HP Insight Cluster Management Utility
Copyright 2013 Hewlett-Packard Development Company, L.P
Contents
Provisioning a cluster with HP Insight CMU
Defining a cluster with HP Insight CMU
Monitoring a cluster with HP Insight CMU
Actions Alerts Alert reactions
Managing a cluster with HP Insight CMU
Advanced topics 112
Detailed installation instructions 131
Troubleshooting 126
Support and other resources 123
HP Insight CMU manpages 139
Glossary 187 Index 189
Figures
Examples
Date command Dmidecode command
Tables
HP Insight CMU configuration
Features
Overview
Compute node monitoring
System disk replication
Compute node administration
Management node hardware requirements
Installing and upgrading HP Insight CMU
Installing HP Insight CMU
Firmware upgrade requirements
Planning for compute node installation
Configuring the local smart array card
Configuring the management cards
NIC 1 PXE boot or PXE enabled NIC 2 Disabled
Configuring the Bios
7.1 DL3xx, DL5xx, DL7xx, Blades
Speed 9600 Bd
IDE
7.2 DL160 G5, DL165c G5, DL165c G6, and DL180 G5 Servers
Dhcp Disabled
Share NIC mode Disabled
NIC1 control Enabled
NIC1 PXE Enabled
NIC2 on the SL2x170z G6 Server
7.4 SL2x170z G6 and DL170h G6 Servers Bios setting
HP Insight CMU kit delivery
Preparing for installation
Preinstallation limitations
Rhel 6 support
Operating system support
HP Insight CMU CD-ROM directory structure
SELinux and HP Insight CMU
HP Insight CMU installation checklist
Login privileges
# chkconfig --add cmu
Installation procedures
Run /opt/cmu/bin/cmumgtconfig -c
Installing HP Insight CMU with high availability
Installing and upgrading HP Insight CMU
HA hardware requirements
Installing HP Insight CMU under HA
Software prerequisites
Overview
Installing and testing
Configuring HA control of HP Insight CMU
HP Insight CMU HA service requirements
# /etc/init.d/cmuserver start
Cmuadmin1# /etc/init.d/cmu start
Start cmuserver
Var/log/cmuservicehostname.log file for errors
Cmu hacmu service needs restart
Cmuadmin2# /opt/cmu/tools/cmuhapostinstall
# /etc/init.d/cmu setaudit # /etc/init.d/cmu stop
Upgrading HP Insight CMU HA service
HP Insight CMU configuration considerations
Cmuha nothing to backup from the cmu HA share
Run cmuhapostinstall on server
Upgrading HP Insight CMU
Stopping the HP Insight CMU service
Dependencies
Upgrading Java Runtime Environment
Restoring the HP Insight CMU database
Saving the HP Insight CMU database
Defining a cluster with HP Insight CMU
HP Insight CMU service status
Launching the HP Insight CMU GUI
HP Insight CMU main window
Quitting administrator mode
Administrator mode
Click Options→Unprivileged Mode
Server requirements
Cluster administration
High-level checklist for building an HP Insight CMU cluster
Node management window
Node management
Scan node dialog
Scanning nodes
Adding nodes
Management card password window
Modifying nodes
Cluster Administration→Node Management
Importing nodes
Network entity management
Deleting nodes
Exporting nodes
Deleting network entities
Adding network entities
Logical group management
Provisioning a cluster with HP Insight CMU
Renaming logical groups
Modifying logical groups
Deleting logical groups
Autoinstall requirements
Autoinstall
Autoinstall templates
Autoinstall calling methods
Enabling autoinstall
Using autoinstall from GUI
Creating an autoinstall logical group
Restart cmuserver
Registering compute nodes
Autoinstall compute nodes
Registering an autoinstall logical group
Using autoinstall from CLI
Adding nodes to autoinstall logical group
Cmu addtologicalgroup node to logicalgroupname
Customization
Where nodes.txt is the list of nodes to autoinstall
Backing up a disk from a compute node in a logical group
Backing up
Restrictions
# /opt/cmu/image/logicalgroupname
Cloning
Cloning procedure
Cloning status
Preconfiguration
Default content of prereconf.sh is
Reconfiguration
Rescan MAC
Node static info
Expanding an image
HP Insight CMU image editor
After editing the image, commit changes
# cmuimagecommit -i rh5u4x8664
Modifying an image
HP Insight CMU diskless environments
Saving a modified cloning image
# /opt/cmu/bin/cmuimagecommit -i rh5u4x8664
Operating systems supported
Modifying the Tftp server configuration
On the management node
On the golden node
Populating the HP Insight CMU database
Activating the diskless feature
Creating a diskless image
Creating a diskless logical group
Adding a new logical group
From the CLI
Adding nodes into the logical group
# /opt/cmu/cmucli
Cmu probekernel
Cmu boot net myTestImage node1 noden
Booting the compute nodes
Understanding the structure of a diskless image
Customizing your diskless image
Using reconf-diskless-image.sh
Best practices for diskless clusters
Do not update the /opt/cmu/image/imageName/root directory
Templates and image file
Node. /opt/cmu/image/imageName/snapshot/nodeName
On Red Hat
Configuring additional NFS servers
# chkconfig nfs on
# chkconfig nfsserver on
When a diskless logical group is created
On Sles
Sample file
Comments on High Availability HA
When a node is added to the diskless logical group
Deploying the monitoring client
Installing the HP Insight CMU monitoring client
Monitoring a cluster with HP Insight CMU
Launch the HP Insight CMU GUI
Monitoring the cluster
Selecting the central frame display
Node and group status
Monitoring window
Global cluster view in the central frame
Resource view overview
Resource view in the central frame
Node view in the central frame
Detail mode in resource view
Gauge widget
Node details
Using time view
Adaptive stacking
Getting started
Tagging nodes
Mouse control
Bindings and options
Technical dependencies
Troubleshooting
Visualizing history data
Archiving user groups
Stopping HP Insight CMU monitoring
Tuning HP Insight CMU monitoring
Action and alert files
Limitations
Alerts
Actions
Alert reactions
Alerts
Text of the Description for this reaction
Name of the alert that caused the reaction
Level of the alert
Installing and starting collectl on compute nodes
Using collectl for gathering monitoring data
# chkconfig --add collectl
# /etc/init.d/collectl start
ActionAndAlertFile.txt file
# collectl -c 1 -s+C --export lexpr
Var/log/collectl *rw,sync,noallsquash,norootsquash
# mkdir /var/log/collectl # vi /etc/exports
# exportfs -r
# cp -a /var/www/html/colplot /opt/cmu/www/colplot
# vi /etc/collectl.conf
# mkdir /var/log/collectl # vi /etc/fstab
Restart collectl
# /etc/init.d/collectl restart
Select plotting options, then click Generate Plot
Monitoring GPUs and coprocessors
Monitoring Nvidia GPUs
Monitoring AMD GPUs
# cmuconfigamd
Monitoring Intel coprocessors
# /opt/cmu/bin/cmuconfigintel
Review the results and verify no errors are reported
HP Insight CMU alert converted to SIM event
Extended metric support
Opt/cmu/bin/cmusubmitextendedmetrics -f datafile
Managing a cluster with HP Insight CMU
Administrator menu
Unprivileged user menu
SSH connection
Management card connection
Power off
Virtual serial port connection
Shutdown
Change UID LED status
Boot
Reboot
Dshbak
Multiple windows broadcast
Single window pdsh
Cmudiff examples
Cmupdsh cmudiff -d
Cmupdsh dmidecode
Cmupdsh cmudiff -h
Parallel distributed copy pdcp
# vi /root/.ssh/config
Adding user groups
User group management
HP Insight firmware management
Viewing and analyzing Bios settings
Deleting user groups
Renaming user groups
Customizing the GUI menu
Checking Bios versions
Installing and upgrading firmware
Basic commands
Saving user settings
HP Insight CMU CLI
Starting a CLI interactive session
Getting help for a command
Help commands
For example, to get more information about the halt command
Cmu help
Specifying nodes
Executing a command on one node
Displaying logical groups of a cluster
Displaying nodes of a logical group
Executing a command on a range of nodes
Executing a command on a list of nodes
Using wildcards
Executing a command on all nodes
Executing a command on specific nodes of a logical group
Administration and cloning commands
Booting a set of nodes
Broadcasting commands to a set of nodes
Halting a set of nodes
Rebooting a set of nodes
Powering off a set of nodes
Cloning a set of nodes
Setting the locator LED on or off
Cmu locate on o185i192
Cmu locate off o185i192
Backing up a node
Adding a new logical group
Adding nodes to a logical group
Cmu backup
Cmu modifypassword ILOILOCMlo100i
Modifying a management card password
Cmu modifypassword lo100i
Discovering MAC address for new nodes
HP Insight CMU Linux shell commands
Administration utilities pdcp and pdsh
# /opt/cmu/bin/pdcp -w cn0001,cn0002 source /tmp/dest
# /opt/cmu/bin/pdsh -w cn0001,cn0002 ls
Advanced topics
Accessing the GUI for non-root users
Custom menu options for non-root users
Configuring sudo support
Bstevens ALL = Nopasswd Cmupower Cmuimage
Cjones ALL = Cmupower
Sbarney ALL = Nopasswd Cmupower Cmuimage Cmuetc
HP Insight CMU diskless API
Name of the new logical group
Build diskless image
Delete diskless image
Boot diskless node
Configure diskless node
Unconfigure diskless node
Diskless check
HP Insight CMU remote hardware control API
Lo100i
None
Osoff
Off
Uidoff
Uidon
CMUVALIDHARDWARETYPES=ILOlo100iILOCMIPMI
CMUVALIDHARDWARETYPES=ILOlo100iILOCM
CMUvSMPPREFIX=vSMP
Support for ScaleMP
Cloning mechanisms
Cloning mechanisms
Advanced topics
Related information
Support and other resources
Contacting HP
Typographic conventions
Command
Computer output
User input
Typographic conventions
HP Insight CMU logs
Troubleshooting
Network boot issues
Troubleshooting network boot
Backup issues
Troubleshooting switch issues
Administration command problems
Cloning issues
GUI problems
If not, restart the HP Insight CMU service
Detailed Java exception is
Certificate error
Troubleshooting
Install required RPMs
Detailed installation instructions
Activating xinetd services
# chkconfig nfs on # /etc/init.d/nfs start
Java installation
Firewall configuration
Verifying the Dhcpd listen interface
On the HP Insight CMU management node
Setting the Java Path
Installing HP Insight CMU licensing
Configuring the HP Insight CMU management server hostname
Method
Starting HP Insight CMU
Cmu service needs restart
Edit the /opt/cmu/etc/cmuserver.conf file
# vi /opt/cmu/etc/cmuserver.conf
Installing HP Insight CMU on the GUI client workstation
Configuring HP Insight CMU to start automatically
Verifying the HP Insight CMU state
Cmuserver utility reports the state of the daemons
Using an X Window server
Configuring the GUI client on Linux workstations
Using an ssh tunnel
Launching the HP Insight CMU directly from the Java file
Launching the HP Insight CMU GUI using a web browser
Activating the HP Insight CMU GUI
HP Insight CMU GUI
HP Insight CMU manpages
Cmushownodes8
# /opt/cmu/bin/cmushownodes
# /opt/cmu/bin/cmushownodes -a -o %n %i %k %m default %b %t
# /opt/cmu/bin/cmushowlogicalgroups -h logicalgroupname
Cmushowlogicalgroups8
Help
# /opt/cmu/bin/cmushowlogicalgroups
# /opt/cmu/bin/cmushownetworkentities -h networkentity
Cmushownetworkentities8
# /opt/cmu/bin/cmushownetworkentities
# /opt/cmu/bin/cmushownetworkentities rack1
# /opt/cmu/bin/cmushowusergroups -h usergroup
Cmushowusergroups8
# /opt/cmu/bin/cmushowusergroups
# /opt/cmu/bin/cmushowusergroups user1
# /opt/cmu/bin/cmushowarchivedusergroups
Cmushowarchivedusergroups8
# /opt/cmu/bin/cmushowarchivedusergroups -p -f
# /opt/cmu/bin/cmushowarchivedusergroups -f -s
Node-number num
Cmuaddnode8
Mgtcardip -T--mgt-card ILOlo100iILOCM -R--arch architecture
Processing 1 node
Command-line mode
# cat nodes.txt
# /opt/cmu/bin/cmuaddnode -f nodes.txt
Filename inputfile
Cmuaddnetworkentity8
# /opt/cmu/bin/cmuaddnetworkentity rack1 rack2
# /opt/cmu/bin/cmuaddnetworkentity -f networkentitylist
Filename inputfile input logical groups from inputfile Name
Cmuaddlogicalgroup8
# /opt/cmu/bin/cmuaddlogicalgroup -n test -d cciss/c0d0
# /opt/cmu/bin/cmuaddlogicalgroup -f logicalgroupfile
Nodenamefile
Cmuaddtologicalgroupcandidates8
Logicalgroup
# /opt/cmu/bin/cmuaddusergroup -f usergrouplist
Cmuaddusergroup8
# /opt/cmu/bin/cmuaddusergroup user1 user2
Usergroup
Cmuaddtousergroup8
# /opt/cmu/bin/cmuaddtousergroup -t group1 cn0003
# /opt/cmu/bin/cmuaddtousergroup -t group1 -f nodenamefile
To flag cn0001 active in logical group rh6u0x8664
Cmuchangeactivelogicalgroup8
# /opt/cmu/bin/cmuchangenetworkentity -t rack1 cn0001
Cmuchangenetworkentity8
Networkentity
Delete one or more nodes from a logical group
Cmudelfromlogicalgroupcandidates8
Delete nodes from this logical group
To delete one node from a logical group rh6u0x8664
# /opt/cmu/bin/cmudelfromnetworkentity -t rack1 node1
Cmudelfromnetworkentity8
Cmudelarchivedusergroup8
# /opt/cmu/bin/cmudelarchivedusergroup -h -v-t timeout -d
Cmudelarchivedusergroup -- Delete an archived user group
Delete an archived user group
# /opt/cmu/bin/cmudelfromusergroup -t user1 node1
Cmudelfromusergroup8
# /opt/cmu/bin/cmudellogicalgroup -f logicalgrouplist
Cmudellogicalgroup8
# /opt/cmu/bin/cmudellogicalgroup rh5u5x8664 sles11sp1x8664
# /opt/cmu/bin/cmudelnetworkentity -f networkentitylist
Cmudelnetworkentity8
# /opt/cmu/bin/cmudelnetworkentity rack1 rack2
# /opt/cmu/bin/cmudelnode -f nodelist
Cmudelnode8
# /opt/cmu/bin/cmudelnode node1 node2
Delete monitoring snapshots from the history database
Cmudelsnapshots8
# /opt/cmu/bin/cmudelusergroup user1 user2
Cmudelusergroup8
# /opt/cmu/bin/cmudelusergroup -f usergrouplist
# /opt/cmu/bin/cmudelusergroup -a -m 60 -f usergrouplist
# /opt/cmu/bin/cmuconsole computenodehostname
Cmuconsole8
Cmuconsole -- Connect to compute node management ports
Cmupower8
Cmupower -p OFF -u user1
Cmupower -p OFF -n cn0001
Cmupower -p Boot -l rh6u0x8664
Cmupower -p Uidon -u user2
Title
Cmucustomrun8
# /opt/cmu/bin/cmucustomrun -l
# /opt/cmu/bin/cmucustomrun -t auditlspci
=node
Cmuclone8
# /opt/cmu/bin/cmuclone -f /tmp/nodelist -i sles11sp1x8664
# /opt/cmu/bin/cmuclone -n node1 -i rh6u2x8664
Cmubackup -l myimage -n node20 -r 3 -e /tmp/err.log
Cmubackup8
Cmubackup -l myimage -n node20 -p cciss/c0d0p3,cciss/c0d0p1
# /opt/cmu/bin/cmuscanmacs -h hostname -p hostnameprefix
Cmuscanmacs8
Options naming
Options general
Example node definitions
Example
173
Cmurescanmac -- Rescan the MAC address of a node
Cmurescanmac8
# /opt/cmu/tools/cmurescanmac -n nodename N NICnum -h
Node name in the HP Insight CMU database
Lg logicalgroup
Cmumodnode8
Filename
# /opt/cmu/bin/cmumodnode -f nodes.txt
Cmumonstat8
# /opt/cmu/bin/cmumonstat --all-sensors --all-nodes
# /opt/cmu/bin/cmumonstat --nodes=n0001,n0002,n0001-11
# /opt/cmu/bin/cmumonstat --all-nodes --all-sensors --stats
# /opt/cmu/bin/cmumonstat --all-nodes --color
# /opt/cmu/bin/cmuimageopen -h -i imagename
Cmuimageopen8
To open the HP Insight CMU image rh5u5x8664
# /opt/cmu/bin/cmuimageopen -i rh5u5x8664
# /opt/cmu/bin/cmuimagecommit -i rh5u5x8664
Cmuimagecommit8
Cmuconfignvidia -- Configure Nvidia GPU monitoring
Cmuconfignvidia8
Cmuconfigamd -- Configure AMD GPU monitoring
Cmuconfigamd8
# /opt/cmu/bin/cmuconfigintel -h -r -n
Cmuconfigintel8
Cmuconfigintel -- Configure Intel coprocessor monitoring
Automatically defaults when reasonable
Cmumgtconfig8
Eth num1num2bond num1
Num
# /opt/cmu/bin/cmumgtconfig -t
# /opt/cmu/bin/cmumgtconfig -c
# /opt/cmu/bin/cmumgtconfig -c -s dhcp
Cmufirmwaremgmt -- Verify and execute firmware
Cmufirmwaremgmt8
Ocmudiffparameters
Specifies a text file listing compute nodes
Ebipa
Glossary
Package management
Index
Index
191