191 | HP Insight Cluster Management Utility instruction

	NVIDIA GPUs, 85	user groups
	O	adding, 98
	O	archived, 76
	operating system support, 20	deleting, 99
	P	renaming, 99
	P	V
	parameters	V
	examples, 15	virtual serial port connection, 92
	pdcp, 97, 111	X
	pdsh, 94, 111	X
	power off, 92	X display, 33
	preconfiguration, 51	X server requirements, 33
	provisioning, 41	xinetd, 131
	R
	RAID configuration, 14
	reboot, 93
	reconfiguration, 52
	related information, 123
	remote hardware control API, 117
	renaming logical groups, 42
	renaming user groups, 99
	rescan MAC, 53
	restore database, 31
	restoring configuration, 31
	S
	save database, 31
	ScaleMP support, 120
	scanning nodes, 36
	secure shell connection, 91
	SELinux, 21
	shell interface, 111
	shutdown, 92
	single window pdsh, 94
	smart array card, 14
	ssh, 91
	starting CMU
	automatically, 135
	with cmuserver, 134
	static info, 53
	status, 32
	gauge, 71
	group overview, 69
	node details, 72
	resource detail, 71
	resource view, 70
	stop monitoring, 77
	T
	time view, 72
	troubleshooting, 126
	tuning monitoring, 77
	typographic conventions, 124
	U
	upgrading, 13, 30
	upgrading Java, 30
	user group management, 98

191

Image 191

HP Insight Cluster Management Utility manual 191

Contents

HP Insight Cluster Management Utility Copyright 2013 Hewlett-Packard Development Company, L.P Contents Provisioning a cluster with HP Insight CMU Defining a cluster with HP Insight CMU Monitoring a cluster with HP Insight CMU Actions Alerts Alert reactions Managing a cluster with HP Insight CMUAdvanced topics 112 HP Insight CMU manpages 139 Troubleshooting 126Detailed installation instructions 131 Support and other resources 123 Glossary 187 Index 189 Figures Examples Date command Dmidecode commandTables Compute node monitoring FeaturesHP Insight CMU configuration Overview System disk replication Compute node administration Management node hardware requirements Installing and upgrading HP Insight CMUInstalling HP Insight CMU Configuring the management cards Planning for compute node installationFirmware upgrade requirements Configuring the local smart array card Speed 9600 Bd Configuring the BiosNIC 1 PXE boot or PXE enabled NIC 2 Disabled 7.1 DL3xx, DL5xx, DL7xx, Blades IDE 7.2 DL160 G5, DL165c G5, DL165c G6, and DL180 G5 Servers NIC1 PXE Enabled Share NIC mode DisabledDhcp Disabled NIC1 control Enabled NIC2 on the SL2x170z G6 Server 7.4 SL2x170z G6 and DL170h G6 Servers Bios setting HP Insight CMU kit delivery Preparing for installationPreinstallation limitations Rhel 6 support Operating system supportHP Insight CMU CD-ROM directory structure SELinux and HP Insight CMU HP Insight CMU installation checklistLogin privileges # chkconfig --add cmu Installation proceduresRun /opt/cmu/bin/cmumgtconfig -c Installing HP Insight CMU with high availability Installing and upgrading HP Insight CMU Overview Installing HP Insight CMU under HAHA hardware requirements Software prerequisites Installing and testing Configuring HA control of HP Insight CMUHP Insight CMU HA service requirements # /etc/init.d/cmuserver start Cmuadmin1# /etc/init.d/cmu startStart cmuserver # /etc/init.d/cmu setaudit # /etc/init.d/cmu stop Cmu hacmu service needs restartVar/log/cmuservicehostname.log file for errors Cmuadmin2# /opt/cmu/tools/cmuhapostinstall Run cmuhapostinstall on server HP Insight CMU configuration considerationsUpgrading HP Insight CMU HA service Cmuha nothing to backup from the cmu HA share Upgrading Java Runtime Environment Stopping the HP Insight CMU serviceUpgrading HP Insight CMU Dependencies Restoring the HP Insight CMU database Saving the HP Insight CMU database HP Insight CMU main window HP Insight CMU service statusDefining a cluster with HP Insight CMU Launching the HP Insight CMU GUI Server requirements Administrator modeQuitting administrator mode Click Options→Unprivileged Mode Cluster administration High-level checklist for building an HP Insight CMU cluster Node management window Node management Scan node dialog Scanning nodes Adding nodes Management card password window Modifying nodes Cluster Administration→Node Management Exporting nodes Network entity managementImporting nodes Deleting nodes Deleting network entities Adding network entities Logical group management Provisioning a cluster with HP Insight CMU Renaming logical groups Modifying logical groupsDeleting logical groups Autoinstall calling methods AutoinstallAutoinstall requirements Autoinstall templates Restart cmuserver Using autoinstall from GUIEnabling autoinstall Creating an autoinstall logical group Registering compute nodes Autoinstall compute nodes Cmu addtologicalgroup node to logicalgroupname Using autoinstall from CLIRegistering an autoinstall logical group Adding nodes to autoinstall logical group Customization Where nodes.txt is the list of nodes to autoinstall Backing up a disk from a compute node in a logical group Backing upRestrictions # /opt/cmu/image/logicalgroupname Cloning Cloning procedure Cloning status Preconfiguration Default content of prereconf.sh is Reconfiguration Rescan MAC Node static info # cmuimagecommit -i rh5u4x8664 HP Insight CMU image editorExpanding an image After editing the image, commit changes # /opt/cmu/bin/cmuimagecommit -i rh5u4x8664 HP Insight CMU diskless environmentsModifying an image Saving a modified cloning image On the golden node Modifying the Tftp server configurationOperating systems supported On the management node Creating a diskless logical group Activating the diskless featurePopulating the HP Insight CMU database Creating a diskless image Adding a new logical group Cmu probekernel Adding nodes into the logical groupFrom the CLI # /opt/cmu/cmucli Customizing your diskless image Booting the compute nodesCmu boot net myTestImage node1 noden Understanding the structure of a diskless image Using reconf-diskless-image.sh Node. /opt/cmu/image/imageName/snapshot/nodeName Do not update the /opt/cmu/image/imageName/root directoryBest practices for diskless clusters Templates and image file On Red Hat Configuring additional NFS servers# chkconfig nfs on Sample file When a diskless logical group is created# chkconfig nfsserver on On Sles Comments on High Availability HA When a node is added to the diskless logical group Deploying the monitoring client Installing the HP Insight CMU monitoring clientMonitoring a cluster with HP Insight CMU Launch the HP Insight CMU GUI Monitoring the cluster Selecting the central frame display Node and group status Monitoring window Global cluster view in the central frame Resource view overview Resource view in the central frame Node view in the central frame Detail mode in resource viewGauge widget Node details Using time view Adaptive stacking Getting startedTagging nodes Mouse control Bindings and options Technical dependencies Troubleshooting Visualizing history data Archiving user groups Limitations Tuning HP Insight CMU monitoringStopping HP Insight CMU monitoring Action and alert files Alerts Actions Alert reactions Alerts Text of the Description for this reaction Name of the alert that caused the reactionLevel of the alert # /etc/init.d/collectl start Using collectl for gathering monitoring dataInstalling and starting collectl on compute nodes # chkconfig --add collectl ActionAndAlertFile.txt file # collectl -c 1 -s+C --export lexpr # cp -a /var/www/html/colplot /opt/cmu/www/colplot # mkdir /var/log/collectl # vi /etc/exportsVar/log/collectl *rw,sync,noallsquash,norootsquash # exportfs -r # /etc/init.d/collectl restart # mkdir /var/log/collectl # vi /etc/fstab# vi /etc/collectl.conf Restart collectl Select plotting options, then click Generate Plot Monitoring GPUs and coprocessorsMonitoring Nvidia GPUs Monitoring AMD GPUs # cmuconfigamd Monitoring Intel coprocessors # /opt/cmu/bin/cmuconfigintel Review the results and verify no errors are reported HP Insight CMU alert converted to SIM event Extended metric support Opt/cmu/bin/cmusubmitextendedmetrics -f datafile SSH connection Administrator menuManaging a cluster with HP Insight CMU Unprivileged user menu Shutdown Power offManagement card connection Virtual serial port connection Change UID LED status BootReboot Dshbak Multiple windows broadcastSingle window pdsh Cmudiff examples Cmupdsh cmudiff -d Cmupdsh dmidecodeCmupdsh cmudiff -h Parallel distributed copy pdcp # vi /root/.ssh/config Adding user groups User group management Renaming user groups Viewing and analyzing Bios settingsHP Insight firmware management Deleting user groups Customizing the GUI menu Checking Bios versionsInstalling and upgrading firmware Starting a CLI interactive session Saving user settingsBasic commands HP Insight CMU CLI Cmu help Help commandsGetting help for a command For example, to get more information about the halt command Displaying nodes of a logical group Executing a command on one nodeSpecifying nodes Displaying logical groups of a cluster Executing a command on all nodes Executing a command on a list of nodesExecuting a command on a range of nodes Using wildcards Broadcasting commands to a set of nodes Administration and cloning commandsExecuting a command on specific nodes of a logical group Booting a set of nodes Halting a set of nodes Rebooting a set of nodesPowering off a set of nodes Cmu locate off o185i192 Setting the locator LED on or offCloning a set of nodes Cmu locate on o185i192 Backing up a node Adding a new logical groupAdding nodes to a logical group Cmu backup Discovering MAC address for new nodes Modifying a management card passwordCmu modifypassword ILOILOCMlo100i Cmu modifypassword lo100i # /opt/cmu/bin/pdsh -w cn0001,cn0002 ls Administration utilities pdcp and pdshHP Insight CMU Linux shell commands # /opt/cmu/bin/pdcp -w cn0001,cn0002 source /tmp/dest Advanced topics Accessing the GUI for non-root users Custom menu options for non-root users Configuring sudo support HP Insight CMU diskless API Cjones ALL = CmupowerBstevens ALL = Nopasswd Cmupower Cmuimage Sbarney ALL = Nopasswd Cmupower Cmuimage Cmuetc Name of the new logical group Build diskless imageDelete diskless image Boot diskless node Configure diskless nodeUnconfigure diskless node None HP Insight CMU remote hardware control APIDiskless check Lo100i Uidon OffOsoff Uidoff CMUVALIDHARDWARETYPES=ILOlo100iILOCMIPMI CMUVALIDHARDWARETYPES=ILOlo100iILOCM CMUvSMPPREFIX=vSMP Support for ScaleMPCloning mechanisms Cloning mechanisms Advanced topics Related information Support and other resourcesContacting HP User input CommandTypographic conventions Computer output Typographic conventions HP Insight CMU logs TroubleshootingNetwork boot issues Troubleshooting network boot Backup issuesTroubleshooting switch issues If not, restart the HP Insight CMU service Cloning issuesAdministration command problems GUI problems Detailed Java exception is Certificate error Troubleshooting # chkconfig nfs on # /etc/init.d/nfs start Detailed installation instructionsInstall required RPMs Activating xinetd services On the HP Insight CMU management node Firewall configurationJava installation Verifying the Dhcpd listen interface Method Installing HP Insight CMU licensingSetting the Java Path Configuring the HP Insight CMU management server hostname # vi /opt/cmu/etc/cmuserver.conf Cmu service needs restartStarting HP Insight CMU Edit the /opt/cmu/etc/cmuserver.conf file Cmuserver utility reports the state of the daemons Configuring HP Insight CMU to start automaticallyInstalling HP Insight CMU on the GUI client workstation Verifying the HP Insight CMU state Using an X Window server Configuring the GUI client on Linux workstationsUsing an ssh tunnel Launching the HP Insight CMU directly from the Java file Launching the HP Insight CMU GUI using a web browserActivating the HP Insight CMU GUI HP Insight CMU GUI HP Insight CMU manpages Cmushownodes8 # /opt/cmu/bin/cmushownodes # /opt/cmu/bin/cmushownodes -a -o %n %i %k %m default %b %t # /opt/cmu/bin/cmushowlogicalgroups Cmushowlogicalgroups8# /opt/cmu/bin/cmushowlogicalgroups -h logicalgroupname Help # /opt/cmu/bin/cmushownetworkentities rack1 Cmushownetworkentities8# /opt/cmu/bin/cmushownetworkentities -h networkentity # /opt/cmu/bin/cmushownetworkentities # /opt/cmu/bin/cmushowusergroups user1 Cmushowusergroups8# /opt/cmu/bin/cmushowusergroups -h usergroup # /opt/cmu/bin/cmushowusergroups # /opt/cmu/bin/cmushowarchivedusergroups -f -s Cmushowarchivedusergroups8# /opt/cmu/bin/cmushowarchivedusergroups # /opt/cmu/bin/cmushowarchivedusergroups -p -f Node-number num Cmuaddnode8Mgtcardip -T--mgt-card ILOlo100iILOCM -R--arch architecture # /opt/cmu/bin/cmuaddnode -f nodes.txt Command-line modeProcessing 1 node # cat nodes.txt # /opt/cmu/bin/cmuaddnetworkentity -f networkentitylist Cmuaddnetworkentity8Filename inputfile # /opt/cmu/bin/cmuaddnetworkentity rack1 rack2 # /opt/cmu/bin/cmuaddlogicalgroup -f logicalgroupfile Cmuaddlogicalgroup8Filename inputfile input logical groups from inputfile Name # /opt/cmu/bin/cmuaddlogicalgroup -n test -d cciss/c0d0 Nodenamefile Cmuaddtologicalgroupcandidates8Logicalgroup # /opt/cmu/bin/cmuaddusergroup -f usergrouplist Cmuaddusergroup8# /opt/cmu/bin/cmuaddusergroup user1 user2 # /opt/cmu/bin/cmuaddtousergroup -t group1 -f nodenamefile Cmuaddtousergroup8Usergroup # /opt/cmu/bin/cmuaddtousergroup -t group1 cn0003 To flag cn0001 active in logical group rh6u0x8664 Cmuchangeactivelogicalgroup8 # /opt/cmu/bin/cmuchangenetworkentity -t rack1 cn0001 Cmuchangenetworkentity8Networkentity To delete one node from a logical group rh6u0x8664 Cmudelfromlogicalgroupcandidates8Delete one or more nodes from a logical group Delete nodes from this logical group # /opt/cmu/bin/cmudelfromnetworkentity -t rack1 node1 Cmudelfromnetworkentity8 Delete an archived user group # /opt/cmu/bin/cmudelarchivedusergroup -h -v-t timeout -dCmudelarchivedusergroup8 Cmudelarchivedusergroup -- Delete an archived user group # /opt/cmu/bin/cmudelfromusergroup -t user1 node1 Cmudelfromusergroup8 # /opt/cmu/bin/cmudellogicalgroup -f logicalgrouplist Cmudellogicalgroup8# /opt/cmu/bin/cmudellogicalgroup rh5u5x8664 sles11sp1x8664 # /opt/cmu/bin/cmudelnetworkentity -f networkentitylist Cmudelnetworkentity8# /opt/cmu/bin/cmudelnetworkentity rack1 rack2 # /opt/cmu/bin/cmudelnode -f nodelist Cmudelnode8# /opt/cmu/bin/cmudelnode node1 node2 Delete monitoring snapshots from the history database Cmudelsnapshots8 # /opt/cmu/bin/cmudelusergroup -a -m 60 -f usergrouplist Cmudelusergroup8# /opt/cmu/bin/cmudelusergroup user1 user2 # /opt/cmu/bin/cmudelusergroup -f usergrouplist # /opt/cmu/bin/cmuconsole computenodehostname Cmuconsole8Cmuconsole -- Connect to compute node management ports Cmupower8 Cmupower -p Uidon -u user2 Cmupower -p OFF -n cn0001Cmupower -p OFF -u user1 Cmupower -p Boot -l rh6u0x8664 # /opt/cmu/bin/cmucustomrun -t auditlspci Cmucustomrun8Title # /opt/cmu/bin/cmucustomrun -l # /opt/cmu/bin/cmuclone -n node1 -i rh6u2x8664 Cmuclone8=node # /opt/cmu/bin/cmuclone -f /tmp/nodelist -i sles11sp1x8664 Cmubackup -l myimage -n node20 -r 3 -e /tmp/err.log Cmubackup8Cmubackup -l myimage -n node20 -p cciss/c0d0p3,cciss/c0d0p1 # /opt/cmu/bin/cmuscanmacs -h hostname -p hostnameprefix Cmuscanmacs8Options naming Options general Example node definitions Example 173 Node name in the HP Insight CMU database Cmurescanmac8Cmurescanmac -- Rescan the MAC address of a node # /opt/cmu/tools/cmurescanmac -n nodename N NICnum -h Lg logicalgroup Cmumodnode8Filename # /opt/cmu/bin/cmumodnode -f nodes.txt Cmumonstat8 # /opt/cmu/bin/cmumonstat --all-nodes --color # /opt/cmu/bin/cmumonstat --nodes=n0001,n0002,n0001-11# /opt/cmu/bin/cmumonstat --all-sensors --all-nodes # /opt/cmu/bin/cmumonstat --all-nodes --all-sensors --stats # /opt/cmu/bin/cmuimageopen -i rh5u5x8664 Cmuimageopen8# /opt/cmu/bin/cmuimageopen -h -i imagename To open the HP Insight CMU image rh5u5x8664 # /opt/cmu/bin/cmuimagecommit -i rh5u5x8664 Cmuimagecommit8 Cmuconfignvidia -- Configure Nvidia GPU monitoring Cmuconfignvidia8 Cmuconfigamd -- Configure AMD GPU monitoring Cmuconfigamd8 # /opt/cmu/bin/cmuconfigintel -h -r -n Cmuconfigintel8Cmuconfigintel -- Configure Intel coprocessor monitoring Num Cmumgtconfig8Automatically defaults when reasonable Eth num1num2bond num1 # /opt/cmu/bin/cmumgtconfig -t # /opt/cmu/bin/cmumgtconfig -c# /opt/cmu/bin/cmumgtconfig -c -s dhcp Specifies a text file listing compute nodes Cmufirmwaremgmt8Cmufirmwaremgmt -- Verify and execute firmware Ocmudiffparameters Ebipa Glossary Package management Index Index 191