Advanced topics | HP Insight Cluster Management Utility specification

122 Advanced topics

Image 122

HP Insight Cluster Management Utility manual Advanced topics

Contents

HP Insight Cluster Management Utility Copyright 2013 Hewlett-Packard Development Company, L.P Contents Defining a cluster with HP Insight CMU Provisioning a cluster with HP Insight CMU Monitoring a cluster with HP Insight CMU Actions Alerts Alert reactions Managing a cluster with HP Insight CMUAdvanced topics 112 Support and other resources 123 Troubleshooting 126Detailed installation instructions 131 HP Insight CMU manpages 139 Glossary 187 Index 189 Figures Examples Date command Dmidecode commandTables Overview FeaturesHP Insight CMU configuration Compute node monitoring Compute node administration System disk replication Management node hardware requirements Installing and upgrading HP Insight CMUInstalling HP Insight CMU Configuring the local smart array card Planning for compute node installationFirmware upgrade requirements Configuring the management cards 7.1 DL3xx, DL5xx, DL7xx, Blades Configuring the BiosNIC 1 PXE boot or PXE enabled NIC 2 Disabled Speed 9600 Bd 7.2 DL160 G5, DL165c G5, DL165c G6, and DL180 G5 Servers IDE NIC1 control Enabled Share NIC mode DisabledDhcp Disabled NIC1 PXE Enabled 7.4 SL2x170z G6 and DL170h G6 Servers Bios setting NIC2 on the SL2x170z G6 Server HP Insight CMU kit delivery Preparing for installationPreinstallation limitations Rhel 6 support Operating system supportHP Insight CMU CD-ROM directory structure SELinux and HP Insight CMU HP Insight CMU installation checklistLogin privileges # chkconfig --add cmu Installation proceduresRun /opt/cmu/bin/cmumgtconfig -c Installing HP Insight CMU with high availability Installing and upgrading HP Insight CMU Software prerequisites Installing HP Insight CMU under HAHA hardware requirements Overview Installing and testing Configuring HA control of HP Insight CMUHP Insight CMU HA service requirements # /etc/init.d/cmuserver start Cmuadmin1# /etc/init.d/cmu startStart cmuserver Cmuadmin2# /opt/cmu/tools/cmuhapostinstall Cmu hacmu service needs restartVar/log/cmuservicehostname.log file for errors # /etc/init.d/cmu setaudit # /etc/init.d/cmu stop Cmuha nothing to backup from the cmu HA share HP Insight CMU configuration considerationsUpgrading HP Insight CMU HA service Run cmuhapostinstall on server Dependencies Stopping the HP Insight CMU serviceUpgrading HP Insight CMU Upgrading Java Runtime Environment Saving the HP Insight CMU database Restoring the HP Insight CMU database Launching the HP Insight CMU GUI HP Insight CMU service statusDefining a cluster with HP Insight CMU HP Insight CMU main window Click Options→Unprivileged Mode Administrator modeQuitting administrator mode Server requirements High-level checklist for building an HP Insight CMU cluster Cluster administration Node management Node management window Scanning nodes Scan node dialog Management card password window Adding nodes Cluster Administration→Node Management Modifying nodes Deleting nodes Network entity managementImporting nodes Exporting nodes Adding network entities Deleting network entities Provisioning a cluster with HP Insight CMU Logical group management Renaming logical groups Modifying logical groupsDeleting logical groups Autoinstall templates AutoinstallAutoinstall requirements Autoinstall calling methods Creating an autoinstall logical group Using autoinstall from GUIEnabling autoinstall Restart cmuserver Autoinstall compute nodes Registering compute nodes Adding nodes to autoinstall logical group Using autoinstall from CLIRegistering an autoinstall logical group Cmu addtologicalgroup node to logicalgroupname Where nodes.txt is the list of nodes to autoinstall Customization Backing up a disk from a compute node in a logical group Backing upRestrictions # /opt/cmu/image/logicalgroupname Cloning procedure Cloning Preconfiguration Cloning status Reconfiguration Default content of prereconf.sh is Node static info Rescan MAC After editing the image, commit changes HP Insight CMU image editorExpanding an image # cmuimagecommit -i rh5u4x8664 Saving a modified cloning image HP Insight CMU diskless environmentsModifying an image # /opt/cmu/bin/cmuimagecommit -i rh5u4x8664 On the management node Modifying the Tftp server configurationOperating systems supported On the golden node Creating a diskless image Activating the diskless featurePopulating the HP Insight CMU database Creating a diskless logical group Adding a new logical group # /opt/cmu/cmucli Adding nodes into the logical groupFrom the CLI Cmu probekernel Understanding the structure of a diskless image Booting the compute nodesCmu boot net myTestImage node1 noden Customizing your diskless image Using reconf-diskless-image.sh Templates and image file Do not update the /opt/cmu/image/imageName/root directoryBest practices for diskless clusters Node. /opt/cmu/image/imageName/snapshot/nodeName On Red Hat Configuring additional NFS servers# chkconfig nfs on On Sles When a diskless logical group is created# chkconfig nfsserver on Sample file When a node is added to the diskless logical group Comments on High Availability HA Deploying the monitoring client Installing the HP Insight CMU monitoring clientMonitoring a cluster with HP Insight CMU Monitoring the cluster Launch the HP Insight CMU GUI Node and group status Selecting the central frame display Global cluster view in the central frame Monitoring window Resource view in the central frame Resource view overview Node view in the central frame Detail mode in resource viewGauge widget Using time view Node details Adaptive stacking Getting startedTagging nodes Bindings and options Mouse control Troubleshooting Technical dependencies Archiving user groups Visualizing history data Action and alert files Tuning HP Insight CMU monitoringStopping HP Insight CMU monitoring Limitations Actions Alerts Alerts Alert reactions Text of the Description for this reaction Name of the alert that caused the reactionLevel of the alert # chkconfig --add collectl Using collectl for gathering monitoring dataInstalling and starting collectl on compute nodes # /etc/init.d/collectl start # collectl -c 1 -s+C --export lexpr ActionAndAlertFile.txt file # exportfs -r # mkdir /var/log/collectl # vi /etc/exportsVar/log/collectl *rw,sync,noallsquash,norootsquash # cp -a /var/www/html/colplot /opt/cmu/www/colplot Restart collectl # mkdir /var/log/collectl # vi /etc/fstab# vi /etc/collectl.conf # /etc/init.d/collectl restart Select plotting options, then click Generate Plot Monitoring GPUs and coprocessorsMonitoring Nvidia GPUs # cmuconfigamd Monitoring AMD GPUs Monitoring Intel coprocessors Review the results and verify no errors are reported # /opt/cmu/bin/cmuconfigintel Extended metric support HP Insight CMU alert converted to SIM event Opt/cmu/bin/cmusubmitextendedmetrics -f datafile Unprivileged user menu Administrator menuManaging a cluster with HP Insight CMU SSH connection Virtual serial port connection Power offManagement card connection Shutdown Change UID LED status BootReboot Dshbak Multiple windows broadcastSingle window pdsh Cmudiff examples Cmupdsh cmudiff -d Cmupdsh dmidecodeCmupdsh cmudiff -h # vi /root/.ssh/config Parallel distributed copy pdcp User group management Adding user groups Deleting user groups Viewing and analyzing Bios settingsHP Insight firmware management Renaming user groups Customizing the GUI menu Checking Bios versionsInstalling and upgrading firmware HP Insight CMU CLI Saving user settingsBasic commands Starting a CLI interactive session For example, to get more information about the halt command Help commandsGetting help for a command Cmu help Displaying logical groups of a cluster Executing a command on one nodeSpecifying nodes Displaying nodes of a logical group Using wildcards Executing a command on a list of nodesExecuting a command on a range of nodes Executing a command on all nodes Booting a set of nodes Administration and cloning commandsExecuting a command on specific nodes of a logical group Broadcasting commands to a set of nodes Halting a set of nodes Rebooting a set of nodesPowering off a set of nodes Cmu locate on o185i192 Setting the locator LED on or offCloning a set of nodes Cmu locate off o185i192 Backing up a node Adding a new logical groupAdding nodes to a logical group Cmu backup Cmu modifypassword lo100i Modifying a management card passwordCmu modifypassword ILOILOCMlo100i Discovering MAC address for new nodes # /opt/cmu/bin/pdcp -w cn0001,cn0002 source /tmp/dest Administration utilities pdcp and pdshHP Insight CMU Linux shell commands # /opt/cmu/bin/pdsh -w cn0001,cn0002 ls Accessing the GUI for non-root users Advanced topics Configuring sudo support Custom menu options for non-root users Sbarney ALL = Nopasswd Cmupower Cmuimage Cmuetc Cjones ALL = CmupowerBstevens ALL = Nopasswd Cmupower Cmuimage HP Insight CMU diskless API Name of the new logical group Build diskless imageDelete diskless image Boot diskless node Configure diskless nodeUnconfigure diskless node Lo100i HP Insight CMU remote hardware control APIDiskless check None Uidoff OffOsoff Uidon CMUVALIDHARDWARETYPES=ILOlo100iILOCM CMUVALIDHARDWARETYPES=ILOlo100iILOCMIPMI CMUvSMPPREFIX=vSMP Support for ScaleMPCloning mechanisms Cloning mechanisms Advanced topics Related information Support and other resourcesContacting HP Computer output CommandTypographic conventions User input Typographic conventions HP Insight CMU logs TroubleshootingNetwork boot issues Troubleshooting network boot Backup issuesTroubleshooting switch issues GUI problems Cloning issuesAdministration command problems If not, restart the HP Insight CMU service Certificate error Detailed Java exception is Troubleshooting Activating xinetd services Detailed installation instructionsInstall required RPMs # chkconfig nfs on # /etc/init.d/nfs start Verifying the Dhcpd listen interface Firewall configurationJava installation On the HP Insight CMU management node Configuring the HP Insight CMU management server hostname Installing HP Insight CMU licensingSetting the Java Path Method Edit the /opt/cmu/etc/cmuserver.conf file Cmu service needs restartStarting HP Insight CMU # vi /opt/cmu/etc/cmuserver.conf Verifying the HP Insight CMU state Configuring HP Insight CMU to start automaticallyInstalling HP Insight CMU on the GUI client workstation Cmuserver utility reports the state of the daemons Using an X Window server Configuring the GUI client on Linux workstationsUsing an ssh tunnel Launching the HP Insight CMU directly from the Java file Launching the HP Insight CMU GUI using a web browserActivating the HP Insight CMU GUI HP Insight CMU GUI HP Insight CMU manpages Cmushownodes8 # /opt/cmu/bin/cmushownodes -a -o %n %i %k %m default %b %t # /opt/cmu/bin/cmushownodes Help Cmushowlogicalgroups8# /opt/cmu/bin/cmushowlogicalgroups -h logicalgroupname # /opt/cmu/bin/cmushowlogicalgroups # /opt/cmu/bin/cmushownetworkentities Cmushownetworkentities8# /opt/cmu/bin/cmushownetworkentities -h networkentity # /opt/cmu/bin/cmushownetworkentities rack1 # /opt/cmu/bin/cmushowusergroups Cmushowusergroups8# /opt/cmu/bin/cmushowusergroups -h usergroup # /opt/cmu/bin/cmushowusergroups user1 # /opt/cmu/bin/cmushowarchivedusergroups -p -f Cmushowarchivedusergroups8# /opt/cmu/bin/cmushowarchivedusergroups # /opt/cmu/bin/cmushowarchivedusergroups -f -s Node-number num Cmuaddnode8Mgtcardip -T--mgt-card ILOlo100iILOCM -R--arch architecture # cat nodes.txt Command-line modeProcessing 1 node # /opt/cmu/bin/cmuaddnode -f nodes.txt # /opt/cmu/bin/cmuaddnetworkentity rack1 rack2 Cmuaddnetworkentity8Filename inputfile # /opt/cmu/bin/cmuaddnetworkentity -f networkentitylist # /opt/cmu/bin/cmuaddlogicalgroup -n test -d cciss/c0d0 Cmuaddlogicalgroup8Filename inputfile input logical groups from inputfile Name # /opt/cmu/bin/cmuaddlogicalgroup -f logicalgroupfile Nodenamefile Cmuaddtologicalgroupcandidates8Logicalgroup # /opt/cmu/bin/cmuaddusergroup -f usergrouplist Cmuaddusergroup8# /opt/cmu/bin/cmuaddusergroup user1 user2 # /opt/cmu/bin/cmuaddtousergroup -t group1 cn0003 Cmuaddtousergroup8Usergroup # /opt/cmu/bin/cmuaddtousergroup -t group1 -f nodenamefile Cmuchangeactivelogicalgroup8 To flag cn0001 active in logical group rh6u0x8664 # /opt/cmu/bin/cmuchangenetworkentity -t rack1 cn0001 Cmuchangenetworkentity8Networkentity Delete nodes from this logical group Cmudelfromlogicalgroupcandidates8Delete one or more nodes from a logical group To delete one node from a logical group rh6u0x8664 Cmudelfromnetworkentity8 # /opt/cmu/bin/cmudelfromnetworkentity -t rack1 node1 Cmudelarchivedusergroup -- Delete an archived user group # /opt/cmu/bin/cmudelarchivedusergroup -h -v-t timeout -dCmudelarchivedusergroup8 Delete an archived user group Cmudelfromusergroup8 # /opt/cmu/bin/cmudelfromusergroup -t user1 node1 # /opt/cmu/bin/cmudellogicalgroup -f logicalgrouplist Cmudellogicalgroup8# /opt/cmu/bin/cmudellogicalgroup rh5u5x8664 sles11sp1x8664 # /opt/cmu/bin/cmudelnetworkentity -f networkentitylist Cmudelnetworkentity8# /opt/cmu/bin/cmudelnetworkentity rack1 rack2 # /opt/cmu/bin/cmudelnode -f nodelist Cmudelnode8# /opt/cmu/bin/cmudelnode node1 node2 Cmudelsnapshots8 Delete monitoring snapshots from the history database # /opt/cmu/bin/cmudelusergroup -f usergrouplist Cmudelusergroup8# /opt/cmu/bin/cmudelusergroup user1 user2 # /opt/cmu/bin/cmudelusergroup -a -m 60 -f usergrouplist # /opt/cmu/bin/cmuconsole computenodehostname Cmuconsole8Cmuconsole -- Connect to compute node management ports Cmupower8 Cmupower -p Boot -l rh6u0x8664 Cmupower -p OFF -n cn0001Cmupower -p OFF -u user1 Cmupower -p Uidon -u user2 # /opt/cmu/bin/cmucustomrun -l Cmucustomrun8Title # /opt/cmu/bin/cmucustomrun -t auditlspci # /opt/cmu/bin/cmuclone -f /tmp/nodelist -i sles11sp1x8664 Cmuclone8=node # /opt/cmu/bin/cmuclone -n node1 -i rh6u2x8664 Cmubackup -l myimage -n node20 -r 3 -e /tmp/err.log Cmubackup8Cmubackup -l myimage -n node20 -p cciss/c0d0p3,cciss/c0d0p1 # /opt/cmu/bin/cmuscanmacs -h hostname -p hostnameprefix Cmuscanmacs8Options naming Options general Example Example node definitions 173 # /opt/cmu/tools/cmurescanmac -n nodename N NICnum -h Cmurescanmac8Cmurescanmac -- Rescan the MAC address of a node Node name in the HP Insight CMU database Lg logicalgroup Cmumodnode8Filename # /opt/cmu/bin/cmumodnode -f nodes.txt Cmumonstat8 # /opt/cmu/bin/cmumonstat --all-nodes --all-sensors --stats # /opt/cmu/bin/cmumonstat --nodes=n0001,n0002,n0001-11# /opt/cmu/bin/cmumonstat --all-sensors --all-nodes # /opt/cmu/bin/cmumonstat --all-nodes --color To open the HP Insight CMU image rh5u5x8664 Cmuimageopen8# /opt/cmu/bin/cmuimageopen -h -i imagename # /opt/cmu/bin/cmuimageopen -i rh5u5x8664 Cmuimagecommit8 # /opt/cmu/bin/cmuimagecommit -i rh5u5x8664 Cmuconfignvidia8 Cmuconfignvidia -- Configure Nvidia GPU monitoring Cmuconfigamd8 Cmuconfigamd -- Configure AMD GPU monitoring # /opt/cmu/bin/cmuconfigintel -h -r -n Cmuconfigintel8Cmuconfigintel -- Configure Intel coprocessor monitoring Eth num1num2bond num1 Cmumgtconfig8Automatically defaults when reasonable Num # /opt/cmu/bin/cmumgtconfig -t # /opt/cmu/bin/cmumgtconfig -c# /opt/cmu/bin/cmumgtconfig -c -s dhcp Ocmudiffparameters Cmufirmwaremgmt8Cmufirmwaremgmt -- Verify and execute firmware Specifies a text file listing compute nodes Glossary Ebipa Package management Index Index 191