Manuals / HP / Computer Equipment / Software

HP Insight Cluster Management Utility manual Advanced topics

Models: Insight Cluster Management Utility

1 191

Download 191 pages 59.43 Kb

Page 122

122 Advanced topics

Image 122

HP Insight Cluster Management Utility manual Advanced topics

Contents

HP Insight Cluster Management Utility Copyright 2013 Hewlett-Packard Development Company, L.P Contents Defining a cluster with HP Insight CMU Provisioning a cluster with HP Insight CMUMonitoring a cluster with HP Insight CMU Actions Alerts Alert reactions Managing a cluster with HP Insight CMUAdvanced topics 112 Support and other resources 123 Troubleshooting 126Detailed installation instructions 131 HP Insight CMU manpages 139Glossary 187 Index 189 Figures Examples Date command Dmidecode commandTables Overview FeaturesHP Insight CMU configuration Compute node monitoringCompute node administration System disk replicationManagement node hardware requirements Installing and upgrading HP Insight CMUInstalling HP Insight CMU Configuring the local smart array card Planning for compute node installationFirmware upgrade requirements Configuring the management cards7.1 DL3xx, DL5xx, DL7xx, Blades Configuring the BiosNIC 1 PXE boot or PXE enabled NIC 2 Disabled Speed 9600 Bd7.2 DL160 G5, DL165c G5, DL165c G6, and DL180 G5 Servers IDENIC1 control Enabled Share NIC mode DisabledDhcp Disabled NIC1 PXE Enabled7.4 SL2x170z G6 and DL170h G6 Servers Bios setting NIC2 on the SL2x170z G6 ServerHP Insight CMU kit delivery Preparing for installationPreinstallation limitations Rhel 6 support Operating system supportHP Insight CMU CD-ROM directory structure SELinux and HP Insight CMU HP Insight CMU installation checklistLogin privileges # chkconfig --add cmu Installation proceduresRun /opt/cmu/bin/cmumgtconfig -c Installing HP Insight CMU with high availability Installing and upgrading HP Insight CMU Software prerequisites Installing HP Insight CMU under HAHA hardware requirements OverviewInstalling and testing Configuring HA control of HP Insight CMUHP Insight CMU HA service requirements # /etc/init.d/cmuserver start Cmuadmin1# /etc/init.d/cmu startStart cmuserver Cmuadmin2# /opt/cmu/tools/cmuhapostinstall Cmu hacmu service needs restartVar/log/cmuservicehostname.log file for errors # /etc/init.d/cmu setaudit # /etc/init.d/cmu stopCmuha nothing to backup from the cmu HA share HP Insight CMU configuration considerationsUpgrading HP Insight CMU HA service Run cmuhapostinstall on serverDependencies Stopping the HP Insight CMU serviceUpgrading HP Insight CMU Upgrading Java Runtime EnvironmentSaving the HP Insight CMU database Restoring the HP Insight CMU databaseLaunching the HP Insight CMU GUI HP Insight CMU service statusDefining a cluster with HP Insight CMU HP Insight CMU main windowClick Options→Unprivileged Mode Administrator modeQuitting administrator mode Server requirementsHigh-level checklist for building an HP Insight CMU cluster Cluster administrationNode management Node management windowScanning nodes Scan node dialogManagement card password window Adding nodesCluster Administration→Node Management Modifying nodesDeleting nodes Network entity managementImporting nodes Exporting nodesAdding network entities Deleting network entitiesProvisioning a cluster with HP Insight CMU Logical group managementRenaming logical groups Modifying logical groupsDeleting logical groups Autoinstall templates AutoinstallAutoinstall requirements Autoinstall calling methodsCreating an autoinstall logical group Using autoinstall from GUIEnabling autoinstall Restart cmuserverAutoinstall compute nodes Registering compute nodesAdding nodes to autoinstall logical group Using autoinstall from CLIRegistering an autoinstall logical group Cmu addtologicalgroup node to logicalgroupnameWhere nodes.txt is the list of nodes to autoinstall CustomizationBacking up a disk from a compute node in a logical group Backing upRestrictions # /opt/cmu/image/logicalgroupname Cloning procedure CloningPreconfiguration Cloning statusReconfiguration Default content of prereconf.sh isNode static info Rescan MACAfter editing the image, commit changes HP Insight CMU image editorExpanding an image # cmuimagecommit -i rh5u4x8664Saving a modified cloning image HP Insight CMU diskless environmentsModifying an image # /opt/cmu/bin/cmuimagecommit -i rh5u4x8664On the management node Modifying the Tftp server configurationOperating systems supported On the golden nodeCreating a diskless image Activating the diskless featurePopulating the HP Insight CMU database Creating a diskless logical groupAdding a new logical group # /opt/cmu/cmucli Adding nodes into the logical groupFrom the CLI Cmu probekernelUnderstanding the structure of a diskless image Booting the compute nodesCmu boot net myTestImage node1 noden Customizing your diskless imageUsing reconf-diskless-image.sh Templates and image file Do not update the /opt/cmu/image/imageName/root directoryBest practices for diskless clusters Node. /opt/cmu/image/imageName/snapshot/nodeNameOn Red Hat Configuring additional NFS servers# chkconfig nfs on On Sles When a diskless logical group is created# chkconfig nfsserver on Sample fileWhen a node is added to the diskless logical group Comments on High Availability HADeploying the monitoring client Installing the HP Insight CMU monitoring clientMonitoring a cluster with HP Insight CMU Monitoring the cluster Launch the HP Insight CMU GUINode and group status Selecting the central frame displayGlobal cluster view in the central frame Monitoring windowResource view in the central frame Resource view overviewNode view in the central frame Detail mode in resource viewGauge widget Using time view Node detailsAdaptive stacking Getting startedTagging nodes Bindings and options Mouse controlTroubleshooting Technical dependenciesArchiving user groups Visualizing history dataAction and alert files Tuning HP Insight CMU monitoringStopping HP Insight CMU monitoring LimitationsActions AlertsAlerts Alert reactionsText of the Description for this reaction Name of the alert that caused the reactionLevel of the alert # chkconfig --add collectl Using collectl for gathering monitoring dataInstalling and starting collectl on compute nodes # /etc/init.d/collectl start# collectl -c 1 -s+C --export lexpr ActionAndAlertFile.txt file# exportfs -r # mkdir /var/log/collectl # vi /etc/exportsVar/log/collectl *rw,sync,noallsquash,norootsquash # cp -a /var/www/html/colplot /opt/cmu/www/colplotRestart collectl # mkdir /var/log/collectl # vi /etc/fstab# vi /etc/collectl.conf # /etc/init.d/collectl restartSelect plotting options, then click Generate Plot Monitoring GPUs and coprocessorsMonitoring Nvidia GPUs # cmuconfigamd Monitoring AMD GPUsMonitoring Intel coprocessors Review the results and verify no errors are reported # /opt/cmu/bin/cmuconfigintelExtended metric support HP Insight CMU alert converted to SIM eventOpt/cmu/bin/cmusubmitextendedmetrics -f datafile Unprivileged user menu Administrator menuManaging a cluster with HP Insight CMU SSH connectionVirtual serial port connection Power offManagement card connection ShutdownChange UID LED status BootReboot Dshbak Multiple windows broadcastSingle window pdsh Cmudiff examples Cmupdsh cmudiff -d Cmupdsh dmidecodeCmupdsh cmudiff -h # vi /root/.ssh/config Parallel distributed copy pdcpUser group management Adding user groupsDeleting user groups Viewing and analyzing Bios settingsHP Insight firmware management Renaming user groupsCustomizing the GUI menu Checking Bios versionsInstalling and upgrading firmware HP Insight CMU CLI Saving user settingsBasic commands Starting a CLI interactive sessionFor example, to get more information about the halt command Help commandsGetting help for a command Cmu helpDisplaying logical groups of a cluster Executing a command on one nodeSpecifying nodes Displaying nodes of a logical groupUsing wildcards Executing a command on a list of nodesExecuting a command on a range of nodes Executing a command on all nodesBooting a set of nodes Administration and cloning commandsExecuting a command on specific nodes of a logical group Broadcasting commands to a set of nodesHalting a set of nodes Rebooting a set of nodesPowering off a set of nodes Cmu locate on o185i192 Setting the locator LED on or offCloning a set of nodes Cmu locate off o185i192Backing up a node Adding a new logical groupAdding nodes to a logical group Cmu backup Cmu modifypassword lo100i Modifying a management card passwordCmu modifypassword ILOILOCMlo100i Discovering MAC address for new nodes# /opt/cmu/bin/pdcp -w cn0001,cn0002 source /tmp/dest Administration utilities pdcp and pdshHP Insight CMU Linux shell commands # /opt/cmu/bin/pdsh -w cn0001,cn0002 lsAccessing the GUI for non-root users Advanced topicsConfiguring sudo support Custom menu options for non-root usersSbarney ALL = Nopasswd Cmupower Cmuimage Cmuetc Cjones ALL = CmupowerBstevens ALL = Nopasswd Cmupower Cmuimage HP Insight CMU diskless APIName of the new logical group Build diskless imageDelete diskless image Boot diskless node Configure diskless nodeUnconfigure diskless node Lo100i HP Insight CMU remote hardware control APIDiskless check NoneUidoff OffOsoff UidonCMUVALIDHARDWARETYPES=ILOlo100iILOCM CMUVALIDHARDWARETYPES=ILOlo100iILOCMIPMICMUvSMPPREFIX=vSMP Support for ScaleMPCloning mechanisms Cloning mechanisms Advanced topics Related information Support and other resourcesContacting HP Computer output CommandTypographic conventions User inputTypographic conventions HP Insight CMU logs TroubleshootingNetwork boot issues Troubleshooting network boot Backup issuesTroubleshooting switch issues GUI problems Cloning issuesAdministration command problems If not, restart the HP Insight CMU serviceCertificate error Detailed Java exception isTroubleshooting Activating xinetd services Detailed installation instructionsInstall required RPMs # chkconfig nfs on # /etc/init.d/nfs startVerifying the Dhcpd listen interface Firewall configurationJava installation On the HP Insight CMU management nodeConfiguring the HP Insight CMU management server hostname Installing HP Insight CMU licensingSetting the Java Path MethodEdit the /opt/cmu/etc/cmuserver.conf file Cmu service needs restartStarting HP Insight CMU # vi /opt/cmu/etc/cmuserver.confVerifying the HP Insight CMU state Configuring HP Insight CMU to start automaticallyInstalling HP Insight CMU on the GUI client workstation Cmuserver utility reports the state of the daemonsUsing an X Window server Configuring the GUI client on Linux workstationsUsing an ssh tunnel Launching the HP Insight CMU directly from the Java file Launching the HP Insight CMU GUI using a web browserActivating the HP Insight CMU GUI HP Insight CMU GUI HP Insight CMU manpages Cmushownodes8 # /opt/cmu/bin/cmushownodes -a -o %n %i %k %m default %b %t # /opt/cmu/bin/cmushownodesHelp Cmushowlogicalgroups8# /opt/cmu/bin/cmushowlogicalgroups -h logicalgroupname # /opt/cmu/bin/cmushowlogicalgroups# /opt/cmu/bin/cmushownetworkentities Cmushownetworkentities8# /opt/cmu/bin/cmushownetworkentities -h networkentity # /opt/cmu/bin/cmushownetworkentities rack1# /opt/cmu/bin/cmushowusergroups Cmushowusergroups8# /opt/cmu/bin/cmushowusergroups -h usergroup # /opt/cmu/bin/cmushowusergroups user1# /opt/cmu/bin/cmushowarchivedusergroups -p -f Cmushowarchivedusergroups8# /opt/cmu/bin/cmushowarchivedusergroups # /opt/cmu/bin/cmushowarchivedusergroups -f -sNode-number num Cmuaddnode8Mgtcardip -T--mgt-card ILOlo100iILOCM -R--arch architecture # cat nodes.txt Command-line modeProcessing 1 node # /opt/cmu/bin/cmuaddnode -f nodes.txt# /opt/cmu/bin/cmuaddnetworkentity rack1 rack2 Cmuaddnetworkentity8Filename inputfile # /opt/cmu/bin/cmuaddnetworkentity -f networkentitylist# /opt/cmu/bin/cmuaddlogicalgroup -n test -d cciss/c0d0 Cmuaddlogicalgroup8Filename inputfile input logical groups from inputfile Name # /opt/cmu/bin/cmuaddlogicalgroup -f logicalgroupfileNodenamefile Cmuaddtologicalgroupcandidates8Logicalgroup # /opt/cmu/bin/cmuaddusergroup -f usergrouplist Cmuaddusergroup8# /opt/cmu/bin/cmuaddusergroup user1 user2 # /opt/cmu/bin/cmuaddtousergroup -t group1 cn0003 Cmuaddtousergroup8Usergroup # /opt/cmu/bin/cmuaddtousergroup -t group1 -f nodenamefileCmuchangeactivelogicalgroup8 To flag cn0001 active in logical group rh6u0x8664# /opt/cmu/bin/cmuchangenetworkentity -t rack1 cn0001 Cmuchangenetworkentity8Networkentity Delete nodes from this logical group Cmudelfromlogicalgroupcandidates8Delete one or more nodes from a logical group To delete one node from a logical group rh6u0x8664Cmudelfromnetworkentity8 # /opt/cmu/bin/cmudelfromnetworkentity -t rack1 node1Cmudelarchivedusergroup -- Delete an archived user group # /opt/cmu/bin/cmudelarchivedusergroup -h -v-t timeout -dCmudelarchivedusergroup8 Delete an archived user groupCmudelfromusergroup8 # /opt/cmu/bin/cmudelfromusergroup -t user1 node1# /opt/cmu/bin/cmudellogicalgroup -f logicalgrouplist Cmudellogicalgroup8# /opt/cmu/bin/cmudellogicalgroup rh5u5x8664 sles11sp1x8664 # /opt/cmu/bin/cmudelnetworkentity -f networkentitylist Cmudelnetworkentity8# /opt/cmu/bin/cmudelnetworkentity rack1 rack2 # /opt/cmu/bin/cmudelnode -f nodelist Cmudelnode8# /opt/cmu/bin/cmudelnode node1 node2 Cmudelsnapshots8 Delete monitoring snapshots from the history database# /opt/cmu/bin/cmudelusergroup -f usergrouplist Cmudelusergroup8# /opt/cmu/bin/cmudelusergroup user1 user2 # /opt/cmu/bin/cmudelusergroup -a -m 60 -f usergrouplist# /opt/cmu/bin/cmuconsole computenodehostname Cmuconsole8Cmuconsole -- Connect to compute node management ports Cmupower8 Cmupower -p Boot -l rh6u0x8664 Cmupower -p OFF -n cn0001Cmupower -p OFF -u user1 Cmupower -p Uidon -u user2# /opt/cmu/bin/cmucustomrun -l Cmucustomrun8Title # /opt/cmu/bin/cmucustomrun -t auditlspci# /opt/cmu/bin/cmuclone -f /tmp/nodelist -i sles11sp1x8664 Cmuclone8=node # /opt/cmu/bin/cmuclone -n node1 -i rh6u2x8664Cmubackup -l myimage -n node20 -r 3 -e /tmp/err.log Cmubackup8Cmubackup -l myimage -n node20 -p cciss/c0d0p3,cciss/c0d0p1 # /opt/cmu/bin/cmuscanmacs -h hostname -p hostnameprefix Cmuscanmacs8Options naming Options general Example Example node definitions173 # /opt/cmu/tools/cmurescanmac -n nodename N NICnum -h Cmurescanmac8Cmurescanmac -- Rescan the MAC address of a node Node name in the HP Insight CMU databaseLg logicalgroup Cmumodnode8Filename # /opt/cmu/bin/cmumodnode -f nodes.txt Cmumonstat8 # /opt/cmu/bin/cmumonstat --all-nodes --all-sensors --stats # /opt/cmu/bin/cmumonstat --nodes=n0001,n0002,n0001-11# /opt/cmu/bin/cmumonstat --all-sensors --all-nodes # /opt/cmu/bin/cmumonstat --all-nodes --colorTo open the HP Insight CMU image rh5u5x8664 Cmuimageopen8# /opt/cmu/bin/cmuimageopen -h -i imagename # /opt/cmu/bin/cmuimageopen -i rh5u5x8664Cmuimagecommit8 # /opt/cmu/bin/cmuimagecommit -i rh5u5x8664Cmuconfignvidia8 Cmuconfignvidia -- Configure Nvidia GPU monitoringCmuconfigamd8 Cmuconfigamd -- Configure AMD GPU monitoring# /opt/cmu/bin/cmuconfigintel -h -r -n Cmuconfigintel8Cmuconfigintel -- Configure Intel coprocessor monitoring Eth num1num2bond num1 Cmumgtconfig8Automatically defaults when reasonable Num# /opt/cmu/bin/cmumgtconfig -t # /opt/cmu/bin/cmumgtconfig -c# /opt/cmu/bin/cmumgtconfig -c -s dhcp Ocmudiffparameters Cmufirmwaremgmt8Cmufirmwaremgmt -- Verify and execute firmware Specifies a text file listing compute nodesGlossary EbipaPackage management Index Index 191