HP Encourages Your Comments

Manpages for third-party software components might be provided as a part of the deliverables for that component.

Using discover(8) as an example, you can use either one of the following commands to display a manpage:

$ man discover $ man 8 discover

If you are not sure about a command you need to use, enter the man command with the -koption to obtain a list of commands that are related to a keyword. For example:

$ man -k keyword

HP encourages comments concerning this document. We are committed to providing documentation that meets your needs. Send any errors found, suggestions for improvement, or compliments to:

docsfeedback@hp.com

Include the document title, manufacturing part number, and any comment, error found, or suggestion for improvement you have concerning this document.

HP Encourages Your Comments 17

Page 17

Image 17

HP XC System 4.x Software manual HP Encourages Your Comments, $ man discover $ man 8 discover, $ man -k keyword

Contents

HP XC System Software XC User Guide Page Table of Contents Compiling and Linking Serial Applications Configuring Your Environment with Modulefiles Srun Roles Srun Modes Debugging Applications 115 107 131 125 11-1 List of Figures10-1 List of Tables List of Examples Intended Audience About This DocumentNew and Changed Information in This Edition Typographic Conventions HP XC and Related HP Products Information HP XC System Software Users Guide Related Information Https//computing.llnl.gov/linux/slurm/documentation.html Http//systemimager.org Http//www-unix.mcs.anl.gov/mpi Manpages $ man -k keyword HP Encourages Your Comments$ man discover $ man 8 discover Page Page Node Specialization Only to the administrator of the HP XC systemDetermining the Node Platform /proc/cpuinfo file is dynamic File System Storage and I/OSAN Storage Local Storage HP XC System Interconnects System Interconnect NetworkFile System Layout User Environment Determining System Configuration InformationNetwork Address Translation NAT 1 LVS Parallel Applications CommandsApplication Development Environment Serial Applications Run-Time EnvironmentSlurm Load Sharing Facility LSF How LSF and Slurm Interact Requested by the HP-MPI mpirun commandHP-MPI Nodes for the job Components, Tools, Compilers, Libraries, and Debuggers Page LVS Login Routing Using the SystemUsing the Secure Shell to Log Logging In to the System Getting Information About Queues IntroductionGetting Information About Resources $ bqueues Performing Other Common User Tasks Determining the LSF Cluster Name and the LSF Execution Host Getting System Help and Information$ lsid $ man sinfo Overview of Modules Configuring Your Environment with Modulefiles Supplied Modulefiles Supplied Modulefiles HP-MPI Viewing Available Modulefiles Modulefiles Automatically Loaded on the SystemViewing Loaded Modulefiles Loading a Modulefile Modulefile Conflicts Unloading a ModulefileViewing Modulefile-Specific Help $ module load modules $ man modulefile Creating a Modulefile Application Development Environment Overview Developing Applications MPI Compiler Compiler CommandsCompilers Examining Nodes and Partitions Before Running Jobs Setting Debugging OptionsInterrupting a Job Developing Serial Applications Serial Application Build Environment Developing Parallel ApplicationsBuilding Serial Applications Parallel Application Build Environment HP-MPI ModulefilesOpenMP Pthreads Intel Fortran and C/C++Compilers MPI LibraryPGI Fortran and C/C++ Compilers GNU C and C++ Compilers Building Parallel Applications Designing Libraries for the CP4000 Platform To compile and link a C application using the mpicc commandDeveloping Libraries To build a 64-bit application, you might enter Linkcommand 32-bit -L/opt/mypackage/lib/i686 -lmystuffLinkcommand 64-bit -L/opt/mypackage/lib/x8664 -lmystuff Developing Applications Submitting a Serial Job with the LSF bsub Command Submitting a Serial Job Using LSFSubmitting Jobs Overview of Job Submission Submitting a Serial Job Through Slurm Only Example 5-2 Submitting a Serial Job Using LSFExample 5-1 Submitting a Job from the Standard Input $ bsub -I srun hostname Submitting a Parallel Job Following is the command line used to compile this programSubmitting a Non-MPI Parallel Job To submit a parallel job $ bsub -n4 -I srun hostname Example 5-5 Submitting a Non-MPI Parallel Job $ bsub -n4 -I mpirun -srun ./helloworld Example 5-7 Submitting an MPI Job Nodelist=list of nodes Arguments for the Slurm External Scheduler $ bsub -n 10 -ext SLURMnodelist=n1-10 srun hostname Example 5-14 Submitting a Job Script Submitting a Batch Job or Job Script$ bsub -n 10 -ext SLURMconstraint=dualcore -I srun hostname $ bqueues -l dualcore grep Slurm $ bsub -n4 -I ./myscript.sh $ bsub -n4 -ext SLURMnodes=4 -I ./myscript.sh Using a Makefile to Submit Multiple Jobs Using a Script to Submit Multiple JobsUse the squeue command to acquire information on the jobs Submitting Multiple MPI Jobs Across the Same Set of Nodes $ tail 113.out Following command line makes the program and executes it$ cat mymake $ cat 117.out Running Preexecution Programs Submitting a Job from a Host Other Than an HP XC HostType=SLINUX64 $ bsub -R type=SLINUX64 -n4 -I srun hostnamePage Debugging Serial Applications Debugging ApplicationsDebugging Parallel Applications TotalView Debugging with TotalView Setting Up TotalViewSSH and TotalView Module load mpi module load totalview Using TotalView with LSF Using TotalView with SlurmSetting TotalView Preferences $ srun -Nx-A $ mpirun -tv -srun application $ mpirun -tv -srun -n2 ./Psimple Debugging an Application Debugging Running Applications $ scancel --user usernameExiting TotalView Run the applicationPage Xtools Utilities Monitoring Node Activity Where Running Performance Health Tests $ ovp -l You can list the available tests with the ovp -lcommand$ ovp --verify=perfhealth/cpuusage $ ovp --verbose --verify=perfhealth/cpuusage HOMEDIRECTORY/ovpn16mmddyy.log HOMEDIRECTORY/ovpn16mmddyyr1.log Page Building a Program Intel Trace Collector and HP-MPI Tuning ApplicationsUsing the Intel Trace Collector and Intel Trace Analyzer Example 8-1 The vtjacobic Example Program Running a Program Intel Trace Collector and HP-MPIFor more information, see the following Web site Example 8-2 C Example Running the vtjacobic Example Program HP-MPI and the Intel Trace Collector Installation KitIntel Trace Collector and Analyzer with HP-MPI on HP XC Following is a Fortran example called vtjacobif Running a Program # bsub -n4 -I mpirun.mpich -np 2 ./vtjacobic Running a Program Across Nodes Using LSFVisualizing Data Intel Trace Analyzer and HP-MPI Page Launching Jobs with the srun Command Using SlurmIntroduction to Slurm Slurm Utilities Using the srun Command with HP-MPI Monitoring Jobs with the squeue CommandUsing the srun Command with LSF Srun Roles and Modes Example 9-5cancels all pending jobs Terminating Jobs with the scancel CommandGetting System Information with the sinfo Command Job Accounting Fault ToleranceSecurity $ sinfo -R Information for LSF Using LSF $ squeue --jobs $SLURMJOBID Overview of LSF Integrated with Slurm Example 10-1 Examples of LSF Job Launch Lsfadmin@n16 ~$ bsub -n4 -I srun hostname Differences Between LSF and LSF Integrated with Slurm Lshosts and lsload commands display for each of these items Connection back to the terminal from which the job was Job TerminologySubmitted. This job may run immediately, or it may run $ ssh n15 lshosts Default, the output is written to Batch jobs are submitted with the srun -bcommand. ByAre submitted with the bsub -Icommand Batch system scheduling policies Useful Commands Using LSF Integrated with Slurm in the HP XC EnvironmentSubmitting Jobs Job Startup and Job Control LSF with Slurm Job Launch Exit Codes LSF-SLURM External SchedulerHow LSF and Slurm Launch and Manage a Job How LSF and Slurm Launch and Manage a Job User logs in to login node n16 Determining Available System Resources Determining the LSF Execution Host Getting Information About the LSF Execution Host Node Following example shows the output from the lshosts commandExamining System Core Status Getting Host Load Information Getting Information About JobsExamining System Queues Getting Information About the lsf Partition This allocation string has the following values Getting Job Allocation InformationLSF job with Slurm allocated resources Than what the job requests Example 10-4 Job Allocation Information for a Finished Job Example 10-5 Using the bjobs Command Short OutputExamining the Status of a Job Example 10-7 Using the bhist Command Short Output Example 10-6 Using the bjobs Command Long OutputOutput Provided by the bhist Command Viewing the Historical Information for a Job Translating Slurm and LSF JOBIDs Example 10-8 Using the bhist Command Long OutputUse the bjobs command to view the Slurm Jobid $ sacct -j Working Interactively Within an Allocation$ bsub -I -n4 -ext SLURMnodes=4 /bin/bash $ bjobs -l 124 grep slurm Alternatively, you can use the following Example 10-9 Launching an Interactive MPI JobThis example assumes 2 cores per node 3describes the srun options and lists their LSF equivalents LSF Equivalents of Slurm srun OptionsLSF Equivalents of Slurm srun Options $ unset Slurmjobid $ unset Slurmnprocs Requests a specific list of nodes. Slurm Suppress informational message Enabling Remote Execution with OpenSSH Advanced TopicsRunning an X Terminal Session from a Remote Node Determining IP Address of Your Local Machine Logging in to HP XC System Running an X terminal Session Using SlurmRunning an X terminal Session Using LSF Options used in this command are Allocate 4 cores Using the GNU Parallel Make CapabilityRun the job on 1 core Monitors display server address $ cd subdir srun -n1 -N1 $MAKE -j4 $ cd subdir srun $MAKE $ make PREFIX=’srun -n1 -N1 MAKEJ=-j4 Example Procedure $ make PREFIX=srun -n1 -N1 MAKEJ=-j4 Local Disks on Compute NodesModified Makefile is invoked as follows 11.5 I/O Performance Considerations Using Mpich on the HP XC SystemCommunication Between Nodes Shared File View Bsub command launches the wrapper script Using Mpich with Slurm AllocationUsing Mpich with LSF Allocation Building and Running a Serial Application ExamplesLaunching a Serial Interactive Shell Through LSF Examine the LSF execution host information Running LSF Jobs with a Slurm Allocation Request Example 2. Four Cores on Two Specific Nodes Launching a Parallel Interactive Shell Through LSF $ hostname n16 $ srun hostname n5 $ bjobs Examine the running jobs information Examine the finished jobs information Submitting a Simple Job Script with LSFShow the environment Display the script Run the job Submitting an Interactive Job with LSFSubmit the job Show the job allocation Show the Slurm job ID Run some commands from the pseudo-terminalExit the pseudo-terminal View the interactive jobs View the running job Submitting an HP-MPI Job with LSFView the node state $ bsub -n 8 -R ALPHA5 SLINUX64 \ -ext SLURMnodes=4-4 myjob Using a Resource Requirements String in an LSF CommandView the finished job 124 Network Availability set Glossary Fcfs Ipmi LVS PXE SVA Index Index LVS PGI 135