List of Examples | HP XC System 4.x Software manual

List of Examples

5-1	Submitting a Job from the Standard Input	50
5-2	Submitting a Serial Job Using LSF	50
5-3	Submitting an Interactive Serial Job Using LSF only	50
5-4	Submitting an Interactive Serial Job Using LSF and the LSF-SLURM External Scheduler	50
5-5	Submitting a Non-MPI Parallel Job	52
5-6	Submitting a Non-MPI Parallel Job to Run One Task per Node	52
5-7	Submitting an MPI Job	53
5-8	Submitting an MPI Job with the LSF-SLURM External Scheduler Option	53
5-9	Using the External Scheduler to Submit a Job to Run on Specific Nodes	55
5-10	Using the External Scheduler to Submit a Job to Run One Task per Node	55
5-11	Using the External Scheduler to Submit a Job That Excludes One or More Nodes	55
5-12	Using the External Scheduler to Launch a Command in Parallel on Ten Nodes	55
5-13	Using the External Scheduler to Constrain Launching to Nodes with a Given Feature	56
5-14	Submitting a Job Script	56
5-15	Submitting a Batch Script with the LSF-SLURM External Scheduler Option	57
5-16	Submitting a Batch Job Script That Uses a Subset of the Allocation	57
5-17	Submitting a Batch job Script That Uses the srun --overcommit Option	57
5-18	Environment Variables Available in a Batch Job Script	58
8-1	The vtjacobic Example Program	76
8-2	C Example – Running the vtjacobic Example Program	76
9-1	Simple Launch of a Serial Program	82
9-2	Displaying Queued Jobs by Their JobIDs	83
9-3	Reporting on Failed Jobs in the Queue	83
9-4	Terminating a Job by Its JobID	83
9-5	Cancelling All Pending Jobs	83
9-6	Sending a Signal to a Job	83
9-7	Using the sinfo Command (No Options)	83
9-8	Reporting Reasons for Downed, Drained, and Draining Nodes	84
10-1	Examples of LSF Job Launch	87
10-2	Examples of Launching LSF Jobs Without the srun Command	88
10-3	Job Allocation Information for a Running Job	97
10-4	Job Allocation Information for a Finished Job	98
10-5	Using the bjobs Command (Short Output)	98
10-6	Using the bjobs Command (Long Output)	99
10-7	Using the bhist Command (Short Output)	99
10-8	Using the bhist Command (Long Output)	100
10-9	Launching an Interactive MPI Job	102
10-10	Launching an Interactive MPI Job on All Cores in the Allocation	102

10 List of Examples

Image 10

HP XC System 4.x Software manual List of Examples

Contents

HP XC System Software XC User Guide Page Table of Contents Configuring Your Environment with Modulefiles Compiling and Linking Serial Applications Debugging Applications Srun Roles Srun Modes 107 115 125 131 10-1 List of Figures11-1 List of Tables List of Examples New and Changed Information in This Edition About This DocumentIntended Audience Typographic Conventions HP XC and Related HP Products Information Related Information HP XC System Software Users Guide Https//computing.llnl.gov/linux/slurm/documentation.html Http//systemimager.org Manpages Http//www-unix.mcs.anl.gov/mpi $ man discover $ man 8 discover HP Encourages Your Comments$ man -k keyword Page Page Determining the Node Platform Only to the administrator of the HP XC systemNode Specialization /proc/cpuinfo file is dynamic SAN Storage Storage and I/OFile System Local Storage File System Layout System Interconnect NetworkHP XC System Interconnects Network Address Translation NAT Determining System Configuration InformationUser Environment 1 LVS Application Development Environment CommandsParallel Applications Slurm Run-Time EnvironmentSerial Applications Load Sharing Facility LSF HP-MPI Requested by the HP-MPI mpirun commandHow LSF and Slurm Interact Nodes for the job Components, Tools, Compilers, Libraries, and Debuggers Page Using the Secure Shell to Log Using the SystemLVS Login Routing Logging In to the System Getting Information About Resources IntroductionGetting Information About Queues $ bqueues Performing Other Common User Tasks $ lsid Getting System Help and InformationDetermining the LSF Cluster Name and the LSF Execution Host $ man sinfo Configuring Your Environment with Modulefiles Overview of Modules Supplied Modulefiles Supplied Modulefiles HP-MPI Viewing Loaded Modulefiles Modulefiles Automatically Loaded on the SystemViewing Available Modulefiles Loading a Modulefile Viewing Modulefile-Specific Help Unloading a ModulefileModulefile Conflicts Creating a Modulefile $ module load modules $ man modulefile Developing Applications Application Development Environment Overview Compilers Compiler CommandsMPI Compiler Interrupting a Job Setting Debugging OptionsExamining Nodes and Partitions Before Running Jobs Developing Serial Applications Building Serial Applications Developing Parallel ApplicationsSerial Application Build Environment Parallel Application Build Environment OpenMP ModulefilesHP-MPI Pthreads PGI Fortran and C/C++ Compilers MPI LibraryIntel Fortran and C/C++Compilers GNU C and C++ Compilers Building Parallel Applications Developing Libraries To compile and link a C application using the mpicc commandDesigning Libraries for the CP4000 Platform Linkcommand 64-bit -L/opt/mypackage/lib/x8664 -lmystuff Linkcommand 32-bit -L/opt/mypackage/lib/i686 -lmystuffTo build a 64-bit application, you might enter Developing Applications Submitting Jobs Submitting a Serial Job Using LSFSubmitting a Serial Job with the LSF bsub Command Overview of Job Submission Example 5-1 Submitting a Job from the Standard Input Example 5-2 Submitting a Serial Job Using LSFSubmitting a Serial Job Through Slurm Only $ bsub -I srun hostname Submitting a Non-MPI Parallel Job Following is the command line used to compile this programSubmitting a Parallel Job To submit a parallel job Example 5-5 Submitting a Non-MPI Parallel Job $ bsub -n4 -I srun hostname Example 5-7 Submitting an MPI Job $ bsub -n4 -I mpirun -srun ./helloworld Arguments for the Slurm External Scheduler Nodelist=list of nodes $ bsub -n 10 -ext SLURMnodelist=n1-10 srun hostname $ bsub -n 10 -ext SLURMconstraint=dualcore -I srun hostname Submitting a Batch Job or Job ScriptExample 5-14 Submitting a Job Script $ bqueues -l dualcore grep Slurm $ bsub -n4 -ext SLURMnodes=4 -I ./myscript.sh $ bsub -n4 -I ./myscript.sh Use the squeue command to acquire information on the jobs Using a Script to Submit Multiple JobsUsing a Makefile to Submit Multiple Jobs Submitting Multiple MPI Jobs Across the Same Set of Nodes $ cat mymake Following command line makes the program and executes it$ tail 113.out $ cat 117.out Type=SLINUX64 Submitting a Job from a Host Other Than an HP XC HostRunning Preexecution Programs $ bsub -R type=SLINUX64 -n4 -I srun hostnamePage Debugging Parallel Applications Debugging ApplicationsDebugging Serial Applications TotalView SSH and TotalView Setting Up TotalViewDebugging with TotalView Module load mpi module load totalview Setting TotalView Preferences Using TotalView with SlurmUsing TotalView with LSF $ srun -Nx-A $ mpirun -tv -srun application Debugging an Application $ mpirun -tv -srun -n2 ./Psimple Exiting TotalView $ scancel --user usernameDebugging Running Applications Run the applicationPage Monitoring Node Activity Xtools Utilities Running Performance Health Tests Where $ ovp --verify=perfhealth/cpuusage You can list the available tests with the ovp -lcommand$ ovp -l $ ovp --verbose --verify=perfhealth/cpuusage HOMEDIRECTORY/ovpn16mmddyy.log HOMEDIRECTORY/ovpn16mmddyyr1.log Page Using the Intel Trace Collector and Intel Trace Analyzer Tuning ApplicationsBuilding a Program Intel Trace Collector and HP-MPI For more information, see the following Web site Running a Program Intel Trace Collector and HP-MPIExample 8-1 The vtjacobic Example Program Example 8-2 C Example Running the vtjacobic Example Program Intel Trace Collector and Analyzer with HP-MPI on HP XC Installation KitHP-MPI and the Intel Trace Collector Running a Program Following is a Fortran example called vtjacobif Visualizing Data Intel Trace Analyzer and HP-MPI Running a Program Across Nodes Using LSF# bsub -n4 -I mpirun.mpich -np 2 ./vtjacobic Page Introduction to Slurm Using SlurmLaunching Jobs with the srun Command Slurm Utilities Using the srun Command with LSF Monitoring Jobs with the squeue CommandUsing the srun Command with HP-MPI Srun Roles and Modes Getting System Information with the sinfo Command Terminating Jobs with the scancel CommandExample 9-5cancels all pending jobs Security Fault ToleranceJob Accounting $ sinfo -R Using LSF Information for LSF Overview of LSF Integrated with Slurm $ squeue --jobs $SLURMJOBID Lsfadmin@n16 ~$ bsub -n4 -I srun hostname Example 10-1 Examples of LSF Job Launch Lshosts and lsload commands display for each of these items Differences Between LSF and LSF Integrated with Slurm Submitted. This job may run immediately, or it may run Job TerminologyConnection back to the terminal from which the job was $ ssh n15 lshosts Are submitted with the bsub -Icommand Batch jobs are submitted with the srun -bcommand. ByDefault, the output is written to Batch system scheduling policies Submitting Jobs Using LSF Integrated with Slurm in the HP XC EnvironmentUseful Commands Job Startup and Job Control How LSF and Slurm Launch and Manage a Job LSF-SLURM External SchedulerLSF with Slurm Job Launch Exit Codes User logs in to login node n16 How LSF and Slurm Launch and Manage a Job Determining the LSF Execution Host Determining Available System Resources Examining System Core Status Following example shows the output from the lshosts commandGetting Information About the LSF Execution Host Node Examining System Queues Getting Information About JobsGetting Host Load Information Getting Information About the lsf Partition LSF job with Slurm allocated resources Getting Job Allocation InformationThis allocation string has the following values Than what the job requests Examining the Status of a Job Example 10-5 Using the bjobs Command Short OutputExample 10-4 Job Allocation Information for a Finished Job Output Provided by the bhist Command Example 10-6 Using the bjobs Command Long OutputExample 10-7 Using the bhist Command Short Output Viewing the Historical Information for a Job Use the bjobs command to view the Slurm Jobid Example 10-8 Using the bhist Command Long OutputTranslating Slurm and LSF JOBIDs $ bsub -I -n4 -ext SLURMnodes=4 /bin/bash Working Interactively Within an Allocation$ sacct -j $ bjobs -l 124 grep slurm This example assumes 2 cores per node Example 10-9 Launching an Interactive MPI JobAlternatively, you can use the following LSF Equivalents of Slurm srun Options LSF Equivalents of Slurm srun Options3describes the srun options and lists their LSF equivalents $ unset Slurmjobid $ unset Slurmnprocs Requests a specific list of nodes. Slurm Suppress informational message Running an X Terminal Session from a Remote Node Advanced TopicsEnabling Remote Execution with OpenSSH Determining IP Address of Your Local Machine Running an X terminal Session Using LSF Running an X terminal Session Using SlurmLogging in to HP XC System Run the job on 1 core Using the GNU Parallel Make CapabilityOptions used in this command are Allocate 4 cores Monitors display server address $ cd subdir srun $MAKE $ cd subdir srun -n1 -N1 $MAKE -j4 Example Procedure $ make PREFIX=’srun -n1 -N1 MAKEJ=-j4 Modified Makefile is invoked as follows Local Disks on Compute Nodes$ make PREFIX=srun -n1 -N1 MAKEJ=-j4 Communication Between Nodes Using Mpich on the HP XC System11.5 I/O Performance Considerations Shared File View Using Mpich with LSF Allocation Using Mpich with Slurm AllocationBsub command launches the wrapper script Launching a Serial Interactive Shell Through LSF ExamplesBuilding and Running a Serial Application Examine the LSF execution host information Running LSF Jobs with a Slurm Allocation Request Launching a Parallel Interactive Shell Through LSF Example 2. Four Cores on Two Specific Nodes Examine the running jobs information $ hostname n16 $ srun hostname n5 $ bjobs Show the environment Submitting a Simple Job Script with LSFExamine the finished jobs information Display the script Submit the job Submitting an Interactive Job with LSFRun the job Show the job allocation Exit the pseudo-terminal Run some commands from the pseudo-terminalShow the Slurm job ID View the interactive jobs View the node state Submitting an HP-MPI Job with LSFView the running job View the finished job Using a Resource Requirements String in an LSF Command$ bsub -n 8 -R ALPHA5 SLINUX64 \ -ext SLURMnodes=4-4 myjob 124 Glossary Network Availability set Fcfs Ipmi LVS PXE SVA Index Index LVS PGI 135