Submitting a Batch Job or Job Script

to the number provided by the -noption of the bsub command. Any additional SLURM srun options are job specific, not allocation-specific.

The mpi-jobnameis the executable file to be run. The mpi-jobnamemust be compiled with the appropriate HP-MPI compilation utility. Refer to the section titled Compiling applications in the HP-MPI User's Guide for more information.

Example 5-7shows an MPI job that runs a hello world program on 4 cores on 2 compute nodes.

Example 5-7 Submitting an MPI Job

$ bsub -n4 -I mpirun -srun ./hello_world

Job <24> is submitted to default queue <normal>. <<Waiting for dispatch ...>>

<<Starting on lsfhost.localdomain>> Hello world!

Hello world! I'm 1 of 4 on host1 Hello world! I'm 3 of 4 on host2 Hello world! I'm 0 of 4 on host1 Hello world! I'm 2 of 4 on host2

You can use the LSF-SLURM External Scheduler option to add capabilities at the job level and queue level by including several SLURM options in the command line. For example, you can use it to submit a job to run one task per node, or to submit a job to run on specific nodes. "LSF-SLURM External Scheduler" discusses this option.

Example 5-8shows an MPI job that uses the LSF-SLURM External Scheduler option to run the same hello world program on each of 4 compute nodes.

Example 5-8 Submitting an MPI Job with the LSF-SLURM External Scheduler Option

$ bsub -n4 -ext "SLURM[nodes=4]" -I mpirun -srun ./hello_world Job <27> is submitted to default queue <normal>.

<<Waiting for dispatch ...>> <<Starting on lsfhost.localdomain>> Hello world!

Hello world! I'm 1 of 4 on host1 Hello world! I'm 2 of 4 on host2 Hello world! I'm 3 of 4 on host3 Hello world! I'm 4 of 4 on host4

Some preprocessing may need to be done:

•If the MPI job requires the use of an appfile

•If there is another reason that prohibits the use of the srun command as the task launcher

This preprocessing should determine the node hostnames to which mpirun's standard task launcher should launch the tasks. In such scenarios, you need to write a batch script; there are several methods available for determining the nodes in an allocation. One method is to use the SLURM_JOBID environment variable with the squeue command to query the nodes. Another method is to use the LSF-HPC environment variables such as LSB_HOSTS and LSB_MCPU_HOSTS, which are prepared by the HP XC job starter script.

Use the following bsub command format to submit a batch job or job script:

bsub -nnum-procs [bsub-options]script-name

The -nnum-procsparameter, which is required for parallel jobs, specifies the number of cores the job requests.

The script-nameargument is the name of the batch job or script. The script can contain one or more srun or mpirun commands.

The script will execute once on the first allocated node, and the srun or mpirun commands within the script will be run on the allocated compute nodes.

Submitting a Batch Job or Job Script