Q-Logic IB6054601-00 D manual Generating the mpihosts File, $ batchmpirun -np n mympiprogram

B – Integration with a Batch Queuing System A Batch Queuing Script

Q

require that his node program be the only application running on each node CPU. In a typical batch environment, the MPI user would still specify the number of node programs, but would depend on the batch system to allocate specific nodes when the required number of CPUs becomes available. Thus, batch_mpirun would take at least an argument specifying the number of node programs and an argument specifying the MPI program to be instantiated. For example,

$ batch_mpirun -np n my_mpi_program

After parsing the command line arguments, the next step of batch_mpirun would be to request an allocation of n processors from the batch system. In SLURM, this would use the command

eval ‘srun --allocate --ntasks=$np --no-shell‘

Make sure to use back-quotes rather than normal single-quotes. $np is the shell variable that your script has set from the parsing of its command line options. The --no-shelloption to srun prevents SLURM from starting a subshell. The srun command is run with eval in order to set the SLURM_JOBID shell variable from the output of the srun command.

With these specified arguments, the SLURM function srun blocks until there are $np processors available to commit to the caller. When the requested resources are available, this command opens a new shell and allocates the requested number of processors to it.

B.1.2

Generating the mpihosts File

Once the batch system has allocated the required resources, your script must generate a mpihosts file, which contains a list of nodes that will be used. To do this, it must find out which nodes the batch system has allocated, and how many processes we can start on each node. This is the part of the script batch_mpirun that performs these tasks:

mpihosts_file=‘mktemp -p /tmp mpihosts_file.XXXXXX‘

srun --jobid=${SLURM_JOBID} hostname -s sort uniq -c \ awk ’{printf "%s:%s\n", $2, $1}’ > $mpihosts_file

The first command creates a temporary hosts file with a random name, and assigns the name to the variable mpihosts file it has generated.

The next instance of the SLURM srun command runs hostname -sonce per process slot that SLURM has allocated to us. If SLURM has allocated two slots on one node, we thus get the output of hostname -stwice for that node.

The sort uniq -ccomponent tells us the number of times each unique line was printed. The awk command converts the result into the mpihosts file format used

B-2 IB6054601-00 D