request. LSF-HPC always tries to pack multiple serial jobs on the same node, with one CPU per job. Parallel jobs and serial jobs cannot coexist on the same node.
After the LSF-HPC scheduler allocates the SLURM resources for a job, the SLURM allocation information is recorded with the job. You can view this information with the bjobs and bhist commands.
When LSF-HPC starts a job, it sets the SLURM_JOBID and SLURM_NPROCS environment variables in the job environment. SLURM_JOBID associates the LSF-HPC job with SLURM’s allocated resources. The SLURM_NPROCS environment variable is set to the originally requested number of processors. LSF-HPC dispatches the job from the LSF-HPC execution host, which is the same node on which LSF-HPC daemons run. The LSF-HPC JOB_STARTER script, which is configured for all queues, uses the srun command to launch a user job on the first node in the allocation. Your job can contain additional srun or mpirun commands to launch tasks to all nodes in the allocation.
While a job is running, all LSF-HPC-supported resource limits are enforced, including core limit, cputime limit, data limit, file size limit, memory limit and stack limit. When you kill a job, LSF-HPC uses the SLURM scancel command to propagate the signal to the entire job.
After a job finishes, LSF-HPC releases all allocated resources.
A detailed description, along with an example and illustration, of how LSF-HPC and SLURM cooperate to launch and manage jobs is provided in Section 7.1.4. It is highly recommended that you review this information.
1.4.4 HP-MPI
HP-MPI is a high-performance implementation of the Message Passing Interface standard and is included with the HP XC system. HP-MPI uses SLURM to launch jobs on an HP XC system — however, it manages the global MPI exchange so that all processes can communicate with each other.
HP-MPI complies fully with the MPI-1.2 standard. HP-MPI also complies with the MPI-2 standard, with some restrictions. HP-MPI provides an application programming interface and software libraries to support parallel, message-passing applications that are efficient, portable, and flexible. HP-MPI version 2.1 is included in this release of HP XC.
HP-MPI 2.1 for HP XC is supported on XC4000 and XC6000 clusters, and includes support for the following system interconnects:
•XC4000 Clusters — Myrinet, Gigabit Ethernet, TCP/IP, InfiniBand
•XC6000 Clusters — Quadrics Elan4, Gigabit Ethernet, TCP/IP
1.5Components, Tools, Compilers, Libraries, and Debuggers
This section provides a brief overview of the some of the common tools, compilers, libraries, and debuggers supported for use on HP XC.
An HP XC system is integrated with several open source software components. HP XC incorporates the Linux operating system, and its standard commands and tools, and does not diminish the Linux ABI in any way. In addition, HP XC incorporates LSF and SLURM to launch and manage jobs, and includes HP-MPI for high performance, parallel, message-passing applications, and HP MLIB math library for intensive computations.
Most standard open source compilers and tools can be used on an HP XC system, however they must be purchased separately. Several open source and commercially available software packages have been tested with the HP XC Software. The following list shows some of the software packages that have been tested for use with HP XC. This list provides an example of what is available on HP XC and is not intended as a complete list. Note that some of the packages listed are actually included as part of the HPC Linux distribution and as such are