Example 8-5: Allocating 12 Processors on 6 Nodes

$ bsub -I -n12 $MPI_ROOT/bin/mpirun -srun -n6 -N6./a.out 1

Note that LSF jobs can be submitted without the -I(interactive) option.

8.3.5 MPI Versioning

The mpirun command includes an option to print the version number. The -versionoption used with mpirun displays the major and minor version numbers. The mpi.h header includes matching constants as HP_MPI and HP_MPI_MINOR.

8.4 System Interconnect Support

HP-MPI on HP XC provides the following system interconnect support:

•Quadrics Elan4 protocol on Itanium® 2-based systems with support for srun startup, which is equivalent to prun on non-XC Elan systems.

•IT-API on InfiniBand. To use InfiniBand on XC, do not link -static. You must use dynamic linking.

•Myrinet GM2.1

•TCP/IP on supported system interconnects (for example, Gigabit Ethernet)

•Shared memory for intranode communication

8.4.1HP-MPI Performance on HP XC with Multiple System Interconnects

HP-MPI attempts to choose the best system interconnect for an application on XC systems. If an HP XC system has more than one high-speed system interconnect available, HP-MPI will attempt to use the fastest available. This choice is based on a default ordering of Quadrics (Elan), Myrinet (GM), InfiniBand (IT-API), or Gigabit Ethernet (TCP/IP). This default ordering can be redefined by using the MPI_IC_ORDER environment variable.

There may be situations where an HP XC system has both Gigabit Ethernet and another high-speed system interconnect. You may prefer to run your application on the Gigabit Ethernet system interconnect. To accomplish this, the mpirun command must be instructed to use the TCP protocol for communication and the system interconnect path must be specified by using the -TCPoption. By default, mpirun uses the system interconnect associated with what is returned by hostname, which usually corresponds to the non-Gigabit Ethernet high-speed system interconnect present. In these situations, you need to explicitly specify the Gigabit Ethernet subnet to be used with the -subnetoption.

To determine the proper subnet, run the following command, or ask your system administrator for the subnet address corresponding to Gigabit Ethernet.

$ /usr/sbin/ifconfig -a

Look for the eth1 interface which should be using a private network address, probably starting with 172.*.*.*. For example:

eth1 Link encap:Ethernet HWaddr 00:30:6E:4A:02:5E

inet addr:172.20.0.136 Bcast:172.255.255.255 Mask:255.0.0.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:84194577 errors:0 dropped:0 overruns:0 frame:0

TX packets:211420929 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000

RX bytes:6252673093 (5963.0 Mb) TX bytes:313590775738 (299063.4Mb)

Interrupt:56

Using HP-MPI 8-7