Manuals / Q-Logic / Computer Equipment / Switch

Q-Logic IB6054601-00 D Lock Enough Memory on Nodes When Using Slurm, # /sbin/fuser -k /dev/ipath

Models: IB6054601-00 D

1 122
Download 122 pages 48.66 Kb
Page 74
Image 74

B – Integration with a Batch Queuing System

Lock Enough Memory on Nodes When Using SLURM

Q

The following command will terminate all processes using the InfiniPath interconnect:

#/sbin/fuser -k /dev/ipath

For more information, see the man pages for fuser(1) and lsof(8).

NOTE: Run these commands as root to insure that all processes are reported.

B.2

Lock Enough Memory on Nodes When Using SLURM

This is identical to information provided in appendix C.8.11. It is repeated here for your convenience.

InfiniPath MPI requires the ability to lock (pin) memory during data transfers on each compute node. This is normally done via /etc/initscript, which is created or modified during the installation of the infinipath RPM (setting a limit of 64MB, with the command "ulimit -l 65536").

Some batch systems, such as SLURM, propagate the user’s environment from the node where you start the job to all the other nodes. For these batch systems, you may need to make the same change on the node from which you start your batch jobs.

If this file is not present or the node has not been rebooted after the infinipath RPM has been installed, a failure message similar to this will be generated:

$ mpirun -m ~/tmp/sm -np 2 -mpi_latency 1000 1000000 node-00:1.ipath_update_tid_err: failed: Cannot allocate memory mpi_latency: /fs2/scratch/infinipath-build-1.3/mpi-1.3/mpich/psm/src mq_ips.c:691:

mq_ipath_sendcts: Assertion ‘rc == 0’ failed. MPIRUN: Node program unexpectedly quit. Exiting.

You can check the ulimit -lon all the nodes by running ipath_checkout. A warning will be given if ulimit -lis less that 4096.

There are two possible solutions to this. If infinipath is not installed on the node where you start the job, set this value in the following way. You must be root to set it:

#ulimit -l 65536

Or, if you have installed infinipath on the node, reboot it to insure that /etc/initscript is run.

B-4

IB6054601-00 D

Page 73 Page 75
Page 74
Image 74
Q-Logic IB6054601-00 D manual Lock Enough Memory on Nodes When Using Slurm, # /sbin/fuser -k /dev/ipath, # ulimit -l
Page 73 Page 75

IB6054601-00 D specifications

The Q-Logic IB6054601-00 D is a high-performance InfiniBand adapter card designed for data centers and enterprise applications requiring robust connectivity and low-latency communication. This adapter is part of QLogic's extensive portfolio of networking solutions, catering to the needs of high-performance computing (HPC), cloud computing, and virtualization environments.

One of the standout features of the IB6054601-00 D is its capability to support data transfer rates of up to 56 Gbps. This makes it ideal for applications demanding large bandwidth and quick data processing. The adapter is optimized for RDMA (Remote Direct Memory Access) technology, which allows data to be transferred directly between the memory of different computers without involving the CPU. This reduces latency and CPU overhead, leading to enhanced overall system performance.

The architecture of the IB6054601-00 D includes support for a dual-port design, which offers increased bandwidth, redundancy, and fault tolerance. This dual-port configuration is especially advantageous for environments that require high availability and reliability, such as financial services and mission-critical applications.

The adapter utilizes advanced error detection and correction mechanisms, ensuring that data integrity is maintained during transmission. With features like adaptive routing and congestion management, the IB6054601-00 D is capable of optimizing the handling of data flows, thereby enhancing performance even under heavy loads.

In terms of compatibility, the Q-Logic IB6054601-00 D supports a wide range of operating systems and virtualization technologies, making it easy to integrate into diverse IT environments. It also includes drivers and software packages that facilitate seamless deployment and management.

In addition to high-speed connectivity, the adapter is designed with power efficiency in mind. It adheres to Energy Star regulations, helping organizations lower their operational costs while minimizing their environmental footprint.

Overall, the Q-Logic IB6054601-00 D stands out for its high throughput, low latency, and reliability. Its combination of advanced features and technologies positions it as an excellent choice for organizations looking to enhance their data center performance and maximize the efficiency of their network infrastructure. With the growing demands for faster and more efficient data transfer, solutions like the IB6054601-00 D are essential in meeting the evolving needs of modern enterprises.