PAR Technologies V5 manual Insight ParaStation5, ParaStation5 pscom communication library

Chapter 5. Insight ParaStation5

This chapter provides more technical details and background information about ParaStation5.

5.1. ParaStation5 pscom communication library

The ParaStation communication library libpscom offers secure and reliable end-to-end connectivity. It hides the actual transport and communication characteristics from the application and higher level libraries.

The libpscom library supports a wide range of interconnects and protocols for data transfers. Using a generic plug-in system, this library may open connections using the following networks and protocols:

•TCP: uses standard TCP/IP sockets to transfer data. This protocol may use any interconnect. Support for this protocol is built-in to the libpscom.

•P4sock: uses an optimized network protocol for Ethernet (see Section 5.2, “ParaStation5 protocol p4sock”, below). Support for this protocol is built-in to the libpscom.

•InfiniBand: based on a vapi kernel layer and a libvapi library, typically provided by the hardware vendor, the libpscom may use InfiniBand to actually transfer data. The corresponding plug-in library is called libpscom4vapi.

•Myrinet: using the GM library and kernel level module, the libpscom library is able to use Myrinet for data transfer. The particular plug-in library is called libpscom4gm.

•Shared Memory: for communication within a SMP node, the libpscom library uses shared memory. Support for this protocol is built-in to the libpscom.

•DAPL: The libpscom supports a DAPL transport layer. Using the libpscom4dapl plug-in, it may transfer data across various networks like Infiniband or 10G Ethernet using a vendor-provided libdapl.

•QsNet: The libpscom supports the QsNetII transport layer. Using the libpscom4elan plug-in, it may transfer data using the libelan.

The interconnect and protocol used between two distinct processes is chosen while opening the connection between those processes. Depending on available hardware, configuration (see Section 4.1, “Configuration of the ParaStation system”) and current environment variables (see Section 5.8, “ControllingParaStation5 communication paths”), the library automatically selects the fastest available communication path.

The library routines for sending and receiving data handle arbitrary large buffers. If necessary, the buffers will be fragmented and reassembled to meet the underlying transport requirements.

The application is dynamically linked with the libpscom.so library. At runtime, this library loads plug- ins for various interconnects, see above. For more information on controlling ParaStation communication pathes, refer to Section 5.8, “ControllingParaStation5 communication paths”.

5.2.ParaStation5 protocol p4sock

ParaStation5 provides its own communication protocol for Ethernet, called p4sock. This protocol is designed for extremely fast and reliable communication within a closed and homogeneous compute cluster environment.

The protocol implements a reliable, connection-oriented communication layer, especially designed for very low overhead. As a result, it delivers very low latencies.

The p4sock protocol is encapsulated within the kernel module p4sock.ko. This module is loaded on system startup or whenever the ParaStation5 daemon psid(8) starts up and the p4sock protocol is enabled within the configuration file parastation.conf(5).

ParaStation5 Administrator's Guide 17