Escali 4.4 manual Communication protocols on DAT-devices, 6 SCI, Channel buffer

Section: 2.3 Communication protocols on DAT-devices

2.2.6 SCI

This is a built-in device that uses the Scali SCI driver and library (ScaSCI). This driver is for the Dolphin SCI network cards. Please see the ScaSCI Release Notes for specific requirements. This device is straight forward and requires no configuration itself, but for multi-dimensional toruses (2D and 3D) the Scali SCI Management system (ScaConf) needs to be running somewhere in your system. Refer to Appendix C for installation and configuration of the Scali SCI Management software.

2.3 Communication protocols on DAT-devices

In SMC, the communication protocol used to transfer data between a sender and a receiver depends on the size of the message to transmit, as illustrated in Figure 2-3:.

Increasing message size

Transporter protocol: message size > eager_size

Eagerbuffering protocol:

channel_inline_threshold < message size <= eager_size

Inlining protocol

0 <= message size <= channel_inline_threshold

Figure 2-3:Thresholds for different communication protocol

The default thresholds that control whether a message belongs to the inlining, eagerbuffering or transporter protocols can be controlled from the application launch program (mpimon) described in chapter 3.

Figure 2-4:illustrates the node resources associated with communication and mechanisms implemented in Scali MPI Connect for handling messages of different sizes. The three communication protocols from Figure 2-3:rely on buffers located in the main memory of the nodes. This memory is allocated as shared, i.e., it is not private to a particular process in the node. Each process has one set of receiving buffers for of the processes it communicates with. As the figure shows all communication relies on the sending process depositing messages directly into the communication buffers of the receiver. For Inline and Eagerbuffering the management of the buffer resources does not require participation from the receiving process, because of their designs as ring buffers.

2.3.1 Channel buffer

The Channel ringbuffer is divided into equally sized entries. The size varies differs for different architectures and networks; see “Scali MPI Connect Release Notes” for details. An entry in the ringbuffer, which is used to hold the information forming the message envelope, is reserved each time a message is being sent, and is used by the inline protocol, the eagerbuffering protocol, and the transporter protocol. In addition, one ore more entries are used by the inline protocol for application data being transmitted.

Scali MPI Connect Release 4.4 Users Guide 16