Escali 4.4 Good programming practice with SMC, Matching MPIRecv with MPIProbe, Using MPIBsend

Section: 3.9 Good programming practice with SMC

3.9 Good programming practice with SMC

3.9.1 Matching MPI_Recv() with MPI_Probe()

During development and testing of SMC, Scali has come across several application programs with the following code sequence:

while (...) {

MPI_Probe(MPI_ANY_SOURCE, MPI_ANY_TAG, comm, sts); if (sts->MPI_TAG == SOME_VALUE) {

MPI_Recv(buf, cnt, dtype, MPI_ANY_SOURCE, MPI_ANY_TAG, comm, sts); doStuff();

}

doOtherStuff();

}

For MPI implementations that have one, and only one, receive-queue for all senders, the program’s code sequence works as desired. However, the code will not work as expected with SMC. SMC uses one receive-queue per sender (inside each MPI-process). Thus, a message from one sender can bypass the message from another sender. In the time-gap between the completion of MPI_Probe() and before MPI_Recv() matches a message, another new message from a different MPI-process could arrive, i.e. it is not certain that the message found by MPI_Probe() is identical to one that MPI_Recv() matches.

To make the program work as expected, the code sequence should be corrected to:

while (...) {

MPI_Probe(MPI_ANY_SOURCE, MPI_ANY_TAG, comm, sts); if (sts->MPI_TAG == SOME_VALUE) {

MPI_Recv(buf, cnt, dtype, sts->MPI_SOURCE, sts->MPI_TAG, comm, sts); doStuff();

}

doOtherStuff();

}

3.9.2 Using MPI_Isend(), MPI_Irecv()

If communication and calculations do not overlap, using immediate calls, e.g., MPI_Isend() and MPI_Irecv(), is usually performance ineffective.

3.9.3 Using MPI_Bsend()

Using buffered send, e.g., MPI_Bsend(), usually degrade performance significantly in comparison with their unbuffered relatives.

3.9.4 Avoid starving MPI-processes - fairness

MPI programs may, if not special care is taken, be unfair and may starve MPI-processes, e.g., by using MPI_Waitany(), as illustrated for a client-server application in examples 3.15 and

3.16in the MPI 1.1 standard [1]. Fairness can also be enforced, e.g. through the use of several tags or separate communicators.

Scali MPI Connect Release 4.4 Users Guide 32