Programming With Sun MPI

Although there are about 190 (non-I/O) routines in the Sun MPI library, you can write programs for a wide range of problems using only six routines:

Table 2-4 Six Basic MPI Routines

MPI_Init	Initializes the MPI library.
MPI_Finalize	Finalizes the MPI library. This includes releasing resources used by the library.
MPI_Comm_size	Determines the number of processes in a specified communicator.
MPI_Comm_rank	Determines the rank of calling process within a communicator.
MPI_Send	Sends a message.
MPI_Recv	Receives a message.

This set of six routines includes the basic send and receive routines. Programs that depend heavily on collective communication may also include MPI_Bcast and MPI_Reduce.

The functionality of these routines means you can have the benefit of parallel operations without having to learn the whole library at once. As you become more familiar with programming for message passing, you can start learning the more complex and esoteric routines and add them to your programs as needed.

See "Sample Code", for two simple Sun MPI code samples, one in C and one in Fortran. See "Sun MPI Routines", for a complete list of Sun MPI routines.

Fortran Support

Sun MPI 4.0 provides basic Fortran support, as described in section 10.2 of the MPI-2 standard. Essentially, Fortran bindings and an mpif.h file are provided, as specified in the MPI-1 standard. The mpif.h file is valid for both fixed- and free-source form, as specified in the MPI-2 standard.

The MPI interface is known to violate the Fortran standard in several ways, which cause few problems for Fortran 77 programs. These standard violations can cause more significant problems for Fortran 90 programs, however, if you do not follow the guidelines recommended in the standard. If you are programming in Fortran, and particularly if you are using Fortran 90, you should consult section 10.2 of the MPI-2 standard for detailed information about basic Fortran support in an MPI implementation.

Recommendations for All-to-All and All-to-One Communication

The Sun MPI library uses the TCP protocol to communicate over a variety of networks. MPI depends on TCP to ensure reliable, correct data flow. TCP's reliability compensates for unreliability in the underlying network, as the TCP retransmission algorithms will handle any segments that are lost or corrupted. In most cases, this works well with good performance characteristics. However, when doing all-to-all and all-to-one communication over certain networks, a large number of TCP segments may be lost, resulting in poor performance.

You can compensate for this diminished performance over TCP in these ways:

• When writing your own algorithms, avoid flooding one node with a lot of data.

• If you need to do all-to-all or all-to-one communication, use one of the Sun MPI routines to do so. They are implemented in a way that avoids congesting a single node with lots of data. The following routines fall into this category:

  • MPI_Alltoall and MPI_Alltoallv - These have been implemented using a pairwise communication pattern, so that every rank is communicating with only one other rank at a given time.

  • MPI_Gather/MPI_Gatherv - The root process sends ready-to-send packets to each nonroot-rank process to tell the processes to send their data. In this way, the root process can regulate how much data it is receiving at any one time. Using this ready-to-send method is, however associated with a minor performance cost. For this reason, you can override this method by setting the MPI_TCPSAFEGATHER environment variable to 0. (See the Sun MPI user's guides for information about environment variables.)

Signals and MPI

When running the MPI library over TCP, nonfatal SIGPIPE signals may be generated. To handle them, the library sets the signal handler for SIGPIPE to ignore, overriding the default setting (terminate the process). In this way, the MPI library can recover in certain situations. You should therefore avoid changing the SIGPIPE signal handler.

The Sun MPI 4.0 Fortran and C⁺⁺ bindings are implemented as wrappers on top of the C bindings. The profiling interface is implemented using weak symbols. This means a profiling library need contain only a profiled version of C bindings.

The SIGPIPEs may occur when a process first starts communicating over TCP. This happens because the MPI library creates connections over TCP only when processes actually communicate with one another. There are some unavoidable conditions where SIGPIPEs may be generated when two processes establish a connection. If you want to avoid any SIGPIPEs, set the environment variable MPI_FULLCONNINIT, which creates all connections during MPI_Init() and avoids any situations which may generate a SIGPIPE. For more information about environment variables, see the Sun MPI user's guides.