The Collector can collect data on calls to the Message Passing Interface (MPI) library. Currently, MPI tracing is only available for system running the Solaris OS. The functions for which data is collected are listed below.
MPI_Allgather |
MPI_Allgatherv |
MPI_Allreduce |
MPI_Alltoall |
MPI_Alltoallv |
MPI_Barrier |
MPI_Bcast |
MPI_Bsend |
MPI_Gather |
MPI_Gatherv |
MPI_Irecv |
MPI_Isend |
MPI_Recv |
MPI_Reduce |
MPI_Reduce_scatter |
MPI_Rsend |
MPI_Scan |
MPI_Scatter |
MPI_Scatterv |
MPI_Send |
MPI_Sendrecv |
MPI_Sendrecv_replace |
MPI_Ssend |
MPI_Wait |
MPI_Waitall |
MPI_Waitany |
MPI_Waitsome |
MPI_Win_fence |
MPI_Win_lock |
MPI tracing data is converted into the following metrics.
Table 2–4 MPI Tracing Metrics
Metric |
Definition |
---|---|
MPI Receives |
Number of receive operations in MPI functions that receive data |
MPI Bytes Received |
Number of bytes received in MPI functions |
MPI Sends |
Number of send operations in MPI functions that send data |
MPI Bytes Sent |
Number of bytes sent in MPI functions |
MPI Time |
Time spent in all calls to MPI functions |
Other MPI Calls |
Number of calls to other MPI functions |
The number of bytes recorded as received or sent is the buffer size given in the call. This number might be larger than the actual number of bytes received or sent. In the global communication functions and collective communication functions, the number of bytes sent or received is the maximum number, assuming direct interprocessor communication and no optimization of the data transfer or retransmission of the data.
The functions from the MPI library that are traced are listed in Table 2–5, categorized as MPI send functions, MPI receive functions, MPI send and receive functions, and other MPI functions.
Table 2–5 Classification of MPI Functions Into Send, Receive, Send and Receive, and Other
Category |
Functions |
---|---|
MPI send functions |
MPI_Bsend, MPI_Isend, MPI_Rsend , MPI_Send, MPI_Ssend |
MPI receive functions |
MPI_Irecv, MPI_Recv |
MPI send and receive functions |
MPI_Allgather, MPI_Allgatherv, MPI_Allreduce, MPI_Alltoall, MPI_Alltoallv, MPI_Bcast, MPI_Gather, MPI_Gatherv, MPI_Reduce, MPI_Reduce_scatter, MPI_Scan, MPI_Scatter, MPI_Scatterv, MPI_Sendrecv, MPI_Sendrecv_replace |
Other MPI functions |
MPI_Barrier, MPI_Wait, MPI_Waitall , MPI_Waitany, MPI_Waitsome, MPI_Win_fence, MPI_Win_lock |
Collecting MPI tracing data can help you identify places where you have a performance problem in an MPI program that could be due to MPI calls. Examples of possible performance problems are load balancing, synchronization delays, and communications bottlenecks.