Memory Considerations

Memory is allocated on a node for each remote MPI process that sends messages to it over RSM. If np_local is the number of processes on a particular node, then the memory requirement on the node for RSM message passing from any one remote process is

np_local * ( MPI_RSM_NUMPOSTBOX * 128 + MPI_RSM_CPOOLSIZE )

bytes when MPI_RSM_SBPOOLSIZE is unset, and

np_local * MPI_RSM_NUMPOSTBOX * 128 + MPI_RSM_SBPOOLSIZE

bytes when MPI_RSM_SBPOOLSIZE is set.

The amount of memory actually allocated may be higher or lower than this requirement:

• The memory requirement is rounded up to some multiple of 8192 bytes with a minimum of 32768 bytes.

• This memory is allocated from a 256-Kbyte (262,144-byte) segment.

  • If the memory requirement is greater than 256 Kbytes, then insufficient memory will be allocated.

  • If the memory requirement is less than 256 Kbytes, some allocated memory will go unused. (There is some, but only limited, sharing of segments.)

If less memory is allocated than is required, then requested values of MPI_RSM_CPOOLSIZE or MPI_RSM_SBPOOLSIZE may be reduced at run time. This can cause the requested value of MPI_RSM_PIPESIZE to be overridden as well.

Each remote MPI process requires its own allocation on the node as described above.

If multiple stripes are employed, the memory requirement increases correspondingly.