Action

The design and implementation of the Solaris ISM (intimate shared memory)--which limits the number of shared memory segments that can be attached to a particular process--caused the ENOMEM failures to the Lotus Notes application.

There is a limit because all shared memory segments are attached in the intimate shared memory (ISM) mode by a system variable that is set in the shmsys:share_page_table system file.

When a shared memory segment is attached in ISM mode, the OS locks that segment into physical memory and arranges the virtual/physical address mappings such that only one copy of the mapping information is shared among all attaching processes. To accomplish this, the OS requires that the virtual starting address of the segment be aligned on a 16 Mbytes (hex 0x1000000) = 16777216-bytes address boundary.

The NULL address lets the system decide what virtual address the segment should be attached to. The system also assigns addresses at 0x3000000 apart, unless forced to attach addresses at 0x1000000 apart.

A sun4d could create and attach up to 220 1-Mbyte ISM segments, and a sun4m could create and attach up to 235 1-Mbyte ISM segments, providing the segments were 0x1000000 apart.

Having established that ISM is the cause of the limit, below are some options:

First, the limit only gives Lotus Notes the ability to attach a total of 80 Mbytes of shared memory. By increasing the segment size to 10 Mbyte, as Lotus has already recommended, 8 ISM segments can handle the load previously needing 80 1-Mbyte segments. The load could conceivably grow to 800 Mbytes now without running into the ISM addressing limit.

Second, the share_page_table (ISM) flag could be turned off. This would give a sun4m the ability to create in excess of 3000 1-Mbyte segments. The problem here is that ISM does improve the performance of shared memory accesses, and, if the user intends to move up to 2.5.1, ISM is required to get around another set of problems related to shared memory loads of this kind.

Third, Lotus could change the Notes server so that it kept track of the attach addresses and always attached at 0x1000000 boundary addresses, instead of having the system default to the 0x3000000 address boundary. This would allow a Notes server to grow to 235 segments on a sun4m.