ENOMEM The available data space is not large enough to accommodate the shared memory segment

Cause

ENOMEM errors occur after 80 segments have been allocated by Lotus Notes.

Action

The design and implementation of Solaris ISM (Intimate Shared Memory) is what caused the ENOMEM failures, from the Lotus Notes application, besause of the limit reached on the number of shared memory segments that can be attached to a particular process.

The limit occurs because all shared memory segments are attached in the Intimate Shared Memory (ISM) mode courtesy of a system variable they have set in the system file called shmsys:share_page_table.

When a shared memory segment is attached in ISM mode, the OS locks that segment into physical memory and arranges the virtual/physical address mappings such that only one copy of the mapping information is shared amongst all attaching processes. To accomplish this, the OS requires the virtual starting address of the segment be aligned on a 16 Meg (hex 0x1000000) = 16777216-bytes address boundary.

The NULL address lets the system decide what virtual address the segment should be attached at. The system also assigns addresses 0x3000000 apart unless forced to attach addresses at 0x1000000 apart.

Doing a few calculations, a sun4d could create and attach up to 220 1-Meg ISM segments and a sun4m could create and attach up to 235 1-Meg ISM segments, providing the segments were 0x1000000 apart.

Given that ISM is what causes the limit, what can we do about it?

First, the limit only gives Lotus Notes the ability to attach a total of 80 Meg of shared memory total. By increasing the segment size to 10 Meg, as Lotus has already recommended, 8 ISM segments will take care of the load previously needing 80 1-Meg segments. The load could conceivably grow to 800-meg now without running into the ISM addressing limit.

Secondly, we could have turned off the share_page_table (ISM) flag. This would give a sun4m the ability to create in excess of 3000 1-Meg segments. The problem here is that ISM does improve the performance of shared memory accesses and if your customer intends to move up to 2.5.x, ISM is required to get around another set of problems that were discovered with shared memory loads of this kind.

Thirdly, Lotus could change the Notes server so that it kept track of the attach addresses and always attached at 0x1000000 boundary addresses instead of letting the system default to the 0x3000000 address boundary. As I've already shown, this would allow a Notes server to grow to 235 segments on a sun4m.