Shared memory multiprocessor computers contain multiple processors. Each processor can access all of the memory in the machine. In some shared memory multiprocessors, the memory architecture enables each processor to access some areas of memory more quickly than other areas. Therefore, allocating memory close to the processor that accesses it will reduce latency and improve application performance.
The Oracle Solaris operating system introduced the locality group (lgroup) abstraction, which is part of the MPO feature. An lgroup is a set of processor-like and memory-like devices in which each processor in the set can access any memory in that set within a bounded latency interval. The library liblgrp.so exports the lgroup abstraction for applications to use for observability and performance tuning. Applications can use the liblgrp.so APIs to perform the following tasks:
Traverse the group hierarchy
Discover the contents and characteristics of a given lgroup
Affect the thread and memory placement on lgroups
By default, the Oracle Solaris operating system attempts to allocate resources for a thread from the thread's home lgroup. For example, by default the operating system attempts to schedule a thread to run on processors in the thread's home lgroup and allocate the thread's memory in the thread's home lgroup.
The following mechanisms can be used to discover and affect thread and memory placement with respect to lgroups:
The meminfo() system call can be used to discover memory placement.
The lgrp_home() function can be used to discover thread placement.
The lgrp_affinity_set() function can be used to affect thread and memory placement by setting a thread's affinity for a given lgroup.
The madvise() function in the standard C library can be used to advise the operating system that a region of user virtual memory is expected to follow a particular pattern of use. The MADV_ACCESS flags passed to madvise() are used to affect memory allocation among lgroups. For example, calling madvise() with the MADV_ACCESS_LWP flag advises the operating system that the next thread to touch the specified address range is the thread that will access the memory region the most. The OS places the memory for this range and the thread accordingly.
For more information about the lgroup APIs, refer to Chapter 4, Locality Group APIs, in Oracle Solaris 11.3 Programming Interfaces Guide . For more information about the madvise() function, see the madvise(3C) man page.