The Oracle Solaris operating system supports features that improve the performance of OpenMP programs. These features include Memory Placement Optimizations (MPO) and Multiple Page Size Support (MPSS).
Shared memory multiprocessor computers contain multiple processors. Each processor can access all of the memory in the machine. In some shared memory multiprocessors, the memory architecture enables each processor to access some areas of memory more quickly than other areas. Therefore, allocating memory close to the processor that accesses it will reduce latency and improve application performance.
The Oracle Solaris operating system introduced the locality group (lgroup) abstraction, which is part of the MPO feature. An lgroup is a set of processor-like and memory-like devices in which each processor in the set can access any memory in that set within a bounded latency interval. The library liblgrp.so exports the lgroup abstraction for applications to use for observability and performance tuning. Applications can use the liblgrp.so APIs to perform the following tasks:
Traverse the group hierarchy
Discover the contents and characteristics of a given lgroup
Affect the thread and memory placement on lgroups
By default, the Oracle Solaris operating system attempts to allocate resources for a thread from the thread's home lgroup. For example, by default the operating system attempts to schedule a thread to run on processors in the thread's home lgroup and allocate the thread's memory in the thread's home lgroup.
The following mechanisms can be used to discover and affect thread and memory placement with respect to lgroups:
The meminfo() system call can be used to discover memory placement.
The lgrp_home() function can be used to discover thread placement.
The lgrp_affinity_set() function can be used to affect thread and memory placement by setting a thread's affinity for a given lgroup.
The madvise() function in the standard C library can be used to advise the operating system that a region of user virtual memory is expected to follow a particular pattern of use. The MADV_ACCESS flags passed to madvise() are used to affect memory allocation among lgroups. For example, calling madvise() with the MADV_ACCESS_LWP flag advises the operating system that the next thread to touch the specified address range is the thread that will access the memory region the most. The OS places the memory for this range and the thread accordingly.
For more information about the lgroup APIs, refer to Chapter 4, Locality Group APIs in Oracle Solaris 11.3 Programming Interfaces Guide. For more information about the madvise() function, see the madvise(3C) man page.
The Multiple Page Size Support (MPSS) feature in Oracle Solaris allows an application to use different page sizes for different regions of virtual memory. The default page size on a specific platform can be obtained with the pagesize command. The -a option on this command lists all the supported page sizes. For details, see the pagesize(1) man page.
The Translation Lookaside Buffer (TLB) is a data structure used to map virtual memory addresses to physical memory addresses. Some performance penalty is associated with accessing memory which does not have the virtual-to-physical mapping information available in the TLB. Larger page sizes let the TLB map more physical memory using the fixed number of TLB entries. Larger pages may therefore reduce the cost of virtual-to-physical memory mapping and increase overall system performance.
There are several ways to change the default page size for an application: