Locality Group Parameters

This section provides generic memory tunables, which apply to any SPARC or x86 system that uses a Non-Uniform Memory Architecture (NUMA).

lpg_alloc_prefer

Description

Controls a heuristic for allocation of large memory pages when the requested page size is not immediately available in the local memory group, but could be satisfied from a remote memory group.

By default, the Solaris OS allocates a remote large page if local free memory is fragmented, but remote free memory is not. Setting this parameter to 1 indicates that additional effort should be spent attempting to allocate larger memory pages locally, potentially moving smaller pages around to coalesce larger pages in the local memory group.

Data Type

Boolean

Default

0 (Prefer remote allocation if local free memory is fragmented and remote free memory is not)

Range

0 (Prefer remote allocation if local free memory is fragmented and remote free memory is not)

1 (Prefer local allocation whenever possible, even if local free memory is fragmented and remote free memory is not)

Dynamic?

No

Validation

None

When to Change

This parameter might be set to 1 if long-running programs on the system tend to allocate memory that is accessed by a single program, or if memory that is accessed by a group of programs is known to be running in the same locality group (lgroup). In these circumstances, the extra cost of page coalesce operations can be amortized over the long run of the programs.

This parameter might be left at the default value (0) if multiple programs tend to share memory across different locality groups, or if pages tend to be used for short periods of time. In these circumstances, quick allocation of the requested size tends to be more important than allocation in a particular location.

Page locations and sizes might be observed by using the NUMA observability tools, available at http://opensolaris.org. TLB miss activity might be observed by using the trapstat -T command.

Commitment Level

Uncommitted

lgrp_mem_default_policy

Description

This variable reflects the default memory allocation policy used by the Solaris OS. This variable is an integer, and its value should correspond to one of the policies listed in the sys/lgrp.h file.

Data Type

Integer

Default

1, LGRP_MEM_POLICY_NEXT indicating that memory allocation defaults to the home lgroup of the thread performing the memory allocation.

Range

Possible values are:

Value	Description	Comment
0	LGRP_MEM_POLICY_DEFAULT	use system default policy
1	LGRP_MEM_POLICY_NEXT	next to allocating thread's home lgroup
2	LGRP_MEM_POLICY_RANDOM_PROC	randomly across process
3	LGRP_MEM_POLICY_RANDOM_PSET	randomly across processor set
4	LGRP_MEM_POLICY_RANDOM	randomly across all lgroups
5	LGRP_MEM_POLICY_ROUNDROBIN	round robin across all lgroups
6	LGRP_MEM_POLICY_NEXT_CPU	near next CPU to touch memory

Dynamic?

No

Validation

None

When to Change

For applications that are sensitive to memory latencies due to allocations that occur from remote versus local memory on systems that use NUMA.

Commitment Level

Uncommitted

lgrp_mem_pset_aware

Description

If a process is running within a user processor set, this variable determines whether randomly placed memory for the process is selected from among all the lgroups in the system or only from those lgroups that are spanned by the processors in the processor set.

For more information about creating processor sets, see psrset(1M).

Data Type

Boolean

Default

0, the Solaris OS selects memory from all the lgroups in the system

Range

• 0, the Solaris OS selects memory from all the lgroups in the system (default)

• 1, try selecting memory only from those lgroups that are spanned by the processors in the processor set. If the first attempt fails, memory can be allocated in any lgroup.

Dynamic?

No

Validation

None

When to Change

Setting this value to a value of one (1) might lead to more reproducible performance when processor sets are used to isolate applications from one another.

Commitment Level

Uncommitted