Go to main content

SPARC T5-8 Server Product Notes

Exit Print View

Updated: May 2016
 
 

Active mempm Should Balance Cage Assignment Across All Nodes (Bug ID15944881)


Note -  This issue was fixed in SRU11.1.6.4.0.

On large SPARC T5 series server domains, certain workloads might exhibit unexpected poor performance or negative performance scaling. In addition, on large SPARC T5 servers with a high number of network adapters, aggregate network throughput might be limited to no more than ~12 GB/second.

The Oracle Solaris kernel manages its data structures inside of a cage, which defines the limits of kernel memory. The cage is allowed to grow over time if needed. Ideally, the cage is distributed across all memory in the domain, so that the kernel does not exhaust the memory within any one locality group (lgrp). Distributing the cage across lgrps is desirable because user-level code running on processors within the lgrp performs better if local memory is available, rather than using remote memory in another lgrp.

Workaround: The tunable mempm parameter defines how the kernel manages cage growth and distribution. For most workloads, the mempm parameter can be left at its default value (0), which allows the power management code to affect cage allocation. However, for some workloads a more even distribution of kernel memory might be obtained using a non-power-aware, traditional cage allocation algorithm.

Set the mempm parameter if you observe unexpected poor performance or negative scaling at the application level on large domains with a high number of CPUs. Setting the mempm parameter to 1 disables the power management cage allocation algorithm, and the kernel reverts to its traditional cage allocation algorithm.

To set the mempm parameter:

  1. Add the following to the /etc/system file:

    set plat_disable_mempm=1
    
  2. Reboot the server and observe memory usage (for example, using the lgrpinfo tool) to see whether memory is more balanced across lgrps.


    Note -  If this issue persists, contact your Oracle service provider for further assistance.