10.2 Configuring HugePages for Oracle Database

The steps in this section are for configuring HugePages on a 64-bit Oracle Linux system running one or more Oracle Database instances.

To configure HugePages:

  1. Verify that the soft and hard values in kilobytes of memlock that are configured in /etc/security/limits.conf are slightly smaller than the amount of installed memory. For example, if the system has 64GB of RAM, the values shown here would be appropriate:

    soft memlock 60397977
    hard memlock 60397977

  2. Log in as the Oracle account owner (usually oracle) and use the following command to verify the value of memlock:

    $ ulimit -l
    60397977

  3. If your system is running Oracle Database 11g or later, disable AMM by setting the values of both of the initialization parameters memory_target and memory_max_target to 0.

    If you start the Oracle Database instances with a server parameter file, which is the default if you created the database with the Database Configuration Assistant (DBCA), enter the following commands at the SQL prompt:

    SQL> alter system set memory_target=0;
    System altered.
    SQL> alter system set memory_max_target=0;
    System altered.

    If you start the Oracle Database instances with a text initialization parameter file, manually edit the file so that it contains the following entries:

    memory_target = 0
    memory_max_target = 0

  4. Verify that all the Oracle Database instances are running (including any Automatic Storage Management (ASM) instances) as they would run on the production system.

  5. Create the file hugepages_settings.sh with the following content (taken from the My Oracle Support (MOS) note 401749.1).

    #!/bin/bash
    #
    # hugepages_settings.sh
    #
    # Linux bash script to compute values for the
    # recommended HugePages/HugeTLB configuration
    #
    # Note: This script does calculation for all shared memory
    # segments available when the script is run, no matter it
    # is an Oracle RDBMS shared memory segment or not.
    # Check for the kernel version
    KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'`
    # Find out the HugePage size
    HPG_SZ=`grep Hugepagesize /proc/meminfo | awk {'print $2'}`
    # Start from 1 pages to be on the safe side and guarantee 1 free HugePage
    NUM_PG=1
    # Cumulative number of pages required to handle the running shared memory segments
    for SEG_BYTES in `ipcs -m | awk {'print $5'} | grep "[0-9][0-9]*"`
    do
       MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q`
       if [ $MIN_PG -gt 0 ]; then
          NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q`
       fi
    done
    # Finish with results
    case $KERN in
       '2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`;
              echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;;
       '2.6') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
        *) echo "Unrecognized kernel version $KERN. Exiting." ;;
    esac
    # End  

  6. Make the file executable, and run it to calculate the recommended value for the vm.nr_hugepages kernel parameter.

    $ chmod u+x ./hugepages_setting.sh  
    $ ./hugepages_settings.sh
    .
    .
    .
    Recommended setting: vm.nr_hugepages = 22960

  7. As root, edit the file /etc/sysctl.conf and set the value of the vm.nr_hugepages parameter to the recommended value.

    vm.nr_hugepages = 22960

  8. Stop all the database instances and reboot the system.

After rebooting the system, verify that the database instances (including any ASM instances) have started, and use the following command to display the state of the huge pages.

# grep ^Huge /proc/meminfo
HugePages_Total:   22960
HugePages_Free:     2056
HugePages_Rsvd:     2016
HugePages_Surp:        0
Hugepagesize:       2048 kB

The value of HugePages_Free should be smaller than that of HugePages_Total, and the value of HugePages_Rsvd should be greater than zero. As the database instances allocate pages dynamically and proactively as required, the sum of the Hugepages_Free and HugePages_Rsvd values is likely to be smaller than the total SGA size.

If you subsequenty change the amount of system memory, add or remove any database instances, or change the size of the SGA for a database instance, use hugepages_settings.sh to recalculate the value of vm.nr_hugepages, readjust the setting in /etc/sysctl.conf, and reboot the system.