Managing Huge Pages

Huge Pages provide considerable performance benefits for Oracle Database on systems with large amounts of memory. Oracle Database Exadata Cloud at Customer provides configuration settings that make use of Huge Pages by default; however, you can make manual adjustments to optimize the configuration of Huge Pages.

Huge Pages is a feature integrated into the Linux kernel 2.6. Enabling Huge Pages makes it possible for the operating system to support large memory pages. Using Huge Pages can improve system performance by reducing the amount of system CPU and memory resources required to manage Linux page tables, which store the mapping between virtual and physical memory addresses. For Oracle Databases, using Huge Pages can drastically reduce the number of page table entries associated with the System Global Area (SGA).

On Exadata Cloud at Customer environments, a standard page is 4 KB, while a Huge Page is 2 MB by default. Therefore, an Oracle Database on Exadata Cloud at Customer with a 50 GB SGA requires 13,107,200 standard pages to house the SGA, compared with only 25,600 Huge Pages. The result is much smaller page tables, which require less memory to store and fewer CPU resources to access and manage.

The configuration for Huge Pages varies depending on when the associated Exadata Cloud at Customer instance was created. Two configurations exist:

Default Configuration of Huge Pages — After Exadata Cloud at Customer release 17.1.5

For Exadata Cloud at Customer instances created with release 17.1.5, or later, a fixed portion of system memory is reserved for Huge Pages in the operating system on each compute node, and this allocation can be used by any database deployment. The precise allocation is determined by the Application Type setting that is associated with the starter database deployment:

Transactional (OLTP) — 70% of the system memory is reserved for Huge Pages.
Decision Support or Data Warehouse — 50% of the system memory is reserved for Huge Pages.

The starter database deployment is configured with the instance parameter setting USE_LARGE_PAGES=ONLY. This setting forces the SGA to use Huge Pages.

Additional database deployments are configured with the instance parameter setting USE_LARGE_PAGES=TRUE. This setting uses available Huge Pages for the SGA and reverts to standard memory pages when the Huge Page allocation is exhausted.

Default Configuration of Huge Pages — Prior to Exadata Cloud at Customer release 17.1.5

For Exadata Cloud at Customer instances created prior to release 17.1.5, Huge Pages are configured only for the starter database deployment, which is the first database deployment that is created after the creation of the Exadata Cloud at Customer instance. The number of Huge Pages configured in the operating system is based on the size of the SGA.

The starter database deployment is configured with the instance parameter setting USE_LARGE_PAGES=ONLY. This setting forces the SGA to use Huge Pages.

Additional database deployments are not configured to use Huge Pages by default. To use Huge Pages with additional databases you must perform a manual configuration.

Adjusting the Configuration of Huge Pages

The configuration of Huge Pages for Oracle Database is a two-step process:

At the operating system level, the overall amount of memory allocated to Huge Pages is controlled by the vm.nr_hugepages system parameter entry. This setting exists separately on each compute node in the environment, and it is strongly recommended that the setting is consistent across all of the compute nodes.

To immediately adjust the Huge Page allocation, use the following command on each compute node as the root user:
```
# sysctl -w vm.nr_hugepages=value
```
In the command, value is the number of Huge Pages that you want to allocate.

On Exadata Cloud at Customer environments, each Huge Page is 2 MB by default. Therefore, to allocate 50 GB of memory to Huge Pages you can execute the following command:
```
# sysctl -w vm.nr_hugepages=25600
```
To persist the Huge Page setting across reboots, you must add an entry in the system parameter file. If your compute nodes use Oracle Linux 6, or earlier, then you can add the entry to the system parameter file at /etc/sysctl.conf. For example:
```
# echo "vm.nr_hugepages=25600" >> /etc/sysctl.conf
```
If your compute nodes use Oracle Linux 7, or later, then you can add the entry to a user-specific parameter file under /etc/sysctl.d/. For example:
```
# touch /etc/sysctl.d/01-custom.conf
# echo "vm.nr_hugepages=25600" >> /etc/sysctl.d/01-custom.conf
```
Check /etc/oracle-release to determine the current Oracle Linux release on your compute nodes.
At the Oracle Database level, the use of Huge Pages is controlled by the USE_LARGE_PAGES instance parameter setting. This setting applies to each database instance in a clustered database and it is strongly recommended that the setting is consistent across all of the database instances associated with a database deployment. The following options are available:
- TRUE — specifies that the database instance can use Huge Pages if they are available. For all versions of Oracle Database after 11.2.0.3, Oracle allocates as much of the SGA as it can using Huge Pages. When the Huge Page allocation is exhausted, standard memory pages are used.
- FALSE — specifies that the database instance does not use Huge Pages. This setting is generally not recommended if Huge Pages are available.
- ONLY — specifies that the database instance must use Huge Pages. With this setting, the database instance fails to start if the entire SGA cannot be accommodated in Huge Pages.

You must ensure that the overall configuration works if you make any adjustments at either the operating system or Oracle Database level.

For more information, see the Oracle Database Administrator's Reference for Linux and UNIX-Based Operating Systems for Release 11.2 or 12.1 for a general overview of Huge Pages and more information about configuring Huge Pages. Also, see USE_LARGE_PAGES in the Oracle Database Reference for Release 11.2, 12.1 or 12.2.