Managing the Off-heap Cache

Configuring the Off-heap Cache

Each Replication Node in the store divides the memory available to it between memory containing store objects, memory used by the host operation system (which includes the file system cache), and the in-memory cache used to contain frequently accessed store data. The amount of memory used by the cache is configurable using the rnHeapPercent parameter. This is expressed as a percentage of the total memory available to the Replication Node (which is, in turn, determined by the amount of memory made available to the Java VM). By default, 70% of the memory available to the Replication Node is reserved for the in-memory cache.

However, this can and probably will fill up. When that happens, objects are evicted from the cache based on a Least-Recently-Used (LRU) algorithm which guarantees that the most commonly accessed objects will remain in the cache.

It is possible to configure your Replication Node so that the in-memory cache is very large, and therefore is less likely to overflow. However, several drawbacks occur when the cache becomes too large. The most important of these is that Java Garbage Collection performance can decrease, which will have an adverse impact on your Replication Node's overall performance. Therefore, it is in general a good idea to keep your heap size below 32 GB. Doing so limits your in-memory cache size to about 22.4 GB.

Note

Be aware that Oracle NoSQL Database limits your maximum heap size to a default value of 32 GB so as to prevent accidentally creating an excessively large cache. This value is managed using the rnHeapMaxMB, which is described in greater detail later in this section.

When your Replication Node's heap is sized lower, it is possible to configure the system such that any memory unused by the heap will be used for the operating system and file system cache. A large file system cache has performance benefits, but it also has drawbacks:

  1. There is a significant redundancy between the main cache and the file system cache because all data and Btree information that is logged (written) by the Replication Node appears in the file system and may also appear in the main cache.

  2. It is not possible for dirty Btree information to be placed in the file system cache without logging it. This logging may be otherwise unnecessary, and the logging creates additional work for internal cleaner threads.

To avoid the problems caused by excessively large in-memory and file system caches, your Replication Node is by default configured to use an off-heap cache. The off-heap cache is used to hold record data and Btree nodes when these are evicted from the "main cache" because it overflows. When the off-heap cache overflows, eviction occurs according to the same LRU algorithm used by the main cache.

Note

You should engage in performance testing before modifying the caches in use by your production store.

Configuring the Off-heap Cache

You do not directly control the amount of memory available to the off-heap cache. Instead, you make memory available to it mainly by limiting the amount of memory available to the operating system. You might also need to control the size of the Java heap, which in turn controls the size of the in-memory cache. Any Replication Node memory that is left over once the heap and operating system requirements are met is used for the off-heap cache. If no memory is left over, the off-heap cache is not used.

The parameters you use to (indirectly) configure the off-heap cache are:

  1. systemPercent

    This defines the percentage of memory available to the Storage Node that is available for operating system use after the heap requirement is subtracted. By default, this value is 10%. Configuring this value to a number that is greater than 10% but less than 100% might leave room for the off-heap cache (depending on the memory on your Storage Node and the value chosen for this parameter). If room is available for the off-heap cache, it will be turned on.

    For most production systems, if you are using the off-heap cache, then the default value of 10% for this parameter should be sufficient.

  2. rnHeapMaxMB

    This is the maximum amount of memory available for the Java heap. (The heap is where the in-memory cache is contained.) The size of your heap will be the lesser of this number, or the size arrived at using the rnHeapPercent parameter value:

     total SN memory * rnHeapPercent = Heap Size  

    rnHeapPercent is by default 85%. So if your Storage Node has 32GB memory, then the heap size will be 32 * 0.85 = 27.2 GB. However, if you set your rnHeapMaxMB to 25,600 (25*1024), then your heap size will be 25GB instead.

    Remember that the heap size is not the same as the in-memory cache size. The in-memory cache size is expressed as a percentage of the heap size. By default, this is 70% (configurable using the rnCachePercent parameter). So if your heap size is 27.2 GB, then your in-memory cache size ends up being 19.04 GB (27.2 * 0.7).

    Note

    Regardless of the values provided to the parameters described here, and the actual memory available for the heap, your heap size is limited to a maximum size of 32 GB.

For example, if your Storage Node has 64GB of memory, and just one Replication Node, then by default:

  • The heap size is 32GB. (64 * .85 = 54.4 which is greater than the 32GB maximum.)

  • The in-memory cache size is 22.4 GB (32 * 0.7).

  • A system memory size of 6.4 GB. (64 *.1)

  • An off-heap cache size of 25.6 GB. This is the amount of memory left over after the heap and system requirements have been met.

If you want to eliminate the off-heap cache, then set system memory to 100%. Do this for each Storage Node in your store. Use the following commands:

  • If the store has not been deployed:

    kv-> change-policy -params systemPercent=100
  • If the store has been deployed:

    kv-> plan change-parameters -service sn1 -wait -params systemPercent=100

Note

The Replication Node (RN) does not re-start by default. You need to re-start the RN (or RNs if the capacity is greater than 1) by using the following commands:

kv-> plan stop-service -all-rns -wait;
Executed plan 7, waiting for completion...
Plan 7 ended successfully
kv-> plan start-service -all-rns -wait;
Executed plan 8, waiting for completion...
Plan 8 ended successfully

This yields:

  • A heap size of 32 GB and in-memory cache size of 22.4 GB. Nothing has changed from the default configuration.

  • The system memory is 32 GB. System memory is 100% of what is left over when the heap is taken away. 64 GB (total available) - 32 GB (heap size) = 32 GB for the operating system and file system cache.