Reviewing OpenSearch Recommendations

These specifications apply to the computer where you install OpenSearch.

• OpenSearch prerequisites

  Before installing the OSK DPK, review the prerequisites on the OpenSearch web site. OpenSearch has strict bootstrap validations. If you do not fulfill the prerequisites, the OpenSearch instance may not start.

• Heap size

  To adjust memory usage after installation, you can adjust the memory settings in the jvm.options properties file. Locate the jvm.options file in the config directory under the installation directory, and modify the values for Xms and Xmx.

  Xms represents the initial size of the total heap space.

  Xmx represents the maximum size of the total heap space.

  The standard recommendations are to set Xms and Xmx to the same value, and to give 50% of the available memory to the OpenSearch heap, while leaving the other 50% free. The memory is used by Lucene for caching in-memory data structures. As a standard practice never set the heap size greater than 30 GB, as setting a higher value would not use JAVA compressed pointers, wastes memory, reduces CPU performance, and makes the garbage collection (GC) struggle with large heaps.

  For example, if the available memory is 20 GB, set both Xms and Xmx to 10 GB:

  -Xms10g
  -Xmx10g

  After you modify the jvm.options file, start and stop OpenSearch.

  See "Performing Additional Tasks," Starting and Stopping an OpenSearch Service.

• Swapping

  Disable swapping. Swapping is expensive in terms of memory required, and thus affects performance.

  You can disable swapping on Linux temporarily by running: sudo swapoff -a. To disable it permanently, you will need to edit the /etc/fstab file and comment out any lines that contain the word “swap.”

  You can disable swapping on Microsoft Window by disabling the paging file entirely. For example, select System Properties, Advanced. Click the Settings button in the Performance area. Select Advanced, Virtual memory, and change the value for the paging file. Alternatively, you can set the sysctl value vm.swappiness to 1.

  If disabling swapping completely is not an option, you can decrease the “swappiness” value. This value controls how aggressively the operating system (OS) tries to swap memory. This prevents swapping under normal circumstances, but still allows the OS to swap under emergency memory situations. A swappiness of 1 is better than 0, since on some kernel versions a swappiness of 0 can invoke the out-of-memory (OOM) killer. If neither approach is possible, you should enable bootstrap.memory_lock. This allows the JVM to lock its memory and prevent it from being swapped by the OS. The recommendation is to set this parameter to true. To enable this parameter, set this value in the opensearch.yml configuration file:

  bootstrap.memory_lock: true

  See "Performing Additional Tasks," Modifying the OpenSearch Configuration File (Optional).

• Type of disk drive

  Solid-state drives (SSDs) are by far superior to any spinning media. SSD-backed nodes see boosts in both query and indexing performance. If you choose to use SSDs, the I/O scheduling should be set to Deadline/noop for optimal utilization of SSDs and increased performance.

  If you use spinning media, try to obtain the fastest disks possible (high-performance server disks, 15k RPM drives). Using RAID 0 is an effective way to increase disk speed, for both spinning disks and SSDs. There is no need to use mirroring or parity variants of RAID, since high availability is built into OpenSearch via replicas. Avoid network-attached storage (NAS). NAS is often slower, displays larger latencies with a wider deviation in average latency, and is a single point of failure.

• File Descriptors and MMAP

  Lucene uses a very large number of files. You should increase your file descriptor count to something very large, such as 64,000. OpenSearch uses a mix of new IO File system (NioFS) and memory-mapped file system (MMapFS) for the various files. Ensure that you configure the maximum map count so that there is ample virtual memory available for mmapped files. This can be set by modifying vm.max_map_count in /etc/sysctl.conf; for example, vm.max_map_count =262144.