The MDEX Engine consistently manages all processor-intensive tasks related to query processing using its preconfigured threading pool.
The
--threads flag reflects the total number of threads in
the MDEX Engine threading pool.
You define the number of threads in the threading pool at MDEX Engine
startup, based on the setting for the
--threads flag.
Recall that the recommended number of threads for the MDEX Engine is typically equal to the number of cores on the MDEX Engine server. By managing the threading pool, the MDEX Engine enables you more accurately to limit the available computation resources to each core. This ensures that the system resources are used effectively for the highly prioritized tasks in the MDEX Engine all of which support query processing and high performance.
The threading pool manages the following MDEX Engine tasks:
All tasks that support query processing in the MDEX Engine. The MDEX Engine allocates these tasks for threads in the threading pool. The tasks include all high-priority, CPU-intensive, frequently performed operations the MDEX Engine runs in production. For example, they include precomputed sorting, background merging of index generations, and operations that support high performance of updates, among others.
Other MDEX Engine operations that do not have a significant impact on CPU usage are not managed by the threading pool.
If you use operating system commands such as
top to examine the number of threads used by the MDEX
Engine server, you may see a number that is larger than the number you specify
with the
--threads flag. This is because in addition to this
number of threads, the MDEX Engine may use additional threads for other tasks.
These additional threads support tasks that are run infrequently, are less-CPU
intensive, and do not affect overall MDEX Engine performance. You cannot
control these additional threads.
Note
Be sure to set the number of MDEX Engine threads to (at most) two less than the number of hardware cores on the machine where the MDEX Engine is running. This will ensure that certain housekeeping tasks run by the MDEX Engine will have enough threads to execute. This text is true regardless of whether partial updates are taking place.
