The MDEX Engine manages all processor-intensive tasks related to query processing using a pre-configured threading pool.
By managing the threading pool, the MDEX Engine ensures that the system resources are used effectively for the prioritized tasks in the MDEX Engine that support high performance query processing.
You define the number of threads in the MDEX Engine threading pool at
MDEX Engine startup, using the
--threads
num flag, where
num can be 1 or greater.
You can also specify the
auto option for the existing
--threads flag, for both dgraph and dgidx. Specifying
--threads auto causes one thread to be used for each
core on the machine.
Note
Although specifying
--threads auto produces good results in some cases, best
results are usually obtained by setting the number of threads explicitly, using
--threads
num after having determined a good value for
num through testing.
The threading pool manages the following MDEX Engine tasks:
All tasks that support query processing in the MDEX Engine. The MDEX Engine allocates these tasks for threads in the threading pool. The tasks include all high-priority, CPU-intensive, frequently performed operations the MDEX Engine runs in production. For example, they include pre-computed sorting, background merging of index generations, and operations that support high performance of updates, among others.
MDEX Engine operations that do not have a significant impact on CPU use are not managed by the threading pool.
If you use operating system commands such as
top to examine the number of threads used by the MDEX
Engine server, you may see a number that is larger than the number you specify
with the
--threads flag. This is because in addition to this
number of threads, the MDEX Engine may use additional threads for other tasks.
These additional threads support tasks that are run infrequently, are less-CPU
intensive, and do not affect overall MDEX Engine performance. You cannot
control these additional threads.
Note
Be sure to set the number of MDEX Engine threads to (at most) two less than the number of hardware cores on the machine where the MDEX Engine is running. This ensures that certain housekeeping tasks run by the MDEX Engine have enough threads to execute. This text is true regardless of whether partial updates are taking place.
