14.11.1 Enterprise Scheduler Configuration Guide

This chapter describes the extra configuration options for the enterprise scheduler.

Note:

These configuration options are only available if the scheduler configuration variable is set to enterprise_scheduler in Configuration Parameters for the Graph Server (PGX) Engine.

The configuration is divided into the following two parts:

1. enteprise_scheduler_config: for setting details about how tasks should be scheduled
2. enterprise_scheduler_flags: where you can configure the enterprise scheduler in more detail

Enterprise Scheduler Fields

Field	Type	Description	Default
analysis_task_config	object	Configuration for analysis tasks.	weight <no-of-CPUs> priority medium max_threads <no-of-CPUs>
fast_analysis_task_config	object	Configuration for fast analysis tasks.	weight 1 priority high max_threads <no-of-CPUs>
max_num_concurrent_io_tasks	integer	Maximum number of concurrent io tasks at a time.	3
num_io_threads_per_task	integer	Number of io threads to use per task.	<no-of-cpus>

Analysis Task Config Fields

Field	Type	Description	Default
max_threads	integer	A hard limit on the number of threads to use for a task.	required
priority	enum[high, medium, low]	The priority of the task. Threads are given to the task with the highest priority at the moment of execution. If there are more threads that have the highest priority, threads are given to the tasks according to their weight	required
weight	integer	The weight of the task. Threads are given to tasks proportionally to their weight. Tasks with higher weight will get more threads than tasks with lower weight. Tasks with the same weight will get the same amount of threads.	required

Enterprise Scheduler Flags

Field	Type	Description	Default
show_allocations	boolean	If true show memory allocation information.	false
show_environment	boolean	If true show version numbers and main environment settings at startup.	false
show_logging	boolean	If true enable summary logging. This is available even in non-debug builds and includes information such as the machine hardware information obtained at start-up, and per-job / per-loop information about the workload.	false
show_profiling	boolean	If true show profiling information.	false
show_scheduler_state	boolean	If true dump scheduler state on each update.	false
show_warnings	boolean	If true enable warnings. These are non-fatal errors. For example, if a NUMA-aware allocation cannot be placed on the intended socket.	true

Example 14-4 Custom Enterprise Scheduler Configuration

This configuration sets the number of io threads per task to 16, increases the maximum number of concurrent io tasks to 5. It also sets the configuration for fast analysis tasks to have a weight of 1, priority of "high" and sets a limit to the maximum number of threads used to 1.

{
  "enterprise_scheduler_config": {
    "num_io_threads_per_task": 16,
    "max_num_concurrent_io_tasks": 5,
    "fast_analysis_task_config": {
      "weight": 1,
      "priority": "high",
      "max_threads": 1
    }
  }
}

Example 14-5 Using the Enterprise Scheduler Flags

This configuration enables extra logging output from the enterprise scheduler.

{
  "enterprise_scheduler_flags": {
    "show_logging": true
  }
}