PGX 20.1.1
Documentation

Enterprise Scheduler Configuration Guide

Not available on all platforms

the enterprise scheduler is currently only available for Linux (x86_64), macOS (x86_64), Solaris (x86_64, sparc)

This chapter describes the extra configuration options for the enterprise scheduler. These configuration options are only available if the scheduler configuration variable is set to enterprise_scheduler. See the PGX configuration guide for more information about this.

The configuration is divided into two parts, the enteprise_scheduler_config for setting details about how tasks should be scheduled and the enterprise_scheduler_flags where you can configure the enterprise scheduler in more detail.

Enterprise Scheduler Fields

Field Type Description Default
analysis_task_configobjectconfiguration for analysis tasks
priority
medium
weight
<no-of-CPUs>
max_threads
<no-of-CPUs>
fast_analysis_task_configobjectconfiguration for fast analysis tasks
priority
high
weight
1
max_threads
<no-of-CPUs>
max_num_concurrent_io_tasksintegermaximum number of concurrent io tasks at a time3
num_io_threads_per_taskintegernumber of io threads to use per task<no-of-cpus>

Analysis Task Config Fields

Field Type Description Default
max_threadsintegera hard limit on the number of threads to use for a task.required
priorityenum[high, medium, low]the priority of the task. Threads are given to the task with the highest priority at the moment of execution. If there are more threads that have the highest priority, threads are given to the tasks according to their weightrequired
weightintegerthe weight of the task. Threads are given to tasks proportionally to their weight. Tasks with higher weight will get more threads than tasks with lower weight. Tasks with the same weight will get the same amount of threads.required

Enterprise Scheduler Flags

Field Type Description Default
show_allocationsbooleanIf true show memory allocation information.false
show_environmentbooleanIf true show version numbers and main environment settings at startup.false
show_loggingbooleanIf true enable summary logging -- this is available even in non-debug builds and includes information such as the machine h/w information obtained at start-up, and per-job / per-loop* information about the workload.false
show_profilingbooleanIf true show profiling information.false
show_scheduler_statebooleanIf true dump scheduler state on each update.false
show_warningsbooleanIf true enable warnings. These are non-fatal errors -- e.g., if a NUMA-aware allocation cannot be placed on the intended socket.true

Examples

Example for Custom Enterprise Scheduler Configuration

This configuration sets the number of io threads per task to 16, increases the maximum number of concurrent io tasks to 5. It also sets the configuration for fast analysis tasks to have a weight of 1, priority of "high" and sets a limit to the maximum number of threads used to 1.

{
  "enterprise_scheduler_config": {
    "num_io_threads_per_task": 16,
    "max_num_concurrent_io_tasks": 5,
    "fast_analysis_task_config": {
      "weight": 1,
      "priority": "high",
      "max_threads": 1
    }
  }
}

Example Using the Enterprise Scheduler Flags

This configuration enables extra logging output from the enterprise scheduler.

{
  "enterprise_scheduler_flags": {
    "show_logging": true
  }
}