12.5.5 Service Job Limits

Keywords: maxjobs, filesperjob

Each time a message is enqueued to a channel, the Job Controller ensures that there is a job running to deliver the message. This might involve starting a new job process, adding a thread, or simply noting that a job is already running. A single service job may not be sufficient to ensure prompt delivery of all messages, however. (For further information on Job Controller concepts and configuration, refer to 10.4.8 Job Controller File, 12.5.4 Processing Pools for Channel Execution Jobs and 8.7 The Job Controller.

For any given installation, there is a reasonable maximum number of processes and threads to be started for delivering messages. This maximum number depends on factors such as the number of processors, the speed of the disks, and the characteristics of the connections. In the MTA configuration, it is possible to control the following:

• The maximum number of processes to start running for a given channel (the maxjobs channel keyword)

• The maximum number of processes to start for a set of channels (the JOB_LIMIT parameter in the relevant pool section of the Job Controller configuration file)

• The number of queued messages received before a new thread or process is started (the threaddepth channel keyword)

• For some channels, the maximum number of threads that will run within a given delivery program (max_client_threads parameter in the channel option file)

The maximum number of processes to start running for a given channel is the minimum of the maxjobs set on the channel and the JOB_LIMIT set for the pool that the channel runs in.

Assume a message needs processing. In general, the Job Controller starts new processes as follows:

• If no process is running for a channel and the pool job limit has not been reached, then the Job Controller starts a new process.

• If a channel program is single-threaded or the thread limit has been reached and the backlog increases past a multiple of threads (specified with threaddepth) and neither the channel nor pool job limit has been reached, the Job Controller starts a new process

• If a channel program is multithreaded and the thread limit has not been reached and the backlog of messages increase past a multiple of threaddepth, a new thread is started.

For SMTP channels in particular, new threads or processes are started as messages are enqueued for different hosts. Thus, for SMTP channels, the Job Controller starts new processes as follows. Assume a message needs processing:

• If no process is running for an SMTP channel and the pool limit has not been reached, the Job Controller starts a new process.

• If the thread limit (MAX_CLIENT_THREADS) has been reached, a message is enqueued for a host not yet being serviced, and neither the channel (maxjobs) nor pool job limit (JOB_LIMIT) has been reached then a new process is started.

• If the thread limit has not been reached and a message is enqueued for a host not yet being serviced, then a new thread is started.

• If the thread limit has not been reached and a message is enqueued that takes the backlog of messages for that host increase past a multiple of threaddepth, a new thread is started.

See also 12.5.8 SMTP Channel Threads.

The filesperjob keyword can be used to cause the MTA to create additional service jobs. This keyword takes a single positive integer parameter which specifies how many queue entries (that is, files) must be sent to the associated channel before more than one service job is created to handle them. If a value less than or equal to zero is given it is interpreted as a request to queue only one service job. Not specifying a keyword is equivalent to specifying a value of zero. The effect of this keyword is maximized; the larger number computed will be the number of service jobs that are actually created.

The filesperjob keyword divides the number of actual queue entries or files by the given value. Note that the number of queue entries resulting from a given message is controlled by a large number of factors, including but not limited to the use of the single and single_sys keywords and the specification of header-modifying actions in mailing lists.

The maxjobs keyword places an upper bound on the total number of service jobs that can be running concurrently. This keyword must be followed by an integer value; if the computed number of service jobs is greater than this value only maxjobs jobs will actually be created. The default for this value if maxjobs is not specified is 100. Normally maxjobs is set to a value that is less than or equal to the total number of jobs that can run simultaneously in whatever service pool or pools the channel uses.