Sun Java System Messaging Server 6 2005Q4 Administration Guide

Configuring Message Processing and Delivery

You can configure when the server attempts to deliver messages based on certain criteria. You can also specify parameters for job processing, such as processing limits for service jobs, or when to spawn a new SMTP channel thread. This section describes the following:

For conceptual information on message processing and delivery, refer to Table 12–24

Configuring Message Processing and Delivery summarizes the keywords described in this section.

Table 12–24 Message Processing and Delivery Keywords


Keyword	Definition
Immediate Delivery	Defines specification for immediate delivery of messages.
`immnonurgent`	Starts delivery immediately after submission for urgent, normal, and non-urgent messages.
Channel Directionality	Specifies type of by which program a channel is served
bidirectional	Channel is served by a master and slave programs.
master	Channel is served by a master program (`master`).
slave	Channel is served by a slave program (`slave`).
Deferred Delivery	Defines specification for delivery of deferred jobs.
`backoff`	Specifies the frequency for attempted redelivery of deferred messages. Can be overridden by `normalbackoff`, `nonurgentbackoff`, `urgentbackoff`.
`deferred`	Implements recognition and honoring of the `Deferred-delivery:` header line.
`nodeferred`	Default. Specifies that `Deferred-delivery:` header line not be honored.
`nonurgentbackoff`	The frequency for attempted redelivery of nonurgent messages.
`normalbackoff`	The frequency for attempted redelivery of normal messages.
`urgentbackoff`	The frequency for attempted redelivery of urgent messages.
Message Priority Based on Size	Defines message priority based on message size.
`nonurgentblocklimit`	Forces messages above this size to lower than nonurgent priority (second class priority), meaning that the messages will always wait for the next periodic job for further processing.
`normalblocklimit`	Forces messages above this size to nonurgent priority.
`urgentblocklimit`	Forces messages above this size to normal priority.
Processing Pools for Channel Execution Jobs	Specifies the pools for processing messages of different urgencies and deferral of jobs
`pool`	Specifies the pool in which channels run.
`after`	Specifies a time delay before channels run.
Service Job Limits	Specifies the number of service jobs and the maximum number of message files to handle per job
`maxjobs`	Specifies the maximum number of jobs that can be running concurrently for the channel.
`filesperjob`	Specifies the number of queue entries to be processed by a single job.
SMTP Channel Threads
`threaddepth`	Number of messages triggering new thread with multithreaded SMTP client.
Multiple Address Expansion	Defines processing for messages with many recipients
`expandlimit`	Processes an incoming message “off-line” when the number of addressees exceeds this limit.
`expandchannel`	Specifies channel in which to perform deferred expansion due to application of expandlimit.
`holdlimit`	Holds an incoming message when the number of addresses exceeds this limit.
Transaction Limits	Specifies connection transaction limits
transactionlimit	Limits the number of messages allowed per connection.
Undeliverable Message Notifications	Specifies when undeliverable message notifications are sent.
`notices`	Specifies the amount of time that may elapse before notices are sent and messages returned.
`nonurgentnotices`	Specifies the amount of time that may elapse before notices are sent and messages returned for messages of non-urgent priority.
`normalnotices`	Specifies the amount of time that may elapse before notices are sent and messages returned for messages of normal priority.
`urgentnotices`	Specify the amount of time which may elapse before notices are sent and messages returned for messages of urgent priority.

Setting Channel Directionality

Keywords: master, slave, bidirectional

Three keywords are used to specify whether a channel is served by a master program (master), a slave program (slave), or both (bidirectional). The default, if none of these keywords are specified, is bidirectional. These keywords determine whether the MTA initiates delivery activity when a message is queued to the channel.

The use of these keywords reflects certain fundamental characteristics of the corresponding channel program or programs. The descriptions of the various channels the MTA supports indicate when and where these keywords should be used.

Implementing Deferred Delivery Dates

Keywords: deferred, nodeferred, immnonurgent

The deferred channel keyword implements recognition and honoring of the Deferred-delivery: header line. Messages with a deferred delivery date in the future are held in the channel queue until they either expire and are returned or the deferred delivery date is reached. See RFC 1327 for details on the format and operation of the Deferred-delivery: header line.

The keyword nodeferred is the default. It is important to realize that while support for deferred message processing is mandated by RFC 1327, actual implementation of it effectively lets people use the mail system as an extension of their disk quota.

The keyword immnonurgent starts delivery immediately after submission for urgent, normal, and non-urgent messages.

Specifying the Retry Frequency for Messages that Failed Delivery

Keywords: backoff, nonurgentbackoff, normalbackoff, urgentbackoff, notices

By default, the frequency of delivery retries for messages that have had delivery failures depends on the message’s priority. The default intervals between delivery attempts (in minutes) is shown below. The first number after the priority indicates the number of minutes after the initial delivery failure that the first delivery retry is attempted:

urgent: 30, 60, 60, 120, 120, 120, 240
normal: 60, 120, 120, 240, 240, 240, 480
nonurgent: 120, 240, 240, 480, 480, 480, 960

For urgent messages, a retry is attempted 30 minutes after the initial delivery failure, 60 minutes after the first delivery retry, 60 minutes after the second retry, 120 minutes after the third and so on. Retries after the last specified attempt repeat at the same interval. Thus, for urgent messages, retries occur every 240 minutes.

Delivery attempts continue for a period of time specified by the notices, nonurgentnotices, normalnotices, or urgentnotices keywords. If a successful delivery cannot be made, a delivery failure notification is generated and the message is returned to sender. (For details on the notices keyword, see To Set Notification Message Delivery Intervals

The backoff keywords allow you to specify customized sets of delivery retry intervals for messages of varying priorities. nonurgentbackoff specifies the intervals for nonurgent messages. normalbackoff specifies the intervals for normal messages. urgentbackoff specifies the intervals for urgent messages. If none of these keywords is specified, backoff specifies the intervals for all messages regardless of priority.

An example, is shown below:

urgentbackoff "pt30m" "pt1h" "pt2h" "pt3h" "pt4h" "pt5h" "pt8h" "pt16h"

Here, delivery retries of urgent messages is attempted 30 minutes after the initial delivery failure, one hour after the first delivery attempt (1 hour 30 minutes after initial failure), two hours after the second delivery attempt, three hours after the third, four hours after the fourth, five hours after the fifth, eight hours after the sixth, 16 hours after the seventh delivery attempt. Subsequent attempts are made every 16 hours until the period of time specified by the notices keyword. If a successful delivery cannot be made, a delivery failure notification is generated and the message is returned to sender. Note that the interval syntax is in ISO 8601P and is described in the Sun Java System Messaging Server Administration Reference.

In this next example,

normalbackoff "pt30m" "pt1h" "pt8h" "p1d" "p2d” "p1w"

a delivery retry of normal messages is attempted 30 minutes after the initial delivery failure, one hour after the first delivery attempt, eight hours after the second attempt, one day after the third, two days after the fourth, one week after the fifth and repeating each week until the period of time specified by the notices keyword. If a successful delivery cannot be made, a delivery failure notification is generated and the message is returned to sender.

In this final example,

backoff "pt30m" "pt120m" "pt16h" "pt36h" "p3d"

all failed message deliveries, regardless of message priority—unless overridden by nonurgentbackoff, normalbackoff, or urgentbackoff—will be retried 30 minutes after the initial delivery failure, two hours after the first retry attempt, 16 hours after the second attempt, 36 hours after the third, three days after the fourth and repeating every three days until the period of time specified by the notices keyword. If a successful delivery cannot be made, a delivery failure notification is generated and the message is returned to sender.

Processing Pools for Channel Execution Jobs

Keywords: pool

You can configure various channels to share resources by running within the same pool. You might want to configure other channels to run in pools dedicated to a particular channel. Within each pool, messages are automatically sorted into different processing queues according to message priority. Higher priority messages in the pool are processed before lower-priority messages. (See Message Priority Based on Size

The pools where the jobs are created can be selected on a channel by channel basis by using the pool keyword. The pool keyword must be followed by the name of the pool to which delivery jobs for the current channel should be pooled. The name of the pool should not contain more than twelve characters.

For further information on Job Controller concepts and configuration, refer to Job Controller File, Job Controller File, and Service Job Limits.

Service Job Limits

Keywords: maxjobs, filesperjob

Each time a message is enqueued to a channel, the Job Controller ensures that there is a job running to deliver the message. This might involve starting a new job process, adding a thread, or simply noting that a job is already running. A single service job may not be sufficient to ensure prompt delivery of all messages, however. (For further information on Job Controller concepts and configuration, refer to Job Controller File, Processing Pools for Channel Execution Jobs and The Job Controller.

For any given installation, there is a reasonable maximum number of processes and threads to be started for delivering messages. This maximum number depends on factors such as the number of processors, the speed of the disks, and the characteristics of the connections. In the MTA configuration, it is possible to control the following:

The maximum number of processes to start running for a given channel (the maxjobs channel keyword)
The maximum number of processes to start for a set of channels (the JOB_LIMIT parameter in the relevant pool section of the Job Controller configuration file)
The number of queued messages received before a new thread or process is started (the threaddepth channel keyword)
For some channels, the maximum number of threads that will run within a given delivery program (max_client_threads parameter in the channel option file)

The maximum number of processes to start running for a given channel is the minimum of the maxjobs set on the channel and the JOB_LIMIT set for the pool that the channel runs in.

Assume a message needs processing. In general, the Job Controller starts new processes as follows:

If no process is running for a channel and the pool job limit has not been reached, then the Job Controller starts a new process.
If a channel program is single-threaded or the thread limit has been reached and the backlog increases past a multiple of threads (specified with threaddepth) and neither the channel nor pool job limit has been reached, the Job Controller starts a new process
If a channel program is multithreaded and the thread limit has not been reached and the backlog of messages increase past a multiple of threaddepth, a new thread is started.

For SMTP channels in particular, new threads or processes are started as messages are enqueued for different hosts. Thus, for SMTP channels, the Job Controller starts new processes as follows. Assume a message needs processing:

If no process is running for an SMTP channel and the pool limit has not been reached, the Job Controller starts a new process.
If the thread limit (MAX_CLIENT_THREADS) has been reached, a message is enqueued for a host not yet being serviced, and neither the channel (maxjobs) nor pool job limit (JOB_LIMIT) has been reached then a new process is started.
If the thread limit has not been reached and a message is enqueued for a host not yet being serviced, then a new thread is started.
If the thread limit has not been reached and a message is enqueued that takes the backlog of messages for that host increase past a multiple of threaddepth, a new thread is started.

Setting Connection Transaction Limits

Keywords: transactionlimit

transactionlimit limits the number of messages allowed per connection. This can be used to thwart attackers in the following way:

An attacker can connect via SMTP and send many RCPT TO commands in an attempt to guess legitimate email addresses. Such an attack can be thwarted by limiting the number of invalid RCPT TOs allowed in a transaction. The attacker may respond by using multiple transactions, but with transactionlimit you can limit the number of transaction allowed in an SMTP session. The attacker can use multiple sessions, but now his cost is getting prohibitive. Connection throttling can be used to limit the number of sessions in various ways making the cost really prohibitive in most cases.

This is not without cost our side, however. Some SMTP clients react badly to recipient limits, transaction limits, or both. Exceptions need to be made for these clients. But TCP channel options apply to the SMTP server unconditionally. The solution is to use channel keywords and switchchannel to route problematic agents to channels with larger limits.

Message Priority Based on Size

Keywords: urgentblocklimit, normalblocklimit, nonurgentblocklimit

The urgentblocklimit, normalblocklimit, and nonurgentblocklimit keywords may be used to instruct the MTA to downgrade the priority of messages based on size. These keywords affect the priority that the Job Controller applies when processing the message.

SMTP Channel Threads

Keywords: threaddepth,

The multithreaded SMTP client sorts outgoing messages to different destinations to different threads. The threaddepth keyword may be used to instruct the multithreaded SMTP client to handle only the specified number of messages in any one thread, using additional threads even for messages all to the same destination (hence normally all handled in one thread). The default for this keyword is 10.

Each time the backlog for a channel increases past a multiple of threaddepth, the Job Controller tries to increase the amount of processing dedicated to processing messages queued for that channel. For multithreaded channels, the Job Controller advises any job processing messages for that channel to start a new thread, or if all jobs have the maximum threads allowed for the channel (MAX_CLIENT_THREADS in the option for the tcp_* channels) it will start a new process. For single-threaded channels it will start new process. Note that the Job Controller will not start a new job if the job limit for the channel (maxjobs) or the pool (JOB_LIMIT) has been reached.

Essentially, threaddepth controls how aggressive jobs are scheduled. Let's consider two different situations:

(1) a normal (outbound) SMTP channel

(2) a forward-to-a-smart-host SMTP channel

The job controller sorts messages destined for a particular channel by the destination host, and it schedules the jobs to process messages based on the backlog on these destinations hosts.

In the first instance, there will be a large number of destination hosts and the backlog for most of them will be small. There will be lots of threads running and everything should work well, except, perhaps, for destination hosts like aol, yahoo, hotmail, and so on, where there can be a large amount of traffic. With a thread depth of 128, you'd only get a second thread delivering to yahoo once the backlog reached 128. This is not desirable.

In the second instance, there is only the one destination host and having many threads delivering to that host are desirable. If anything, the default value of 10 could be too small.

Use of threaddepth may be of particular interest for achieving multithreading on a daemon router TCP/IP channel—a TCP/IP channel that connects to a single specific SMTP server—when the SMTP server to which the channel connects can handle multiple simultaneous connections.

Expansion of Multiple Addresses

Keywords: expandlimit, expandchannel, holdlimit

Most channels support the specification of multiple recipient addresses in the transfer of each inbound message. The specification of many recipient addresses in a single message may result in delays in message transfer processing (online delays). If the delays are long enough, network time-outs can occur, which in turn can lead to repeated message submission attempts and other problems.

The MTA provides a special facility to force deferred (offline) processing if more than a given number of addresses are specified for a single message. Deferral of message processing can decrease on-line delays enormously. Note, however, that the processing overhead is deferred, not avoided completely.

This special facility is activated by using a combination of, for instance, the generic reprocessing channel and the expandlimit keyword. The expandlimit keyword takes an integer argument that specifies how many addresses should be accepted in messages coming from the channel before deferring processing. The default value is infinite if the expandlimit keyword is not specified. A value of 0 will force deferred processing on all incoming addresses from the channel.

The expandlimit keyword must not be specified on the local channel or the reprocessing channel itself; the results of such a specification are unpredictable.

The channel actually used to perform the deferred processing may be specified using the expandchannel keyword; the reprocessing channel is used by default, if expandchannel is not specified, but use of some other reprocessing or processing channel may be useful for special purposes. If a channel for deferred processing is specified via expandchannel, that channel should be a reprocessing or processing channel; specification of other sorts of channels may lead to unpredictable results.

The reprocessing channel, or whatever channel is used to perform the deferred processing, must be added to the MTA configuration file in order for the expandlimit keyword to have any effect. If your configuration was built by the MTA configuration utility, then you should already have a reprocessing channel.

Extraordinarily large lists of recipient addresses are often a characteristic of unsolicited bulk email. The holdlimit keyword tells the MTA that messages coming in the channel that result in more than the specified number of recipients should be marked as .HELD messages and enqueued to the reprocess channel (or to whatever channel is specified via the expandchannel keyword). The files will sit unprocessed in the reprocess queue awaiting manual intervention by the MTA postmaster.

Enable Service Conversions

Keywords: service, noservice

The service keyword unconditionally enables service conversions regardless of CHARSET-CONVERSION entry. If the noservice keyword is set, service conversions for messages coming into this channel must be enabled via CHARSET-CONVERSION.