Chapter 1 MTA SDK Concepts and Overview

The Sun Java System Messaging Server MTA SDK is a low-level interface, with routines falling into three categories: those that enqueue messages, those that dequeue messages, and miscellaneous routines that typically query or set MTA states, or parse message structures, such as lists of RFC 822 addresses.

The Callable Send facility, described in Chapter 5, Decoding Messages and Chapter 6, MTA SDK Reference and used only for originating mail from the local host, can be used simultaneously with the MTA SDK.

This chapter contains the following topics:

• Channel Programs and Message Queuing

• Managing Multiple Threads Using Contexts

• Enqueuing Messages

• Dequeuing Messages

• String-valued Call Arguments

• Item Codes and Item Lists

Channel Programs and Message Queuing

Message enqueuing and dequeuing are generally done by channel programs also referred to simply as channels. There are two types of channel programs, master channel that dequeue messages, and channels (sometimes referred to as slave channels) that enqueue messages. Each MTA channel has its own message queue, referred to as a channel queue. Channel programs may also perform intermediate roles by dequeuing messages from one message queue and re-enqueuing them to another while, typically, processing the message at the same time. For example, the message processing might be to convert the message body from one format to another.

Managing Multiple Threads Using Contexts

A number of SDK operations require multiple, sequential calls to the SDK routines. To manage this, the SDK provides the caller with a pointer to an opaque data structure called a context. This mechanism allows for management of state information across calls to the SDK. Use of the contexts allows multiple threads within a single program to make simultaneous calls to the same SDK routine. The only limitation is that a single, specific context may not be simultaneously used by different threads in calls to the SDK. When such usage is detected in an SDK call, an MTA_THREAD error results.

In some cases these contexts are automatically created for you, such as dequeue and decode contexts. In all other cases, for example for enqueue contexts, you must make an explicit call to create them. The calls that automatically create contexts also automatically dispose of them. In all other cases, a call must be made to explicitly dispose of a context. It is important to dispose of contexts when you no longer need them as so doing releases resources such as virtual memory.

For more information on contexts, see Threads and Enqueue Contexts and Threads and Dequeue Contexts.

Enqueuing Messages

Messages are introduced to the MTA by enqueuing them. Each enqueued message contains two required components, the message envelope and the message header, and may optionally contain a third component, the message body. The contents of the envelope and header must be provided by the program using the SDK.

For instructions on how to enqueue messages, see Chapter 2, MTA SDK Programming Considerations.

For an example of how to enqueue a message, see A Simple Example of Enqueuing a Message.

Message Components

This section describes the three message components: envelope, header and body.

Envelope

The message envelope contains the envelope From: address, and the list of envelope To: addresses. The envelope is created by the SDK as the message is enqueued. The addresses to be placed in the envelope must conform to RFC 2822. The envelope To: addresses are often referred to as envelope recipient addresses.

Programs should rely solely upon the MTA SDK routines to read and write envelope information, since the queued message file formats are subject to change. Using the SDK routines insulates programmers from format changes.

The routines mtaEnqueueStart( and mtaEnqueueTo( are used to construct a message envelope.

Header

The message header contains RFC 2822 style header lines. The program enqueuing the message has nearly complete control over the contents of the header and can specify as many or as few header lines as it sees fit, with a few exceptions. A header must have at a minimum three lines: From:, Date:, and at least one recipient header line, such as To:, Cc:, or Bcc:.

As the message is enqueued, the SDK will do its best to supply any mandatory header lines that are missing as well as take some measures to ensure that the contents of the header lines conform to any relevant standards. If the From: header line is omitted by the program using the SDK, the SDK code will construct a default header line from the envelope From: address. This may not always be appropriate; for instance, when mail is addressed to a mailing list that specifies an Errors-to: address, then the Errors-to: address should be used as the envelope From: address. In this case, it is not appropriate to derive the header From: line from the envelope From: address. If the Date: header line is omitted, the SDK code will supply it, as well as a Date-warning: header line. Finally, if no recipient header lines are present, then the SDK code will generate them using the envelope recipient addresses.

Any addresses appearing in the message header should conform to RFC 2822.

The header is written line-by-line using the routines mtaEnqueueWrite() and mtaEnqueueWriteLine().

Body

The optional message body contains the content of the message. As with the message header, the program enqueuing the message has nearly complete control over the contents of the message body. The only exception to this is when the message is structured with multiple parts or requires encoding, for example if it contains binary data, or lines requiring wrapping. In such cases, the SDK will ensure that the message body conforms to MIME standards (RFCs 2045– 2049).

As with the message header, message body lines are written with the routines mtaEnqueueWrite() and mtaEnqueueWriteLine().

Example 1–1 Sample Enqueued Message

Enqueued messages may be seen in the MTA queue directories and are merely ASCII text files. In the following sample message, lines 1 and 2 are the message envelope, the next four lines are the header, and the rest of the lines are the body.

jdoe@siroe.com
msmith@siroe.com

Date: Tues, 1 Apr 2003 15:01 PST
From: John Doe
To: Mike Smith
Subject: Lunch today

Mike,
Just confirming our lunch appointment today I’ll meet you at the
restaurant at noon.
John

            

As stated earlier, do not directly read from or write messages to the MTA message queues. Always use the SDK routines or Callable Send. The file structure of messages in the MTA queues are subject to change. In addition, site specific constraints may be placed on things such as encodings, and character set usage. The SDK routines automatically handle these and other issues.

Threads and Enqueue Contexts

Each individual message being enqueued to the MTA is represented within the SDK by an opaque enqueue context of type mta_nq_t. This enqueue context is created by mtaEnqueueStart() and destroyed by mtaEnqueueFinish(). Throughout the enqueue process, the message being enqueued is referenced through its enqueue context. A program using the SDK may simultaneously enqueue multiple messages, each message represented by its own enqueue context. Indeed, multiple threads may simultaneously enqueue one or more messages per thread. The only requirement is that a specific enqueue context not be simultaneously used by two or more threads. In the event that the SDK detects simultaneous usages, it returns the MTA_THREAD error.

Enqueuing Dequeued Mail

If a message being enqueued is the result of dequeuing a message, then all envelope fields can automatically be carried over from the old message to the new message. Both per-message fields (such as envelope IDs) and per-recipient fields (such as delivery receipt requests) can be preserved. This preservation is achieved by supplying the associated dequeue context to the routines mtaEnqueueStart(), or mtaEnqueueTo(), or both. Supplying the dequeue context to mtaEnqueueStart() preserves per-message envelope fields, while supplying the dequeue context to mtaEnqueueTo() preserves the per-recipient fields for the specified envelope recipient.

For information on message dequeuing and message dequeue contexts, see Dequeuing Messages.

Dequeuing Messages

Messages stored in the MTA message queues are removed from their queues by dequeuing them. This is typically done by channel programs (see Channel Programs and Message Queuing). When a message is dequeued, it is literally removed from the MTA message queues and, as far as the MTA is concerned, no longer exists. That is, dequeuing a message relieves the MTA of all further responsibility for the message. The responsibility is assumed to have been passed on to some other entity such as another MTA or a message store.

The channel name used by the program identifies the MTA message queue being serviced by the program. The channel name can either be explicitly specified by the program or determined from the run time environment using the PMDF_CHANNEL environment variable.

Channel naming conventions: the name must be 32 bytes or less, should be in lower case, and if the channel will have multiple instantiations, then it should be given a generic name, such as tcp, and then each instantiation can be given a specific version of it, such as tcp_local, tcp_auth, tcp_intranet.

Multiple programs may simultaneously process the same message queue. The SDK and Job Controller will automatically coordinate such efforts, using file locks to prevent two or more programs or threads from simultaneously processing the same message. When the message processing program (see Dequeue Message Processing Routine Tasks) is called, the message to be process is locked so that no other jobs may access that message. The message is then unlocked when mtaDequeueMessageFinish() is called, or when the program exits, normally or abnormally.

Threads and Dequeue Contexts

Each individual message being dequeued from the MTA is represented within the SDK by an opaque dequeue context of type mta_dq_t. Each dequeue context is created by mtaDequeueStart() and passed to a caller-supplied processing procedure. Each dequeue context is then destroyed when mtaDequeueMessageFinish() is called. Throughout the dequeue process, the message being dequeued is referenced through its dequeue context. Under typical usage, a program will have multiple threads operating, each simultaneously dequeuing a message. However, it is not permitted for two threads to simultaneously use the same dequeue context in calls to the SDK. In the event the SDK detects simultaneous usages, it returns the MTA_THREAD error.

Message Processing Threads

When mtaDequeueStart() is called, a communication path with the MTA Job Controller is established. The Job Controller is then asked if there are messages to be processed for the channel. Typically there will be messages to process since the Job Controller normally only starts channel programs when there are queued messages in need of processing. Based upon information obtained from the Job Controller, mtaDequeueStart() will then begin to create non-joinable processing threads. Each processing thread immediately begins processing the queued messages.

For further information about the exact steps a message processing thread goes through, see Debugging Programs and Logging Diagnostics.

String-valued Call Arguments

Strings passed as call arguments to the MTA SDK routines also have an associated length argument. Use of the length argument is optional; that is, if you do not know the length or do not wish to supply it, then supply a value of zero for the length argument. However, in that case the supplied string must be NULL terminated so that the SDK routine can determine the string’s length. When a non-zero length is supplied, then the string does not need to be NULL terminated. Wherever possible, the SDK routines return pointers to output strings rather than returning the strings themselves. These pointers are always thread safe; however, when associated with an SDK context they often are only valid as long as the context itself is valid. Such limits will be noted in the description of the individual routines in Chapter 4, Dequeuing Messages. In some cases, an output string buffer must be supplied, as with the mtaDateTime() and mtaUniqueString() routines.

Internally, the MTA has several basic string sizes. Users of the SDK generally do not need to concern themselves with this fact. However, at times it may be helpful to be aware of them as they can provide an upper bound on the length of various strings you might encounter. As shown in the following table, for instance, channel names will never be longer than CHANLENGTH bytes; channel option values will never exceed a length of BIGALFA_SIZE bytes; and envelope addresses will never exceed a length of ALFA_SIZE bytes:

Item Codes and Item Lists

A number of the MTA SDK routines accept a variable length list of item code arguments. For instance, mtaInit() has the call syntax:

int mtaInit(int item_code, ...)

That is to say, it accepts one or more integer-valued call arguments. These call arguments are referred to as an “item code list” or, more simply, an “item list.” Each item list must be terminated by a call argument with the value 0. As such, the call syntax for mtaInit() can be expressed as

int mtaInit([int item_code[, ...]], 0)

There can be zero or more item codes with non-zero values which must then be followed by an item code with the value zero.

In the MTA SDK, item lists serve two purposes. First, they allow code using the SDK to specify optional behaviors and actions to the SDK. Second, they provide an extension mechanism for future versions of the SDK to extend the functionality of routines through the introduction of new item codes.

However, there is a drawback to the use of item lists; the number of items passed to an SDK routine must be known at compile time. That is, it is difficult if not impossible for a program at run time to adjust the number of item codes that it wishes to pass. In recognition of this limitation, all SDK routines that accept an item code list also accept a pointer to an arbitrary length array of item codes. Such an array is referred to as an “item list array” and is specified with the MTA_ITEM_LIST item code. This mechanism allows programs to dynamically construct the array at run time, while still using a fixed number of arguments at compile time.

The MTA_ITEM_LIST item code is always followed by an additional call argument whose value is a pointer to an array of mta_item_list_t type elements. Each array entry has the following five fields:

The end of the array is signified by an array entry whose item_code field has the value zero (MTA_END_LIST). As an example of using MTA_ITEM_LIST, consider the following mtaInit() call:

istat = mtaInit(MTA_DEBUG_SDK, MTA_DEBUG_OS, MTA_DEBUG_MM, 4,
                MTA_DEBUG_DEQUEUE, MTA_DEBUG_DECODE, 0);

      

In the above call, the decision to enable the listed debug modes is made at compile time. Using an item list array allows the decision to be made at run time as illustrated in the following example:

mta_item_list_t item_list[6];
int index;

index = 0;
if (debug_sdk)
   item_list[index++].item_code = MTA_DEBUG_SDK;
if (debug_os)
   item_list[index++].item_code = MTA_DEBUG_OS;
if (debug_mm)
{
   item_list[index].item_code = MTA_DEBUG_MM;
   item_list[index++].item_length = 4;
}
if (debug_dq)
   item_list[index++].item_code = MTA_DEBUG_DEQUEUE;
if (debug_decode)
   item_list[index++].item_code = MTA_DEBUG_DECODE;
item_list[index].item_code = MTA_END_LIST;
istat = mtaInit(MTA_ITEM_CODE, item_list, 0);

The list of item code arguments must still be terminated with a call argument with value zero. Further, item codes may simultaneously be passed as distinct call arguments and also in item list arrays. For example:

mtaInit(MTA_DEBUG_SDK, MTA_ITEM_LIST, item_list1,
        MTA_INTERACTIVE, MTA_ITEM_LIST, item_list2, 0);

      

In the above, the item codes MTA_DEBUG_SDK, MTA_ITEM_LIST, MTA_INTERACTIVE, and MTA_ITEM_LIST are all explicitly passed as call arguments. Additional item codes are passed via the item list arrays item_list1 and item_list2.

When processing item codes, they are processed from left to right as the call argument list is interpreted. Using the above example, mtaInit() processes MTA_DEBUG_SDK, then MTA_ITEM_LIST, MTA_INTERACTIVE, MTA_ITEM_LIST, and finally the terminating 0 call argument which terminates the item code processing. When processing the first occurrence of MTA_ITEM_LIST, the entries of item_list1 are processed starting with the first entry (index 0), then the second, and so on until an entry with an item code value of 0 is encountered. The same processing occurs for item_list2.

If two item codes set the same underlying option or value, the last processed instance of that item code will prevail. For example, the call:

mtaInit(MTA_DEBUG_ENQUEUE, MTA_DEBUG_MM, 10, 0);

will leave the enqueue debug level set to 10. While the MTA_DEBUG_ENQUEUE item code sets it to 5, the subsequent MTA_DEBUG_MM item code changes the setting to 10.