Figure 9–1 shows a simplified standalone view of Messaging Server. While this particular deployment is not recommended for large deployment because it does not scale well, it does illustrate the individual components of the server.
The preceding figure shows the following Messaging Server software components:
Message Transfer Agent or MTA. Receives, routes, transports, and delivers mail messages using the SMTP protocol. An MTA delivers messages to a local mailbox or to another MTA.
Message Store. Consists of a set of components that store, retrieve, and manipulate messages for mail clients. Mail can be retrieved by POP, IMAP, or HTTP clients. POP clients download messages to the client machine for reading and storage. IMAP and HTTP clients read and manipulate messages on the server.
LDAP directory. Stores, retrieves, and distributes mail directory information for Messaging Server. This includes user routing information, distribution lists, and other information necessary to support delivery and access of email. The LDAP directory also stores passwords, and any other information needed by the MTA or Message Store to authenticate users.
In addition to storing messages, the Message Store uses the directory server to verify user login name and passwords for mail clients accessing their mail. The directory also stores information about quota limits, default message store type, and so on.
DNS Server. Translates domain names into IP addresses. This component needs to be present before Messaging Server is installed.
Incoming messages from the Internet or local clients are received by the MTA through the Simple Mail Transport Protocol (SMTP). If the address is internal, that is, within the Messaging Server domain, the MTA delivers the message to the Message Store. If the message is external, that is, addressed to a domain outside of Messaging Server control, the MTA relays the message to another MTA on the Internet.
Although it is possible to deliver mail to the /var/mail file system (UNIX systems only), which was the case in previous versions of the Messaging Server, local messages are usually delivered to the more optimized Messaging Server Message Store. Messages are then retrieved by IMAP4, POP3, or HTTP mail client programs.
Outgoing messages from mail clients go directly to the MTA, which sends the message to the appropriate server on the Internet. If the address is local, the MTA sends the message to the Message Store.
New users and groups are created by adding user and group entries to the directory. Entries can be created or modified by using the Delegated Administrator utility or by modifying the directory using LDAP.
Messaging Server provides a set of command-line administrative interfacs and configuration files. Some of the more common administrative tasks are adding, modifying, and removing users and groups to the mail system, and configuring the operation of the MTA, directory server, and Message Store.
The MTA routes, transfers, and delivers Internet mail messages for Messaging Server. Mail flows through interfaces known as channels. Each channel consists of one or a pair of agent programs and a set of configuration information. The agent programs are a slave program, which handles mail coming into the channel, and a master program, which handles mail as it leaves the channel. There is a message queue for storing messages that are destined to be sent to one or more of the interfaces associated with any channel. Messaging Server provides a number of default channels, including:
SMTP Channel. Enables TCP/IP-based message delivery and receipt. Both master and slave channels are provided.
LMTP Channel. Enables routing of messages directly from MTAs to the Message Store in a two-tiered configuration. These channels communicate with the Message Store on another system over LMTP instead of SMTP. Both master and slave channels are provided.
Pipe Channel. Used for alternative message delivery programs. Enables delivery of messages to programs such as a mail sorter rather than directly to a user’s inbox. A master channel is provided.
Local Channel. Delivers mail to /var/mail. Provides for compatibility with older UNIX mail clients. A master channel is provided.
Reprocessing Channel. Useful for messages that are resubmitted. A master channel is provided.
Defragmentation Channel. Reassembles partial messages into the original complete message to support the MIME message/partial content type. A master channel is provided.
Conversion Channel. Performs body part by body part conversion on messages. Useful for rewriting addresses or re-formatting messages. A master channel is provided.
Message Store Channel. Provides for local delivery to the message store.
Figure 9–2 illustrates the process. You can configure channels individually and direct mail to specific channels based on the address.
Channel programs perform one of two functions:
SMTP slave programs enqueue messages into message queues for further processing by the MTA, or reject the message so it is not accepted onto the system, and accept messages from other interfaces.
Master programs process the message from its queue area, and enqueue it on the same system for further processing by another channel, or transmit messages off system to other interfaces, deleting them from their queue after they are sent, or deliver the message to the final destination on the system, such as the Message Store.
Channels are configurable by using the imta.cnf configuration text file. Through channel configuration, you can set a variety of channel keywords to control how messages are handled. Channel keywords affect performance tuning as well as reporting aspects of the system. For example, you can define multiple channels to segment traffic by destination, define message size limits to limit traffic, and define delivery status notification rules according to the needs of your business. Diagnostic attributes are also configurable on a per-channel basis. The number of configuration parameters that can be set on a channel basis is large.
See Chapter 8, MTA Concepts, in Sun Java System Messaging Server 6.3 Administration Guide for more information on MTA concepts.
The MTA looks up the information directly from the LDAP server. The direct lookup provides a scalable, fast, and configurable relationship between the MTA and the LDAP server. The results of the LDAP queries are cached in the process, with configurable size and time, so performance is tunable. See the Sun Java System Messaging Server 6.3 Administration Guide for more information.
Mail is routed to a channel based on the result of running the destination addresses through domain rewriting rules, or rewrite rules for short. Rewrite rules are used to convert addresses into true domain addresses and to determine their corresponding channels. These rules are used to rewrite addresses appearing in both the transport layer and the message header. The transport layer is the message’s envelope. It contains routing information and is invisible to the user, but is the actual information used to deliver the message to the appropriate recipient.
The rewrite rules and the table of channels cooperate to determine the disposition of each address. The result of the rewrite process is a rewritten address and a routing system, that is, the system (channel) to which the message is to be sent/queued. Depending upon the topology of the network, the routing system might only be the first step along the path the message takes to reach its destination, or it might be the final destination system itself.
After the rewrite process has finished, a search is made for the routing system among the channel portion of the imta.cnf file. Each channel has one or more host names associated with it. The routing system name is compared against each of these names to determine to which channel to enqueue the message. A simple rewrite rule is shown here:
example.com $U%example.com@tcp_siroe-daemon
This rule matches addresses for the domain example.com only. Such matching addresses would be rewritten using the template $U%$D, where:
Indicates the user portion or left-hand side of the address (before the @)
Indicates the @ sign
Indicates the domain portion or right-hand side of the address (after the @)
Thus, a message of the form wallaby@thor.example.com would be rewritten to wallaby@example.com, and would be sent to the channel whose channel host name is tcp_siroe-daemon.
Rewrite rules can perform sophisticated substitutions based on mapping tables, LDAP directory lookups, and database references. While occasionally cryptic, they are useful in the fact that they operate at a low level and impose little direct overhead on the message processing cycle. For full information on these and other features available in the rewrite process, see Chapter 11, Configuring Rewrite Rules, in Sun Java System Messaging Server 6.3 Administration Guide.
Master channel programs are run under the control of the job controller, a program that controls the message queues and invokes the channel programs to do the actual message delivery. The job controller is a multithreaded process and is one of the few processes that is always present in the Messaging Server system. The channel processing jobs themselves are created by the job controller but are transient and might not be present when there is no work for them to do.
Job controller configuration settings determine if there is always at least one instance of a channel processing program. In many cases, these are set so that there is always at least one instance of the service program even when there is no immediate work to do. In other cases, there will be an instance for a set period of time after it last did some work but there is nothing to do currently.
Slave channels, which accept external messages, by queueing a message, notify the job controller of a newly created message file. The job controller enters this information into its internal data structure and if necessary creates a master channel job to process the message in that queue. This job creation might not be necessary if the job controller determines that an existing channel job can process the newly queued message file. When the master channel job starts, it gets its message assignment from the job controller. When it is finished with the message, the master channel updates the job controller as to the status of its processing. The status is either that the message is successfully dequeued or the message should be rescheduled for retrying.
The job controller maintains information about message priority and previous delivery attempts that failed, allowing for advantageous scheduling of channel jobs. The job controller also keeps track of the state of each job. The state can be idle, how long the job has been idle, or whether the job is busy. Tracking state enables the job controller to keep an optimal pool of channel jobs.
There are currently only two slave channels, SMTP slave and LMTP slave. These programs are controlled by the dispatcher, which is described next.
The dispatcher is another process that is always present on a Messaging Server system. It is a multithreaded traffic dispatcher, which dispatches incoming SMTP or LMTP connections to the pool of SMTP or LMTP server threads for protocol-specific processing. The SMTP and LMTP server programs provide a pool of worker threads at the disposal of the dispatcher. After it processes a message by either rejecting the message or enqueuing it into its destination channel, the worker thread is ready to accept more work from the dispatcher.
The dispatcher can block incoming traffic based on IP address and throttles it to prevent denial of service attacks. It also creates and shuts down SMTP or LMTP server processes based on load and configuration. Therefore the SMTP or LMTP slave channel programs are under the control of the dispatcher, not the job controller.
As of the Messaging Server 6.0 release, you can configure LMTP for delivery to the Message Store in a multi-tiered deployment. In these scenarios, where you are using inbound relays and back-end Message Stores, the relays become responsible for address expansion and delivery methods such as autoreply and forwarding and also for mailing list expansion.
Delivery to the back-end stores historically has been over SMTP, which requires the back-end system to look up the recipient addresses in the LDAP directory again, thereby engaging the full machinery of the MTA. For speed and efficiency, the MTA can use LMTP rather than SMTP to deliver messages to the back-end store. See Chapter 16, LMTP Delivery, in Sun Java System Messaging Server 6.3 Administration Guide for more information.
By design, LMTP is intended for use in multi-tier deployments. It is not possible to use LMTP with single-system deployments. Also, the Messaging Server’s LMTP service as implemented is not designed to work with other LMTP servers or other LMTP clients.
The Message Store is a dedicated data store for the delivery, retrieval, and manipulation of Internet mail messages. The Message Store works with the IMAP4 and POP3 client access servers to provide flexible and easy access to messaging. The Message Store also works through the HTTP server (mshttpd) to provide messaging capabilities to Communications Express in a web browser. In addition to this section, see Chapter 20, Managing the Message Store, in Sun Java System Messaging Server 6.3 Administration Guide for more information.
The Message Store is organized as a set of folders or user mailboxes. The folder or mailbox is a container for messages. Each user has an INBOX where new mail arrives. Each IMAP or Webmail user can also have one or more folders where mail can be stored. Folders can contain other folders arranged in a hierarchical tree. Mailboxes owned by an individual user are private folders. Private folders can be shared at the owner’s discretion with other users on the same Message Store. Messaging Server supports sharing folders across multiple stores by using the IMAP protocol.
There are two general areas in the Message Store, one for user files and another for system files. In the user area, the location of each user’s INBOX is determined by using a two-level hashing algorithm. Each user mailbox or folder is represented by another directory in its parent folder. Each message is stored as a file. When there are many messages in a folder, the system creates hash directories for that folder. Using hash directories eases the burden on the underlying file system when there are many messages in a folder. In addition to the messages themselves, the Message Store maintains an index and cache of message header information and other frequently used data to enable clients to rapidly retrieve mailbox information and do common searches without the need to access the individual message files.
A Message Store can contain many message store partitions for user files. A Message Store partition is contained by a file system volume. As the file system becomes full, you can create additional file system volumes and Message Store partitions on those file system volumes to store new users.
If the partition gets full, users on the partition will not be able to store additional messages. There are several ways to address the problem:
Reduce the size of user mailboxes
If you are using volume management software, add additional disks
Create additional partitions and move mailboxes to the new partitions
For more information, see Chapter 20, Managing the Message Store, in Sun Java System Messaging Server 6.3 Administration Guide.
The Message Store maintains only one copy of each message per partition. This is sometimes referred to as a single-copy message store. When the Message Store receives a message addressed to multiple users or a group or distribution list, it adds a reference to the message in each user’s INBOX. Rather than saving a copy of the message in each user’s INBOX, Message Store avoids the storage of duplicate data. The individual message status flag (seen, read, answered, deleted, and so on) is maintained per folder for each user.
The system area contains information on the entire Message Store in a database format for faster access. The information in the system area can be reconstructed from the user area. Messaging Server contains a database snapshot function. When needed, you can quickly recover the database to a known state. Messaging Server also has fast recovery, so that in case of database corruption, you can shut down the Message Store and bring it back immediately without having to wait for a lengthy database reconstruction.
The Message Store supports per-user quotas. Enforcement of quota can be turned on or off. You can configure a user quota by using number of bytes or number of messages. You can also set a threshold so that if the quota reaches the threshold, a warning message can be sent to the user. When the user is over quota, new messages can be held up for retry during a grace period. After the grace period, messages sent to the over-quota user are returned to the sender with a non-delivery notification.
For special applications where quota is used, but messages must be delivered regardless of the quota status of the users, there is a guaranteed message delivery channel. This channel can be used to deliver all messages regardless of quota status. Utilities are available for reporting quota usage and for sending over quota warnings.
Messaging Server is bundled with Sun Java System Directory Server. Directory Server is a Lightweight Directory Access Protocol (LDAP) directory service. Directory Server provides the central repository for information critical to the operation of Messaging Server. This information includes user profiles, distribution lists, and other system resources.
The directory stores data in the form of a tree, known as the Directory Information Tree (DIT). The DIT is a hierarchical structure, with one major branch at the top of the tree and branches and subbranches below. The DIT is flexible enough to enable you to design a deployment that fits your organization’s needs. For example, you might choose to arrange the DIT according to your actual business organizational structure, or by the geographical layout of your business. You also might want to design a DIT that follows a one-to-one mapping to your DNS layers. Use care when designing your DIT, as changing it after the fact is not an easy task.
The DIT is also flexible enough to accommodate a wide range of administration scenarios. You can administer the DIT in either a centralized or distributed manner. In centralized administration, one authority manages the entire DIT. You would use centralized administration where the entire DIT resides on one mail server. In distributed administration, multiple authorities manage the DIT. Usually you implement distributed administration when the DIT is divided into portions, or subtrees, residing on different mail servers.
When the DIT is large, or when mail servers are geographically disbursed, consider delegating management of portions of the DIT. Typically, you assign an authority to manage each subtree of the DIT. Messaging Server enables you to manage multiple subtrees from one authority. However, for security reasons, an authority can only make changes to the subtree of the DIT that the authority owns.
The default schema used by Messaging Server when Access Manager is not used is different from the one used by Access Manager. Messaging Server supports both Sun Java System LDAP Schema 1 and 2, and allows for transition and migration of the schemas.
Directory Server supports replication, enabling a variety of configurations that provide redundancy and efficiency. Enabling replication of all or part of the DIT from one host to other hosts provides the following configuration capabilities:
The directory information is more accessible, because it is replicated on multiple servers rather than on a single server.
The directory information can be cached on a local directory server, reducing effort of accessing the information from a remote directory server. Caching the directory information enhances performance, especially in deployments with limited network bandwidth to the central directory.
Depending on the actual configuration, multiple directory servers can process mail client requests faster than a single centralized server.
For more information on directory replication, directory performance tuning, and DIT structure and design, see the Sun Java System Directory Server documentation:
http://docs.sun.com/app/docs/coll/1316.1
See Chapter 8, Understanding Schema and Provisioning Options for information on schema and provisioning options for Messaging Server users.
Messaging Server supports archiving through the AXS-One message archiving system. Whether you require archiving for regulatory, compliance or legislative purposes, or you wish to manage the growth of your message store or reduce the storage costs, the AXS-One messaging archiving system can help them to achieve both, individually, or simultaneously.
For more information on how to connect the AXS-One archiving system to Sun Java System Messaging Server, see Message Archiving Using the Sun Compliance and Content Management Solution.