Sun Java Communications Suite 5 Deployment Planning Guide

Chapter 9 Introduction to Messaging Server Software

Sun Java System Messaging Server is a powerful Internet messaging server designed for high-capacity, reliable handling of the messaging needs of both enterprises and service providers. The server consists of several modular, independently configurable components that provide support for several email protocols.

Messaging Server uses a centralized LDAP database for storing information about users, groups, and domains. Some information is stored in a set of local configuration files.

The Messaging Server product suite provides tools to support user provisioning and server configuration.

This chapter contains the following sections:

What Is a Messaging System?

A good email system architecture quickly delivers email with embedded sound, graphics, video files, and HTML forms, while providing for future upgrade and scalability. At a simplistic level, the Messaging Server architecture should:

Central to an email system architecture is the messaging server itself, a collection of components used to send and deliver messages. In addition to components provided in Messaging Server, the email system also requires an LDAP server and a DNS server. The DNS server must be in place before deploying your email system.

Several factors other than efficiency and scalability influence the Messaging Server architecture. Specifically these are:

See Chapter 11, Developing a Messaging Server Architecture for more information on these topics.

Messaging Server Support for Standards and Functionality

This section describes standards supported by Messaging Server, as well as other supported functionality.

Support for Standard Protocols

Messaging Server supports most national, international, and industry standards related to electronic messaging. For a complete list, see Appendix A, Supported Standards, in Sun Java System Messaging Server 6.3 Administration Reference.

Support for Hosted Domains

Messaging Server provides full support for hosted domains—email domains that are outsourced by an ISP. That is, the ISP provides email domain hosting for an organization by operating and maintaining the email services for that organization remotely. A hosted domain can share the same Messaging Server host with other hosted domains. In earlier LDAP-based email systems, a domain was supported by one or more email server hosts. With Messaging Server, many domains can be hosted on a single server. For each hosted domain, there is an LDAP entry that points to the user and group container for the domain and provides various domain-specific default settings.

When you define a domain, there must be a corresponding domain entry in the directory; that is, you must have an LDAP entry for the domain. Attributes such as mailAlternateAddress and mailEquivalentAddress depend on the existence of domain entries in the directory. Contrast this with vanity domains, which are domain names associated with an individual user, not with a specific server or hosted domain. The vanity domain does not have an LDAP entry for the domain name.


Note –

Because of their increased operational overhead, vanity domains are not recommended.


Support for User Provisioning

Messaging Server uses a centralized LDAP database for storing information about users, groups, and domains. At this time, Messaging Server supports two schema options, Sun Java System LDAP Schema version 1 (Schema 1) and Sun Java System LDAP Schema version 2 (Schema 2). The provisioning options will depend on which schema you have chosen. See Appendix A, Messaging Server Pre-Installation Considerations and Procedures, in Sun Java Communications Suite 5 Installation Guide for more information.

Messaging Server provisioning for Schema 2 is done using Delegated Administrator, as documented in the Sun Java System Delegated Administrator 6.4 Administration Guide. Delegated Administrator provides a graphical user interface and a set of command-line utilities for managing the users, groups, domains, resources, and service packages within an organization.

Schema 1 is supported by the iPlanet Delegated Administrator for Messaging product, which provides a graphical user interface and a set of command-line utilities for managing the users, groups, and domains within an organization. You can also use the following documentation, pertaining to previous software releases, for managing users, groups, and domains in Schema 1:


Note –

Access Manager console provides minimal Messaging Server and Calendar Server LDAP user entry provisioning using Access Manager Services. Because the interface provides no input validation, user entries that cannot receive email or otherwise don’t function will be created without reporting any errors. As a result, use this interface for demonstration purposes only.

Delegated Administrator, which is described in the Sun Java System Delegated Administrator 6.4 Administration Guide, is the recommended mechanism for provisioning Communications Suite users.


Support for Unified Messaging

Messaging Server provides the basis for a complete unified messaging solution: the concept of using a single message store for email, voicemail, fax, video, and other forms of communication.

Support for Webmail

Messaging Server currently ships with two native client user interfaces (UI):

Going forward, no new features will be added to the Messenger Express user interface. It has been deprecated in favor of the Communications Express user interface. Sun Microsystems, Inc. will announce an end-of-life timeline for Messenger Express at a future date.

See the Communications Express documentation for more information:

http://docs.sun.com/app/docs/coll/1312.1

Messaging Server Security and Access Control

Messaging Server provides the following security and access control features:

Messaging Server Administration User Interfaces

Messaging Server consists of several modular, independently configurable components that provide support for email transport and access protocols.

To configure the Message Transfer Agent (MTA), Messaging Server provides a complete set of command-line utilities and configuration files stored locally on the server. To configure the Message Store and message access services, Messaging Server provides a complete set of command-line utilities.

See the Sun Java System Messaging Server 6.3 Administration Guide for more information.

Messaging Server Software Architecture

Figure 9–1 shows a simplified standalone view of Messaging Server. While this particular deployment is not recommended for large deployment because it does not scale well, it does illustrate the individual components of the server.

Figure 9–1 Standalone Messaging Server, Simplified Components View

This diagram shows a simplified view of the Messaging
Server software components.

The preceding figure shows the following Messaging Server software components:

Message Path Through the Simplified Messaging Server System

Incoming messages from the Internet or local clients are received by the MTA through the Simple Mail Transport Protocol (SMTP). If the address is internal, that is, within the Messaging Server domain, the MTA delivers the message to the Message Store. If the message is external, that is, addressed to a domain outside of Messaging Server control, the MTA relays the message to another MTA on the Internet.

Although it is possible to deliver mail to the /var/mail file system (UNIX systems only), which was the case in previous versions of the Messaging Server, local messages are usually delivered to the more optimized Messaging Server Message Store. Messages are then retrieved by IMAP4, POP3, or HTTP mail client programs.

Outgoing messages from mail clients go directly to the MTA, which sends the message to the appropriate server on the Internet. If the address is local, the MTA sends the message to the Message Store.

New users and groups are created by adding user and group entries to the directory. Entries can be created or modified by using the Delegated Administrator utility or by modifying the directory using LDAP.

Messaging Server provides a set of command-line administrative interfacs and configuration files. Some of the more common administrative tasks are adding, modifying, and removing users and groups to the mail system, and configuring the operation of the MTA, directory server, and Message Store.

The Message Transfer Agent (MTA)

The MTA routes, transfers, and delivers Internet mail messages for Messaging Server. Mail flows through interfaces known as channels. Each channel consists of one or a pair of agent programs and a set of configuration information. The agent programs are a slave program, which handles mail coming into the channel, and a master program, which handles mail as it leaves the channel. There is a message queue for storing messages that are destined to be sent to one or more of the interfaces associated with any channel. Messaging Server provides a number of default channels, including:

Figure 9–2 illustrates the process. You can configure channels individually and direct mail to specific channels based on the address.

Figure 9–2 Channel Architecture

This diagram shows the Messaging Server channel architecture.

Channel programs perform one of two functions:

Channels are configurable by using the imta.cnf configuration text file. Through channel configuration, you can set a variety of channel keywords to control how messages are handled. Channel keywords affect performance tuning as well as reporting aspects of the system. For example, you can define multiple channels to segment traffic by destination, define message size limits to limit traffic, and define delivery status notification rules according to the needs of your business. Diagnostic attributes are also configurable on a per-channel basis. The number of configuration parameters that can be set on a channel basis is large.

See Chapter 8, MTA Concepts, in Sun Java System Messaging Server 6.3 Administration Guide for more information on MTA concepts.

Direct LDAP Lookup

The MTA looks up the information directly from the LDAP server. The direct lookup provides a scalable, fast, and configurable relationship between the MTA and the LDAP server. The results of the LDAP queries are cached in the process, with configurable size and time, so performance is tunable. See the Sun Java System Messaging Server 6.3 Administration Guide for more information.

Rewrite Rules

Mail is routed to a channel based on the result of running the destination addresses through domain rewriting rules, or rewrite rules for short. Rewrite rules are used to convert addresses into true domain addresses and to determine their corresponding channels. These rules are used to rewrite addresses appearing in both the transport layer and the message header. The transport layer is the message’s envelope. It contains routing information and is invisible to the user, but is the actual information used to deliver the message to the appropriate recipient.

The rewrite rules and the table of channels cooperate to determine the disposition of each address. The result of the rewrite process is a rewritten address and a routing system, that is, the system (channel) to which the message is to be sent/queued. Depending upon the topology of the network, the routing system might only be the first step along the path the message takes to reach its destination, or it might be the final destination system itself.

After the rewrite process has finished, a search is made for the routing system among the channel portion of the imta.cnf file. Each channel has one or more host names associated with it. The routing system name is compared against each of these names to determine to which channel to enqueue the message. A simple rewrite rule is shown here:

example.com     $U%example.com@tcp_siroe-daemon

This rule matches addresses for the domain example.com only. Such matching addresses would be rewritten using the template $U%$D, where:

$U

Indicates the user portion or left-hand side of the address (before the @)

%

Indicates the @ sign

$D

Indicates the domain portion or right-hand side of the address (after the @)

Thus, a message of the form wallaby@thor.example.com would be rewritten to wallaby@example.com, and would be sent to the channel whose channel host name is tcp_siroe-daemon.

Rewrite rules can perform sophisticated substitutions based on mapping tables, LDAP directory lookups, and database references. While occasionally cryptic, they are useful in the fact that they operate at a low level and impose little direct overhead on the message processing cycle. For full information on these and other features available in the rewrite process, see Chapter 11, Configuring Rewrite Rules, in Sun Java System Messaging Server 6.3 Administration Guide.

The Job Controller

Master channel programs are run under the control of the job controller, a program that controls the message queues and invokes the channel programs to do the actual message delivery. The job controller is a multithreaded process and is one of the few processes that is always present in the Messaging Server system. The channel processing jobs themselves are created by the job controller but are transient and might not be present when there is no work for them to do.

Job controller configuration settings determine if there is always at least one instance of a channel processing program. In many cases, these are set so that there is always at least one instance of the service program even when there is no immediate work to do. In other cases, there will be an instance for a set period of time after it last did some work but there is nothing to do currently.

Slave channels, which accept external messages, by queueing a message, notify the job controller of a newly created message file. The job controller enters this information into its internal data structure and if necessary creates a master channel job to process the message in that queue. This job creation might not be necessary if the job controller determines that an existing channel job can process the newly queued message file. When the master channel job starts, it gets its message assignment from the job controller. When it is finished with the message, the master channel updates the job controller as to the status of its processing. The status is either that the message is successfully dequeued or the message should be rescheduled for retrying.

The job controller maintains information about message priority and previous delivery attempts that failed, allowing for advantageous scheduling of channel jobs. The job controller also keeps track of the state of each job. The state can be idle, how long the job has been idle, or whether the job is busy. Tracking state enables the job controller to keep an optimal pool of channel jobs.


Note –

There are currently only two slave channels, SMTP slave and LMTP slave. These programs are controlled by the dispatcher, which is described next.


Dispatcher

The dispatcher is another process that is always present on a Messaging Server system. It is a multithreaded traffic dispatcher, which dispatches incoming SMTP or LMTP connections to the pool of SMTP or LMTP server threads for protocol-specific processing. The SMTP and LMTP server programs provide a pool of worker threads at the disposal of the dispatcher. After it processes a message by either rejecting the message or enqueuing it into its destination channel, the worker thread is ready to accept more work from the dispatcher.

The dispatcher can block incoming traffic based on IP address and throttles it to prevent denial of service attacks. It also creates and shuts down SMTP or LMTP server processes based on load and configuration. Therefore the SMTP or LMTP slave channel programs are under the control of the dispatcher, not the job controller.

Local Mail Transfer Protocol (LMTP)

As of the Messaging Server 6.0 release, you can configure LMTP for delivery to the Message Store in a multi-tiered deployment. In these scenarios, where you are using inbound relays and back-end Message Stores, the relays become responsible for address expansion and delivery methods such as autoreply and forwarding and also for mailing list expansion.

Delivery to the back-end stores historically has been over SMTP, which requires the back-end system to look up the recipient addresses in the LDAP directory again, thereby engaging the full machinery of the MTA. For speed and efficiency, the MTA can use LMTP rather than SMTP to deliver messages to the back-end store. See Chapter 16, LMTP Delivery, in Sun Java System Messaging Server 6.3 Administration Guide for more information.


Note –

By design, LMTP is intended for use in multi-tier deployments. It is not possible to use LMTP with single-system deployments. Also, the Messaging Server’s LMTP service as implemented is not designed to work with other LMTP servers or other LMTP clients.


The Message Store

The Message Store is a dedicated data store for the delivery, retrieval, and manipulation of Internet mail messages. The Message Store works with the IMAP4 and POP3 client access servers to provide flexible and easy access to messaging. The Message Store also works through the HTTP server (mshttpd) to provide messaging capabilities to Communications Express in a web browser. In addition to this section, see Chapter 20, Managing the Message Store, in Sun Java System Messaging Server 6.3 Administration Guide for more information.

The Message Store is organized as a set of folders or user mailboxes. The folder or mailbox is a container for messages. Each user has an INBOX where new mail arrives. Each IMAP or Webmail user can also have one or more folders where mail can be stored. Folders can contain other folders arranged in a hierarchical tree. Mailboxes owned by an individual user are private folders. Private folders can be shared at the owner’s discretion with other users on the same Message Store. Messaging Server supports sharing folders across multiple stores by using the IMAP protocol.

There are two general areas in the Message Store, one for user files and another for system files. In the user area, the location of each user’s INBOX is determined by using a two-level hashing algorithm. Each user mailbox or folder is represented by another directory in its parent folder. Each message is stored as a file. When there are many messages in a folder, the system creates hash directories for that folder. Using hash directories eases the burden on the underlying file system when there are many messages in a folder. In addition to the messages themselves, the Message Store maintains an index and cache of message header information and other frequently used data to enable clients to rapidly retrieve mailbox information and do common searches without the need to access the individual message files.

A Message Store can contain many message store partitions for user files. A Message Store partition is contained by a file system volume. As the file system becomes full, you can create additional file system volumes and Message Store partitions on those file system volumes to store new users.


Note –

If the partition gets full, users on the partition will not be able to store additional messages. There are several ways to address the problem:

For more information, see Chapter 20, Managing the Message Store, in Sun Java System Messaging Server 6.3 Administration Guide.


The Message Store maintains only one copy of each message per partition. This is sometimes referred to as a single-copy message store. When the Message Store receives a message addressed to multiple users or a group or distribution list, it adds a reference to the message in each user’s INBOX. Rather than saving a copy of the message in each user’s INBOX, Message Store avoids the storage of duplicate data. The individual message status flag (seen, read, answered, deleted, and so on) is maintained per folder for each user.

The system area contains information on the entire Message Store in a database format for faster access. The information in the system area can be reconstructed from the user area. Messaging Server contains a database snapshot function. When needed, you can quickly recover the database to a known state. Messaging Server also has fast recovery, so that in case of database corruption, you can shut down the Message Store and bring it back immediately without having to wait for a lengthy database reconstruction.

The Message Store supports per-user quotas. Enforcement of quota can be turned on or off. You can configure a user quota by using number of bytes or number of messages. You can also set a threshold so that if the quota reaches the threshold, a warning message can be sent to the user. When the user is over quota, new messages can be held up for retry during a grace period. After the grace period, messages sent to the over-quota user are returned to the sender with a non-delivery notification.

For special applications where quota is used, but messages must be delivered regardless of the quota status of the users, there is a guaranteed message delivery channel. This channel can be used to deliver all messages regardless of quota status. Utilities are available for reporting quota usage and for sending over quota warnings.

Messaging Server and Directory Services

Messaging Server is bundled with Sun Java System Directory Server. Directory Server is a Lightweight Directory Access Protocol (LDAP) directory service. Directory Server provides the central repository for information critical to the operation of Messaging Server. This information includes user profiles, distribution lists, and other system resources.

Directory Information Tree

The directory stores data in the form of a tree, known as the Directory Information Tree (DIT). The DIT is a hierarchical structure, with one major branch at the top of the tree and branches and subbranches below. The DIT is flexible enough to enable you to design a deployment that fits your organization’s needs. For example, you might choose to arrange the DIT according to your actual business organizational structure, or by the geographical layout of your business. You also might want to design a DIT that follows a one-to-one mapping to your DNS layers. Use care when designing your DIT, as changing it after the fact is not an easy task.

The DIT is also flexible enough to accommodate a wide range of administration scenarios. You can administer the DIT in either a centralized or distributed manner. In centralized administration, one authority manages the entire DIT. You would use centralized administration where the entire DIT resides on one mail server. In distributed administration, multiple authorities manage the DIT. Usually you implement distributed administration when the DIT is divided into portions, or subtrees, residing on different mail servers.

When the DIT is large, or when mail servers are geographically disbursed, consider delegating management of portions of the DIT. Typically, you assign an authority to manage each subtree of the DIT. Messaging Server enables you to manage multiple subtrees from one authority. However, for security reasons, an authority can only make changes to the subtree of the DIT that the authority owns.

The default schema used by Messaging Server when Access Manager is not used is different from the one used by Access Manager. Messaging Server supports both Sun Java System LDAP Schema 1 and 2, and allows for transition and migration of the schemas.

Directory Replication

Directory Server supports replication, enabling a variety of configurations that provide redundancy and efficiency. Enabling replication of all or part of the DIT from one host to other hosts provides the following configuration capabilities:

For more information on directory replication, directory performance tuning, and DIT structure and design, see the Sun Java System Directory Server documentation:

http://docs.sun.com/app/docs/coll/1316.1

Provisioning Messaging Users

See Chapter 8, Understanding Schema and Provisioning Options for information on schema and provisioning options for Messaging Server users.

Message Archiving Using the AXS-One System

Messaging Server supports archiving through the AXS-One message archiving system. Whether you require archiving for regulatory, compliance or legislative purposes, or you wish to manage the growth of your message store or reduce the storage costs, the AXS-One messaging archiving system can help them to achieve both, individually, or simultaneously.

For more information on how to connect the AXS-One archiving system to Sun Java System Messaging Server, see Message Archiving Using the Sun Compliance and Content Management Solution.