CHAPTER 9 |
Sun Message Store |
The Sun Message Store is the primary message store that is used by the SunTM Internet Mail ServerTM 4.0. This provides a significant advance in reliability, performance, and scalability among open systems message stores.
Topics in this chapter include:
|
|
Sun Message Store key features |
|
|
How does the message store work? |
|
|
User identification for message store and message access |
|
|
Mail folders |
|
|
Sun OpenWindows Mail Tool V3 format conversion |
|
|
Simultaneous connections |
|
|
Message access protocols |
|
|
Message access service specifications |
The key features of the Sun Message Store component are:
|
|
Supported Internet Standards--The message store stores any message that conforms to RFC 822 specifications. It recognizes MIME content format and supports direct address ability of any header or body part. It is specifically designed for IMAP4 message access. |
|
|
Reliable Scalable Design--Write-once data store and two-level indexing simplify access, reduce contention, and facilitate multithreading. Committed transactions also facilitate multithreading and ensure that no messages are lost or corrupted. |
|
|
High Storage Efficiency--The message store retains exactly one copy of each message, regardless of the number of recipients. |
|
|
Optimized Access--Messages are prepared and indexed when inserted into the store. No parsing is necessary when messages are accessed. The degree of preparing is tunable. The benefits of preparing decrease as message size increases. POP users do not need parsing at all. |
|
|
Optimized File System Usage--Time-based sorting of messages within the data store provides good locality of reference and more effective use of disk caches. |
|
|
Optimized Updates--Once in the store, messages are never modified. Status changes and folder updates are stored in lightweight index files that are rapidly updated. |
|
|
Managed Backup, Migration, Archival, and Purge--Bulk dump and load facility supports backup, restore, and migration of individual users, groups, or entire stores. Deletion and purge tools support archival and guaranteed delete. |
The Sun Message Store is organized into the following major subcomponents:
This is the IMTA channel queue interface that accepts messages from the IMTA and inserts them into the message store. The implementation is as follows:
|
|
Messages are appended to the central data store. |
|
|
A data store index entry is created. |
|
|
A pointer to the index is added to the inbox folder of each recipient. |
The retrieval interface is the c-client driver interface, used by c-client applications to manipulate messages in the message store. The Sun Message Store driver can be used only in protocol server applications, like the IMAP4 and POP3 servers.
This facility is the generalized bulk dump and load facility that can be used for message store backup, moving users from one message store to another.
The message store provides extensive statistics, including disk space in use, number of messages in folders, oldest messages, and most recent activity. Monitoring parameters are exposed through the c-client driver interface. The set of managed objects is specified by the message access component.
These command-line utilities are provided for periodic maintenance.
The Sun Message Store maintains only one copy of each message. If the Sun Message Store receives a message addressed to multiple users or a group (distribution list), it adds a reference to the message in each user's Inbox rather than having a copy of the message in each user's Inbox, thereby saving disk space. The individual message status (new, unread, replied to, deleted, and the like) is maintained per Mailbox.
FIGURE 9-1 shows the major elements of the Sun Message Store and the flow of messages into the Sun Message Store from the IMTA and to the mail client users.
SIMS also supports /var/mail. A site can implement both the Sun Message Store and /var/mail. Mail users can access both the Sun Message Store and /var/mail using the Internet Mail Access Protocol version 4 (IMAP4) or the Post Office Protocol version 3 (POP3).
Access to the Sun Message Store and /var/mail is multithreaded. This feature enables a single process to manage a large number of connections. Each connection is handled by a thread. Multithreaded access maximizes both performance and scaleability by minimizing the system resources required for the management of each connection.
Each SIMS email user has three distinct identification (ID) strings:
|
|
Email ID--The ID used to send mail to this users. It is also called the mail address. Example: mayflower@bridge.net |
|
|
Login ID--The ID a client uses to access his mailbox. It is also called the mail access ID. Examples: mayflower, bluebell+stream.com |
|
|
Message Store ID--The user identification required by message store commands such as imbackup, imrestore, imcheck, iminitquota, imexpire, imimportmbox, imdeluser, and so on. |
The login ID is different from the message store ID depending on whether the LDAP attribute simsRecursive is set to 1 or 0 and the user entry is in a subdomain of the default domain. If simsRecursive is set to 1, the message store ID of users in a subdomain do not need to be appended with the domain. If simsRecursive is set to 0, the Login ID and message store ID of users in a subdomain need to be appended with the domain. In a hosted domain, both the Login ID and message store ID of users in a subdomain need to be appended with the domain. In either case, Email IDs do not change. Below are three examples:
Example 1, default domain: stream.com, simsRecursive=1
Distinguished Name for mayflower: cn=mayflower,ou=People,dc=stream,dc=com,o=internet
Distinguished Name for bluebell: cn=bluebell,ou=People,dc=eng,dc=stream,dc=com,o=internet
TABLE 9-1 Default Domain: stream.com, simsRecursive=1
Message Store ID
Login ID
mayflower
mayflower or mayflower+stream.com
bluebell
bluebell or bluebell+eng.stream.com
Example 2, default domain: stream.com, simsRecursive=0:
Distinguished Name for mayflower: cn=mayflower,ou=People,dc=stream,dc=com,o=internet
Distinguished Name for bluebell: cn=bluebell,ou=People,dc=eng,dc=stream,dc=com,o=internet
TABLE 9-2 Default Domain: stream.com, simsRecursive=0
Message Store ID
Login ID
Example 3, Hosted domain:
Distinguished Name for mayflower: cn=mayflower,ou=People,dc=string,dc=com,o=internet
Distinguished Name for bluebell: cn=bluebell,ou=People,dc=eng,dc=stroller,dc=com,o=internet
TABLE 9-3 Hosted Domain
Message Store ID
Login ID
The Sun Message Store is where user messages are stored and retrieved. Each user has an Inbox where new mail arrives. Users can also create additional private folders or mailboxes, where they file and store messages they have read. Folders can contain other folders in a hierarchical tree.
In addition to the Inbox and private folders, users can also own shared folders. A shared folder is similar to a email group, but instead of messages going into each members inbox, messages addressed to the shared folder go into a private folder in each users message store.
For example, the network design team of the Stream Corporation have created a shared folder called network-design@bridge.net. Messages sent to this address go into private shared folders called network-design in each members message store space. Each member brings up that mailbox in order to see the mail sent to that address. The client syntax for accessing the mailbox is #shared/<folder name>. In this example members would enter #shared/network-design to bring up the shared folder. On recent IMAP clients that support namespace, these folders are visible automatically.
Members can read then delete messages from their view of the shared folder, but the actual messages remain in the mailbox of shared folder's owner. The owner of the shared folder is designated when the shared folder is first created. The owner is the only member of the group who can permanently expunge the message from the shared folder. The designated owner of the shared folder uses the syntax
#shared.<folder name>/Inbox to access the shared folder. (In our example that would be #shared.network-design/Inbox.)
Before a message can be processed by the Sun Message Store, the body of the message must be in Multipurpose Internet Mail Extensions (MIME) format. (The Sun Message Store stores messages in MIME format only.) When a message enters the Sun Message Store queue, it checks the message body format. If the message body is in MIME format, the Sun Message Store automatically processes the message. If the message body is in Sun OpenWindowsMail Tool V3 format (the message was generated in Mail Tool), the Sun Message Store converts the message body format to MIME. This feature is not user configurable.
SIMS allows simultaneous IMAP and POP client connections. While this is standard for most IMAP clients, for POP clients this means that if a connection is dropped, or a user wants more than one connection, then a user can immediately reconnect and have access to his messages without having to wait for the POP server to time-out.
When multiple IMAP clients connect to a mailbox, all flag changes (new, seen, deleted, answered, and so on) and message states (new mail, removed message) are shared between clients by means of the IMAP protocol.
When IMAP and POP clients connect to a mailbox, all flag changes (new, seen, deleted, answered, and so on) and message states (new mail, remove message) are shared between IMAP clients. POP clients, however, will not see IMAP deleted messages. When a POP client connects to a mailbox, it grabs a snapshot of the mailbox which persists until the POP session is terminated.
When multiple POP clients connect to a mailbox, each grabs a unique snapshot of the mailbox state at connect time. Each connection's unique snapshot persists until the POP session is terminated. Any changes to a mailbox's state are seen by clients that connect after these changes are made.
APOP is a POP command that the mail client can use as an alternative to USER/PASS (RFC 1939) for secure authentication. Unlike USER/PASS, APOP does not use the user's plaintext password for authentication. Instead, it uses an encoding of the password together with a challenge string. This encoding uses the md5 method.
This section provides further explanation of the message access protocols, including:
|
|
Message access protocol transparency |
|
|
Security between mail client and mail server |
The Sun Message Store and /var/mail support the Internet Mail Access Protocol version 4 (IMAP4) and the Post Office Protocol version 3 (POP3). Therefore, a mail client user can access either message store using either protocol. For example, a user's Inbox and other folders can be stored on the Sun Message Store and accessed using IMAP4 or POP3 or they can be stored in /var/mail and accessed using IMAP4 or POP3.
Message access services refers to the protocol servers, software drivers, and libraries that support client access to the message store. The key to this component is the Internet Message Access Protocol version 4 (IMAP4), implemented via the c-client Mail API library. This component is also responsible for the Post Office Protocol version 3 (POP3). The key features of this component are:
|
|
IMAP4 Revision 1--This protocol has been extended to optimize network usage and improve low-bandwidth performance. |
|
|
Advanced POP3--In addition to the mandatory POP3 command set, message access services supports the optional TOP and UIDL commands. The UIDL implementation is based on IMAP4 Universal Identifiers. |
|
|
Orthogonal Message Store Access--Multiple access protocols (IMAP and POP3) are supported from a common message store. Similarly, both message stores (Solaris Mailbox Format and Sun Message Store) support both access protocols. |
|
|
Support for Sun Mail tool V3 Attachment Format--Documents received in Sun V3 format are automatically converted in the message stores to MIME. |