This section provides information about the utilities you use to perform maintenance and recovery tasks for the message store. You should always read your postmaster mail for warnings and alerts that the server might send. You should also monitor the log files for information about how the server is performing. For more information about log files, see Chapter 21, Managing Logging
This section contains the following:
The Messaging Server message store contains the user mailboxes for a particular Messaging Server instance. The size of the message store increases as the number of mailboxes, folders, and log files increase.
As you add more users to your system, your disk storage requirements increase. Depending on the number of users your server supports, the message store might require one physical disk or multiple physical disks. Messaging Server enables you an add more stores as needed. One approach to adding more stores is by using storage appliances. See for information on how to configure Network Appliance storage appliances with Messaging Server.
This section describes the following utilities for managing and monitoring mailboxes: mboxutil, hashdir, readership.
Use the mboxutil command to perform typical maintenance tasks on mailboxes. mboxutil tasks include the following:
List mailboxes
List and remove orphaned and inactive mailboxes
Create mailboxes
Rename mailboxes
Move mailboxes from one partition to another
Delete orphaned or inactive mailboxes
You can also use the mboxutil command to view information about quotas. For more information, see To Monitor Quota Limits.
Note that you should not to kill the mboxutil process in the middle of execution. If it is killed with SIGKILL (kill -9), it may potentially require that every server get restarted and a recovery be done.
Table 18–11 lists the mboxutil commands. For detailed syntax and usage requirements, see the Sun Java System Messaging Server 6 2005Q4 Administration Reference.
Table 18–11 mboxutil Options
Option |
Description |
---|---|
-a |
Obsolete. Used to list all user quota information. Use. imquotacheck |
-c mailbox |
Creates the specified mailbox. Can be used with -f. A mailbox must exist before creating a secondary mailbox. |
-d mailbox |
Delete the specified mailbox. To delete a user from the message store, use the following value for -d mailbox: user/userid/INBOX For example, to delete the user john from the message store, use -d user/john/INBOX. To delete the mm folder in the user john’s mailbox, use -d user/john/mm. The recommended method to delete a user is to mark the user status as deleted in the LDAP directory (by using the Delegated Administrator utility commadmin user delete command or the Delegated Administrator console.) Next, use the commadmin user purge command to purge the users that have been marked as deleted for a period longer than the specified number of days. If you use the Delegated Administrator utility as described in the preceding paragraph, you do not have to use the mboxutil -d command to delete a mailbox. |
-e |
Expunges all deleted messages in the message store. This option also can be used with the -p pattern option to expunge all deleted mailboxes with names that match pattern. |
-f file |
Specifies a file that stores mailbox names. The -f option can be used with the -c, -d, or -r options. The file contains a list of mailboxes on which the mboxutil command is executed. The following is an example of entries in a data file: user/daphne/INBOXuser/daphne/projxuser/daphne/mm |
-k mailbox cmd |
Obsolete. Locks the specified mailbox at the folder level; runs the specified command; after command completes, unlocks the mailbox. |
-l |
Lists all of the mailboxes on a server. If you create multi-byte folders for different language locales, you should edit: msg_svr_base/sbin/bundles/encbylang.properties to associate the appropriate character set with the LANG environment variable. |
-o |
Checks for orphaned accounts. This option searches for inboxes in the current messaging server host which do not have corresponding entries in LDAP. For example, the -o option finds inboxes of owners who have been deleted from LDAP or moved to a different server host. For each orphaned account it finds, mboxutil writes the following command to the standard output: mboxutil-d user/userid/INBOX unless -w is specified |
-p MUTF7_IMAP_pattern |
When used with the -l option, lists only those mailboxes with names that match MUTF7_IMAP_pattern. Can also be used with the -d or -e option to delete or expunge mailboxes with names that match MUTF7_IMAP_pattern. You can use IMAP wildcards. This option expects a pattern in IMAP M-UTF-7 format. This is not the recommended way to search for non ascii mailboxes. To search for non ascii mailboxes, use the -P option. |
-P regexp |
Lists only those mailboxes with names that match the specified POSIX regular expression. This option expects the regexp in the local language |
-q domain |
Obsolete. Use imquotacheck -d domain |
-r oldname newname[partition] |
Renames the mailbox from oldname to newname. To move a folder from one partition to another, specify the new partition with the partition option. Can be used with the -f flag to use a file. This option can be used to rename a user. For example, mboxutil -r user/user1/INBOX user/user2/INBOX moves all mail and mailboxes from user1 to user2, and new messages will appear in the new INBOX. (If user2 already exists, this operation will fail.) |
-R mailbox |
Restores deleted messages that have not yet been purged. When a mailbox is expunged or expired, the uids of the deleted messages are stored in a store.exp file. The messages are physically removed by imexpire after the cleanup page has passed. When expunge or expire is issued by mistake, this option can by used to restore the deleted messages that have not been purged by imexpire into the original mailbox. |
-s |
When used with the -l option, displays only the mailbox name. No other data is displayed. |
-t num |
Lists the mailboxes that have not been accessed in a specified number of days (num). The -t option must be used with the -o option, which identifies orphaned mailboxes. Thus, the -t option identifies inactive mailboxes (based on last-accessed date) together with orphaned mailboxes (mailboxes that do not have corresponding user entries in the LDAP directory). To identify (list) the orphaned and inactive mailboxes, use mboxutil -o -w file -t num. To mark these orphaned and inactive mailboxes for deletion, use mboxutil -d -f file, where file is the same file as the one used in the preceding -w file. To use this feature, the config variable local.enablelastaccess must be enabled for at least the number of days specified with the -t option. |
-u user |
Obsolete. Used to list user information. Use imquotacheck -u user |
-w file |
Used with the -o option. Writes to a file the mailbox names generated by the -o option (which identifies orphaned accounts). |
-x |
When used with the -l option, shows the path and access control for a mailbox. |
POSIX regular expressions can be used in the mboxutil command.
You must specify mailbox names in the following format: user/userid/mailbox, where userid is the user that owns the mailbox and mailbox is the name of the mailbox. For hosted domains, userid is uid@domain.
For example, the following command creates the mailbox named INBOX for the user whose user ID is crowe. INBOX is the default mailbox for mail delivered to the user crowe.
mboxutil -c user/crowe/INBOX
Important: The name INBOX is reserved for each user’s default mailbox. INBOX is the only folder name that is case-insensitive. All other folder names are case-sensitive.
To list all mailboxes for all users:
mboxutil -l
To list all mailboxes and also include path and ACL information:
mboxutil -l -x
To create the default mailbox named INBOX for the user daphne:
mboxutil -c user/daphne/INBOX
To delete a mail folder named projx for the user delilah:
mboxutil -d user/delilah/projx
To delete the default mailbox named INBOX and all mail folders for the user druscilla:
mboxutil -d user/druscilla/INBOX
To rename the mail folder memos to memos-april for the user desdemona:
mboxutil -r user/desdemona/memos user/desdemona/memos-april
To move the mail account for the user dimitria to a new partition:
mboxutil -r user/dimitria/INBOX user/dimitria/INBOX partition
where partition specifies the name of the new partition.
To move the mail folder named personal for the user dimitria to a new partition:
mboxutil -r user/dimitria/personal user/dimitria/personal partition
To search for orphaned accounts (orphaned accounts are mailboxes that do not have corresponding entries in LDAP) use the following command:
mboxutil -o |
Command output follows:
mboxutil: Start checking for orphaned mailboxes user/annie/INBOX user/oliver/INBOX mboxutil: Found 2 orphaned mailbox(es) mboxutil: Done checking for orphaned mailboxes
Use the following command to create a file listing orphaned mailboxes that can be turned into a script file that deletes the orphaned mailboxes (example filename is orphans.cmd):
mboxutil -o -w orphans.cmd |
The command output is as follows:
mboxutil: Start checking for orphaned mailboxes mboxutil: Found 2 orphaned mailbox(es) mboxutil: Done checking for orphaned mailboxes
Delete the orphan files with the following command:
mboxutil -d -f orphans.cmd |
The mailboxes in the message store are stored in a hash structure for fast searching. Consequently, to find the directory that contains a particular user’s mailbox, use the hashdir utility.
This utility identifies the directory that contains the message store for a particular account. This utility reports the relative path to the message store, such as d1/a7/. The path is relative to the directory level just before the one based on the user ID. The utility sends the path information to the standard output.
For example, to find the relative path to the mailbox for user crowe:
hashdir crowe
The readership utility reports on how many users other than the mailbox owner have read messages in a shared IMAP folder.
An owner of a IMAP folder may grant permission for others to read mail in the folder. A folder that others are allowed to access is called a shared folder. Administrators can use the readership utility to see how many users other than the owner are accessing a shared folder.
This utility scans all mailboxes and produces one line of output per shared folder, reporting the number of readers followed by a space and the name of the mailbox.
Each reader is a distinct authentication identity that has selected the shared folder within the past specified number of days. Users are not counted as reading their own personal mailboxes. Personal mailboxes are not reported unless there is at least one reader other than the folder’s owner.
For example, the following command counts as a reader any identity that has selected the shared IMAP folder within the last 15 days:
readership -d 15
Monitor quota usage and limits by using imqutoacheck, which generates a report listing defined quotas and limits, and provides information on quota usage. Quotas and usage figures are reported in kilobytes. This utility can also compare mailbox size with a user’s assigned quota. As an option, you can email a notification to users who have exceeded a set percentage of their assigned quota.
Certain functions have changed in imquotacheck. (In Messaging Server 6.x, imquotacheck utility has superseded the quotacheck utility.) In Messaging Server 5.x, when you used the quotacheck utility to retrieve a list of users, quotacheck searched the local mboxlist database. This function duplicated the search function in the mboxutil utility.
In Messaging Server 6.x, this duplicate function was removed from the imquotacheck utility. If you perform a user search with imquotacheck, the search is performed against the LDAP directory, not the local mboxlist database. To retrieve a list of users from the local mboxlist database, use the mboxutil utility.
To list the usage of all users whose quota exceeds the least threshold in the rule file:
imquotacheck
List quota information for a the domain siroe.com:
imquotacheck -d siroe.com
To send a notification to all users in accordance to the default rule file:
imquotacheck -n
To send a notification to all users in accordance to a specified rulefile, myrulefile, and to a specified mail template file, mytemplate.file (for more information, refer to imquotacheck in Sun Java System Messaging Server 6 2005Q4 Administration Reference):
imquotacheck -n -r myrulefile -t mytemplate.file
To list the usage of all users and (will ignore the rule file):
imquotacheck -i
To list per folder usages for users user1 (will ignore the rule file):
imquotacheck -u user1 -e
You can specify how often the system should monitor disk space and partition usage, and under what circumstances the system should send a warning. See Monitoring Disk Space for details.
The stored utility performs the following monitoring and maintenance tasks for the server:
Background and daily messaging tasks.
Deadlock detection and rollback of deadlocked database transactions.
Cleanup of temporary files on startup.
Implementation of aging policies.
Periodic monitoring of server state, disk space, service response times, and so on (see stored).
Issuing of alarms if necessary.
Database recovery as necessary (see Message Store Startup and Recovery
The stored utility automatically performs cleanup and expiration operations once a day at 11 PM. You can choose to run additional cleanup and expiration operations.
Table 18–12 lists some of the stored options. Some common usage examples follow the table. For detailed syntax and usage requirements, see the Sun Java System Messaging Server 6 2005Q4 Administration Reference.
Table 18–12 stored Options
Option |
Description |
---|---|
-d |
OBSOLETE. Use start-msg store to start up stored which will run as daemon, performing system checks and activating alarms, deadlock detection, and database repair. |
-t |
Checks the status of stored. The return code of this command indicates the status. |
-v |
Verbose output. |
-v -v |
More verbose output. |
To print the status, enter:
stored -t -v
If you want to change the time of the automatic cleanup and expiration operations, use the configutil utility as follows:
configutil -o store.expirestart -v 21
Occasionally, you might need to restart the stored utility; for example, if the mailbox list database becomes corrupted. To restart stored on UNIX, use the following commands at the command line:
msg_svr_base/sbin/stop-msg store msg_svr_base/sbin/start-msg store
If any server daemon crashes, you must stop all daemons and restart all daemons including stored.
When a message is sent to multiple recipients, that message is placed in each recipient’s mailbox. Some messaging systems store separate copies of the same message in each recipient’s mailbox. By contrast, the Sun Java System Messaging Server strives to retain a single copy of a message regardless of the number of mailboxes in which that message resides. It does this by creating hard links to that message in the mailboxes containing that message.
When other messaging systems are migrated to the Sun Java Messaging Server, these multiple message copies may be copied over with the migration process. With a large message store, this mean that a lot of messages are duplicated unnecessarily. In addition, multiple copies of the same message can be accumulated in normal server operation, for example, from IMAP append operations or other sources.
Messaging Server provides a new command called relinker that removes the excess message copies and replaces them with hard links to a single copy.
The relinking function can be run in the command or realtime mode. When the relinker command is run, it scans through the message store partitions, creates or updates the MD5 message digest repository (as hard links), deletes excess message files, and creates the necessary hard links.
The digest repository consists of hard links to the messages in the message store. It is stored in the directory hierarchy partition_path/=md5. This directory is parallel to the user mailbox hierarchy partition_path/=user (see Figure 18–1). Messages in the digest repository are uniquely identified by their MD5 digest. For example, if the digest for fredb/00/1.msg is 4F92E5673E091B43415FFFA05D2E47, then partition/=user/hashdir/hashdir/=fredb/00/1.msg is linked to partition/=md5/hashdir/hashdir/4F92E5673E091B43415FFFA05D2E47EA.msg. If another mailbox has this same message, for example, partition_path/=user/hashdir/hashdir/gregk/00/17.msg, that message will also be hard linked to partition_path/=md5/4F/92/4F92E5673E091B43415FFFA05D2E47EA.msg. This is shown in Figure 18–5.
For this message, the link count will be three. If both messages are deleted from the mailboxes of fredb and gregk, then the link count will be one and the message can be purged.
The relinker process can also be run in the realtime mode for similar functionality. See Using Relinker in the Realtime Mode for details.
relinker scans through a message store partition, creates or updates the MD5 message repository (as hard links) and deletes excess message files. After relinker scans a store partition, it outputs statistics on the number of unique messages and size of the partition before and after relinking. To run more quickly on an already hashed store, relinker only computes the digest of the messages not yet present in =md5. It also has an option to erase the entire digest repository (which doesn’t affect the user mailboxes).
The syntax for the command is as follows:
relinker [-p partitionname] [-d]
where partitionname specifies the partition to be processed (default: all partitions) and -d specifies that the digest repository be deleted. Sample output is shown below:
# relinker Processing partition: primary Scanning digest repository... Processing user directories.............................. --------------------------------------------------------- Partition statistics Before After --------------------------------------------------------- Total messages 4531898 4531898 Unique messages 4327531 3847029 Message digests in repository 0 3847029 Space used 99210Mb 90481Mb Space savings from single-copy 3911Mb 12640Mb --------------------------------------------------------- # relinker -d Processing partition: primary Purging digest repository... --------------------------------------------------------- Partition statistics Before After --------------------------------------------------------- Message digests in repository 3847029 0 --------------------------------------------------------- |
Relinker can take a long time to run, especially for the first time if there are no messages are in the repository. This is because it has to calculate the digest for every message (if the relinker criteria is configured include all messages—see Configuring Relinker for information on configuring relinker criteria.) For example, it could take six hours to process a 100 Gigabyte message store. However, if run-time relinking is enabled (see Using Relinker in the Realtime Mode
If the relinker command line mode is used exclusively, and not the run-time option, it is necessary to purge the digest repository (=md5), otherwise messages purged in the store (=user) will not become available disk space since they still have a link in the digest repository (they become orphaned). If you are just performing a one-time optimization of the store—for example after a migration—you can run relinker once, then delete the entire repository with relinker -d. For repeated purging—during migration—it is sufficient to just run the relinker command repeatedly, since each time it runs it also purges the expired or orphaned messages from the repository.
It is safe to run multiple instances of relinker in parallel with each processing a different partition (using the -p option). Messages are only relinked inside the same partition.
The relinker function can be enabled in the realtime mode by setting the configutil parameter local.store.relinker.enabled to yes. Using relinker in the realtime mode will compute the digest of every message delivered (or restored, IMAP appended, and so forth) which matches the configured relinker criteria (Configuring Relinker), then look in the repository to see if that digest is already present. If the digest is present, it creates a link to it in the destination mailbox instead of a creating new copy of the message. If there is no digest, it creates the message and adds a link to it in the repository afterwards.
stored scans the digest repositories of each partition and purges the messages having a link count of 1, or which don’t match the relinker criteria. The scan is done one directory at a time over a configurable time period. This is so that the I/O load is evenly distributed and doesn’t noticeably impact other server operations. By default the purge cycle is 24 hours, which means messages can still be present on the disk for up to 24 hours after they’ve been deleted from the store or have exceeded the configured maximum age. This task is enabled when the relinker realtime mode is enabled.
Table 18–13 shows the parameters used to set relinker criteria.
Table 18–13 relinker configutil Parameters
Parameter |
Description |
---|---|
Enables real-time relinking of messages in the append code and stored purge. The relinker command-line tool may be run even if this option is off, however since stored will not purge the repository, relinker -d must be used for this task. Turning this option on affects message delivery performance in exchange for the disk space savings. Default: no |
|
Maximum age in hours for messages to be kept in the repository, or considered by the relinker command-line. -1 means no age limit, that is, only purge orphaned messages from the repository. For relinker it means process existing messages regardless of age. Shorter values keep the repository smaller thus allow relinker or stored purge to run faster and reclaim disk space faster, while longer values allow duplicate message relinking over a longer period of time, for example, when users copy the same message to the store several days apart, or when running a migration over several days or weeks. Default: 24 |
|
Minimum size in kilobytes for messages to be considered by run-time or command-line relinker. Setting a non-zero value gives up the relinker benefits for smaller messages in exchange for a smaller repository. Default: 0 |
|
Approximate duration in hours of an entire stored purge cycle. The actual duration depends on the time it takes to scan each directory in the repository. Smaller values will use more I/O and larger values will not reclaim disk space as fast. 0 means run purge continuously without any pause between directories. -1 means don’t run purge in stored (then purge must be performed using the relinker -d command). Default: 24 |