20.11 Performing Message Store Maintenance Procedures

This section provides information about the utilities you use to perform maintenance and recovery tasks for the message store. You should always read your postmaster mail for warnings and alerts that the server might send. You should also monitor the log files for information about how the server is performing. For more information about log files, see Chapter 25, Managing Logging

This section contains the following:

• 20.11.1 Adding More Physical Disks to the Message Store

• 20.11.2 To Manage Mailboxes

• 20.11.3 Maximum Mailbox Size

• 20.11.4 To Monitor Quota Limits

• 20.11.5 To Monitor Disk Space

• 20.11.6 The stored Daemon

• 20.11.7 Reducing Message Store Size Due to Duplicate Storage of Identical Messages

20.11.1 Adding More Physical Disks to the Message Store

The Messaging Server message store contains the user mailboxes for a particular Messaging Server instance. The size of the message store increases as the number of mailboxes, folders, and log files increase.

As you add more users to your system, your disk storage requirements increase. Depending on the number of users your server supports, the message store might require one physical disk or multiple physical disks. Messaging Server enables you an add more stores as needed. One approach to adding more stores is by using storage appliances. See Using NetApp Filers with Sun Java System Messaging Server Message Store for information on how to configure Network Appliance storage appliances with Messaging Server.

20.11.2 To Manage Mailboxes

This section describes the following utilities for managing and monitoring mailboxes: mboxutil, hashdir, readership.

20.11.2.1 The mboxutil Utility

Use the mboxutil command to perform typical maintenance tasks on mailboxes. mboxutil tasks include the following:

• List mailboxes

• List and remove orphaned and inactive mailboxes

• Create mailboxes

• Rename mailboxes

• Move mailboxes from one partition to another

• Expunge mailboxes

• Restore expunged messages that have not been purged

• List personal mailbox subscriptions and unsubscribed mailboxes that no longer exist

• You can also use the mboxutil command to view information about quotas. For more information, see 20.11.4 To Monitor Quota Limits.

Note that you should not kill the mboxutil process in the middle of execution. If it is killed with SIGKILL (kill -9), it may potentially require that every server get restarted and a recovery be done.

See mboxutil in Sun Java System Messaging Server 6.3 Administration Reference for detailed syntax and usage requirements.

Examples

To list all mailboxes for all users:

mboxutil -l

To list all mailboxes and also include path and ACL information:

mboxutil -l -x

To create the default mailbox named INBOX for the user daphne:

mboxutil -c user/daphne/INBOX

To delete a mail folder named projx for the user delilah:

mboxutil -d user/delilah/projx

To delete the default mailbox named INBOX and all mail folders for the user druscilla:

mboxutil -d user/druscilla/INBOX

To rename the mail folder memos to memos-april for the user desdemona:

mboxutil -r user/desdemona/memos user/desdemona/memos-april

To move the mail account for the user dimitria to a new partition:

mboxutil -r user/dimitria/INBOX user/dimitria/INBOX partition

where partition specifies the name of the new partition.

To move the mail folder named personal for the user dimitria to a new partition:

mboxutil -r user/dimitria/personal user/dimitria/personal partition

20.11.2.2 To Remove Orphan Accounts

To search for orphaned accounts (orphaned accounts are mailboxes that do not have corresponding entries in LDAP) use the following command:

mboxutil -o

Command output follows:

  mboxutil: Start checking for orphaned mailboxes
  user/annie/INBOX
  user/oliver/INBOX
  mboxutil: Found 2 orphaned mailbox(es)
  mboxutil: Done checking for orphaned mailboxes

Use the following command to create a file listing orphaned mailboxes that can be turned into a script file that deletes the orphaned mailboxes (example filename is orphans.cmd):

mboxutil -o -w orphans.cmd

The command output is as follows:

  mboxutil: Start checking for orphaned mailboxes
  mboxutil: Found 2 orphaned mailbox(es)
  mboxutil: Done checking for orphaned mailboxes

Delete the orphan files with the following command:

mboxutil -d -f orphans.cmd

20.11.2.3 The hashdir Utility

The mailboxes in the message store are stored in a hash structure for fast searching. Consequently, to find the directory that contains a particular user’s mailbox, use the hashdir utility.

This utility identifies the directory that contains the message store for a particular account. This utility reports the relative path to the message store, such as d1/a7/. The path is relative to the directory level just before the one based on the user ID. The utility sends the path information to the standard output.

For example, to find the relative path to the mailbox for user crowe:

hashdir crowe

20.11.2.4 The readership Utility

The readership utility reports on how many users other than the mailbox owner have read messages in a shared IMAP folder.

An owner of a IMAP folder may grant permission for others to read mail in the folder. A folder that others are allowed to access is called a shared folder. Administrators can use the readership utility to see how many users other than the owner are accessing a shared folder.

This utility scans all mailboxes and produces one line of output per shared folder, reporting the number of readers followed by a space and the name of the mailbox.

Each reader is a distinct authentication identity that has selected the shared folder within the past specified number of days. Users are not counted as reading their own personal mailboxes. Personal mailboxes are not reported unless there is at least one reader other than the folder’s owner.

For example, the following command counts as a reader any identity that has selected the shared IMAP folder within the last 15 days:

readership -d 15

20.11.3 Maximum Mailbox Size

A mailbox has a maximum size of about one million messages. More than this will cause messages to stop being delivered to the user and could cause message store performance problems. See 20.14.4.8 User Mail Not Delivered Due to Mailbox Overflow for details.

20.11.4 To Monitor Quota Limits

Monitor quota usage and limits by using imquotacheck, which generates a report listing defined quotas and limits, and provides information on quota usage. Quotas and usage figures are reported in kilobytes. This utility can also compare mailbox size with a user’s assigned quota. As an option, you can email a notification to users who have exceeded a set percentage of their assigned quota.

Certain functions have changed in imquotacheck. (In Messaging Server 6.x, imquotacheck utility has superseded the quotacheck utility.) In Messaging Server 5.x, when you used the quotacheck utility to retrieve a list of users, quotacheck searched the local mboxlist database. This function duplicated the search function in the mboxutil utility.

In Messaging Server 6.x, this duplicate function was removed from the imquotacheck utility. If you perform a user search with imquotacheck, the search is performed against the LDAP directory, not the local mboxlist database. To retrieve a list of users from the local mboxlist database, use the mboxutil utility.

To list the usage of all users whose quota exceeds the least threshold in the rule file:

imquotacheck

List quota information for a the domain siroe.com:

imquotacheck -d siroe.com

To send a notification to all users in accordance to the default rule file:

imquotacheck -n

To send a notification to all users in accordance to a specified rulefile, myrulefile, and to a specified mail template file, mytemplate.file (for more information, refer to imquotacheck in Sun Java System Messaging Server 6.3 Administration Reference):

imquotacheck -n -r myrulefile -t mytemplate.file

To list per folder usages for one user user1 (will ignore the rule file):

imquotacheck -u user1 -e

20.11.5 To Monitor Disk Space

You can specify how often the system should monitor disk space and partition usage, and under what circumstances the system should send a warning. See 27.3.2 Monitoring Disk Space for details.

20.11.6 The stored Daemon

The stored daemon performs the following maintenance tasks for the message store:

• Performs checkpoint database transactions. .

• Deadlock detection and rollback of deadlocked database transactions.

• Cleanup of temporary files and lock files on startup.

• Creates a database snapshot archive

• Database recovery as necessary (see 20.14.2 Message Store Startup and Recovery

If any server daemon crashes, you must stop all daemons and restart all daemons including stored.

20.11.7 Reducing Message Store Size Due to Duplicate Storage of Identical Messages

When a message is sent to multiple recipients, that message is placed in each recipient’s mailbox. Some messaging systems store separate copies of the same message in each recipient’s mailbox. By contrast, the Sun Java System Messaging Server strives to retain a single copy of a message regardless of the number of mailboxes in which that message resides. It does this by creating hard links to that message in the mailboxes containing that message.

When other messaging systems are migrated to the Sun Java Messaging Server, these multiple message copies may be copied over with the migration process. With a large message store, this means that a lot of messages are duplicated unnecessarily. In addition, multiple copies of the same message can be accumulated in normal server operation, for example, from IMAP append operations or other sources.

Messaging Server provides a new command called relinker that removes the excess message copies and replaces them with hard links to a single copy.

20.11.7.1 Relinker Theory of Operations

The relinking function can be run in the command or realtime mode. When the relinker command is run, it scans through the message store partitions, creates or updates the MD5 message digest repository (as hard links), deletes excess message files, and creates the necessary hard links.

The digest repository consists of hard links to the messages in the message store. It is stored in the directory hierarchy partition_path/=md5. This directory is parallel to the user mailbox hierarchy partition_path/=user (see Figure 20–1). Messages in the digest repository are uniquely identified by their MD5 digest. For example, if the digest for fredb/00/1.msg is 4F92E5673E091B43415FFFA05D2E47, then partition/=user/hashdir/hashdir/=fredb/00/1.msg is linked to partition/=md5/hashdir/hashdir/4F92E5673E091B43415FFFA05D2E47EA.msg. If another mailbox has this same message, for example, partition_path/=user/hashdir/hashdir/gregk/00/17.msg, that message will also be hard linked to partition_path/=md5/4F/92/4F92E5673E091B43415FFFA05D2E47EA.msg. This is shown in Figure 20–4.

Figure 20–4 Message Store Digest Repository

For this message, the link count will be three. If both messages are deleted from the mailboxes of fredb and gregk, then the link count will be one and the message can be purged.

The relinker process can also be run in the realtime mode for similar functionality. See 20.11.7.3 Using Relinker in the Realtime Mode for details.

20.11.7.2 Using Relinker in the Command Line Mode

relinker scans through a message store partition, creates or updates the MD5 message repository (as hard links) and deletes excess message files. After relinker scans a store partition, it outputs statistics on the number of unique messages and size of the partition before and after relinking. To run more quickly on an already hashed store, relinker only computes the digest of the messages not yet present in =md5. It also has an option to erase the entire digest repository (which doesn’t affect the user mailboxes).

The syntax for the command is as follows:

relinker [-P. partitionname] [-d]

where partitionname specifies the partition to be processed (default: all partitions) and -d specifies that the digest repository be deleted. Sample output is shown below:

# relinker

Processing partition: primary
Scanning digest repository...
Processing user directories..............................
---------------------------------------------------------
Partition statistics           Before            After 
---------------------------------------------------------
Total messages                 4531898         4531898
Unique messages                4327531         3847029
Message digests in repository        0         3847029
Space used                       99210Mb         90481Mb
Space savings from single-copy    3911Mb         12640Mb
---------------------------------------------------------


# relinker -d 
Processing partition: primary
Purging digest repository...
---------------------------------------------------------
Partition statistics                 Before         After
---------------------------------------------------------
Message digests in repository       3847029             0
---------------------------------------------------------

Relinker can take a long time to run, especially for the first time if there are no messages are in the repository. This is because it has to calculate the digest for every message (if the relinker criteria is configured to include all messages—see 20.11.7.4 Configuring Relinker for information on configuring relinker criteria.) For example, it could take six hours to process a 100 Gigabyte message store. However, if run-time relinking is enabled see 20.11.7.3 Using Relinker in the Realtime Mode.

If the relinker command line mode is used exclusively, and not the run-time option, it is necessary to purge the digest repository (=md5), otherwise messages purged in the store (=user) will not become available disk space since they still have a link in the digest repository (they become orphaned). If you are just performing a one-time optimization of the store—for example after a migration—you can run relinker once, then delete the entire repository with relinker -d. For repeated purging during migration, it is sufficient to just run the relinker command repeatedly, since each time it runs it also purges the expired or orphaned messages from the repository.

It is safe to run multiple instances of relinker in parallel with each processing a different partition (using the -p option). Messages are only relinked inside the same partition.

20.11.7.3 Using Relinker in the Realtime Mode

The relinker function can be enabled in the realtime mode by setting the configutil parameter local.store.relinker.enabled to yes. Using relinker in the realtime mode will compute the digest of every message delivered (or restored, IMAP appended, and so forth) which matches the configured relinker criteria (20.11.7.4 Configuring Relinker), then look in the repository to see if that digest is already present. If the digest is present, it creates a link to it in the destination mailbox instead of creating a new copy of the message. If there is no digest, it creates the message and adds a link to it in the repository afterwards.

stored scans the digest repositories of each partition and purges the messages having a link count of 1, or which don’t match the relinker criteria. The scan is done one directory at a time over a configurable time period. This is so that the I/O load is evenly distributed and doesn’t noticeably impact other server operations. By default the purge cycle is 24 hours, which means messages can still be present on the disk for up to 24 hours after they’ve been deleted from the store or have exceeded the configured maximum age. This task is enabled when the relinker realtime mode is enabled.

20.11.7.4 Configuring Relinker

Table 20–11 shows the parameters used to set relinker criteria.