CHAPTER 11 |
SIMS Periodic Maintenance Procedures |
This chapter describes tasks that are performed on a regular basis, either scheduled or as needed.
The IMTA runs a periodic job called post.sh every four hours by default. The post.sh program scans through all the channels currently defined in the configuration file and checks the corresponding queues for messages. Processing jobs are unconditionally submitted to run the master channel programs for any channels, with master programs so as to poll remote systems that cannot establish their own connections. Jobs are also submitted for channels that support master channel programs and have messages queued. After this is done post.sh then terminates. It will run again in another four hours.
The post.sh program is the shell script, /opt/SUNWmail/imta/lib/post.sh, which the cron daemon is normally scheduled to run every four hours. IMTA's suggested default behavior of running the periodic delivery job once every four hours is appropriate for most sites. Indeed, at busy sites, running the periodic delivery job too frequently tends to be counterproductive.
If a site has a special need to run post.sh more frequently, they can change the crontab. Note, however, that RFC 1123, Requirements for Internet Hosts requires that Internet mail wait at least 30 minutes before being retried. Do not run your channel to the Internet more frequently than every half hour.
Finally, IMTA normally performs some periodic cleanup tasks when post.sh runs. IMTA's defaults are tuned for the case where the periodic job only runs every couple of hours. If you will be running the periodic job more frequently, you should adjust IMTA's cleanup task frequency: the IMTA_SYNCH_CACHE_PERIOD and IMTA_VERSION_LIMIT_PERIOD IMTA tailor options should be set to integer values so that these tasks are still performed only every couple of hours or so. (Refer to SIMS Reference Manual for more details on these strings.)
The IMTA runs a second periodic job called return.sh which has as its primary job the returning of old, undeliverable messages which have sat around in the message queues for too long. The return.sh shell script is at
/opt/SUNWmail/imta/lib/return.sh. The cron daemon normally schedules it to run once a day at 30 minutes after midnight.The return.sh scans the channels listed in the configuration file, checking the values established with the notices keyword. The messages queued to each channel are then checked. A warning message is sent for every message whose age in days matches any of the values specified with the notices keyword on the associated channel. The default ages if no notices keyword is specified for 3, 6, 9, and 12 days. If the message is as old or older than the final notices value, the entire message is returned and the original message is deleted from the channel queue; no further attempts to deliver the message will be made. (See the SIMS Reference Manual for notices channel keyword.)
The text of the warning and failure notices issued by the message return system is contained in the pair of files return_warning.txt and return_failure.txt located in the /opt/SUNWmail/imta/locale/{C}/LC_MESSAGES directory. These files can be edited to provide different notification text if desired.
IMTA maintains a history of delivery attempts for each message, which sometimes will include information about why the delivery attempts failed. This information is included in returned messages if RETURN_DELIVERY_HISTORY is set to 1 in the IMTA Option file (this is the default). A value of 0 disables the inclusion of this information.
The imta_tailor file options can be used to control the periodicity of the various subfunctions of the message return system. The IMTA_RETURN_SYNCH_PERIOD option in /opt/SUNWmail/imta/imta_tailor controls queue synchronization; the IMTA_RETURN_PERIOD IMTA tailor file option controls the return of expired messages and the generation of warnings; and the IMTA_RETURN_SPLIT_PERIOD IMTA tailor file option controls splitting of the mail.log file. If any of these options is set to an integer value N, then the action associated with the tailor file option will only be performed every N times the message return job runs. The value of these options is taken to be 1 if the option is not specified in the IMTA tailor file.
If the IMTA return job is running once an hour, then the default will be to issue warning notices for messages which have remained undeliverable for 3, 6, or 9 hours. Message which have remained undeliverable for 12 or more hours are returned in their entirety to their sender and no further delivery attempts are made.
Note - When RETURN_UNITS=1, these defaults will result in mail being bounced much too soon; therefore, sites are strongly encouraged to use the notices channel keyword to set "bounce" ages in excess of 12 hours.
The return.sh also performs various IMTA periodic cleanup tasks tuned on the assumption that the return job will only be running once a day. When return.sh is run more frequently, various IMTA parameters should be adjusted accordingly. In particular, the IMTA_RETURN_SYNCH_PERIOD and IMTA_RETURN_SPLIT_PERIOD IMTA tailor file options should generally be adjusted so that these tasks are still performed only once a day. See the imta purge and imta cache-sync utilities in the SIMS Reference Guide for details on the cleanup programs used.
The Sun Message Store contains the content of the email system--messages and attachments. This section describes the maintenance procedures for the Sun Message Store. TABLE 11-2 summarizes the maintenance utilities provided for the Sun Message Store.
TABLE 11-2 Sun Message Store Maintenance Utilities Utility
Description
Supported Interface/Reference
Purge
Removes messages that are no longer referenced in user and group folders and returns space to the Sun Message Store file system.
Admin Console. See "Configuring Purge Options" on page 173 and "To Configure the Purge Schedule" on page 175.
Backup
Copies contents of folders to specified backup device. Can perform full or incremental backups of all folders or the folder of a specified user or group.
Command-line utility (imbackup). See "Message Store Backup and Restore" on page 237.
Restore
Restores contents of all folders or one specified user or group folder from the backup device to the Sun Message Store.
Command-line utility (imrestore). See "Message Store Backup and Restore" on page 237.
Message Store data check
Scans through the Sun Message Store and the user folders verifying links.
Command-line utility (imcheck).
Import mailbox
Imports existing user's mailbox in to the Sun Message Store.
Command line utility (imimportmbox).
Delete user
Deletes the following from the Sun Message Store: Inbox, private folders, and private shared folders of a specified user; and public shared folders.
Command-line utility (imdeluser). Refer to "Deleting the User" on page 246.
Reinitialize user quota
Reinitializes a user's quota file in the user admin directory /usr/<hash number>/<username>/Adm.
Command-line utility (iminitquota). Refer to "To Activate Message Store Quota Enforcement on an Installed System" on page 163.
BackUp/Restore Message Store Configuration
Back up or restore Message Store Configuration
Admin Console. See "Message Store Configuration Backup and Restore" on page 158.
Deleting old messages
Deleting messages of a specified age.
Command-line utility (imexpire). Refer to "Deleting Old Messages" on page 245.
Some of the maintenance utilities impose a session-locking mechanism to prevent certain other maintenance utilities from being run in parallel. TABLE 11-3 outlines which utilities cannot run in parallel.
imcheck should not be run with any other utilities.
TABLE 11-4 outlines the recommended maintenance schedule for the Sun Message Store.
When a message is delivered into the Sun Message Store, a reference is created in the Inbox of each of the message recipients. The reference points to the stored message. As each recipient reads, deletes, and removes (expunges) the message via their respective mail clients, the associated reference to the message is removed. When all references are removed, the message can be purged from the Sun Message Store.
The purge utility removes messages no longer referenced from any user or shared folder and returns the space to the Sun Message Store file system. The purge utility removes unreferenced messages starting with mail two days old and older. It does not remove unreferenced messages in today's and yesterday's mail. You must use the purge utility periodically; otherwise, the size of the Sun Message Store will grow unbounded.
For information on the other maintenance utilities with which purge can run concurrently, refer to TABLE 11-3.
You can invoke the purge utility by issuing the impurge command (see the SIMS Reference Manual and "To Configure the Purge Schedule" on page 175), or you can configure purge options and a purge schedule using the Admin Console.
SIMS allows you to backup and restore the message store with a great deal of flexibility:
|
|
You can backup and restore various parts of the message store. For example, you can backup all the mailboxes in the message store, the mailboxes of groups of users, the mailboxes specific users, or individual mail boxes. |
|
|
You can do concurrent backups of various parts of the message store. For example, if your message store is very large with many users, you can create groups of user mailboxes, and backup multiple groups on multiple devices concurrently. |
|
|
You can do full backups (backing up entire mailboxes) or incremental backups (backing up only the changes since the previous backup). |
We recommend using Solstice Backup, a file backup and restore product that is part of the Solstice System Management Suite, to backup the SIMS message store. If you do not have Solstice Backup, use the command line utilities imbackup and imrestore. See http://docs.sun.com for documentation on Solstice Backup.
Note that Solstice Backup and the Legato Networker product are identical. The instructions here are applicable to both products. Read the Solstice Backup or Legato Networker documentation set before attempting to backup the message store.
Note - Do not use ufsdump to backup the message store.
Note - Message store backup and restore cannot run at the same time as some other maintenance utilities. Refer to TABLE 11-3 for session locking information.
The SIMS message store is not structured in a hierarchical file system by user and mailbox. However, in order to backup and restore user mailboxes with Solstice Backup, the message store mailboxes need to be created up in a hierarchical file system by user and mailbox. This is accomplished by running mkbackupdir, which creates a dummy directory tree reflecting the desired hierarchical file system. Note that using the command line utility imbackup does not require using mkbackupdir.
Note - mkbackupdir only creates a directory tree showing the user/mailbox hierarchy. This directory tree does NOT contain the message store data.
This hierarchy is used to structure the message store into a hierarchical file system with each mailbox consisting of a file. The files in this hierarchy are empty, and only used to provide this hierarchical information to Solstice Backup. If the directory already exists, mkbackupdir synchronizes the directory structure with the current folder/mailbox hierarchy.
Each of the nodes in the directory represents a set of mailbox files which can be saved by Solstice Backup.
|
|
ADM_ROOT is the administration root directory defined in /etc/opt/SUNWmail/ims/ims.cnf. By default the directory is /var/opt/SUNWmail/ims/adm/. |
|
|
backup is the top-level node of the message store hierarchy. To back up all the message data at once on a single Solstice Backup Save Set, set the Client Save Set to this directory. Example: /var/opt/SUNWmail/ims/adm/backup |
|
|
groups are optional directories under which user mail folders can be grouped. For example, you may wish to group all the mailboxes of users with the login name starting with the letter a in a group called A-group, users with the login name starting with the letter b in a group called B-group, and so on. Groups can be used by both Solstice Backup and imbackup. |
| Groups can be created by manually modifying the default hierarchical directory created by mkbackupdir. Groups can also be created automatically by specifying a series of regular expressions which will divide users by name. These methods are described in "Full Message Store Backups Using Solstice Backup" on page 240. |
|
|
user directories contain the mailbox hierarchy for each mail user. Under each user directory is a file called INBOX which stores all incoming messages and a Mail directory containing the user's mailboxes and folders. (In this picture a mailbox is a file that holds messages, and a folder is a directory that hold mailboxes.) |
Backup Choices and Recommendations
You have the following choices for backing up the message store using Solstice Backup:
| 1. | You can choose to back up at any hierarchical level in the directory just like a file system. For example you can back up a mailbox, all of a single users mailboxes, the mailboxes of a group of users, or all the mailboxes in the entire message store. | |
| 2. | You can divide your message store space into user groups and backup multiple groups concurrently. | |
| 3. | You can perform a full backup (save all user mailboxes) or an incremental backup (save only mailboxes that have changed since a specified date). |
We recommend that you perform a full backup every week, and an incremental backup every other day.
 |
Full Message Store Backups Using Solstice Backup |
| 1. | If you decide to do concurrent backups or backup of groups of users as opposed to all the users in the message store, break your messages store into groups of users and create a hierarchical file system that reflects these groups. If you do not require groups of users, go to step 2. |
| Groups can be created manually or automatically. If you decide to create groups manually, skip to step 2. If you wish to create groups automatically, specify your groups in the /var/opt/SUNWmail/ims/backup.cnf file. The format for specifying groups is: |
groupname: <user names>
where, groupname is the name of the directory under which the user and mailbox directories will be stored, and <user names> is a UNIX regular expression specifying user directory names that will go under the groupname directory.
Example 1: You have the following message store users:
ally, amy, ann, bea, ben, bob, cam, cat, coral, 123
The following entries in backup.cnf will create three group directories called groupA, groupB, and groupC. Below these group* directories will be user directories corresponding to the user names beginning with a, b, and c (usernames are not case-sensitive):
| groupA: ^a.* groupB: ^b.* groupC: ^c.* |
| backup-groups.cnf: |
bridge.net: .*@bridge.net$
stream.com: .*@stream.com$
string.com: .*@string.com$
| 2. | Run mkbackupdir at the command line. |
| mkbackupdir creates a directory tree showing the user folder hierarchy. This directory tree WILL NOT contain the message store data--even after running Solstice Backup. The files in this hierarchy are only to provide hierarchical file information to Solstice Backup. If the hierarchical directory already exists, mkbackupdir synchronizes the directory with the current folder/mailbox hierarchy. |
| 3. | (Optional) Create user groups manually. |
| If you have created groups automatically (Step 1) or if you do not want to create groups, skip to step 4. If you want to create groups manually use the mkdir command to create group directories under /var/opt/SUNWmail/ims/adm/backup. Each directory should be a group name. |
| Example: |
| After your group directories are created, manually move the user directories under /var/opt/SUNWmail/ims/adm/backup/DEFAULT to the desired group directories. |
| 4. | Start Solstice Backup. |
| Do not use the Solstice Backup incremental backup feature. To do an incremental backup see "Incremental Message Store Backup Using Solstice Backup" on page 244. Specify concurrent backups as necessary. |
Note - The .nsr files are generated by the mkbackupdir command. It contains standard Networker directives and should never be edited.
| 5. | Automate this procedure. |
| The preceding steps describe how to run Solstice Backup manually. We recommend that you set up a cron job to run mkbackupdir every week and then use the Solstice Backup GUI to schedule backups after mkbackupdir is run. (Instead of setting up a cron job, you can also specify mkbackupdir as a Backup Command in the Solstice Backup program). |
| |
Full Message Store Backups Using imbackup |
| 1. | If you decide to do concurrent backups or backup of groups of users as opposed to all the users in the message store, break your messages store into groups of users. If you do not require groups of users, go to step 2. |
| Groups can be created manually or automatically. If you wish to create groups automatically, specify your groups in the /var/opt/SUNWmail/ims/backup.cnf file. The format for specifying groups is: |
groupname: <user names>
where, groupname is the name of the directory under which the user and mailbox directories will be stored, and <user names> is a UNIX regular expression specifying user directory names that will go under the groupname directory.
# imbackup -g groupA
| backup-groups.cnf: |
| groupA: ^a.* groupB: ^b.* groupC: ^c.* |
| # imbackup -g stream.com |
| backup-groups.cnf: |
| bridge.net: .*@bridge.net$ stream.com: .*@stream.com$ string.com: .*@string.com$ |
| 2. | Run imbackup. |
| |
Incremental Message Store Backup Using Solstice Backup |
A full backup saves the user's current mailbox. An incremental backup saves changes to the mailbox from specified date.
| 1. | Set up a cron job or specify mkbackupdir -d as a Backup Command in the Solstice Backup program. |
| The proper format is: |
| mkbackupdir -d <date (yyyymmdd) since last full backup> |
| This will create an incremental backup directory hierarchy for only mailboxes that have been modified since a specified date. Solstice Backup will only backs up those mailboxes. For example, if you run a full backup using mkbackupdir on 1 January 2000, you can do one week incremental backup by running: |
| mkbackupdir -d 20000108 |
| |
Restoring the Message Store |
Utility: imrestore
You cannot use imrestore to restore mailboxes backed up with Solstice Backup, and you cannot use Solstice Restore to restore mailboxes backed up with imbackup. For more detailed information on how to use Solstice Restore and imrestore, refer to the Solstice Backup documents (http://docs.sun.com) or the imrestore man page.
If you use Solstice Backup to recover a message store, you will receive the message "File already exists. Do you want to overwrite, skip, backup, or rename?" Choose overwrite. This message appears because the backup tree is just the directory hierarchy, that is, it consists of empty files and stays that way permanently.
Note - If you use the Solstice recover command, then you can use the -A and -iy arguments to suppress this message.
Utility: imcheck
The folder check utility scans through the Sun Message Store and the user folders verifying links. That is, it verifies that all the messages in the folders are accessible. In addition to running the folder check utility at regular intervals for maintenance, you can also run this utility after a system failure to ensure that all message deliveries were made while the system was in a questionable state. If the utility determines that messages are not in user folders and hence were not delivered, it will redeliver the messages.
You can invoke the folder check utility by issuing the imcheck command at a command-line interface. For information on the imcheck command, refer to the SIMS Reference Manual.
Utility: imimportmbox
The import mailbox utility automatically imports an existing user's Inbox folder and all messages from /var/mail to the Sun Message Store. You must manually import a /var/mail user's private folders.
You can invoke the import mailbox utility by issuing the imimportmbox command at a command-line interface. For information on the imimportmbox command and importing /var/mail users refer to "Migrating /var/mail Mailboxes" on page 328 and the SIMS Reference Manual.
For information on the other maintenance utilities with which import mailbox can run concurrently, refer to TABLE 11-3.
Utility: imexpire
The imexpire command allows administrators to mark as permanently deleted or "expired" any user messages older than a specified date or older than a number of specified days. The deleted messages are expunged from the user mailbox when the user connects or disconnects from the server. The actual data is removed from the message store when impurge is run. Refer to the SIMS Reference Manual for further details.
| |
To Disable Automatic Quota Synchronization |
| If you want to change or disable the automatic quota synchronization (for example you need to do message store maintenance and you want the synchronization to occur), edit the following crontab entry: |
| 10,30,50 * * * * [ -x /opt/SUNWmail/imta/sbin/imta ] && /opt/SUNWmail/imta/sbin/imta dirsync -C /opt/SUNWmail/lib/libquota.so.1:update_quota:5 |
Utility: imadmin-delete-user
To delete a user's entry and mailbox, use imadmin-delete-user. (See "To Delete a User or Group Entry from the Directory" on page 41.) To remove only the user mailbox and not the user entry in the directory, use the imdeluser command.
Note - You will need the Sun Directory Services 3.1 Administration Guide for more details on some of these tasks.
When entries are deleted, they still occupy space in the directory database. Regenerate the attribute index periodically to retrieve this space.
Note - Regenerating a large database can take a considerable amount of time (2-5 hours for 50,000 entries depending upon the system configuration). During regeneration users cannot log into their mailboxes, although mail is still delivered to mailboxes.
You can regenerate the attribute index by using the Sun Directory Services Admin Console. You need to do this for each data store individually. Refer to the Sun Directory Services 3.1 Administration Guide for details. Alternatively, you can use the idxgen command to regenerate indexes. Refer to the dsidxgen man page for more information.
Note - If you add any additional index definitions to dsserv.conf, you must regenerate the indexes before running dsservd again. This is true even if you have not yet added data matching the new indexes to the Directory.
The directory data base is stored in a binary format called LDBM. Back up the LDBM database by using the LDBM Data Store backup feature in the Sun Directory Services Admin Console (see the Sun Directory Services 3.1 Administration Guide) or simply backup LDBM database files using the tar command. The location of the LDBM database files is defined by the directory parameter in /etc/opt/SUNWconn/ldap/current/dsserv.conf. Before using the tar command on these files, put the directory in the read-only mode so LDBM database files don't get written to during backup.
Because the LDBM database is stored in binary files, they cannot be properly restored if the files themselves are corrupted. For this reason you may also choose to backup the directory database in the text-based LDIF format. You can make LDIF backups of your LDBM directory database by using the command ldbmcat. To restore the LDIF database files back to LDBM, use the command ldif2ldbm. Refer to the man pages for more information.
Use the Backup Configuration feature in the Sun Directory Services Admin Console (see the Sun Directory Services 3.1 Administration Guide) to backup the directory service configuration file. This is particularly important if you change the Sun Directory Services configuration from the default. The Sun Directory Services Admin Console also provides a configuration restore feature to restore any of your backed up directory service configurations.