Sun Java System Messaging Server 6.3 Administration Guide

20.9.2 To Deploy the Automatic Message Removal Feature

Automatic message removal requires three steps:

  1. Define automatic message removal policy: Which messages will be automatically removed? What users, folders, domains and partitions will have messages automatically removed? What size, message age, headers will define the removal criteria. Define the scope of messages to be removed. See 20.9.2.1 To Define Automatic Message Removal Policy

  2. Specify theimexpire rules to implement this policy. See 20.9.2.2 To Set Rules Implementing Automatic Message Removal Policy

  3. Specify the imexpire scheduling. See 20.9.2.3 To Schedule Automatic Message Removal and Logging Level

20.9.2.1 To Define Automatic Message Removal Policy

Define your automatic message removal policy by specifying the criteria for removal. imexpire allows for removal using the following criteria:

Age of Message. Automatically remove messages older than X days. Attribute: messagedays.

Message Count. Automatically remove messages in a folder exceeding X messages. Attribute: messagecount.

Age of Oversized Message. Automatically remove messages that exceed X bytes after Y days grace period. Attributes: messagesize and messagesizedays.

Seen and Deleted Message Flag. Automatically remove messages with the Seen or Deleted flag set. These criteria can be set to “and” or “or.” If set to or, the message’s Seen/Delete flag will cause automatic deletion regardless of other criteria. If set to and, the message’s Seen/Delete flag must be set along with passing all other specified criteria. Attributes: seen and deleted.

Header Field of Message. Allows you specify a header and string as criteria for removing a message. For example, removing all messages with the header “Subject: Work from Home!”

Folder of Messages. Allows you to specify the folder on which to remove messages. Attribute: folderpattern. Note that this attribute only uses the modified UTF-7 character set.


Note –

imexpire does not allow you to delete or preserve messages based on how long it has been since that message was read. For example, you cannot specify that messages that have not been read for 200 days will be removed.


Examples of Automatic Message Removal Policy

Example 1: Remove all messages 365 days old in a folder exceeding 1,000 messages.

Example 2: Remove messages in domain siroe.com that are older than 180 days.

Example 3: Remove all messages that have been marked as deleted.

Example 4: Remove messages in sesta.com that have been marked as seen, are older than 30 days, are larger than 100 kilobytes, from folders exceeding 1,000 messages, with the header X-spam.

20.9.2.2 To Set Rules Implementing Automatic Message Removal Policy

To implement the automatic message removal policy defined in the previous section, you must set the imexpire rules. Rules are set by putting them into a store.expirerule file. An example of two global store.expirerule rules is shown below:


Rule1.regexp: 1
Rule1.folderpattern: user/.*/trash
Rule1.messagedays: 2
Rule2:regexp: 1
Rule2.folderpattern: user/.*
Rule2.messagedays: 14

            

In this example, Rule 1 specifies that all messages in the trash folder will be remove after two days. Rule 2 specifies that all messages in the message store will be removed after 14 days.

This section consists of the following subsections:

Expiration Rules Guidelines

This section sets the guidelines for the store.expirerule file rules.


Note –

In earlier Messaging Server releases, expiration rules could be set with configutil parameters store.expirerule.attribute (see configutil Parameters in Sun Java System Messaging Server 6.3 Administration Reference.) This is still true, but expire rules using header constraints (example: expiring a message with a specific subject line) are not supported. Also, regular expressions in the expire rules created with configutil need to be POSIX compliant rules. If you want to use UNIX compliant regular expressions you will have to use the store.expire file. In addition, using both configutil options and the global store.expirerule configuration file is not supported. If the configuration file is present, configutil options are not used. In any case, it is best to use store.expirerule to specify all expiration rules.


Table 20–8 imexpire Attributes

Attribute 

Description (Attribute Value) 

action

Specifies an action to perform on the messages caught by the expire rules. The possible values are: 

discard discards the message. This is the default.

report action prints the mailbox name, uid-validity and uid to stdout.

archive archives the message with Sun Compliance and Content Management System and then discards the message.

fileinto: folder action folders the message into the specific folder. The shared folder prefix can be used to folder messages to folders owned by another user.

exclusive

Specifies whether or not this is an exclusive rule. If specified as exclusive, only this rule applies to the specified mailbox(s) and all other rules are ignored. If more than one exclusive rule exists, the last exclusive rule loaded will be used. For example, if a global and a local exclusive rule are specified, the local rule will be used. If there is more than one global exclusive rule, the last global rule listed by configutil is used. (1/0)

folderpattern

Specifies the folders affected by this rule. The format must start with a user/, which represents the directory store_root/partition/*/ See Table 20–9. (POSIX regular expression)

messagecount

Maximum number of messages in a folder. Oldest messages are expunged as additional messages are delivered. (integer) 

foldersize

Maximum size of folder before the oldest messages are expunged when additional messages are delivered. (integer in bytes) 

messagedays

Number of days in the message store before being expunged. (integer) 

messagesize

Maximum size of message in bytes before it is marked to be expunged. (integer) 

messagesizedays

Grace period. Days an over-sized message should remain in a folder. (integer) 

messageheader.header

Specifies a header field and string that marks a message for removal. Values are not case-sensitive and regular expressions are not recognized.Example: Rule1.messageheader.Subject: Get Rich Now!

For the header Expires and Expiry-Date, imexpire will remove the message if the date value specified with these header fields is older than the messagedays attribute. If multiple expiration header fields are specified, the earliest expiration date will be used. (string).

regexp

Enable UNIX regular expressions in rules creation. (1 or 0). If not specified, IMAP expressions will be used. 

savedays

Number of days the messages are saved in a folder until they are expunged. 

seen

seen is a message status flag set by the system when the user opens a message. If the attribute seen is set to and, then the message must be seen and other criteria must be met before the rule is fulfilled. If the attribute seen is set to or, then the message only needs to be seen or another criteria be met before the rule is fulfilled. (and/or).

sieve

A Sieve rule specifying message selection criteria. Example: Rule17.sieve: header :contains “Subject” “Vigara”

deleted

deleted is a message status flag set by the system when the user deletes a message. If the attribute deleted is set to and, then the message must be deleted and another criteria must be met before the rule is fulfilled. If the attribute deleted is set to or, then the message only needs to be seen or another criteria be met before the rule is fulfilled. (and/or)

Localized Mailbox Names

The IMAP protocol specifies that mailbox names use modified UTF-7 encoding. Messaging Server supports localized character sets on external interfaces so that mailbox names can be localized. Internally, however, the system converts the localized name to UTF-7. Thus, a folder that has a localize mailbox name on a client will have a corresponding mailbox file name in UTF-7. (Note that IMAP error messages will output mailbox names in UTF-7 and not the localized character set.)

In general, most message store utilities that require mailbox names expect the names in the localized character set, although they may have an option flag that allows a different character set to be used. These utilities include reconstruct, mboxutil, imsbackup, imsrestore, and hashdir. However, imexpire requires that the mailbox name, specified as the attribute folderpattern,be in UTF-7. Using a localized name will not work.

To obtain the appropriate folderpattern for imexpire it may be necessary to convert a localized mailbox name to the modified UTF-7 equivalent. This can be done using the mboxutil -E command as follows:

mboxutil showing localized filename and modified UTF-7
filename.

The first mboxutil shows the localized filename. The second mboxutil shows the filename in modified UTF-7. It is also possible to use the IMAP list command:

IMAP list command

To convert the local charset to modified UTF-7 encoding, use the mboxutil command with the -E option:

mboxutil with —E option.

Note that mboxutil -E can be used for any command that requires the use of a UTF-7 mailbox name including imexpire.

Setting imexpire Rules Textually

Automatic message removal rules are set by specifying rules in a store.expirerule file. The store.expirerule file contains one expire criteria per line. An expire criteria of the global rule configuration file (msg-svr-base/data/store/store.expirerule) has the following format:

rule_name.attribute: value

An expiration rule for a user or mailbox rule configuration file has this format:

attribute: value

Example 20–4 shows a set of global expiration rules in msg-svr-base/config/store.expirerule.

Rule 1 sets the global expiration policy (that is, policy that applies to all messages), as follows:

Rule 2 sets the automatic message removal policy for users at the hosted domain siroe.com. It limits mailbox sizes to 1 megabyte, removes messages that have been deleted, and removes messages older than 14 days.

Rule 3 sets the automatic message removal policy for messages in the inbox folder of user f.dostoevski. It removes messages with a subject line having the expression “On-line Casino.”


Example 20–4 Example imexpire Rules


Rule1.regexp: 1
Rule1.folderpattern: user/.*
Rule1.messagesize: 100000
Rule1.messagesizedays: 3
Rule1.deleted: or
Rule1.Subject: Vigara Now!
Rule1.Subject: XXX Porn!
Rule1.messagecount: 1000
Rule1.messagedays: 365
Rule2.regexp: 1
Rule2.folderpattern: user/.*@siroe.com/.*
Rule2.exclusive: 1
Rule2.deleted: or
Rule2.messagedays: 14
Rule2.messagecount: 1000
Rule3.folderpattern: user/f.dostoevski/inbox
Rule3.Subject: *On-line Casino*
                  

Setting imexpire Folder Patterns

Folder patterns can be specified using POSIX regular expressions by setting the imexpire attribute regex to 1. If not specified, IMAP expressions will be used. The format must start with a user/ followed by a pattern. Table 20–9 shows the folder pattern for various folders.)

Table 20–9 imexpire Folder Patterns Using Regular Expressions

Folder Pattern 

Scope 

user/userid/.*

Apply rule to all messages in all folders of userid.

user/userid/Sent

Apply rule to messages of userid in folder Sent:

user/.*

Apply rule to entire message store. 

user/.*/trash

Apply rule to the trash folder of all users.

user/.*@siroe.com/.*

Apply rule to folders in hosted domain siroe.com: 

user/[^@]*/.*

Apply rule to folders in default domain. 

20.9.2.3 To Schedule Automatic Message Removal and Logging Level

Automatic message removal is activated by the imsched scheduling daemon. By default, imsched invokes imexpire at 23:00 every day and messages are both expunged and purged. This schedule can be customized by setting the configutil parameter local.schedule.expire and store.cleanupage described in Table 20–10.

Expire and purge can take a long time to complete on a large message store, so you may wish to experiment and decide how often to run these processes. For example, if an expire/purge cycle takes 10 hours, you may not want the default schedule of running expire and purge once a day. Schedule expire and purge using the imexpire command and the automatic task scheduling parameter (see4.6 To Schedule Automatic Tasks). For example:


configutil -o local.schedule.expire -v "0 1 * * 6 /opt/SUNWmsgsr/sbin/imexpire -e"
configutil -o local.schedule.mspurge -v "0 23 * * * /opt/SUNWmsgsr/sbin/imexpire -c"

In this example, messages are expired at 1AM Saturdays and purged every night at 11PM. If no purge schedule is set, , imexpire will perform purge after an expire.

Table 20–10 Expire and Purge configutil Log and Scheduling Parameters

Parameter  

Description 

local.schedule.expire

Interval for running imexpire. Uses UNIX crontab format: minute hour day-of-month month-of-year day-of-week

The values are separated by a space or tab and can be 0-59, 0-23, 1-31, 1-12 and 0-6 (with 0=Sunday) respectively. Each time field can be either an asterisk (meaning all legal values), a list of comma-separated values, or a range of two values separated by a hyphen. Note that days can be specified by both day of the month and day of the week, however, it is not typical to use them both since the number of such occurrences are very small. If they are both specified, then both will be required. For example, setting the 17th day of the month and Tuesday will require both values to be true. 

Note that you can also use the -e and -c flags with imexpire to and expire only or purge only respectively. See imexpire in Sun Java System Messaging Server 6.3 Administration Reference.

Interval Examples:

1) Run imexpire at 12:30am, 8:30am, and 4:30pm:

30 0,8,16 * * * /opt/SUNWmsgsr/sbin/imexpire

2) Run imexpire at weekday morning at 3:15 am:

15 3 * * 1-5 /opt/SUNWmsgsr/sbin/imexpire

3) Run imexpire only on Mondays:

0 0 * * 1 /opt/SUNWmsgsr/sbin/imexpire

Default:  

0 23 * * * /opt/SUNWmsgsr/sbin/imexpire

To disable: set local.schedule.expire.enable to NO.

store.cleanupage

Age (in hours) of expired or expunged message before purge will permanently remove it.

Default: None 

local.store.expire.loglevel

Specify a log level: 

1 = log summary for the entire expire session.  

2 = log one message per mailbox expired.  

3 = log one message per message expired. 

Default: 1 

Setting imexpire Logging Levels

imexpire will log a summary to the default log file upon completion. If expire is invoked from the command line, the -v (verbose) and -d (debug) option can be used to instruct imexpire to log detail status/debug messages to stderr. If imexpire is invoked by imsched, the configutil parameter local.store.expire.loglevel can be set to 1,2 or 3 for different levels of logging. Loglevel 1 is the default, it will log a summary for the entire expire session. Loglevel 2 will log one message per mailbox expired. Loglevel 3 will log one message per message expired.

Excluding Specified Users from Automatic Message Remove

Exclude specified users from the expire rules by adding their user ID, one per line, in a file called expire_exclude_list in msg-svr-base/config/. Or, configure a dummy exclusive expire rule under the user's mailbox.