This chapter describes message expiration concepts. See "Configuring Message Expiration (Tasks)" for information on message removal tasks.
Message expiration automatically removes messages from the message store based on criteria that you set. For example, you can remove old messages, overly large messages, seen or deleted messages, messages with specific Subject: lines, messages of a certain type, and so on.
Note:
Oracle Communications Messaging Server removes messages without giving a warning, so it is important to inform users about message expiration policies. Unexpected message removal can be a source of consternation for users and administrators.Message expiration is performed by the imexpire utility, which performs a specific action to the expired messages. (See "Deleting, Expunging, Purging, and Cleaning Up Messages" for details on the message removal process.) You can launch the imexpire utility from the command-line or schedule it to launch through the imsched daemon. You specify a set of expiration rules in the store.expirerule file. You can have multiple rules files, each located the directory that pertains to the scope of the rules. That is, rules that apply globally to the entire message store are put in one directory, rules that apply to a partition in another, rules that apply to users in yet another, and so on.
Note:
Although global expiration rules can be specified with store.expire.attribute configutil options, use store.expirerule files to specify these rules. If too many rules are created by using configutil, performance problems can result.imexpire loads all of the expire rules at start up. By default, imexpire creates one thread per partition. Each thread goes through the list of user folders under its assigned partition and loads the local expire rule files as it goes. The expire function checks each folder against the expire rules applicable to this folder and expunges messages as needed.
It is also possible to exclude specified users from the expire rules by adding their user ID, one per line, in a file called expire_exclude_list in the MessagingServer_home/config/ directory.
Message expiration requires the following three steps:
Define message expiration policy: Which messages will be expired? What users, folders, domains, and partitions will have messages expired? What size, message age, and headers will define the removal criteria? Define the scope of messages to be removed. See "To Define Message Expiration Policy" for more information.
Specify the imexpire rules to implement this policy. See "To Set Rules Implementing Message Expiration Policy" for more information.
Specify the imexpire scheduling. See "To Schedule Message Expiration and Logging Level" for more information.
You can define message expiration based on criteria such as:
Age of Message – Expire messages older than X days. Attribute: messagedays.
Message Count – Expire messages in a folder exceeding X messages. Attribute: messagecount.
Age of Oversized Message – Expire messages that exceed X bytes after Y days grace period. Attributes: messagesize and messagesizedays.
Seen and Deleted Message Flag – Expire messages with the Seen or Deleted flag set. These criteria can be set to "and" or "or." If set to or, the message's Seen/Delete flag will cause expiration regardless of other criteria. If set to and, the message's Seen/Delete flag must be set along with passing all other specified criteria. Attributes: seen and deleted.
Header Field of Message – Allows you to specify a header and string as criteria for expiring a message, for example, removing all messages with the header "Subject: Work from Home." Note that this feature also allows you to use message type as a criteria too. See "Expiring Messages by Message Type" for more information.
Folder of Messages – Allows you to specify the folder on which to expire messages. Attribute: folderpattern. Note that this attribute only uses the modified UTF-7 character set.
Note:
imexpire does not allow you to delete or preserve messages based on how long it has been since that message was read. For example, you cannot specify that messages that have not been read for 200 days will be removed.Example 1: Remove all messages 365 days old in a folder exceeding 1,000 messages.
Example 2: Remove messages in domain example.org that are older than 180 days.
Example 3: Remove all messages that have been marked as deleted.
Example 4: Remove messages in example.com that have been marked as seen, are older than 30 days, are larger than 100 kilobytes, from folders exceeding 1,000 messages, with the header X-spam.
Rules are set by putting them into a store.expirerule file. An example of two global store.expirerule rules is shown below:
Rule1.regexp: 1 Rule1.folderpattern: user/.*/trash Rule1.messagedays: 2 Rule2.regexp: 1 Rule2.folderpattern: user/.* Rule2.messagedays: 14
In this example, Rule 1 specifies that all messages in all users' trash folders are removed after two days. Rule 2 specifies that all messages in any folder in the message store are removed after 14 days.
This section sets the guidelines for the store.expirerule file rules.
Note:
In earlier Messaging Server releases, expiration rules could be set with configutil options store.expirerule.attribute (see Messaging Server Reference.) This is still true, but expire rules using header constraints (example: expiring a message with a specific subject line) are not supported. Also, regular expressions in the expire rules created with configutil need to be POSIX compliant rules. If you want to use UNIX compliant regular expressions you must use the store.expire file. In addition, using both configutil options and the global store.expirerule configuration file is not supported. If the configuration file is present, configutil options are not used. In any case, it is best to use store.expirerule to specify all expiration rules.Rules are specified in a file called store.expirerule.
Multiple expiration criteria can be specified with the same rule. (See preceding example.)
Rules can apply to the entire message store (global rules), a partition, a user, or a folder.
The global rules are stored in the MessagingServer_home/config/store.expirerule file.
Note:
Each global rule will be checked against every mailbox, which can cause some processing overhead depending on the number of global rules you specify. For this reason, you should not specify partition, mailbox or user rules in the global rules file. In general, you should try not to put any more expiration rules than necessary in this file.Partition rules are stored in store_root/partition/partition_name/store.expirerule (more accurately, the location specified by the store.partition.*.path configutil option).
User rules are specified in store_root/partition/partition_name/userid/store.expirerule or by specifying the folderpattern rule to be user/userid/.*.
Folder rules are specified in store_root/partition/partition_name/userid/folder/store.expirerule or by specifying the folderpattern rule to be user/userid/folder.
Multiple non-global rules (user, folder, partition) using rule_name was only implemented starting in Messaging Server 6.2p4.
Multiple expire rules can be applied to a mailbox at the same time. An expire policy for a mailbox consists of global rules and local rules. Local rules apply to the mailbox under the same directory and all of its sub-folders.
imexpire unifies all of the expiration rules applying to a mailbox, unless there is an exclusive rule specified for this mailbox (see Table 43-1). The resulting rule set represents the most restrictive expiration policy based on all applicable rules. For example, if rule X expires messages such that the maximum message life is 10 days, and rule Y specifies 5 days, the union will be 5 days.
Note:
When join: and attribute is specified in a rule, all expiration criteria specified in a rule must be satisfied for the message to expire (see Table 43-1). This option is valid if the rules are configured as exclusive rule. Example:rule.regexp: 1 rule.folderpattern: user/* rule.messagedays: 60 rule.messageheader.X-MESSAGE-TYPE: email rule.exclusive: 1 rule.join: and
In this example, the rule specifies to remove the messages that are older than 30 days AND contains message header X-MESSAGE-TYPE: Email.
Table 43-1 imexpire Attributes
| Attribute | Description (Attribute Value) |
|---|---|
|
action |
Specifies an action to perform on the messages caught by the expire rules. The possible values are:
|
|
exclusive |
Specifies whether or not this is an exclusive rule. If specified as exclusive, only this rule applies to the specified mailbox(es) and all other rules are ignored. If more than one exclusive rule exists, the last exclusive rule loaded will be used. For example, if a global and a local exclusive rule are specified, the local rule will be used. If there is more than one global exclusive rule, the last global rule listed by configutil is used. (1/0) |
|
expires |
imexpire will select the message if the date value specified with these header fields is older than the expiration date based on the messagedays attribute. If multiple expiration header fields are specified, the earliest expiration date will be used. (string) |
|
expiry-date |
imexpire will select the message if the date value specified with these header fields is older than the expiration date based on the messagedays attribute. If multiple expiration header fields are specified, the earliest expiration date will be used. (string) |
|
foldpattern |
Specifies the folders affected by this rule. The format must start with a user/, which represents the directory store_root/partition/*/. See Table 43-2, "imexpire Folder Patterns Using Regular Expressions". (POSIX regular expression) |
|
messagecount |
Maximum number of messages in a folder. Oldest messages are expunged as additional messages are delivered. (integer) |
|
foldersize |
Maximum size of folder before the oldest messages are expunged when additional messages are delivered. (integer in bytes) |
|
messagedays |
Number of days in the message store before being expunged. (integer) |
|
messagesize |
Maximum size of message in bytes before it is marked to be expunged. (integer) |
|
messagesizedays |
Grace period. Days an over-sized message should remain in a folder. (integer) |
|
messageheader.header |
Specifies a header field and string. Values are not case-sensitive and regular expressions are not recognized. Example: Rule1.messageheader.Subject: Get Rich Now! Headers other than Subject: can be used. |
|
regexp |
Enable UNIX regular expressions in rules creation. (1 or 0). If not specified, IMAP expressions will be used. |
|
savedays |
Number of days the messages are saved in a folder until they are expunged. |
|
seen |
seen is a message status flag set by the system when the user opens a message. If the attribute seen is set to and, then the message must be seen and other criteria must be met before the rule is fulfilled. If the attribute seen is set to or, then the message only needs to be seen or another criteria be met before the rule is fulfilled. (and/or) |
|
sieve |
A Sieve test specifying message selection criteria. Example: Rule17.sieve: header :contains "Subject" "Vigara" |
|
deleted |
deleted is a message status flag set by the system when the user deletes a message. If the attribute deleted is set to and, then the message must be deleted and another criteria must be met before the rule is fulfilled. If the attribute deleted is set to or, then the message only needs to be deleted or another criteria be met before the rule is fulfilled. (and/or) |
|
join |
join may only be specified in an exclusive rule. The default value is or. Specifying join: and means all criteria must be met before the rule is fulfilled. (and /or) |
|
userflag.flag-name |
Valid values are and/or. Specify user IMAP flags in expiration rules. The following example shows a rule expires those messages with the junk flag set, and which are older than 30 days: messagedays:30 userflag.junk: and |
|
channel |
Specify the name of an MTA channel as which to run for purposes of spam/virus filtering. In order for a channel attribute value to take effect, the expiresieve Message Store option must be enabled. (string) |
|
rescanhours |
When using imexpire to perform post-delivery spam/virus filtering, the rescanhours tells imexpire to rescan those message that have not been scanned for the specified number of hours. In order for any rescanhours attribute value to take effect, the expiresieve Message Store option must be enabled. (integer) |
The IMAP protocol specifies that mailbox names use modified UTF-7 encoding. Messaging Server supports localized character sets on external interfaces so that mailbox names can be localized. Internally, however, the system converts the localized name to MUTF-7. Thus, a folder that has a localized mailbox name on a client will have a corresponding mailbox file name in MUTF-7. (Note that IMAP error messages will output mailbox names in MUTF-7 and not the localized character set.)
In general, most message store utilities that require mailbox names expect the names in the localized character set, although they may have an option flag that allows a different character set to be used. These utilities include reconstruct, mboxutil, imsbackup, imsrestore, and hashdir. However, imexpire requires that the mailbox name, specified as the attribute folderpattern, be in MUTF-7. Using a localized name will not work.
To obtain the appropriate folderpattern for imexpire it may be necessary to convert a localized mailbox name to the modified UTF-7 equivalent. This can be done using the mboxutil -E command as follows:
mboxutil -l -p user/han/*
msgs Kbytes last msg partition quotaroot mailbox
57 100 2010/04/29 11:18 primary 5242880 user/han/INBOX
1 1 2010/04/30 12:56 primary - user/han/<multibyte_mailbox>
mboxutil -l -E MUTF-7 -p user/han/*
57 100 2010/04/29 11:18 primary 5242880 user/han/INBOX
1 1 2010/04/30 12:56 primary - user/han/&kAFP4W4IMH8wojCkMMYw4A-
The first mboxutil shows the localized mailbox name. The second mboxutil shows the mailbox name in MUTF-7. The MUTF-7 mailbox name is identical to the IMAP list command:
x list "" * * LIST (\NoInferiors) "/" INBOX * LIST (\HasNoChildren) "/" &kAFP4W4IMH8wojCkMMYw4A-
To convert the local charset to modified UTF-7 encoding, use the mboxutil command with the -E option:
mboxutil -l -E MUTF-7 -P user/han/<multibyte_mailbox>
msgs Kbytes last msg partition quotaroot mailbox
1 1 2010/04/30 12:56 primary - user/han/&kAFP4W4IMH8wojCkMMYw4A-
Note that mboxutil -E can be used for any command that requires the use of a MUTF-7 mailbox name including imexpire.
Message expiration rules are set by specifying expire criteria in a store.expirerule file. The store.expirerule file contains one expire criteria per line. An expire criteria of the global rule configuration file (DataRoot/store/store.expirerule) has the following format:
rule_name.attribute: value
"Example imexpire Rules" shows a set of global expiration rules in MessagingServer_home/config/store.expirerule.
Rule 1 sets the global expiration policy (that is, policy that applies to all messages), as follows:
Enable UNIX regular expressions in rules creation.
Removes messages larger than 100,000 bytes after 3 days.
Removes messages deleted by the user.
Removes any message with the strings "Viagra Now!" or "XXX Porn!" in the Subject: header.
Limits all folders to 1,000 messages, the system removes the oldest messages on a folder to keep the total to 1.000.
Removes all messages older than 365 days.
Rule 2 sets the message expiration policy for users at the hosted domain example.org. It limits mailbox sizes to 1 megabyte, removes messages that have been deleted, and removes messages older than 14 days.
Rule 3 sets the message expiration policy for messages in the inbox folder of user f.dostoevski. It removes messages with a subject line having the expression "On-line Casino."
Rule1.regexp: 1 Rule1.folderpattern: user/.* Rule1.messagesize: 100000 Rule1.messagesizedays: 3 Rule1.deleted: or Rule1.Subject: Vigara Now! Rule1.Subject: XXX Porn! Rule1.messagecount: 1000 Rule1.messagedays: 365 Rule2.regexp: 1 Rule2.folderpattern: user/.*@example.org/.* Rule2.exclusive: 1 Rule2.deleted: or Rule2.messagedays: 14 Rule2.messagecount: 1000 Rule3.folderpattern: user/f.dostoevski/inbox Rule3.Subject: On-line Casino
Folder patterns can be specified using POSIX regular expressions by setting the imexpire attribute regex to 1. If not specified, IMAP expressions will be used. The format must start with a user/ followed by a pattern. Table 43-2 shows the folder pattern for various folders.
Table 43-2 imexpire Folder Patterns Using Regular Expressions
| Scope | Folder Patterns (regex=0) | Folder Pattern (regex=1) |
|---|---|---|
|
Apply rule to all messages in all folders of userid. |
user/userid/* |
user/userid/.* |
|
Apply rule to messages of userid in folder Sent:. |
user/userid/Sent |
user/userid/Sent |
|
Apply rule to entire message store. |
user/* |
user/.* |
|
Apply rule to any folder called Trash anywhere in any user's hierarchy. |
user/*/Trash |
user/.*/Trash |
|
Apply rule to folders in hosted domain example.org. |
user/*@example.org/* |
user/.*@example.org/.* |
|
Apply rule to folders in default domain. |
Not applicable |
user/[^@]*/.* |