53 Administering Very Large Mailboxes
Oracle Communications Messaging Server enables the use of "very large" mailboxes.
Topics:
Very Large Mailboxes Overview
To increase the mailbox size limit and improve expunge performance, the index records and cache records have been split into separate files with support added for multiple cache files. An index file contains a mailbox header and a 128-byte fixed-length record for every message in the mailbox. The index record contains the location and size of the corresponding cache record. The cache files contain frequently used data in variable length records. The cache file size is configurable (store.maxcachefilesize). The default cache size is 500 Mbs, with a maximum size 2 Gbs. New cache records are appended to the newest cache file. As a cache file fills up, a new cache file is created, enabling a mailbox to continue to grow.
To optimize expunge performance, only the index file store.idx is purged when a mailbox is expunged. Expunge removes the obsoleted index records from the index file. Cache record and message file removal is deferred until the size of the expunged data exceeds a configurable threshold (store.purge.count or store.purge.percentage for cache records; store.cleanupsize for the number of messages). When the expunged size exceeds the threshold, expunge enqueues a purge request to the "Message Store Maintenance Queue." impurge dequeues the request, removes the unused message files and purge the cache files when the expunged size exceeds the purge threshold.
The Structure of a Mailbox
The following figure is an example of a large mailbox:
Mailbox Size Limit
IMAP defines the UID as a 32-bit value. Therefore, the maximum number of messages in a mailbox is limited to 4,294,967,295. For a 64 bit messaging server, the maximum number of messages in a folder is 4,294,967,295. For a 32-bit messaging server, the maximum number of messages in a folder is 16,777,215. The msconfig option store.maxmessages can be used to limit the size of a folder. The default limit is 16,000,000.
Note, the 4 billion limit is a hard limit. Effective limit is much smaller (normally less than 1 million). Mailbox expunge has to rewrite the store.idx file. The larger the mailbox, the longer it takes to expunge.
Mailbox Migration
Mailboxes are migrated to the new format automatically when they are opened. The operation is transparent to the end users.
Pre-Deployment Preparations
The high-level steps for preparing to use very large mailboxes include:
-
Configuring the maximum cache file size
-
Configuring the mailbox expunge size
-
Configuring store.maxmessages, quotas or expire rules to prevent mailboxes from getting too big
The new msconfig options for managing large mailboxes are:
Checking Mailbox Data
The imcheck command line utility can be used to dump the data of mailboxes in readable format. For example, imcheck -m mailbox dumps the content of a store.idx file:
$ imcheck -m user/dumbo/INBOX -------------------------------------------------------------------------------- Name: user/dumbo Version: 103 Exists: 4 Flags: 0 Largest Msg: 1521 bytes Last Append: 20080131104858 Last Repair: - Last UID: 4 Oldest Msg: 20080131104843 Oldest Uid: 1 Quota Used: 2394 Bytes Expunged: 0 UID Validity: 1201805323 Last CacheId: 1 Start Offset: 256 Append CacheId: 1 ACL: dumbo lrswipcdan Subscribed: 0 Partition: primary Path: /var/opt/SUNWmsgsr/store/partition/primary/=user/94/60/=dumbo Msg Path: /var/opt/SUNWmsgsr/store/partition/primary/=user/94/60/=dumbo MsgNo Uid Internal-Date Sent-Date Size HSize Cache-Id C-Offset C-Len Last-Updated Save-Date MT SFlags UFlags Original-Uid Message-id --------------------------------------------------------------------------------------------------------------------------------------------------------------- 1 1 20080131104843 20080131104843 257 244 1 16 864 20080131104843 20080131104843 2 R 0.0.0 1201805323-1 - 2 2 20080131104851 20020301191039 338 229 1 880 816 20080131104851 20080131104851 2 R 0.0.0 1201805323-2 <001@red.iplanet.com> 3 3 20080131104855 20020301191039 1521 241 1 1696 840 20080131104855 20080131104855 2 R 0.0.0 1201805323-3 <002@red.iplanet.com> 4 4 20080131104858 20050919110532 278 252 1 2536 860 20080131104858 20080131104858 1 R 0.0.0 1201805323-4 <003@red.iplanet.com>
imcheck -m mailbox -c msgno displays the cache records of a message:
$ imcheck -m user/dumbo/INBOX -c 1
Cache items of user/dumbo message number 1:
ENVELOPE {300}
("Thu, 31 Jan 2008 10:48:43 -0800" "welcome" (("Mail Administrator" NIL "Postmaster" "puzzle.red.iplanet.com")) (("Mail Administrator" NIL "Postmaster" "puzzle.red.iplanet.com")) (("Mail Administrator" NIL "Postmaster" "puzzle.red.iplanet.com")) ((NIL NIL "dumbo" "red.iplanet.com")) NIL NIL NIL NIL)
BODYSTRUCTURE {75}
("TEXT" "PLAIN" ("CHARSET" "us-ascii") NIL NIL "7BIT" 13 1 NIL NIL NIL NIL)
BODY {59}
("TEXT" "PLAIN" ("CHARSET" "us-ascii") NIL NIL "7BIT" 13 1)
SECTION {48}
Header offset: 0 len: 244
Body offset: 244 len: 13
1 subparts
(1) offset: 244 len: 13 charset: 0 encoding: 0
CACHEHEADERS {244}
Subject: welcome
To: dumbo@red.iplanet.com
Date: Thu, 31 Jan 2008 10:48:43 -0800
From: Mail Administrator <Postmaster@puzzle.red.iplanet.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
FROM {53}
mailadministrator <postmaster@puzzle.red.iplanet.com>
TO {23}
<dumbo@red.iplanet.com>
CC {0}
BCC {0}
SUBJECT {9}
"welcome"
XSENDER {0}