36 Tuning the mboxlist Database Cache in Unified Configuration

Setting the Mailbox Database Cache Size

Messaging Server makes frequent calls to the mailbox database. For this reason, it helps if this data is returned as quickly as possible. A portion of the mailbox database is cached to improve Message Store performance. Setting the optimal cache size can make a big difference in overall Message Store performance. You set the size of the cache with the store.dbcachesize option.

You should use the store.dbtmpdir option to redefine the location of the mailbox temporary files to a tmpfs, that is, /tmp/msgDBtmpdir. On Linux, the tmpfs location is usually /dev/shm. The default store.dbtmpdir value (/tmp/.xxx) is not appropriate, so you should change it manually to use the tmpfs location. On reboot, the tmpfs will be cleared, therefore be sure that permissions are stored so that the correct directory location can be recreated.

To define the location of the mailbox temporary files:

1. Create the directory, for example:

   mkdir /tmp/msgDBtmpdir
   

2. Assign the appropriate ownership and permissions so that this directory is owned, readable, and writable by the mailsrv user. For example, if the mailsrv user name is mailuser and the group name is mail:

   chown mailuser:mail /tmp/msgDBtmpdir
   chmod 700 /tmp/msgDBtmpdir
   

3. Set the store.dbtmpdir option, for example:

   msconfig
   msconfig> set store.dbtmpdir /tmp/msgDBtmpdir
   msconfig# write
   msconfig> exit
   

The files stored in the store.dbtmpdir location are temporarily memory mapped files used by all processes connecting to the database. Due to their usage pattern, the pages of these files will most likely be in memory all the time. So setting this to be on a tmpfs will not really increase memory usage. What it does is save I/O. When the Oracle Solaris virtual memory system sees a memory mapped file on a tmpfs, it knows it does not really need to write the modified pages back to the file. So there is only one copy in memory and it saves I/O.

The mailbox database is stored in data pages. When the various daemons make calls to the database (stored, imapd, popd), the system checks to see if the desired page is stored in the cache. If it is, the data is passed to the daemon. If not, the system reads the desired page and writes it in the cache. If there is no clean page available, the system must write one page from the cache back to disk.

Lowering the number of disk read/writes helps performance, so setting the cache to its optimal size is important:

• If the cache is too small, the desired data will have to be retrieved from disk more frequently than necessary.

• If the cache is too large, dynamic memory (RAM) is wasted, and it takes longer to synchronize the disk to the cache.

Of these two situations, a cache that is too small will degrade performance more than a cache that is too large.

Cache efficiency is measured by hit rate. Hit rate is the percentage of times that a database call can be handled by cache. An optimally sized cache will have a 98 to 99 percent hit rate (that is, 98 to 99 percent of the desired database pages will be returned to the daemon without having to grab pages from the disk). The goal is to set the smallest cache so that it holds several pages such that the cache will be able to return at least 98 to 99 percent of the requested data. If the direct cache return is less than 98 percent, then you must increase the cache size.

To Adjust the Mailbox Database Cache Size

Use the msconfig command to set the size of the cache with the store.dbcachesize option, for example:

msconfig
msconfig> set store.dbcachesize 25165825
msconfig# write
msconfig> exit

It is important to tune the cache size to smallest size that will accomplish the desired hit rate.

The store.dbcachesize option controls the size of a shared memory segment used by all processes connected to the database, including stored, imap, popd, imsbackup, imsrestore, ims_master, tcp_lmtp_server, and so on. While the maximum value for store.dbcachesize is 2 GB, setting it to the maximum wastes memory. Instead, start with the default value of 64 MB and monitor the cache hit rate over a period of days. Increase the value only if the hit rate is under 98%.

Also consider the transaction checkpoint function (performed by stored). Set the store.checkpoint.debug option and refresh stored to see log messages to provide more exact data about transaction checkpoint function time. For example:

msconfig
msconfig> set -restricted store.checkpoint.debug 1
msconfig# write
msconfig> exit
refresh store
Refreshing store server 7585 ... done

This process must examine all buffers in the cache and hold a region lock during the checkpoint. Other threads needing the lock must wait.

To Monitor the Mailbox Database Cache Size

1. Use the imcheck command to measure the cache hit rate.

   imcheck -s mpool > imcheck-s.out
   

   Note:

   The imcheck -s command is only valid for Berkeley Database message store.

2. Find the cache information section in the output file, for example:

   2MB 513KB 604B Total cache size.
   1 Number of caches.
   1       Maximum number of caches
   2MB 520KB Pool individual cache size.
   

   Then there will be several blocks of output - a summary and one for each database file - look for these lines in each block:

   0 Requested pages mapped into the process' address space.
   55339 Requested pages found in the cache (99%).
   

   In this case, the hit rate is 99 percent. This could be optimal or, more likely, it could be that the cache is too large. To test, lower the cache size until the hit rate moves to below 99 percent. When you hit 98 percent, you have optimized the DB cache size. Conversely, if see a hit rate of less than 95 percent, then you should increase the cache size with the store.dbcachesize option.

As your user base changes, the hit rate can also change. Periodically check and adjust this option as necessary.