This chapter describes how to back up and restore Oracle Communications Messaging Server mailboxes. For conceptual information on the message store, see the following topics:
Mailbox backup and restore is one of the most common and important administrative tasks. You must implement a backup and restore policy for your message store to ensure that data is not lost if the following problems occur:
System crashes
Hardware failure
Accidental deletion of messages or mailboxes
Problems when reinstalling or upgrading a system
Natural disasters (for example, earthquakes, fire, hurricanes)
Migrating users
You can back up and restore mailboxes by using the "imsbackup" and "imsrestore" command-line utilities or the integrated backup and restore solution that uses Oracle StorageTek Enterprise Backup Software (EBS).
Messaging Server provides a single-copy backup procedure. Regardless of how many user folders contain a particular message, during backup, the message file is backed up only once using the first message file found. The second message copy is backed up as a link to the name of the first message file, and so on. imsbackup maintains a hash table of all messages using the device and inode of the message files as the index. This method does have implications when restoring data, however. See "Considerations for Partial Restore" for more information.
Note:
You can also back up and restore the message store by backing up all relevant message files and directories. See "Message Store Disaster Backup and Recovery" for more information.Backing up mailboxes includes three steps:
Your backup policy will depend on several factors, such as:
Take into account peak business loads when scheduling backups for your system as this can reduce system load during peak hours. For example, backups are probably best scheduled for early morning hours such as 2:00 AM.
Incremental backups (see "Incremental Backup") scan the message store for changed data and back up only what has changed. Full backups back up the entire message store. Determine how often the system should perform full as opposed to incremental backups. For example, you probably want to perform incremental backups as a daily maintenance procedure and full backups once a week.
When user data is stored on multiple disks, you can back up user groups in parallel. Depending on system resources, parallel backups can speed up the overall backup procedure. However, you might want to use serial backups to reduce backup impact on the server's performance. Whether to use parallel or serial backups can depend on many factors, including system load, hardware configuration, how many tape drives are available, and so on.
A backup group is an arbitrary set of user mailboxes defined by regular expressions. By organizing user mailboxes into backup groups, you can define more flexible backup management.
For example, you could create three backup groups, the first containing user IDs starting with the letters A through L, the second with users whose user IDs begin with M through Z, and the third with users whose user IDs begin with a number. Administrators could use these backup groups to back up mailboxes in parallel, or perhaps only certain groups on one day and other groups on another.
Consider the following points about backup groups:
They are arbitrary virtual groups of mail users that do not precisely map to the "Classic Message Store Directory Layout", although backup groups could resemble the message store directory.
Administrators define backup groups by using UNIX regular expressions. The regular expressions are defined in the MessagingServer_home/config/backup-groups.conf file.
When backup groups are referenced in imsbackup and imsrestore, they use the path format: /partition_name/backup_group
When you run the imsbackup command, it evaluates the entire backup-groups.conf, and if it finds more than one group that matches a user, it uses the first match. For example, the following backup-groups.conf contains these definitions:
groupA=a.* ... groupN=.*n$
Because both groups match the user ID admin, the imsbackup command uses the first match, which is groupA. Thus, groupA includes the admin mailbox. Furthermore, the groupN backup does not include the admin mailbox.
The format of backup-groups.conf is as follows:
group_name=definition group_name=definition . . .
Using the example described in the previous paragraph, you would use the following definitions to create the three backup groups:
groupA=[a-l].* groupB=[m,-z].* groupC=[0-9].*
You can now scope imsbackup and imsrestore at several levels. You can backup the whole message store by using the following backup commands:
imsbackup -f <device> /
To back up all mailboxes for all users in groupA use the following command:
imsbackup -f <device> /<partition>/groupA
The default partition is called primary.
Oracle Communications Messaging Server includes one predefined backup group that is available without creating the backup-groups configuration file. This group is called user and includes all users. For example, the following command backs up all users on the primary partition:
imsbackup -f backupfile /primary/user
To back up and restore your mailboxes, Messaging Server provides the "imsbackup" and "imsrestore" utilities. The imsbackup and imsrestore utilities do not have the advanced features found in general purpose tools like EBS. For example, the utilities have only very limited support for tape auto-changers, and they cannot write a single store to multiple concurrent devices. Comprehensive backup is achieved by using plug-ins to generalized tools like EBS. See "To Use StorageTek Enterprise Backup Software" for more information about using EBS.
With imsbackup, you can write selected contents of the message store to any serial device, including magnetic tape, a UNIX pipe, or a plain file. The backup or selected parts of the backup can later be recovered by using the imsrestore utility. The output of imsbackup can be piped to imsrestore.
The following example backs up the entire message store to /dev/rmt/0:
imsbackup -f /dev/rmt/0 /
This example backs up the mailboxes of user ID joe to /dev/rmt/0:
imsbackup -f /dev/rmt/0 /primary/user/joe
This example backs up all the mailboxes of all the users defined in the backup group groupA to backupfile (see "To Create Backup Groups"):
imsbackup -f /primary/groupA > backupfile
The following example backs up messages stored from May 1, 2004 at 1:10 pm to the present. The default is to back up all the messages regardless of their dates:
imsbackup -f /dev/rmt/0 -d 20040501:131000 /
This command uses the default blocking factor of 20. See "imsbackup" for a complete syntax description.
Regarding date-time stamp:
20040501:131000 YYYYMMDD:HHMMSS 2004 05 01 : 13 10 00 YYYY MM DD : HH MM SS
When you perform a backup operation, you can specify mailboxes that will be excluded from being backed up. By excluding bulk or trash mailboxes that can accrue large numbers of unimportant messages, you can streamline the backup session, reduce the time to complete the operation, and minimize the disk space required to store the backup data.
To exclude mailboxes, specify a value for the store.backupexclude option.
You can specify a single mailbox or a list of mailboxes separated by the "%' character. ("%' is an illegal character in a mailbox name.) For example, you could specify the following values:
Trash
Trash%Bulk Mail%Third Class Mail
In the first example, the folder Trash is excluded. In the second example, the folders Trash, Bulk Mail, and Third Class Mail are excluded.
Example commands:
cd /opt/sun/comms/messaging64/bin msconfig set store.backupexclude "Trash%Bulk Mail%Third Class Mail" msconfig show store.backupexclude role.store.backupexclude = Trash%Bulk Mail%Third Class Mail
The backup utility backs up all folders in a user mailbox except those folders specified in the store.backupexclude option.
This feature works with the Messaging Server backup utility, StorageTek Enterprise Backup Software, and third-party backup software.
You can override the store.backupexclude setting and back up an excluded mailbox by specifying its full logical name during the operation. For example, suppose the Trash folder has been excluded. You can still back up Trash by specifying the following:
/primary/user/user1/trash
However, if you specify
/primary/user/user1
the Trash folder is excluded.
To restore messages from the backup device, use the "imsrestore" command. For example, the following command restores messages for user1 from the file backupfile.
imsrestore -f backupfile /primary/user1
A partial restore is when only a part of the message store is restored. A full restore is when the entire message store is restored. The message store uses a single-copy message system. That is, only a single copy of any message is saved in the store as a single file. Any other instances of that message (for example, when a message is sent to multiple mailboxes) are stored as links to that copy. Because of this, there are implications when restoring messages. For example:
Full Restore. During a full restore, linked messages still point to the same inode as the message file to which they are linked.
Partial Backup/Restore. During a partial backup and partial restore, however, the single-copy characteristic of the message store might not be preserved.
The following examples demonstrate what happens to a message that is used by multiple users when a partial restore is performed. Assume there are three messages, all the same, belonging to three users A, B, and C, as follows:
A/INBOX/1 B/INBOX/1 C/INBOX/1
Example 1. In the first example, the system performs a partial backup and full restore procedure as follows:
Back up mailboxes for users B and C.
Delete mailboxes of users B and C.
Restore the backup data from step 1.
In this example, B/INBOX/1 and C/INBOX/1 are assigned a new inode number and the message data is written to a new place on the disk. Only one message is restored. The second message is a hard link to the first message.
Example 2. In this example, the system performs a full backup and a partial restore as follows:
Perform full backup.
Delete mailboxes for user A.
Restore mailboxes for user A.
A/INBOX/1 is assigned a new inode number.
Example 3. In this example, partial restore might require more than one attempt:
Perform full backup.
B/INBOX/1 and C/INBOX/1 are backed up as links to A/INBOX/1.
Delete mailboxes for users A and B.
Restore mailboxes for user B.
The restore utilities ask the administrator to restore A/INBOX first.
Restore mailboxes for users A and B.
Delete mailboxes for user A (optional).
Note:
If you want to ensure that all messages are restored for a partial restore, you can run the imsbackup command with the -i option. The -i option backs up every message multiple times if necessary.If the backup device is seekable (for example, a drive or tape), imsrestore seeks to the position containing A/INBOX/1 and restores it as B/INBOX/1. If the backup device is non-seekable (for example, a UNIX pipe), imsrestore logs the object ID and the ID of the depending (linked) object to a file, and the administrator must invoke imsrestore again with the -r option to restore the missing message references.
If you are restoring messages from a mailbox that has been incrementally backed-up, and if that mailbox exists on the server on which you want to restore the messages, then restoring the messages requires a straightforward imesrestore. However, if you want to restore messages from a mailbox that has been incrementally backed-up, and if that mailbox no longer exists, you must follow different restore procedures.
Use one of the following procedures to restore messages to a mailbox that does not exist on the message store server:
During the restore operation, disable delivery of messages to the user. Do this by setting the LDAP attribute mailDeliveryOption to hold.
Before you use imesrestore, create the mailbox with the mboxutil -c command.
The reason why these instructions must be followed for restoring an incremental backup is as follows: When a mailbox has been deleted or is being migrated, the imsrestore utility recreates the mailbox with the mailbox unique identification validity and message unique identifications (UIDs) stored in the backup archive.
In the past, when imsrestore would recreate a deleted or migrated mailbox, it would assign a new UID validity to the mailbox and new UIDs to the messages. In that situation, a client with cached messages would have to resynchronize the mailbox UID validity and message UIDs. The client would have to download the new data again, increasing the workload on the server.
With the new imsrestore behavior, the client cache remains synchronized, and the restore process operates transparently with no negative impact on performance.
If a mailbox exists, imsrestore assigns new UIDs to the restored messages so that the new UIDs remain consistent with the UIDs already assigned to existing messages. To ensure UID consistency, imsrestore locks the mailbox during the restore operation. However, because imsrestore now uses the mailbox UID validity and message UIDs from the backup archive instead of assigning new UID values, UIDs could become inconsistent if you perform incremental backups and restores.
If you perform incremental backups with the -d date option of the imsbackup utility, you might have to invoke imsrestore multiple times to complete the restore operation. If incremental backups were performed, you must restore the latest full backup and all subsequent incremental backups.
New messages can be delivered to the mailbox between the restore operations, but in this case, the message UIDs can become inconsistent. To prevent inconsistency in the UIDs, you need to take one of the actions previously described on this page.
Messaging Server includes a backup API that provides an interface with third-party backup tools, such as EBS. The physical message store structure and data format are encapsulated within the backup API. The backup API interacts directly with the message store. It presents a logical view of the message store to the backup service. The backup service uses the conceptual representation of the message store to store and retrieve the backup objects.
Messaging Server provides an Application Specific Module (ASM) that can be invoked by the EBS's save and recover commands to back up and restore the message store data. The ASM then invokes the Messaging Server imsbackup and imsrestore utilities.
Note:
This section provides information about how to use EBS with the Messaging Server message store. To understand the EBS interface, see your StorageTek Enterprise Backup Software documentation.Create a symbolic link from /usr/lib/nsr/imsasm to MessagingServer_home/lib/msg/imsasm.
From Oracle or EMC, obtain a copy of the nsrfile binary and copy it to the following directory:/usr/bin/nsr
This is required only if you are using an older version of Networker (5.x). With Networker 6.0 and above, nsrfile is automatically installed under /usr/bin/nsr.
If you want to back up users by groups, perform the following steps:
Create a backup group file. See "To Create Backup Groups" for more information.
To verify your configuration, run mkbackupdir.sh. Look at the directory structure created by mkbackupdir.sh. The structure should look similar to that shown in "Classic Message Store Directory Layout". If you do not specify a backup-groups.conf file, the backup process uses the default backup group ALL for all users.
In the directory /nsr/res/, create a res file for your save group to invoke the mkbackupdir.sh script before the backup. See "Classic Message Store Directory Layout" for an example.
Note:
Earlier versions of Networker have a limitation of 64 characters for the save set name. If the name of this directory plus the logical name of the mailbox (for example, /primary/groupA/fred) is greater than 64 characters, then you must run mkbackupdir.sh -p. Therefore, you should use a short path name for the -p option of mkbackupdir.sh. For example the following command will create the backup image under the directory /backup:mkbackupdir.sh -p /backup
Important: The backup directory must be writable by the message store owner (example: mailsrv).
The following is a sample backup groups directory structure.
/backup/primary/groupA/amy /bob /carly /groupB/mary /nancy /zelda /groupC/123go /1bill /354hut
The following example shows a sample res file named IMS.res in the /nsr/res directory:
type: savepnpc; precmd: "echo mkbackupdir started", "/usr/example/server5/msg-example/bin/mkbackupdir.sh -p /backup"; pstcmd: "echo imsbackup Completed"; timeout: "12:00 pm";
You are now ready to run the EBS interface as follows:
Create the Messaging Server save group if necessary.
Run nwadmin.
Select Customize | Group | Create.
Create a backup client using savepnpc as the backup command:
Set the save set to the directory created by mkbackupdir. For a single session backup, use /backup.
For parallel backups, use /backup/server/group. Be sure you have already created group as defined in "To Create Backup Groups". You must also set the parallelism to the number of backup sessions. See "To Back Up Data By Using StorageTek Enterprise Backup Software" for more information.
Select Group Control | Start to test your backup configuration. Example. Creating A Backup Client in EBS: To create a backup client in EBS. From nwadmin, select Client | Client Setup | Create
Name: example Group: IMS Savesets:/backup/primary/groupA /backup/secondary/groupB /backup/tertiary/groupC . . Backup Command:savepnpc Parallelism: 4
To recover data, you can use the EBS nwrecover interface or the recover command-line utility. The following example recovers user a1's INBOX:
recover -a -f -s example /backup/example/groupA/a1/INBOX
The next example recovers the entire message store:
recover -a -f -s example /backup/example
Messaging Server provides two message store backup solutions, the command line imsbackup and the StorageTek Enterprise Backup Software. A large message store running a single imbackup to back up the entire message store can take a significant amount of time. The EBS solution supports concurrent backup sessions on multiple backup devices. Concurrent backup can shorten backup time dramatically (backups of 25GB of data per hour have been achieved).
If you are using another third party concurrent backup software (for example, Netbackup), you can use the following method to integrate your backup software with the Messaging Server.
Divide your users into groups (see "To Create Backup Groups") and create a backup-groups.conf file under the directory MessagingServer_home/config/.
Note:
This backup solution requires additional disk space. To backup all the groups concurrently, the disk space requirement is two times the message store size. If you do not have that much disk space, divide your users into smaller groups, and then backup a set of groups at a time. For example group1 - group5, group6 - group10. Remove the group data files after backup.Run imsbackup to back up each group into files under a staging area. The command is imsbackup -f <device>/<instance>/<group> You can run multiple imsbackup processes simultaneously. For example:
imsbackup -f- /primary/groupA > /bkdata/groupA & imsbackup -f- /primary/groupB > /bkdata/groupB & . . .
Use your third party backup software to back up the group data files in the staging area (in our example that is /bkdata).
To restore a user, identify the group filename of the user, restore that file from tape, and then use imsrestore to restore the user from the data file.
This section describes common backup and restore problems and their solutions.
Problem: msprobe restarts everything during a long imsrestore during message store migration. This can also happen with imsbackup, imsimport, or any processing intensive utility.
Solution: When imsrestore or any processing intensive operation takes significantly more system resources than normal, and continues doing so longer than the msprobe interval, there might be a temporary backlog of DB transaction log files to be cleared. If there are more files than specified in store.maxlog, then msprobe may erroneously restart all the processes during a restore. To prevent this from happening, disable msprobe during the imsrestore.
Problem: When I do an restore of a folder or INBOX using imsrestore or imsasm, it appends all the messages in that folder onto the current folder. This results in multiple copies of the messages in that folder.
Solution: Make sure the -i flag of imsrestore is not set in the imsasm script.
Problem: I want to do an incremental backup of just new messages added in a mail folder, but when I try, the entire folder gets backed up. How do I just back up the new messages?
Solution: Set the -ddatetime flag on imsbackup. This will backup messages stored from the specified date and time to the present. The default is to back up all messages regardless of their dates.
A disaster refers to a catastrophic failure of the entire message store as opposed to a mailbox or set of mailboxes. That is, a situation where all data on the message store servers are lost. A complete message store disaster restore will consist of restoring the following lost data:
All message store data. These can be backed up using the procedures described above. If file system backup method is used, be sure to back up the following data:
All message store partitions
The message store database files at DataRoot/store/mboxlist.
The message store database snapshots at DataRoot/store/dbdata/snapshots (Note that the location of message store database snapshot files can be configured with the store.snapshotpath option.)
Configuration data. Including the local configuration file at DataRoot/config. See also "Classic Message Store Directory Layout".
If you want to back up your message store for disaster recovery, you can use file system snapshot tools to take a snapshot of the file system. The snapshot must be a point-in-time file system snapshot.
It is best to capture all the data (message store partitions, database files and so on) at the same point-in-time, however, if this cannot be done, then you must backup the data in this order:
Database snapshots
Database files
Message Store indexes if separated from the messages using the partition:partition-name.messagepath option (store.partition.*.messagepath in legacy configuration). For additional information, see the discussion about the messagepath option in Messaging Server Reference.
Messages
Configuration data
If database files and Message Store indexes are not backed up with the same point-in-time snapshot (or database files, Message Store indexes, and Messages, if indexes are not separate) then reconstruct -m is required after restore.