Complete Contents
Introduction
Chapter 1 Preparing for Installation
Chapter 2 Installing Netscape Messaging Server 4.1
Chapter 3 Ugrading an Existing Installation
Appendix A Installing a 3.x Directory Server
Index
Messaging Server Installation Guide: Upgrading an Existing Installation
Previous Next Contents Index


Chapter 3 Upgrading an Existing Installation

This chapter describes how to upgrade an existing Messaging Server installation and how to migrate mailboxes and message queues from version 3.01 or later to Messaging Server 4.1. It contains the following sections:

Note. The following procedures do not support upgrading from versions earlier than 3.01.


Upgrade Process Overview
There are three paths for upgrading an existing Messaging Server installation to version 4.1:

The following sections describe the processes involved for the various upgrade paths.


Upgrade Considerations
The installation program first detects if there is an existing Messaging server installed on your system. If it detects an existing Messaging Server, the installation program asks if you want to upgrade the installation or perform a completely new installation.

If you are upgrading a 3.x Messaging Server, the number of mailboxes and messages to be migrated will determine how long the upgrade process will take. The more mailboxes and messages you need to transfer, the longer the process will take. Your machine and network parameters and load are also factors influencing how long a migration will take.

If an existing server is detected, the installation program presents additional options depending on your selection:

If you decline the upgrade requested by the installation program, the standard installation procedure is executed as described in the earlier chapters of this document. Note the following important points:


Upgrading a 3.x Server to a 4.1 Server
When you run the installation program, it automatically detects if you have an existing Messaging Server 3.x installation and asks if you want to perform the upgrade and migration as part of the installation procedure:

Before You Run the Upgrade Utility
Before running the upgrade utility, note the following points:

Using the Installation Program to Migrate 3.x Mailboxes to 4.1
During the installation procedure, the installation program will ask if you want to migrate the configuration settings and mailboxes from the existing 3.x installation:

If you choose to migrate your 3.x mailboxes, you should note the following important points:

Once the migration has completed successfully, the installation program completes the upgrade and restarts all services.

Using the Upgrade Utility to Migrate 3.x Mailboxes to 4.1
This section describes how to migrate mailboxes from Messaging Server 3.x to Messaging Server 4.1 with the upgrade utility. (The upgrade utility performs functions similar to the migrate utility provided with Messaging Server 3.x.)

The upgrade utility supports multi-processes and muti-threading. In addition, the upgrade utility can read a list of users contained in a text file so that you can run several simultaneous upgrades for different 3.x mailbox store paths.

The upgrade process consists of two major operations:

Before upgrading messages, you must first upgrade folders (upgrade -s).

Upgrading Folders
You upgrade folders by running the upgrade utility with the -s option. The upgrade utility creates new files in the 3.x default mailbox directory for later message upgrade.

Some files are used as a status marker, such as:

Some files are used as the mapping database, such as:

The folder upgrade operation can be run only as a single-process, which means you can perform one upgrade process at a time to migrate folders (upgrade -s). Later, however, you can run several upgrade processes simultaneously to migrate messages (upgrade -m).

The folder upgrade process searches LDAP entries to find all users in the existing message server, retrieves all folders names in each user's physical 3.x mailbox directory, and creates a mapping table (stored in a file called __upfolder.db) that it uses to add the folders names into 4.x folder database.

The mapping table is later used for migrating messages without to having search the LDAP directory again and as a reference to the 4.x Messaging Server which has a new folder name syntax that requires some characters in 3.x to be remapped to their 4.x equivalents.

The upgrade -s operation does not create the actual physical 4.x mailboxes, but only the folders name entries in which the messages are stored and marks the messages as "TRANSITION".

Upgrading Folders with Multiple Mail Store Paths
In Messaging Server 4.x, the logical name mapped to the actual physical path is called the partition. If you are upgrading a Messaging Server 3.x installation that has multiple mail store paths (in addition to the default), the upgrade utility attempts to create a new partition name that maps to the mail store path. It also creates a new subdirectory in the old 3.x mail store paths, so that the new 4.1 mailboxes can be created in the same mail store path.

  1. Upgrade generates a new partition name.

    2. The upgrade utility creates a partition name based on the last subdirectory name in mail store path, stripping out the non-alphanumeric characters. and converting it lowercase. For example:

    To verify that duplicate partition names do not exist, the upgrade utility searches through all user names and generates a list of all mail store paths. If a duplication exits, the upgrade utility stops and reports the error. To correct the problem, the administrator must change the mail store path and move users into a different directory and change all user LDAP entries.

    For example, if two Messaging Server 3.x partitions exist, such as:

    /disk1/partition.one

    /disk2/partition.one

    the upgrade process will fail. In this case, you will need to rename your partitions and then rerun the upgrade utility.

  2. Upgrade creates a 4.x directory.

    3. For example, if your 3.x mailbox store path is /mail/marketing, the upgrade utility migrates it to mail/marketing/.0004x.marketing. In this way, the 3.x mailbox store path is preserved.

    If you want to change /mail/marketing/.0004x.marketing to something else, after you run upgrade -s (folder upgrade), use the configutil program to change the partition to the path you want, before you run upgrade -m (message upgrade).

    Similar to a default mail store path in Messaging Server version 3.x, Messaging Server 4.1 uses primary as the partition name and maps it to the default path you specified during installation, such as:

    /usr/netscape/server4/msg-tango/store/partition/primary

    If you run configutil to get a list, you will see response similar to the following:

    store.partition.primary.path|/usr/netscape/server4/msg-tango/store/ partition/primary

  3. Upgrade generates a user id list file.

    4. The upgrade utility creates a file to store all user ids n the mail store path for the newly created partition. It names the file __up.partition.txt where partition is the name of the partition. For example, if you use default mail store value (primary), the file name is __up.primary.txt. If the partition name is marketing, the file name is __up.marketing.txt. The user id list file lets an administrator run the upgrade utility with the new option -u in a dedicated upgrade process to migrate a mail store path. For example:

    upgrade -u __up.primary.txt

Upgrading Messages
You can use any of the following methods to migrate 3.x messages to Messaging Server 4.1:

Before upgrading messages, you should:

  1. Use the command line utility configutil to examine the default value for all 4.x partitions. For more information, see Appendix A, "Command-line Utilities," in the Messaging Server 4.1 Administrator's Guide.

  2. Decide where to put the messages by specifying the new location of the message store, because the default directory generated when you upgraded the folders may not be the location what you want.
The message upgrade process provides two options:

-m

-u userlist

The -m option is a folder based operation whereas the -u userlist option is a per mail store and per user based operation. Both options require that you first run upgrade -s.

Both the -m and -u options support muti-threading. The upgrade utility creates one thread to retrieve all user folder names to put into a queue and several threads to transfer messages from the queue and migrate them from 3.x to 4.x. During the process of migrating messages, the upgrade utility creates the 4.x physical mailbox and index files.

By default, the number of the threads created is 5 and the queue size is 50 folders. The message transfer thread uses the database __upfolder.db to find the physical location of the folder in the 3.x mail store, lock the 3.x folder, and migrate the messages. After migrating all messages, the folder is marked as "NON-TRANSITION" in the 4.x database.

The message transfer thread does not delete the 3.x physical directory, even if you use the -r option to remove the 3.x messages after migration. It removes only the 3.x message files in the directory, but not the directory structure. If an error occurs while moving a message, the upgrade utility reports the error and leaves the message in its original location.

To migrate mailboxes and messages to a new 4.1 Messaging Server, follow these steps:

  1. Login as to root (Unix) or administrator (NT) on the Messaging Server machine.

  2. Change to the directory that contains the upgrade utility: For example:

    3. cd /usr/netscape/server4/bin/msg/admin/bin

  3. Create new 4.1 mailbox folders to match the existing 3.x folders.

    4. upgrade -s

  4. Transfer the 3.x mailbox messages to the 4.1 server.

    5. upgrade -m

    upgrade -m -t 10

    upgrade -m -r

    Alternatively, instead of using -m, you can use the -u userlist option to migrate users from one disk partition at a time or only migrate messages for a specific list of user IDs. See Migrating Users in a Multi-Partition Environment and Migrating Specific Users below for details.

Migrating Users in a Multi-Partition Environment
The -m option transfers messages for all 3.x users to the new 4.1 server regardless of the disk partition they are stored in. Instead of using -m, you may be able to improve message transfer efficiency by using the -u userlist option to transfer all messages from a single disk partition simultaneously (you first run upgrade -s on the partition).

When you ran upgrade -s, it created a separate users file for each partition. For example, a file named __up.primary.txt lists the users on the primary partition. If you have a secondary partition, upgrade -s also created a __up.secondary.txt file for the users on that partition. The __up.primary.txt file (or files) are stored in the 3.x default mailbox directory.

To migrate users one partition at a time, use upgrade -u userlist where userlist is the name of the partition file created when you ran upgrade -s. For example, the following command migrates just the users stored in the primary partition:

upgrade -u __up.primary.txt

The upgrade -u userlist command reads the user IDs to be migrated from a text file. You create your own text file listing the users whose messages are to be migrated as described below.

The -u option lets an administrator run an upgrade on a given mail store or migrate a single user folder that previously failed to upgrade. The -u option will read each user id, one by one, from a user list file that you specify and process all folders that belong to that ID. Because it uses the information in the user ID list file to locate the 3.x physical mailbox, it does not need to ask the LDAP server to locate the user's mail store.

The userlist file can be named whatever you want. The first three lines are mandatory and use a fixed format to store the physical location of a 3.x user mailbox as shown in the following example:

[4x partition]:primary

[3x MailStorePath]:/mail/mailbox-3.x

#---------List of mail user id ------

Following these three lines, you add the list of user IDs to be transferred with each user ID on a separate line and no blank lines or spaces between them. The message transfer process starts by reading user ids listed. For example, a userlist file should look like the following example:

[4x partition]:primary

[3x MailStorePath]:/mail/mailbox-3.x

#---------List of mail user id ------

postmaster

username1

user2

anotheruser

Specifying Non-Default 4.x Mail Store Directories
If you want to use a non-default directory to store messages, you need to run the configutil utility to change the partition name created by the upgrade utility before running upgrade -m (or upgrade -u).

For example, assume you have two 3.x message store directories:

You run upgrade -s and then run configutil to list all of your configuration parameters. If you check:

You will see that the values are:

Suppose, however, you want to use the directories /mailstore/sales and /mailstore/research. To set up your mail system with these non- default directories, you must use configutil to change the directory structure before running upgrade -m (or upgrade -u). For example:

configutil -o store.partition.store1.path -v "/mailstore/sales"

configutil -o store.partition.store2.path -v "/mailstore/research"

Make sure that the /mailstore/sales and /mailstore/research directories have been created with the same permissions as those of server-root/msg-instance/store.

Note. If you later rerun upgrade -s, the partition information that upgrade utility relies on will be reset back to the original default. Thus, you must remember to use configutil to change the partition paths every time you run upgrade -s.

Message Migration Errors
Errors that occur during the migration process are recorded in the log. The log file is named default and is located in the server-root/msg-instance/log/default/ directory. For example:

/usr/netscape/server4/msg-tango/log/default/default

If the upgrade utility fails, there are two recovery procedures you can use depending on whether it completed the process with errors or halted before completion:

Upgrade Completed the Process With Errors

If the errors did not cause the upgrade to halt abnormally, you can correct whatever caused the problem and then resume message migration by running upgrade -m (or upgrade -u) again.

Upgrade Failed or Was Halted Before Completion

If upgrade failed without completing the process or was halted abnormally, follow these steps which will remove all migrated mailbox and perform the entire upgrade process from scratch:

  1. Start the message store and then shut it down after it successfully starts. This action cleans up database locking and makes sure that there are no database problems. For example:

    2. /usr/netscape/server4/ms-tango/start-msg store

    /usr/netscape/server4/ms-tango/stop-msg store

    where /usr/netscape/server4 is the server-root directory and msg-tango is the server instance directory.

  2. Remove all files in the server-root/msg-instance/store/mboxlist directory.

    3. For example, remove all files in the directory:

    /usr/netscape/server4/msg-tango/store/mboxlist

  3. Remove all files and subdirectories from the server-root/msg-instance/store/partition/primary directory.

    4. For example, remove all files and subdirectories in the directory:

    /usr/netscape/server4/msg-tango/store/partition/primary

  4. Remove all files and subdirectories from the server-root/msg-instance/store/user directory.

    5. For example, remove all files and subdirectories in the directory:

    /usr/netscape/server4/msg-tango/store/user

  5. Remove the __lock.share file in the server-root/msg-instance/lock directory. (Note that this filename begins with two underscore characters.)

  6. Remove all __up.* files from the old 3.x default mailbox directory.

  7. Remove the upgrade4x.conf file from the old 3.x default mailbox directory.

  8. If they exist, remove the __greeting_done__ message file and the __snmp_done__ configuration file from the old 3.x default mailbox directory.
Using Dynamic Message Upgrade
Using a dynamic message upgrade can significantly reduce the migration downtime for a large mail store (such as those larger than 10GB). A dynamic message upgrade occurs after all folders have been migrated and all configuration information (such as partition, remove message, and so forth) has been set up correctly.

A dynamic message upgrade is executed in response to any process that first accesses the folder (including SMTP, POP, IMAP, STORE, HTTP, or the upgrade utility). During the folder upgrade stage (upgrade -s), all folders are marked "TRANSITION". If the first process to access the folder detects the folder is in TRANSITION, that process immediately searches through the upgrade database to find the 3.x message store mapping and then migrates 3.x messages to the 4.x message store. The 3.x message store will be locked (per folder base) to prevent any other processes from starting a migration.

After you have run upgrade -s successfully and have set up the correct new message store, you can start the messaging server (POP/IMAP/SMTP/HTTP). At the same time, you can also run upgrade -m to start migrating messages.

To set up dynamic upgrade, use the following steps:

  1. Ensure that the STORE process is started and running. (You should do this before starting any upgrade process.)

  2. Run upgrade -s to upgrade all the mail folders.

  3. Using the configutil utility, set up the correct primary or non-primary 4.x mail store path (if necessary).

  4. Start up Messaging Server 4.1.

  5. The mailbox is marked as in "TRANSITION" and the service to first access the folder initiates 3.x message migration.
Migrating a 3.x Message Queue to Messaging Server 4.1
The qconvert utility migrates the Messaging Server 3.x message queue to the Netscape Messaging Server 4.1 format.

If you do not specify the location of the 3.x message queue or the 4.x message queue, the qconvert utility reads the 3.x and 4.1 configuration files to locate the message queue directories.

The utility automatically converts 3.x access rights to 4.x access rights so that users have access to the appropriate files.

For more information, see Appendix A, "Command-line Utilities," in the Messaging Server 4.1 Administrator's Guide.


Upgrading an Older 4.x Server to a Newer 4.1 Server
If you choose to upgrade during the installation process, and your existing Messaging Server is a 4.x version, the installation program detects this automatically. It presents existing file, directory, and configuration values as the defaults during the installation process and fills in missing values with standard default suggestions. You are free to specify non-default values where desired.

Mailboxes and message queues do not require migration for a 4.x to 4.1 system upgrade.

 

© Copyright 1999 Netscape Communications Corp., a subsidiary of America Online, Inc. All rights reserved.