This section provides the information you need to configure the Veritas Cluster Server or Sun Cluster high availability clustering software and prepare it for use with the Messaging Server. It is assumed you have read Chapter 6, Designing for Service Availability, in Sun Java Communications Suite 5 Deployment Planning Guide as well as the appropriate Veritas or Sun Cluster Server documentation for detailed planning, installation instructions, required patches, and other information as needed.
This chapter consists of the following sections:
See What’s New in This Release of Messaging Server in Sun Java Communications Suite 5 Release Notes for the latest supported versions and platforms.
There are different high availability models that can be used with Messaging Server. Three of the more basic ones are:
Each of these models is described in greater detail in the following subsections.
Note that different HA products may or may not support different models. Refer to the HA documentation to determine which models are supported.
The basic asymmetric or hot standby high availability model consists of two clustered host machines or nodes. A logical IP address and associated host name are designated to both nodes.
In this model, only one node is active at any given time; the backup or hot standby node remains idle most of the time. A single shared disk array between both nodes is configured and is mastered by the active or primary node. The message store partitions and Mail Transport Agent (MTA) queues reside on this shared volume.
The preceding figure shows two physical nodes, Physical-A and Physical-B. Before failover, the active node is Physical-A. Upon failover, Physical-B becomes the active node and the shared volume is switched so that it is mastered by Physical-B. All services are stopped on Physical-A and started on Physical-B.
The advantage of this model is that the backup node is dedicated and completely reserved for the primary node. Additionally, there is no resource contention on the backup node when a failover occurs. However, this model also means that the backup node stays idle most of the time and this resource is therefore under utilized.
The basic symmetric or "dual services" high availability model consists of two hosting machines, each with its own logical IP address. Each logical node is associated with one physical node, and each physical node controls one disk array with two storage volumes. One volume is used for its local message store partitions and MTA queues, and the other is a mirror image of its partner's message store partitions and MTA queues.
The following figure shows the symmetric high availability mode. Both nodes are active concurrently, and each node serves as a backup node for the other. Under normal conditions, each node runs only one instance of Messaging Server.
Upon failover, the services on the failing node are shut down and restarted on the backup node. At this point, the backup node is running Messaging Server for both nodes and is managing two separate volumes.
The advantage of this model is that both nodes are active simultaneously, thus fully utilizing machine resources. However, during a failure, the backup node will have more resource contention as it runs services for Messaging Server from both nodes. Therefore, you should repair the failed node as quickly as possible and switch the servers back to their dual services state.
This model also provides a backup storage array. In the event of a disk array failure, its redundant image can be picked up by the service on its backup node.
To configure a symmetric model, you need to install shared binaries on your shared disk. Note that doing so might prevent you from performing rolling upgrades, a feature that enables you to update your system during Messaging Server patch releases. (This feature is planned for future releases.)
The N + 1 or "N over 1" model operates in a multi-node asymmetrical configuration. N logical host names and N shared disk arrays are required. A single backup node is reserved as a hot standby for all the other nodes. The backup node is capable of concurrently running Messaging Server from the N nodes.
The figure below illustrates the basic N + 1 high availability model.
Upon failover of one or more active nodes, the backup node picks up the failing node's responsibilities.
The advantages of the N + 1 model are that the server load can be distributed to multiple nodes and that only one backup node is necessary to sustain all the possible node failures. Thus, the machine idle ratio is 1/N as opposed to 1/1, as is the case in a single asymmetric model.
To configure an N+1 model, you need to install binaries only on the local disks (that is, not shared disks as with the symmetric model). The current Messaging Server installation and setup process forces you to put the binaries on the shared disk for any symmetric, 1+1, or N+1 asymmetrical or symmetrical HA solution.
The following table summarizes the advantages and disadvantages of each high availability model. Use this information to help you determine which model is right for your deployment.
Table 3–1 HA Model Comparison
The following table illustrates the probability that on any given day the messaging service will be unavailable due to system failure. These calculations assume that on average, each server goes down for one day every three months due to either a system crash or server hang, and that each storage device goes down one day every 12 months. These calculations also ignore the small probability of both nodes being down simultaneously.
Table 3–2 HA Down Probabilities
Model |
Server Down Time Probability |
Single server (no high availability) |
Pr(down) = (4 days of system down + 1 day of storage down)/365 = 1.37% |
Asymmetric |
Pr(down) = (0 days of system down + 1 day of storage down)/365 = 0.27% |
Symmetric |
Pr(down) = (0 days of system down + 0 days of storage down)/365 = (near 0) |
N + 1 Asymmetric |
Pr(down) = (5 hours of system down + 1 day of storage down)/(365xN) = 0.27%/N |
After selecting the appropriate HA model for your deployment, you'll want to choose between the Sun Cluster HA or the Veritas HA. This section provides preliminary HA deployment information. Subsequent sections will provide specific information on Sun Cluster and Veritas High Availability solutions.
A cluster agent is a Messaging Server program that runs under the cluster framework.
The Sun Cluster Messaging Server agent (SUNWscims) is installed when you select Sun Cluster through the Java Enterprise System installer. The Veritas Cluster Messaging Server agent (SUNWmsgvc) can be found in the Messaging Server Product subdirectory on theSun Java Communications Suite CD, Solaris_sparc/Product/messaging_svr/Packages/SUNWmsgvc. (Note that you must use the pkgadd(1M) command to install the VCS cluster agent.)
Some items of note regarding the Messaging Server and high availability (applies to both Veritas Cluster and Sun Cluster) installation:
High availability for the Messaging Server is not installed by default; be sure to select High Availability Components from the Custom Installation menu of the Java Enterprise System installer.
When running the installation, make sure that the HA logical host name and associated IP addresses for Messaging Servers are functioning (for example, active). The reason for this is because portions of the installation will make TCP connections using them. Run the installation on the cluster node currently pointed at by the HA logical host name for the messaging server.
Make sure that the msg_svr_base is on the shared file system; otherwise, high availability will not work correctly. For example, after failing over to another node, the servers will no longer see the data accumulated by the servers on the failed node.
When you are asked for the fully-qualified domain name of the Messaging Server host during initial runtime configuration, be sure to specify the fully-qualified HA logical host name for the Messaging Server. During the install, TCP connections using this logical host name will be attempted.
When you are asked for the IP address of Messaging Server when you run ha_ip_config, be sure to specify the IP address associated with the logical host name for Messaging Server. Do not use the IP address for the physical host.
Clustering software needs to be installed before installing and configuring the current version of Messaging Server. Run the installation on the cluster node currently pointed at by the HA logical host name for Messaging Server. Use the cluster alias when prompted for any node names.
When running the Messaging Server Initial Runtime Configuration (see 1.3 Creating the Initial Messaging Server Runtime Configuration) sure to specify the fully-qualified HA logical host name of the cluster of Messaging Server.
Configure Messaging Server using the cluster host name. If you do otherwise, then you’ll need to re-configure a second time using the cluster host name.
The useconfig utility allows you to share a single configuration between multiple nodes in an HA environment. This utility is not meant to upgrade or update an existing configuration.
For example, if you are upgrading your first node, you will install through the Communications Suite Installer and then configure Messaging Server. You will then failover to the second node where you will install the Messaging Server package through the Communications Suite Installer, but you will not have to run the Initial Runtime Configuration Program (configure) again. Instead, you can use the useconfig utility.
To enable the utility, run useconfig to point to your previous Messaging Server configuration:
msg-svr-base/sbin/useconfig install/configure_YYYYMMDDHHMMSS |
where configure_YYYYMMDDHHMMSS is your previous configuration settings file.
On a brand new node, you can find the configure_YYYYMMDDHHMMSS in the msg-svr-base/data/setup directory on the shared disk.
The following sections on 3.5 Veritas Cluster Server Agent Installation and 3.4 Sun Cluster Installation describe when you can use the useconfig utility.
This section describes how to install and configure the Messaging Server as a Sun Cluster Highly Available (HA) Data Service. The following topics are covered in this section:
See also Sun Cluster documentation.
Note that Veritas File System (VxFS) is supported with Sun Cluster 3.1.
This section presumes the following:
Sun Cluster is installed and configured on Solaris operating system with required patches.
The Sun Cluster agent SUNWscims is installed on your systems.
If logical volumes are being created, either Solstice DiskSuite or Veritas Volume Manager is used.
It is highly recommended that you use the HAStoragePlus resource type to make locally mounted file systems highly available within a Sun Cluster environment. Local file systems, also called Failover File Systems (FFS) provide better Input/Output performance than cluster file systems (CFS), also called global file systems. HAStoragePlus supports both FFS and CFS. In contrast. HAStorage only supports CFS.
HAStoragePlus has a number of benefits:
HAStoragePlus bypasses the global file service layer completely. For disk-IO intensive data services, this leads to a significant performance increase.
HAStoragePlus can work with any file system (like UFS, VxFS, and so forth), even those that might not work with the global file service layer. If a file system is supported by the Solaris operating system, it will work with HAStoragePlus.
To determine whether to create HAStorage or HAStoragePlus resources in a data service resource group, consider the following criteria:
Use HAStoragePlus if you are using Sun Cluster 3.0 Release May 2002 or Sun Cluster 3.1
Use HAStorage if you are using Sun Cluster 3.0 December 2001 or earlier
For more information on HAStoragePlus, refer to appropriate Sun Cluster docs. For example, http://docs.sun.com/app/docs/coll/573.10.
This section describes how to configure HAStorage and HAStoragePlus for Messaging Server for Sun Cluster. The first section describes the general steps. Subsequent sections show specific examples for symmetric and asymmetric deployments.
After configuring HA, be sure to review 3.4.4 Binding IP Addresses on a Server for additional steps associated with HA support.
The following description assumes that Messaging Server has been configured with an HA logical host name and IP address. The physical host names are assumed to be mars and venus, with an HA logical host name of meadow. Figure 3–4 depicts the nested dependencies of the different HA resources you will create in configuring Messaging Server HA support.
While we describe how to configure HAStorage and HAStoragePlus, we highly recommend HAStoragePlus better I/O performance. See 3.4.2 About HAStoragePlus.
This section contains the following subsections:
To Configure Messaging Server with Sun Cluster HAStorage or HAStoragePlus—Generic Example
To Unconfigure Messaging Server HA Support for Sun Cluster 3.x—Generic Example
To Configure a Two-node HA Asymmetric Messaging Server—Example
This section provides the generic steps for configuring Messaging Server for HA. After reviewing these steps, refer to the specific asymmetric or symmetric examples in the following sections. In these instructions the physical hosts are called mars and venus. The logical host name is meadow.
Figure 3–4 depicts the nested dependencies of the different HA resources you will create in configuring Messaging Server HA support.
Become the superuser and open a console.
All of the following Sun Cluster commands require that you have logged in as superuser. You will also want to have a console or window for viewing messages output to /dev/console.
On all the nodes, install the required Messaging Sun Cluster Data Service Agents package (SUNWscims).
On each node in the cluster create the Messaging Server runtime user and group under which the Messaging Server will run.
The user ID and group ID numbers must be the same on all nodes in the cluster. The runtime user ID is the user name under which Messaging Server runs. This name should not be root. The default is mailsrv. The runtime Group ID is the group under which Messaging Server runs. The default is mail
Although the configure utility can create these names for you, you can also create them before running configure as part of the preparation of each node as described in this chapter. The runtime user and group ID names must be in the following files:
mailsrv, or the name you select, must in /etc/passwd on all nodes in the cluster
mail, or the name you select, must be in /etc/group on all nodes in the cluster
Add required resource types to Sun Cluster.
Configure Sun Cluster to know about the resources types we will be using. To register Messaging Server as your resource use the following command:
# scrgadm -a -t SUNW.ims |
To register HAStoragePlus as a resource type, use this command:
# scrgadm -a -t SUNW.HAStoragePlus |
To do the same with HAStorage as a resource type, use this command:
# scrgadm -a -t SUNW.HAStorage |
Create a failover resource group for the Messaging Server.
If you have not done so already, create a resource group and make it visible on the cluster nodes which will run the Messaging Server. The following command creates a resource group named MAIL-RG, making it visible on the cluster nodes mars and venus:
# scrgadm -a -g MAIL-RG -h mars,venus
You may, of course, use whatever name you wish for the resource group.
Create an HA logical host name resource and bring it on-line.
If you have not done so, create and enable a resource for the HA logical host name placing that resource in the resource group. The following command does so using the logical host name meadow. Since the -j switch is omitted, the name of the resource created will also be meadow. meadow is the logical host name by which clients communicate with the services in the resource group.
# scrgadm -a -L -g MAIL-RG -l meadow # scswitch -Z -g MAIL-RG |
Create an HAStorage or HAStoragePlus resource.
Next, you need to create an HA Storage or HAStoragePlus resource type for the file systems on which Messaging Server is dependent. The following command creates an HAStoragePlus resource named disk-rs, and the file system disk_sys_mount_point is placed under its control:
# scrgadm -a -j disk-rs -g MAIL-RG \ -t SUNW.HAStoragePlus \ -x FilesystemMountPoints=disk_sys_mount_point-1, disk_sys_mount_point-2 -x AffinityOn=True |
SUNW.HAStoragePlus represents the device groups, cluster and local file systems which are to be used by one or more data service resources. One adds a resource of type SUNW.HAStoragePlus to a resource group and sets up dependencies between other resources and the SUNW.HAStoragePlus resource. These dependencies ensure that the data service resources are brought online after:
All specified device services are available (and collocated if necessary)
All specified file systems are mounted following their checks
The FilesystemMountPoints extension property allows for the specification of either global or local file systems. That is, file systems that are either accessible from all nodes of a cluster or from a single cluster node. Local file systems managed by a SUNW.HAStoragePlus resource are mounted on a single cluster node and require the underlying devices to be Sun Cluster global devices. SUNW.HAStoragePlus resources specifying local file systems can only belong in a failover resource group with affinity switch overs enabled. These local file systems can therefore be termed failover file systems. Both local and global file system mount points can be specified together.
A file system whose mount point is present in the FilesystemMountPoints extension property is assumed to be local if its /etc/vfstab entry satisfies both of the following conditions:
Non-global mount option
Mount at boot flag is set to no
Instances of the SUNW.HAStoragePlus resource type ignore the mount at boot flag for global file systems.
For the HAStoragePlus resource, the comma-separated list of FilesystemMountPoints are the mount points of the Cluster File Systems (CFS) or Failover File Systems (FFS) on which Messaging Server is dependent. In the above example, only two mount points, disk_sys_mount_point-1 and disk_sys_mount_point-2, are specified. If one of the servers has additional file systems on which it is dependent, then you can create an additional HA storage resource and indicate this additional dependency in Step 15.
For HAStorage use the following:
# scrgadm -a -j disk-rs -g MAIL-RG \ -t SUNW.HAStorage -x ServicePaths=disk_sys_mount_point-1, disk_sys_mount_point-2 -x AffinityOn=True |
For the HAStorage resource, the comma-separated list of ServicePaths are the mount points of the cluster file systems on which Messaging Server is dependent. In the above example, only two mount points, disk_sys_mount_point-1 and disk_sys_mount_point-2, are specified. If one of the servers has additional file systems on which it is dependent, then you can create an additional HA storage resource and indicate this additional dependency in Step 15.
Install the required Messaging Server packages on the primary node. Choose the Configure Later option.
Use the Communications Suite installer to install the Messaging Server packages.
For symmetric deployments: Install Messaging Server binaries and configuration data on files systems mounted on a shared disk of the Sun Cluster. For example, for Messaging Server binaries could be under /disk_sys_mount_point-1/SUNWmsgsr and the configuration data could be under /disk_sys_mount_point-2/config.
For asymmetric deployments: Install Messaging Server binaries on local file systems on each node of the Sun Cluster. Install configuration data on a shared disk. For example, the configuration data could be under /disk_sys_mount_point-2/config.
Configure the Messaging Server. See 1.3 Creating the Initial Messaging Server Runtime Configuration.
In the initial runtime configuration, you are asked for the Fully Qualified Host Name. You must use the HA logical hostname instead of the physical hostname.
In the initial runtime configuration, you are asked to specify a configuration directory in 1.3 Creating the Initial Messaging Server Runtime Configuration. Be sure to use the shared disk directory path of your HAStorage or HAStoragePlus resource.
Run the ha_ip_config script to set service.listenaddr and service.http.smtphost and to configure the dispatcher.cnf and job_controller.cnf files for high availability.
The script ensures that the logical IP address is set for these parameters and files, rather than the physical IP address. It also enables the watcher process (sets local.watcher.enable to 1), and auto restart process (local.autorestart to 1).
For instructions on running the script, see 3.4.4 Binding IP Addresses on a Server.
The ha_ip_config script should only be run once on the primary node.
Modify the imta.cnf file and replace all occurrences of the physical hostname with the logical hostname of the cluster.
Fail the resource group from the primary to the secondary cluster node in order to ensure that the failover works properly.
Manually fail the resource group over to another cluster node. (Be sure you have superuser privileges on the node to which you failover.)
Use the scstat command to see what node the resource group is currently running on (“online” on). For instance, if it is online on mars, then fail it over to venus with the command:
# scswitch -z -g MAIL-RG -h venus
If you are upgrading your first node, you will install through theCommunications Suite Installer and then configure Messaging Server. You will then failover to the second node where you will install the Messaging Server package through the Communications Suite Installer, but you will not have to run the Initial Runtime Configuration Program (configure) again. Instead, you can use the useconfig utility.
Install the required Messaging Server packages on the secondary node. Choose the Configure Later option.
After failing over to the second node, install the Messaging Server packages using the Communications Suite Installer.
For symmetric deployments: Do not install Messaging Server.
For asymmetric deployments: Install Messaging Server binaries on local file systems on the local file system.
Run useconfig on the second node of the cluster.
The useconfig utility allows you to share a single configuration between multiple nodes in an HA environment. You don't need to run the initial runtime configure program (configure). Instead use the useconfig utility (see 3.3.3 Using the useconfig Utility.
Create an HA Messaging Server resource.
It’s now time to create the HA Messaging Server resource and add it to the resource group. This resource is dependent upon the HA logical host name and HA disk resource.
In creating the HA Messaging Server resource, we need to indicate the path to the Messaging Server top-level directory—the msg-svr-base path. These are done with the IMS_serverroot extension properties as shown in the following command.
# scrgadm -a -j mail-rs -t SUNW.ims -g MAIL-RG \ -x IMS_serverroot=msg-svr-base \ -y Resource_dependencies=disk-rs,meadow |
The above command, creates an HA Messaging Server resource named mail-rs for the Messaging Server, which is installed on IMS_serverroot in the msg-svr-base directory. The HA Messaging Server resource is dependent upon the HA disk resource disk-rs as well as the HA logical host name meadow.
If the Messaging Server has additional file system dependencies, then you can create an additional HA storage resource for those file systems. Be sure to include that additional HA storage resource name in the Resource_dependencies option of the above command.
Enable the Messaging Server resource.
It’s now time to activate the HA Messaging Server resource, thereby bringing the messaging server online. To do this, use the command
# scswitch -e -j mail-rs
The above command enables the mail-rs resource of the MAIL-RG resource group. Since the MAIL-RG resource was previously brought online, the above command also brings mail-rs online.
Verify that things are working.
Use the scstat -pvv command to see if the MAIL-RG resource group is online.
You may also want to look at the output directed to the console device for any diagnostic information. Also look in the syslog file, /var/adm/messages. For more debugging options and information, refer to 3.4.3.1 How to Enable Debugging on Sun Cluster.
This section describes how to undo the HA configuration for Sun Cluster. This section assumes the simple example configuration (described in the 3.4 Sun Cluster Installation (for example, Step 3) may be different but will otherwise follow the same logical order.
Become the superuser.
All of the following Sun Cluster commands require that you be running as user superuser.
Bring the resource group offline.
To shut down all of the resources in the resource group, issue the command
# scswitch -F -g MAIL-RG
This shuts down all resources within the resource group (for example, the Messaging Server and the HA logical host name).
Disable the individual resources.
Next, remove the resources one-by-one from the resource group with the commands:
# scswitch -n -j mail-rs # scswitch -n -j disk-rs # scswitch -n -j budgie |
Remove the individual resources from the resource group.
Once the resources have been disabled, you may remove them one-by-one from the resource group with the commands:
# scrgadm -r -j mail-rs # scrgadm -r -j disk-rs # scrgadm -r -j budgie |
Remove the resource group.
Once the all the resources have been removed from the resource group, the resource group itself may be removed with the command:
# scrgadm -r -g MAIL-RG
Remove the resource types (optional).
Should you need to remove the resource types from the cluster, issue the commands:
# scrgadm -r -t SUNW.ims # scrgadm -r -t SUNW.HAStoragePlus |
In this example we assume two cluster nodes with the physical hostnames mars.red.siroe.com and venus.red.siroe.com. The installation and configuration directory locations need to be unique. A contention problem will occur if the installation and configuration directories on each node have the same directory names, for example /opt/SUNWmsgsr and /var/opt/SUNWmsgsr. The contention problem occurs when venus does a failover to mars, and two instances of Messaging Server compete with the same install and configuration directories.
A good practice for creating unique names for the installation and configuration directories would be to use the format /opt/NodeMember/SUNWmsgsr for the installation directory and /var/opt/NodeMember/SUNWmsgsr for the configuration directory. You can use any directory to install your binaries and configuration data as long as they are unique.
In this example we assume two cluster nodes with physical hostnames mars.red.siroe.com and venus.red.siroe.com.
For mars.red.siroe.com, binaries are installed at /opt/mars/SUNWmsgsr and configuration data is installed at /var/opt/mars/SUNWmsgsr.
For venus.red.siroe.com, binaries are installed at /opt/venus/SUNWmsgsr and configuration data is installed at /var/opt/venus/SUNWmsgsr.
We have two logical hostnames called meadow and pasture with their respective logical IP addresses. For example, the /etc/hosts file on both nodes look like this:
192.18.75.155 meadow.red.siroe.com meadow 192.18.75.157 pasture.red.siroe.com pasture |
Install the Messaging Server Sun Cluster agent package (SUNWscims) on both nodes.
Create four file systems.
These files systems can either be Cluster File Systems or local file systems (Failover File Systems).
/var/opt/mars/SUNWmsgsr /var/opt/venus/SUNWmsgsr /opt/mars/SUNWmsgsr /opt/venus/SUNWmsgsr |
These files systems should be mounted on a shared disk. For example below we show four Cluster File Systems. The contents of the/etc/vfstab shown below should be similar on all nodes of the cluster.
# cat /etc/vfstab #device device mount FS fsck mount mount to mount to fsck point type pass at_boot_options /dev/md/penguin/dsk/d500 /dev/md/penguin/rdsk/d500 /opt/mars/SUNWmsgsr ufs 2 yes logging,global /dev/md/penguin/dsk/d400 /dev/md/penguin/rdsk/d400 /var/opt/mars/SUNWmsgsr ufs 2 yes logging,global /dev/md/polarbear/dsk/d200 /dev/md/polarbear/rdsk/d200 /opt/venus/SUNWmsgsr ufs 2 yes logging,global /dev/md/polarbear/dsk/d300 /dev/md/polarbear/rdsk/d300 /var/opt/venus/SUNWmsgsr ufs 2 yes logging,global |
If you want to make the four file systems shown above as local file systems (Fail Over File Systems), set the mount at boot option to no and remove the mount option global keyword:
|
Configure the primary node
Add the required resource types on the primary node.
This configures Sun Cluster to know about the resources types that will be used. To register Messaging Server and the HAStoragePlus resource, use the following commands:
# scrgadm -a -t SUNW.HAStoragePlus # scrgadm -a -t SUNW.ims |
Create a fail over resource group for Messaging Server called MS_RG_MARS.
# scrgadm -a -g MS_RG_MARS -h mars,venus |
Create one logical hostname resource called meadow, add it to the resource group and bring it on-line.
# scrgadm -a -L -g MS_RG_MARS -l meadow # scrgadm -c -j meadow -y R_description="LogicalHostname resource for meadow" # scswitch -Z -g MS_RG_MARS |
Create a HAStoragePlus resource called ms-hasp-mars with the file systems created earlier.
# scrgadm -a -j ms-hasp-mars -g MS_RG_MARS -t SUNW.HAStoragePlus -x FileSystemMountPoints ="/opt/mars/SUNWmsgsr, /var/opt/mars/SUNWmsgsr" -x AffinityOn=TRUE |
Enable the HAStoragePlus resource:
# scswitch -e -j ms-hasp-mars |
Install the Messaging Server on the primary node.
Install the Messaging Server packages using Communications Suite installer. Make sure you install the Messaging Server binaries and configuration data on the shared file system (see Step 2). For example, for this instance of Messaging Server, the messaging binaries are under /opt/mars/SUNWmsgsr and the configuration data is under /var/opt/mars/SUNWmsgsr.
Install and configure the Messaging Server on the primary node (see 1.3 Creating the Initial Messaging Server Runtime Configuration).
The initial runtime configuration program asks for the Fully Qualified Host Name. Enter the logical hostname meadow.red.siroe.com. The program also asks to specify a configuration directory. Enter /var/opt/mars/SUNWmsgsr.
Run the ha_ip_config script on the primary node and provide the logical IP address.
This is only run on the primary node and not on the secondary node. The ha_ip_config script is located under the installation directory under the sbin directory. For example:
# /opt/mars/SUNWmsgsr/sbin/ha_ip_config Please specify the IP address assigned to the HA logical host name. Use dotted decimal form, a.b.c.d Logical IP address: 192.18.75.155 # This value is the logical IP address of the logical hostname. Refer # to the /etc/hosts file. Please specify the path to the top level directory in which iMS is installed. iMS server root: /opt/mars/SUNWmsgsr . . . Updating the file /opt/mars/SUNWmsgsr/config/dispatcher.cnf Updating the file /opt/mars/SUNWmsgsr/config/job_controller.cnf Setting the service.listenaddr configutil parameter Setting the local.snmp.listenaddr configutil parameter Setting the service.http.smtphost configutil parameter Setting the local.watcher.enable configutil parameter Setting the local.autorestart configutil parameter Setting the metermaid.config.bindaddr configutil parameters Setting the metermaid.config.serveraddr configutil parameters Setting the local.ens.port parameter Configuration successfully updated |
Modify the imta.cnf file and replace all occurrences of the physical hostname, that is, mars, with the HA logical host name (meadow).
Fail over the resource group to the secondary node (venus).
After failing over, you will then configure the secondary node (venus).
# scswitch -z -g MS_RG_VENUS -h mars |
On the secondary node (venus) run useconfig utility. See 3.3.3 Using the useconfig Utility
You do not have to run the initial runtime configuration program (configure) or install the Messaging Server packages.
In the following example, /var/opt/mars/SUNWmsgsr is the shared configuration directory.
# useconfig /var/opt/mars/SUNWmsgsr/setup/configure_20061201124116 cp /var/opt/mars/SUNWmsgsr/setup/configure_20061201124116/Devsetup.properties /opt/mars/SUNWmsgsr/lib/config-templates/Devsetup.properties /usr/sbin/groupadd mail /usr/sbin/useradd -g mail -d / mailsrv /usr/sbin/usermod -G mail mailsrv sed -e "s/local.serveruid/mailsrv/" -e "s/local.serveruid/mail/" -e "s:<msg·RootPath>:/opt/mars/SUNWmsgsr:" /opt/mars/SUNWmsgsr/lib/config-templates/devtypes.txt.template > /opt/mars/SUNWmsgsr/lib/config-templates/devtypes.txt sed -e "s/local.serveruid/mailsrv/" -e "s/local.serveruid/mail/" -e "s:<msg·RootPath>:/opt/mars/SUNWmsgsr:" /opt/mars/SUNWmsgsr/lib/config-templates/config.ins.template > /opt/mars/SUNWmsgsr/lib/config-templates/config.ins /opt/mars/SUNWmsgsr/lib/devinstall -l sepadmsvr:pkgcfg:config -v -m -i /opt/mars/SUNWmsgsr/lib/config-templates/config.ins /opt/mars/SUNWmsgsr/lib/config-templates /opt/mars/SUNWmsgsr/lib/jars /opt/mars/SUNWmsgsr/lib devinstall returned 0 crle -c /var/ld/ld.config -s /usr/lib/secure:/opt/SUNWmsgsr/lib:/opt/venus/SUNWmsgsr/lib:/opt/mars/SUNWmsgsr/lib -s /opt/mars/SUNWmsgsr/lib See /opt/mars/SUNWmsgsr/install/useconfiglog_20061211155037 for more details |
Create the HA Messaging Server Resource and enable it.
# scrgadm -a -j ms-rs-mars -t SUNW.ims -g MS_RG_MARS -x IMS_serverroot =/opt/mars/SUNWmsgsr -y Resource_dependencies=meadow,ms-hasp-mars # scswitch -e -j mail-rs-mars |
The above command, creates an HA Messaging Server resource named ms-rs-mars for the Messaging Server, which is installed on /opt/mars/SUNWmsgsr. This HA Messaging Server resource is dependent upon the HA disk resource, that is, the file systems created earlier as well as the HA logical host name meadow.
Verify that everything is working.
Failover the Messaging Server resource back to the primary node.
# scswitch -z -g MAIL-RG -h mars |
Similarly, create another failover resource group for the second instance of Messaging Server with venus as the primary and mars as the secondary (or standby node).
Repeat the steps from 3 to 10 with venus as the primary node for this resource group, MS_RG_VENUS as the resource group, pasture as the logical hostname and ms-hasp-venus as the HAStoragePlus resource. Thus the commands will look like this:
To create the resource group MS_RG_VENUS:
# scrgadm -a -g MS_RG_VENUS -h venus,mars |
To create a logical hostname resource called pasture, add it to the resource group and bring it on-line;
# scrgadm -a -L -g MS_RG_VENUS -l pasture # scrgadm -c -j pasture -y R_description="LogicalHostname resource for pasture" # scswitch -Z -g MS_RG_VENUS |
To create an HAStoragePlus resource called ms-hasp-venus with the file systems created earlier:
# scrgadm -a -j ms-hasp-venus -g MS_RG_VENUS -t SUNW.HAStoragePlus -x FileSystemMountPoints ="/opt/venus/SUNWmsgsr, /var/opt/venus/SUNWmsgsr" -x AffinityOn=TRUE |
To enable the HAStoragePlus resource:
# scswitch -e -j ms-hasp-venus |
To run the ha_ip_config script on the primary node and provide the logical IP address:
# /opt/venus/SUNWmsgsr/sbin/ha_ip_config |
To create the HA Messaging Server Resource and enable it:
# scrgadm -a -j ms-rs-venus -t SUNW.ims -g MS_RG_VENUS -x IMS_serverroot =/opt/venus/SUNWmsgsr -y Resource_dependencies=pasture,ms-hasp-venus # scswitch -e -j mail-rs-venus |
To fail over the resource group to the secondary node (venus):
# scswitch -z -g MS_RG_MARS -h venus |
To run useconfig on the secondary node (mars) run useconfig utility:
# useconfig /var/opt/venus/SUNWmsgsr/setup/configure_20061201124116 |
To verify that everything is working by failing over the Messaging Server resource back to the primary node:
# scswitch -z -g MAIL-RG -h venus |
Unconfiguring is done when you need to upgrade the Messaging Server or the Sun Cluster, or when you need to uninstall the Messaging Server. It is assumed that the system was configured the using the previous example.
The first step is to remove each resource group in the cluster. In the example, there are two resource groups, MS_RG_MARS and MS_RG_VENUS. Both must be removed.
Remove resource group MS_RG_MARS from the cluster.
Use the following commands on just one node. You do not have to do this on each node.
Brings the resource group off-line on all nodes of the cluster:
# scswitch -F -g MS_RG_MARS |
Disable all the specific Messaging Server resources:
# scswitch -n -j ms-rs-mars # scswitch -n -j meadow # scswitch -n -j ms-hasp-mars |
Remove all the specific MS resources:
# scrgadm -r -j ms-rs-mars # scrgadm -r -j meadow # scrgadm -r -j ms-hasp-mars |
Remove the resource group:
scrgadm -r -g MS_RG_MARS |
Remove resource group MS_RG_VENUS from the cluster.
Use the following commands on just one node. You do not have to do this on each node.
Brings the resource group off-line on all nodes of the cluster:
# scswitch -F -g MS_RG_VENUS |
Disable all the specific Messaging Server resources:
# scswitch -n -j ms-rs-venus # scswitch -n -j pasture # scswitch -n -j ms-hasp-venus |
Remove all the specific MS resources:
# scrgadm -r -j ms-rs-venus # scrgadm -r -j pasture # scrgadm -r -j ms-hasp-venus |
Remove the resource group:
scrgadm -r -g MS_RG_VENUS |
Unregister the Resource Types that are not in use.
# scrgadm -r -t SUNW.HAStoragePlus # scrgadm -r -t SUNW.ims |
In this example we assume two node cluster with physical hostnames daisy.red.siroe.com and lavender.red.siroe.com with a logical hostname called budgie.
For daisy.red.siroe.com, binaries are installed at /opt/SUNWmsgsr and configuration data is installed at /var/opt/SUNWmsgsr.
The logical hostname budgie is assigned logical IP address. For example, the /etc/hosts file could look like this:
192.18.75.157 budgie.red.siroe.com budgie |
Install the Messaging Sun Cluster agents (SUNWscims) on both nodes.
Create the file system.
In this example, the file system /var/opt/SUNWmsgsr is mounted on a shared disk. This file system can either be a Cluster File System or local file systems (Failover File Systems).
Configure the primary node (daisy).
Add the required resource types on the primary node.
This configures Sun Cluster to know about the resources types that will be used. To register Messaging Server and the HAStoragePlus resource, use the following commands:
# scrgadm -a -t SUNW.HAStoragePlus # scrgadm -a -t SUNW.ims |
Create a resource group for the Messaging Server instance called MS_RG_DAISY.
# scrgadm -a -g MS_RG_daisy -h daisy,lavender |
Create a logical hostname resource called meadow, add it to the resource group and bring it on-line.
# scrgadm -a -L -g MS_RG_DAISY -l meadow # scrgadm -c -j meadow -y R_description="LogicalHostname resource for meadow" # scswitch -Z -g MS_RG_DAISY |
Create an HAStoragePlus resource called ms-hasp-daisy with the file systems created earlier.
# scrgadm -a -j ms-hasp-daisy -g MS_RG_DAISY -t SUNW.HAStoragePlus -x FileSystemMountPoints ="/var/opt/SUNWmsgsr" -x AffinityOn=TRUE |
Enable the HAStoragePlus resource:
# scswitch -e -j ms-hasp-daisy |
Install and configure the Messaging Server on the primary node (see 1.3 Creating the Initial Messaging Server Runtime Configuration).
The initial runtime configuration program asks for the Fully Qualified Host Name. Enter the logical hostname meadow.red.siroe.com. The program also asks to specify a configuration directory. Enter /var/opt/SUNWmsgsr.
Run the ha_ip_config script on the primary node and provide the logical IP address.
This is only run on the primary node and not on the secondary node. The ha_ip_config script is located under the installation directory under the sbin directory. For Example:
# /opt/SUNWmsgsr/sbin/ha_ip_config Please specify the IP address assigned to the HA logical host name. Use dotted decimal form, a.b.c.d Logical IP address: 192.18.75.155 # This value is the logical IP address of the logical hostname. Refer # to the /etc/hosts file. Please specify the path to the top level directory in which iMS is installed. iMS server root: /opt/SUNWmsgsr . . . Updating the file /opt/SUNWmsgsr/config/dispatcher.cnf Updating the file /opt/SUNWmsgsr/config/job_controller.cnf Setting the service.listenaddr configutil parameter Setting the local.snmp.listenaddr configutil parameter Setting the service.http.smtphost configutil parameter Setting the local.watcher.enable configutil parameter Setting the local.autorestart configutil parameter Setting the metermaid.config.bindaddr configutil parameters Setting the metermaid.config.serveraddr configutil parameters Setting the local.ens.port parameter Configuration successfully updated |
Modify the imta.cnf file and replace all occurrences of the physical hostname (daisy) with the HA logical host name (meadow).
Fail over the resource group to the secondary node (lavender).
After failing over, you will then configure the secondary node (lavender).
# scswitch -z -g MS_RG_LAVENDER -h daisy |
On the secondary node (lavender) install Messaging Server and run the useconfig utility. See 3.3.3 Using the useconfig Utility
You do not have to run the initial runtime configuration program (configure).
In the following example, /var/opt/SUNWmsgsr is the shared configuration directory.
# useconfig /var/opt/SUNWmsgsr/setup/configure_20061201124116 cp /var/opt/SUNWmsgsr/setup/configure_20061201124116/Devsetup.properties /opt/SUNWmsgsr/lib/config-templates/Devsetup.properties /usr/sbin/groupadd mail /usr/sbin/useradd -g mail -d / mailsrv /usr/sbin/usermod -G mail mailsrv sed -e "s/local.serveruid/mailsrv/" -e "s/local.serveruid/mail/" -e "s:<msg·RootPath>:/opt/SUNWmsgsr:" /opt/SUNWmsgsr/lib/config-templates/devtypes.txt.template > /opt/SUNWmsgsr/lib/config-templates/devtypes.txt sed -e "s/local.serveruid/mailsrv/" -e "s/local.serveruid/mail/" -e "s:<msg·RootPath>:/opt/SUNWmsgsr:" /opt//SUNWmsgsr/lib/config-templates/config.ins.template > /opt/SUNWmsgsr/lib/config-templates/config.ins /opt/SUNWmsgsr/lib/devinstall -l sepadmsvr:pkgcfg:config -v -m -i /opt/SUNWmsgsr/lib/config-templates/config.ins /opt/SUNWmsgsr/lib/config-templates /opt/SUNWmsgsr/lib/jars /opt/SUNWmsgsr/lib devinstall returned 0 crle -c /var/ld/ld.config -s /usr/lib/secure:/opt/SUNWmsgsr/lib:/opt/SUNWmsgsr/lib:/opt/SUNWmsgsr/lib -s /opt/SUNWmsgsr/lib See /opt/SUNWmsgsr/install/useconfiglog_20061211155037 for more details |
Create the HA Messaging Server Resource and enable it.
# scrgadm -a -j ms-rs-daisy -t SUNW.ims -g MS_RG_DAISY -x IMS_serverroot =/opt/SUNWmsgsr -y Resource_dependencies=meadow,ms-hasp-daisy # scswitch -e -j mail-rs-daisy |
The above command creates an HAMessaging Server resource named ms-rs-daisy for the Messaging Server, which is installed on /opt/SUNWmsgsr. This HAMessaging Server resource is dependent upon the HA disk resource, that is, the file system created earlier as well as the HA logical host name meadow.
Verify that everything is working.
Failover the Messaging Server resource back to the primary node.
# scswitch -z -g MAIL-RG -h daisy |
Messaging Server Data Service Sun Cluster agents uses two APIs to log debug messages:
scds_syslog_debug()writes a debugging message to the system log at 1 level.
scds_syslog() writes a message to the system log at daemon.notice, daemon.info and daemon.error levels.
All syslog messages are prefixed with the following:
SC[resourceTypeName, resourceGroupName, resourceName,methodName] |
For example:
Dec 11 18:24:46 mars SC[SUNW.ims,MS-RG,mail-rs,ims_svc_start]: [ID 831728daemon.debug] Groupname mail exists. Dec 11 18:24:46 mars SC[SUNW.ims,MS-RG,mail-rs,ims_svc_start]: [ID 383726daemon.debug] Username mailsrv exists. Dec 11 18:24:46 mars SC[SUNW.ims,MS-RG,mail-rs,ims_svc_start]: [ID 244341daemon.debug] IMS_serverroot = /opt/mars/SUNWmsgsr Dec 11 15:55:52 mars SC[SUNW.ims,MS_RG,MessagingResource,ims_svc_validate]: [ID 855581daemon.error] Failed to get the configuration info Dec 11 18:24:46 mars SC[SUNW.ims,MS-RG,mail-rs,ims_svc_start]: [ID 833212daemon.info] Attempting to start the data service under process monitor facility. |
To log messages from the Messaging Server Resource Type SUNW.ims, create the Resource Type Directory under /var/cluster as shown below:
mkdir -p /var/cluster/rgm/rt/SUNW.ims |
To see all debugging messages for resource type SUNW.ims, issue the following command on all the nodes of your cluster:
echo 9 > /var/cluster/rgm/rt/SUNW.ims/loglevel |
To suppress debugging messages for resource type SUNW.iws, issue the following command on all nodes of your cluster:
echo 0 > /var/cluster/rgm/rt/SUNW.ims/loglevel |
To log debug messages from the Sun Cluster Data services and the most common debugging information from the Messaging Server Agents, edit the syslog.conf file. For example, to log all syslog messages to the file /var/adm/clusterlog, add the following line to the syslog.conf file:
daemon.debug /var/adm/clusterlog |
This will log all messages at the following levels (emerg, alert, critical, error, warning, notice, information, debug). See syslog.conf man page for more information
Now restart the syslogd daemon:
pkill -HUP syslogd |
If you are using the Symmetric or N + 1 high availability models, there are some additional things you should be aware of during configuration in order to prepare the Sun Cluster Server for Messaging Server.
Messaging Server running on a server requires that the correct IP address binds it. This is required for proper configuration of Messaging in an HA environment.
Part of configuring Messaging Server for HA involves configuring the interface address on which the Messaging Servers bind and listen for connections. By default, the servers bind to all available interface addresses. However, in an HA environment, you want the servers to bind specifically to the interface address associated with an HA logical host name.
A script is therefore provided to configure the interface address used by the servers belonging to a given Messaging Server instance. Note that the script identifies the interface address by means of the IP address which you have or will be associating with the HA logical host name used by the servers.
The script effects the configuration changes by modifying or creating the following configuration files. For the file
msg-svr-base/config/dispatcher.cnf
it adds or changes INTERFACE_ADDRESS option for the SMTP and SMTP Submit servers. For the file
msg-svr-base/config/job_controller.cnf
it adds or changes the INTERFACE_ADDRESS option for the Job Controller.
Finally it sets the configutil service.listenaddr and service.http.smtphost parameters used by the POP, IMAP, and Messenger Express HTTP servers.
Note that the original configuration files, if any, are renamed to *.pre-ha.
Run the script as follows:
Become superuser.
Execute msg-svr-base/sbin/ha_ip_config
The script presents the questions described below. The script may be aborted by typing control-d in response to any of the questions. Default answers to the questions will appear within square brackets, [ ]. To accept the default answer, simply press the RETURN key.
Logical IP address: Specify the IP address assigned to the logical host name which Messaging Server will be using. The IP address must be specified in dotted decimal form, for example, 123.456.78.90.
The logical IP address is automatically set in the configutil parameter service.http.smtphost which allows you to see which machine is running your messaging system in a cluster. For example, if you are using Messenger Express, your server will be able to determine from which mail host to send outgoing mail.
Messaging Server Base ( msg-svr-base): Specify the absolute path to the top-level directory in which Messaging Server is installed.
Do you wish to change any of the above choices: answer “no” to accept your answers and effect the configuration change. Answer “yes” if you wish to alter your answers.
In addition, the ha_ip_config script automatically enables two new processes watcher and msprobe with the following parameters: local.autorestart and local.watcher.enable. These new parameters help to monitor the health of the messaging server. Process failures and unresponsive services result in log messages indicating specific failures. The cluster agents now monitor the watcher process and failover whenever it exits. Note that the parameters must be enabled in order for Sun Cluster to function properly.
For more information on the watcher and msprobe processes, see 4.5 Automatic Restart of Failed or Unresponsive Services
To enable the Messaging Server Resource:
# scswitch -e -j messaging-resource |
To disable the Messaging Server Resource:
# scswitch -n -j cal-resource |
To list all the resources and the resource groups:
# scstat -pvv |
To determine the process monitoring facility (PMF) tag, that is, the process monitored by PMF:
# pmfadm -L |
To list all the resources and resource groups and their status:
# scstat -g |
To manage the Sun Cluster:
scsetup |
Messaging Server can be configured with Veritas Cluster Server 3.5, 4.0, 4.1, and 5.0.
Be sure to review the Veritas Cluster Server documentation prior to following these procedures.
After installing Messaging Server using the Communications Suite Installer and configuring HA, be sure to review 3.4.4 Binding IP Addresses on a Server for additional steps associated with configuring HA support. This section contains the following subsections:
Veritas Cluster Software is already installed and configured as described in the following instructions (3.5.2 VCS Installation and Configuration Notes) along with the Messaging Server software on both nodes.
The following instructions describe how to configure Messaging Server as an HA service, by using Veritas Cluster Server.
The default main.cf configuration file sets up a resource group called ClusterService that launches the VCSweb application. This group includes network logical host IP resources like csgnic and webip. In addition, the ntfr resource is created for event notification.
Launch Cluster Explorer from one of the nodes.
Note that these Veritas Cluster Server instructions assume you are using the graphical user interface to configure Messaging Server as an HA service.
To launch Cluster Explorer, run the following command:
# /opt/VRTSvcs/bin/hagui |
The VRTScscm package must be installed in order to use the GUI.
Using the cluster explorer, add a service group called MAIL-RG.
Add s1ms_dg disk group resource of type DiskGroup to the service group MAIL-RG and enable it.
Add s1ms_mt mount resource of type Mount to the service group MAIL-RG.
Create a link between s1ms_mt and s1ms_dg. Enable the resource s1ms_mt.
The figure depicts the dependency tree:
Run the Communications Suite Installer to install the Messaging Server.
Run the Messaging Server Initial Runtime Configuration from the primary node (for example, Node_A) to install Messaging Server.
Install the Veritas Cluster Server agent package, SUNWmsgvc, (located in the Messaging Server Product subdirectory on the Sun Java Communications Suite CD) by using the pkgadd(1M) command.
Messaging Server and the Veritas agent are now installed on Node_A.
Switch to the backup node (for example, Node_B).
Run the Communications Suite Installer to install Messaging Server on the backup node (Node_B).
After installing Messaging Server, use the useconfig utility to obviate the need for creating an additional initial runtime configuration on the backup node (Node_B). The useconfig utility allows you to share a single configuration between multiple nodes in an HA environment. This utility is not meant to upgrade or update an existing configuration. See 3.3.3 Using the useconfig Utility.
The Veritas agent is now installed on Node_B.
From the Veritas Cluster Server Cluster Manager, Select Import Types... from the File menu which will display a file selection box.
Import the MsgSrvTypes.cf file from the /etc/VRTSvcs/conf/config directory. Import this type file. Note that you need to be on a cluster node to find this file.
Now create a resource of type MsgSrv (for example, Mail). This resource requires the logical host name property to be set.
The Mail resource depends on s1ms_mt and webip. Create links between the resources as shown in the following dependency tree:
Switch over to Node_A and check if the High Availability configuration is working.
This section describes MsgSrv additional attributes and arguments that govern the behavior of the mail resource. To configure Messaging Server with Veritas Cluster Server, see Table 3–3.
Table 3–3 Veritas Cluster Server Attributes
Attribute |
Description |
---|---|
FaultOnMonitorTimeouts |
If unset (=0), monitor (probe) time outs are not treated as resource fault. Recommend setting this to 2. If the monitor times out twice, the resource will be restarted or failed over. |
ConfInterval |
Time interval over which faults/restarts are counted. Previous history is erased if the service remains online for this duration. Suggest 600 seconds. |
ToleranceLimit |
Number of times the monitor should return OFFLINE for declaring the resource FAULTED. Recommend leaving this value at ”0’ (default). |
Table 3–4 MsgSrv Arguments
Parameter |
Description |
---|---|
State |
Indicates if the service is online or not in this system. This value is not changeable by the user. |
LogHostName |
The logical host name that is associated with this instance. |
PrtStatus |
If set to TRUE, the online status is printed to the Veritas Cluster Server log file. |
DebugMode |
If set to TRUE, the debugging information is sent to the Veritas Cluster Server log file. |
This section describes how to unconfigure high availability. To uninstall high availability, follow the instructions in your Veritas or Sun Cluster documentation.
The High Availability unconfiguration instructions differ depending on whether you are removing Veritas Cluster Server or Sun Cluster.
The following topics are covered in this section:
This section describes how to unconfigure the high availability components for Veritas Cluster Server:
Bring the MAIL-RG service group offline and disable its resources.
Remove the dependencies between the mail resource, the logical_IP resource, and the mountshared resource.
Bring the MAIL-RG service group back online so the sharedg resource is available.
Delete all of the Veritas Cluster Server resources created during installation.
Stop the Veritas Cluster Server and remove following files on both nodes:
/etc/VRTSvcs/conf/config/MsgSrvTypes.cf /opt/VRTSvcs/bin/MsgSrv/online /opt/VRTSvcs/bin/MsgSrv/offline /opt/VRTSvcs/bin/MsgSrv/clean /opt/VRTSvcs/bin/MsgSrv/monitor /opt/VRTSvcs/bin/MsgSrv/sub.pl |
Remove the Messaging Server entries from the /etc/VRTSvcs/conf/config/main.cf file on both nodes.
Remove the /opt/VRTSvcs/bin/MsgSrv/ directory from both nodes.