Previous Contents Index DocHome Next |
iPlanet Application Server Administrator's Guide |
Chapter 13 Managing Distributed Data Synchronization
This chapter describes how to group iPlanet Application Server instances into clusters that participate in data synchronization.The following subjects are described in this chapter:
About Distributed Data Synchronization
About Distributed Data Synchronization
Distributed data synchronization maintains the integrity of shared state and session information across multiple iPlanet Application Server instances. This is crucial for partitioned and distributed applications that are hosted on multiple iPlanet Application Server instances.In most enterprises, several iPlanet Application Server instances support one or more distributed applications. For such distributed applications to run successfully, each server must have access to the relevant information for that application, such as state and session information.
Support for this distribution of information is provided through a system-level distributed data synchronization service that is built into iPlanet Application Server.
How Failover Keeps Data Accessible
The distributed data synchronizer is a system-level service that controls how distributed data, such as application session information, is maintained and made accessible across multiple iPlanet Application Server instances.Each iPlanet Application Server instance is made up of the following four "engines:"
Administrative Server (KAS) - The Administrative Server brings up and monitors the other engines and makes sure that any engines that fail are brought up again.
Executive Server (KXS) - Only an Executive Server can be the primary synchronization engine (the synchronizer) for an iPlanet Application Server cluster.
Zero or more Java Servers (KJS)
- In a cluster of iPlanet Application Server instances, one of the Executive Servers maintains the distributed (synchronized) information and sets up server roles for all the other servers participating in the cluster. All engines in a cluster know how to access this primary engine and the information that is on this primary engine.
Zero or more C++ Servers (KCS)
- If KJS or KCS fails, KAS simply restarts the failed engine. However, if KXS fails, KAS performs the following actions:
What Is a Cluster?
A cluster is a group of iPlanet Application Server instances that share information related to the state of individual user sessions. The primary benefit of using clusters is to improve reliability, to continue user sessions even after process and hardware failures. The secondary benefit is improved load distribution across all available resources.Servers in a cluster can belong to the same network or to different networks, or to different subnets within the same network. While keeping the servers in a cluster on a common subnet enhances performance, this is not necessary. Members in a cluster communicate through TCP/IP, which uses the IP Addresses and port numbers of the servers in the cluster, to access the KAS and the KXS engines of each instance. As long as this communication takes place, the cluster will be operable. See "Using Firewalls for Security", for more information on how this communication takes place.
In a cluster, the state/session data is stored in the memory of the KXS process for the Primary Sync Server. The configuration data of the cluster is stored in Directory Server. All the iPlanet Application Server instances in your cluster can share a single Directory Server; if the iPlanet Application Server instances in your cluster do not share a single Directory Server, cluster settings must be copied from one Directory Server to another so each server has access to identical cluster information.
This section describes the following topics:
Setting Up Data Synchronization
Setting Up Data Synchronization
To set up data synchronization between servers, you need to decide the role that you want a server to perform in the cluster. You can then edit each cluster entry to set up the server roles and register the cluster with the synchronizer service. Finally, you need to start each iPlanet Application Server in the order that is determined by server roles.
Server Roles Within Clusters
You can configure a server's role within a cluster as either a Sync Alternate or a Sync Local. The server that you start first becomes your Primary Sync Server and the server that you start next becomes your Backup Sync Server. This goes on till the number of backup servers you have defined have been started. If you have defined 2 backup servers, then the 2 servers that you start after the Primary Sync Server is started play the role of backup Sync Servers.The next server that is started (once the defined number of backup servers have been started), is your Sync Alternate. This server takes over if the Primary and the backups fail.
See "Example: Coordination Within a Seven-Server Cluster, to learn more about how servers in a cluster coordinate with each other, while performing the roles that they have been assigned.
The roles that the servers in a cluster can perform are described in the following table.
How a Cluster Communicates
Servers in a cluster need to communicate with each other. To enable this communication, it is necessary that each server identifies with the cluster to which it belongs. An iPlanet Application Server instance becomes an active part of a cluster when you map its synchronizer to the cluster. This procedure is described in .When an application component requests "write" access to a distributed data source, the write occurs first on the Sync Primary. When the data changes on the Sync Primary, the Sync Primary immediately updates the Sync Backups.
You can map the synchronizer for each iPlanet Application Server instance to only one cluster at a time.
Information Flow Within a Cluster
Sync Backups, Sync Alternates, and Sync Locals communicate with the Sync Primary in a star configuration, as shown in the following illustration:
In this illustration, notice that all servers are communicating with the Sync Primary. Also note that no Sync Local is assigned a priority number.
Note also that this illustration is an ideal representation of a cluster that has probably just started and has not experienced failover, as the priority numbers correspond gracefully to the currently assigned roles.
Configuring Clusters
You can set up and manage clusters using the iPlanet Application Server Administration Tool (iASAT). iASAT is a stand-alone graphical user interface tool, using which you can manage all the administrative aspects of iPlanet Application Server. When you configure clusters using iASAT, the relevant back-end entries are created in iPlanet Registry, which you can verify and edit.You can also configure clusters directly in iPlanet Registry, using iPlanet Registry Editor. However, it is recommended that iASAT be used to create and manage clusters.
The following sections describe how to create and manage clusters, using both iASAT and iPlanet Registry Editor.
Configuring Clusters Using iASAT
Configuring Clusters Using iPlanet Registry Editor
Configuring Clusters Using iASAT
When you install iPlanet Application Server on your instance, your host server is automatically taken as the default cluster, with a single node. You can add other registered iPlanet Application Server instances to the existing cluster (your host server), or create a new cluster, or create a completely new cluster.This section describes the following topics:
To Create a New Cluster
To Create a New Cluster
To create a new cluster using iASAT, perform the following tasks:
In the iASAT toolbar, click General to open the General window.
When you restart the server, you can add other iPlanet Application Server instances to the new cluster as described in the following section.From the left pane of the General window, select the iPlanet Application Server instance with which you want to create a new cluster.
In the right pane of the General window, click the Cluster tab, as shown in the following figure:
In the Cluster Name drop-down list, delete the default cluster name entry, using either the Backspace or the Delete key.
Provide the name of the new cluster and press Enter. You can choose any unique name for the new cluster.
Click Apply Changes. You need to start the server again for the new cluster configuration to take effect.
To Add a Server to a Cluster
To add an unassigned server to a cluster or to reassign a server to a different cluster using iASAT, perform the following steps:
From the iASAT toolbar, click General to open the General window.
In the left pane of the General window, click the name of the server you want to add to a cluster.
- The following window appears:
- A list of all registered servers is displayed in the left pane of the General window. A list of the server(s) belonging to the existing cluster(s) is displayed in the Priority List of Servers box, in the right pane of the General window. These servers are sorted by their priority in a cluster.
- The Priority List of Servers box also shows the cluster status of a server. Server conditions can be Normal, Dual Primary or No Primary. Click the Refresh List button to immediately update the Priority List of Servers box. By default, this box is updated every 15 seconds.
From the Cluster Name drop-down box, select the name of the cluster to which you want to add the server.
Click Apply Changes.
- The Cluster Name drop-down list is populated with the cluster names to which all registered servers belong.
Stop and start every server in the cluster, including the server you just added.
Note If you remove a server from a cluster and add it to another, you need to restart all the servers in both clusters, for the change in configuration to take effect.
To Remove a Server From a Cluster
To remove a server from a cluster, perform the following steps:
From the iASAT toolbar, click General to open the General window.
From the left pane of the General window, select an iPlanet Application Server that is a member of the cluster from which you want to remove a server.
Click the Clusters tab in the right pane of the General window, as shown in the following figure:
In the Cluster Name drop-down list, select the cluster from which you want to remove a server. The Priority List of Servers box shows all the servers belonging to the selected cluster.
- A list of all registered servers is displayed in the left pane of the General window. A list of the server(s) belonging to the existing cluster(s) is displayed in the Priority List of Servers box, in the right pane of the General window.
In the Priority List of Servers box, select the server you want to delete and click Remove from Cluster.
Shut down and restart every server in the cluster, including the server you just removed.
Configuring Clusters Using iPlanet Registry Editor
When you configure clusters using iASAT, the necessary back-end entries are created in iPlanet Registry. You can directly create these entries in iPlanet Registry and configure clusters, without using iASAT. Although this facility is available, it is strongly recommended that you use iASAT to set up and manage your clusters.This section describes the following topics:
To Create a New Cluster
To Create a New Cluster
When you install iPlanet Application Server, a default cluster is automatically created, using the host server. The easiest way to set up and configure clusters is to modify the back-end entries created in iPlanet Registry for this default cluster.The default cluster bears the name <hostname>-NoDsync, where hostname is the name of your local instance. For instance, if you install iPlanet Application Server on a machine named Bozo, the default cluster entry will be named Bozo-NoDysnc. The default values for this cluster are configured during iPlanet Application Server installation. You can configure the existing default cluster, or configure a completely new cluster, as described in the sections that follow.
To create a new cluster, perform the following tasks:
Start iPlanet Registry. (See "About iPlanet Registry Editor)
When you add the required values to your cluster, make sure that the type for each value is set to Integer. You can do this by selecting Integer from the Type drop-down list, in the Add Value dialog box.
From the Edit menu, choose Add Key. The Add Key dialog box appears, as shown in the following figure:
- SOFTWARE\iPlanet\Application Server\Clusters\
In the Key field, provide a name for the new cluster and press OK. A new cluster is created. You now need to configure the cluster, as described in the ensuing steps.
Select the new cluster that you created, and choose Add Value from the Edit menu. The Add Value dialog box appears, as shown in the following figure:
When you finish adding a value, click OK to confirm. To add another value, select the new cluster you have created and choose Add Value from the Edit menu.
You need to add the following values to the new cluster:
MaxBackups
When you finish adding these values, select the new cluster key and choose Add Key from the Edit menu.
MaxSyncHeartBeat
- This value indicates the maximum number of servers that can be assigned as backup servers. The default value for this key is 0. If you do not specify a number for backup, your session data will not be backed up. While not having a backup increases performance, there is a trade-off with respect to reliability. We recommend that you specify at least one backup server.
- The maximum number of backup servers that you can specify is the total number of servers in your cluster, minus one.
SyncHeartBeatInterval
- This value specifies the maximum number of heartbeat messages that an engine will send to any other engine. The heart-beat mechanism is used to detect an abnormal cluster condition. The default value for this key is 10.
SyncTimerInterval
- This value specifies the number of seconds between two heartbeat messages sent from one server to another. The default value is 30.
- This key specifies the intervals, in seconds, at which the synchronization service wakes up and checks to see whether any data has expired. This key specifies how often the timer thread goes through the node list and removes all the nodes that have expired. The default value is 30.
- See Setting Cluster Parameters, for more information on configuring these values for best results.
In the AddKey dialog box that appears, type SyncServers.
- This creates a folder called SyncServers under the new cluster key that you have created. When you add servers to your cluster, you will need to add them under this key.
- Your new cluster is created and configured. You can now add servers to your cluster, as described in the next section.
Adding a Server to a Cluster
To add servers to a cluster, perform the following tasks:
Start iPlanet Registry.
You can now continue to add more servers to your cluster. When you finish adding servers to your cluster, you need to map the servers to the cluster to which they belong. See Mapping the Synchronizer to the Cluster.
Open the following key:
- (See About iPlanet Registry Editor)
From the Edit menu, choose Add Value. The Add Value dialog box appears, as shown in the following figure:
- SOFTWARE\iPlanet\Application Server\Clusters\<clustername>\SyncServers
In the Name text field, provide the IP Address of the instance which you want to add to the cluster. Type a colon (:) after the IP Address and enter the KXS port number, as shown in the following example:
In the Value text field, provide a value that indicates server priority in the cluster. For example, 0 indicates the highest priority and 1 indicates the next level of priority. The lowest priority value is 65,535.
- host IP address:KXS port number
- Your entry should look like this.
- 129.158.228.63:10800
Select Integer from the Type drop-down list and click OK to register your entry in iPlanet Registry.
To Remove a Server From a Cluster
You can remove a server from a cluster that you have added, by deleting the server's IP Address entry from the SyncServers key.To remove a server from a cluster using iPlanet Registry Editor, perform the following tasks:
Start iPlanet Registry.
Open the following key:
- (See About iPlanet Registry Editor)
To remove a server, select the required IP Address entry and choose Delete from the Edit menu. Alternatively, you can select the required entry and press the Delete key.
- SOFTWARE\iPlanet\Application Server\Clusters\<clustername>\SyncServers
- You will see the IP Address and port number entries of the servers that have been added to the cluster.
You will be prompted to confirm the action. Click Yes to confirm. The server is removed from the cluster.
Note Note that you can add the server that you removed, again to the same cluster, or to a different cluster. See To Add a Server to a Cluster for more information.
You can't rename a cluster in iPlanet Registry. You will have to delete the cluster name entry and create a new cluster configuration. See To Create a New Cluster for more information.
Determining Sync Server Priority
You can set priority values to sync servers in a cluster. Priority values are used to select between Sync Servers in the same status (either between a group of Sync Backups or between a group of Sync Alternates). Only the order in which instances of iPlanet Application Server are started, not priority, determines which server should be the Sync Primary and which Sync Servers will start out as Sync Backups or Sync Alternates.You can set the sync server priority using either iASAT or iPlanet Registry Editor.
The following sections describe how you can assign or change priority to a sync server:
To Change Sync Server Priority Using iASAT
To Change Sync Server Priority Using iPlanet Registry Editor
To Change Sync Server Priority Using iASAT
To assign a new Sync Server priority to a server that is in a cluster, perform the following tasks:
From the iASAT toolbar, click General to open the General window.
In the left pane of the General window, click a server that is a member of the cluster whose Sync Server priority you want to change.
In the right pane of the General window, click the Clusters tab. The following window appears:
In the Priority List of Servers box, click the name of the server whose Sync Server priority you want to change.
- A list of registered servers is displayed in the left pane of the General window. Another list of servers, sorted by priority in a cluster, is displayed in the right pane.
- The list also shows the status of each server in the cluster. The status should always be Normal. If an abnormal cluster condition exists, the status could be Dual Primary or No Primary. To ensure that these conditions are corrected, see Setting Cluster Parameters. To check for status, click the Refresh List button. By default, server status is updated every 15 seconds.
To change the Sync Server priority of the server, click one of the following buttons next to the Priority List of Servers box:
For example, if a server has a Sync Server priority of third in line to take over for the Sync Primary, clicking Increase once changes the priority from third to second. A lower number indicates higher priority. For example, 0 indicates the highest priority and 1 indicates the next level of priority. The lowest priority value is 65,535.
Click Apply Changes, when you complete reassigning priorities for the server(s).
For changes in Sync Server priority to apply across a cluster, you must restart each server so that all servers are aware of their new priority sequence, relative to one another.
- The changes you made in iASAT will be reflected in iPlanet Registry. You can also change the sync server priority directly in iPlanet Registry. The next section describes how you can do this.
To Change Sync Server Priority Using iPlanet Registry Editor
To change the sync server priority directly in iPlanet Registry using iPlanet Registry Editor, perform the following tasks:
Start iPlanet Registry.
Open the following key:
- (See "About iPlanet Registry Editor")
You will see the IP Address and port number entries of the servers that are added to the cluster. Select the sync server whose priority you want to change.
- SOFTWARE\iPlanet\Application Server\Clusters\<clustername>\SyncServers
Double-click the entry to open it, or, choose Modify
Value from the Edit menu. The Modify Value dialog box appears.Change the value as required. Note that a lower value indicates higher priority. For example, 0 indicates the highest priority and 1 indicates the next level of priority. The lowest priority value is 65,535.
Restart every server in the cluster, including the one whose priority you just changed. For changes in Sync Server priority to apply across a cluster, you must restart each server so that all servers are aware of their new priority sequence, relative to one another.
Setting Cluster Parameters
For a cluster to function efficiently, you need to set specific parameters that will affect its performance. You can set these parameters using either iASAT or iPlanet Registry Editor. The following sections describe how you can set cluster parameters using either tool:
To Set Cluster Parameters Using iASAT
To Set Cluster Parameters Using iASAT
You can set the following parameters for each cluster, using iASAT:
Maximum Number of Sync Backups
To set cluster parameters using iASAT, perform the following steps:
Restart in case of abnormal cluster
- You can specify the maximum number of Sync Backups the Sync Primary will use. In clusters that have many servers, specifying the maximum number of Sync Backups allows you to control how many servers are used as backups.
- You can also enable the process to restart if an abnormal cluster condition is detected. An abnormal cluster condition is either a cluster that has more than one iPlanet Application Server instances with the Sync Primary (dual-primary) role or no iPlanet Application Server instances with the Sync Primary role.
From the iASAT toolbar, click General to open the General window.
Restart every server in the cluster. For changes to apply across a cluster, you must restart each instance in the cluster, so that all the instances are aware of the changes.In the left pane of the General window, select the iPlanet Application Server that is a member of the cluster you want to modify.
In the right pane of the General window, click the Clusters tab. The following window appears
In the right pane of the General window, enter the maximum number of Sync Backups allowed during a single cluster session in the Maximum Number of Sync Backups text field.
Mark the Restart in case of abnormal cluster checkbox to correct any abnormal cluster conditions that are detected.
Note You need to first register a server with iASAT, before you add the server to a cluster.
To Set Cluster Parameters Using iPlanet Registry Editor
To set cluster parameters using iPlanet Registry Editor, perform the following steps:
Stop the server whose iPlanet Registry you want to edit.
After correctly completing these steps, you have redefined the default cluster. Now, follow the procedure in to enable communication between the servers in the cluster
Open the following folder:
- (See "About iPlanet Registry Editor")
SOFTWARE\iPlanet\Application Server\Clusters\<clustername>\
Modify the following values under the <clustername> key, as required:
- In the given example, the default cluster is named bozo-NoDysnc and contains one Sync Server with a priority of 1, as shown in the following figure:
Tip To modify a value in iPlanet Registry, perform the following steps:
MaxSyncHeartBeat
- The maximum number of backup data synchronization servers determines how many Sync Backups are updated with data from the Sync Primary at the same time. For more information about backup data synchronization servers, see What Is a Cluster?.
- As all Sync Backups are updated at the same time, an extra load is created for each additional backup server when you increase the MaxBackups value. Consider the performance impact when you set the number of backup servers, and choose a number that is high enough to provide safety, while not so high as to negatively affect performance. The default value of 1 is usually sufficient.
Note The next entry is the MaxHops key. This key relates to an unsupported feature. You do not need to modify the value of this key.
- Check and modify the MaxSyncHeartBeat value as required.
- This value specifies the maximum number of heartbeat messages that an engine will send to any other engine. The heart-beat mechanism is used to detect an abnormal cluster condition. The default value is 10.
- Each heartbeat message consists of the:
SyncHeartBeatInterval
- Whenever a heartbeat message is received, an iPlanet Application Server engine will send back a response identifying its role in the cluster.
- A heartbeat starts when a Sync Backup server is promoted to a Sync Primary. The new Sync Primary starts to send heart-beat messages to the original Sync Primary engine. In the case of a temporary network failure, the two engines will become Sync Primaries, thus creating a double-primary (split-primary) abnormal condition. This condition can be automatically corrected.
SyncTimerInterval
- Check and modify the SyncHeartBeatInteveral value as required.
- This value specifies the number of seconds between two heartbeat messages sent from one server to another. The default value is 30 seconds.
Close iPlanet Registry Editor when you finish setting the values you want.
- Check and modify the SyncTimerInterval value as required.
- This key specifies the intervals, in seconds, at which the synchronization service wakes up and checks to see whether any data has expired. This key specifies how often the timer thread goes through the node list and removes all the nodes that have expired.
- If this value is too large, expired data will still be accessible. If this value is too small, the frequent waking up and checking can degrade system performance. The default value of 60 seconds is good for most clusters.
Restart all the servers that will be affected by the modifications.
Note You can also set cluster parameters using iASAT. See To Set Cluster Parameters Using iASAT for more information.
Mapping the Synchronizer to the Cluster
Each iPlanet Application Server has an inbuilt data synchronizer, which helps synchronize the data within the server and also the server's communication with other servers.For a cluster to communicate, the synchronizer in each server must know to which cluster the server belongs. If you create clusters using iASAT, the server synchronizer is mapped automatically. If you create clusters manually, using iPlanet Registry Editor, you need to perform the following tasks, to map the synchronizer to its cluster:
Stop the server whose iPlanet Registry you want to edit.
The synchronizer is now be mapped to the cluster.
Open the following key:
- (See "About iPlanet Registry Editor")
The default cluster name will be listed. You can delete the default cluster name and specify a new cluster name for the cluster.
How Sync Server Prioritization Improves Coordination
This section discusses general priority issues and gives a comprehensive example of cluster coordination.Priority values are used only to select between Sync Servers in the same status (either between a group of Sync Backups or between a group of Sync Alternates). Only the order in which instances of iPlanet Application Server are started, not priority, determines which server should be the Sync Primary and which Sync Servers will start out as Sync Backups or Sync Alternates.
A Sync Local is not assigned a priority because it is not eligible to become a Sync Server, so a Sync Local cannot become a Sync Primary, Sync Backup, or Sync Alternate.
Which Sync Server becomes the Sync Primary in a cluster is determined simply by which Sync Server is started first. The next Sync Servers that start, up to the value in MaxBackups, become Sync Backups. When the Sync Primary fails, the Sync Backup with the highest priority, which is the lowest integer value, becomes the new Sync Primary.
When a Sync Backup becomes a Sync Primary, the number of Sync Backups falls below the value of MaxBackups. To restore the number of Sync Backups, the Sync Alternate with the highest priority becomes a Sync Backup.
Example: Coordination Within a Seven-Server Cluster
The following example illustrates cluster coordination through server roles, and the part that priority plays in determining those roles. As you trace the role changes through the example, keep in mind that server fallibility has been purposely exaggerated to provide many scenarios.Although not required, you can ease cluster maintenance by assigning the highest priority to the iPlanet Application Server instance that you will start as the Sync Primary, and the next highest priorities (in descending order) to the Sync Backups. Be aware that the cluster in this example does not do this. Also, notice that this cluster does not follow the recommended practice of starting the servers in priority order.
Assume a seven-server cluster with iPlanet Application Server instances that are numbered 0 to 6. Servers 0 through 4 are Sync Servers that are assigned the same priorities as their server numbers (for example, server 0 has a priority of zero). Servers 5 and 6 are Sync Locals. MaxBackups for the cluster is set to 2.
Server 3 is brought up first, so it becomes the Sync Primary.
Server 3 fails and goes down. Between the two Sync Backups, server 4 and server 1, server 1 has the higher priority (lower integer value) and it becomes the new Sync Primary. This leaves server 4 as the only Sync Backup.Server 4 is started next, and it becomes a Sync Backup.
Server 6 is started next, and it is a Sync Local.
Server 1 is started next, and it becomes a Sync Backup.
Server 2 is started next, and it becomes a Sync Alternate.
Because MaxBackups is set to 2, one of the Sync Alternates is converted to a Sync Backup. Server 0 becomes the new Sync Backup because it has a higher priority than the other remaining Sync Alternate, server 2. At this point:
Server 3 comes back online. It becomes a Sync Alternate. Even though it was originally a Sync Primary, the synchronizer now sees it as just another Sync Server, so the server does not resume its Sync Primary role. At this point:
Server 1 is the Sync Primary.
Server 0 fails. Server 2 becomes a Sync Backup because it has the higher priority (lower integer value) among the Sync Alternates. At this point:Servers 0 and 4 are Sync Backups.
Server 1 is the Sync Primary.
Server 0 comes back online and becomes a Sync Alternate. Server 1, the Sync Primary, fails. Among the Sync Backups, server 2 has a higher priority than server 4, so server 2 becomes the new Sync Primary. Server 0 becomes a Sync Backup. At this point:Servers 2 and 4 are Sync Backups.
Server 2 fails. Server 0 becomes the Sync Primary and server 3 becomes a Sync Backup. At this point:
Server 3 fails. Even though only one Sync Backup remains, neither server 5 nor server 6 is considered because neither is a Sync Server. At this point:
Previous Contents Index DocHome Next
Copyright © 2001 Sun Microsystems, Inc. Some preexisting portions Copyright © 2001 Netscape Communications Corp. All rights reserved.
Last Updated November 09, 2001