C H A P T E R 2 |
Configuring the Sun StorageTek Availability Suite Software |
Note - This guide assumes that you have already installed the volume manager software and the Sun Cluster software on each node in your cluster. |
![]() |
Caution - Do not install the Sun StorageTek Availability Suite software on a system running the initial release of the Sun Cluster 3.0 software. |
The Sun StorageTek Availability Suite 4.0 Software Installation and Configuration Guide, listed in Related Documentation, describes how to install the Sun StorageTek Availability Suite software in a non-clustered environment. The installation steps to install this software in a Sun Cluster environment are generally the same as described in the installation guides. This chapter describes the differences when you install the software in a Sun Cluster environment.
The topics in this chapter include:
When installed in a Sun Cluster operating environment, AVS software requires a configuration database for information specific to Sun Cluster. This Sun Cluster configuration database is in addition to the local configuration database, which is still required (see the Sun StorageTek Availability Suite 4.0 Software Installation and Configuration Guide for more details).
A single Sun Cluster configuration location must be shared by all cluster nodes, and must be explicitly set on each node. Running dscfgadm with no arguments will prompt you to set the cluster configuration location, if it has not yet been set on that node. The location may also be changed later by running dscfgadm -s on all nodes of the cluster.
The cluster configuration database must be located on a raw slice of a did device. For help in finding an unused did device, the scdidadm -L command shows the local and shared disks by device ID. Refer to TABLE 2-1 for the requirements for this configuration location.
Note - Ensure that the slice does not contain disk label private areas (for example, slice 2 on a Solaris OS-formatted volume). The disk label region is contained in the first sectors of cylinder 0 of a disk. See VTOC Information. |
In addition to the local configuration information (see the Sun StorageTek Availability Suite 4.0 Software Installation and Configuration Guide), the cluster-specific configuration information should also be periodically backed up. You may wish to set up a cron(1M) job to periodically back up the Sun Cluster configuration database automatically. In addition, back up the configuration information whenever you change your configuration, for example, adding and deleting volumes.
|
1. On each node of the Sun Cluster, back up the local configuration database (see the Sun StorageTek Availability Suite 4.0 Software Installation and Configuration Guide).
2. On one node of the Sun Cluster, perform the following two steps:
a. Back up the /etc/dscfg_cluster reference file, containing the path to the AVS cluster database.
b. Back up the contents of the cluster-specific configuration database to a file, using the dd command. The size of the database is 5.5 MB.
3. To determine the device you have specified for the cluster-specific configuration database, run a cat command on the /etc/dscfg_cluster file or run
dscfgadm -i.
Bitmap volumes are used by the Remote Mirror and Point-in-Time Copy software to track differences between volumes and provide information for volume updates. The Sun StorageTek Availability Suite software documentation listed in Related Documentation describes the bitmap size and other requirements.
In a Sun Cluster environment, bitmap volumes must be part of the same disk device group or cluster resource group as the corresponding primary host or secondary hosts data volume.
The Remote Mirror and Point-in-Time Copy software include two configuration files that determine how bitmap volumes are written to and saved:
A bitmap maintained on disk can persist across a system crash, depending on the setting of rdc_bitmap_mode in /usr/kernel/drv/rdc.conf. The default setting is 0. Set the bitmap mode to 1, as in the following example:
Open the rdc.conf file and locate the following section. Edit the value for the bitmap mode, save the file, and close it.
The /usr/kernel/drv/ii.conf file contains one setting that sets the Point-in-Time Copy bitmap save mode:
A bitmap maintained on disk can persist across a system crash when this field is set to 1.
|
1. Open the /usr/kernel/drv/ii.conf file using a text editor such as vi(1).
2. In a Sun Cluster environment, set the bitmap mode to 1.
4. Disable and re-enable the data services as follows:
This step ensures that the host names in the /etc/hosts file are read and known by machines running the Availability Suite software.
|
Place the names and IP addresses of all machines you plan to use with the Remote Mirror software in the /etc/hosts file. Make sure you include the logical host names and IP addresses of the logical hosts you plan to use with the Remote Mirror software in the /etc/hosts file.
Add the names and IP addresses of all machines you plan to use with the remote mirror software to the /etc/hosts file.
Edit this file on each machine where you are installing and running the remote mirror software.
Consider the following when using autosynchronization with Sun Cluster:
For example, you cannot have a primary volume with a disk device group name of sndrdg and a primary bitmap volume with a disk device group name of sndrdg2 in the same remote mirror volume set.
To improve the failover affinity between a configured resource group name and the Remote Mirror device group configured within, it is suggested that the resource group name you specify consist of the disk device group name appended with -stor-rg. For example, if the group name is sndrdg, then the resource group name would be sndrdg-stor-rg.
Typically, the remote mirror primary host is part of one cluster configuration, while the replicating secondary host might or might not be part of a different cluster.
Three configurations for the Remote Mirror software are supported:
In this configuration, the remote mirror primary host is the logical host you created in the remote mirror resource group for the remote mirror disk group using the scrgadm command; for example, see To configure Sun Cluster for HAStorage or HAStoragePlus.
If you have configured the remote mirror autosynchronization feature on the primary host, the Remote Mirror software starts an update resynchronization from the primary host for all affected remote mirror volume sets following a switchover or failover event, if the autosynchronization feature is enabled for those volume sets. This operation is performed after the resource group and network switchover operation is complete. See the sndradm man page and the Sun StorageTek Availability Suite 4.0 Remote Mirror Software Administration Guide for a description of the sndradm -a command to set the autosynchronization feature.
In this configuration, the remote mirror secondary host is the logical host you created in the remote mirror resource group for the remote mirror disk group using the scrgadm command; for example, see To configure Sun Cluster for HAStorage or HAStoragePlus.
Operations such as update resynchronizations occur and are issued from the primary host machine. Following a switchover or failover event, the Remote Mirror software attempts to start an update resynchronization for all affected remote mirror volume sets, if the autosynchronization feature is enabled for those volume sets. However, the remote mirror secondary host in a remote mirror volume set cannot initiate an update resynchronization.
This operation is performed after the resource group and network switchover operation is complete. In this case, the remote mirror secondary host switchover appears to be a short network outage to the remote mirror primary host.
If you have configured the remote mirror autosynchronization feature on the primary host, the sndrsyncd synchronization daemon attempts to resynchronize the volume sets if the system reboots or link failures occur. See the sndradm man page and the Sun StorageTek Availability Suite 4.0 Remote Mirror Software Administration Guide for a description of the sndradm -a command to set the autosynchronization feature.
If this feature is disabled (its default setting) and volume sets are logging but not replicating, perform the updates manually using the sndradm command.
Remote mirror replication within the cluster is not supported. That is, remote mirror replication when the primary and secondary hosts reside in the same cluster and the primary, secondary, and bitmap volumes in a volume set reside in the same disk device group.
If the remote mirror primary and secondary hosts are configured in different clusters, for operating considerations see Remote Mirror Primary Host On a Cluster Node and Remote Mirror Secondary Host On a Cluster Node.
Following are some rules for Point-in-Time Copy software:
For example, you cannot have a master volume with a disk device group name of
ii-group and a shadow volume with a disk device group name of ii-group2 in the same volume set.
The Solstice DiskSuite and VERITAS Volume Manager (VxVM) can arrange disk devices into a group to be mastered by a cluster node. You can then configure these disk device groups to fail over to another cluster node, as described in Configuring the Sun Cluster Environment.
The Solstice DiskSuite and VxVM device paths contain the disk device group. When operating in a Sun Cluster environment, the Sun StorageTek Availability Suite commands sndradm and iiadm automatically detect and use the disk device group as configured in Configuring the Sun Cluster Environment.
You can also use the sndradm and iiadm commands to select specified disk device groups or to operate on a volume set as a local node-only configuration entry. See Using the Sun StorageTek Availability Suite iiadm and sndradm Commands
The Sun StorageTek Availability Suite supports the use of raw devices in a Sun Cluster OE (Operating Environment), although the volume configuration procedures to handle raw device are very different from devices under the control of VERITAS Volume Manager (VxVM) or Solaris Volume Manager (SVM).
In a Sun Cluster OE, the Availability Suite supports raw devices only through the use of Sun Cluster Global Devices (as in /dev/global/rdsk/d8s0). Availability Suite can not use Sun Cluster DID devices (such as/dev/did/rdsk/d8s0), because Sun Cluster's disk fencing and data pathing software causes connectivity problems when directly using DID devices.
When the Point-in-Time Copy software is configured on a Global Device, any node in a Sun Cluster can concurrently access the data on the Point-in-Time Copy software master or shadow volume. One node in the Sun Cluster has direct I/O access; all other nodes have cluster-interconnect I/O access to the Point-in-Time Copy software sets that are configured.
Depending on Point-in-Time Copy software set usage patterns, when a specific node is dominating the I/O workload of either the master or shadow volume, the Global Device can be moved to the node of high application usage as follows:
Doing so will improve performance and decrease the amount of cluster interconnect traffic.
Global devices are very much like SVM and VxVM volumes, and can be switched between nodes. Although Availability Suite is supported on Global Devices, because of the default DID device behavior of Sun Cluster, each global device is in its own "named" Sun Cluster device group. For example, /dev/global/rdsk/d8 is in device group "dsk/d8". This default Sun Cluster behavior (all constituent volumes in a Point-in-Time Copy must be in the same Sun Cluster device group) would force you to put the Point-in-Time Copy software master, shadow and bitmap volume into the same Global Device, yielding very poor I/O performance.
By default in a Sun Cluster OE, all DID devices are mapped to global devices of the same name. For instance the following DID devices:
are mapped to the following global devices:
If you tried to put a Point-in-Time Copy software master, shadow and bitmap on separate global devices, this enable operation would fail as follows because of the constituent volume rule stated above.
|
1. Bring the devices to the node.
2. Try to enable a Point-in-Time Copy set spanning the device groups.
In this example, the groups are dsk/d6, dsk/d7, and dsk/d8.
# iiadm -e dep /dev/global/rdsk/d6s0 /dev/global/rdsk/d7s0 /dev/global/rdsk/d8s0 iiadm: Volumes are not in same disk group: |
To resolve this issue, you need to reconfigure these different global devices into a "named" global device group.
3. Sun Cluster "switch" the device groups into maintenance mode.
4. Sun Cluster unconfigure the old names from the configuration.
5. Define a new named device group configuration name containing the DID devices.
In this example, the named group is "AVSuite."
6. Bring the named global device group (AVSuite) to the current Sun Cluster node.
In this example, the named group is "AVSuite."
7. Try again to enable a Point-in-Time Copy set spanning the named groups.
In this example, the named groups are dsk/d6, dsk/d7, and dsk/d8.
This time, the command is successful.
8. Verify that the cluster tag was set successfully.
Note that after running iiadm -i /dev/global/rdsk/d7s0, the cluster tag is AVSuite, and not dsk/d7.
9. The newly named global device can now be switched to other Sun Cluster nodes, using its new device group name.
|
After you have created "named" global devices, there may be a need to remove them at some future time. Following are the steps to restore the Global Device configuration back to its initial state.
1. Disable the use of the "named" Global Devices.
2. Sun Cluster "switch" the device groups into maintenance mode.
In this example, the named group is "AVSuite."
3. Sun Cluster unconfigure the named global device configuration.
4. Define a (default) dsk/dn device group configuration name for all previously configured DID devices.
5. Bring the named global device groups to the current Sun Cluster node.
6. Verify that the original restriction is now back.
# iiadm -e dep /dev/global/rdsk/d6s0 /dev/global/rdsk/d7s0 /dev/global/rdsk/d8s0 iiadm: Volumes are not in same disk group: |
The procedures in this section describe how to configure the Sun Cluster software for use with the Remote Mirror and Point-in-Time Copy software. The Sun Cluster 3.2 Data Installation and Configuration Guide contains more information about configuring and administering Sun Cluster data services. See the scrgadm(1M) and scswitch(1M) man pages for more information.
The general configuration steps are:
1. Log on to any node in the cluster.
2. Configure a disk device group using your volume manager.
3. Register the SUNW.HAStorage or SUNW.HAStoragePlus resource type.
5. Add SUNW.HAStorage or SUNW.HAStoragePlus to the disk device group.
6. (Remote Mirror step only) Add a logical failover host to the resource group.
7. Enable the resource group and bring it online.
|
1. Log on as the root user on any node in the cluster.
2. Configure a disk device group using your volume manager software.
See the documentation that came with your volume manager software. Also you might check the currently configured groups before configuring a new disk device group. For example, use the metaset(1M), vxdg, or vxprint commands, depending on your volume manager software.
3. Register SUNW.HAStorage or SUNW.HAStoragePlus as a resource type.
4. Create a resource group for the devicegroup.
specifies the cluster nodes that can master this resource group. If you do not specify these nodes, it defaults to all the nodes in the cluster. |
5. For a SUNW.HAStorage resource, use the following command to add the resource to the resource group.
# scrgadm -a -j devicegroup-stor -g devicegroup-stor-rg \-t SUNW.HAStorage \-x ServicePaths=devicegroup -x AffinityOn=True |
6. For a a SUNW.HAStoragePlus resource, use the following command to add the resource to the resource group.
# scrgadm -a -j devicegroup-stor -g devicegroup-stor-rg \-t SUNW.HAStoragePlus \-x GlobalDevicePaths=devicegroup -x AffinityOn=True |
7. Add a logical hostname resource to the resource group.
Note - Perform this step for the remote mirror volumes only. This step is not needed for Point-in-Time Copy volumes. |
# scrgadm -a -L [-j lhost-stor] -g devicegroup-stor-rg \-l lhost1, lhost2,..., lhostN -n nafo0@node,nafo0@node |
8. Enable the resources in the resource group, manage the resource group, and bring the resource group online.
9. Verify that the resource is online.
a. Run the following command on any cluster node.
b. Look for the resource group state field to determine if the resource group is online on the nodes specified in the node list.
10. For the HAStoragePlus resource, verify that the resource group can be failed between nodes.
The above command fails the resource group to the specified node.
The above command fails ALL resources from the specified node.
This example shows how to configure a resource group on a locally mounted Sun Cluster global device partition.
You can configure the HAStoragePlus resource to fail over resource groups as well as individual volume sets to another node in the cluster. When configuring a resource type with volume sets, consider the following:
specifies the extension property GlobalDevicePath and raw device volume names for the point-in-time copy volume set. |
Copyright © 2006, Sun Microsystems, Inc. All Rights Reserved.