Skip Navigation Links | |
Exit Print View | |
Oracle Solaris Cluster Geographic Edition Data Replication Guide for Hitachi TrueCopy and Universal Replicator Oracle Solaris Cluster 3.3 3/13 |
1. Replicating Data With Hitachi TrueCopy and Universal Replicator Software
2. Administering Hitachi TrueCopy and Universal Replicator Protection Groups
Strategies for Creating Hitachi TrueCopy and Universal Replicator Protection Groups
Creating a Protection Group While the Application Is Offline
Creating a Protection Group While the Application Is Online
Ensuring Data Consistency for Hitachi Universal Replicator in Asynchronous Mode
Understanding Data Consistency in Geographic Edition
Using Consistency Group IDs to Ensure Data Consistency
Requirements to Support Oracle Real Application Clusters With Data Replication Software
How to Create a Protection Group for Oracle Real Application Clusters
How the Data Replication Subsystem Validates the Device Group
How to Modify a Hitachi TrueCopy or Universal Replicator Protection Group
Validating a Hitachi TrueCopy or Universal Replicator Protection Group
How to Validate a Hitachi TrueCopy or Universal Replicator Protection Group
How to Delete a Hitachi TrueCopy or Universal Replicator Protection Group
Administering Hitachi TrueCopy and Universal Replicator Application Resource Groups
Administering Hitachi TrueCopy and Universal Replicator Data Replication Device Groups
Validations Made by the Data Replication Subsystem
How the State of the Hitachi TrueCopy or Universal Replicator Device Group Is Validated
Determining the State of an Individual Hitachi TrueCopy or Universal Replicator Device Group
Determining the Aggregate Hitachi TrueCopy or Universal Replicator Device Group State
Validating the Local Role of the Protection Group Against the Aggregate Device Group State
How to Modify a Hitachi TrueCopy or Universal Replicator Data Replication Device Group
Activating a Hitachi TrueCopy or Universal Replicator Protection Group
How to Activate a Hitachi TrueCopy or Universal Replicator Protection Group
Deactivating a Hitachi TrueCopy or Universal Replicator Protection Group
How to Deactivate a Hitachi TrueCopy or Universal Replicator Protection Group
Resynchronizing a Hitachi TrueCopy or Universal Replicator Protection Group
How to Resynchronize a Protection Group
Checking the Runtime Status of Hitachi TrueCopy and Universal Replicator Data Replication
Displaying a Hitachi TrueCopy or Universal Replicator Runtime Status Overview
How to Check the Overall Runtime Status of Replication
Displaying a Detailed Hitachi TrueCopy or Universal Replicator Runtime Status
3. Migrating Services That Use Hitachi TrueCopy and Universal Replicator Data Replication
A. Geographic Edition Properties for Hitachi TrueCopy and Universal Replicator
This section contains procedures for the following tasks:
Ensuring Data Consistency for Hitachi Universal Replicator in Asynchronous Mode
Requirements to Support Oracle Real Application Clusters With Data Replication Software
How to Create a Protection Group for Oracle Real Application Clusters
How the Data Replication Subsystem Validates the Device Group
How to Modify a Hitachi TrueCopy or Universal Replicator Protection Group
Validating a Hitachi TrueCopy or Universal Replicator Protection Group
How to Delete a Hitachi TrueCopy or Universal Replicator Protection Group
Note - You can create protection groups that are not configured to use data replication. To create a protection group that does not use a data replication subsystem, omit the -d datareplicationtype option when you use the geopg command. The geoadm status command shows a state for these protection groups of Degraded.
For more information, see Creating a Protection Group That Does Not Require Data Replication in Oracle Solaris Cluster Geographic Edition System Administration Guide.
Use the steps in this task to create and configure a Hitachi TrueCopy or Universal Replicator protection group. If you want to use Oracle Real Application Clusters (Oracle RAC), see How to Create a Protection Group for Oracle Real Application Clusters.
Before You Begin
Before you create a protection group, ensure that the following conditions are met:
The local cluster is a member of a partnership.
The protection group you are creating does not already exist.
Note - Protection group names are unique in the global Geographic Edition namespace. You cannot use the same protection group name in two partnerships on the same system.
You can also replicate the existing configuration of a protection group from a remote cluster to the local cluster. For more information, see Replicating the Hitachi TrueCopy or Universal Replicator Protection Group Configuration to a Secondary Cluster.
You must be assigned the Geo Management RBAC rights profile to complete this procedure. For more information about RBAC, see Geographic Edition Software and RBAC in Oracle Solaris Cluster Geographic Edition System Administration Guide.
This command creates a protection group on all nodes of the local cluster.
# geopg create -s partnershipname -o localrole -d truecopy [-p property [-p…]] \ protectiongroupname
Specifies the role of this protection group on the local cluster as either primary or secondary.
Specifies that the protection group data is replicated by the Hitachi TrueCopy or Universal Replicator software.
Specifies the properties of the protection group.
You can specify the following properties:
Description – Describes the protection group.
Timeout – Specifies the timeout period for the protection group in seconds.
Nodelist – Lists the host names of the machines that can be primary for the replication subsystem.
Ctgid – Specifies the consistency group ID (CTGID) of the protection group.
Cluster_dgs – Lists the device groups where the data is written. The Oracle Solaris Cluster device groups must exist and have the same name on both the primary cluster and the secondary cluster.
For more information about the properties you can set, see Appendix A, Standard Geographic Edition Properties, in Oracle Solaris Cluster Geographic Edition System Administration Guide.
Specifies the name of the protection group.
For information about the names and values that are supported by Geographic Edition software, see Appendix B, Legal Names and Values of Geographic Edition Entities, in Oracle Solaris Cluster Geographic Edition System Administration Guide.
For more information about the geopg command, refer to the geopg(1M) man page.
Example 2-2 Creating and Configuring a Hitachi TrueCopy or Universal Replicator Protection Group
This example creates a Hitachi TrueCopy or Universal Replicator protection group on cluster-paris, which is set as the primary cluster.
# geopg create -s paris-newyork-ps -o primary -d truecopy \ -p Nodelist=phys-paris-1,phys-paris-2 tcpg
Example 2-3 Creating a Hitachi TrueCopy or Universal Replicator Protection Group for Application Resource Groups That Are Online
This example creates a Hitachi TrueCopy or Universal Replicator protection group, tcpg, for an application resource group, resourcegroup1, that is currently online on cluster-newyork.
Create the protection group without the application resource group.
# geopg create -s paris-newyork-ps -o primary -d truecopy \ -p nodelist=phys-paris-1,phys-paris-2 tcpg
Activate the protection group.
# geopg start -e local tcpg
Add the application resource group.
# geopg add-resource-group resourcegroup1 tcpg
This section describes the protection group configuration that is required in Geographic Edition software to guarantee data consistency in asynchronous mode replication. Asynchronous mode replication is implemented by using the async fence level of Hitachi Universal Replicator. The following discussion therefore applies only to the async fence level and to Hitachi Universal Replicator as implemented in the Geographic Edition module.
The Geographic Edition module supports Hitachi TrueCopy and Universal Replicator device groups in asynchronous mode replication. Routine operations for both Hitachi TrueCopy and Universal Replicator provide data consistency in asynchronous mode. However, in the event of a temporary loss of communications or of a “rolling disaster” where different parts of the system fail at different times, only Hitachi Universal Replicator software can prevent loss of consistency of replicated data for asynchronous mode. In addition, Hitachi Universal Replicator software can only ensure data consistency with the configuration described in this section and in Configuring the /etc/horcm.conf File on the Nodes of the Primary Cluster and Configuring the /etc/horcm.conf File on the Nodes of the Secondary Cluster.
In Hitachi Universal Replicator software, the Hitachi storage arrays replicate data from primary storage to secondary storage. The application that produced the data is not involved. Even so, to guarantee data consistency, replication must preserve the application's I/O write ordering, regardless of how many disk devices the application writes.
During routine operations, Hitachi Universal Replicator software on the storage secondary array pulls data from cache on the primary storage array. If data is produced faster than it can be transferred, Hitachi Universal Replicator can commit backlogged I/O and a sequence number for each write to a journal volume on the primary storage array. The secondary storage array pulls that data from primary storage and commits it to its own journal volumes, from where it is transferred to application storage. If communications fail and are later restored, the secondary storage array begins to resynchronize the two sites by continuing to pull backlogged data and sequence numbers from the journal volume. Sequence numbers control the order in which data blocks are committed to disk so that write ordering is maintained at the secondary site despite the interruption. As long as journal volumes have enough disk space to record all data that is generated by the application that is running on the primary cluster during the period of failure, consistency is guaranteed.
In the event of a rolling disaster, where only some of the backlogged data and sequence numbers reach the secondary storage array after failures begin, sequence numbers determine which data should be committed to data LUNs to preserve consistency.
Note - In the Geographic Edition module with Hitachi Universal Replicator, journal volumes are associated with application storage in the /etc/horcm.conf file. That configuration is described in Journal Volumes and Configuring the /etc/horcm.conf File on the Nodes of the Primary Cluster. For information about how to configure journal volumes on a storage array, see the Hitachi documentation for that array.
Along with journal volumes, consistency group IDs (CTGIDs) ensure data consistency even if the storage for an application data service includes devices in multiple Hitachi device groups. A CTGID is an integer that is assigned to one or more Hitachi device groups. It designates those devices that must be maintained in a state of replication consistent with each other. Consistency is maintained among all devices with the same CTGID whether the devices are members of a single Hitachi device group or several Hitachi device groups. For example, if Hitachi Universal Replicator stops replication on the devices of one device group that is assigned the CTGID of 5, it stops replication on all other devices in device groups with the CTGID of 5.
To ensure data consistency, an exact correspondence must therefore exist between the device groups that are used by a single application data service and a CTGID. All device groups that are used by a single data service must have the same unique CTGID. No device group can have that CTGID unless it is used by the data service.
To ensure this correspondence, the Geographic Edition software allows the administrator to set a CTGID property on each protection group. The device groups that are added to the protection group must all have the same CTGID as the protection group. If other device groups are assigned the same CTGID as the device groups in the protection group, the Geographic Edition software generates an error. For example, if the protection group app1-pg has been assigned the CTGID of 5, all device groups included in app1-pg must have the CTGID of 5. Moreover, all CTGIDs of device groups that are included in app1-pg must have the CTGID of 5.
You are not required to set a CTGID on a protection group. The Hitachi storage software will automatically assign a unique CTGID to an asynchronously replicated device group when it is initialized. Thereafter, the pairs in that device group will be maintained in a state of consistency with each other. Thus, if an application data service in a protection group uses storage in just one asynchronously replicated Hitachi device group, you can let the Hitachi storage array assign the device group's CTGID. You do not have to also set the CTGID of the protection group.
Similarly, if you do not need data consistency, or if your application does not write asynchronously to your Hitachi device groups, then setting the CTGID on the protection group has little use. However, if you do not assign a CTGID to a protection group, any later configuration changes to the device group or to the protection group might lead to conflicts. Assignment of a CTGID to a protection group provides the most flexibility for later changes and the most assurance of device group consistency.
You can assign a consistency group ID (CTGID) to a protection group by setting the property ctgid=consistency-group-ID as an option to the geopg create command. You can assign CTGID values to device groups in one of two ways:
You can add uninitialized device groups to the protection group. They are initialized and acquire the CTGID of the protection group when the protection group is started with the geopg start command.
You can initialize a device group with the CTGID that you plan to use for the protection group that will hold that device group. After you create the protection group with that CTGID, you must assign the device group to it.
The following procedure demonstrates these two methods of setting the CTGID for the devices that are used by an application data service. The procedure configures a protection group named app1-pg with a CTGID of 5. This protection group contains the app1-rg resource group and the Hitachi Universal Replicator devgroup1 device group, which uses the async fence level.
Before You Begin
Configure a Hitachi Universal Replicator device group with journal volumes in the /etc/horcm.conf file as described inConfiguring the /etc/horcm.conf File on the Nodes of the Primary Cluster and Configuring the /etc/horcm.conf File on the Nodes of the Secondary Cluster.
Configure the devices in each device group as raw-disk devices.
Configure a Oracle Solaris Cluster resource group that includes a resource of type HAStoragePlus in addition to any other resources that are required for its application data service. This HAStoragePlus resource must use the disk devices of a previously configured Hitachi Universal Replicator device group as described in How to Configure a Highly Available Local File System for Hitachi TrueCopy or Universal Replicator Replication.
phys-paris-1# geopg create -s paris-newyork-ps -o primary -d truecopy -p ctgid=5 \ -p nodelist=phys-paris-1,phys-paris-2 app1-pg
phys-paris-1# geopg add-resource-group app1-rg app1-pg
Add device groups that have been configured in the /etc/horcm.conf file but have not been initialized by using the paircreate command.
phys-paris-1# geopg add-device-group -p fence_level=async devgroup1 app1-pg
Assign CTGIDs to device groups when they are initialized by using the Hitachi paircreate command, and add the device groups to the protection group that has the same value for the CTGID property.
In the following example, a device group is initialized with the CTGID of 5 and then added to the app1-pg protection group:
phys-paris-1# paircreate -g devgroup1 -vl -f async 5
phys-paris-1# geopg add-device-group -p fence_level=async devgroup1 app1-pg
phys-paris-1# geopg start -e local app1-pg
Uninitialized device groups, if any, are initialized and assigned the CTGID of 5.
Geographic Edition software supports Oracle Real Application Clusters (Oracle RAC) with Hitachi TrueCopy and Universal Replicator software. Observe the following requirements when you configure Oracle RAC:
Each CRS OCR and Voting Disk Location must be in its own device group on each cluster and cannot be replicated.
Static data such as CRS and database binaries are not required to be replicated. But this data must be accessible from all nodes of both clusters.
You must create a SUNW.ScalDeviceGroup resource in its own resource group for the device group that holds dynamic database files. This resource group must be separate from the resource group that holds the clusterware SUNW.ScalDeviceGroup resource.
To be able to leave RAC infrastructure resource groups outside of Geographic Edition control, you must run Geographic Edition binaries on both cluster partners and set the RAC protection group External_Dependency_Allowed property to true.
Do not add the CRS OCR and Voting Disk device group to the protection group's cluster_dgs property.
Do not add RAC infrastructure resource groups to the protection group. Only add the rac_server_proxy resource group and resource groups for device groups that are replicated to the protection group. Also, you must set to false the auto_start_on_new_cluster resource group property for the rac_server_proxy resource group and resource groups and for device groups that are replicated.
When you use a cluster file system for an Oracle RAC file system, such as a flash recovery area, alert, or trace log files, you must manually create on both clusters a separate resource group that uses the HAStoragePlus resource to bring online these corresponding file systems. You must set a strong resource dependency from nonClusterware SUNW.ScalDeviceGroup resources to this HAStoragePlus resource. Then add this HAStoragePlus resource group to the RAC protection group.
Before You Begin
Before you create a protection group for Oracle Real Application Clusters (Oracle RAC), ensure that the following conditions are met:
Read Requirements to Support Oracle Real Application Clusters With Data Replication Software.
The node list of the protection group must be the same as the node list of Oracle RAC framework resource group.
If one cluster is running Oracle RAC on a different number of nodes than another cluster, ensure that all nodes on both clusters have the same resource groups defined.
You must be assigned the Geo Management RBAC rights profile to complete this procedure. For more information about RBAC, see Geographic Edition Software and RBAC in Oracle Solaris Cluster Geographic Edition System Administration Guide.
This command creates a protection group on all nodes of the local cluster.
# geopg create -s partnershipname -o localrole -d truecopy \ -p External_Dependency_Allowed=true [-p property [-p…]] protectiongroupname
Specifies the name of the partnership.
Specifies the role of this protection group on the local cluster as primary.
Specifies that the protection group data is replicated by the Hitachi TrueCopy or Universal Replicator software.
Specifies the properties of the protection group.
You can specify the following properties:
Description – Describes the protection group.
External_Dependency_Allowed - Specifies whether to allow any dependencies between resource groups and resources that belong to this protection group and resource groups and resources that do not belong to this protection group. For RAC, set this property to true.
Timeout – Specifies the timeout period for the protection group in seconds.
Nodelist – Lists the host names of the machines that can be primary for the replication subsystem.
Ctgid – Specifies the consistency group ID (CTGID) of the protection group.
For more information about the properties you can set, see Appendix A, Standard Geographic Edition Properties, in Oracle Solaris Cluster Geographic Edition System Administration Guide.
Specifies the name of the protection group.
For information about the names and values that are supported by Geographic Edition software, see Appendix B, Legal Names and Values of Geographic Edition Entities, in Oracle Solaris Cluster Geographic Edition System Administration Guide.
For more information about the geopg command, refer to the geopg(1M) man page.
# geopg add-device-group [-p property [-p…]] protectiongroupname
Specifies the properties of the protection group.
You can specify the Fence_level properties which defines the fence level that is used by the disk device group. The fence level determines the level of consistency among the primary and secondary volumes for that disk device group. You must set this to never.
Caution - To avoid application failure on the primary cluster, specify a Fence_level of never or async. If the Fence_level parameter is not set to never or async, data replication might not function properly when the secondary site goes down. If you specify a Fence_level of never, the data replication roles do not change after you perform a takeover. Do not use programs that would prevent the Fence_level parameter from being set to data or status because these values might be required in special circumstances. If you have special requirements to use a Fence_level of data or status, consult your Oracle representative. |
For more information about the properties you can set, see Appendix A, Standard Geographic Edition Properties, in Oracle Solaris Cluster Geographic Edition System Administration Guide.
Specifies the name of the protection group.
Note - Do not add the RAC framework resource group to the protection group. This ensures that, if the protection group becomes secondary on the node, the framework resource group does not become unmanaged. In addition, multiple RAC databases can be on the cluster, and the databases can be under Geographic Edition control or not under its control.
# geopg add-resource-group resourcegroup protectiongroupname
Specifies a comma-separated list of resource groups to add to or delete from the protection group. The specified resource groups must already be defined.
The protection group must be online before you add a resource group. The geopg add-resource-group command fails when a protection group is offline and the resource group that is being added is online.
Note - If a protection group has already been started at the time that you add a resource group, the resource group remains unmanaged. You must start the resource group manually by running the geopg start command.
Specifies the name of the protection group.
Example 2-4 Creating a Protection Group for Oracle RAC
This example creates the protection group pg1 which uses Oracle RAC and the cluster feature.
A cluster feature disk group racdbdg controls the data which is replicated by the Hitachi TrueCopy or Universal Replicator device group VG01. The node list of the Oracle RAC framework resource group is set to all nodes of the cluster.
Create the protection group on the primary cluster with the cluster feature disk group racdbdg.
# geopg create -s pts1 -o PRIMARY -d Truecopy \ -p cluster_dgs=racdbdg -p external_dependency_allowed=true pg1 Protection group "pg1" successfully created.
Add the Hitachi TrueCopy or Universal Replicator device group VG01 to protection group pg1.
# geopg add-device-group --property fence_level=never VG01 pg1 Device group "VG01" successfully added to the protection group "pg1".
Add the rac_server_proxy-rg resource group and the replicated device-group resource groups, hasp4rac-rg and scaldbdg-rg, to the protection group.
# geopg add-resource-group rac_server_proxy-rg,hasp4rac-rg,\ scaldbdg-rg pg1
Before creating the protection group, the data replication layer validates that the horcmd daemon is running.
The data replication layer validates that the horcmd daemon is running on at least one node that is specified in the Nodelist property.
If the Cluster_dgs property is specified, then the data replication layer verifies that the device group specified is a valid Oracle Solaris Cluster device group. The data replication layer also verifies that the device group is of a valid type.
Note - The device groups that are specified in the Cluster_dgs property must be written to only by applications that belong to the protection group. This property must not specify device groups that receive information from applications outside the protection group.
A Oracle Solaris Cluster resource group is automatically created when the protection group is created.
This resource in this resource group monitors data replication. The name of the Hitachi TrueCopy or Universal Replicator data replication resource group is rg-tc-protectiongroupname.
Caution - These automatically created replication resource groups are for Geographic Edition internal implementation purposes only. Use caution when you modify these resource groups by using Oracle Solaris Cluster commands. |
Before You Begin
Before modifying the configuration of your protection group, ensure that the protection group you want to modify exists locally.
You must be assigned the Geo Management RBAC rights profile to complete this procedure. For more information about RBAC, see Geographic Edition Software and RBAC in Oracle Solaris Cluster Geographic Edition System Administration Guide.
This command modifies the properties of a protection group on all nodes of the local cluster. If the partner cluster contains a protection group of the same name, this command also propagates the new configuration information to the partner cluster.
# geopg set-prop -p property [-p...] protectiongroupname
Specifies the properties of the protection group.
For more information about the properties you can set, see Appendix A, Standard Geographic Edition Properties, in Oracle Solaris Cluster Geographic Edition System Administration Guide.
Specifies the name of the protection group.
For information about the names and values that are supported by Geographic Edition software, see Appendix B, Legal Names and Values of Geographic Edition Entities, in Oracle Solaris Cluster Geographic Edition System Administration Guide.
For more information about the geopg command, refer to the geopg(1M) man page.
Example 2-5 Modifying the Configuration of a Protection Group
This example modifies the Timeout property of the protection group that was created in Example 2-2.
# geopg set-prop -p Timeout=400 tcpg
During protection group validation, the Hitachi TrueCopy or Universal Replicator data replication subsystem validates the following:
The horcmd daemon is running on at least one node that is specified in the Nodelist property of the protection group. The data replication layer also confirms that a path to a Hitachi TrueCopy or Universal Replicator storage device exists from the node on which the horcmd daemon is running.
The device group specified is a valid Oracle Solaris Cluster device group. The data replication layer also verifies that the device group is of a valid type.
The properties are validated for each Hitachi TrueCopy or Universal Replicator device group that has been added to the protection group.
When the geoadm status output displays that the Configuration status of a protection group is Error, you can validate the configuration by using the geopg validate command. This command checks the current state of the protection group and its entities.
If the protection group and its entities are valid, then the Configuration status of the protection groups is set to OK. If the geopg validate command finds an error in the configuration files, then the command displays a message about the error and the configuration remains in the error state. In such a case, you can fix the error in the configuration, and run the geopg validate command again.
Before You Begin
Ensure that the protection group you want to validate exists locally and that the Common Agent Container is online on all nodes of both clusters in the partnership.
You must be assigned the Geo Management RBAC rights profile to complete this procedure. For more information about RBAC, see Geographic Edition Software and RBAC in Oracle Solaris Cluster Geographic Edition System Administration Guide.
This command validates the configuration of the protection group on the local cluster only. To validate the protection group configuration on the partner cluster, run the command again on the partner cluster.
# geopg validate protectiongroupname
Specifies a unique name that identifies a single protection group.
Example 2-6 Validating the Configuration of a Protection Group
This example validates a protection group.
# geopg validate tcpg
Before You Begin
If you want to delete the protection group everywhere, you must run the geopg delete command on each cluster where the protection group exists.
Before deleting a protection group, ensure that the following conditions are met:
The protection group you want to delete exists locally.
The protection group is offline on the local cluster.
Note - You must remove the application resource groups from the protection group in order to keep the application resource groups online while deleting the protection group. See Example 2-8 and Example 2-10 for examples of this procedure.
You must be assigned the Geo Management RBAC rights profile to complete this procedure. For more information about RBAC, see Geographic Edition Software and RBAC in Oracle Solaris Cluster Geographic Edition System Administration Guide.
This command deletes the configuration of the protection group from the local cluster. The command also removes the replication resource group for each Hitachi TrueCopy or Universal Replicator device group in the protection group. This command does not alter the pair state of the Hitachi TrueCopy or Universal Replicator device group.
# geopg delete protectiongroupname
Specifies the name of the protection group.
Example 2-7 Deleting a Protection Group
This example deletes a protection group from both partner clusters.
cluster-paris is the primary cluster. For a reminder of the sample cluster configuration, see Example Geographic Edition Cluster Configuration in Oracle Solaris Cluster Geographic Edition System Administration Guide.
# rlogin phys-paris-1 -l root phys-paris-1# geopg delete tcpg # rlogin phys-newyork-1 -l root phys-newyork-1# geopg delete tcpg
Example 2-8 Deleting a Hitachi TrueCopy or Universal Replicator Protection Group While Keeping Application Resource Groups Online
This example keeps online two application resource groups, apprg1 and apprg2, while deleting their protection group, tcpg. Remove the application resource groups from the protection group, then delete the protection group.
# geopg remove-resource-group apprg1,apprg2 tcpg # geopg stop -e global tcpg # geopg delete tcpg