JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Oracle Solaris Cluster Geographic Edition Data Replication Guide for Hitachi TrueCopy and Universal Replicator
search filter icon
search icon

Document Information

Preface

1.  Replicating Data With Hitachi TrueCopy and Universal Replicator Software

2.  Administering Hitachi TrueCopy and Universal Replicator Protection Groups

Strategies for Creating Hitachi TrueCopy and Universal Replicator Protection Groups

Creating a Protection Group While the Application Is Offline

Creating a Protection Group While the Application Is Online

Creating, Modifying, Validating, and Deleting a Hitachi TrueCopy or Universal Replicator Protection Group

How to Create and Configure a Hitachi TrueCopy or Universal Replicator Protection Group That Does Not Use Oracle Real Application Clusters

Ensuring Data Consistency for Hitachi Universal Replicator in Asynchronous Mode

Understanding Data Consistency in Geographic Edition

Using Consistency Group IDs to Ensure Data Consistency

Requirements to Support Oracle Real Application Clusters With Data Replication Software

How to Create a Protection Group for Oracle Real Application Clusters

How the Data Replication Subsystem Validates the Device Group

How to Modify a Hitachi TrueCopy or Universal Replicator Protection Group

Validating a Hitachi TrueCopy or Universal Replicator Protection Group

How to Validate a Hitachi TrueCopy or Universal Replicator Protection Group

How to Delete a Hitachi TrueCopy or Universal Replicator Protection Group

Administering Hitachi TrueCopy and Universal Replicator Application Resource Groups

How to Add an Application Resource Group to a Hitachi TrueCopy or Universal Replicator Protection Group

How to Delete an Application Resource Group From a Hitachi TrueCopy or Universal Replicator Protection Group

Administering Hitachi TrueCopy and Universal Replicator Data Replication Device Groups

How to Add a Data Replication Device Group to a Hitachi TrueCopy or Universal Replicator Protection Group

Validations Made by the Data Replication Subsystem

How the State of the Hitachi TrueCopy or Universal Replicator Device Group Is Validated

Determining the State of an Individual Hitachi TrueCopy or Universal Replicator Device Group

Determining the Aggregate Hitachi TrueCopy or Universal Replicator Device Group State

Validating the Local Role of the Protection Group Against the Aggregate Device Group State

How to Modify a Hitachi TrueCopy or Universal Replicator Data Replication Device Group

How to Delete a Data Replication Device Group From a Hitachi TrueCopy or Universal Replicator Protection Group

Replicating the Hitachi TrueCopy or Universal Replicator Protection Group Configuration to a Secondary Cluster

How to Replicate the Hitachi TrueCopy or Universal Replicator Protection Group Configuration to a Secondary Cluster

Activating a Hitachi TrueCopy or Universal Replicator Protection Group

How to Activate a Hitachi TrueCopy or Universal Replicator Protection Group

Deactivating a Hitachi TrueCopy or Universal Replicator Protection Group

How to Deactivate a Hitachi TrueCopy or Universal Replicator Protection Group

Resynchronizing a Hitachi TrueCopy or Universal Replicator Protection Group

How to Resynchronize a Protection Group

Checking the Runtime Status of Hitachi TrueCopy and Universal Replicator Data Replication

Displaying a Hitachi TrueCopy or Universal Replicator Runtime Status Overview

How to Check the Overall Runtime Status of Replication

Displaying a Detailed Hitachi TrueCopy or Universal Replicator Runtime Status

3.  Migrating Services That Use Hitachi TrueCopy and Universal Replicator Data Replication

A.  Geographic Edition Properties for Hitachi TrueCopy and Universal Replicator

Index

Creating, Modifying, Validating, and Deleting a Hitachi TrueCopy or Universal Replicator Protection Group

This section contains procedures for the following tasks:


Note - You can create protection groups that are not configured to use data replication. To create a protection group that does not use a data replication subsystem, omit the -d datareplicationtype option when you use the geopg command. The geoadm status command shows a state for these protection groups of Degraded.

For more information, see Creating a Protection Group That Does Not Require Data Replication in Oracle Solaris Cluster Geographic Edition System Administration Guide.


How to Create and Configure a Hitachi TrueCopy or Universal Replicator Protection Group That Does Not Use Oracle Real Application Clusters

Use the steps in this task to create and configure a Hitachi TrueCopy or Universal Replicator protection group. If you want to use Oracle Real Application Clusters (Oracle RAC), see How to Create a Protection Group for Oracle Real Application Clusters.

Before You Begin

Before you create a protection group, ensure that the following conditions are met:


Note - Protection group names are unique in the global Geographic Edition namespace. You cannot use the same protection group name in two partnerships on the same system.


You can also replicate the existing configuration of a protection group from a remote cluster to the local cluster. For more information, see Replicating the Hitachi TrueCopy or Universal Replicator Protection Group Configuration to a Secondary Cluster.

  1. Log in to a cluster node.

    You must be assigned the Geo Management RBAC rights profile to complete this procedure. For more information about RBAC, see Geographic Edition Software and RBAC in Oracle Solaris Cluster Geographic Edition System Administration Guide.

  2. Create a new protection group by using the geopg create command.

    This command creates a protection group on all nodes of the local cluster.

    # geopg create -s partnershipname -o localrole -d truecopy [-p property [-p…]] \
    protectiongroupname
    -s partnershipname
    Specifies the name of the partnership.
    -o localrole

    Specifies the role of this protection group on the local cluster as either primary or secondary.

    -d truecopy

    Specifies that the protection group data is replicated by the Hitachi TrueCopy or Universal Replicator software.

    -p propertysetting

    Specifies the properties of the protection group.

    You can specify the following properties:

    • Description – Describes the protection group.

    • Timeout – Specifies the timeout period for the protection group in seconds.

    • Nodelist – Lists the host names of the machines that can be primary for the replication subsystem.

    • Ctgid – Specifies the consistency group ID (CTGID) of the protection group.

    • Cluster_dgs – Lists the device groups where the data is written. The Oracle Solaris Cluster device groups must exist and have the same name on both the primary cluster and the secondary cluster.

    For more information about the properties you can set, see Appendix A, Standard Geographic Edition Properties, in Oracle Solaris Cluster Geographic Edition System Administration Guide.

    protectiongroupname

    Specifies the name of the protection group.

    For information about the names and values that are supported by Geographic Edition software, see Appendix B, Legal Names and Values of Geographic Edition Entities, in Oracle Solaris Cluster Geographic Edition System Administration Guide.

    For more information about the geopg command, refer to the geopg(1M) man page.

Example 2-2 Creating and Configuring a Hitachi TrueCopy or Universal Replicator Protection Group

This example creates a Hitachi TrueCopy or Universal Replicator protection group on cluster-paris, which is set as the primary cluster.

# geopg create -s paris-newyork-ps -o primary -d truecopy \
-p Nodelist=phys-paris-1,phys-paris-2 tcpg

Example 2-3 Creating a Hitachi TrueCopy or Universal Replicator Protection Group for Application Resource Groups That Are Online

This example creates a Hitachi TrueCopy or Universal Replicator protection group, tcpg, for an application resource group, resourcegroup1, that is currently online on cluster-newyork.

  1. Create the protection group without the application resource group.

    # geopg create -s paris-newyork-ps -o primary -d truecopy \
    -p nodelist=phys-paris-1,phys-paris-2 tcpg
  2. Activate the protection group.

    # geopg start -e local tcpg
  3. Add the application resource group.

    # geopg add-resource-group resourcegroup1 tcpg

Ensuring Data Consistency for Hitachi Universal Replicator in Asynchronous Mode

This section describes the protection group configuration that is required in Geographic Edition software to guarantee data consistency in asynchronous mode replication. Asynchronous mode replication is implemented by using the async fence level of Hitachi Universal Replicator. The following discussion therefore applies only to the async fence level and to Hitachi Universal Replicator as implemented in the Geographic Edition module.

Understanding Data Consistency in Geographic Edition

The Geographic Edition module supports Hitachi TrueCopy and Universal Replicator device groups in asynchronous mode replication. Routine operations for both Hitachi TrueCopy and Universal Replicator provide data consistency in asynchronous mode. However, in the event of a temporary loss of communications or of a “rolling disaster” where different parts of the system fail at different times, only Hitachi Universal Replicator software can prevent loss of consistency of replicated data for asynchronous mode. In addition, Hitachi Universal Replicator software can only ensure data consistency with the configuration described in this section and in Configuring the /etc/horcm.conf File on the Nodes of the Primary Cluster and Configuring the /etc/horcm.conf File on the Nodes of the Secondary Cluster.

In Hitachi Universal Replicator software, the Hitachi storage arrays replicate data from primary storage to secondary storage. The application that produced the data is not involved. Even so, to guarantee data consistency, replication must preserve the application's I/O write ordering, regardless of how many disk devices the application writes.

During routine operations, Hitachi Universal Replicator software on the storage secondary array pulls data from cache on the primary storage array. If data is produced faster than it can be transferred, Hitachi Universal Replicator can commit backlogged I/O and a sequence number for each write to a journal volume on the primary storage array. The secondary storage array pulls that data from primary storage and commits it to its own journal volumes, from where it is transferred to application storage. If communications fail and are later restored, the secondary storage array begins to resynchronize the two sites by continuing to pull backlogged data and sequence numbers from the journal volume. Sequence numbers control the order in which data blocks are committed to disk so that write ordering is maintained at the secondary site despite the interruption. As long as journal volumes have enough disk space to record all data that is generated by the application that is running on the primary cluster during the period of failure, consistency is guaranteed.

In the event of a rolling disaster, where only some of the backlogged data and sequence numbers reach the secondary storage array after failures begin, sequence numbers determine which data should be committed to data LUNs to preserve consistency.


Note - In the Geographic Edition module with Hitachi Universal Replicator, journal volumes are associated with application storage in the /etc/horcm.conf file. That configuration is described in Journal Volumes and Configuring the /etc/horcm.conf File on the Nodes of the Primary Cluster. For information about how to configure journal volumes on a storage array, see the Hitachi documentation for that array.


Using Consistency Group IDs to Ensure Data Consistency

Along with journal volumes, consistency group IDs (CTGIDs) ensure data consistency even if the storage for an application data service includes devices in multiple Hitachi device groups. A CTGID is an integer that is assigned to one or more Hitachi device groups. It designates those devices that must be maintained in a state of replication consistent with each other. Consistency is maintained among all devices with the same CTGID whether the devices are members of a single Hitachi device group or several Hitachi device groups. For example, if Hitachi Universal Replicator stops replication on the devices of one device group that is assigned the CTGID of 5, it stops replication on all other devices in device groups with the CTGID of 5.

To ensure data consistency, an exact correspondence must therefore exist between the device groups that are used by a single application data service and a CTGID. All device groups that are used by a single data service must have the same unique CTGID. No device group can have that CTGID unless it is used by the data service.

To ensure this correspondence, the Geographic Edition software allows the administrator to set a CTGID property on each protection group. The device groups that are added to the protection group must all have the same CTGID as the protection group. If other device groups are assigned the same CTGID as the device groups in the protection group, the Geographic Edition software generates an error. For example, if the protection group app1-pg has been assigned the CTGID of 5, all device groups included in app1-pg must have the CTGID of 5. Moreover, all CTGIDs of device groups that are included in app1-pg must have the CTGID of 5.

You are not required to set a CTGID on a protection group. The Hitachi storage software will automatically assign a unique CTGID to an asynchronously replicated device group when it is initialized. Thereafter, the pairs in that device group will be maintained in a state of consistency with each other. Thus, if an application data service in a protection group uses storage in just one asynchronously replicated Hitachi device group, you can let the Hitachi storage array assign the device group's CTGID. You do not have to also set the CTGID of the protection group.

Similarly, if you do not need data consistency, or if your application does not write asynchronously to your Hitachi device groups, then setting the CTGID on the protection group has little use. However, if you do not assign a CTGID to a protection group, any later configuration changes to the device group or to the protection group might lead to conflicts. Assignment of a CTGID to a protection group provides the most flexibility for later changes and the most assurance of device group consistency.

Configuring Consistency Group IDs for Hitachi Universal Replicator Device Groups in Asynchronous Mode

You can assign a consistency group ID (CTGID) to a protection group by setting the property ctgid=consistency-group-ID as an option to the geopg create command. You can assign CTGID values to device groups in one of two ways:

The following procedure demonstrates these two methods of setting the CTGID for the devices that are used by an application data service. The procedure configures a protection group named app1-pg with a CTGID of 5. This protection group contains the app1-rg resource group and the Hitachi Universal Replicator devgroup1 device group, which uses the async fence level.

Before You Begin

  1. On the primary cluster, create the Geographic Edition protection group with a specified CTGID, and add the resource group.
    phys-paris-1# geopg create -s paris-newyork-ps -o primary -d truecopy -p ctgid=5 \
    -p nodelist=phys-paris-1,phys-paris-2 app1-pg
    phys-paris-1# geopg add-resource-group app1-rg app1-pg
  2. Add device groups to the protection group by using one of the following methods:
    • Add device groups that have been configured in the /etc/horcm.conf file but have not been initialized by using the paircreate command.

      phys-paris-1# geopg add-device-group -p fence_level=async devgroup1 app1-pg
    • Assign CTGIDs to device groups when they are initialized by using the Hitachi paircreate command, and add the device groups to the protection group that has the same value for the CTGID property.

      In the following example, a device group is initialized with the CTGID of 5 and then added to the app1-pg protection group:

      phys-paris-1# paircreate -g devgroup1 -vl -f async 5
      phys-paris-1# geopg add-device-group -p fence_level=async devgroup1 app1-pg
  3. Start the protection group.
    phys-paris-1# geopg start -e local app1-pg

    Uninitialized device groups, if any, are initialized and assigned the CTGID of 5.

Requirements to Support Oracle Real Application Clusters With Data Replication Software

Geographic Edition software supports Oracle Real Application Clusters (Oracle RAC)with Hitachi TrueCopy and Universal Replicator software. Observe the following requirements when you configure Oracle RAC:

How to Create a Protection Group for Oracle Real Application Clusters

Before You Begin

Before you create a protection group for Oracle Real Application Clusters (Oracle RAC), ensure that the following conditions are met:

When a cluster and the VxVM cluster feature software restart, the Oracle RAC framework automatically tries to import all cluster feature device groups that were imported already before cluster went down. Therefore, the attempt to import the device groups to the original primary fails.

  1. Log in to a cluster node on the primary cluster.

    You must be assigned the Geo Management RBAC rights profile to complete this procedure. For more information about RBAC, see Geographic Edition Software and RBAC in Oracle Solaris Cluster Geographic Edition System Administration Guide.

  2. Create a new protection group by using the geopg create command.

    This command creates a protection group on all nodes of the local cluster.

    # geopg create -s partnershipname -o localrole -d truecopy  \
    -p External_Dependency_Allowed=true [-p property [-p…]] protectiongroupname
    -s partnershipname

    Specifies the name of the partnership.

    -o localrole

    Specifies the role of this protection group on the local cluster as primary.

    -d truecopy

    Specifies that the protection group data is replicated by the Hitachi TrueCopy or Universal Replicator software.

    -p propertysetting

    Specifies the properties of the protection group.

    You can specify the following properties:

    • Description – Describes the protection group.

    • External_Dependency_Allowed - Specifies whether to allow any dependencies between resource groups and resources that belong to this protection group and resource groups and resources that do not belong to this protection group. For RAC, set this property to true.

    • Timeout – Specifies the timeout period for the protection group in seconds.

    • Nodelist – Lists the host names of the machines that can be primary for the replication subsystem.

    • Ctgid – Specifies the consistency group ID (CTGID) of the protection group.

    • Cluster_dgs – Specifies the VxVM cluster feature disk group where the data is written.

    For more information about the properties you can set, see Appendix A, Standard Geographic Edition Properties, in Oracle Solaris Cluster Geographic Edition System Administration Guide.

    protectiongroupname

    Specifies the name of the protection group.

    For information about the names and values that are supported by Geographic Edition software, see Appendix B, Legal Names and Values of Geographic Edition Entities, in Oracle Solaris Cluster Geographic Edition System Administration Guide.

    For more information about the geopg command, refer to the geopg(1M) man page.

  3. Add a Hitachi TrueCopy or Universal Replicator device group to the protection group.
    # geopg add-device-group [-p property [-p…]] protectiongroupname
    -p propertysetting

    Specifies the properties of the protection group.

    You can specify the Fence_level properties which defines the fence level that is used by the disk device group. The fence level determines the level of consistency among the primary and secondary volumes for that disk device group. You must set this to never.


    Caution

    Caution - To avoid application failure on the primary cluster, specify a Fence_level of never or async. If the Fence_level parameter is not set to never or async, data replication might not function properly when the secondary site goes down.

    If you specify a Fence_level of never, the data replication roles do not change after you perform a takeover.

    Do not use programs that would prevent the Fence_level parameter from being set to data or status because these values might be required in special circumstances.

    If you have special requirements to use a Fence_level of data or status, consult your Oracle representative.


    For more information about the properties you can set, see Appendix A, Standard Geographic Edition Properties, in Oracle Solaris Cluster Geographic Edition System Administration Guide.

    protectiongroupname

    Specifies the name of the protection group.

  4. Add to the protection group only the rac_server_proxy resource group and resource groups for device groups that are replicated.

    Note - Do not add the RAC framework resource group to the protection group. This ensures that, if the protection group becomes secondary on the node, the framework resource group does not become unmanaged. In addition, multiple RAC databases can be on the cluster, and the databases can be under Geographic Edition control or not under its control.


    # geopg add-resource-group resourcegroup protectiongroupname
    resourcegroup

    Specifies a comma-separated list of resource groups to add to or delete from the protection group. The specified resource groups must already be defined.

    The protection group must be online before you add a resource group. The geopg add-resource-group command fails when a protection group is offline and the resource group that is being added is online.


    Note - If a protection group has already been started at the time that you add a resource group, the resource group remains unmanaged. You must start the resource group manually by running the geopg start command.


    protectiongroupname

    Specifies the name of the protection group.

Example 2-4 Creating a Protection Group for Oracle RAC

This example creates the protection group pg1 which uses Oracle RAC and the cluster feature.

A cluster feature disk group racdbdg controls the data which is replicated by the Hitachi TrueCopy or Universal Replicator device group VG01. The node list of the Oracle RAC framework resource group is set to all nodes of the cluster.

  1. Create the protection group on the primary cluster with the cluster feature disk group racdbdg.

    # geopg create -s pts1 -o PRIMARY -d Truecopy \ -p cluster_dgs=racdbdg -p external_dependency_allowed=true pg1
    Protection group "pg1" successfully created.
  2. Add the Hitachi TrueCopy or Universal Replicator device group VG01 to protection group pg1.

    # geopg add-device-group --property fence_level=never VG01 pg1
    Device group "VG01" successfully added to the protection group "pg1".
  3. Add the rac_server_proxy-rg resource group and the replicated device-group resource groups, hasp4rac-rg and scaldbdg-rg, to the protection group.

    # geopg add-resource-group rac_server_proxy-rg,hasp4rac-rg,\
    scaldbdg-rg pg1

How the Data Replication Subsystem Validates the Device Group

Before creating the protection group, the data replication layer validates that the horcmd daemon is running.

The data replication layer validates that the horcmd daemon is running on at least one node that is specified in the Nodelist property.

If the Cluster_dgs property is specified, then the data replication layer verifies that the device group specified is a valid Oracle Solaris Cluster device group. The data replication layer also verifies that the device group is of a valid type.


Note - The device groups that are specified in the Cluster_dgs property must be written to only by applications that belong to the protection group. This property must not specify device groups that receive information from applications outside the protection group.


A Oracle Solaris Cluster resource group is automatically created when the protection group is created.

This resource in this resource group monitors data replication. The name of the Hitachi TrueCopy or Universal Replicator data replication resource group is rg-tc-protectiongroupname.


Caution

Caution - These automatically created replication resource groups are for Geographic Edition internal implementation purposes only. Use caution when you modify these resource groups by using Oracle Solaris Cluster commands.


How to Modify a Hitachi TrueCopy or Universal Replicator Protection Group

Before You Begin

Before modifying the configuration of your protection group, ensure that the protection group you want to modify exists locally.

  1. Log in to a cluster node.

    You must be assigned the Geo Management RBAC rights profile to complete this procedure. For more information about RBAC, see Geographic Edition Software and RBAC in Oracle Solaris Cluster Geographic Edition System Administration Guide.

  2. Modify the configuration of the protection group.

    This command modifies the properties of a protection group on all nodes of the local cluster. If the partner cluster contains a protection group of the same name, this command also propagates the new configuration information to the partner cluster.

    # geopg set-prop -p property [-p...] protectiongroupname
    -p propertysetting

    Specifies the properties of the protection group.

    For more information about the properties you can set, see Appendix A, Standard Geographic Edition Properties, in Oracle Solaris Cluster Geographic Edition System Administration Guide.

    protectiongroupname

    Specifies the name of the protection group.

    For information about the names and values that are supported by Geographic Edition software, see Appendix B, Legal Names and Values of Geographic Edition Entities, in Oracle Solaris Cluster Geographic Edition System Administration Guide.

    For more information about the geopg command, refer to the geopg(1M) man page.

Example 2-5 Modifying the Configuration of a Protection Group

This example modifies the Timeout property of the protection group that was created in Example 2-2.

# geopg set-prop -p Timeout=400 tcpg

Validating a Hitachi TrueCopy or Universal Replicator Protection Group

During protection group validation, the Hitachi TrueCopy or Universal Replicator data replication subsystem validates the following:

When the geoadm status output displays that the Configuration status of a protection group is Error, you can validate the configuration by using the geopg validate command. This command checks the current state of the protection group and its entities.

If the protection group and its entities are valid, then the Configuration status of the protection groups is set to OK. If the geopg validate command finds an error in the configuration files, then the command displays a message about the error and the configuration remains in the error state. In such a case, you can fix the error in the configuration, and run the geopg validate command again.

How to Validate a Hitachi TrueCopy or Universal Replicator Protection Group

Before You Begin

Ensure that the protection group you want to validate exists locally and that the Common Agent Container is online on all nodes of both clusters in the partnership.

  1. Log in to a cluster node.

    You must be assigned the Geo Management RBAC rights profile to complete this procedure. For more information about RBAC, see Geographic Edition Software and RBAC in Oracle Solaris Cluster Geographic Edition System Administration Guide.

  2. Validate the configuration of the protection group.

    This command validates the configuration of the protection group on the local cluster only. To validate the protection group configuration on the partner cluster, run the command again on the partner cluster.

    # geopg validate protectiongroupname 
    protectiongroupname

    Specifies a unique name that identifies a single protection group

Example 2-6 Validating the Configuration of a Protection Group

This example validates a protection group.

# geopg validate tcpg

How to Delete a Hitachi TrueCopy or Universal Replicator Protection Group

Before You Begin

If you want to delete the protection group everywhere, you must run the geopg delete command on each cluster where the protection group exists.

Before deleting a protection group, ensure that the following conditions are met:


Note - You must remove the application resource groups from the protection group in order to keep the application resource groups online while deleting the protection group. See Example 2-8 and Example 2-10 for examples of this procedure.


  1. Log in to a node on the primary cluster.

    You must be assigned the Geo Management RBAC rights profile to complete this procedure. For more information about RBAC, see Geographic Edition Software and RBAC in Oracle Solaris Cluster Geographic Edition System Administration Guide.

  2. Delete the protection group.

    This command deletes the configuration of the protection group from the local cluster. The command also removes the replication resource group for each Hitachi TrueCopy or Universal Replicator device group in the protection group. This command does not alter the pair state of the Hitachi TrueCopy or Universal Replicator device group.

    # geopg delete protectiongroupname 
    protectiongroupname

    Specifies the name of the protection group

  3. To delete the protection group on the secondary cluster, repeat step 1 and step 2 on cluster-newyork.

Example 2-7 Deleting a Protection Group

This example deletes a protection group from both partner clusters.

cluster-paris is the primary cluster. For a reminder of the sample cluster configuration, see Example Geographic Edition Cluster Configuration in Oracle Solaris Cluster Geographic Edition System Administration Guide.

# rlogin phys-paris-1 -l root
phys-paris-1# geopg delete tcpg
# rlogin phys-newyork-1 -l root
phys-newyork-1# geopg delete tcpg

Example 2-8 Deleting a Hitachi TrueCopy or Universal Replicator Protection Group While Keeping Application Resource Groups Online

This example keeps online two application resource groups, apprg1 and apprg2, while deleting their protection group, tcpg. Remove the application resource groups from the protection group, then delete the protection group.

# geopg remove-resource-group apprg1,apprg2 tcpg
# geopg stop -e global tcpg
# geopg delete tcpg