Recovering Oracle Data Guard Data After a Takeover

After a successful takeover operation, the standby cluster, cluster-newyork, becomes the primary for the protection group, and the services are online on the standby cluster. After the recovery of the original primary cluster, the services can be brought online again on the original primary cluster by using a process called failback.

Sun Cluster Geographic Edition software supports the following two kinds of failback:

• Failback switchover. During a failback switchover, applications are brought online again on the original primary cluster, cluster-paris, after the primary cluster data has been resynchronized with the data on the standby cluster cluster-newyork.

  For a reminder of which clusters are cluster-paris and cluster-newyork, see Example Sun Cluster Geographic Edition Cluster Configuration in Sun Cluster Geographic Edition System Administration Guide.

• Failback takeover. During a failback takeover, applications are brought online again on the original primary cluster and use the current data on the primary cluster. Any updates that occurred on the standby cluster are discarded.

If you want to leave the new primary, cluster-newyork, as the primary cluster and the original primary cluster, cluster-paris, as the standby cluster after the original primary cluster starts again , you can resynchronize and revalidate the protection group configuration. You can resynchronize and revalidate the protection group without performing a switchover or takeover.

This section describes how to perform the following procedures:

• How to Resynchronize and Revalidate the Protection Group Configuration

• How to Perform a Failback Switchover on a System That Uses Oracle Data Guard Replication

• How to Perform a Failback Takeover on a System That Uses Oracle Data Guard Replication

How to Resynchronize and Revalidate the Protection Group Configuration

Follow this procedure to resynchronize and revalidate data on the original primary cluster, cluster-paris, with the data on the current primary cluster cluster-newyork.

Before You Begin

Before you resynchronize and revalidate the protection group configuration, a takeover has occurred on cluster-newyork. The clusters now have the following roles:

• The protection group on cluster-newyork is assigned the primary role.

• The protection group on cluster-paris has either the primary role or secondary role, depending on whether cluster-paris could be reached during the takeover from cluster-newyork.

1. If the original primary cluster, cluster-paris, has been down, confirm that the cluster is booted and that the Sun Cluster Geographic Edition infrastructure is enabled on the cluster.

   For more information about booting a cluster, see Booting a Cluster in Sun Cluster Geographic Edition System Administration Guide.

2. Resynchronize the original primary cluster, cluster-paris, with the current primary cluster cluster-newyork.

   The cluster cluster-paris forfeits its own configuration and replicates the cluster-newyork configuration locally. Resynchronize both the partnership and protection group configurations.

   1. On cluster-paris, deactivate the protection group on the local cluster.

      phys-paris-1# geopg stop -e local protectiongroupname

      -e local

      Specifies the scope of the command.

      By specifying a local scope, the command operates on the local cluster only.

      ---

      Note –

      The property values, such as global and local, are not case sensitive.

      ---

      protectiongroupname

      Specifies the name of the protection group.

      If the protection group is already deactivated, the state of the resource group in the protection group is probably Error because the application resource groups are managed and offline.

      If you deactivate the protection group, the application resource groups are no longer managed, clearing the Error state.

   2. On cluster-paris, resynchronize the partnership.

      phys-paris-1# geops update partnershipname

      ---

      Note –

      You need to perform this step only once, even if you are resynchronizing multiple protection groups.

      ---

      For more information about synchronizing partnerships, see Resynchronizing a Partnership in Sun Cluster Geographic Edition System Administration Guide.

   3. On cluster-paris, resynchronize each protection group.

      Because the role of the protection group on cluster-newyork is primary, this step ensures that the role of the protection group on cluster-paris is secondary.

      phys-paris-1# geopg update protectiongroupname

      For more information about synchronizing protection groups, see Resynchronizing an Oracle Data Guard Protection Group.

3. On cluster-paris, validate the configuration for each protection group.

   phys-paris-1# geopg validate protectiongroupname

   For more information, see How to Validate an Oracle Data Guard Protection Group.

4. On cluster-paris, activate each protection group.

   When you activate a protection group, the protection group's application resource groups are also brought online.

   phys-paris-1# geopg start -e global protectiongroupname

   -e global

   Specifies the scope of the command.

   By specifying a global scope, the command operates on both clusters where the protection group is located.

   ---

   Note –

   The property values, such as global and local, are not case sensitive.

   ---

   protectiongroupname

   Specifies the name of the protection group.

   ---

   Caution –

   Do not use the -n option because the data needs to be synchronized from the current primary cluster, cluster-newyork, to the current standby cluster, cluster-paris.

   Because the protection group has a role of secondary, the data is synchronized from the current primary cluster, cluster-newyork, to the current standby cluster, cluster-paris.

   For more information about the geopg start command, see How to Activate an Oracle Data Guard Protection Group.

   ---

5. Confirm that all data is synchronized.

   1. Confirm that the state of the protection group on cluster-newyork is OK.

      phys-newyork-1# geoadm status

      Refer to the Protection Group section of the output.

   2. Confirm that all resources in the replication resource group, ODGprotectiongroupname-odg-rep-rg, report a status of OK.

      phys-newyork-1# clresource status -v ODGprotectiongroupname-odg-rep-rs

How to Perform a Failback Switchover on a System That Uses Oracle Data Guard Replication

Follow this procedure to restart an application on the original primary cluster, cluster-paris, after the data on the cluster has been resynchronized with the data on the current primary cluster, cluster-newyork.

The failback procedures apply only to clusters in a partnership. You need to perform the following procedure only once for each partnership.

Before You Begin

Before you perform a failback switchover, a takeover has occurred on cluster-newyork. The clusters now have the following roles:

• The protection group on cluster-newyork is assigned the primary role.

• The protection group on cluster-paris has either the primary role or the secondary role, depending on whether the cluster-paris cluster could be reached during the takeover from the cluster-newyork cluster.

1. If the original primary cluster, cluster-paris, failed, confirm that the cluster is restarted and that the Sun Cluster Geographic Edition infrastructure is enabled on the cluster.

   For more information about restarting a cluster, see Booting a Cluster in Sun Cluster Geographic Edition System Administration Guide.

2. Recover and restore the failed Oracle Data Guard primary database as the new standby.

   Refer to the Oracle documentation, which describes how to perform this step.

3. Ensure that the original primary cluster, cluster-paris, is working correctly as part of the Oracle Data Guard configuration.

   oracle (phys-paris-1)$ dgmgrl sys/sysdba_password@sales-svc DGMGRL> show configuration;

   If the original primary cluster, cluster-paris, is working correctly, the show configuration command displays the SUCCESS state.

   If the original primary cluster was down at the point of failure, it is marked as a deactivated primary cluster. If the original primary cluster was up at the point of failure, it is marked as a deactivated secondary.

4. Resynchronize the original primary cluster, cluster-paris, with the current primary cluster cluster-newyork.

   The cluster cluster-paris forfeits its own configuration and replicates the cluster-newyork configuration locally. Resynchronize both the partnership and protection group configurations.

   1. On cluster-paris, resynchronize the partnership.

      phys-paris-1# geops update partnershipname

      ---

      Note –

      You need to perform this step only once for each partnership, even if you are performing a failback switchover for multiple protection groups in the partnership.

      ---

      For more information about synchronizing partnerships, see Resynchronizing a Partnership in Sun Cluster Geographic Edition System Administration Guide.

   2. Determine whether the protection group on the original primary cluster, cluster-paris, is active.

      phys-paris-1# geoadm status

   3. If the protection group on the original primary cluster is active, stop the protection group.

      phys-paris-1# geopg stop -e local protectiongroupname

      -e local

      Specifies the scope of the command.

      By specifying a local scope, the command operates on the local cluster only.

      ---

      Note –

      The property values, such as global and local, are not case sensitive.

      ---

      protectiongroupname

      Specifies the name of the protection group.

      If the protection group is already deactivated, the state of the resource group in the protection group is probably Error because the application resource groups are managed and offline.

      If you deactivate the protection group, the application resource groups are no longer managed, clearing the Error state.

   4. Verify that the protection group is stopped.

      phys-paris-1# geoadm status

   5. On cluster-paris, resynchronize each protection group.

      Because the local role of the protection group on the cluster-newyork cluster is now primary, this step ensures that the role of the protection group on the cluster-paris cluster becomes secondary.

      phys-paris-1# geopg update protectiongroupname

      For more information about synchronizing protection groups, see Resynchronizing an Oracle Data Guard Protection Group.

5. On cluster-paris, validate the configuration for each protection group.

   A protection group cannot be started when it is in an Error state. Ensure that the protection group is not in an Error state.

   phys-paris-1# geopg validate protectiongroupname

   For more information, see How to Validate an Oracle Data Guard Protection Group.

6. On cluster-paris, activate each protection group.

   When you activate a protection group, its application resource groups are also brought online.

   phys-paris-1# geopg start -e global protectiongroupname

   -e global

   Specifies the scope of the command.

   By specifying a global scope, the command operates on both clusters where the protection group is located.

   ---

   Note –

   The property values, such as global and local, are not case sensitive.

   ---

   protectiongroupname

   Specifies the name of the protection group.

7. Confirm that the data is completely synchronized.

   1. Confirm that the state of the protection group on cluster-newyork is OK.

      phys-newyork-1# geoadm status

      Refer to the Protection Group section of the output.

   2. Confirm that all resources in the replication resource group, ODGprotectiongroupname-odg-rep-rg, report a status of OK.

      phys-newyork-1# clresource status -v ODGprotectiongroupname-odg-rep-rs

8. On both partner clusters, ensure that the protection group is activated.

   phys-paris-1# geoadm status … phys-newyork-1# geoadm status …

9. For each protection group, on either cluster, perform a switchover from cluster-newyork to cluster-paris.

   phys-node-n# geopg switchover [-f] -m cluster-paris protectiongroupname

   For more information, see How to Switch Over an Oracle Data Guard Protection Group From the Primary to the Standby Cluster.

   The cluster-paris cluster resumes its original role as primary cluster for the protection group.

10. Ensure that the switchover was performed successfully.

    phys-node-n# geoadm status

    Verify that the protection group is now primary on cluster-paris and secondary on cluster-newyork and that the states that are shown for the Data replication and the Resource groups properties are OK on both clusters.

11. Check the runtime status of the application resource group and data replication for each Oracle Data Guard protection group.

    phys-node-n# clresourcegroup status -v resourcegroupname # clresource status -v ODGConfigurationName-odg-rep-rs

    Refer to the Status and Status Message fields that are presented for the Oracle Data Guard Broker configuration that you want to check. For more information about these fields, see Table 2–1.

    For more information about the runtime status of data replication, see Checking the Runtime Status of Oracle Data Guard Data Replication.

How to Perform a Failback Takeover on a System That Uses Oracle Data Guard Replication

Follow this procedure to restart an application on the original primary cluster, cluster-paris, and to use the current data on the original primary cluster.

Any updates that occurred on the standby cluster, cluster-newyork, while it was acting as primary are discarded.

The failback procedures apply only to clusters in a partnership. You need to perform the following procedure only once for each partnership.

Conditionally, you can resume using the data on the original primary cluster-paris. However, you must not have replicated data from the new primary, cluster-newyork, to the original primary cluster, cluster-paris, at any point after the takeover operation on cluster-newyork.

Before You Begin

Before you begin the failback takeover procedure, the clusters must have the following roles:

• The protection group on cluster-newyork is assigned the primary role.

• The protection group on cluster-paris has either the primary role or secondary role, depending on whether the protection group was reached during the takeover.

1. If the original primary cluster, cluster-paris, failed, confirm that the cluster is restarted and that the Sun Cluster Geographic Edition infrastructure is enabled on the cluster.

   For more information about restarting a cluster, see Booting a Cluster in Sun Cluster Geographic Edition System Administration Guide.

2. Recover and revert the new Oracle Data Guard primary database to a standby for the original primary to a point in time before the original primary failed.

   Refer to the Oracle documentation, which describes how to perform this step.

   ---

   Note –

   You might need to use the dgmgrl command to remove and recreate the Oracle Data Guard Broker configuration.

   ---

3. Ensure that the original primary cluster, cluster-paris, is again working correctly as the primary as part of the Oracle Data Guard configuration.

   oracle (phys-paris-1)$ dgmgrl sys/sysdba_password@sales-svc DGMGRL> show configuration;

   If the original primary cluster, cluster-paris, is working correctly, the show configuration command displays the SUCCESS state.

   If the original primary cluster was up at the point of failure, it is marked as a deactivated secondary cluster. Also, the original standby cluster is marked as an activated primary.

   If the original primary cluster was down at the point of failure, it is marked as a deactivated primary cluster. Also, the original standby cluster is marked as an activated primary.

4. Was the original primary cluster, cluster-paris, up or down at the point of failure?

   • If the original primary cluster, cluster-paris, was down at the point of failure, update the original standby cluster, cluster-newyork, to a secondary.

     1. On the original standby cluster, that is, the cluster that has become the new primary cluster, stop the protection group.

        phys-newyork-1# geopg stop -e local protectiongroupname

     2. On the original standby cluster, that is, the cluster that has become the new primary cluster, update the protection group.

        phys-newyork-1# geopg update protectiongroupname

        The roles are now correct, but both clusters are marked as deactivated.

        For more information about synchronizing protection groups, see How to Resynchronize an Oracle Data Guard Protection Group.

     3. On cluster-paris and on cluster-newyork, locally validate the configuration for each protection group.

        Ensure that the protection group is not in an Error state. You cannot start a protection group when the protection group is in an Error state.

        phys-paris-1# geopg validate protectiongroupname phys-newyork-1# geopg validate protectiongroupname

        For more information, see How to Validate an Oracle Data Guard Protection Group.

     4. From any node in either cluster, globally activate the protection group on both clusters.

        # geopg start -e global protectiongroupname

        Once the protection groups are activated on both clusters, you have successfully performed the failback takeover.

   • If the original primary cluster, cluster-paris, was up at the point of failure, determine the status of the secondary (that is, the original primary) configuration.

     phys-newyork-1# geoadm status

     • If the Configuration status is set to OK, synchronize the configurations.

       1. Initiate a takeover for each protection group on the original primary cluster-paris.

          phys-paris-1# geopg takeover [-f] protectiongroupname

       2. If the configuration for the original standby cluster, cluster-newyork, is marked as Error, validate the configuration for each protection group.

          cluster-newyork# geopg validate protectiongroupname

          For more information, see How to Validate an Oracle Data Guard Protection Group.

       3. Globally activate the protection groups on both clusters.

          cluster-newyork# geopg start -e global protectiongroupname

          Once the protection groups are activated on both clusters, you have successfully performed the failback takeover.

     • If the Configuration status is set to Error, resolve the problem.

       1. Deactivate the secondary (that is, the original primary) configuration that is in the Error state.

          phys-newyork-1# geopg stop -e local protectiongroupname

       2. Force a takeover to make the secondary configuration a primary configuration again and to match the underlying Oracle dgmgrl configuration.

          phys-newyork-1# geopg takeover -f protectiongroupname

       3. On both the cluster-paris and on the cluster-newyork clusters, locally validate the configuration for each protection group.

          phys-paris-1# geopg validate protectiongroupname phys-newyork-1# geopg validate protectiongroupname

          For more information, see How to Validate an Oracle Data Guard Protection Group.

       4. From any node in either cluster, globally activate the protection group on both clusters.

          # geopg start -e global protectiongroupname

          Once the protection groups are activated on both clusters, you have successfully performed the failback takeover.