Automate Database Role Changes for Manually Configured Oracle Data Guard in OCI Database Services Using OCI Full Stack DR and Custom Scripts

Introduction

Oracle Cloud Infrastructure Full Stack Disaster Recovery (OCI Full Stack DR) orchestrates the transition of compute, database, and applications between Oracle Cloud Infrastructure (OCI) regions from around the globe with a single click. Customers can automate the steps needed to recover one or more business systems without redesigning or re-architecting existing infrastructure, databases, or applications and without needing specialized management or conversion servers.

OCI Full Stack DR offers fully integrated support for various Oracle Database services on OCI. These databases can be added as members of a OCI Full Stack DR protection group, enabling coordinated disaster recovery operations.

For Oracle Databases managed services in OCI, it is strongly recommended to configure Oracle Data Guard using the OCI Console, Oracle Cloud Infrastructure Command Line Interface (OCI CLI), or OCI SDKs. Doing so ensures that OCI Full Stack DR can automatically detect the Data Guard configuration and generate built-in plan groups for role transitions as part of your DR plan.

However, there are scenarios where manual Oracle Data Guard configuration (outside of OCI’s native interfaces) becomes necessary due to specific technical or operational requirements, such as:

In such cases, while the OCI Database services control plane may not recognize the Oracle Data Guard setup, OCI Full Stack DR still offers flexibility. You can handle role transitions by creating custom scripts and integrating them into user-defined plan groups within your DR plans.

Please note that this solution is not compatible with Oracle Database Cloud services where Data Guard configurations are managed through the OCI Console, SDKs, or APIs.

In this tutorial, we will walk through a standardized approach for managing Oracle Data Guard role transitions using custom database handler scripts for OCI Database services where Oracle Data Guard has been configured manually.

Note: This custom script solution applies to the following OCI Database services:

Architecture Description

In this tutorial, we will use Oracle Base Database Service with two DB Systems deployed across two OCI regions, where Oracle Data Guard has been configured manually.

custom-data-guard-configuration
Figure A: Custom Data Guard configuration using Oracle Base Database Service

Definitions and Assumptions throughout the Tutorial

Objectives

The following tasks will be covered in this tutorial:

Prerequisites

We will use the following resources to start with the tutorial.

Resources Region 1 - Ashburn Region 2 - Phoenix
Compartment app app
DB System adghol-12345 adghol-12345
DB name adghol adghol
DB unique name adghol-site0 adghol-site1
DB role primary standby
Compute VM script-iad script-phx
Bucket IAD PHX

Note: Complete all the required prerequisites before proceeding further. These steps lay the foundation for a smooth and successful OCI Full Stack DR setup.

  1. Admin Access or Required Oracle Cloud Infrastructure Identity and Access Management (OCI IAM) Policies.

    Ensure you have administrator privileges or configure the necessary OCI IAM policies and dynamic groups to use OCI Full Stack DR. In this solution, the database handler script internally launches an OCI container instance, so you must add policies accordingly.

    Note: Replace all occurrences of <compartment_ocid> and <compartment_name> with your actual OCI compartment OCID and name.

    1. Create a Dynamic Group: Create a dynamic group with the example name (FullStackDR_Database_DG) and add the following matching rules:

       Any {instance.compartment.id = '<compartment_ocid>'}
 Any {resource.type = 'instance', resource.compartment.id = '<compartment_ocid>'}
 Any {resource.type = 'computecontainerinstance', resource.compartment.id = '<compartment_ocid>'}
 Any {resource.type = 'drprotectiongroup', resource.compartment.id = '<compartment_ocid>'}

    2. Create an OCI IAM Policy: Create a policy with the example name (FullStackDR_Database_Group_Policies) and add the following allow statements:

       Allow dynamic-group FullStackDR_Database_DG to read secret-family in compartment <compartment_name>
 Allow dynamic-group FullStackDR_Database_DG to manage virtual-network-family in compartment <compartment_name>
 Allow dynamic-group FullStackDR_Database_DG to manage instance-agent-command-family in compartment <compartment_name>
 Allow dynamic-group FullStackDR_Database_DG to manage instance-agent-command-execution-family in compartment <compartment_name>
 Allow dynamic-group FullStackDR_Database_DG to manage objects in compartment <compartment_name>
 Allow dynamic-group FullStackDR_Database_DG to manage database-family in compartment <compartment_name>
 Allow dynamic-group FullStackDR_Database_DG to manage compute-container-family in compartment <compartment_name>

    For more information, see OCI Disaster Recovery Policies – Official Docs and Configuring IAM Policies (Oracle Blog).

  2. Provision OCI Compute instances in both regions: Create OCI Compute instance in each region to serve as Jumphost for hosting and executing the scripts.For detailed steps, see Create OCI Instance. For simplicity, we will refer to this OCI Compute instance as the Jumphost throughout the tutorial.If you already have existing OCI Compute instances that can function as Jumphost, you can skip this step. Make sure the Jumphost has the Oracle Cloud Agent is running and run command plugin is enabled. For more details, see Oracle Cloud Agent.

  3. Access to Run Commands on OCI Compute Instances: Ensure that you set up the run command prerequisites in the jumphost, as we are using user defined plan groups for running scripts during DR operations. For more information, see Running Commands on an Instance.

  4. Install OCI CLI in the jumphost on both regions: Based on the operating system of the jumphost, install OCI CLI on both regions and make sure you can invoke OCI CLI commands, in the script we will use instance principal. For more information, see OCI CLI Installation.

  5. Set Up VCNs in both Regions with Remote VCN Peering: Create VCNs in both the primary and standby regions and setup remote VCN peering. This is required for setting up the cross region Oracle Data Guard. For more information, see OCI Base DB networking configuration.

  6. Manual Oracle Data Guard Configuration: Manually configure the Oracle Data Guard setup based on the requirements with Oracle Data Guard broker. For more information, see Oracle Data Guard Broker and Oracle Clusterware.

  7. Download the Database Handler Scripts in the Jumphost: The scripts should be downloaded and kept in the jumphost in both regions.

    1. Download the Oracle Data Guard database handler scripts from the following repository: Data Guard DB handler scripts.

    2. Copy the scripts to the /home/opc/ directory (or any other preferred path) on the jumphost in both regions.

    3. Ensure the script files have executable permissions.

    4. full_stack_dr_non_std_db_handler.py is the Python script responsible for handling the Oracle Data Guard role transitions.The associated bash scripts are provided as templates and can be modified to suit your specific requirements. Do not modify the Python role change script itself.

  8. Create OCI Vault and Secrets: You must create an OCI Vault and store database credentials as secrets in both regions.

    1. Create an OCI Vault in each region using the OCI Console or CLI.
    2. Create a secret within the vault to store the SYS user password for the database.

    For more information, see OCI Vaults.

  9. Connectivity Checks from the Jumphost: Ensure the OCI Database service, OCI Vault service, and OCI container instance service are accessible from the compute instance. This is required because the OCI Full Stack DR scripts perform introspection to fetch database details from both Primary and Standby regions.

    1. Connectivity must work from the the jumphost. Run the following commands from the jumphost.

        # Primary Region
  curl -v telnet://database.<primary_region>.oraclecloud.com:443
  curl -v telnet://secrets.vaults.<primary_region>.oci.oraclecloud.com:443
  curl -v telnet://iaas.<primary_region>.oraclecloud.com:443
  curl -v telnet://compute-containers.<primary_region>.oci.oraclecloud.com:443

  # Standby Region
  curl -v telnet://database.<standby_region>.oraclecloud.com:443
  curl -v telnet://secrets.vaults.<standby_region>.oci.oraclecloud.com:443
  curl -v telnet://iaas.<standby_region>.oraclecloud.com:443
  curl -v telnet://compute-containers.<standby_region>.oci.oraclecloud.com:443

      Note: Replace <primary_region> and <standby_region> with actual OCI region identifiers.
      For example:

      • us-ashburn-1 for Ashburn
      • us-phoenix-1 for Phoenix

      For a complete list, see OCI Region Identifiers.

      Expected Output: Each command should return a message similar to Connected to ....

      If any connection fails, check the security lists, route tables, and service gateway configuration of the VCN/subnets of the jumphost.

  10. Create Object Storage buckets: Create OCI Object Storage buckets in the primary and standby regions to store logs generated by OCI Full Stack DR during recovery operations as explained here: Preparing Log Location for Operation Logs.

  11. Database Handler Script Usage and Customization: You can download the database handler scripts from the mentioned GitHub repo. Here is the database handler script usage and the required parameters.

    db-handler-script.png
    Fig B: DB handler script usage

    Supported --db_operation options are:

    • SWITCHOVER
    • SWITCHOVER_PRECHECK
    • FAILOVER
    • FAILOVER_PRECHECK
    • CONVERT_PHYSICAL_TO_SNAPSHOT_STANDBY
    • CONVERT_PHYSICAL_TO_SNAPSHOT_STANDBY_PRECHECK
    • REVERT_SNAPSHOT_TO_PHYSICAL_STANDBY_PRECHECK
    • REVERT_SNAPSHOT_TO_PHYSICAL_STANDBY

    OCI Full Stack DR expects all required parameters to be passed when running the database handler script. For better usability and repeatability, it is recommended to create a wrapper bash script that:

    • Provides the required parameters.
    • Enables logging for auditing and troubleshooting.

    Sample Example: Database Switchover Script (db-switchover-iad-phx.sh).

    #!/bin/bash

# Define log file with date and time
LOG_FILE="db-switchover-iad-phx-$(date +%Y%m%d_%H%M%S).log"

# Define Python script and argument
PYTHON_SCRIPT="full_stack_dr_non_std_db_handler.py"
ARGUMENT="--database_ocid="ocid1.database.oc1.phx.xxxxxxxx" --vault_ocid="ocid1.vaultsecr
et.oc1.phx.xxxxx" --region="us-phoenix-1" --primary_db_unique_name="adghol_site0" --st
andby_db_unique_name="adghol_site1" --drpg_ocid="ocid1.drprotectiongroup.oc1.phx.axxxxxxax
" --db_operation="SWITCHOVER" --auth_type=INSTANCE_PRINCIPAL"

# Execute Python script and log output
echo "Executing Python script: $PYTHON_SCRIPT with argument: $ARGUMENT" | tee -a $LOG_FILE
/usr/bin/python3 $PYTHON_SCRIPT $ARGUMENT 2>&1 | tee -a $LOG_FILE

echo "Execution completed. Logs saved in $LOG_FILE"

    Note: Store this wrapper script in the same location as your database handler scripts and ensure it is executable. OCID’s are anonymized in the script.

    chmod +x db-switchover-wrapper.sh

    DR Plan Script Mapping where DB running in Region 1 (Ashburn) is Primary and Region 2 (Phoenix) is Standby

    DR Plan Type Target Instance Script Name Comment
    Switchover script-phx db-prechk-switchover-iad-phx.sh Prechk DB Switchover from IAD to PHX
    Switchover script-phx db-switchover-iad-phx.sh DB Switchover from IAD to PHX
    Failover script-phx db-prechk-failover-iad-phx.sh Prechk DB Failover from IAD to PHX
    Failover script-phx db-failover-iad-phx.sh DB Failover from IAD to PHX
    Start drill script-phx db-prechk-startdrill-phx.sh Prechk Start DR Drill in PHX
    Start drill script-phx db-startdrill-phx.sh Start DR Drill in PHX
    Stop drill script-phx db-prechk-stopdrill-phx.sh Prechk Stop DR Drill in PHX
    Stop drill script-phx db-stopdrill-phx.sh Stop DR Drill in PHX

    DR Plan Script Mapping where DB running in Region 2 (Phoenix) is Primary and Region 2 (Ashburn) is Standby

    DR Plan Type Target Instance Script Name Comment
    Switchover script-iad db-prechk-switchover-phx-iad.sh Prechk DB Switchover from PHX to IAD
    Switchover script-iad db-switchover-phx-iad.sh DB Switchover from PHX to IAD
    Failover script-iad db-prechk-failover-phx-iad.sh Prechk DB Failover from PHX to IAD
    Failover script-iad db-failover-phx-iad.sh DB Failover from PHX to IAD
    Start drill script-iad db-prechk-startdrill-iad.sh Prechk Start DR Drill in IAD
    Start drill script-iad db-startdrill-iad.sh Start DR Drill in IAD
    Stop drill script-iad db-prechk-stopdrill-iad.sh Prechk Stop DR Drill in IAD
    Stop drill script-iad db-stopdrill-iad.sh Stop DR Drill in IAD

Note: For better clarity and usability, we have created multiple bash wrapper scripts tailored to specific DR plan types and regions. These scripts use a shared Python script for database role transitions, which you can customize to suit your own requirements and environment.

Task 1: Verify the Oracle Data Guard Configuration and Update the Tag

In this task, we will verify the Oracle Data Guard manual configuration using Oracle Base Database Service. We will create a *tag** on the database to indicate a non-standard Oracle Data Guard setup. This allows OCI Full Stack DR to create a DR plan without relying on any built-in plan groups.

  1. Log in to the OCI Console and navigate to Oracle Database and click Oracle Base Database Service.

  2. Ensure the OCI region context is set to Region 1 (Ashburn).

  3. Select the DB system, in our example it is adghol0-12345.

    iad-db-system.png
    Figure 1.1: DB system in region 1

  4. Navigate to the Databases tab and select the database adghol.

    iad-db-system-db.png
    Figure 1.2: Database in region 1

  5. Verify the Data Guard Status is Not enabled, this confirms Oracle Data Guard is not set up using OCI Console.

    iad-db-dataguard-cp-status.png
    Figure 1.3: Data Guard control plane status in region 1

  6. Navigate to the Tags tab and create the free-form tags. You must replace the value of Peer DB OCID based on your setup. In this example, you must add database OCID of the Phoenix region.

    Tag Key Value
    FsdrNonStandardDataGuardPeerDatabaseId Peer DB OCID
    FsdrNonStandardDataGuardFlag True

    iad-db-dataguard-tags
    Figure 1.4: DB tags in region 1

  7. Go to the DB Systems and select Nodes.

    iad-db-node.png
    Figure 1.5: DB node in region 1

    Note: This is a Non-RAC setup, so you will see a single database node. If this were an Oracle Real Application Clusters setup, you would see two nodes.

  8. Connect to the database node and switch to the oracle user. Use the Oracle Data Guard broker to verify the roles.

    dgmgrl
show configuration

    iad-db-dataguard-status.png
    Figure 1.6: Data Guard status in region 1

    Expected Output:

    • adghol_site0 should appear as the Primary Database.
    • adghol_site1 should appear as the Physical Standby Database.
    • Configuration Status should appear Success.

    If there are errors and the database roles are not as expected, you must fix the errors using the Oracle Data Guard documentation.

  9. Log in to the OCI Console and navigate to Oracle Database and click Oracle Base Database Service.

  10. Ensure the OCI region context is set to Region 2 (Phoenix).

  11. Select the DB system, in our example it is adghol1-12345

    phx-db-system.png
    Figure 1.7: DB system in region 2

  12. Navigate to the Databases tab and select the database adghol.

    phx-db-system-db.png
    Figure 1.8: Database in region 2

  13. Verify the Data Guard Status is Not enabled, this confirms Oracle Data Guard is not set up using OCI Console.

    phx-db-dataguard-cp-status.png
    Figure 1.9: Data Guard control plane status in region 2

  14. Navigate to the Tags tab and create the free-form tags. You must replace the value of Peer DB OCID based on your setup. In this example, you must add database OCID of the Ashburn region.

    Tag Key Value
    FsdrNonStandardDataGuardPeerDatabaseId Peer DB OCID
    FsdrNonStandardDataGuardFlag True

    phx-db-dataguard-tags
    Figure 1.10: DB tags in region 2

  15. Repeat step 7 and 8, but this time connect to the database node in Region 2 (Standby Region).

    • Ensure you are logged in as the oracle user on the Region 2 database node.
    • Verify the Oracle Data Guard roles using dgmgrl just like in step 8.

Task 2: Create and Associate DR Protection Groups

Create DR protection groups in Region 1 and Region 2 if the protection groups for this application stack do not exist yet.

Task 2.1: Create a Protection Group in Region 1

  1. Go to the OCI Console and navigate to DR Protection Groups as shown in Figure 2.1.

    1. Ensure the OCI region context is set to Region 1 (Ashburn).
    2. Click Migration & Disaster Recovery.
    3. Click DR Protection Groups.

    dg-iad-drpg.png
    Figure 2.1: Navigate to DR protection groups

  2. Create a basic DR protection group in Region 1 as shown in Figure 2.2. The peer, role and members will be assigned in later steps.

    1. Select the compartment where you want the DR protection group to be created.
    2. Click Create DR protection group.
    3. Use a meaningful name for the DR protection group.
    4. Select OCI Object Storage bucket for OCI Full Stack DR logs.
    5. Click Create.

    dg-drpg-create-iad-finish.png
    Figure 2.2: Parameters needed to create DR protection group in region 1

Task 2.2: Create a Protection Group in Region 2

  1. Go to the OCI Console, navigate to DR Protection Groups as shown in Figure 2.3.

    1. Ensure the OCI region context is set to Region 2 (Phoenix).
    2. Click Migration & Disaster Recovery.
    3. Click DR Protection Groups.

    dg-iad-drpg.png
    Figure 2.3: Navigate to DR protection groups

  2. Create a basic DR protection group in Region 2 as shown in Figure 2.4. The peer, role and members will be assigned in later steps.

    1. Select the compartment where you want the DR protection group to be created.
    2. Click Create DR protection group.
    3. Use a meaningful name for the DRPG.
    4. Select OCI Object Storage bucket for OCI Full Stack DR logs.
    5. Click Create.

    dg-drpg-create-phx-finish.pn
    Figure 2.4: Parameters needed to create DR protection group in region 2

Task 2.3: Associate Protection Groups in Region 1 and Region 2

Associate the DRPGs in each region as peers of each other and assign the peer roles of primary and standby. The roles of primary and standby are automatically changed by OCI Full Stack DR as part of any DR operation/DR plan execution; there is no need to manage the roles manually at any time.

  1. Go to the DR protection group details page.

    1. Ensure OCI region context is set to Region 1 (Ashburn).
    2. Click Actions drop-down menu and click Associate to begin the process.

    drpg-assoc-begin-iad.png
    Fig 2.5: Begin DRPG association

  2. Enter the following information.

    1. Role: Select Primary role. OCI Full Stack DR will assign the standby role to Region 2 automatically.
    2. Peer region: Select Region 2 (Phoenix), where the other DR protection group was created.
    3. Peer DR protection group: Select the peer DR protection group that was created.
    4. Click Associate.

    drpg-assoc-finish-iad.png
    Fig 2.6: Parameters needed to associate the DRPGs

OCI Full Stack DR will show something like as shown in the following image, once the association is completed.

drpg-assoc-completed-iad.png
Fig 2.7: Showing the peer relationship from the individual DRPG perspective

The same information can be found whenever the context/view is from a global perspective showing all DR protection groups as shown in the following image.

drpg-assoc-completed-dxb.png
Fig 2.8: Showing the peer relationship from the global DRPG perspective

Task 3: Add Members to the DR Protection Groups

In this task, we will add the following OCI resources to the primary DR protection group in Region 1.

  1. The OCI Compute instance hosting the database handler script will be added as a non-moving VM.
  2. The primary DB system.

Task 3.1: Add Members to DR Protection Group in Region 1

  1. Select the DR protection group in Region 1 as shown in the following image.

    1. Ensure the OCI region context is Region 1 (Ashburn).
    2. Select the DR protecion group in Region 1.
    3. Navigate to the Members tab.
    4. Click Manage members.

    drpg-add-nav-iad.png
    Fig 3.1: How to begin adding members to DR protection group in region 1

  2. Add a compute instance for the database handler scripts.

    1. Select Add member
    2. Select Instance under Compute as a member Resource type.
    3. Select the compute Instance hosting the database handler scripts.
    4. Select Non-moving instance.
    5. Click Add.

    drpg-add-compute-iad.png
    Fig 3.2: Compute Instance Add to the DRPG in Region 1

    Verify the added compute instance.

    drpg-add-compute-iad-added.png
    Fig 3.2: Compute Instance added to the DRPG in Region 1

  3. Add the primary database. Click Add Member

    1. Select Add member
    2. Select Oracle Database -> Database (Base DB,ExaDB-D,ExaCC,ExaXS) as member Resource type.
    3. Select Oracle Base Database as Database type.
    4. Select Database system.
    5. Select Database home.
    6. Select Database.
    7. Select Database password secret
    8. Click Add

    drpg-add-db-iad.png
    Fig 3.3: Parameters needed to add the Primary DB

    Verify the added primary database and publish both the members.

    drpg-add-db-iad-complete.png
    Fig 3.4: Primary DB added to the DRPG in Region 1

    drpg-add-db-iad-complete1.png
    Fig 3.5: Publish members to the DRPG in Region 1

    After few minutes both members should be available under the members.

    drpg-add-db-iad-published.png
    Fig 3.6: Members added to the DRPG in Region 1

Task 3.2: Add Members to DR Protection Group in Region 2

  1. Select the DR protection group in Region 2 as shown in the following image.

    1. Ensure the OCI region context is Region 2 (Phoenix).
    2. Select the DR protecion group in Region 2.
    3. Navigate to the Members tab.
    4. Click Manage members.

    drpg-add-nav-phx.png
    Fig 3.7: How to begin adding members to DR protection group in region 2

  2. Add a compute instance for the database handler scripts.

    1. Select Add member
    2. Select Instance under Compute as a member Resource type.
    3. Select the compute Instance hosting the database handler scripts.
    4. Select Non-moving instance.
    5. Click Add.

    drpg-add-compute-phx.png
    Fig 3.8: Compute Instance Add to the DRPG in Region 2

    Verify the added compute instance.

    drpg-add-compute-phx-added.png
    Fig 3.9: Compute Instance added to the DRPG in Region 2

  3. Add the standby database. Click Add Member

    1. Select Add member
    2. Select Oracle Database -> Database (Base DB,ExaDB-D,ExaCC,ExaXS) as member Resource type.
    3. Select Oracle Base Database as Database type.
    4. Select Database system.
    5. Select Database home.
    6. Select Database.
    7. Select Database password secret
    8. Click Add

    drpg-add-db-phx.png
    Fig 3.10: Parameters needed to add the standby DB

    Verify the added standby database and publish both the members.

    drpg-add-db-phx-complete.png
    Fig 3.11: Standby DB added to the DRPG in Region 2

    drpg-add-db-phx-complete1.png
    Fig 3.12: Publish members to the DRPG in Region 2

    After few minutes both members should be available under the members.

    drpg-add-db-phx-published.png
    Fig 3.13: Members added to the DRPG in Region 2

Task 4: Create and Customize the DR Plans in Region 2

In this task, we will create the initial Switchover, Failover, and Start Drill plans associated with the standby DR protection group in Region 2 (Phoenix).

These plans are designed to seamlessly transition the workload from the primary region (Region 1) to the standby region (Region 2).

DR plans are always created in the protection group holding the standby role.

Since Region 2 (Phoenix) is currently the standby, we will create all initial DR plans there.

Task 4.1: Create DR Plans

  1. Create DR plans by selecting the DRPG in Region 2 (Phoenix)

    1. Ensure the OCI region context is Region 2 (Phoenix).
    2. Select the standby DRPG in Region 2.
    3. Navigte to Plans tab.
    4. Click Create Plan.

    plan-create-nav-phx.png
    Fig 4.1: How to begin creating basic DR plans in Region 2

  2. Create a switchover plan.

    1. Enter a simple but meaningful Name for the switchover plan. The name should be as short as possible but easy to understand at a glance to help reduce confusion and human error during a crisis.
    2. Select Plan type as Switchover (planned).

    plan-create-so-phx-png
    Fig 4.2: The parameters needed to create DR switchover plan

  3. Create a failover plan.

    Follow the same process to create a basic failover plan as shown in the following image.

    1. Enter Name of the failover plan simple but meaningful.
    2. Select Plan type as Failover (unplanned).

    plan-create-fo-phx.png
    Fig 4.3: The parameters needed to create DR failover plan

  4. Create a start drill plan.

    Follow the same process to create a basic failover plan as shown in the following image.

    1. Enter Name of the start drill plan simple but meaningful.
    2. Select Plan type as Failover (unplanned).

    plan-create-startdrill-phx.png
    Fig 4.4: The parameters needed to create DR start drill plan

    The standby DR protection group in Region 2 should now have the three DR plans as shown in the following image. These will handle transitioning workloads from Region 1 to Region 2. You will create similar plans at Region 1 to transition workloads from Region 2 back to Region 1 in a later task.

    plan-create-phx-completed.png
    Fig 4.5: Showing the three DR plans that must exist in region 2 before proceeding any further

Note: OCI Full Stack DR allows you to create a stop drill plan only after the start drill plan has been executed successfully, and the DR protection group is in the Inactive (Drill in progress) state.

Task 4.2: Customise the DR Plans with User-Defined Plan Groups

The DR plans created in Task 4.1 will not generate any built-in plan groups for the Oracle database role transitions since the Oracle Data Guard setup was done manually.

In this task, you will:

This ensures that your DR plans can fully handle failover, switchover, and drill scenarios involving a manually configured Data Guard environment.

  1. Navigate to the switchover plan created in Task 4.1 and select Plan groups.

    plan-custom-so-phx-nav.png
    Fig 4.6: How to begin customizing the switchover plan in region 2

  2. Start by adding custom, user-defined DR plan groups to tailor the DR workflow to the specific needs of the Oracle database role changes. This plan groups will invoke the necessary scripts from the jumphost script-iad in Region 1.

  3. Add a user-defined plan group to DB Switchover.

    1. Click Add group.
    2. Enter a simple but descriptive plan group Name. In this example, we will use DB Switchover.

    3. Click Add Step to open the dialog where we will specify the script to perform the DB role change.

      plan-custom-so-phx-grp-db-role-change.png
      Fig 4.7: Parameters to create plan group to perform db role change

      In Add plan group step, enter the following information.

    4. Enter a simple but descriptive Step name. In this example, we will use DB Switchover from IAD to PHX.
    5. Select Instance region which contains the instance on which this step will run. In this example, the script will be executed on the jumphost in Region 2, so select Phoenix.
    6. Select Run local script.
    7. Select Target instance which is the jumphost script-phx in Region 2.
    8. In Script parameters, enter the full path of the script with the parameters. For example: /home/opc/db-switchover-iad-phx.sh.
    9. In Run as user, enter opc.

    10. Click Add Step.

      plan-custom-so-phx-grp-db-role-change-step.png
      Fig 4.8: Parameters to add a step for db role change

    11. Verify the added step.

      plan-custom-so-phx-grp-db-role-change-step-added.png
      Fig 4.9: Added for db role change

    12. Click Add.

      plan-custom-so-phx-grp-db-role-change-add
      Fig 4.10: DB role change group add

      Plan group will be available now.

      plan-custom-so-phx-grp1.png
      Fig 4.11: DB switchover plan group

  4. Add user-defined precheck step to a switchover DR plan. The user-defined precheck will be executed along with built-in precheck steps.

    1. Go to Plan Groups, click … infront of Prechecks-Built and select Add user defined precheck.

      plan-custom-so-phx-grp-db-role-precheck-change.png
      Fig 4.12: Add custom precheck for DB switchover

    2. Enter a simple but descriptive Step name. In this example, we will use DB Switchover precheck from IAD to PHX.
    3. Select Instance region which contains the instance on which this step will run. In this example, the script will be executed on the jumphost in Region 2, so select Phoenix.
    4. Select Run local script.
    5. Select Target instance which is the jumphost script-phx in Region.
    6. In Script parameters, enter the full path of the script with the parameters. For example: /home/opc/db-prechk-switchover-iad-phx.sh.
    7. In Run as user, enter opc.

    8. Click Add Step.

      plan-custom-so-phx-grp-db-precheck-role-change-step.png
      Fig 4.13: Parameters to add a custom precheck step for db role change

    9. Verify the added step.

      plan-custom-so-phx-grp-db-role-precheck-change-step-added.png
      Fig 4.14: Added custom precheck step for db role change

      The customization of the switchover plan has been successfully completed.

      plan-custom-so-phx-complete.png
      Fig 4.15: Final Switchover plan

  5. Similarly customize the failover, start drill plans, use the correct Target Instance, Scripts using the tabular details available in Prerequisites: DB handler script usage and customization.

  6. Failover and Start Drill plans will look as below once the user-defined plan group and custom precheck step are added.

    plan-custom-fo-phx-complete.png
    Fig 4.16: Final failover plan

    plan-custom-startdrill-phx-complete.png
    Fig 4.17: Final start drill plan

Task 5: Run Prechecks for the DR Plans in Region 2

With switchover, failover, start drill DR plans have been successfully created in the standby Region 2. These plans enable OCI Full Stack DR to transition workloads from Region 1 to Region 2 or perform DR drill. The subsequent task involves running the prechecks for the DR plans to ensure readiness and validate the transition process.

Task 5.1: Begin Prechecks for the Switchover Plan

Run prechecks for the switchover DR plan.

  1. Ensure the region context is set to standby Region 2.
  2. Ensure the correct DR protection group in Region 2 is selected, it should be the standby role.
  3. Click Actions drop-down menu.
  4. Click Run prechecks.
  5. Select the DB Switchover from IAD to PHX plan.
  6. Enter Plan execution name, if not it will auto generate.

  7. Click Run prechecks.

    prechecks-so-phx-begin.png
    Fig 5.1: Showing how to run prechecks of the switchover plan

  8. Verify the Succeeded status from the Plan executions tab.

    prechecks-so-phx-complete.png
    Fig 5.2: Showing a Completed prechecks of the switchover plan

Task 5.2: Begin Prechecks for the Failover Plan

Run the prechecks for the failover DR plan.

  1. Ensure the region context is set to standby Region 2.
  2. Ensure the correct DR protection group in Region 2 is selected, it should be the standby role.
  3. Click Actions drop-down menu.
  4. Click Run prechecks.
  5. Select the DB Failover from IAD to PHX plan.
  6. Enter Plan execution name, if not it will auto generate.

  7. Click Run prechecks.

    prechecks-fo-phx-begin.png
    Fig 5.3: Showing how to run prechecks of the failover plan

  8. Verify the Succeeded status from the Plan executions tab.

    prechecks-fo-phx-complete.png
    Fig 5.4: Showing a Completed prechecks of the failover plan

Task 5.3: Begin Prechecks for the Start Drill Plan

Run the prechecks for the start drill DR plan.

  1. Ensure the region context is set to standby Region 2.
  2. Ensure the correct DR protection group in Region 2 is selected, it should be the standby role.
  3. Click Actions drop-down menu.
  4. Click Run prechecks.
  5. Select the Start drill in PHX plan.
  6. Enter Plan execution name, if not it will auto generate.

  7. Click Run prechecks.

    prechecks-stadr-phx-begin.png
    Fig 5.5: Showing how to run prechecks of the start drill plan

  8. Verify the Succeeded status from the Plan executions tab.

    prechecks-stardr-phx-complete.png
    Fig 5.6: Showing a Completed prechecks of the failover plan

Task 6: Run the Switchover Plan in Region 2

Run the switchover DR plan to initiate the Oracle database role transition from Region 1 (Primary) to Region 2 (Standby).

  1. Ensure the region context is set to standby Region 2.
  2. Ensure the correct DR protection group in Region 2 is selected, it should be the standby role.
  3. Click Actions drop-down menu.

  4. Click Execute Plan.

    execute-so-phx-begin.png
    Fig 6.1: Showing how to Run the switchover plan

  5. Select the DB Switchover from IAD to PHX plan.
  6. This task runs the switchover plan in Region 2.
  7. Deselect Enable prechecks, since the prechecks was already executed in Task 5. If you want to run again, you can enable it.
  8. Enter Plan execution name, if not it will auto generate.

  9. Click Execute plan.

    exec-so-phx-begin.png
    Fig 6.2: Showing how to execute the switchover plan

  10. Navigate to Plan Executions tab and select the switchover plan execution.

    exec-so-phx-in-progress.png
    Fig 6.3: Showing the switchover plan in Progress

  11. Monitor the switchover plan until the complete workload has been fully transitioned from Region 1 to Region 2. The execution of the switchover plan was successfully completed in approximately 8 minutes.

    exec-so-phx-complete.png
    Fig 6.4: Showing a Completed switchover plan execution.

  12. You can verify the database role status using the details provided in Task 1.8.

    iad-db-dataguard-status-afterso.png
    Figure 6.5: Data Guard status after switchover

    Expected Output:

    • adghol_site1 should appear as the Primary Database.
    • adghol_site0 should appear as the Physical Standby Database.
    • Configuration Status should appear Success.

  13. The role in the DR protection group will be swapped, Region 2 will show as Primary and Region 1 will show as Standby.

    drpg-roles-after-so-phx
    Fig 6.6: DR protection role change after switchover

Task 7: Create and Customize DR Plans in Region 1

With the successful execution of the switchover DR plan from Region 1 to Region 2, Region 2 has now assumed the role of the primary, while Region 1 has transitioned to the standby role.

Follow the same approach detailed in Task 4 to create and customize the switchover, failover, and DR Drill plans within the DR protection group for Region 1, which now serves as the standby peer region.

Once these plans are created and updated with user-defined plan groups and a custom pre-check step, they will look like as the following images.

plan-create-iad.png
Fig 7.1: Created DR plans in Region 1

plan-so-customize-iad.png
Fig 7.2: Switchover plan in Region 1

plan-fo-customize-iad.png
Fig 7.3 : Failover plan in Region 1

plan-sd-customize-iad.png
Fig 7.4 : Start drill plan in Region 1

If needed, you can run the switchover plan to promote the database in Region 1 back to the primary role, while the database in Region 2 becomes the standby.

Similarly, you can run the start drill plan to simulate a disaster recovery event, and then create and execute the stop drill plan to revert the system to its original state.

Next Steps

For more information, see the OCI Full Stack DR, Oracle Base Database Service, Oracle Data Guard documentation in the Related Links section.

Acknowledgments

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.