Automate Cold Disaster Recovery for OCI Database with PostgreSQL using OCI Full Stack Disaster Recovery

Introduction

Oracle Cloud Infrastructure Full Stack Disaster Recovery (OCI Full Stack DR) orchestrates the transition of compute, database, and applications between OCI regions from around the globe with a single click. Customers can automate the steps needed to recover one or more business systems without redesigning or re-architecting existing infrastructure, databases, or applications and without needing specialized management or conversion servers.

OCI Database with PostgreSQL is a fully managed PostgreSQL-compatible service with intelligent sizing, tuning, and high durability. The service automatically scales storage as database tables are created and dropped, making management easier on you and optimizing storage spend. Data is encrypted both in-transit and at rest. OCI Database with PostgreSQL is designed for high availability by offering durability even in the event of an availability domain (AD) failure.

In this tutorial, you will learn how to automate a cold disaster recovery for OCI Database with PostgreSQL. It outlines the procedures for utilizing the OCI Full Stack DR service to manage the switchover and failover processes.

Note: This type of Disaster Recovery (DR) strategy relies on a backup and restore mechanism, making it best suited for non-critical applications where the business requirements for Recovery Time Objectives (RTO) and Recovery Point Objectives (RPO) are not overly demanding.

Architecture Description

The architecture presented in this tutorial showcases a typical Web application running on OCI virtual machine (VM) seamlessly integrated with OCI Database with PostgreSQL.

An OCI Load Balancer is deployed within a public subnet in both regions to efficiently manage external user connections. Application users are routed to the currently active backends in Region X through DNS Traffic Steering.

Description of the illustration fsdr_psql_backup_restore_dr-Physical_Architecture.png

Disaster Recovery Architecture Description

The DR strategy for this web application involves a comprehensive approach, including the complete replication of the boot volume of the VM with volume group replication.

For OCI Database with PostgreSQL, backup copy is enabled to regularly copy the automatic backup to the remote region to ensure data protection and disaster recovery readiness.

Description of the illustration fsdr_psql_backup_restore_dr-Physical_DR_Architecture.png

The recovery solution for this deployment requires OCI Full Stack DR to run a series of custom Python scripts during a recovery operation such as a failover or switchover.

The scripts referenced in this tutorial are provided by the EMEA Technology Engineering team and available here: full-stack-disaster-recovery, which is specifically tailored for this DR solution.

This tutorial explains how to download the scripts and how to use them in a later task.

Note: The following scripts are provided for generic guidance. You can either use your own scripts or customize the scripts according to your corporate policy and security requirements.

A load balancer is pre-provisioned in the remote region, ensuring it is ready to handle traffic seamlessly when the web application VM is transitioned to the remote region during a switchover or failover scenario, as visible in the following diagram, the seamless switchover of the load balancer is guaranteed by a DNS Steering Policy with a healthcheck attached routing to the currently available load balancer backend-set. For more information, see OCI Traffic Management.

The load balancer listener is set up on ports 80 (frontend) and 8000 (API) and respectively routed to ports 3000 and 8585 in default configuration.

Description of the illustration fsdr_psql_backup_restore_dr-Physical_Network_Architecture.png

The following diagram illustrates the workflow for the automatic backup copy from the primary region to the remote region.

Description of the illustration fsdr_psql_backup_restore_dr-Logical_Workflow_Auto_Copy_Backup_to_Remote.png

In this case, backups could be scheduled to run daily and then copied to a remote region for additional redundancy, resulting in a Recovery Point Objective (RPO) of 24 hours.

Note:

• Make sure to enable automatic backup and enable backup copy to the remote region. Without this step, the restore process may fail due to the backup not being available in the secondary region.
• While in this tutorial we focus on automating the tasks of creating, transferring to another region and creating a new database system from the backup those very same steps can be carried out manually by following our OCI Database with PostgreSQL documentation outlined.
  • Backing Up a Database System Manually.
  • Copying an OCI Database with PostgreSQL Backup to Another Region.
  • Creating a Database System from a Backup.

If the use case requires a shorter RPO, the solution would be to schedule more frequent backups and use the provided scripts (psql_create_bkp.py and psql_copy_bkp.py) to manually trigger backups and copy them to the remote region, in accordance with business requirements. Additionally psql_copy_config.py can be used to copy the OCI Database with PostgreSQL configuration to the remote region which is necessary to restore the database systems extensions and parameters.

Description of the illustration fsdr_psql_backup_restore_dr-Logical_Workflow_Manual_Backup_Copy_to_Remote.png

All the scripts are available on GitHub and are thoroughly detailed in the following sections.

Note: Make sure to schedule this script (or a similar one) as per your business requirements to regularly take a backup and copy it to the remote region. Without this step, the restore process may fail due to the backup not being available in the secondary region.

How the Recovery Works?

After a planned switchover is executed, the roles will reverse: the primary workload will run in Region 2, while the standby will operate in Region 1. The architecture will appear as follows:

Description of the illustration fsdr_psql_backup_restore_dr-Physical_Switchover_Architecture.png

In the current setup, we leverage the OCI private DNS service to manage a DNS record that directs traffic to the active OCI Database with PostgreSQL endpoint. During the recovery process, this DNS record is updated through a custom script(psql_update_dns.py) to reflect the new OCI Database with PostgreSQL, ensuring seamless switchover or failover and continuity of service.

The following diagram illustrates the workflow for restoring the most recent OCI Database with PostgreSQL backup in the standby region, which then becomes the new primary region.

Description of the illustration fsdr_psql_backup_restore_dr-Logical_Workflow_Switchover_to_Remote.png

Definitions and Assumptions throughout the Tutorial

• Regions:

  • Region 1 (Dubai): Dubai will initially serve as the primary region. However, this role will transition to standby during the switchover process, which will be performed in the later tasks as part of the disaster recovery plan.

  • Region 2 (Abu Dhabi): Abu Dhabi will initially function as the standby region. This role will later transition to primary following the switchover process, which will be carried out in the subsequent tasks as part of the disaster recovery procedure.

• Compartments: You are free to organize this deployment and OCI Full Stack DR into any compartment scheme that works within your standards for IT governance. We have chosen to organize all the OCI resources for this tutorial in one single compartment.

Web Application Virtual Machine

The web application in this tutorial is meant to showcase the end to end scenario architecture, a single deployment script (deploy_application_demo.sh) will configure and deploy the application stack in the web application VM. Further instructions about deploying the application can be found in the repository here: link-to-web-app-sample

Frontend Preview: Rows visible in the report are sourced directly from OCI Database with PostgreSQL.

Description of the illustration webapp-frontend-preview.png

Objectives

The following tasks will be covered in this tutorial:

1. Task 1: Prepare the environment for Disaster Recovery.
2. Task 2: Create DR Protection Groups (DRPG) in both Regions.
3. Task 3: Add Members to the DR Protection Group.
4. Task 4: Create Basic DR Plans in Region 2.
5. Task 5: Customize the Switchover Plan in Region 2.
6. Task 6: Customize the Failover Plan in Region 2.
7. Task 7: Run the Prechecks of the DR plans in Region 2.
8. Task 8: Run the switchover plan in Region 2.
9. Task 9: Create and Customize the DR plans in Region 1.
10. Task 10: Run the Failover Plan in Region 1.

Task 1: Prepare the Environment for Disaster Recovery

Task 1.1: Create Volume Groups and Enable Replication

Create a volume group for the sample web application VM in Region 1 and ensure it is replicated in Region 2. Ensure the boot volume for each application VM (Sample WebApp) is a member of the volume group and the volume group is replicated to Region 2.

The following images displaying the created volume group, which includes the boot volumes of the web application VM, with replication successfully enabled to Region 2. For more information, see Creating a Volume Group.

Description of the illustration psql-webapp-create-vol-grp.png

Description of the illustration psql-webapp-create-vol-grp-2.png

Description of the illustration psql-webapp-create-vol-grp-3.png

After configuring cross region replication, click Next until you get to the Summary and click Create.

Task 1.2: Prepare Web Application VM for Automated Deployment

1. Download/Clone the GitHub repository from here: (oci-postgressql-colddr) in /home/opc folder.

2. Make the script executable.

   chmod +x deploy_application_demo.sh
   

3. Update the deploy_application_demo.sh script with the necessary details for the web application and OCI Database with PostgreSQL in both regions.

   • WEBAPP_HOME: Root location of the deployment (location where you execute the script from). For example, /home/opc.
   • WEBAPP_URL: The URL pointing to the compute with the API http://webapi.yoururl.eu:8000.
   • PRIMARY_REGION: OCI primary region. Format is eu-frankfurt-1.
   • STANDBY_REGION: OCI standby region.
   • PRIMARY_SECRET_OCID: The OCI vault secret OCID with the PostgreSQL password in primary region (ocid1.vaultsecret.oc1.me-dubai-1.xxxxxx).
   • STANDBY_SECRET_OCID: The OCI vault secret OCID with the PostgreSQL password in standby region (ocid1.vaultsecret.oc1.me-abudhabi-1.xxxxxx).
   • PG_USER: The database user used for loading the sample data(postgresql_sample.sql).
   • PG_DB: The database used for loading the sample data. Default value is postgres.
   • PG_HOST: The FQDN of the private zone entry for the database.

   Note:

   • Ensure all values are accurately updated.
   • Other parameters can be left unchanged when using the default configuration.

4. Execute the deploy_application_demo.sh script.

   ./deploy_application_demo.sh
   

   This will deploy the frontend, backend, configure services and firewall.

   After successful deployment, the message Deployment Complete! is displayed.

As part of the tutorial, we will use this same VM for running user-defined scripts. Ensure the VM acting as the DR control node has been configured to run commands. For more information, see Invoke custom scripts using the run command with Oracle Cloud Infrastructure Full Stack Disaster Recovery.

Task 1.3: Create Web Steering Policy

To guarantee that our application always points to the active region, a failover traffic management steering policy can be leveraged. The steering policy in combination with OCI Health Checks service validates the application endpoint every 60 seconds (Default TTL) through the HTTP Monitor.

1. Go to the OCI Console and navigate to Networking.

2. Click Traffic management steering policies.

   

   Description of the illustration oci-web-steering-policies-nav.png

3. Click Create traffic management steering policy.

   

   Description of the illustration psql-webapp-dns-create.png

   1. Select Failover as Policy type.

      1. Enter a Name for the steering policy.
      2. Select the compartment for the steering policy.
      3. Select a TTL as per your requirements.

      

      Description of the illustration psql-webapp-dxb-dns-create-failover.png

   2. Configure Answer Pool 1, pointing to the load balancer in Region 1.

      1. Enter a meaningful name for the pool.
      2. Select A record as Type.
      3. Enter the IP of the web application load balancer in Region 1.

      

      Description of the illustration psql-webapp-dxb-dns-create-pool-1.png

   3. Configure Answer Pool 2, pointing to the load balancer in Region 2.

      1. Enter a Name for the pool.
      2. Select A record as Type.
      3. Enter the IP of the web application load balancer in Region 2.

      

      Description of the illustration psql-webapp-dxb-dns-create-pool-2.png

   4. Configure Pool Priority, this context priority is given to Region 1.

      Description of the illustration psql-webapp-dxb-dns-create-pool-priority.png

   5. Click Add New to attach a OCI Health Checks policy.

      1. Select the Request type as HTTP.
      2. Enter a Name for the health check.
      3. Select the compartment for the health check.

      

      Description of the illustration psql-webapp-dxb-dns-create-healthcheck.png

   6. Click Advanced options.

      1. Enter the port of the web application, in this case 80.
      2. Enter the path of the web application, in this case /.
      3. Select GET as Method.
      4. Select a timeout for the request.

      

      Description of the illustration psql-webapp-dxb-dns-create-healthcheck-details.png

   7. Configure Attached domains.

      1. Enter the subdomain for the webapi endpoint.
      2. Select the compartment containing the zone (domain).
      3. Select the appropriate zone.
      4. Repeat steps 1 to 3 for the webapp endpoint.

      

      Description of the illustration psql-webapp-dxb-dns-create-domains.png

   8. Verify if all the details are correct and click Create traffic management steering policy. After some time the creation should show completed.

      

      Description of the illustration psql-webapp-dxb-dns-create-complete.png

   9. You can see the overview of the steering policy post creation. At the end of the page, the Attached health check can be seen.

      

      Description of the illustration psql-webapp-dxb-dns-create-details.png

      1. Click Attached health check.
      2. Navigate to Health check history.

      3. Here you can view the load balancer currently available as well as past availability.

         Description of the illustration psql-webapp-dxb-healthcheck-history.png

Task 1.4: Create Oracle Cloud Infrastructure Identity and Access Management Policies for OCI Full Stack DR

To configure the required OCI IAM policies for OCI Full Stack DR, see:

• Policies for OCI Full Stack DR

• Configuring Identity and Access Management (IAM) policies to use Full Stack DR

Task 1.5: Create OCI IAM Policies for other Services managed by OCI Full Stack DR

OCI Full Stack DR must have the ability to control and manage other key OCI services such as compute, networking, storage and other miscellaneous services. To configure the required OCI IAM policies for other services, see Policies for Other Services Managed by Full Stack Disaster Recovery and OCI IAM policies.

Task 2: Create DR Protection Groups (DRPG) in both Regions

Create DR protection groups in Region 1 and Region 2 if the protection groups for this application stack do not exist yet.

Task 2.1: Create a Protection Group in Region 1

1. Go to the OCI Console and navigate to DR Protection Groups.

   1. Ensure the OCI region context is set to Region 1 (Dubai).
   2. Click Migration & Disaster Recovery.
   3. Click DR Protection Groups.

   

   Description of the illustration psql-webapp-dxb-drpg-intro.png

2. Create a basic DR protection group (DRPG) in Region 1. The peer, role and members will be assigned in later steps.

   1. Select the compartment where you want the DRPG to be created.
   2. Click Create DR protection group to open the dialog.
   3. Enter Name for the DRPG.
   4. Select OCI Object Storage bucket for OCI Full Stack DR logs.
   5. Click Create.

   

   Description of the illustration psql-webapp-dxb-drpg.png

Task 2.2: Create a Protection Group in Region 2

1. Go to the OCI Console, navigate to DR Protection Groups.

   1. Ensure the OCI region context is set to Region 2 (Abu Dhabi).
   2. Click Migration & Disaster Recovery.
   3. Click DR Protection Groups.

   

   Description of the illustration psql-webapp-dxb-drpg-intro.png

2. Create a basic DR protection group (DRPG) in Region 2. The peer, role and members will be assigned in later steps.

   1. Select the compartment where you want the DRPG to be created.
   2. Click Create DR protection group to open the dialog.
   3. Enter Name for the DRPG.
   4. Select OCI Object Storage bucket for OCI Full Stack DR logs.
   5. Click Create.

   

   Description of the illustration psql-webapp-auh-drpg.png

Task 2.3: Associate Protection Groups in Region 1 and Region 2

Associate the DRPGs in each region as peers of each other and assign the peer roles of primary and standby. The roles of primary and standby are automatically changed by OCI Full Stack DR as part of any DR operation/DR plan execution; there is no need to manage the roles manually at any time.

1. Go to the DR protection group details page.

   1. Ensure OCI region context is set to Region 1 (Dubai).
   2. Click Associate to begin the process.

   

   Description of the illustration psql-webapp-dxbauh-drpg.png

2. Enter the parameters as shown in the following image.

   1. Role: Select Primary role. OCI Full Stack DR will assign the standby role to Region 2 automatically.
   2. Peer region: Select Region 2 (Abu Dhabi), where the other DRPG was created.
   3. Peer DR protection group: Select the peer DRPG that was created.
   4. Click Associate.

   

   Description of the illustration psql-webapp-dxbauh-drpg-associate.png

OCI Full Stack DR will show something like as shown in the following image, once the association is completed.

1. The current primary peer DRPG is Dubai (region 1).
2. The current standby peer DRPG is Abu Dhabi (region 2).

Description of the illustration psql-webapp-dxbauh-drpg-primary.png

The same information can be found whenever the context/view is from a global perspective showing all DR protection groups as shown in the following image.

1. The current primary peer DRPG is Dubai (region 1).
2. The current standby peer DRPG is Abu Dhabi (region 2).

Description of the illustration psql-webapp-dxbauh-drpg-standby.png

Task 3: Add Members to the DR Protection Group

In this task, we will add the following OCI resources to the primary DRPG in Region 1.

1. The compute instances hosting the web application will be added as a moving VM.
2. The volume group that contains the boot volume of the web application compute node.
3. The primary load balancer.

Task 3.1: Add Members to DRPG in Region 1

1. Select the DRPG in Region 1 as shown in the following image.

   1. Ensure the OCI region context is Region 1 (Dubai).
   2. Select the DRPG in Region 1.
   3. Select Members.
   4. Click Add Member to begin the process.

   

   Description of the illustration psql-webapp-dxb-drpg-add.png

2. Add a compute instance for the web application VM.

   1. Acknowledge warning about DR plans.
   2. Enter Compute as a member Resource type.
   3. Select the compute instance hosting the web application.
   4. Select Moving instance.
   5. Click Add VNIC mapping to select which VCN and subnet to assign to the VNIC(s) at Region 2 during a recovery.
   6. Click Show advanced options.
   7. In Settings, select Retain fault domain.
   8. Verify the details and click Add.

   

   Description of the illustration psql-webapp-dxb-drpg-add-compute.png

   

   Description of the illustration psql-webapp-dxb-drpg-add-compute-vnic.png

   

   Description of the illustration psql-webapp-dxb-drpg-add-compute-vnic-details.png

   

   Description of the illustration psql-webapp-dxb-drpg-add-compute-detail.png

3. Add the block volume group containing boot volume of the web application VM.

   1. Acknowledge warning about DR plans.
   2. Select Volume group as member Resource type.
   3. Ensure the correct compartment containing the volume group is selected and select the volume group.
   4. Verify the details and click Add.

   

   Description of the illustration psql-webapp-dxb-drpg-add-volgrp.png

4. In this example, we will add OCI Load Balancer as a member of the DRPG in Region 1.

   1. Acknowledge warning about DR plans.
   2. Select Load Balancer as member Resource type.
   3. Ensure correct compartments for load balancer is selected and select the load balancer you want to add.
   4. Select Destination load balancer to be used in Region 2.
   5. Select Source Backend Set, this is the backend set used by the web application VM. An OCI Load Balancer can be shared among multiple applications and may have multiple backend sets configured. During a DR switchover, only the backend sets specified here will have their configuration moved to the standby region.
   6. Select Destination Backend Set, this is the empty backend set created in Region 2.
   7. Verify the details and click Add.

   

   Description of the illustration psql-webapp-dxb-drpg-add-lb.png

   

   Description of the illustration psql-webapp-dxb-drpg-add-lb-details.png

5. Verify the members section contains the load balancer, volume group and the compute instance.

   

   Description of the illustration psql-webapp-dxb-drpg-members.png

Task 3.2: Add Members to DRPG in Region 2

1. Select the DRPG in Region 2 as shown in the following image.

   1. Ensure the OCI region context is Region 2 (Abu Dhabi).
   2. Select the DRPG in Region 2.
   3. Select Members.
   4. Click Add Member to begin the process.

   

   Description of the illustration psql-webapp-auh-drpg-add.png

2. In this example, we will add OCI Load Balancer as a member of the DRPG in Region 2.

   1. Acknowledge warning about DR plans.
   2. Select Load Balancer as member Resource type.
   3. Ensure correct compartment for load balancer is selected and select the load balancer you want to add.
   4. Select the Destination load balancer to be used in Region 1.
   5. Select Source Backend Set, this is the backend set used by the web application VM. An OCI Load Balancer can be shared among multiple applications and may have multiple backend sets configured. During a DR switchover, only the backend sets specified here will have their configuration moved to the standby region.
   6. Select Destination Backend Set, this backend set is created in Region 2.
   7. Verify the details and click Add.

   

   Description of the illustration psql-webapp-aux-drpg-add-lb.png

   

   Description of the illustration psql-webapp-dxb-drpg-add-lb-details.png

3. Verify the members section contains the load balancer.

   

   Description of the illustration psql-webapp-aux-drpg-members.png

Task 4: Create Basic DR Plans in Region 2

In this task, we will create the initial switchover and failover plans associated with the standby DR protection group in Region 2 (Abu Dhabi).

The purpose of these plans is to seamlessly transition the workload from the primary region (Region 1) to the standby region (Region 2). As part of any DR operation, the roles of the DR protection groups in both regions are automatically reversed: the protection group in Region 1 becomes the standby, while the protection group in Region 2 assumes the primary role following a failover or switchover.

OCI Full Stack DR will pre-populate these plans with built-in steps derived from the member resources added during the previous tasks. These plans will later be customized to manage tasks specific to OCI Database with PostgreSQL during the recovery process.

Switchover plans are always created within the protection group holding the standby role. Since Region 2 (Abu Dhabi) is currently the standby protection group, we will begin creating the plans there.

Task 4.1: Create DR Plans

1. Create a basic plan by selecting the DRPG in Region 2 (Abu Dhabi)

   1. Ensure the OCI region context is Region 2 (Abu Dhabi).
   2. Select the standby DRPG in Region 2.
   3. Select Plans.
   4. Click Create Plan to begin the process.

   

   Description of the illustration psql-webapp-auh-drpg-create.png

2. Create a switchover plan.

   1. Enter a simple but meaningful Name for the switchover plan. The name should be as short as possible but easy to understand at a glance to help reduce confusion and human error during a crisis.
   2. Select Plan type as Switchover (planned).
   3. Click Create.

   

   Description of the illustration psql-webapp-auh-drpg-create-switchover.png

3. Create a failover plan. Follow the same process to create a basic failover plan as shown in the following image.

   1. Enter Name of the failover plan.
   2. Select Plan type as Failover (unplanned).
   3. Click Create.

   

   Description of the illustration psql-webapp-auh-drpg-create-failover.png

The standby DR protection group in Region 2 should now have the two DR plans as shown in the following image. These will handle transitioning workloads from Region 1 to Region 2. You will create similar plans at Region 1 to transition workloads from Region 2 back to Region 1 in a later task.

Description of the illustration psql-webapp-auh-drpg-plans.png

Task 5: Customize the Switchover Plan in Region 2

The basic DR plans created in Task 4 contain pre-populated steps for recovery tasks that are built into OCI Full Stack DR and do not contain anything to manage recovery tasks specific to OCI Database with PostgreSQL. This task explains how to add custom, user-defined DR plan groups and steps to manage the tasks that need to be accomplished during a switchover:

• Create a database backup after gracefully shutting down the web application VM to ensure data consistency.
• Transfer the latest database backup to the remote OCI Region 2 for redundancy and disaster recovery.
• Restore the most recent database backup in Region 2 to prepare the environment for switchover.
• Update the private database DNS record, enabling the application VM to seamlessly connect to the database in Region 2.
• Terminate the source OCI Database with PostgreSQL database in Region 1.

Task 5.1: Select the Switchover Plan

Navigate to the switchover plan created in Task 4. In Task 5.3, you will add new groups.

Description of the illustration psql-webapp-auh-drpg-create-switchover-add.png

Task 5.2: (Optional) Enable DR Plan Groups that Terminate Artifacts

There are three plan groups that are disabled by default in the switchover plans, as shown in the following image. These plan groups are disabled to provide reassurance during testing, ensuring that no artifacts are deleted and that a viable backup copy remains intact in case of issues during the test phase.

However, these three plan groups are designed to terminate (delete) artifacts that will no longer be needed for any future DR operations. Without these plan groups enabled, unused artifacts will continue to accumulate over time as you perform switchovers between the two regions, which can lead to confusion about which compute instances and volume groups should be active.

Optionally, enabling these plan groups now will help avoid the need for manual clean up of unnecessary artifacts before going into production. This proactive step can streamline the transition to production and maintain a cleaner, more manageable environment.

Description of the illustration psql-webapp-auh-drpg-create-switchover-details-1.png

1. To enable the plan groups, select Enable all steps from the context menu to the right of the plan group name.

   

   Description of the illustration psql-webapp-auh-drpg-create-switchover-enable.png

   

   Description of the illustration psql-webapp-auh-drpg-create-switchover-enable-check.png

2. Verify that all the Plan groups are enabled.

   

   Description of the illustration psql-webapp-auh-drpg-create-switchover-enable-complete.png

Task 5.3: Create a Plan group to Run Custom Scripts

Start by adding custom, user-defined DR plan groups to tailor the DR workflow to the specific needs of the OCI Database with PostgreSQL backup/restore DR process.

These plan groups will invoke the necessary scripts from the web application VM in Region 1. This would ensure that the OCI Database with PostgreSQL is fully restored and operational before the application VM are brought online.

The following custom plan should be added to the pre-populated switchover plan: PostgreSQL - Switchover, PostgreSQL - Update DNS and PostgreSQL - Terminate.

1. Add a custom plan group to execute the switchover for OCI Database with PostgreSQL. This operation entails, backing up the database in the Primary Region, copying the database configuration to the Standby region, copying the database backup to the Standby region and finally restoring the database in the Standby region.

   1. Click Add group.
   2. Enter a simple but descriptive plan group Name. In this example, we will use PostgreSQL - Switchover.
   3. Select a position where the plan group will be inserted into the DR plan. In this example, select Add after to insert our user-defined plan group after the built-in plan group Compute Instances - Launch.
   4. Click Add Step to open the dialog where we will specify the script to execute the PostgreSQL switchover.

   

   Description of the illustration psql-webapp-auh-drpg-create-switchover-add-psql-so.png

   In Add plan group step, enter the following information.

   1. Enter Step name. In this example, we will use PostgreSQL - Switchover. Same as the plan group name.
   2. Select a region which contains the instance on which this step will run. In this example, the script will be executed on the web application VM in Region 1.
   3. Select Run local script.
   4. Select Target instance which is the web application VM in Region 1.
   5. In Script parameters, enter the full path of the script with the parameters. For example: /home/opc/full-stack-dr-scripts-main/oci-postgresql/backup-restore/psql_exec_cold_dr.py -c amo-psql-dbs.json -o switchover.
   6. In Run as user, enter opc.
   7. Click Add Step.

   

   Description of the illustration psql-webapp-auh-drpg-create-switchover-add-psql-so-details.png

   Click Add.

   

   Description of the illustration psql-webapp-auh-drpg-create-switchover-add-psql-details.png

2. Add a custom plan group to update the PostgreSQL database DNS.

   1. Click Add group.
   2. Enter a plan group Name. In this example, we will use PostgreSQL - Update DNS.
   3. Select a position where the plan group will be inserted into the DR plan. In this example, select Add after to insert our user-defined plan group after the built-in plan group PostgreSQL - Switchover.
   4. Click Add Step to open the dialog where we will specify the script to update the PostgreSQL database DNS.

   

   Description of the illustration psql-webapp-auh-drpg-create-switchover-add-psql-dns.png

   In Add plan group step, enter the following information.

   1. Enter Step name. In this example, we will use PostgreSQL - Update DNS. Same as the plan group name.
   2. Select a region which contains the instance on which this step will run. In this example, the script will be executed on the web application VM in Region 1.
   3. Select Run local script.
   4. Select the Target instance which is the web application VM in Region 1.
   5. In Script parameters, enter the full path of the script with the parameters. For example: /home/opc/full-stack-dr-scripts-main/oci-postgresql/backup-restore/psql_update_dns.py -c amo-psql-dbs.json -d amo-psql-dbs.amo.vcn01.internal -o switchover.
   6. In Run as user, enter opc.
   7. Click Add Step.

   

   Description of the illustration psql-webapp-auh-drpg-create-switchover-add-psql-dns-details.png

   Click Add.

   

   Description of the illustration psql-webapp-auh-drpg-create-switchover-add-dns-details.png

3. Add a custom plan group to terminate the PostgreSQL source database.

   1. Click Add group.
   2. Enter a plan group Name. In this example, we will use PostgreSQL - Terminate.
   3. Select a position where the plan group will be inserted into the DR plan. In this example, select Add after to insert our user-defined plan group after the built-in plan group Volume Groups - Remove from DR Protection Group.
   4. Click Add Step to open the dialog where we will specify the script to terminate the PostgreSQL database source.

   

   Description of the illustration psql-webapp-auh-drpg-create-switchover-add-psql-terminate.png

   In Add plan group step, enter the following information.

   1. Enter Step name. In this example, we will use PostgreSQL - Terminate. Same as the plan group name.
   2. Select a region which contains the instance on which this step will run. In this example, the script will be executed on the web application VM in Region 1.
   3. Select Run local script.
   4. Select the Target instance which is the web application VM in Region 1.
   5. In Script parameters, enter the full path of the script with the parameters. For example: /home/opc/full-stack-dr-scripts-main/oci-postgresql/backup-restore/psql_exec_cold_dr.py -c amo-psql-dbs.json -o terminate.
   6. In Run as user, enter opc.
   7. Click Add Step.

   

   Description of the illustration psql-webapp-auh-drpg-create-switchover-add-psql-terminate-details.png

   Click Add.

   

   Description of the illustration psql-webapp-auh-drpg-create-switchover-add-terminate-details.png

4. Verify that the custom plan groups created show Enabled status.

   Description of the illustration psql-webapp-auh-drpg-create-switchover-details-2.png

The customization of the switchover plan has been successfully completed.

Task 6: Customize the Failover Plan in Region 2

In this task, add custom, user-defined DR plan groups and steps to manage the tasks that need to be accomplished during a failover.

• Restore the most recent database backup in Region 2 to prepare the environment for switchover.

• Update the private database DNS record, enabling the application VM to seamlessly connect to the database in Region 2.

Task 6.1: Select the Failover Plan

Navigate to the failover plan created in Task 4.

Description of the illustration psql-webapp-auh-drpg-create-failover-details-1.png

Task 6.2: Create a Plan group to Run Custom Scripts in Region 2

Start by adding custom, user-defined DR plan groups to tailor the DR workflow to the specific needs of the OCI Database with PostgreSQL Backup/Restore DR process.

These plan groups will invoke the necessary scripts from the web application VM.

The following custom plan should be added to the pre-populated failover plan: PostgreSQL - Failover and PostgreSQL - Update DNS.

1. Add custom plan group to execute the PostgreSQL failover script.

   1. Click Add group.
   2. Enter a plan group Name. In this example, we will use PostgreSQL - Failover.
   3. Select a position where the plan group will be inserted into the DR plan. In this example, select Add after to insert our user-defined plan group after the built-in plan group Compute Instances - Launch.
   4. Click Add Step to open the dialog where we will specify the script to failover the PostgreSQL database.

   

   Description of the illustration psql-webapp-auh-drpg-create-failover-add-psql.png

   In Add plan group step, enter the following information.

   1. Enter a Step name. In this example, we will use PostgreSQL - Failover. Same as the plan group name.
   2. Select a region which contains the instance on which this step will run. In this example, the script will be executed on the web application VM in Region 1.
   3. Select Run local script.
   4. Select the Target instance which is the web application VM in Region 1.
   5. In Script parameters, enter the full path of the script with the parameters. For example: /home/opc/full-stack-dr-scripts-main/oci-postgresql/backup-restore/psql_exec_cold_dr.py -c amo-psql-dbs.json -o failover.
   6. In Run as user, enter opc.
   7. Click Add Step.

   

   Description of the illustration psql-webapp-auh-drpg-create-failover-add-psql-fo-details.png

   Click Add

   

   Description of the illustration psql-webapp-auh-drpg-create-failover-add-psql-details.png

2. Add custom plan group to update PostgreSQL database DNS.

   1. Click Add group.
   2. Enter a plan group Name. In this example, we will use PostgreSQL - Update DNS.
   3. Select a position where the plan group will be inserted into the DR plan. In this example, select Add after to insert our user-defined plan group after the built-in plan group PostgreSQL - Failover.
   4. Click Add Step to open the dialog where we will specify the script to update the PostgreSQL database DNS.

   

   Description of the illustration psql-webapp-auh-drpg-create-failover-add-dns.png

   In Add plan group step, enter the following information.

   1. Enter a Step name. In this example, we will use PostgreSQL - Update DNS. Same as the plan group name.
   2. Select a region which contains the instance on which this step will run. In this example, the script will be executed on The web application VM in Region 1.
   3. Select Run local script.
   4. Select the Target instance which is the web application VM in Region 1.
   5. In Script parameters, enter the full path of the script with the parameters. For example: /home/opc/full-stack-dr-scripts-main/oci-postgresql/backup-restore/psql_update_dns.py -c amo-psql-dbs.json -d xxx-psql-dbs.xxx.vcn01.internal -o failover.
   6. In Run as user, enter opc.
   7. Click Add Step.

   

   Description of the illustration psql-webapp-auh-drpg-create-failover-add-dns-fo-details.png

   Click Add.

   

   Description of the illustration psql-webapp-auh-drpg-create-failover-add-dns-details.png

3. Verify that the custom plan groups created show Enabled status.

   Description of the illustration psql-webapp-auh-drpg-create-failover-details-2.png

The customization of the failover plan has been successfully completed.

Task 7: Run Prechecks in Region 2

Both switchover and failover DR plans have been successfully created in the standby Region 2. These plans enable OCI Full Stack DR to transition workloads from Region 1 to Region 2. The subsequent task involves running the prechecks for both the switchover and failover plans to ensure readiness and validate the transition process.

Task 7.1: Begin Prechecks for the Switchover Plan

Run prechecks for the switchover DR plan.

1. Ensure the region context is set to standby Region 2.

2. Ensure the correct DR protection group in Region 2 is selected, it should be the standby role.

3. Click the switchover plan name.

4. Click Actions.

5. Click Run prechecks.

   

   Description of the illustration psql-webapp-auh-drpg-create-switchover-precheck.png

6. Click Run prechecks.

   

   Description of the illustration psql-webapp-auh-drpg-create-switchover-precheck-run.png

7. Verify if all prechecks are completed successfully.

   Description of the illustration psql-webapp-auh-drpg-create-switchover-precheck-details-2.png

Task 7.2: Begin Prechecks for the Failover Plan

Run the prechecks for the failover DR plan.

1. Ensure the region context is set to standby region 2.

2. Ensure the correct DR protection group in Region 2 is selected, it should be the standby role.

3. Click the failover plan name.

4. Click Actions.

5. Click Run prechecks.

   

   Description of the illustration psql-webapp-auh-drpg-create-failover-precheck.png

6. Click Run prechecks.

   

   Description of the illustration psql-webapp-auh-drpg-create-failover-precheck-run.png

7. Verify if all prechecks are completed successfully.

   

   Description of the illustration psql-webapp-auh-drpg-create-failover-precheck-details-2.png

Task 8: Run the Switchover Plan in Region 2

Run the switchover DR plan to begin the process of transitioning the web application with the OCI Database with PostgreSQL from Region 1 to Region 2.

1. Ensure the region context is set to standby Region 2.

2. Ensure the correct DR protection group in Region 2 is selected, it should be the standby role.

3. Click the switchover plan name.

4. Click Actions

5. Click Execute plan.

   

   Description of the illustration psql-webapp-auh-drpg-switchover-execute.png

This task runs the switchover plan in Region 2.

1. Deselect Enable prechecks, since they were already executed in Task 7.
2. Click Execute plan.

Description of the illustration psql-webapp-auh-drpg-create-switchover-execute-run.png

Monitor the switchover plan until the complete workload has been fully transitioned from Region 1 to Region 2.

The execution of the switchover plan was successfully completed in approximately 39 minutes.

Description of the illustration psql-webapp-auh-drpg-create-switchover-execute-details-1.png

Description of the illustration psql-webapp-auh-drpg-create-switchover-execute-details-2.png

After the Switchover: The web application will once again be accessible in Region 2, using the same URL as before.

Description of the illustration webapp-frontend-preview.png

Task 9: Create and Customize DR Plans in Region 1

With the successful completion of the switchover by OCI Full Stack DR, Region 2 has now assumed the role of the primary region, while Region 1 has transitioned to serve as the standby region.

Follow the same approach detailed in Task 1 to 8, proceed to create and customize the switchover and failover plans within the DR protection group for Region 1, which now serves as the standby peer region.

Description of the illustration psql-webapp-dxb-drpg-plan-create.png

Note: PostgreSQL - Terminate is not visible in the following screenshot.

Description of the illustration psql-webapp-dxb-drpg-create-switchover-details-1.png

Description of the illustration psql-webapp-dxb-drpg-create-failover-details-1.png

Task 10: Run the Failover Plan in Region 1

The execution of the failover plan is intended for testing purposes, where we simulate a disaster scenario to validate the effectiveness of our DR strategy. In this simulation, the disaster is mimicked by intentionally stopping the virtual machine (VM) in Region 2.

To recover from this simulated failure, we run the failover DR plan, which initiates the failover process, where the web application VM is brought up the most recent available backup of the OCI Database with PostgreSQL is restored in Region 1.

Note: The failover DR plan should only be used during a catastrophic unplanned event. Data loss may occur during the failover process.

1. Ensure the region context is set to standby Region 1.
2. Ensure the correct DR protection group in Region 1 is selected, it should be the standby role.
3. Click the failover plan name.
4. Click Actions.
5. Click Execute plan.

Description of the illustration psql-webapp-dxb-drpg-failover-execute.png

This task runs the Failover plan in Region 1.

1. Keep Enable Prechecks unchecked, as this is an actual failover exercise.
2. Click Execute plan to begin.

Description of the illustration psql-webapp-dxb-drpg-failover-execute-run.png

Monitor the failover plan until the complete workload has been fully transitioned from Region 2 to Region 1.

The execution of the failover plan was successfully completed in approximately 10 minutes.

Description of the illustration psql-webapp-dxb-drpg-create-failover-execute-details-1.png

Description of the illustration psql-webapp-dxb-drpg-create-failover-execute-details-2.png

Related Links

• Oracle Cloud Infrastructure (OCI) Full Stack Disaster Recovery

• Database with PostgreSQL

• OCI Full Stack Disaster Recovery

• OCI Database with PostgreSQL

• Traffic management steering policy

• Join #full-stack-dr slack channel

Acknowledgments

• Authors - Antoun Moubarak (Architect, Office of CTO), Piotr Kurzynoga (Open Source Data Black Belt)

• Contributor - Suraj Ramesh (Product Manager for OCI Full Stack DR)

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.

---

Title and Copyright Information

Automate Cold Disaster Recovery for OCI Database with PostgreSQL using OCI Full Stack Disaster Recovery

G37250-02

Copyright ©2025, Oracle and/or its affiliates.