Implement cross-region disaster recovery for Exadata Database Service on Google Cloud

When designing applications, it is essential to establish a robust disaster recovery mechanism for restoring operations in the event of an outage to ensure business continuity. Traditional, replicated disaster recovery setups in data centers may cost more. Thus many organizations are turning to the cloud to minimize upfront infrastructure costs and deploy scalable solutions that adapt to their needs.

For many years, customers have trusted Oracle Exadata Database Service using Oracle Maximum Availability Architecture (MAA) to ensure resiliency for mission-critical applications both on premises and on Oracle Exadata Database Service. Oracle Exadata Database Service on Oracle Database@Google Cloud offers feature and price parity with Exadata on Oracle Cloud Infrastructure (OCI) and can be deployed across multiple Google Cloud regions to ensure high availability and disaster recovery.

Architecture

A highly available Google Cloud application is deployed in two Google Cloud regions: a Primary region and a Standby region.

This architecture shows a highly available, Google Cloud application with Oracle Exadata Database Service on Oracle Database@Google Cloud in a cross-region disaster recovery topology.

Description of exadb-dr-db-google-cloud.png follows

Description of the illustration exadb-dr-db-google-cloud.png

exadb-dr-db-google-cloud-oracle.zip

For data protection, the Oracle Database is running in an Exadata virtual machine (VM) cluster in the primary region, with Active Data Guard replicating the data to the standby database running on an Exadata VM cluster in the standby region.

You can route Active Data Guard traffic through the Google Cloud network. However, this architecture focuses on Active Data Guard network traffic through the OCI network to optimize network throughput and latency.

The Oracle Exadata Database Service on Oracle Database@Google Cloud network is connected to the Exadata client subnet by using a Dynamic Routing Gateway (DRG) managed by Oracle. A DRG is also required to create a peer connection between Virtual Cloud Networks (VCNs) in different regions. Because only one DRG is allowed per VCN in OCI, a second VCN with its own DRG is required to connect the primary and standby VCNs in each region.

The database Transparent Data Encryption (TDE) keys are stored in Oracle Cloud Infrastructure Vault and replicated between the OCI regions.

The automatic backups are configured on both the primary and standby regions to Oracle Database Autonomous Recovery Service, which can reside either in Google Cloud or in OCI.

Oracle Database Autonomous Recovery Service is the preferred solution for automatic backups. Alternatively, OCI Object Storage can be used.

The architecture has the following components:

Google Cloud region
A Google Cloud region is a geographical area that contains data centers and infrastructure for hosting resources. Regions are made up of zones, which are isolated from each other within the region.
Region
An Oracle Cloud Infrastructure region is a localized geographic area that contains one or more data centers, hosting availability domains. Regions are independent of other regions, and vast distances can separate them (across countries or even continents).
Google Virtual Private Cloud
Google Virtual Private Cloud (VPC) provides networking functionality to Compute Engine virtual machine (VM) instances, Google Kubernetes Engine (GKE) containers, database services, and serverless workloads. VPC provides global, scalable, and flexible networking for your cloud-based service.
Virtual cloud network (VCN) and subnet
A VCN is a customizable, software-defined network that you set up in an Oracle Cloud Infrastructure region. Like traditional data center networks, VCNs give you control over your network environment. A VCN can have multiple non-overlapping CIDR blocks that you can change after you create the VCN. You can segment a VCN into subnets, which can be scoped to a region or to an availability domain. Each subnet consists of a contiguous range of addresses that don't overlap with the other subnets in the VCN. You can change the size of a subnet after creation. A subnet can be public or private.
Oracle Exadata Database Service
enables you to leverage the power of Exadata in the cloud. Oracle Exadata Database Service delivers proven Oracle Database capabilities on purpose-built, optimized Oracle Exadata infrastructure in the public cloud. Built-in cloud automation, elastic resource scaling, security, and fast performance for all Oracle Database workloads helps you simplify management and reduce costs.
Virtual cloud network (VCN) and subnets
A VCN is a customizable, software-defined network that you set up in an Oracle Cloud Infrastructure region. Like traditional data center networks, VCNs give you control over your network environment. A VCN can have multiple non-overlapping CIDR blocks that you can change after you create the VCN. You can segment a VCN into subnets, which can be scoped to a region or to an availability domain. Each subnet consists of a contiguous range of addresses that don't overlap with the other subnets in the VCN. You can change the size of a subnet after creation. A subnet can be public or private.
Object storage
OCI Object Storage provides access to large amounts of structured and unstructured data of any content type, including database backups, analytic data, and rich content such as images and videos. You can safely and securely store data directly from the internet or from within the cloud platform. You can scale storage without experiencing any degradation in performance or service reliability.

Use standard storage for "hot" storage that you need to access quickly, immediately, and frequently. Use archive storage for "cold" storage that you retain for long periods of time and seldom or rarely access.
Oracle Cloud Infrastructure Vault
Oracle Cloud Infrastructure Vault enables you to create and centrally manage the encryption keys that protect your data and the secret credentials that you use to secure access to your resources in the cloud. The default key management is Oracle-managed keys. You can also use customer-managed keys which use OCI Vault. OCI Vault offers a rich set of REST APIs to manage vaults and keys.
Oracle Database Autonomous Recovery Service
Oracle Database Autonomous Recovery Service is an Oracle Cloud service that protects Oracle databases. Backup automation and enhanced data protection capabilities for OCI databases allow you to offload all backup processing and storage requirements to Oracle Database Autonomous Recovery Service, which reduces backup infrastructure costs and manual administration overhead.
Managed service
A managed service provides specific functionality without requiring you to perform maintenance tasks related to optimizing performance, availability, scaling, security, or upgrading. A managed service enables you to focus on delivering features for your customers instead of worrying about the complexity of operations. A managed service provides a scalable and secure component for cloud-native development. Use managed services to develop and run your app and to store its data. You get best-in-class solutions without needing expertise in each domain to build and operate your app.
Local peering
Local peering enables you to peer one VCN with another VCN in the same region. Peering means the VCNs communicate using private IP addresses, without the traffic traversing the internet or routing through your on-premises network.
Remote peering
Remote peering allows resources within different VCNs to communicate using private IP addresses. Remote peering eliminates the need for an internet gateway or public IP addresses for instances that need to communicate with another VCN in a different region.
Data Guard
Oracle Data Guard and Oracle Active Data Guard provide a comprehensive set of services that create, maintain, manage, and monitor one or more standby databases and that enable production Oracle databases to remain available without interruption. Oracle Data Guard maintains these standby databases as copies of the production database by using in-memory replication. If the production database becomes unavailable due to a planned or an unplanned outage, Oracle Data Guard can switch any standby database to the production role, minimizing the downtime associated with the outage. Oracle Active Data Guard provides the additional ability to offload read-mostly workloads to standby databases and also provides advanced data protection features.
Oracle Database@Google Cloud
Oracle Database@Google Cloud is an Oracle Cloud database service that runs Oracle Database workloads in your Google Cloud environment. All hardware for Oracle Database@Google Cloud is colocated in Google Cloud's data centers and uses Google Cloud networking. The service benefits from the simplicity, security, and low latency of a single operating environment within Google Cloud. You can manage the service on the Google Cloud console or by using Google Cloud automation tools. Google Cloud IAM and Admin provide federated identity and access management for Oracle Exadata Database Service.

Recommendations

Use the following recommendations as a starting point when performing cross-region disaster recovery for Oracle Exadata Database Service on Oracle Database@Google Cloud. Your requirements may differ from the architecture described here.

Perform regular application and database switchover operations every three to six months, or conduct full application and database failover tests.
Deploy the required Exadata infrastructure in both primary and standby regions. For each Exadata instance, deploy an Exadata VM cluster in the same Virtual Private Cloud (VPC) in a different subnet. The Oracle Real Application Clusters (Oracle RAC) database can then be instantiated on the cluster. In the same VPC, deploy your application in a separate subnet. Configure Active Data Guard to replicate data from one Oracle Database to the other, across regions.
Use Oracle Cloud automation tools such as OCI Console, OCI CLI, SDKs, REST APIs, to automatically provision the Active Data Guard standby database following Oracle's Maximum Availability Architecture (MAA) best practices.

Considerations

When performing cross-region disaster recovery for Oracle Exadata Database Service on Oracle Database@Google Cloud, consider the following.

Prepare using a comprehensive disaster scenario that considers different business requirements and availability architectures. Your plan must be actionable and include high-availability (HA), disaster recovery (DR) requirements. This architecture provides guidelines to help you select the approach that best fits your application deployment by using simple and effective failover for the disaster recovery configuration in your OCI and Google Cloud environments.
(Optional) Enable automatic failover (Fast-Start Failover) to reduce the recovery time in case of failures by installing the Oracle Data Guard Observer on a separate VM, preferably in a separate location or in the application network.
You can configure automatic backups to Oracle Database Autonomous Recovery Service located on OCI or Google Cloud.
You can configure either Oracle Data Guard or Active Data Guard during the creation of the Standby database using OCI automation.

Deploy

To configure cross-region VCN peering and establish network communication between regions shown in the architecture diagram, complete the following high-level steps.

Primary Region

Create a virtual cloud network (VCN), HUB VCN Primary, in the Oracle Cloud Infrastructure (OCI) primary region.
Deploy two local peering gateways (LPGs), Primary-LPG and Hub-Primary-LPG, in VCN Primary and HUB VCN Primary respectively.
Establish a peer LPG connection between the LPGs for HUB VCN Primary and VCN Primary.
Create a dynamic routing gateway (DRG), Primary-DRG in the Hub VCN Primary VCN.
In the HUB VCN Primary VCN, create the route table, primary_hub_transit_drg, and assign the destination of the VCN Primary client subnet, a target type of LPG, and the target Hub-Primary-LPG. For example:
```
10.5.0.0/24 target type: LPG, Target: Hub-Primary-LPG
```
In the HUB VCN Primary VCN, create a second route table, primary_hub_transit_lpg, and assign the destination of the VCN Standby client subnet, a target type DRG, and a target Primary-DRG. For example:
```
10.6.0.0/24 target type: DRG, Target: Primary-DRG
```
From the Hub VCN Primary VCN, attach Hub VCN Primary to the DRG. Edit the DRG VCN attachments, and under advanced options, edit the tab VCN route table to associate it with the the primary_hub_transit_drg route table. This configuration permits transit routing.
From the Hub VCN Primary VCN, associate the primary_hub_transit_lpg route table with the Hub-Primary-LPG gateway.
In the Hub VCN Primary default route table, add a route rule for the VCN Primary client subnet IP Address range to use the LPG. Add another route rule for the VCN Standby client subnet IP Address range to use the DRG. For example:
```
10.5.0.0/24 LPG Hub-Primary-LPG
10.6.0.0/24 DRG Primary-DRG
```
From Primary-DRG, select the DRG route table, Autogenerated DRG Route Table for RPC, VC, and IPSec attachments. Add a static route to the VCN Primary subnet client IP Address range that uses the Hub VCN Primary VCN with a next hop attachment type of VCN and the next hop attachment name Primary Hub attachment. For example:
```
10.5.0.0/24 VCN Primary Hub attachment
```
Use the Primary-DRG remote peering connection attachments menu to create a remote peering connection, RPC.
In the VCN Primary client subnet, update the network security group (NSG) to create a security rule to allow ingress for TCP port 1521. Optionally, you can add SSH port 22 for direct SSH access to the database servers.

Note:
For a more precise configuration, disable the import route distribution of the Autogenerated DRG Route Table for RPC, VC, and IPSec attachments route table. For Autogenerated DRG Route Table for VCN attachments, create and assign a new import route distribution including only the required RPC attachment.

Standby Region

Create the VCN, HUB VCN Standby, in the OCI standby region.
Deploy two LPGs, Standby-LPG and Hub-Standby-LPG, in the VCN Standby and the HUB VCN Standby VCNs respectively.
Establish a peer LPG connection between LPGs for VCN Standby and HUB VCN Standby.
Create a DRG, Standby-DRG in the Hub VCN Standby VCN.
In the HUB VCN Standby VCN, create a route table, standby_hub_transit_drg, and assign the destination of the VCN Standby client subnet, a target type of LPG, and a target Hub-Standby-LPG. For example:
```
10.6.0.0/24 target type: LPG, Target: Hub-Standby-LPG
```
In the HUB VCN Standby VCN, create a second route table, standby_hub_transit_lpg and assign the destination of the VCN Primary client subnet, a target type DRG, and a target Standby-DRG. For example:
```
10.5.0.0/24 target type: DRG, Target: Standby-DRG
```
From the HUB VCN Standby VCN, attach the Hub VCN Standby VCN to the DRG. Edit The DRG VCN attachments and under advanced options, edit the VCN route table to associate it with the standby_hub_transit_drg route table. This configuration permits transit routing.
From the HUB VCN Standby VCN, in the Hub VCN Standby default route table, add route rules for the VCN Standby client subnet IP Address range to use the LPG and for the VCN Primary client subnet IP Address range to use the DRG. For example:
```
10.6.0.0/24 LPG Hub-Standby-LPG
10.5.0.0/24 DRG Standby-DRG
```
Associate the route table, standby_hub_transit_lpg with the Hub-Standby-LPG gateway.
From Standby-DRG, select the DRG route table Autogenerated Drg Route Table for RPC, VC, and IPSec attachments. Add a static route to the VCN Standby subnet client IP Address range that use the Hub VCN Standby VCN with a next hop attachment type of VCN and the next hop attachment name Standby Hub attachment. For example:
```
10.6.0.0/24 VCN Standby Hub attachment
```
Use the Standby-DRG remote peering connection attachments menu to create a remote peering connection, RPC.
Select the remote peering connection, select Establish Connection, and provide the Primary-DRG OCID. The peering status becomes peered. Both regions are connected.
In the VCN Standby client subnet, update the NSG to create a security rule to allow ingress for TCP port 1521. Optionally, you can add SSH port 22 for direct SSH access to the database servers.

Data Guard

Follow these steps to enable Oracle Data Guard or Active Data Guard in Oracle Database:

On the Oracle Database details page, click Data Guard group, then click Add Standby.
On the Add Standby page:
1. Select the standby peer region.
2. Select the standby availability domain mapped to the Google Cloud zone.
3. Select the standby Exadata infrastructure.
4. Select the desired standby VM cluster.
5. Choose either Oracle Data Guard or Active Data Guard.
  
  Note:
  Oracle MAA recommends Active Data Guard for auto block repair of data corruptions and the ability to offload reporting.
6. For cross-region Oracle Data Guard associations, only the maximum performance protection mode is supported.
7. Select an existing database home or create one.
  
  Note:
  Oracle recommends that you use the same database software image of the primary database for the standby database home, so that both have the same patches available.
8. Enter the password for the SYS user.
9. Click Add Standby.
After Oracle Data Guard is enabled, the standby database will be listed in the Oracle Data Guard group section.
(Optional) Enable automatic failover (Fast-Start Failover) to reduce the recovery time in case of failures by installing Oracle Data Guard Observer on a separate VM, preferably in a separate location or in the application network.

Explore More

Learn more about Oracle Database@Google Cloud.

Review the following reference architectures:

Review these additional resources:

Acknowledgments

Authors: Sinan Petrus Toma, Glen Hawkins, Julien Silverston
Contributors: Andy Steinorth, Lawrence To, Sreya Dutta