Oracle Maximum Availability Architecture for Oracle Database@Azure

Microsoft Azure is a strategic Oracle Multicloud partner. Oracle Maximum Availability Architecture (MAA) evaluates MAA reference architectures on Oracle Database@Azure, the results of which are shown here.

To learn more about MAA Silver, MAA Gold, and MAA Platinum evaluations and their benefits after certification, see MAA Evaluations on Multicloud Solutions.

Oracle MAA evaluated Oracle’s solution in Azure. Oracle MAA continues to re-evaluate periodically to ensure the solution meets all the expected benefits and to highlight any new MAA for Oracle Database@Azure benefits and capabilities. Certification is only given after the MAA evaluation meets the requirements.

Oracle MAA has evaluated and endorsed Oracle Database@Azure for:

Evaluation environments:

Network Results

The results shown in the following use cases are based on network tests described in Oracle Database Multicloud Evaluations by Oracle MAA.

Use Case: Application VMs to Exadata VM Cluster (Same Region)

RTT Latency Observed: Varies based on placement

Network Throughput Observed: Variable based on placement and VM size

MAA Recommendations:

Use Case: Between Exadata VM Clusters Across AZs (London, England)

Evaluated for:

RTT Latency Observed: 1.2 milliseconds (for both peering options)

Network Single Process Throughput Observed:

Maximum Parallel Throughput:

MAA Recommendations:

Use Case: Between Exadata VM Clusters Across Regions (Ashburn, Virginia, USA and London, England)

Evaluated for:

RTT Latency Observed: ~76 milliseconds (for both peering options)

Network Single Process Throughput Observed:

Maximum Parallel Throughput:

MAA Recommendations:

Azure Network Recommendations and Considerations

MAA Silver Network Topology and Evaluation

The following MAA Silver HA Diagram illustrates Oracle RAC on ExaDB-D with backup to Autonomous Recovery Service running in Azure or OCI.

Figure 1: MAA Silver Architecture for Oracle Database@Azure

Description of image follows

Oracle MAA evaluated and endorsed Oracle Database@Azure for the MAA Silver reference architecture, and observed the following results.

Application Network Layer on Azure

The proximity of the application tier to the database cluster affects application response time.

If you require a very low latency response time (200-400 microseconds), deploy the application VMs in the same Availability Zone (AZ) as the database cluster. Latency increases to 1 millisecond or more when application and database servers are configured across VNets or AZs.

Deploy the application tier over at least two AZs to ensure high availability. The deployment process and solution over multiple AZs varies depending on the application’s components, Azure services, and resources involved. For example, with Azure Kubernetes Services (AKS), you can deploy the worker nodes in different AZs. Kubernetes control plane maintains and synchronizes the pods and the workload.

Backup and Restore Observations

RMAN nettest was used for backups, and the expected results were met. See KB115145 for details about nettest.

Oracle Database backup and restore throughput to Oracle’s Autonomous Recovery Service or Oracle’s Object Storage Service were within performance expectations. For example, an ExaDB-D 2 node cluster (using 16+ OCPUs) and three storage cells may observe a 4 TB/hour backup rate and approximately 8.7 TB/hour restore rate with no other workloads.

By increasing the RMAN channels, you can leverage available network bandwidth or storage bandwidth and achieve as much as 42 TB/hour backup rate and 8.7 TB/hour restore rate for 3 Exadata storage cells. The restore rates can increase as you add more Exadata storage cells. The performance varies based on existing workloads and network traffic on the shared infrastructure.

The Autonomous Recovery Service provides the following additional benefits:

MAA Gold and Platinum Network Topology and Evaluation

The recommended MAA Gold and MAA Platinum architectures in Azure consist of:

MAA Gold and MAA Platinum Option 1 (Oracle AI Database 26ai or higher and Oracle Active Data Guard) evaluation builds upon the Silver MAA evaluation, plus:

The following diagram shows MAA Gold and MAA Platinum DR architecture in the same Azure Region.

Description of image follows

Description of the illustration azure-gold-sameregion.png

The following diagram shows MAA Gold and MAA Platinum DR architecture across two Azure Regions.

Description of image follows

Description of the illustration azure-gold-crossregion.png

Network Layer

Oracle Data Guard maintains an exact physical copy of the primary database by transmitting (and applying) all data changes (redo) to the standby database across the network, making network throughput, and in some cases latency, critical to the implementation’s success. Use Data Guard switchover for planned maintenance or disaster recovery tests. If the primary database becomes unavailable, use Data Guard failover to resume service.

Peering Networks Between Primary and Standby

This is the most critical decision to ensure your standby can keep up. If the network does not have sufficient bandwidth to support single-process redo throughput, the standby will have a growing transport lag.

The primary and standby Exadata Clusters are deployed in separate networks. Oracle Database@Azure Exadata Clusters are always deployed using separate Virtual Cloud Networks (VCN) in OCI. These separate VCNs must be connected to allow traffic to pass between them, that is they must be “peered” before enabling Data Guard with Oracle cloud automation. For this reason, the networks must use separate, non-overlapping IP CIDR ranges.

Peering across Availability Zones (AZs) can be done using the OCI or Azure networks. Both options provide reasonable throughput; however, Azure peering comes with a chargeback for cross-VNet traffic. OCI provides up to 10TB of data egress per month at no charge, and lower rates when that limit is exceeded. Note that Azure VNets are virtual networks similar to an OCI VCN (Virtual Cloud Network).

For cross-region network peering, OCI is the recommended option for any Enterprise application and databases due to the higher throughput observed during testing. OCI VCN peering provides higher single-process network throughput when compared to Azure VNet peering between the same regions. The latency between database clusters will vary based on physical location and network topology. There is no chargeback for cross-region traffic for the first 10TB per month with OCI VCN peering. Note that single process throughput, maximum network throughput, and latency may vary based on data center locations.

Oracle Database@Azure service network is connected to the Exadata client subnet by a Dynamic Routing Gateway (DRG) managed by Oracle. A DRG is also required to peer VCNs between regions, and only one DRG is allowed for each VCN in OCI. Therefore, to connect the primary and standby VCNs, the communication requires transit over a second VCN with its own DRG in each region.

Recommended OCI VCN Peering for Data Guard

Alternative Option of Azure VNet Peering for Data Guard

Choose this option only if the database workload and redo generation rate are much lower than the single process network bandwidth results. To peer the Azure VNets for Data Guard redo traffic, see Virtual network peering.

When networks are peered through Azure, single-process network throughput can be significantly lower than Oracle OCI peered networks.. This is relevant because Data Guard redo transport is a single process for each database instance; therefore, if a single instance produces redo at a higher rate, transport lag may increase and accumulate. There is an additional cost for ingress and egress network traffic in each VNet when networks are peered through Azure.

Enabling Data Guard

After the network is peered by one of the above options, you can Enable Data Guard (see Use Oracle Data Guard with Exadata Cloud Infrastructure).

Data Guard Role Transitions

The timings of the Data Guard switchover and failover were within expectations compared to a similar setup in Oracle OCI. Application downtime when performing a Data Guard switchover and failover can range from 30 seconds to a few minutes. For guidance on tuning Data Guard role transition timings or examples of role transition timings, see Tune and Troubleshoot Oracle Data Guard.

Setting Up Networking Across Availability Zones

Oracle MAA recommends best practices for networking across availability zones in a disaster recovery setup.

Ensure the following

Description of image follows

Description of the illustration azure-gold-sameregion.png

Configuring the Application Tier on Azure

  1. Deploy the application tier over at least two AZs.

    The process and solution for deploying over multiple AZs varies depending on the applicationâ€TMs components, Azure services, and resources involved. For example, with Azure Kubernetes Services (AKS), you can deploy the worker nodes in different AZs. The Kubernetes control plane maintains and synchronizes the pods and the workload.

  2. Follow the steps described in Configuring Continuous Availability for Applications.

Configuring the Database Tier on OCI

When Exadata Clusters are created in Azure, each cluster is in a different Virtual Cloud Network (VCN) in OCI. Connectivity between VCNs is required for Data Guard redo transport. This connectivity, or peering, must be configured before enabling Data Guard in Azure. For resources in different VCNs to communicate with each other, as is required by Data Guard, additional steps are required to peer the VCNs and allow the IP address ranges access to each other.

Oracle Data Guard maintains a standby database by transmitting and applying redo data from the primary database. Use Data Guard switchover for planned maintenance or disaster recovery tests. If the primary database becomes unavailable, use Data Guard failover to resume service.

OCI is the preferred network for performance (latency, throughput), and no egress or ingress costs are incurred. When Exadata Clusters are created in Azure, each cluster will be configured in a different OCI Virtual Cloud Network (VCN). As Data Guard requires, you must perform additional steps to peer the VCNs and allow the IP address ranges access to each other for resources in different VCNs to communicate with each other.

The following steps describe the process to enable Data Guard across AZs using the OCI-managed network.

  1. Log in to the OCI Console and create a Local Peering Gateway (LPG) in the VCNs of the primary and standby Exadata VM Clusters.

    For details see Creating a Local Peering Gateway.

  2. Establish a Peer Connection between primary and standby LPG, and select the Unpeered Peer Gateway in the standby VCN.

    For details see Connecting to Another LPG.

    Each VCN can have only one Local Peering Gateway (LPG), if there are multiple databases on a given Exadata Cluster which will have standby databases on different Exadata Clusters, a Hub VCN will need to be configured. See Transit Routing inside a hub VCN

  3. Update the default route table to route the traffic between the primary and standby databases over the OCI network without incurring any inbound and outbound data transfer costs.

    Note:

    To update the default route table, you currently need to create a support ticket SR providing the tenancy name and dynamic routing gateway (DRG) OCID.

    If you encounter the error “Authorization failed or requested resource not found”, open a service ticket with the following information:

    • Title of ticket: “Required VCN Route Table Update permission”

    • Include information for each VNC and its DRG attachment: Region, Tenancy OCID, VCN OCID, DRG OCID

  4. Update the primary and standby Network Security Group to create a security rule to allow primary and standby client subnet ingress for TCP port 1521.

    Optionally you can add SSH port 22 for direct SSH access to the database servers.

  5. Enable Data Guard or Oracle Active Data Guard for the primary database.

    From the Oracle Database details page, click the Data Guard Associations link, then click Enable Data Guard.

    On the Enable Data Guard page:

    1. Select the standby Availability Domain mapped to Azure AZ.

    2. Select the standby Exadata Infrastructure.

    3. Select the desired standby VM Cluster.

    4. Choose Data Guard or Active Data Guard (MAA recommends Active Data Guard for auto block repair of data corruptions and ability to offload reporting).

    5. Choose a protection mode and redo transport type that satisfies your RTO and RPO.

    6. Select an existing database home or create a new one.

      It is recommended that you use the same custom database software image of the primary database for the standby database home, so that both have the same patches available.

    7. Enter the password for the SYS user and Enable Data Guard.

    Optionally, to reduce the recovery time in case of failures, enable automatic failover (Fast-Start Failover) by installing Data Guard Observer or a separate VM, preferably in a separate location or in the application network. For more information, see Fast-Start Failover in the Oracle Data Guard Broker guide andConfigure Fast Start Failover to Bound RTO and RPO (MAA Gold Requirement). Currently these steps are not part of cloud automation and are manual.

    After Data Guard is enabled, the standby database is listed in the Data Guard Associations section.

Setting Up Networking Across Regions

Oracle MAA recommends these best practices for networking across Azure regions in an Oracle Database@Azure DR configuration.

The architecture diagram below shows a containerized Azure Kubernetes Service (AKS) application in two Azure regions. The container images are stored in the Azure container registry and replicated between primary and standby regions. Users access the application externally through a public load balancer. The database is running in an Exadata VM Cluster in primary/standby mode with Oracle Data Guard for data protection. The database TDE encryption keys are stored in OCI Private Vault and are replicated between the regions. The automatic backups are in OCI object storage in each region.

The following diagram shows MAA Gold and MAA Platinum DR architecture across two Azure Regions.

Description of image follows

Description of the illustration azure-gold-crossregion.png

Network Principles for Oracle Database@Azure Across Regions

Oracle MAA recommends:

For additional details and setup instructions see Perform Cross-Regional Disaster Recovery for Exadata Database on Oracle Database@Azure.

Note: Once configured, it is possible to enable automatic failover (Fast-Start Failover) to reduce recovery time in case of failure by installing Data Guard Observer on a separate VM, preferably in a separate location or in the application network. For more information, see the documentation for Fast-Start Failover, and Configure and Deploy Oracle Data Guard. (These are currently manual steps and not part of cloud automation.)