Integrate Oracle Database@Azure with your Azure streaming platform using OCI GoldenGate

Before You Begin

In order to take advantage of this reference architecture, the following are required:

Access to an Azure subscription and directory
Access to an Oracle Cloud Infrastructure (OCI) tenancy
Active Oracle Database@Azure multicloud link between the Azure and OCI clouds
Non overlapping CIDR blocks between any Azure VNets and OCI VCNs
Adequate Oracle Exadata Database Service Limits, Oracle Autonomous Database Serverless and OCI GoldenGate Service limits prior to provisioning

Note:

With a multicloud solution, networking is a key determinant of overall system performance. The customer is responsible for ensuring that the cloud-to-cloud network (bandwidth and latency) is thoroughly tested to ensure the application performance meets the defined business requirements.

Architecture

This architecture shows how to use Oracle Database@Azure and OCI GoldenGate with your real-time data streaming platform on Azure.

The platforms are connected by an Oracle Cloud Infrastructure (OCI) Managed Network and a VCN that spans both regions and includes a local peering gateway. Oracle Database@Azure resides within the VCN in the Azure region and uses local peering to send data through the OCI Managed Network to the services located in the HUB VCN in OCI. OCI GoldenGate is accessible using a private endpoint (PE) from within the OCI network that secures access to OCI resources.

In addition, Oracle Interconnect for Azure and a site-to-site VPN provide a path for data going from OCI to Azure. Data flows from the dynamic routing gateway on the OCI HUB VCN to the Site-to-Site VPN and Oracle Interconnect for Azure. The data from both sources flows through a virtual network gateway on the Azure VNet that includes the private link and endpoint. From there, it flows to the Azure resources.

The following diagram illustrates this reference architecture.

Description of oracle-db-azure-streaming.png follows

Description of the illustration oracle-db-azure-streaming.png

oracle-db-azure-streaming-oracle.zip

The architecture has the following components:

The architecture has the following Oracle Cloud Infrastructure (OCI) components:

Region
An Oracle Cloud Infrastructure region is a localized geographic area that contains one or more data centers, called availability domains. Regions are independent of other regions, and vast distances can separate them (across countries or even continents).
Availability domains
Availability domains are standalone, independent data centers within a region. The physical resources in each availability domain are isolated from the resources in the other availability domains, which provides fault tolerance. Availability domains don’t share infrastructure such as power or cooling, or the internal availability domain network. So, a failure at one availability domain shouldn't affect the other availability domains in the region.
Virtual cloud network (VCN) and subnets
A VCN is a customizable, software-defined network that you set up in an Oracle Cloud Infrastructure region. Like traditional data center networks, VCNs give you control over your network environment. A VCN can have multiple non-overlapping CIDR blocks that you can change after you create the VCN. You can segment a VCN into subnets, which can be scoped to a region or to an availability domain. Each subnet consists of a contiguous range of addresses that don't overlap with the other subnets in the VCN. You can change the size of a subnet after creation. A subnet can be public or private.
Dynamic routing gateway (DRG)
The DRG is a virtual router that provides a path for private network traffic between VCNs in the same region, between a VCN and a network outside the region, such as a VCN in another Oracle Cloud Infrastructure region, an on-premises network, or a network in another cloud provider.

A DRG is required to set up a private interconnection using OCI FastConnect between a VCN in an OCI region and a VNet in an Azure region.
Local peering gateway (LPG)
An LPG enables you to peer one VCN with another VCN in the same region. Peering means the VCNs communicate using private IP addresses, without the traffic traversing the internet or routing through your on-premises network.
Site-to-site VPN
Provides a site-to-site IPSec VPN between your on-premises network and your VCN over a secure, encrypted connection.
Network security group (NSG)
Network security group (NSG) acts as a virtual firewall for your cloud resources. With the zero-trust security model of Oracle Cloud Infrastructure, all traffic is denied, and you can control the network traffic inside a VCN. An NSG consists of a set of ingress and egress security rules that apply to only a specified set of VNICs in a single VCN.
Route table
Virtual route tables contain rules to route traffic from subnets to destinations outside a VCN, typically through gateways.
Service gateway
The service gateway provides access from a VCN to other services, such as Oracle Cloud Infrastructure Object Storage. The traffic from the VCN to the Oracle service travels over the Oracle network fabric and does not traverse the internet.
Oracle Exadata Database Service
Oracle Exadata is an enterprise database platform that runs Oracle Database workloads of any scale and criticality with high performance, availability, and security. Exadata’s scale-out design employs unique optimizations that let transaction processing, analytics, machine learning, and mixed workloads run faster and more efficiently. Consolidating diverse Oracle Database workloads on Exadata platforms in enterprise data centers, on Oracle Cloud Infrastructure (OCI), and in multicloud environments helps organizations increase operational efficiency, reduce IT administration, and lower costs.

Oracle Exadata Database Service enables you to leverage the power of Exadata in the cloud. Oracle Exadata Database Service delivers proven Oracle Database capabilities on purpose-built, optimized Oracle Exadata infrastructure in the public cloud and on Cloud@Customer. Built-in cloud automation, elastic resource scaling, security, and fast performance for all Oracle Database workloads helps you simplify management and reduce costs.
Oracle Exadata Database Service on Dedicated Infrastructure
Oracle Exadata Database Service on Dedicated Infrastructure provides Oracle Exadata Database Machine as a service in an Oracle Cloud Infrastructure (OCI) data center. The Oracle Exadata Database Service on Dedicated Infrastructure instance is a virtual machine (VM) cluster that resides on Exadata racks in an OCI region.

Oracle Exadata Database Service delivers proven Oracle Database capabilities on purpose-built, optimized Oracle Exadata infrastructure in the public cloud. Built-in cloud automation, elastic resource scaling, security, and fast performance for OLTP, in-memory analytics, and converged Oracle Database workloads help simplify management and reduce costs.

Exadata Cloud Infrastructure X9M brings more CPU cores, increased storage, and a faster network fabric to the public cloud. Exadata X9M storage servers include Exadata RDMA Memory (XRMEM), creating an additional tier of storage, boosting overall system performance. Exadata X9M combines XRMEM with innovative RDMA algorithms that bypass the network and I/O stack, eliminating expensive CPU interrupts and context switches.

Exadata Cloud Infrastructure X9M increases the throughput of its 100 Gbps active-active Remote Direct Memory Access over Converged Ethernet (RoCE) internal network fabric, providing a faster interconnect than previous generations with extremely low-latency between all compute and storage servers.
Oracle Autonomous Database Serverless
Oracle Autonomous Database is a fully managed, preconfigured database environments that you can use for transaction processing and data warehousing workloads. You do not need to configure or manage any hardware, or install any software. Oracle Cloud Infrastructure handles creating the database, as well as backing up, patching, upgrading, and tuning the database.

Oracle Autonomous Database Serverless is an Oracle Autonomous Database. You have a fully elastic database where Oracle autonomously operates all aspects of the database lifecycle from database placement to backup and updates.

Oracle Database Autonomous Serverless is also available with Oracle Database@Azure as the world’s first autonomous data management in the cloud to deliver automated patching, upgrades, and tuning, without human intervention. Autonomous Database Serverless is built on Oracle Exadata infrastructure, is self-managing, self-securing, and self-repairing, helping eliminate manual database management and human errors.
Oracle Database@Azure
Oracle Database@Azure is the Oracle Database service (Oracle Exadata Database Service on Dedicated Infrastructure and Oracle Autonomous Database Serverless) running on Oracle Cloud Infrastructure (OCI), deployed in Microsoft Azure data centers. The service offers features and price parity with OCI, users purchase the service on Azure Marketplace.

Oracle Database@Azure integrates Oracle Exadata Database Service, Oracle Real Application Clusters (Oracle RAC), and Oracle Data Guard technologies into the Azure platform. Oracle Database@Azure service offers the same low latency as other Azure-native services and meets mission-critical workloads and cloud-native development needs. Users manage the service on the Azure console and with Azure automation tools. The service is deployed in Azure Virtual Network (VNet) and integrated with the Azure identity and access management system. The OCI and Oracle Database metrics and audit logs are natively available in Azure. The service requires users to have an Azure tenancy and an OCI tenancy. Oracle Autonomous Database Serverless is also available with Oracle Database@Azure as the world’s first autonomous data management, fully managed in the cloud, to deliver automated patching, upgrades, and tuning, without human intervention. Autonomous Database is built on Oracle Exadata infrastructure, is self-managing, self-securing, and self-repairing, helping eliminate manual database management and human errors. Autonomous Database enables development of scalable AI-powered apps with any data using built-in AI capabilities using your choice of large language model (LLM) and deployment location.

Both Oracle Exadata Database Service and Oracle Autonomous Database Serverless are easily provisioned through the native Azure Portal, enabling access to the broader Azure ecosystem.
OCI GoldenGate
Oracle Cloud Infrastructure GoldenGate is a managed service providing a real-time data mesh platform, which uses replication to keep data highly available, and enables real-time analysis. You can design, execute, and monitor your data replication and stream data processing solutions without allocating or managing compute environments.
Transparent Data Encryption (TDE)
Transparent Data Encryption (TDE) transparently encrypts data at rest in an Oracle Database. It stops unauthorized attempts from the operating system to access database data stored in files, without impacting how applications access the data using SQL. TDE is fully integrated with Oracle Database and can encrypt entire database backups (RMAN), Data Pump exports, entire application tablespaces, or specific sensitive columns. Encrypted data remains encrypted in the database, whether it is in tablespace storage files, temporary tablespaces, undo tablespaces, or other files such as redo logs.
FastConnect
Oracle Cloud Infrastructure FastConnect provides an easy way to create a dedicated, private connection between your data center and Oracle Cloud Infrastructure. FastConnect provides higher-bandwidth options and a more reliable networking experience when compared with internet-based connections.

The architecture has the following Microsoft Azure components:

Azure region
An Azure region is a geographical area in which one or more physical Azure data centers, called availability zones, reside. Regions are independent of other regions, and vast distances can separate them (across countries or even continents).

Azure and OCI regions are localized geographic areas. For Oracle Database@Azure, an Azure region is connected to an OCI region, with availability zones (AZs) in Azure connected to availability domains (ADs) in OCI. Azure and OCI region pairs are selected to minimize distance and latency.
Azure availability zone
An availability zone is a physically separate data center within a region that is designed to be available and fault tolerant. Availability zones are close enough to have low-latency connections to other availability zones.
Virtual network (VNet) and subnet
A VNet is a virtual network that you define in Azure. A VNet can have multiple non-overlapping CIDR blocks subnets that you can add after your create the VNet. You can segment a VNet into subnets, which can be scoped to a region or to an availability zones. Each subnet consists of a contiguous range of addresses that don't overlap with the other subnets in the VNet. Use VNet to isolate your Azure resources logically at the network level.
ExpressRoute
Azure ExpressRoute lets you set up a private connection between a VNet and another network, such as your on-premises network or a network in another cloud provider. ExpressRoute is a more reliable and faster alternative to typical internet connections, because the traffic over ExpressRoute doesn't traverse the public internet.
Virtual network gateway
A virtual network gateway allows traffic between an Azure VNet and a network outside Azure, either over the public internet or using ExpressRoute, depending on the gateway type that you specify.
Route table
Route tables direct traffic between Azure subnets, VNets, and networks outside Azure.
Network security group
A network security group contains rules to control network traffic between the Azure resources within a VNet. Each rule specifies the source or destination, port, protocol, and direction of network traffic that's allowed or denied.
Local peering gateway (LPG)
A local peering gateway (LPG) is a component on a VCN for routing traffic to a locally peered VCN.
Site-to-site VPN
Provides a site-to-site IPSec VPN between your on-premises network and your VCN over a secure, encrypted connection.

Recommendations

Use the following recommendations as a starting point. Your requirements might differ from the architecture described here.

Choice of location
We recommend creating the OCI GoldenGate resources in the same OCI tenancy and region as Oracle Database@Azure to achieve optimal end-to-end latency.
Provisioning
Choose the right size of the Oracle Cloud Infrastructure FastConnect to support the bandwidth needs for the workload.
Parallelism
It is recommended that parallel replication in OCI GoldenGate be set up to speed up applying lag where possible.

Considerations

Consider the following points when deploying this reference architecture.

Networking Setup for Oracle Database@Azure and OCI GoldenGate
Microsoft Azure Environment:
- Utilizes an Azure Virtual Network (VNet) for networking.
- Creates virtual network interface cards (vnics) inside a pre-created delegated subnet.
OCI Environment:
- Oracle Database@Azure connects to a client subnet within an OCI VCN.
- To connect the database VCN with the HUB VCN, communication transits through a LPG.
Client Connectivity
- This network setup allows client connectivity from both Azure and OCI resources.
- However, it does not permit direct communication between OCI resources and Azure resources.
This comprehensive setup ensures robust, secure connectivity across OCI and Azure environments, supporting versatile deployment scenarios. For more information, see the Access to Microsoft Azure link in the Explore More section.

Cost
For OCI GoldenGate, we provide a web-based flexible and elastic service to move data in real-time.
Performance
- Network latency is vital for performance. For OCI GoldenGate, check and measure the network latency as part of application performance testing.
- Network latency between applications and databases hosted in different cloud data centers must be less than 10 ms. We recommend selecting nearby application and database regions to achieve optimal end-to-end performance.
Connectivity
- An additional network connectivity setup is required to enable direct communication between OCI GoldenGate and Azure.
  - In several regions, a dedicated OCI Azure Interconnect is available.
  - This interconnect allows cross-cloud workloads without requiring the traffic between clouds to traverse the internet.
  - It enables the connection of a Microsoft Azure VNet with an OCI VCN, facilitating seamless cross-cloud workloads.
  - A Site-to-Site VPN can be established as an alternative in regions where the OCI Azure Interconnect is unavailable.
- Use a Dedicated endpoint and select a Private Subnet when creating the Connection to Oracle Database@Azure.
- Use the Redirect session mode when creating the Connection to Oracle Exadata Database Service on Dedicated Infrastructure to use the Single Client Access Name (SCAN) listener.
- The proper routing, DNS, and security rules must be in place to allow OCI GoldenGate to access your resources

Explore More

Learn more about Oracle Cloud Infrastructure .

Review these additional resources:

Acknowledgments

Authors: Julien Testut, Suzanne Holliday, Thomas van Buggenhout
Contributors: Leo Alvarado