Network Setup
This topic describes the network requirements for deploying Oracle Data Infrastructure Cloud@Customer.
To provide secure and reliable connectivity for different application and management functions, Oracle Data Infrastructure Cloud@Customer uses separate networks.
Data Infrastructure Network
This network is configured according to Oracle specifications and should not be modified without Oracle approval.
Control Plane Network
The control plane network connects the two Control Plane Proxy Services VMs in the Data Infrastructure to Oracle Cloud Infrastructure (OCI). It serves two primary purposes:
- Enables secure customer-initiated operations through the Oracle Cloud Console and OCI APIs, such as provisioning, scaling, and managing resources.
- Enables Oracle to securely monitor and administer the Oracle-managed infrastructure from OCI.
The control plane network is dedicated to control plane communications and is separate from customer data traffic, helping ensure secure and reliable infrastructure management.
For the control plane to function properly, the Control Plane Proxy Services VMs must be able to connect to specific OCI endpoints. Enable outbound TCP port 443 access to the required endpoints in your OCI region, as described in following table.
| Description / Purpose | Open Port | Location |
|---|---|---|
| Outgoing Tunnel Service for Cloud Automation delivery | 443 outbound | Use this URL format, replacing <oci_region> with your region:https://wss.dbinfracc.<oci_region>.ocp.oraclecloud.com |
| Secure Tunnel Service for remote Oracle operator access | 443 outbound | Use this URL format, replacing <oci_region> with your region:https://mgmt.dbinfracc.<oci_region>.ocp.oraclecloud.com |
| Object Storage Service to retrieve system updates, infrastructure monitoring, and log collection | 443 outbound | Use this URL format, replacing <oci_region> with your region:https://objectstorage.<oci_region>.oraclecloud.com |
| Monitoring Service to record and process Infrastructure Monitoring Metrics (IMM) | 443 outbound | Use this URL format, replacing <oci_region> with your region:https://telemetry-ingestion.<oci_region>.oraclecloud.com |
| Identity Service for Authorization and Authentication | 443 outbound | Use this URL format, replacing <oci_region> with your region:https://identity.<oci_region>.oraclecloud.comhttps://auth.<oci_region>.oraclecloud.com |
| Resource Principal based authentication and Database service delivery | 443 outbound | Use this URL format, replacing <oci_region> with your region:https://datacc.<oci_region>.ocp.oraclecloud.comhttps://database.<oci_region>.oraclecloud.com |
| Operating System Management Service for OS updates | 443 outbound | Use this URL format, replacing <oci_region> with your region:https://osmh.<oci_region>.oci.oraclecloud.comhttps://osmh.yum.<oci_region>.oci.oraclecloud.comhttps://management-agent.<oci_region>.oci.oraclecloud.com |
Access from the Data Infrastructure to the service endpoints listed in the preceding table is required for full functionality. Failure to allow access to all required endpoints can result in reduced functionality or features not operating as expected.
The Control Plane Proxy Services VMs require outbound TCP port 443 access only. Inbound TCP port 443 access is not required and can be blocked for additional security. After the secure outbound connection is established, bidirectional communication can occur over the connection.
The Control Plane Proxy Services VMs also require customer DNS and NTP services to function properly. The minimum bandwidth requirement for the connection to OCI is 50 Mbps download and 10 Mbps upload.
Some environments require proxies for outbound internet connectivity. Use an HTTP open proxy server for Control Plane Proxy Services VM connections to OCI. Reverse proxy servers, challenge proxies, and traffic inspection are not supported.
If you use firewall rules based on IP address filtering, allow traffic for all relevant OCI region CIDR ranges listed in the following file:
https://docs.oracle.com/en-us/iaas/tools/public_ip_ranges.json
System Network
This internal system network connects the Data Infrastructure servers and Control Plane Proxy Services VMs. It is used for storage (ASM) traffic, high-performance interconnect traffic, and administration of infrastructure components.
This network is fully contained within the Data Infrastructure and does not connect directly to your corporate network. However, the Data Infrastructure is indirectly connected to your corporate network through the Control Plane Proxy Services VMs. Therefore, the IP addresses allocated to the system network must not exist elsewhere in your corporate network.
Each Data Infrastructure server and storage server has three network interfaces connected to the system network from the following ports:
- NET0: Host Networking port
- NET MGMT - Integrated Lights Out Manager (ILOM) port
- Ports on the PCIe Card slot #1
The NET0 and NET MGMT (ILOM) ports are cross-connected between the two Data Infrastructure servers. Ports on PCIe Slot #1 on the two servers are connected for private cluster interconnect between the two servers. Modification of the cabling or configuration of these interfaces is not permitted.
Customer network
Customer-managed networks required for the Data Infrastructure data plane to access related systems.
Client Network
The client network connects the VM Clusters and Application VMs on your Data Infrastructure to your existing client network and is used for client access to these VMs. Applications access Oracle AI Databases through this network by using Single Client Access Name (SCAN) and Oracle Real Application Clusters (Oracle RAC) virtual IP (VIP) interfaces.
When Data Guard is enabled, data replication uses the client network by default.
Backup Network
The backup network is similar to the client network because it connects the VM Clusters and Application VMs on your Data Infrastructure to your existing network. It can be used to access VMs for various purposes, including backups and bulk data transfers.
Like the client network, the backup network must be physically connected to the customer network.
If you use customer-managed on-premises storage, such as NFS or Oracle Zero Data Loss Recovery Appliance (Recovery Appliance), exclusively as the backup destination, the backup network does not require external connectivity to OCI.
Data Center Network Services
Before deploying Data Infrastructure, ensure that your data center network meets the required prerequisites.
Domain Name System (DNS)
As part of the deployment process, you must determine the host names and IP addresses used for the client and backup network interfaces. Register these host names and IP addresses in your corporate DNS.
At least one reliable DNS server is required and must be accessible to the Control Plane Proxy Services VMs and all servers on the client network. You can configure up to three DNS servers to provide redundancy if a server becomes unavailable.
Network Time Protocol (NTP)
Data Infrastructure uses Network Time Protocol (NTP) to synchronize time across all system components.
At least one reliable NTP server is required and must be accessible to the Control Plane Proxy Services VMs and all servers on the client network. You can configure up to three NTP servers to provide redundancy if a server becomes unavailable.
IP Addresses and Subnets
You must allocate IP address ranges for the required Data Infrastructure networks.
System Network
No overlap is permitted between the address ranges used for the system network and other networks in your corporate network. All IP addresses must be unique within your corporate network. You must also allocate IP addresses from your corporate network for the Control Plane Proxy Services VMs. Specify these network configuration details when creating the Data Infrastructure.
When creating the Data Infrastructure, the Console prepopulates default values for the system network CIDR block. You can use the suggested CIDR block if it does not overlap with existing IP addresses in a corporate network.
Review the IP address requirements for the system network. The table specifies the maximum and minimum CIDR block prefix lengths allowed for the network. The maximum CIDR block prefix length defines the smallest IP address range required for the network. To allow for future expansion, work with your network team to reserve sufficient IP addresses for anticipated growth.
| Network Type | IP Address Requirements |
|---|---|
| System network | RFC 1918 private CIDR block with a /24 prefix |
| Control plane network | Two IP addresses, one for each Control Plane Proxy Services VM |
For more information about system network CIDR requirements, see Create a Data Infrastructure.
Host Name and IP Address
To connect to a corporate network, Data Infrastructure requires host names and IP addresses for network interfaces on the client and backup networks. The number of required IP addresses depends on the system shape. Specify these network configuration details, including host names and IP addresses, when creating a VM Network. All IP addresses must be statically assigned and cannot use Dynamic Host Configuration Protocol (DHCP). The client and backup networks must use separate subnets.
Client Network
- A two-node High Availability VM Cluster requires 6 client network IPs (2 host IPs, 2 VIPs, and 2 SCAN IPs).
- A one-node VM Cluster with SCAN requires 3 client network IPs (1 host IP, 1 VIP, and 1 SCAN IP).
- A one-node VM Cluster without SCAN requires 1 client network IP (host IP only).
- An Application VM requires 1 client network IP.
Backup Network
- A two-node VM Cluster: 2 backup network IPs (one per host).
- A one-node VM Cluster: 1 backup network IP (for the single host).
- An Application VM: 1 backup network IP.
Uplinks
Ensure that the Data Infrastructure meets the uplink requirements for the control plane network, client network and the backup network.
Control Plane Proxy Services VM Network
- 4 × 10G RJ45 ports for the Control Plane Proxy Services VM Network
- Oracle supplies CAT 6 cables to connect the control plane network ports on the Data Infrastructure servers to your data center switches.
Dual-Port Configuration for Client and Backup Networks
- 4 × 10/25G SFP28 ports for the client network
- 4 × 10/25G SFP28 ports for the backup network
- Oracle supplies transceivers for the client and backup ports on the Data Infrastructure servers. You must use your own cables and transceivers for the data center switch ports.
Quad-Port Configuration for Client and Backup Networks
- 4 × 10G RJ45 ports for the client network
- 4 × 10G RJ45 ports for the backup network
- You must use your own CAT 6 cables to connect the client and backup ports on the Data Infrastructure servers to your data center switch ports.
Establish a Secure Connection Between the Control Plane Proxy Services VMs and OCI Using FastConnect
Consider using OCI FastConnect if you require additional isolation for connectivity between the Control Plane Proxy Services VMs and OCI beyond the default TLS tunnel approach.
For more information, see OCI FastConnect.
Data Infrastructure supports both the public peering and private peering FastConnect connectivity models.
The control plane network connects through the FastConnect provider to the Oracle edge network. Existing FastConnect connectivity can also be used to connect Data Infrastructure to the OCI region by using public peering.
Configuring FastConnect for Data Infrastructure
You can configure OCI FastConnect either before or after deploying Data Infrastructure.
Data Infrastructure Egress Network Configuration for FastConnect
Configure the Control Plane Proxy Services VM Network egress rules to route traffic through FastConnect. Also ensure that the network can reach an internet-facing customer DNS service. Data Infrastructure uses the customer DNS service to resolve OCI public endpoints.
Using a corporate HTTP proxy between the Control Plane Proxy Services VMs and the OCI region is not recommended when using FastConnect because FastConnect already provides dedicated network connectivity. If a corporate proxy is required, configure additional routing to ensure that Data Infrastructure traffic is routed through FastConnect.
If you use private peering, ensure that transit routing is configured on the VCN side.