Creating and Managing Exadata Networks

Oracle Private Cloud Appliance supports direct connectivity to Oracle Exadata clusters.

This section describes creating and managing Exadata networks from the Service Enclave. Before you can create an Exadata network, you must physically connect your Private Cloud Appliance to an Oracle Exadata rack. For instructions, see the "Optional Connection to Exadata" section in the chapter Configuring Oracle Private Cloud Appliance of the Oracle Private Cloud Appliance Installation Guide.

In order to use an Exadata network, the VCNs containing compute instances that connect to the database nodes, must have a dynamic routing gateway (DRG) configured. The enabled subnet needs a route rule with the Exadata CIDR as destination and the DRG as target.

For more information about Oracle Exadata Integration, see the "Network Infrastructure" section in the Hardware Overview chapter of the Oracle Private Cloud Appliance Concepts Guide.

Creating an Exadata Network

To set up a network connection between Private Cloud Appliance and an Oracle Exadata system, you need this set of parameters:

Note:

When connecting more than one Oracle Exadata system to the Private Cloud Appliance rack, you must be careful that the configured IP address ranges do not overlap. This cannot be detected or prevented any other way.

Parameter	Example Value	Description
cidr	10.nn.nn.0/24	Choose a valid CIDR range that is within the CIDR range of the Oracle Exadata.
spine1Ip	10.nn.nn.2	A valid IP address in the CIDR specified.
spine2Ip	10.nn.nn.3	A valid IP address in the CIDR specified.
spineVip	10.nn.nn.1	A valid IP address in the CIDR specified.
vlan	3062	Choose a VLAN from 2 to 3899 that isn't in use by the uplink VLAN or other Oracle Exadata VLANs. This parameter can be unspecified for attaching a device not supporting VLAN tagging.
ports	7/1,7/2	Valid ports are '7/1','7/2','7/3','7/4','8/1','8/2','8/3','8/4', '9/1','9/2','9/3','9/4','10/1','10/2','10/3','10/4'.
advertiseNetwork	True	True or False - enables or disables the visibility of the Exadata network to the customer's data center servers. `advertiseNetwork=true` is only available for dynamic routing configurations.

Using the Service Web UI

Determine the Exadata network parameters listed in the table above.
In the Dashboard, click the Rack Units quick action tile.
In the PCA Config navigation menu on the Rack Units page, click Exadata Networks.
In the top-right corner above the table, click Create Exadata Network.
Fill out the Exadata Network form using the parameters you collected in advance.

By default the network is not advertised to the data center network. You have to click the slider to set it to "on"/"true".
Click Submit to create the new network. It appears in the Exadata Networks table and its Lifecycle State changes to Available when the configuration has been applied successfully.
Next, add a subnet to the Exadata network. See Enabling Oracle Exadata Access.

Using the Service CLI

Determine the Exadata network parameters listed in the table above.

Create the Exadata network by entering the parameters.

PCA-ADMIN> exaDataCreateNetwork cidr="10.nn.nn.0/24" vlan=2001 spine1Ip="10.nn.nn.101" \
spine2Ip="10.nn.nn.102" spineVip="10.nn.nn.1" ports="7/1,7/2"
Command: exaDataCreateNetwork cidr="10.nn.nn.0/24" vlan=2001 spine1Ip="10.nn.nn.101" \ 
spine2Ip="10.nn.nn.102" spineVip="10.nn.nn.1" ports="7/1,7/2"
Status: Success
Time: 2021-11-22 06:10:05,260 UTC
Data: ocid1.exadata.unique_id

Next, add a subnet to the Exadata network. See Enabling Oracle Exadata Access.

Enabling Oracle Exadata Access

Enabling access from a subnet to the Exadata network must be done through the Service CLI.

Subnets that have been granted access, appear in the Exadata network detail page under Access Lists, grouped by their parent VCN.

Using the Service CLI

Get the OCID of the Exadata network you want to enable, using the exaDataGetNetwork command.

Enable access to a configured Exadata network.

PCA-ADMIN> exaDataEnableAccess exadataNetworkId=ocid1.exadata.unique_id \
subnetId=ocid1.subnet.unique_id
Command: exaDataEnableAccess exadataNetworkId=ocid1.exadata.unique_id \
subnetId=ocid1.subnet.unique_id
Status: Success
Time: 2021-11-17 18:56:45,251 UTC
Data:
 id
 --
 ocid1.vcn.unique_id

If you are using a secondary NIC to access the Exadata network, you must add a route to the Exadata CIDR address range for interface eth1 (the secondary NIC).
```
PCA-ADMIN> route add Exadata-CIDR-address-range gw Addr-default-router eth1
```
For example, if the Exadata address range is 192.168.0.0/24 and the gateway is 192.168.0.1:
```
PCA-ADMIN> route add 192.169.0.0/24 gw 192.168.0.1 eth1 
```
This entry appears as a second interface in the IP routing table:
```
Destination Gateway      Genmask       Flags Metric Ref Use Iface
 . . . . . . . . . . . . . . . . . . . . 
 192.168.1.0 192.168.1.1 255.255.255.0 0     0      0       eth0 
 192.168.0.0 192.168.0.1 255.255.255.0 0     0      0       eth1 
```
A ping from the secondary NIC, eth1, now succeeds to the Exadata network.

List Exadata Networks

Using the Service Web UI

In the Dashboard, click the Rack Units quick action tile.
In the PCA Config navigation menu on the Rack Units page, click Exadata Networks. The table contains all configured Exadata networks.

Using the Service CLI

Use the exaDataListNetwork command to display configured Exadata networks, including their OCIDs.

PCA-ADMIN> exaDataListNetwork
Command: exaDataListNetwork
Status: Success
Time: 2021-11-22 06:10:17,617 UTC
Data:
  id                        vlan   cidr            spine1Ip       spine2Ip       spineVip     ports
  --                        ----   ----            --------       --------       --------     -----
  ocid1.exadata.unique_id   2001   10.nn.nn.0/24   10.nn.nn.101   10.nn.nn.102   10.nn.nn.1   7/1,7/2

Get Exadata Network Details

Using the Service Web UI

Navigate to the Exadata Network page.
In the overview table, click the name (OCID) of the network for which you want to display details.

The Exadata Network detail page shows the configuration parameters, the state of the network, and the subnets that have been granted access.

Using the Service CLI

Get the OCID of the Exadata network for which you want details, using the exaDataListNetwork command.

Use the exaDataGetNetwork command to display details about a specific Exadata network, including the state of the network, subnet and VCN IDs.

PCA-ADMIN> exaDataGetNetwork exadataNetworkId=ocid1.exadata.unique_id
Command: exaDataGetNetwork exadataNetworkId=ocid1.exadata.unique_id
Status: Success
Time: 2021-11-22 19:34:56,917 UTC
Data:
  CIDR = 10.nn.nn.0/24
  Vlan = 2001
  Spine1Ip = 10.nn.nn.101
  Spine2Ip = 10.nn.nn.102
  SpineVip = 10.nn.nn.1
  Ports = 7/1,7/2
  advertiseNetwork = false
  Access List 1 - Vcn Id = ocid1.vcn.unique_id
  Access List 1 - Subnet Ids 1 = ocid1.subnet.unique_id
  Access List 1 - Subnet Ids 1 = ocid1.subnet.unique_id
  Access List 2 - Vcn Id = ocid1.vcn.unique_id
  Access List 2 - Subnet Ids 1 = ocid1.subnet.unique_id
  Lifecycle State = AVAILABLE

Disabling Oracle Exadata Access

Disabling access from a subnet to the Exadata network must be done through the Service CLI.

Subnets that have been granted access, appear in the Exadata network detail page under Access Lists, grouped by their parent VCN. When you disable access for a given subnet, it is removed from the Access Lists.

Using the Service CLI

Get the OCID of the Exadata network you want to disable, using the exaDataListNetwork command.
Get the OCID of the subnet ID for the Exadata network using the exaDataGetNetwork command.

Disable access to a configured Exadata network.

PCA-ADMIN> exaDataDisableAccess exadataNetworkId=ocid1.exadata.unique_id \
subnetId=ocid1.subnet.unique_id
Command: exaDataDisableAccess exadataNetworkId=ocid1.exadata.unique_id  \
subnetId=ocid1.subnet.unique_id
Status: Success
Time: 2021-11-02 11:29:49,873 UTC
PCA-ADMIN> exaDatadisableAccess exadataNetworkId=ocid1.exadata.unique_id \ 
subnetId=ocid1.subnet.unique_id \
Command: exaDatadisableAccess exadataNetworkId=ocid1.exadata.unique_id \ 
subnetId=ocid1.subnet.unique_id \
Status: Success
Time: 2021-12-15 11:26:40,344 UTC
Data:
  id
  --
  ocid1.vcn.unique_id \
PCA-ADMIN>

Deleting an Exadata Network

Using the Service Web UI

Make sure that, for the Exadata network you intend to delete, access has been disabled first.
Navigate to the Exadata Network page.
Choose one of these options to delete the Exadata network:
- In the overview table, open the Actions menu on the right hand side of the row and select Delete. When prompted, click Confirm.
- Open the Exadata network detail page, then click the Delete button in the top-right corner.

Using the Service CLI

Make sure that, for the Exadata network you intend to delete, access has been disabled first.
Get the OCID of the Exadata network you want to delete, using the exaDataListNetwork command.

Delete the Exadata network.

PCA-ADMIN> exaDatadeleteNetwork exadataNetworkId=ocid1.exadata.unique_id
Command: exaDatadeleteNetwork exadataNetworkId=ocid1.exadata.unique_id
Status: Success
Time: 2021-11-16 05:59:54,177 UTC