2.5 Network Customization

The Oracle PCA controller software allows you to add custom networks at the appliance level. This means that the Fabric Interconnects and other hardware components are reconfigured to enable the additional connectivity. These networks are then configured automatically in your Oracle VM environment, where they can be used for isolating and optimizing network traffic beyond the capabilities of the default network configuration. All custom networks, both internal and public, are VLAN-enabled.

Caution

The following network limitations apply:

The maximum number of custom external networks is 7.
The maximum number of custom internal networks is 3.
The maximum number of VLANs is a total of 128 across all internal, external, default, and custom networks combined.

Caution

When configuring custom networks, make sure that no provisioning operations or virtual machine environment modifications take place. This might lock Oracle VM resources and cause your Oracle PCA CLI commands to fail.

Creating custom network requires use of the CLI in this software release. The administrator chooses between two types: a network internal to the appliance, or a network with external connectivity. Custom networks appear automatically in Oracle VM Manager with the virtual machine network role. For networks with external connectivity the Fabric Interconnect I/O ports must be specified so that these are reconfigured to route the external traffic. These ports must be cabled to create the physical uplink to the next-level switches in the data center.

Creating a Custom Network

Using SSH and an account with superuser privileges, log into the active management node.
Note
The default root password is Welcome1.
```
# ssh root@10.100.1.101
root@10.100.1.101's password:
root@ovcamn05r1 ~]#
```

Launch the Oracle PCA command line interface.

# pca-admin
Welcome to PCA! Release: 2.1.1
PCA>

If your custom network must be externally routable, you need to use one or more Fabric Interconnect ports. Verify the number of I/O modules and ports available and carefully plan your network customizations accordingly. The following example shows how to retrieve that information from your system:

PCA> list network-card --sorted-by Director

Slot   Director        Type                              State      Number_Of_Ports
----   --------        ----                              -----      ---------------
     3 ovcasw15r1      sanFc2Port8GbLrCardEthIb          up             2
    18 ovcasw15r1      sanFc2Port8GbLrCardEthIb          up             2
    16 ovcasw15r1      nwEthernet4Port10GbCardEthIb      up             4
     5 ovcasw15r1      nwEthernet4Port10GbCardEthIb      up             4
    17 ovcasw15r1      nwEthernet4Port10GbCardEthIb      up             4
     4 ovcasw15r1      nwEthernet4Port10GbCardEthIb      up             4
    16 ovcasw22r1      nwEthernet4Port10GbCardEthIb      up             4
     5 ovcasw22r1      nwEthernet4Port10GbCardEthIb      up             4
    18 ovcasw22r1      sanFc2Port8GbLrCardEthIb          up             2
    17 ovcasw22r1      nwEthernet4Port10GbCardEthIb      up             4
     4 ovcasw22r1      nwEthernet4Port10GbCardEthIb      up             4
     3 ovcasw22r1      sanFc2Port8GbLrCardEthIb          up             2
-----------------
12 rows displayed

Status: Success
PCA> list network-port --filter-column Type --filter nwEthernet* --sorted-by State

Port      Director        Type                   State      Networks
----      --------        ----                   -----      --------
4:4       ovcasw15r1      nwEthernet10GbPort     down       None
4:3       ovcasw15r1      nwEthernet10GbPort     down       None
4:2       ovcasw15r1      nwEthernet10GbPort     down       None
5:4       ovcasw15r1      nwEthernet10GbPort     down       None
5:3       ovcasw15r1      nwEthernet10GbPort     down       None
5:2       ovcasw15r1      nwEthernet10GbPort     down       None
10:4      ovcasw15r1      nwEthernet10GbPort     down       None
10:3      ovcasw15r1      nwEthernet10GbPort     down       None
10:2      ovcasw15r1      nwEthernet10GbPort     down       None
10:1      ovcasw15r1      nwEthernet10GbPort     down       None
11:4      ovcasw15r1      nwEthernet10GbPort     down       None
11:3      ovcasw15r1      nwEthernet10GbPort     down       None
11:2      ovcasw15r1      nwEthernet10GbPort     down       None
11:1      ovcasw15r1      nwEthernet10GbPort     down       None
4:4       ovcasw22r1      nwEthernet10GbPort     down       None
4:3       ovcasw22r1      nwEthernet10GbPort     down       None
4:2       ovcasw22r1      nwEthernet10GbPort     down       None
5:4       ovcasw22r1      nwEthernet10GbPort     down       None
5:3       ovcasw22r1      nwEthernet10GbPort     down       None
5:2       ovcasw22r1      nwEthernet10GbPort     down       None
10:4      ovcasw22r1      nwEthernet10GbPort     down       None
10:3      ovcasw22r1      nwEthernet10GbPort     down       None
10:1      ovcasw22r1      nwEthernet10GbPort     down       None
11:3      ovcasw22r1      nwEthernet10GbPort     down       None
11:2      ovcasw22r1      nwEthernet10GbPort     down       None
11:1      ovcasw22r1      nwEthernet10GbPort     down       None
4:1       ovcasw15r1      nwEthernet10GbPort     up         mgmt_public_eth, vm_public_vlan
5:1       ovcasw15r1      nwEthernet10GbPort     up         mgmt_public_eth, vm_public_vlan
4:1       ovcasw22r1      nwEthernet10GbPort     up         mgmt_public_eth, vm_public_vlan
5:1       ovcasw22r1      nwEthernet10GbPort     up         mgmt_public_eth, vm_public_vlan
10:2      ovcasw22r1      nwEthernet10GbPort     up         None
11:4      ovcasw22r1      nwEthernet10GbPort     up         None
-----------------
32 rows displayed

Status: Success

Create a new network and select either rack_internal_network or external_network. Use the following syntax:
- For an internal-only network, specify a network name.
```
PCA> create network MyInternalNetwork rack_internal_network
Status: Success
```
- For an external network, specify a network name and the Fabric Interconnect ports to be configured for external routing.
```
PCA> create network MyPublicNetwork external_network '4:2 5:2'
Status: Success
```
  Note
  The port arguments are specified as 'x:y' where x is the I/O module slot number and y is the number of the port on that module. The example above shows how to retrieve that information.

Connect the required servers to the new custom network. You must provide the network name and the names of the servers to connect.

PCA> add network MyPublicNetwork ovcacn07r1
Status: Success
PCA> add network MyPublicNetwork ovcacn08r1
Status: Success
PCA> add network MyPublicNetwork ovcacn09r1
Status: Success

Verify the configuration of the new custom network.
```
PCA> show network MyPublicNetwork

----------------------------------------
Network_Name         MyPublicNetwork
Trunkmode            True
Description          User defined network
Ports                ['4:2', '5:2']
vNICs                ovcacn09r1-eth8, ovcacn07r1-eth8, ovcacn08r1-eth8
Status               ready
Network_Type         external_network
Compute_Nodes        ovcacn07r1, ovcacn08r1, ovcacn09r1
----------------------------------------

Status: Success
```
As a result of these commands, a bond of two new vNICs is configured on each of the servers to connect them to the new custom network. These configuration changes are reflected in the Networking tab and the Servers and VMs tab in Oracle VM Manager. Figure 2.4 shows a custom network named MyPublicNetwork, which is VLAN-enabled and uses the compute node's bond5 interface consisting of Ethernet ports (vNICs) eth8 and eth8B.
Figure 2.4 Oracle VM Manager View of Custom Network Configuration

Click to view larger image ovmm_servertab_customnetcfg.png
To disconnect servers from the custom network use the remove network command.
Warning
Before removing the network connection of a server, make sure that no virtual machines are relying on this network.
When a server is no longer connected to a custom network, make sure that its port configuration is cleaned up in Oracle VM.
```
PCA> remove network MyPublicNetwork ovcacn09r1
************************************************************
 WARNING !!! THIS IS A DESTRUCTIVE OPERATION.
************************************************************
Are you sure [y/N]:y

Status: Success
```

Deleting a Custom Network

Caution

Before deleting a custom network, make sure that all servers have been disconnected from it first.

Using SSH and an account with superuser privileges, log into the active management node.
Note
The default root password is Welcome1.
```
# ssh root@10.100.1.101
root@10.100.1.101's password:
root@ovcamn05r1 ~]#
```

Launch the Oracle PCA command line interface.

# pca-admin
Welcome to PCA! Release: 2.1.1
PCA>

Verify that all servers have been disconnected from the custom network. No vNICs or nodes should appear in the network configuration.

Caution

Related configuration changes in Oracle VM must be cleaned up as well.

PCA> show network MyPublicNetwork

----------------------------------------
Network_Name         MyPublicNetwork
Trunkmode            True
Description          User defined network
Ports                ['4:2', '5:2']
vNICs                None
Status               ready
Network_Type         external_network
Compute_Nodes        None
----------------------------------------

Status: Success

Delete the custom network.

PCA> delete network MyPublicNetwork
************************************************************
 WARNING !!! THIS IS A DESTRUCTIVE OPERATION.
************************************************************
Are you sure [y/N]:y

Status: Success

Caution

If a custom network is left in an invalid or error state, and the delete command fails, you may use the --force option and retry.