This section describes the network connection requirements and IP address requirements to connect the Oracle Private Cloud Appliance with InfiniBand-based network architecture to your existing network infrastructure.
Before installation, you must run network cables from your existing network infrastructure to the Oracle Private Cloud Appliance installation site.
You must connect two 10 Gigabit Ethernet (GbE) IO module ports labeled “Public” on each Fabric Interconnect to your public Ethernet network.
The IO modules only support 10 GbE transport and cannot be connected to gigabit Ethernet switches. The Oracle Private Cloud Appliance must be connected externally to 10GbE optical switch ports.
Figure 3.3 shows the location of the 10 GbE Public IO module ports on the Fabric Interconnect.
It is critical that both Fabric Interconnects have two 10GbE connections each to a pair of next-level data center switches. This configuration with four cable connections provides redundancy and load splitting at the level of the Fabric Interconnects, the 10GbE ports and the data center switches. This outbound cabling should not be crossed or meshed, because the internal connections to the pair of Fabric Interconnects are already configured that way. The cabling pattern plays a key role in the continuation of service during failover scenarios involving Fabric Interconnect outages and other components.
Do not enable Spanning Tree Protocol (STP) in the upstream switch ports connecting to the Oracle Private Cloud Appliance.
Do not configure any type of link aggregation group (LAG) across the 10GbE ports: LACP, network/interface bonding or similar methods to combine multiple network connections are not supported.
To provide additional bandwidth to the environment hosted by the Oracle Private Cloud Appliance, create custom networks. For detailed information about network customization, refer to the Oracle Private Cloud Appliance Administrator's Guide.
Optional Data Center Administration Network Uplink
In addition to the public Ethernet connection, you may connect one of the two Oracle Switch ES1-24 switches to a management or machine administration network at your installation site. The system contains two Oracle Switch ES1-24 switches, making up one logical internal management network. The daisy-chained Oracle Switch ES1-24 arrangement has a special high availability (HA) configuration. Link tracking is done in software on the Oracle Switch ES1-24.
If you choose to use an uplink to the data center administration network, consider it as a long-term alternative to the temporary workstation connection described in Section 3.2.2, “IP Address Requirements for InfiniBand-based Systems”. Configure the administration uplink after the initialization of the appliance, when the appliance default network settings have been reconfigured.
Connect port 24 on one Oracle Switch ES1-24 – never both.
Make sure that the data center Ethernet switch used in this connection is configured to prevent DHCP leakage to the 192.168.4.0/24 subnet used by Oracle Private Cloud Appliance.
Use a workstation connected to the data center administration network and statically assign the IP address 192.168.4.254. Make sure there is no other machine directly connected to an internal Oracle Switch ES1-24, using the same IP address and causing IP conflicts.
A connection to the appliance internal management network, either directly into an Oracle Switch ES1-24 or through an additional Ethernet switch in the data center, is not required to access the appliance management functionality of the Oracle Private Cloud Appliance Dashboard. The primary role of the appliance internal management network is to allow the controller software on the management nodes to interact with the compute nodes and other rack components. Connecting to this network from outside the appliance allows you to gain direct administrator access to each component, for example to control the ILOMs.
The Oracle Private Cloud Appliance Dashboard, in contrast, is not accessed over the internal management network, but through the management node cluster's virtual IP address. The virtual IP address is public, in the sense that it should be accessible from within your data center network. You reserve this IP address in your data center network in advance, as described in Section 3.3, “General Network Configuration”, and then configure it in the appliance network settings during software initialization. Instructions for first access to the Oracle Private Cloud Appliance Dashboard are provided in Section 5.3.5, “Connect the Appliance to the Network”.
The Oracle Private Cloud Appliance requires a large number of IP addresses. For lists of default IP addresses that are preassigned to your Oracle Private Cloud Appliance, see Appendix A, Default IP Addresses.
To gain initial access to the Oracle Private Cloud Appliance Dashboard, you must connect a terminal or workstation with a web browser directly to the management network. Connect an Ethernet cable to the available port 19 in one of the Oracle Switch ES1-24 switches, and statically configure the wired network connection of the workstation to use the IP address 192.168.4.254.
The IP address 192.168.4.254 is the only one available for customer use in the appliance management network.
During the initial software configuration of the Oracle Private Cloud Appliance, you reconfigure the network settings of the management nodes. For this purpose, you should reserve three IP addresses in the public (data center) network: one for each management node, and one to be used as virtual IP address by both management nodes. The virtual IP address provides access to the Dashboard once the software initialization is complete.
To avoid network interference and conflicts, you must ensure that the data center network does not overlap with any of the infrastructure subnets of the Oracle Private Cloud Appliance default configuration. These are the subnets you should keep clear:
192.168.140.0/24
192.168.40.0/24
192.168.4.0/24