Network Requirements Overview
Oracle SuperCluster T5-8 includes SPARC T5-8 servers, Exadata Storage Servers, and the ZFS storage appliance, as well as equipment to connect the SPARC T5-8 servers to your network. The network connections enable the servers to be administered remotely and enable clients to connect to the SPARC T5-8 servers.
Each SPARC T5-8 server consists of the following network components and interfaces:
4 embedded Gigabit Ethernet ports (NET0, NET1, NET2, and NET3) for connection to the host management network
1 Ethernet port (NET MGT) for Oracle Integrated Lights Out Manager (Oracle ILOM) remote management
Either 4 (Half Rack) or 8 (Full Rack) dual-ported Sun QDR InfiniBand PCIe Low Profile host channel adapters (HCAs) for connection to the InfiniBand private network
Either 4 (Half Rack) or 8 (Full Rack) dual-ported Sun Dual 10-GbE SFP+ PCIe 2.0 Low Profile network interface cards (NICs) for connection to the 10-GbE client access network
Note - The QSFP modules for the 10-GbE PCIe 2.0 network cards are purchased separately.
Each Exadata Storage Server consists of the following network components and interfaces:
1 embedded Gigabit Ethernet port (NET0) for connection to the host management network
1 dual-ported Sun QDR InfiniBand PCIe Low Profile host channel adapter (HCA) for connection to the InfiniBand private network
1 Ethernet port (NET MGT) for Oracle ILOM remote management
Each ZFS storage controller consists of the following network components and interfaces:
1 embedded Gigabit Ethernet port for connection to the host management network:
NET0 on the first storage controller (installed in slot 25 in the rack)
NET1 on the second storage controller (installed in slot 26 in the rack)
1 dual-port QDR InfiniBand Host Channel Adapter for connection to the InfiniBand private network
1 Ethernet port (NET0) for Oracle ILOM remote management using sideband management. The dedicate Oracle ILOM port is not used due to sideband.
The Cisco Catalyst 4948 Ethernet switch supplied with Oracle SuperCluster T5-8 is minimally configured during installation. The minimal configuration disables IP routing, and sets the following:
Host name
IP address
Subnet mask
Default gateway
Domain name
Domain Name Server
NTP server
Time
Time zone
Additional configuration, such as defining multiple virtual local area networks (VLANs) or enabling routing, might be required for the switch to operate properly in your environment and is beyond the scope of the installation service. If additional configuration is needed, then your network administrator must perform the necessary configuration steps during installation of Oracle SuperCluster T5-8.
To deploy Oracle SuperCluster T5-8, ensure that you meet the minimum network requirements. There are three networks for Oracle SuperCluster T5-8. Each network must be on a distinct and separate subnet from the others. The network descriptions are as follows:
Management network – This required network connects to your existing management network, and is used for administrative work for all components of Oracle SuperCluster T5-8. It connects the servers, Oracle ILOM, and switches connected to the Ethernet switch in the rack. There is one uplink from the Ethernet switch in the rack to your existing management network.
Note - Network connectivity to the PDUs is only required if the electric current will be monitored remotely.Each SPARC T5-8 server and Exadata Storage Server use two network interfaces for management. One provides management access to the operating system through the 1-GbE host management interface(s), and the other provides access to the Oracle Integrated Lights Out Manager through the Oracle ILOM Ethernet interface.
Note - The SPARC T5-8 servers have four 1-GbE host management interfaces (NET 0 - NET3). All four NET interfaces are physically connected, and use IPMP to provide redundancy. See Understanding the Software Configurations for more information.The method used to connect the ZFS storage controllers to the management network varies depending on the controller:
Storage controller 1: NET0 used to provide access to the Oracle ILOM network using sideband management, as well as access to the 1-GbE host management network.
Storage controller 2: NET0 used to provide access to the Oracle ILOM network using sideband management, and NET1 used to provide access to the 1-GbE host management network.
Oracle SuperCluster T5-8 is delivered with the 1-GbE host management and Oracle ILOM interfaces connected to the Ethernet switch on the rack. The 1-GbE host management interfaces on the SPARC T5-8 servers should not be used for client or application network traffic. Cabling or configuration changes to these interfaces is not permitted.
Client access network – This required 10-GbE network connects the SPARC T5-8 servers to your existing client network and is used for client access to the servers. Database applications access the database through this network using Single Client Access Name (SCAN) and Oracle RAC Virtual IP (VIP) addresses.
InfiniBand private network – This network connects the SPARC T5-8 servers, ZFS storage appliance, and Exadata Storage Servers using the InfiniBand switches on the rack. For SPARC T5-8 servers configured with Database Domains, Oracle Database uses this network for Oracle RAC cluster interconnect traffic and for accessing data on Exadata Storage Servers and the ZFS storage appliance. For SPARC T5-8 servers configured with the Application Domain, Oracle Solaris Cluster uses this network for cluster interconnect traffic and to access data on the ZFS storage appliance. This non-routable network is fully contained in Oracle SuperCluster T5-8, and does not connect to your existing network. This network is automatically configured during installation.
Note - All networks must be on distinct and separate subnets from each other.
The following figure shows the default network diagram.
Figure 18 Network Diagram for Oracle SuperCluster T5-8
