2 Understanding the Network Requirements for Oracle Exadata Database Machine

This chapter describes the network requirements for Oracle Exadata Database Machine.

Note:

For ease of reading, the name "Oracle Exadata Rack" is used when information refers to both Oracle Exadata Database Machine and Oracle Exadata Storage Expansion Rack.

2.1 Overview of Network Requirements

Oracle Exadata Database Machine includes database servers, Exadata Storage Servers, as well as equipment to connect the servers to your network. The network connections allow the servers to be administered remotely, and clients to connect to the database servers.

The information in this section should be used in conjunction with Oracle Exadata Deployment Assistant to configure your environment for Oracle Exadata Database Machine.

Note:

The name used for the InfiniBand interface changed from BONDIB0 to IB0 and IB1 on Oracle Exadata Database Machine X4-2 systems using release 11.2.3.3.0 and later.

As of release 11.2.2.1.0, the names used for bonding changed. The names are BONDIB0 for the InfiniBand bonding and BONDETH0 for Ethernet bonding. In earlier releases, the names were BOND0 and BOND1, respectively.

To deploy Oracle Exadata Database Machine ensure that you meet the minimum network requirements. Oracle Exadata Database Machine requires a minimum of three networks, and there are interfaces available for additional networks. Each network must be on a distinct and separate subnet from the others. Oracle recommends one common additional network for backup, which you can configure with OEDA. The network descriptions are as follows:

  • Management network: This required network connects to your existing management network, and is used for administrative work for all components of Oracle Exadata Database Machine. It connects the servers, ILOM, and switches connected to the Ethernet switch in the rack. There is one uplink from the Ethernet switch in the rack to your management network, one uplink from the KVM switch in the rack to your management network, and one uplink for each power distribution unit (PDU).

    Notes:

    • Network connectivity to the PDUs is only required if the electric current is monitored remotely.

    • A properly secured configuration requires the management network to be fully isolated from all other networks.

    Each database server and Exadata Storage Server has two network interfaces for management. One provides management access to the operating system through the embedded NET0 Ethernet port on Oracle Exadata Database Machine Two-Socket systems, or the NEM0/NET1 Ethernet port on Oracle Exadata Database Machine Eight-Socket systems. The other network interface provides access to the ILOM through the NET MGT Ethernet port. Oracle Exadata Database Machines are delivered with the NET MGT and NET0 or NEM0/NET0 ports connected to the Ethernet switch in the rack. The NET0 or NEM0/NET0 port on the database servers should not be used for client or application network traffic. Cabling or configuration changes to these interfaces on Exadata Storage Servers is not permitted.

  • Client access network: This required network connects the database servers to your existing client network and is used for client access to the database servers. Applications access the database through this network using Single Client Access Name (SCAN) and Oracle RAC Virtual IP (VIP) addresses. Database servers support channel bonding to provide higher bandwidth or availability for client connections to the database. Oracle recommends channel bonding for the client access network.

    The following list describes the channel bonding configurations:

    • Oracle Exadata Database Machine Two-Socket systems, use either:

      • The embedded NET1 and NET2 ports

      • The two 10 GbE ports or two 10/25GbE ports on Oracle Exadata Database Machine X7-2

    • Oracle Exadata Database Machine Eight-Socket systems, X4-8 or higher

      • Use two ports, with each port on a different PCIe card

    • Oracle Exadata Database Machine X3-8 Full Rack and Oracle Exadata Database Machine X2-8 Full Rack

      • Use ports from each Network Express Module (NEM)

        For the Oracle Exadata Database Machine X3-8 Full Rack and Oracle Exadata Database Machine X2-8 Full Rack database servers, using the same ports on the Network Express Modules, such as NEM0/NET1 and NEM0/NET2, works but does not provide for high availability.

    Note:

    If your system will use more than 10 VIP addresses in the cluster and you have VLAN configured for the Oracle Clusterware client network, then you must use 3 digit VLAN ids. Do not use 4 digit VLAN ids because the VLAN name can exceed the 15 character operating system interface name limit.

  • Additional networks (optional): Database servers can connect to additional existing networks using the available open ports not used by the management and client networks.

  • InfiniBand private network: This network connects the database servers and Oracle Exadata Storage Servers using the InfiniBand switches on the rack. Starting with the Oracle Exadata Storage X4-2L servers, the InfiniBand fabric is connected using the IB0 and IB1 interfaces. Oracle Exadata Storage Server X3-2L servers and earlier use the BONDIB0 interface. Oracle Database uses this network for Oracle RAC cluster interconnect traffic and for accessing data on Oracle Exadata Storage Servers. This non-routable network is fully contained in Oracle Exadata Database Machine, and does not connect to your existing network. This network is automatically configured during installation.

Note:

All networks must be on distinct and separate subnets from each other. For better security, the management network should be isolated from all other networks.

Oracle Exadata Database Machine database server networking requirements are identical with the following exceptions:

  • Oracle Exadata Database Machine X7-2 embedded ports are 1/10/25 GbE.

  • Oracle Exadata Database Machine X6-2, Oracle Exadata Database Machine X5-2, Oracle Exadata Database Machine X4-2, and Oracle Exadata Database Machine X3-2 embedded ports are 1 GbE/10 GbE.

  • Oracle Exadata Database Machine X2-2 embedded ports are 1 GbE.

  • Only Oracle Exadata Database Machine X2-2 has a KVM.

At start time on an 8-socket system, each database server generates a cellaffinity.ora configuration file. The cellaffinity.ora file resides in the /etc/oracle/cell/network-config directory, and must be readable by the Oracle database.

The file contains a mapping between the NUMA node numbers and the IP address of the network interface card closest to each node. Oracle Database uses the file to select the closest network interface card when communicating with Exadata Storage Servers, thereby optimizing performance. This file is only generated and used on an 8-socket system. On a 2-socket system, there is no performance to be gained in this manner, and no cellaffinity.ora file. The file is not intended to be directly edited with a text editor.

The following image shows the default network diagram for Oracle Exadata Database Machine X7-2. Use the files from Oracle Exadata Database Machine Deployment Assistant to verify the cable connections when cabling Oracle Exadata Database Machine to the existing network.

Figure 2-1 Network Diagram for Oracle Exadata Database Machine X7-2

Description of Figure 2-1 follows
Description of "Figure 2-1 Network Diagram for Oracle Exadata Database Machine X7-2"

The following image shows the default network diagram for Oracle Exadata Database Machine X4-2 and later models. Use the files from Oracle Exadata Database Machine Deployment Assistant to verify the cable connections when cabling Oracle Exadata Database Machine to the existing network.

Figure 2-2 Network Diagram for Oracle Exadata Database Machine X4-2 and Later Models

Description of Figure 2-2 follows
Description of "Figure 2-2 Network Diagram for Oracle Exadata Database Machine X4-2 and Later Models"

The following image shows the default network diagram for Oracle Exadata Database Machine X3-2 and Oracle Exadata Database Machine X2-2. Use the files from Oracle Exadata Database Machine Deployment Assistant to verify the cable connections when cabling Oracle Exadata Database Machine to the existing network.

Figure 2-3 Network Diagram for Oracle Exadata Database Machine X3-2 and Oracle Exadata Database Machine X2-2

Description of Figure 2-3 follows
Description of "Figure 2-3 Network Diagram for Oracle Exadata Database Machine X3-2 and Oracle Exadata Database Machine X2-2"

Notes:

  • Oracle Exadata Database Machine X3-8 Full Rack, and Oracle Exadata Database Machine X2-8 Full Rack have BONDIB0, BONDIB1, and so on for the InfiniBand private network.

  • The KVM switch is only available in Oracle Exadata Database Machine X2-2 racks.

  • Additional configuration, such as defining multiple virtual local area networks (VLANs) or enabling routing, may be required for the switch to operate properly in your environment, and is not included in the installation service. If additional configuration is needed, then the network administrator performs the necessary configuration during Oracle Exadata Database Machine installation.

2.2 Network Components and Interfaces

Each Oracle Exadata Database Machine contains several network components and interfaces.

2.2.1 Oracle Exadata Database Machine X7-2 Database Server

Each Oracle Exadata Database Machine X7-2 database server consists of the following network components and interfaces:

  • 1 embedded 1 GbE/10 GbE port and 2 embedded 10 GbE/25GbE ports

  • 1 dual-port 4X QDR (40 Gb/s) InfiniBand Host Channel Adapter (HCA)

  • 1 embedded Ethernet port for Sun Integrated Lights Out Manager (ILOM) remote management

  • 1 dual-port 25GbE PCIe 3.0 network card with Broadcom BCM57414 10Gb/25Gb Ethernet Controller

Note:

The SFP28 modules for the 25GbE PCIe 3.0 network cards are purchased separately.

2.2.2 Oracle Exadata Database Machine X5-2 and X6-2 Database Server

Each Oracle Exadata Database Machine X5-2 and X6-2 database server consists of the following network components and interfaces:

  • 4 embedded 1 GbE/10 GbE ports

  • Oracle Exadata Database Machine X6-2 or X5-2: 1 dual-port 4X QDR (40 Gb/s) InfiniBand Host Channel Adapter (HCA)

  • Oracle Exadata Database Machine SL6 only: 2 dual-port 4X QDR (40 Gb/s) InfiniBand Host Channel Adapters (HCA)

  • 1 embedded Ethernet port for Sun Integrated Lights Out Manager (ILOM) remote management

  • 1 dual-port 10 GbE PCIe 2.0 network card with Intel 82599 10 GbE controller

Note:

The Small Form-Factor Pluggable (SFP) modules for the 10 GbE PCIe 2.0 network cards are purchased separately.

2.2.3 Oracle Exadata Database Machine X5-8 and X6-8 Full Rack Database Server

Each Oracle Exadata Database Machine X5-8 or X6-8 Full Rack database server consists of the following network components and interfaces:

  • 4 dual-port 4x QDR (40 Gb/s) InfiniBand Host Channel Adapter (HCA)

  • 10 x 1 GbE Base-T Ethernet Ports (2 Quad-port PCIe 2.0 network card, and 2 embedded ports based on the Intel I350 1GbE Controller technology)

  • 8 x 10 GbE Ethernet SFP+ Ports (4 Dual-port 10GbE PCIe 2.0 network card based on the Intel 82599 10GbE Controller technology)

Note:

The Small Form-Factor Pluggable (SFP) modules for the 10 GbE PCIe 2.0 network cards are purchased separately.

2.2.4 Oracle Exadata Database Machine X4-2 Database Server

Each Oracle Exadata Database Machine X4-2 database server consists of the following network components and interfaces:

  • 4 embedded 1 GbE/10 GbE ports

  • 1 dual-port 4X QDR (40 Gb/s) InfiniBand Host Channel Adapter (HCA)

  • 1 embedded Ethernet port for Sun Integrated Lights Out Manager (ILOM) remote management

  • 1 dual-port 10 GbE PCIe 2.0 network card with Intel 82599 10 GbE controller

Note:

The Small Form-Factor Pluggable (SFP) modules for the 10 GbE PCIe 2.0 network cards are purchased separately.

2.2.5 Oracle Exadata Database Machine X4-8 Full Rack Database Server

Each Oracle Exadata Database Machine X4-8 Full Rack database server consists of the following network components and interfaces:

  • 4 dual-port 4x QDR (40 Gb/s) InfiniBand Host Channel Adapter (HCA)

  • 10 x 1 GbE Base-T Ethernet Ports (2 Quad-port PCIe 2.0 network card, and 2 embedded ports based on the Intel I350 1GbE Controller technology)

  • 8 x 10 GbE Ethernet SFP+ Ports (4 Dual-port 10GbE PCIe 2.0 network card based on the Intel 82599 10GbE Controller technology)

Note:

The Small Form-Factor Pluggable (SFP) modules for the 10 GbE PCIe 2.0 network cards are purchased separately.

2.2.6 Oracle Exadata Database Machine X3-2 Database Server

Each Oracle Exadata Database Machine X3-2 database server consists of the following network components and interfaces:

  • 4 embedded 1 GbE ports

  • 1 dual-port 4X QDR (40 Gb/s) InfiniBand Host Channel Adapter (HCA)

  • 1 embedded Ethernet port for Sun Integrated Lights Out Manager (ILOM) remote management

  • 1 dual-port 10 GbE PCIe 2.0 network card with Intel 82599 10 GbE controller

Note:

The Small Form-Factor Pluggable (SFP) modules for the 10 GbE PCIe 2.0 network cards are purchased separately.

2.2.7 Oracle Exadata Database Machine X3-8 Full Rack and Oracle Exadata Database Machine X2-8 Full Rack Database Server

Each Oracle Exadata Database Machine X3-8 Full Rack and Oracle Exadata Database Machine X2-8 Full Rack database server consists of the following network components and interfaces:

  • 2 Network Express Modules (NEMs) providing the total of the following:

    • 8 x 10 GbE network ports

    • 8 x 1 Gb Ethernet network ports

  • 4 dual-port 4X QDR InfiniBand PCIe 2.0 Express Modules (EMs)

  • 1 Ethernet port for Sun Integrated Lights Out Manager (ILOM) remote management

Note:

The SFP+ modules for the NEMs are purchased separately.

2.3 Using Network VLAN Tagging with Oracle Exadata Database Machine

Compute nodes and storage nodes can use VLANs for the management network, ILOM, client network, and the backup access network.

By default, the network switch is minimally configured, without VLAN tagging. If VLAN tagging is to be used, then it must be enabled by the customer after the initial deployment. This applies to both physical and Oracle VM deployments.

Client and backup VLAN networks must be bonded. The admin network is never bonded.

Notes:

  • VLANs do not exist in InfiniBand. For equivalent functionality, use InfiniBand partitioning as described in Oracle Exadata Database Machine Maintenance Guide.

  • Network VLAN tagging is supported for Oracle Real Application Clusters on the public network.

  • If the backup network is on a tagged VLAN network, the client network must also be on a separate tagged VLAN network.

  • The backup and client networks can share the same network cables.

  • Oracle Exadata Deployment Assistant (OEDA) supports VLAN tagging for both physical and virtual deployments.

  • Virtual deployments do not support IPv6 VLANs.

  • If your system will use more than 10 VIP addresses in the Oracle RAC cluster and you have VLAN configured for the Oracle Clusterware client network, then you must use 3 digit VLAN ids. Do not use 4 digit VLAN ids because the VLAN name can exceed the 15 character operating system interface name limit.

The following table shows IPv4/IPv6 support on the admin, client, and backup networks for the different Exadata systems and Oracle Database versions.

Version of Oracle Database VLAN Tagging on Admin Network Client and Backup Networks
11.2.0.4

Only supported with IPv4 addresses on X3-2 and above for two-socket servers, and X4-8 and above for eight-socket servers.

Supported with IPv4 and IPv6 on all hardware models.

12.1.0.2

Only supported with IPv4 addresses on X3-2 and above for two-socket servers, and X4-8 and above for eight-socket servers.

Supported with IPv4 on all hardware models.

Supported with IPv6 on all hardware models with fix for 22289350.

2.4 Network Channel Bonding Support

The network port on a database server used for the client access network can be bonded with another network on the database server to provide higher bandwidth or availability for client connections to the database. In a bonded network configuration, Oracle RAC SCAN and VIP addresses are defined on the BONDETH0 interface on database servers.

2.4.1 Bonded Network Configurations

A bonded network configuration is used when there is a need to aggregate multiple network interfaces together for client connections to the database.

The bonded network configuration is the default configuration.

The following sections describe the bonded network configuration for Oracle Exadata Database Machine:

2.4.1.1 Bonded Network Configuration for Oracle Exadata Database Machine Two-Socket Systems

When using a bonded network configuration for database client access on Oracle Exadata Database Machine Two-Socket Systems, the database servers are configured as follows:

  • Port NET0: Provides access to the operating system using the management network through the ETH0 interface.

  • Port NET1: Provides access to the database using the client access network, and is bonded with NET2 port to create the BONDETH0 interface.

  • Port NET2: Provides access to the database using the client access network, and is bonded with NET1 port to create the BONDETH0 interface.

  • Port NET MGT: Provides access to ILOM using the management network.

When channel bonding is configured for the client access network during initial configuration, the Linux bonding module is configured for active-backup mode (mode=1). If a different bonding policy is preferred, then you may reconfigure the bonding module after initial configuration. For configuration details, refer to the Linux Ethernet Bonding Driver HOWTO in the /usr/share/doc/kernel-doc-2.6.39/Documentation/networking/bonding.txt directory on any Oracle Exadata Database Machine database server.

The following table shows the port connections for Oracle Exadata Database Machine Two-Socket Systems database servers when using channel bonding. The client access network and the backup network must use different ports, in that one network must use the embedded GbE ports and the other network must use the PCIe ports.

Table 2-1 Channel Bonding Connections for Oracle Exadata Database Machine X7-2

Network Using 1 GbE Ports Using 10/25 GbE Ports

Management (1 GbE NET0)

The administration network connection uses the 1 GbE port even when using 10 GbE ports for the other networks.

  • Port: 1 GbE NET0

  • Connect to Cisco switch

  • Port: 1 GbE NET0

  • Connect to Cisco switch

Client (BONDETH0)

  • Ports: 1 GbE NET1 and 1 GbE NET2

  • Connect to client access 1 GbE switches

  • Ports: 10/25 GbE PCI Ports 1 and 2

  • Connect to client access 10/25 GbE switches

Backup (BONDETH1)

  • Ports: 10/25 GbE PCI Ports 1 and 2

  • Connect to backup 10/25 GbE switches

  • Ports: 10/25 GbE NET2 and NET3

  • Connect to backup 10/25 GbE switches

Table 2-2 Channel Bonding Connections for Other Oracle Exadata Database Machine Two-Socket Systems

Network Using 1 GbE Ports Using 10 GbE Ports

Management (1 GbE NET0)

The administration network connection uses the 1 GbE port even when using 10 GbE ports for the other networks.

  • Port: 1 GbE NET0

  • Connect to Cisco switch

  • Port: 1 GbE NET0

  • Connect to Cisco switch

Client (BONDETH0)

  • Ports: 1 GbE NET1 and 1 GbE NET2

  • Connect to client access 1 GbE switches

  • Ports: 10 GbE PCI1/PORTA and 10 GbE PCI1/PORTB

  • Connect to client access 10 GbE switches

Backup (BONDETH1)

  • Ports: 10 GbE PCI1/PORTA and 10 GbE PCI1/PORTB

  • Connect to backup 10 GbE switches

  • Ports: 1 GbE NET2 and 1 GbE NET3

  • Connect to backup 1 GbE switches

Note:

When using a bonded network configuration, you must provide a network switch capable of supporting the chosen bonding mode. For example, if mode 4 is configured (IEEE 802.3ad Dynamic link aggregation), then you must supply and configure the network switch capable of supporting this bonding mode. Requirements for specific bonding policies are documented in the Linux Ethernet Bonding Driver HOWTO in the /usr/share/doc/kernel-doc-2.6.39/Documentation/networking/bonding.txt directory on any Oracle Exadata Database Machine database server.

2.4.1.2 Bonded Network Configuration for Oracle Exadata Database Machine X7-8

When using a bonded network configuration for database client access on Oracle Exadata Database Machine X7-8, the database servers are configured as follows:

  • Port NET0: Provides access to the operating system using the management network through the ETH0 interface.

  • P1 at 3 PCIe 4: Provides access to the database using the client access network, and is bonded with port P1 at 13 PCIe 12, to create the BONDETH0 interface.

  • P2 at 13 PCIe 6: Provides access to the database using the client access network, and is bonded with port P2 at 3 PCIe 14 to create the BONDETH1 interface.

  • Port NET MGT: Provides access to ILOM using the management network.

The following table shows the port connections for Oracle Exadata Database Machine X7-8 database servers when using channel bonding.

Table 2-3 Channel Bonding Connections for Oracle Exadata Database Machine X7-8

Network Using 1 GbE Ports Using 25 GbE Ports

Management (NET0)

The administration network connection uses the 1 GbE port even when using 10 GbE ports for the other networks.

  • Port: 1 GbE NET0

  • Connect to Cisco switch

  • Port: 1 GbE NET0

  • Connect to Cisco switch

Client (BONDETH0)

  • SMOD0 and SMOD1’s Net2

  • Connect to client access 1 GbE switches

  • Port1 in PCIe 4 and 12 cards

  • Connect to client access 25 GbE switches

Backup (BONDETH1)

  • SMOD0 and SMOD1’s Net3

  • Connect to backup 1 GbE switches

  • Port1 in PCIe 6 and 14 cards

  • Connect to backup 25 GbE switches

2.4.1.3 Bonded Network Configuration for Oracle Exadata Database Machine X4-8, X5-8, and X6-8

When using a bonded network configuration for database client access on Oracle Exadata Database Machine X4-8, X5-8, and X6-8, the database servers are configured as follows:

  • Port NET0: Provides access to the operating system using the management network through the ETH0 interface.

  • P0 at 3 PCIe3: Provides access to the database using the client access network, and is bonded with port P0 at 13 PCIe3, to create the BONDETH0 interface.

  • P0 at 13 PCIe3: Provides access to the database using the client access network, and is bonded with port P0 at 3 PCIe3 to create the BONDETH1 interface.

  • Port NET MGT: Provides access to ILOM using the management network.

The following table shows the port connections for Oracle Exadata Database Machine X4-8, X5-8, and X6-8 database servers when using channel bonding.

Table 2-4 Channel Bonding Connections for Oracle Exadata Database Machine X4-8, X5-8, and X6-8

Network Using 1 GbE Ports Using 10 GbE Ports

Management (NET0)

The administration network connection uses the 1 GbE port even when using 10 GbE ports for the other networks.

  • Port: 1 GbE NET0

  • Connect to Cisco switch

  • Port: 1 GbE NET0

  • Connect to Cisco switch

Client (BONDETH0)

  • 1 GbE Ports: P0 at 3 PCIe3, and at 13 PCIe3

  • Connect to client access 1 GbE switches

  • 10 GbE Ports: 2 PCIe3 PORTA, and 4 PCIe 3 PORTA

  • Connect to client access 10 GbE switches

Backup (BONDETH1)

  • 10 GbE ports: 2 PCIe3 PORTA, and 4 PCIe3 PORTA

  • Connect to backup 10 GbE switches

  • 1 GbE ports: P0 at 3 PCIe3, and at 13 PCIe3

  • Connect to backup 1 GbE switches

2.4.1.4 Bonded Network Configuration for Oracle Exadata Database Machine X3-8 Full Rack and Oracle Exadata Database Machine X2-8 Full Rack

When using a bonded network configuration for database client access on Oracle Exadata Database Machine X3-8 Full Rack and Oracle Exadata Database Machine X2-8 Full Rack, the database servers are configured as follows:

  • Port NEM0/NET0: Provides access to the operating system using the management network through the ETH0 interface.

  • Port NEM0/NET1: Provides access to the database using the client access network, and is bonded with port NEM1/NET3 to create the BONDETH0 interface.

  • Port NEM1/NET3: Provides access to the database using the client access network, and is bonded with port NEM0/NET1 to create the BONDETH0 interface.

  • Port NET MGT: Provides access to ILOM using the management network.

Table 2-5 shows the port connections for Oracle Exadata Database Machine X3-8 Full Rack and Oracle Exadata Database Machine X2-8 Full Rack database servers when using channel bonding. The NEM0 ports are on the left, and the NEM1 ports are on the right.

Table 2-5 Channel Bonding Connections for Oracle Exadata Database Machine X2-8 and X3-8 Full Rack

Network Using 1 GbE Ports Using 10 GbE Ports

Management (NET0)

  • Port: 1 GbE NET0

  • Connect to Cisco switch

  • Port: 1 GbE NET0

  • Connect to Cisco switch

Client (BONDETH0)

  • Ports: 1 GbE NET1 and 1 GbE NET2

  • Connect to client access 1 GbE switches

  • Ports: 10 GbE NEM0/PORTA and 10 GbE NEM1/PORTB

  • Connect to client access 10 GbE switches

Backup (BONDETH1)

  • Ports: 10 GbE NEM0/PORTA and 10 GbE NEM1/PORTB

  • Connect to backup 10 GbE switches

  • Ports: 1 GbE NET1 and 1 GbE NET2

  • Connect to backup 1 GbE switches

When channel bonding is configured for the client access network during initial configuration, the Linux bonding module is configured for active-backup mode (mode=1). If a different bonding policy is preferred, then you may reconfigure the bonding module after initial configuration. For configuration details, refer to the Linux Ethernet Bonding Driver HOWTO in the /usr/share/doc/kernel-doc-2.6.18/Documentation/networking/bonding.txt directory on any Oracle Exadata Database Machine database server.

Note:

When using a bonded network configuration, you must provide a network switch capable of supporting the chosen bonding mode. For example, if mode 4 is configured (IEEE 802.3ad Dynamic link aggregation), then you must supply and configure the network switch capable of supporting this bonding mode. Requirements for specific bonding policies are documented in the Linux Ethernet Bonding Driver HOWTO.

2.4.2 Non-bonded Network Configuration

A non-bonded network configuration is used when there is no need or ability to implement network bonding.

The following sections describe the non-bonded network configuration for Oracle Exadata Database Machine:

2.4.2.1 Non-bonded Network Configuration for Oracle Exadata Database Machine Two-Socket Systems

When using a non-bonded network configuration for Oracle Exadata Database Machine Two-Socket Systems, the database servers are configured as follows:

  • Port NET0: Provides access to the operating system using the management network through the ETH0 interface.

  • Port NET1: Provides access to the database using the client access network through the ETH1 interface. The Oracle RAC SCAN and VIP addresses are defined on this interface.

  • Port NET MGT: Provides access to the Integrated Lights Out Manager (ILOM) using the management network.

The following diagram shows the network diagram for Oracle Exadata Database Machine Two-Socket Systems when not using bonding.

Figure 2-4 Network Diagram for Oracle Exadata Database Machine Two-Socket Systems When Not Using Bonding

Description of Figure 2-4 follows
Description of "Figure 2-4 Network Diagram for Oracle Exadata Database Machine Two-Socket Systems When Not Using Bonding"

2.4.2.2 Non-bonded Network Configuration for Oracle Exadata Database Machine X5-8, X6-8, and X7-8

When using a non-bonded network configuration for Oracle Exadata Database Machine X5-8, X6-8, and X7-8, the database servers are configured as follows:

  • Port NET0: Provides access to the operating system using the management network through the ETH0 interface.

  • Port NET1: Provides access to the database using the client access network through the ETH1 interface. The Oracle RAC SCAN and VIP addresses are defined on this interface.

  • Port NET MGT: Provides access to the ILOM using the management network.

Note:

Oracle Exadata Database Machine X5-8, X6-8, and X7-8 have BONDIB0, BONDIB1, and so on for the InfiniBand private network.

2.4.2.3 Non-bonded Network Configuration for Oracle Exadata Database Machine X4-8 Full Rack

When using a non-bonded network configuration for Oracle Exadata Database Machine X4-8 Full Rack, the database servers are configured as follows:

  • Port NET0: Provides access to the operating system using the management network through the ETH0 interface.

  • Port NET1: Provides access to the database using the client access network through the ETH1 interface. The Oracle RAC SCAN and VIP addresses are defined on this interface.

  • Port NET MGT: Provides access to the ILOM using the management network.

Note:

Oracle Exadata Database Machine X4-8 Full Rack has BONDIB0, BONDIB1, and so on for the InfiniBand private network.

2.4.2.4 Non-bonded Network Configuration for Oracle Exadata Database Machine X3-8 Full Rack and Oracle Exadata Database Machine X2-8 Full Rack

When using a non-bonded network configuration for Oracle Exadata Database Machine X4-8 Full Rack, Oracle Exadata Database Machine X3-8 Full Rack and Oracle Exadata Database Machine X2-8 Full Rack, the database servers are configured as follows:

  • Port NEM0/NET0: Provides access to the operating system using the management network through the ETH0 interface. This network must use the 1 GbE port.

  • Port NEM0/NET1: Provides access to the database using the client access network through the ETH1 interface. The Oracle RAC SCAN and VIP addresses are defined on this interface.

  • Port NET MGT: Provides access to the Integrated Lights Out Manager (ILOM) using the management network.

Note:

Oracle Exadata Database Machine X4-8 Full Rack, Oracle Exadata Database Machine X3-8 Full Rack, and Oracle Exadata Database Machine X2-8 Full Rack have BONDIB0, BONDIB1, and so on for the InfiniBand private network.

2.4.3 Additional Networks

  • On Oracle Exadata Database Machine X7-2:

    • Non-bonded network configuration— There are four open ports (2x 1/10GbE copper or 2x 10/25GbE optical ports and two additional 10/25GbE PCI ports) on each database server.

    • Bonded network configuration— There are two open ports (10/25GbE PCI ports) on each database server.

  • On Oracle Exadata Database Machine two-socket systems (X2-2 to X6-2):

    • Non-bonded network configuration— There are four open ports (NET2, NET3, and two additional 10 GbE ports) on each database server

    • Bonded network configuration— There are three open ports (NET3 and two additional 10 GbE ports) on each database server

  • On Oracle Exadata Database Machine X7-8:

    • Non-bonded network configuration— There are 14 open ports (6 1/10GbE ports and 8 10/25GbE PCI ports) on each database server

    • Bonded network configuration— There are 13 open ports (5 1/10 GbE ports and 8 open 10/25 GbE PCI ports) on each database server

  • On Oracle Exadata Database Machine X5-8 and X6-8:

    • Non-bonded network configuration— There are 18 open ports (10 1GbE ports and 8 10GbE ports) on each database server

    • Bonded network configuration— There are 15 open ports (7 1 GbE ports and 8 open 10 GbE ports) on each database server

  • On Oracle Exadata Database Machine X4-8 Full Rack:

    • Non-bonded network configuration— There are 16 open ports (8 1GbE ports and 8 10GbE ports) on each database server

    • Bonded network configuration— There are 15 open ports (7 1 GbE ports and 8 open 10 GbE ports) on each database server

  • On Oracle Exadata Database Machine X3-8 Full Rack and X2-8 Full Rack:

    • Non-bonded network configuration— There are 14 open ports (NEM0/NET2, NEM0/NET3, NEM1/NET0, NEM1/NET1, NEM1/NET2, NEM1/NET3, and eight additional 10 GbE ports) on each database server

    • Bonded network configuration— There are 13 open ports (NEM0/NET2, NEM0/NET3, NEM1/NET0, NEM1/NET1, NEM1/NET2, and eight additional 10 GbE ports) on each database server

Use of these additional network ports is optional, and the configuration varies depending on your requirements and available network infrastructure. For example, these interfaces may be used for connectivity to a dedicated tape backup network or connectivity to a disaster recovery network.

2.5 Default IP Addresses

Starting with Oracle Exadata Storage Server Software release 12.1.2.1.0, the NET0 IP addresses are assigned dynamically by the elastic configuration procedure during the first start of the system. The addresses are in the 172.16.2.1 to 172.16.7.254 range. In earlier releases, Oracle Exadata Database Machine had default IP addresses set at the factory, and the range of IP addresses was 192.168.1.1 to 192.168.1.203. The subnet mask is 255.255.252.0.

Note:

Prior to connecting Oracle Exadata Database Machine to the network, ensure these IP addresses do not conflict with other addresses on the network. Use the checkip.sh script generated by Oracle Exadata Deployment Assistant (OEDA) to check for conflicts. You run the checkip.sh script on the network after the DNS entries for the Exadata Database Machine have been created, but before the Exadata Database Machine is configured or connected to the network. Oracle recommends running the script to avoid configuration delays, even if a check was performed as part of planning process before the machine was delivered.

If you ran OEDA on a Windows system, then the generated script is checkip.bat.

The following table lists the default IP addresses for Oracle Exadata Database Machine X5-2, X6-2, and X7-2. The subnet mask is 255.255.252.0.

Table 2-6 Default IP Addresses for Oracle Exadata Database Machine X5-2 and Higher

Component NET0 IP Addresses ILOM IP Addresses InfiniBand Active Bonded IP Addresses

Exadata Storage Server 14

Assigned during first start of the system

192.168.1.122

Not applicable

Exadata Storage Server 13

Assigned during first start of the system

192.168.1.121

Not applicable

Exadata Storage Server 12

Assigned during first start of the system

192.168.1.120

Not applicable

Exadata Storage Server 11

Assigned during first start of the system

192.168.1.119

Not applicable

Exadata Storage Server 10

Assigned during first start of the system

192.168.1.118

Not applicable

Exadata Storage Server 9

Assigned during first start of the system

192.168.1.117

Not applicable

Exadata Storage Server 8

Assigned during first start of the system

192.168.1.116

Not applicable

Oracle Database server 8

Assigned during first start of the system

192.168.1.115

Not applicable

Oracle Database server 7

Assigned during first start of the system

192.168.1.114

Not applicable

Oracle Database server 6

Assigned during first start of the system

192.168.1.113

Not applicable

Oracle Database server 5

Assigned during first start of the system

192.168.1.112

Not applicable

Oracle Database server 4

Assigned during first start of the system

192.168.1.111

Not applicable

Oracle Database server 3

Assigned during first start of the system

192.168.1.110

Not applicable

Oracle Database server 2

Assigned during first start of the system

192.168.1.109

Not applicable

Oracle Database server 1

Assigned during first start of the system

192.168.1.108

Not applicable

Exadata Storage Server 7

Assigned during first start of the system

192.168.1.107

Not applicable

Exadata Storage Server 6

Assigned during first start of the system

192.168.1.106

Not applicable

Exadata Storage Server 5

Assigned during first start of the system

192.168.1.105

Not applicable

Exadata Storage Server 4

Assigned during first start of the system

192.168.1.104

Not applicable

Exadata Storage Server 3

Assigned during first start of the system

192.168.1.103

Not applicable

Exadata Storage Server 2

Assigned during first start of the system

192.168.1.102

Not applicable

Exadata Storage Server 1

Assigned during first start of the system

192.168.1.101

Not applicable

Sun Datacenter InfiniBand Switch 36 switch 3

192.168.1.203

Not applicable

Not applicable

Sun Datacenter InfiniBand Switch 36 switch 2

192.168.1.202

Not applicable

Not applicable

Sun Datacenter InfiniBand Switch 36 switch 1

192.168.1.201

Not applicable

Not applicable

Ethernet switch

192.168.1.200

Not applicable

Not applicable

The following table lists the default IP addresses for Oracle Exadata Database Machine X4-2. The subnet mask is 255.255.252.0.

Table 2-7 Default IP Addresses for Oracle Exadata Database Machine X4-2

Component NET0 IP Addresses ILOM IP Addresses InfiniBand Active Bonded IP Addresses

Exadata Storage Server 14

192.168.1.22

192.168.1.122

192.168.10.44

192.168.10.43

Exadata Storage Server 13

192.168.1.21

192.168.1.121

192.168.10.42

192.168.10.41

Exadata Storage Server 12

192.168.1.20

192.168.1.120

192.168.10.40

192.168.10.39

Exadata Storage Server 11

192.168.1.19

192.168.1.119

192.168.10.38

192.168.10.37

Exadata Storage Server 10

192.168.1.18

192.168.1.118

192.168.10.36

192.168.10.35

Exadata Storage Server 9

192.168.1.17

192.168.1.117

192.168.10.34

192.168.10.33

Exadata Storage Server 8

192.168.1.16

192.168.1.116

192.168.10.32

192.168.10.31

Oracle Database server 8

192.168.1.15

192.168.1.115

192.168.10.30

192.168.10.29

Oracle Database server 7

192.168.1.14

192.168.1.114

192.168.10.28

192.168.10.27

Oracle Database server 6

192.168.1.13

192.168.1.113

192.168.10.26

192.168.10.25

Oracle Database server 5

192.168.1.12

192.168.1.112

192.168.10.24

192.168.10.23

Oracle Database server 4

192.168.1.11

192.168.1.111

192.168.10.22

192.168.10.21

Oracle Database server 3

192.168.1.10

192.168.1.110

192.168.10.20

192.168.10.19

Oracle Database server 2

192.168.1.9

192.168.1.109

192.168.10.18

192.168.10.17

Oracle Database server 1

192.168.1.8

192.168.1.108

192.168.10.16

192.168.10.15

Exadata Storage Server 7

192.168.1.7

192.168.1.107

192.168.10.14

192.168.10.13

Exadata Storage Server 6

192.168.1.6

192.168.1.106

192.168.10.12

192.168.10.11

Exadata Storage Server 5

192.168.1.5

192.168.1.105

192.168.10.10

192.168.10.9

Exadata Storage Server 4

192.168.1.4

192.168.1.104

192.168.10.8

192.168.10.7

Exadata Storage Server 3

192.168.1.3

192.168.1.103

192.168.10.6

192.168.10.5

Exadata Storage Server 2

192.168.1.2

192.168.1.102

192.168.10.4

192.168.10.3

Exadata Storage Server 1

192.168.1.1

192.168.1.101

192.168.10.2

192.168.10.1

Sun Datacenter InfiniBand Switch 36 switch 3

192.168.1.203

Not applicable

Not applicable

Sun Datacenter InfiniBand Switch 36 switch 2

192.168.1.202

Not applicable

Not applicable

Sun Datacenter InfiniBand Switch 36 switch 1

192.168.1.201

Not applicable

Not applicable

Ethernet switch

192.168.1.200

Not applicable

Not applicable

The following table lists the default IP addresses for Oracle Exadata Database Machine X3-2 and Oracle Exadata Database Machine X2-2.

Table 2-8 Default IP Addresses for Oracle Exadata Database Machine X3-2 and Oracle Exadata Database Machine X2-2

Component NET0 IP Addresses ILOM IP Addresses InfiniBand Bonded IP Addresses

Exadata Storage Server 14 (Oracle Exadata Database Machine Full Rack only)

192.168.1.22

192.168.1.122

192.168.10.22

Exadata Storage Server 13 (Oracle Exadata Database Machine Full Rack only)

192.168.1.21

192.168.1.121

192.168.10.21

Exadata Storage Server 12 (Oracle Exadata Database Machine Full Rack only)

192.168.1.20

192.168.1.120

192.168.10.20

Exadata Storage Server 11 (Oracle Exadata Database Machine Full Rack only)

192.168.1.19

192.168.1.119

192.168.10.19

Exadata Storage Server 10 (Oracle Exadata Database Machine Full Rack only)

192.168.1.18

192.168.1.118

192.168.10.18

Exadata Storage Server 9 (Oracle Exadata Database Machine Full Rack only)

192.168.1.17

192.168.1.117

192.168.10.17

Exadata Storage Server 8 (Oracle Exadata Database Machine Full Rack only)

192.168.1.16

192.168.1.116

192.168.10.16

Oracle Database server 8 (Oracle Exadata Database Machine Full Rack only)

192.168.1.15

192.168.1.115

192.168.10.15

Oracle Database server 7 (Oracle Exadata Database Machine Full Rack only)

192.168.1.14

192.168.1.114

192.168.10.14

Oracle Database server 6 (Oracle Exadata Database Machine Full Rack only)

192.168.1.13

192.168.1.113

192.168.10.13

Oracle Database server 5 (Oracle Exadata Database Machine Full Rack only)

192.168.1.12

192.168.1.112

192.168.10.12

Oracle Database server 4 (Oracle Exadata Database Machine Full Rack and Half Rack only)

192.168.1.11

192.168.1.111

192.168.10.11

Oracle Database server 3 (Oracle Exadata Database Machine Full Rack and Half Rack only)

192.168.1.10

192.168.1.110

192.168.10.10

Oracle Database server 2

192.168.1.9

192.168.1.109

192.168.10.9

Oracle Database server 1

192.168.1.8

192.168.1.108

192.168.10.8

Exadata Storage Server 7 (Oracle Exadata Database Machine Full Rack and Half Rack only)

192.168.1.7

192.168.1.107

192.168.10.7

Exadata Storage Server 6 (Oracle Exadata Database Machine Full Rack and Half Rack only)

192.168.1.6

192.168.1.106

192.168.10.6

Exadata Storage Server 5 (Oracle Exadata Database Machine Full Rack and Half Rack only)

192.168.1.5

192.168.1.105

192.168.10.5

Exadata Storage Server 4 (Oracle Exadata Database Machine Full Rack and Half Rack only)

192.168.1.4

192.168.1.104

192.168.10.4

Exadata Storage Server 3

192.168.1.3

192.168.1.103

192.168.10.3

Exadata Storage Server 2

192.168.1.2

192.168.1.102

192.168.10.2

Exadata Storage Server 1

192.168.1.1

192.168.1.101

192.168.10.1

Sun Datacenter InfiniBand Switch 36 switch 3

192.168.1.203

Not applicable

Not applicable

Sun Datacenter InfiniBand Switch 36 switch 2

192.168.1.202

Not applicable

Not applicable

Sun Datacenter InfiniBand Switch 36 switch 1 (Oracle Exadata Database Machine Full Rack and Half Rack only)

192.168.1.201

Not applicable

Not applicable

Ethernet switch

192.168.1.200

Not applicable

Not applicable

The following table lists the default IP addresses for Oracle Exadata Database Machine X5-8, X6-8, and X7-8 Full Rack. The subnet mask is 255.255.252.0.

Table 2-9 Default IP Addresses for Oracle Exadata Database Machine X5-8 and Higher Full Rack Systems

Component NET0 IP Addresses ILOM IP Addresses InfiniBand Bonded IP Addresses

Exadata Storage Server 14

192.168.1.22

192.168.1.122

192.168.10.43

192.168.10.44

Exadata Storage Server 13

192.168.1.21

192.168.1.121

192.168.10.41

192.168.10.42

Exadata Storage Server 12

192.168.1.20

192.168.1.120

192.168.10.39

192.168.10.40

Exadata Storage Server 11

192.168.1.19

192.168.1.119

192.168.10.37

192.168.10.38

Exadata Storage Server 10

192.168.1.18

192.168.1.118

192.168.10.35

192.168.10.36

Exadata Storage Server 9

192.168.1.17

192.168.1.117

192.168.10.33

192.168.10.34

Exadata Storage Server 8

192.168.1.16

192.168.1.116

192.168.10.31

192.168.10.32

Oracle Database server 2

192.168.1.9

192.168.1.109

192.168.10.29

192.168.10.30

192.168.10.27

192.168.10.28

192.168.10.25

192.168.10.26

192.168.10.23

192.168.10.24

Oracle Database server 1

192.168.1.8

192.168.1.108

192.168.10.21

192.168.10.22

192.168.10.19

192.168.10.20

192.168.10.17

192.168.10.18

192.168.10.15

192.168.10.16

Exadata Storage Server 7

192.168.1.7

192.168.1.107

192.168.10.13

192.168.10.14

Exadata Storage Server 6

192.168.1.6

192.168.1.106

192.168.10.11

192.168.10.12

Exadata Storage Server 5

192.168.1.5

192.168.1.105

192.168.10.9

192.168.10.10

Exadata Storage Server 4

192.168.1.4

192.168.1.104

192.168.10.7

192.168.10.

Exadata Storage Server 3

192.168.1.3

192.168.1.103

192.168.10.5

192.168.10.6

Exadata Storage Server 2

192.168.1.2

192.168.1.102

192.168.10.3

192.168.10.4

Exadata Storage Server 1

192.168.1.1

192.168.1.101

192.168.10.1

192.168.10.2

Sun Datacenter InfiniBand Switch 36 switch 3

192.168.1.203

Not applicable

Not applicable

Sun Datacenter InfiniBand Switch 36 switch 2

192.168.1.202

Not applicable

Not applicable

Sun Datacenter InfiniBand Switch 36 switch 1

192.168.1.201

Not applicable

Not applicable

Ethernet switch

192.168.1.200

Not applicable

Not applicable

The following table lists the default IP addresses for Oracle Exadata Database Machine X4-8 Full Rack. The subnet mask is 255.255.252.0.

Note:

For Oracle Exadata Database Machine X4-8 racks with Exadata Storage Server X5-2 Servers, the NETO IP addresses are assigned during the elastic configuration process.

Table 2-10 Default IP Addresses for Oracle Exadata Database Machine X4-8 Full Rack

Component NET0 IP Addresses ILOM IP Addresses InfiniBand Bonded IP Addresses

Exadata Storage Server 14

192.168.1.22

192.168.1.122

192.168.10.43

192.168.10.44

Exadata Storage Server 13

192.168.1.21

192.168.1.121

192.168.10.41

192.168.10.42

Exadata Storage Server 12

192.168.1.20

192.168.1.120

192.168.10.39

192.168.10.40

Exadata Storage Server 11

192.168.1.19

192.168.1.119

192.168.10.37

192.168.10.38

Exadata Storage Server 10

192.168.1.18

192.168.1.118

192.168.10.35

192.168.10.36

Exadata Storage Server 9

192.168.1.17

192.168.1.117

192.168.10.33

192.168.10.34

Exadata Storage Server 8

192.168.1.16

192.168.1.116

192.168.10.31

192.168.10.32

Oracle Database server 2

192.168.1.9

192.168.1.109

192.168.10.29

192.168.10.30

192.168.10.27

192.168.10.28

192.168.10.25

192.168.10.26

192.168.10.23

192.168.10.24

Oracle Database server 1

192.168.1.8

192.168.1.108

192.168.10.21

192.168.10.22

192.168.10.19

192.168.10.20

192.168.10.17

192.168.10.18

192.168.10.15

192.168.10.16

Exadata Storage Server 7

192.168.1.7

192.168.1.107

192.168.10.13

192.168.10.14

Exadata Storage Server 6

192.168.1.6

192.168.1.106

192.168.10.11

192.168.10.12

Exadata Storage Server 5

192.168.1.5

192.168.1.105

192.168.10.9

192.168.10.10

Exadata Storage Server 4

192.168.1.4

192.168.1.104

192.168.10.7

192.168.10.

Exadata Storage Server 3

192.168.1.3

192.168.1.103

192.168.10.5

192.168.10.6

Exadata Storage Server 2

192.168.1.2

192.168.1.102

192.168.10.3

192.168.10.4

Exadata Storage Server 1

192.168.1.1

192.168.1.101

192.168.10.1

192.168.10.2

Sun Datacenter InfiniBand Switch 36 switch 3

192.168.1.203

Not applicable

Not applicable

Sun Datacenter InfiniBand Switch 36 switch 2

192.168.1.202

Not applicable

Not applicable

Sun Datacenter InfiniBand Switch 36 switch 1

192.168.1.201

Not applicable

Not applicable

Ethernet switch

192.168.1.200

Not applicable

Not applicable

The following table lists the default IP addresses for Oracle Exadata Database Machine X3-8 Full Rack and Oracle Exadata Database Machine X2-8 Full Rack. The subnet mask is 255.255.252.0.

Table 2-11 Default IP Addresses for Oracle Exadata Database Machine X3-8 Full Rack and Oracle Exadata Database Machine X2-8 Full Rack

Component NET0 IP Addresses ILOM IP Addresses InfiniBand Bonded IP Addresses

Exadata Storage Server 14

192.168.1.22

192.168.1.122

192.168.10.22

Exadata Storage Server 13

192.168.1.21

192.168.1.121

192.168.10.21

Exadata Storage Server 12

192.168.1.20

192.168.1.120

192.168.10.20

Exadata Storage Server 11

192.168.1.19

192.168.1.119

192.168.10.19

Exadata Storage Server 10

192.168.1.18

192.168.1.118

192.168.10.18

Exadata Storage Server 9

192.168.1.17

192.168.1.117

192.168.10.17

Exadata Storage Server 8

192.168.1.16

192.168.1.116

192.168.10.16

Oracle Database server 2

192.168.1.9

192.168.1.109

192.168.10.15

192.168.10.14

192.168.10.13

192.168.10.12

Oracle Database server 1

192.168.1.8

192.168.1.108

192.168.10.11

192.168.10.10

192.168.10.9

192.168.10.8

Exadata Storage Server 7

192.168.1.7

192.168.1.107

192.168.10.7

Exadata Storage Server 6

192.168.1.6

192.168.1.106

192.168.10.6

Exadata Storage Server 5

192.168.1.5

192.168.1.105

192.168.10.5

Exadata Storage Server 4

192.168.1.4

192.168.1.104

192.168.10.4

Exadata Storage Server 3

192.168.1.3

192.168.1.103

192.168.10.3

Exadata Storage Server 2

192.168.1.2

192.168.1.102

192.168.10.2

Exadata Storage Server 1

192.168.1.1

192.168.1.101

192.168.10.1

Sun Datacenter InfiniBand Switch 36 switch 3

192.168.1.203

Not applicable

Not applicable

Sun Datacenter InfiniBand Switch 36 switch 2

192.168.1.202

Not applicable

Not applicable

Sun Datacenter InfiniBand Switch 36 switch 1

192.168.1.201

Not applicable

Not applicable

Ethernet switch

192.168.1.200

Not applicable

Not applicable

The following table lists the default IP addresses for Oracle Exadata Storage Expansion Rack X4-2.

Table 2-12 Default IP Addresses for Oracle Exadata Storage Expansion Rack X4-2

Component NET0 IP Addresses ILOM IP Addresses InfiniBand Active Bonded IP Addresses

Exadata Storage Server 18

192.168.1.68

192.168.1.168

192.168.10.86

192.168.10.85

Exadata Storage Server 17

192.168.1.67

192.168.1.167

192.168.10.84

192.168.10.83

Exadata Storage Server 16

192.168.1.66

192.168.1.166

192.168.10.82

192.168.10.81

Exadata Storage Server 15

192.168.1.65

192.168.1.165

192.168.10.80

192.168.10.79

Exadata Storage Server 14

192.168.1.64

192.168.1.164

192.168.10.78

192.168.10.77

Exadata Storage Server 13

192.168.1.63

192.168.1.163

192.168.10.76

192.168.10.75

Exadata Storage Server 12

192.168.1.62

192.168.1.162

192.168.10.74

192.168.10.73

Exadata Storage Server 11

192.168.1.61

192.168.1.161

192.168.10.72

192.168.10.71

Exadata Storage Server 10

192.168.1.60

192.168.1.160

192.168.10.70

192.168.10.69

Exadata Storage Server 9

192.168.1.59

192.168.1.159

192.168.10.68

192.168.10.67

Exadata Storage Server 8

192.168.1.58

192.168.1.158

192.168.10.66

192.168.10.65

Exadata Storage Server 7

192.168.1.57

192.168.1.157

192.168.10.64

192.168.10.63

Exadata Storage Server 6

192.168.1.56

192.168.1.156

192.168.10.62

192.168.10.61

Exadata Storage Server 5

192.168.1.55

192.168.1.155

192.168.10.60

192.168.10.59

Exadata Storage Server 4

192.168.1.54

192.168.1.154

192.168.10.58

192.168.10.57

Exadata Storage Server 3

192.168.1.53

192.168.1.153

192.168.10.56

192.168.10.55

Exadata Storage Server 2

192.168.1.52

192.168.1.152

192.168.10.54

192.168.10.53

Exadata Storage Server 1

192.168.1.51

192.168.1.151

192.168.10.52

192.168.10.51

Sun Datacenter InfiniBand Switch 36 switch 3

192.168.1.223

Not applicable

Not applicable

Sun Datacenter InfiniBand Switch 36 switch 2

192.168.1.222

Not applicable

Not applicable

Sun Datacenter InfiniBand Switch 36 switch 1 (Oracle Exadata Database Machine Full Rack and Half Rack only)

192.168.1.221

Not applicable

Not applicable

Ethernet switch

192.168.1.220

Not applicable

Not applicable

PDU-A

192.168.1.212

Not applicable

Not applicable

PDU-B

192.168.1.213

Not applicable

Not applicable

The following table lists the default IP addresses for Oracle Exadata Storage Expansion Rack X3-2, and Oracle Exadata Storage Expansion Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers.

Table 2-13 Default IP Addresses for Oracle Exadata Storage Expansion Rack X3-2, and Oracle Exadata Storage Expansion Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers

Component NET0 IP Addresses ILOM IP Addresses InfiniBand Bonded IP Addresses

Exadata Storage Server 18

192.168.1.68

192.168.1.168

192.168.10.68

Exadata Storage Server 17

192.168.1.67

192.168.1.167

192.168.10.67

Exadata Storage Server 16

192.168.1.66

192.168.1.166

192.168.10.66

Exadata Storage Server 15

192.168.1.65

192.168.1.165

192.168.10.65

Exadata Storage Server 14

192.168.1.64

192.168.1.164

192.168.10.64

Exadata Storage Server 13

192.168.1.63

192.168.1.163

192.168.10.63

Exadata Storage Server 12

192.168.1.62

192.168.1.162

192.168.10.62

Exadata Storage Server 11

192.168.1.61

192.168.1.161

192.168.10.61

Exadata Storage Server 10

192.168.1.60

192.168.1.160

192.168.10.60

Exadata Storage Server 9

192.168.1.59

192.168.1.159

192.168.10.59

Exadata Storage Server 8

192.168.1.58

192.168.1.158

192.168.10.58

Exadata Storage Server 7

192.168.1.57

192.168.1.157

192.168.10.57

Exadata Storage Server 6

192.168.1.56

192.168.1.156

192.168.10.56

Exadata Storage Server 5

192.168.1.55

192.168.1.155

192.168.10.55

Exadata Storage Server 4

192.168.1.54

192.168.1.154

192.168.10.54

Exadata Storage Server 3

192.168.1.53

192.168.1.153

192.168.10.53

Exadata Storage Server 2

192.168.1.52

192.168.1.152

192.168.10.52

Exadata Storage Server 1

192.168.1.51

192.168.1.151

192.168.10.51

Sun Datacenter InfiniBand Switch 36 switch 3

192.168.1.223

Not applicable

Not applicable

Sun Datacenter InfiniBand Switch 36 switch 2

192.168.1.222

Not applicable

Not applicable

Sun Datacenter InfiniBand Switch 36 switch 1

192.168.1.221

Not applicable

Not applicable

Ethernet switch

192.168.1.220

Not applicable

Not applicable

PDU-A

192.168.1.212

Not applicable

Not applicable

PDU-B

192.168.1.213

Not applicable

Not applicable

2.6 Port Assignments When Using a Firewall

The following table lists the ports used by services on Oracle Exadata Database Machine. When network communication requires access through a firewall, review the list and open the necessary ports. All ports are on the management network, except for port 1521 which is on the client network.

Table 2-14 Open Ports for Firewall

Source Target Protocol Port Network Application

Any

Database servers, storage cells, and ILOMs for database servers, Exadata Storage Servers, and InfiniBand switches

SSH over TCP

22

Management

SSH

Any

KVM

SSH over TCP

22

Management

SSH for serial sessions to MPUIQ-SRL module

Exadata Storage Servers

SMTP e-mail server

SMTP

25

Management

SMTP (Simple Mail Transfer Protocol)

Any

ILOMs for database servers, Exadata Storage Servers, and InfiniBand switches

HTTP

80

Management

Web (user configurable)

Any

PDU

HTTP

80

Management

Browser interface

Any

KVM

HTTP

80

Management

Avocent video viewer download for Java applet

Any

rpcbind

TCP

111

Management

rpcbind

Database management

NTP servers

NTP over UDP

123

Client

Outgoing Network Time Protocol (NTP)

Database servers, Exadata Storage Servers, ILOMs, InfiniBand switches, and Cisco switch

NTP servers

NTP over UDP

123

Management

Outgoing NTP

Any

ILOMs for database servers, Exadata Storage Servers, and InfiniBand switches

SNMP over UDP

161

Management

SNMP (Simple Network Management Protocol) (user configurable)

Any

PDU

SNMP over UDP

161

Management

SNMP (user configurable)

Any

KVM

SNMP over UDP

161

Management

SNMP (user configurable)

Exadata Storage Servers

SNMP subscriber such as Oracle Enterprise Manager Grid Control or an SNMP manager

SNMP

162

Management

SNMP version 1 (SNMPv1) outgoing traps (user-configurable)

Database servers, Exadata Storage Servers, and ILOMs for database servers, Exadata Storage Servers, and InfiniBand switches

ASR Manager

SNMP

162

Management

Telemetry messages sent to ASR Manager

ILOMs for database servers, Exadata Storage Servers, and InfiniBand switches

Any

IPMI over UDP

162

Management

Outgoing IPMI (Intelligent Platform Management Interface) Platform Event Trap (PET)

PDU

SNMP trap receivers

SNMP over UDP

162

Management

Outgoing SNMPv2 traps

KVM

SNMP trap receivers

SNMP over UDP

162

Management

Outgoing SNMPv2 traps

ASR Manager

ASR backend

HTTPS

443

Management

Telemetry messages sent to ASR backend

Any

ILOMs for database servers, Exadata Storage Servers, and InfiniBand switches

HTTPS

443

Management

Web (user configurable)

Any

PDU

HTTPS

443

Management

Browser interface

Any

KVM

HTTPS

443

Management

Browser interface for MergePoint Utility switch and KVM sessions

Exadata Storage Servers

SMTPS client

SMTPS

465

Management

Simple Mail Transfer Protocol, Secure (if configured)

Database servers, Exadata Storage Servers, and ILOMs for database servers, Exadata Storage Servers, and InfiniBand switches

Syslog server

Syslog over UDP

514

Management

Outgoing Syslog

PDU

Syslog server

Syslog over UDP

514

Management

Outgoing Syslog

KVM

Syslog server

Syslog over UDP

514

Management

Outgoing Syslog

Any

ILOMs for database servers, Exadata Storage Servers, and InfiniBand switches

IPMI over UDP

623

Management

IPMI

Any

plathwsvcd

TCP

723

Management

 

Any

evnd

TCP

791

Management

 

Any

partitiond

TCP

867

Management

 

Any

Database servers

TCP

1521

Client

Database listener

Any

KVM

TCP

2068

Management

KVM session data for keyboard and mouse transmission, or for video transmission

Any

tgtd

TCP

3260

Management

SCSI target daemon

Any

 

TCP

3872

Management

Java EM agent

Any

Cisco Smart Install

TCP

4786

Management

Cisco Smart Install

Any

ILOMs for database servers, and Exadata Storage Servers

TCP

5120

Management

ILOM remote console: CD

Any

ILOMs for database servers, and Exadata Storage Servers

TCP

5121

Management

ILOM remote console: keyboard and mouse

Any

ILOMs for database servers, and Exadata Storage Servers

TCP

5123

Management

ILOM remote console: diskette

Any

ILOMs for database servers, and Exadata Storage Servers

TCP

5555

Management

ILOM remote console: encryption

Any

ILOMs for database servers, and Exadata Storage Servers

TCP

5556

Management

ILOM remote console: authentication

Any

Oracle Notification Service

TCP

6200

Management

Oracle Notification Service

ASR Manager

ILOMs for database servers, and Exadata Storage Servers

HTTP

6481

Management

Service tag listener for asset activation

Any

ILOMs for database servers, and Exadata Storage Servers

TCP

6481

Management

ILOM remote console: Servicetag daemon

Any

ILOMs for database servers, and Exadata Storage Servers

TCP

7578

Management

ILOM remote console: video

Any

ILOMs for database servers, and Exadata Storage Servers

TCP

7579

Management

ILOM remote console: serial

Any

Database servers and Exadata Storage Servers

TCP

7777

Both

Oracle Enterprise Manager Grid Control HTTP console port

Any

Database servers and Exadata Storage Servers

TCP

7799

Both

Oracle Enterprise Manager Grid Control HTTPS console port

Any

Oracle WebLogic

TCP

7878

8888

Management

Java WebLogic

Any

rpc.statd

TCP

21408

40801

41460

47431

Management

rpc.statd

Any

gpnpd.bin

TCP

19193 (IPv6)

25194

Management

gpnpd.bin

Any

Java agent (yuehou)

TCP

26222 (IPv6)

Management

Java agent (yuehou)

Any

Java agent (sveerana)

TCP

42034 (IPv6)

Management

Java agent (sveerana)

Any

Java EM Agent

TCP

47632

Management

 

See Also:

Appendix E, "Managing Oracle Database Port Numbers", in the Oracle Real Application Clusters Installation Guide for Linux and UNIX.