6 Connecting Exalogic and Exadata Machines

This chapter describes how to connect Exalogic machine racks and Oracle Exadata Database Machine racks together on the same InfiniBand fabric.

It includes the following topics:

Section 6.1, "Before You Begin"
Section 6.2, "Connecting an Exalogic Machine Full Rack and an Oracle Exadata Database Machine Full Rack Together"
Section 6.3, "Connecting an Exalogic Machine Quarter Rack and an Oracle Exadata Database Machine Quarter Rack Together"

6.1 Before You Begin

Before you start connecting Exalogic machines and Oracle Exadata Database Machines together, ensure that you have completed all of the multirack cabling tasks, as described in Section 1.7, "Multirack Cabling Tasks".

6.2 Connecting an Exalogic Machine Full Rack and an Oracle Exadata Database Machine Full Rack Together

This section contains the following topics:

Connectivity Diagram
Procedure

6.2.1 Connectivity Diagram

Figure 6-1 shows how an Exalogic machine full rack is connected to an Oracle Exadata Database machine full rack.

Figure 6-1 Connectivity Between Exalogic Machine Full Rack and Oracle Exadata Database Machine Full Rack in a Multirack Scenario

Description of "Figure 6-1 Connectivity Between Exalogic Machine Full Rack and Oracle Exadata Database Machine Full Rack in a Multirack Scenario"

Each Sun Network QDR InfiniBand Gateway Switch (NM2-GW) in the Exalogic machine full rack is connected to its local spine switch (Sun Datacenter InfiniBand Switch 36 (NM2-36P). See the red connecting lines in Figure 6-1 (EL Rack #1).

A spine switch is included in the Exalogic machine full rack configuration, but it is not connected, by default. Then each leaf switch in the Oracle Exadata Database Machine full rack is connected to its local spine switch. See the red connecting lines in Figure 6-1 (ED X2-2 Rack #2).

Subsequently, each leaf switch in the Exalogic machine full rack is connected to the spine switch of the Oracle Exadata Database Machine. See the blue connecting lines in Figure 6-1.

Finally, each leaf switch in the Oracle Exadata Database Machine is connected to the spine switch of the Exalogic machine. See the green connecting lines in Figure 6-1.

6.2.2 Procedure

Connecting an Exalogic machine full rack to an Oracle Exadata Database Machine full rack involves the following steps:

Remove Factory-Installed Cables (Cross-Connects) in Exalogic Machine Full Rack
Remove Factory-Installed Cables (Cross-Connects) in Oracle Exadata Database Machine Full Rack
Connect Each Leaf Switch in Exalogic Machine Full Rack to Its Spine Switch
Connect Each Leaf Switch in Oracle Exadata Database Machine Full Rack to the Spine Switch in Oracle Exadata DataBase Machine Rack
Connect Exalogic's Leaf Switches to Spine Switch of Oracle Exadata Database Machine
Connect Leaf Switches of Oracle Exadata Database Machine to Spine Switch of Exalogic Machine

6.2.2.1 Remove Factory-Installed Cables (Cross-Connects) in Exalogic Machine Full Rack

Table 6-1 lists the factory-installed Exalogic machine cross-connects (cables) that should be removed.

The system naming conventions in Table 6-1 are as follows. "R" indicates an Exalogic machine rack. For example, R1 indicates rack 1. "U" indicates the unit height in the rack. For example, U24 indicates rack unit 24. U1 is located at the bottom of the rack, and U42 is at the top of the rack. For more information about Exalogic machine rack layout, see Section 1.5, "Exalogic Machine Rack Layout".

Table 6-1 Factory-Installed Cross-Connects in Exalogic Machine to be Removed

Cable Coming from Slot (Exalogic Machine)	Port	Connecting to Slot (Exalogic Machine)	Port	Length
R1-U16	0A	R1-U26	0A	2 meter
R1-U16	0B	R1-U26	0B	2 meter
R1-U16	1A	R1-U26	1A	2 meter
R1-U16	1B	R1-U26	1B	2 meter
R1-U16	2A	R1-U24	2A	2 meter
R1-U16	2B	R1-U24	2B	2 meter
R1-U16	3A	R1-U24	3A	2 meter
R1-U16	3B	R1-U24	3B	2 meter
R1-U23	0A	R1-U24	0A	1 meter
R1-U23	0B	R1-U24	0B	1 meter
R1-U23	1A	R1-U24	1A	1 meter
R1-U23	1B	R1-U24	1B	1 meter
R1-U23	2A	R1-U26	2A	1 meter
R1-U23	2B	R1-U26	2B	1 meter
R1-U23	3A	R1-U26	3A	1 meter
R1-U23	3B	R1-U26	3B	1 meter

Note:

R1 in the above table indicates an Exalogic machine full rack that is being connected to an Oracle Exadata Database Machine full rack

6.2.2.2 Remove Factory-Installed Cables (Cross-Connects) in Oracle Exadata Database Machine Full Rack

Table 6-2 lists the factory-installed Oracle Exadata Database Machine cross-connects (cables) that should be removed.

The system naming conventions in Table 6-2 are as follows. "R" indicates an Oracle Exadata Database Machine rack. For example, R1 indicates rack 1. "U" indicates the unit height in the rack. For example, U24 indicates rack unit 24. U1 is located at the bottom of the rack, and U42 is at the top of the rack. For more information about Oracle Exadata Database machine rack layout, see the Oracle Exadata Database Machine Owner's Guide.

Table 6-2 Factory-Installed Cross-Connects in Oracle Exadata Database Machine to be Removed

Cable Coming from Slot (Oracle Exadata Database Machine)	Port	Connecting to Slot (Oracle Exadata Database Machine)	Port	Length
R2-U20	8A	R2-U24	8A	2 meter
R2-U20	9A	R2-U24	9B	2 meter
R2-U20	9B	R2-U24	9A	2 meter
R2-U20	10A	R2-U24	10B	2 meter
R2-U20	10B	R2-U24	10A	2 meter
R2-U20	11A	R2-U24	11B	2 meter
R2-U20	11B	R2-U24	11A	2 meter
R2-U24	8B	R2-U1	0B	3 meter
R2-U20	8B	R2-U1	1B	3 meter

Note:

R2 in the above table indicates an Oracle Exadata Database Machine full rack that is being connected to an Exalogic machine full rack (R1).

6.2.2.3 Connect Each Leaf Switch in Exalogic Machine Full Rack to Its Spine Switch

Table 6-3 lists the 3-meter cables that should be used to connect leaf switches (NM2-GW) to the spine switch (NM2-36P) in the Exalogic machine rack. The 3-meter cables are bundled with the Exalogic machine. Only Exalogic machine full rack and half rack configurations include a spine switch in U1. The spine switch is not connected, by default.

For more information about system naming conventions (Unit numbers in the Exalogic machine), see the Exalogic machine rack layout in Section 1.5, "Exalogic Machine Rack Layout".

Table 6-3 Connect 3-Meter Cables from Each Leaf Switch to the Spine Switch in Exalogic Machine Full Rack

Cable Coming from Slot (Exalogic Machine)	Port	Connecting to Slot (Exalogic Machine)	Port	Length
R1-U26	0A	R1-U1	3A	3 meter
R1-U26	0B	R1-U1	3B	3 meter
R1-U26	1A	R1-U1	4A	3 meter
R1-U26	1B	R1-U1	4B	3 meter
R1-U24	0A	R1-U1	5A	3 meter
R1-U24	0B	R1-U1	5B	3 meter
R1-U24	1A	R1-U1	6A	3 meter
R1-U24	1B	R1-U1	6B	3 meter
R1-U23	0A	R1-U1	7A	3 meter
R1-U23	0B	R1-U1	7B	3 meter
R1-U23	1A	R1-U1	8A	3 meter
R1-U23	1B	R1-U1	8B	3 meter
R1-U16	0A	R1-U1	9A	3 meter
R1-U16	0B	R1-U1	9B	3 meter
R1-U16	1A	R1-U1	10A	3 meter
R1-U16	1B	R1-U1	10B	3 meter

Note:

R1 in the above table indicates an Exalogic machine full rack that is being connected to an Oracle Exadata Database Machine full rack.

6.2.2.4 Connect Each Leaf Switch in Oracle Exadata Database Machine Full Rack to the Spine Switch in Oracle Exadata DataBase Machine Rack

Table 6-4 lists the 3-meter cables that should be used to connect leaf switches in the Oracle Exadata Database Machine to the spine switch (NM2-36P) in the Oracle Exadata Database Machine full rack. The 3-meter cables are bundled with the Oracle Exadata Database Machine.

Table 6-4 Connect 3-Meter Cables from Each Leaf Switch in Oracle Exadata Database Machine to Spine Switch in Oracle Exadata Database Machine

Cable Coming from Slot (Oracle Exadata Database Machine)	Port	Connecting to Slot (Oracle Exadata Database Machine)	Port	Length
R2-U20	8A	R2-U1	11B	3 meter
R2-U20	8B	R2-U1	12B	3 meter
R2-U20	9A	R2-U1	13B	3 meter
R2-U20	9B	R2-U1	14B	3 meter
R2-U24	8A	R2-U1	11A	3 meter
R2-U24	8B	R2-U1	12A	3 meter
R2-U24	9A	R2-U1	13A	3 meter
R2-U24	9B	R2-U1	14A	3 meter

Note:

R2 in the above table indicates an Oracle Exadata Database Machine full rack that is being connected to an Exalogic machine full rack (R1).

6.2.2.5 Connect Exalogic's Leaf Switches to Spine Switch of Oracle Exadata Database Machine

Table 6-5 lists the 5-meter cables that should be used to connect leaf switches (NM2-GW) to the spine switch (NM2-36P) in the Exalogic machine rack. The 3-meter cables are bundled with the Exalogic machine. Only Exalogic machine full rack and half rack configurations include a spine switch in U1. The spine switch is not connected, by default.

For more information about system naming conventions (Unit numbers in the Exalogic machine), see the Exalogic machine rack layout in Section 1.5, "Exalogic Machine Rack Layout". For more information about system naming conventions related to Oracle Exadata Database Machine (Unit numbers in the Oracle Exadata Database Machine), see the Oracle Exadata Database Machine rack layout described in your Oracle Exadata Database Machine documentation.

Table 6-5 Connect 5-Meter Cables from Each Leaf Switch to the Spine Switch in Exalogic Machine Full Rack

Cable Coming from Slot (Exalogic Machine)	Port	Connecting to Slot (Oracle Exadata Database Machine)	Port	Length
R1-U26	2A	R2-U1	3A	5 meter
R1-U26	2B	R2-U1	3B	5 meter
R1-U26	3A	R2-U1	4A	5 meter
R1-U26	3B	R2-U1	4B	5 meter
R1-U24	2A	R2-U1	5A	5 meter
R1-U24	2B	R2-U1	5B	5 meter
R1-U24	3A	R2-U1	6A	5 meter
R1-U24	3B	R2-U1	6B	5 meter
R1-U23	2A	R2-U1	7A	5 meter
R1-U23	2B	R2-U1	7B	5 meter
R1-U23	3A	R2-U1	8A	5 meter
R1-U23	3B	R2-U1	8B	5 meter
R1-U16	2A	R2-U1	9A	5 meter
R1-U16	2B	R2-U1	9B	5 meter
R1-U16	3A	R2-U1	10A	5 meter
R1-U16	3B	R2-U1	10B	5 meter

Note:

R1 in the above table indicates an Exalogic machine full rack that is being connected to an Oracle Exadata Database Machine full rack (R2).

6.2.2.6 Connect Leaf Switches of Oracle Exadata Database Machine to Spine Switch of Exalogic Machine

Table 6-6 lists the 5-meter cables that should be used to connect leaf switches of Oracle Exadata Database Machine to the spine switch (NM2-36P) of the Exalogic machine full rack. The 5-meter cables are bundled with the Oracle Exadata Database Machine.

For more information about system naming conventions (Unit numbers in the Oracle Exadata Database Machine), see the Oracle Exadata Database Machine rack layout described in your Oracle Exadata Database Machine documentation. For more information about system naming conventions (Unit numbers in the Exalogic machine), see the Exalogic machine rack layout in Section 1.5, "Exalogic Machine Rack Layout".

Table 6-6 Connect 5-Meter Cables from Each Leaf Switch of Oracle Exadata Database Machine to the Spine Switch of Exalogic Machine

Cable Coming from Slot (Oracle Exadata Database Machine)	Port	Connecting to Slot (Exalogic Machine)	Port	Length
R2-U20	10A	R1-U1	11B	5 meter
R2-U20	10B	R1-U1	12B	5 meter
R2-U20	11A	R1-U1	13B	5 meter
R2-U20	11B	R1-U1	14B	5 meter
R2-U24	10A	R1-U1	11A	5 meter
R2-U24	10B	R1-U1	12A	5 meter
R2-U24	11A	R1-U1	13A	5 meter
R2-U24	11B	R1-U1	14A	5 meter

Note:

R2 in the above table indicates an Oracle Exadata Database Machine full rack that is being connected to an Exalogic machine full rack (R1).

6.3 Connecting an Exalogic Machine Quarter Rack and an Oracle Exadata Database Machine Quarter Rack Together

In this multirack configuration scenario, all connections with both racks will remain as they are when delivered to the customer site. No connections will be moved or removed to make the cross connections. In addition, a spine switch in either rack is not required.

All cross-connections between the Exalogic machine quarter rack and the Oracle Exadata Database Machine quarter rack are made using the four Infiniband switch ports 1[a,b] and 2[a,b]. Table 6-7 describes the connections between the leaf switches of the two racks being connected together.

Table 6-7 Cross-Connections Between Exalogic Machine Quarter Rack and Oracle Exadata Database Machine Quarter Rack

Oracle Exadata Database Machine		Exalogic Machine
Leaf Switch	Port	Leaf Switch	Port
R2-U24	1A	R1-U23	1A
R2-U24	1B	R1-U23	1B
R2-U24	2A	R1-U16	2A
R2-U24	2B	R1-U16	2B
R2-U20	1A	R1-U16	1A
R2-U20	1B	R1-U16	1B
R2-U20	2A	R1-U23	2A
R2-U20	2B	R1-U23	2B

Note:

R2 in the above table indicates an Oracle Exadata Database Machine full rack that is being connected to an Oracle Exadata Database Machine full rack (R1).

In the above table, ports 1A and 1B are horizontal connections across the two machine racks being connected (top leaf switch to top leaf switch). Ports 2A and 2B are the cross connections between the racks (top leaf switch to bottom leaf switch).

You require only 8 cables to connect an Exalogic machine quarter rack and an Oracle Exadata Database Machine together. If these systems are side by side, you can use the 5-meter Infiniband cables included in the spares kit to complete these connections.