Skip Headers
Oracle® Exalogic Elastic Cloud Multirack Cabling Guide
Release EL X2-2, X3-2, and X4-2

E18481-08
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
PDF · Mobi · ePub

5 Connecting Two Exalogic Machines

This chapter describes how to connect two Exalogic machines on the same InfiniBand fabric.

It includes the following topics:

5.1 Before You Begin

Before you start connecting multiple Exalogic machines together, ensure that you have completed all of the multirack cabling tasks described in Section 1.4, "Multirack Cabling Tasks".

5.2 Connecting Two Exalogic Machine Full Racks Together

This section discusses how to connect two Exalogic machine full racks together. It contains the following topics:

5.2.1 Connectivity Diagram

Figure 5-1 shows how an Exalogic machine full rack is connected to another Exalogic machine full rack.

Figure 5-1 Connectivity Between Two Exalogic Machine Full Racks in a Multirack Scenario

Description of Figure 5-1 follows
Description of "Figure 5-1 Connectivity Between Two Exalogic Machine Full Racks in a Multirack Scenario"

Sun Network QDR InfiniBand Gateway Switch (NM2-GW) 4x4 (four switches in each Exalogic machine full rack) cross-connects are recabled to their respective local spine switches (Sun Datacenter InfiniBand Switch 36 (NM2-36P). See the red connecting lines in Figure 5-1. A spine switch is included in the Exalogic machine full rack configuration, but it is not connected by default.

In addition, each leaf switch is connected to the other rack's spine switch. For example, leaf switches in rack 1 are connected to the spine switch in rack 2. See the blue and green connecting lines in Figure 5-1.

5.2.2 Procedure

Connecting an Exalogic machine full rack to another Exalogic machine full rack involves the following steps:

  1. Remove Factory-Installed Cables

  2. Remove Cables Between Gateway Switches

  3. Connect Each Leaf Switch to the Spine Switch

  4. Connect Each Leaf Switch in One Exalogic Rack to Spine Switches in the Other Exalogic Rack

5.2.2.1 Remove Factory-Installed Cables

Table 5-1 lists the factory-installed Exalogic machine cross-connects (cables) that should be removed. Remove these cables in each of the Exalogic machine full racks that will be connected on the same InfiniBand fabric.

The system naming conventions are as follows. "R" indicates an Exalogic machine rack or an Exadata Database Machine rack. For example, R1 indicates rack 1. "U" indicates the unit height in the rack. For example, U24 indicates rack unit 24. U1 is located at the bottom of the rack, and U42 is at the top of the rack. For more information about Exalogic machine rack layout, see the Oracle Exalogic Elastic Cloud Machine Owner's Guide.

Table 5-1 Remove Factory-Installed Cross-Connects in Exalogic Machines

Cable Coming from Slot Port Connecting to Slot Port Length

U16

0A

U26

0A

2 meter

U16

0B

U26

0B

2 meter

U16

1A

U26

1A

2 meter

U16

1B

U26

1B

2 meter

U16

2A

U24

2A

2 meter

U16

2B

U24

2B

2 meter

U16

3A

U24

3A

2 meter

U16

3B

U24

3B

2 meter

U23

0A

U24

0A

1 meter

U23

0B

U24

0B

1 meter

U23

1A

U24

1A

1 meter

U23

1B

U24

1B

1 meter

U23

2A

U26

2A

1 meter

U23

2B

U26

2B

1 meter

U23

3A

U26

3A

1 meter

U23

3B

U26

3B

1 meter


Note:

Remove the factory-installed cables as shown in Table 5-1 from both Exalogic machines.

5.2.2.2 Remove Cables Between Gateway Switches

You must remove the cables connected in ports 4A, 4B, 5A, 5B, 6A, and 6B between gateway switches in both the racks.

5.2.2.3 Connect Each Leaf Switch to the Spine Switch

Table 5-2 lists the 3-meter cables that should be used to connect leaf switches (NM2-GW) to the spine switch (NM2-36P) in the Exalogic machine rack. The 3-meter cables are bundled with the Exalogic machine. Only Exalogic machine full rack and half rack configurations include a spine switch in U1. The spine switch is not connected by default. You should connect leaf switches to the spine switches in each of the Exalogic machine full racks that will be connected.

Table 5-2 Connect 3-Meter Cables from Each Leaf Switch to the Spine Switch

Cable Coming from Slot Port Connecting to Slot Port Length

U26

0A

U1

1A

3 meter

U26

0B

U1

1B

3 meter

U26

1A

U1

2A

3 meter

U26

1B

U1

2B

3 meter

U24

0A

U1

3A

3 meter

U24

0B

U1

3B

3 meter

U24

1A

U1

4A

3 meter

U24

1B

U1

4B

3 meter

U23

0A

U1

5A

3 meter

U23

0B

U1

5B

3 meter

U23

1A

U1

6A

3 meter

U23

1B

U1

6B

3 meter

U16

0A

U1

7A

3 meter

U16

0B

U1

7B

3 meter

U16

1A

U1

8A

3 meter

U16

1B

U1

8B

3 meter


Note:

Make the connections between leaf switches and spine switch as shown in Table 5-2 for both Exalogic machines.

5.2.2.4 Connect Each Leaf Switch in One Exalogic Rack to Spine Switches in the Other Exalogic Rack

Table 5-3 lists the 5-meter cables (or longer) that should be used to connect leaf switches (NM2-GW) from each Exalogic machine full rack to the spine switches (NM2-36P) in the other Exalogic machine full rack. The 5-meter cables are bundled with the Exalogic machine.

Table 5-3 Connect 5-Meter Cable One Rack's Leaf Switches to the Second Rack's Spine Switch

Cable Coming from Slot Port Connecting to Slot Port Length

R1-U26

2A

R2-U1

9A

5 meter

R1-U26

2B

R2-U1

9B

5 meter

R1-U26

3A

R2-U1

10A

5 meter

R1-U26

3B

R2-U1

10B

5 meter

R1-U24

2A

R2-U1

11A

5 meter

R1-U24

2B

R2-U1

11B

5 meter

R1-U24

3A

R2-U1

12A

5 meter

R1-U24

3B

R2-U1

12B

5 meter

R1-U23

2A

R2-U1

13A

5 meter

R1-U23

2B

R2-U1

13B

5 meter

R1-U23

3A

R2-U1

14A

5 meter

R1-U23

3B

R2-U1

14B

5 meter

R1-U16

2A

R2-U1

15A

5 meter

R1-U16

2B

R2-U1

15B

5 meter

R1-U16

3A

R2-U1

16A

5 meter

R1-U16

3B

R2-U1

16B

5 meter


Note:

Connect leaf switches in each Exalogic machine full rack to the spine switch in the other Exalogic machine full rack, as shown in Table 5-3.

For network connectivity diagrams of other Exalogic-to-Exalogic scenarios, see Appendix A.