5.2 Connecting Two Exalogic Machine Full Racks Together
This section discusses how to connect two Exalogic machine full racks together. It contains the following topics:
5.2.1 Connectivity Diagram
Figure 5-1 shows how an Exalogic machine full rack is connected to another Exalogic machine full rack.
Figure 5-1 Connectivity Between Two Exalogic Machine Full Racks in a Multirack Scenario
Description of "Figure 5-1 Connectivity Between Two Exalogic Machine Full Racks in a Multirack Scenario"
Sun Network QDR InfiniBand Gateway Switch (NM2-GW) 4x4 (four switches in each Exalogic machine full rack) cross-connects are recabled to their respective local spine switches Sun Datacenter InfiniBand Switch 36 (NM2-36P). See the red connecting lines in Figure 5-1.
In addition, each leaf switch is connected to the other rack's spine switch. For example, leaf switches in rack 1 are connected to the spine switch in rack 2. See the blue and green connecting lines in Figure 5-1.
5.2.2 Connect Two Exalogic Machine Full Racks Together
Connecting an Exalogic machine full rack to another Exalogic machine full rack involves the following steps:
5.2.2.1 Remove the Factory-Installed Cables
Table 5-1 lists the factory-installed Exalogic machine cross-connects (cables) that should be removed. Remove these cables in each of the Exalogic machine full racks that will be connected on the same InfiniBand fabric.
The system naming conventions are as follows. "R" indicates an Exalogic machine rack or an Exadata Database Machine rack. For example, R1 indicates rack 1. "U" indicates the unit height in the rack. For example, U24 indicates rack unit 24. U1 is located at the bottom of the rack, and U42 is at the top of the rack. For more information about Exalogic machine rack layout, see the Oracle Exalogic Elastic Cloud Machine Owner's Guide.
Table 5-1 Remove Factory-Installed Cross-Connects in Exalogic Machines
| Cable Coming from Slot | Port | Connecting to Slot | Port | Length |
|---|---|---|---|---|
|
U16 |
0A |
U26 |
0A |
2 meter |
|
U16 |
0B |
U26 |
0B |
2 meter |
|
U16 |
1A |
U26 |
1A |
2 meter |
|
U16 |
1B |
U26 |
1B |
2 meter |
|
U16 |
2A |
U24 |
2A |
2 meter |
|
U16 |
2B |
U24 |
2B |
2 meter |
|
U16 |
3A |
U24 |
3A |
2 meter |
|
U16 |
3B |
U24 |
3B |
2 meter |
|
U23 |
0A |
U24 |
0A |
1 meter |
|
U23 |
0B |
U24 |
0B |
1 meter |
|
U23 |
1A |
U24 |
1A |
1 meter |
|
U23 |
1B |
U24 |
1B |
1 meter |
|
U23 |
2A |
U26 |
2A |
1 meter |
|
U23 |
2B |
U26 |
2B |
1 meter |
|
U23 |
3A |
U26 |
3A |
1 meter |
|
U23 |
3B |
U26 |
3B |
1 meter |
Note:
Remove the factory-installed cables as shown in Table 5-1 from both Exalogic machines.
5.2.2.2 Remove Cables Between Gateway Switches
You must remove the cables connected in ports 4A, 4B, 5A, 5B, 6A, and 6B between gateway switches in both the racks.
5.2.2.3 Connect Each Leaf Switch to the Spine Switch
Table 5-2 lists the 3-meter cables that should be used to connect leaf switches (NM2-GW) to the spine switch (NM2-36P) in the Exalogic machine rack. The 3-meter cables are bundled with the Exalogic machine. You should connect leaf switches to the spine switches in each of the Exalogic machine full racks that will be connected.
Table 5-2 Connect 3-Meter Cables from Each Leaf Switch to the Spine Switch
| Cable Coming from Slot | Port | Connecting to Slot | Port | Length |
|---|---|---|---|---|
|
U26 |
0A |
U1 |
1A |
3 meter |
|
U26 |
0B |
U1 |
1B |
3 meter |
|
U26 |
1A |
U1 |
2A |
3 meter |
|
U26 |
1B |
U1 |
2B |
3 meter |
|
U24 |
0A |
U1 |
3A |
3 meter |
|
U24 |
0B |
U1 |
3B |
3 meter |
|
U24 |
1A |
U1 |
4A |
3 meter |
|
U24 |
1B |
U1 |
4B |
3 meter |
|
U23 |
0A |
U1 |
5A |
3 meter |
|
U23 |
0B |
U1 |
5B |
3 meter |
|
U23 |
1A |
U1 |
6A |
3 meter |
|
U23 |
1B |
U1 |
6B |
3 meter |
|
U16 |
0A |
U1 |
7A |
3 meter |
|
U16 |
0B |
U1 |
7B |
3 meter |
|
U16 |
1A |
U1 |
8A |
3 meter |
|
U16 |
1B |
U1 |
8B |
3 meter |
Note:
Make the connections between leaf switches and spine switch as shown in Table 5-2 for both Exalogic machines.
5.2.2.4 Connect Each Leaf Switch in One Exalogic Rack to Spine Switches in the Other Exalogic Rack
Table 5-3 lists the 5-meter cables (or longer) that should be used to connect leaf switches (NM2-GW) from each Exalogic machine full rack to the spine switches (NM2-36P) in the other Exalogic machine full rack. The 5-meter cables are bundled with the Exalogic machine.
Table 5-3 Connect 5-Meter Cable One Rack's Leaf Switches to the Second Rack's Spine Switch
| Cable Coming from Slot | Port | Connecting to Slot | Port | Length |
|---|---|---|---|---|
|
R1-U26 |
2A |
R2-U1 |
9A |
5 meter |
|
R1-U26 |
2B |
R2-U1 |
9B |
5 meter |
|
R1-U26 |
3A |
R2-U1 |
10A |
5 meter |
|
R1-U26 |
3B |
R2-U1 |
10B |
5 meter |
|
R1-U24 |
2A |
R2-U1 |
11A |
5 meter |
|
R1-U24 |
2B |
R2-U1 |
11B |
5 meter |
|
R1-U24 |
3A |
R2-U1 |
12A |
5 meter |
|
R1-U24 |
3B |
R2-U1 |
12B |
5 meter |
|
R1-U23 |
2A |
R2-U1 |
13A |
5 meter |
|
R1-U23 |
2B |
R2-U1 |
13B |
5 meter |
|
R1-U23 |
3A |
R2-U1 |
14A |
5 meter |
|
R1-U23 |
3B |
R2-U1 |
14B |
5 meter |
|
R1-U16 |
2A |
R2-U1 |
15A |
5 meter |
|
R1-U16 |
2B |
R2-U1 |
15B |
5 meter |
|
R1-U16 |
3A |
R2-U1 |
16A |
5 meter |
|
R1-U16 |
3B |
R2-U1 |
16B |
5 meter |
Note:
Connect leaf switches in each Exalogic machine full rack to the spine switch in the other Exalogic machine full rack, as shown in Table 5-3.
For network connectivity diagrams of other Exalogic-to-Exalogic scenarios, see Multirack Cabling Diagrams for Exalogic-to-Exalogic Scenarios.