6.2 Two-Rack Cabling with InfiniBand Network Fabric
Review this information before cabling two racks together with InfiniBand Network Fabric.
-
The spine switch is in U1 (also referred to as IB1) for all racks.
-
In the following lists, the leaf switches are referred to as IB2 and IB3. Use the rack unit that is appropriate for your system.
-
In Oracle Exadata Database Machine X2-2 racks and Oracle Exadata Storage Expansion Racks with Exadata Storage Server with Sun Fire X4270 M2 Servers, the leaf switches are in U20 and U24, also referred to as IB2 and IB3, respectively.
-
In Oracle Exadata Database Machine 8-socket (X8-8, X7-8, X6-8, X5-8, X4-8, X3-8 and X2-8) Full Rack systems, the leaf switches are in U21 and U23, also referred to as IB2 and IB3, respectively.
-
In Oracle Exadata Database Machine X4-2 and later racks, Oracle Exadata Database Machine X3-2 Full Racks and Oracle Exadata Storage Expansion Rack X3-2 racks, the leaf switches are in U20 and U22, also referred to as IB2 and IB3, respectively.
-
The cable lengths shown in the following lists assume that the racks are adjacent to each other. If the racks are not adjacent or use overhead cabling trays, then they may require longer cables lengths. Cables lengths up to 100 meters are supported.
The following table shows the cable connections for the first spine switch (R1-U1) when cabling two full racks together.
Table 6-1 Leaf Switch Connections for the First Rack in a Two-Rack System
Leaf Switch | Connection | Cable Length |
---|---|---|
R1 IB3 within Rack 1 |
R1-IB3-P8A to R1-U1-P3A |
3 meters |
R1 IB3 to Rack 2 |
R1-IB3-P10A to R2-U1-P7A |
5 meters |
R1 IB2 within Rack 1 |
R1-IB2-P8A to R1-U1-P3B |
3 meters |
R1 IB2 to Rack 2 |
R1-IB2-P10A to R2-U1-P7B |
5 meters |
The following table shows the cable connections for the second spine switch (R2-U1) when cabling two full racks together.
Table 6-2 Leaf Switch Connections for the Second Rack in a Two-Rack System
Leaf Switch | Connection | Cable Length |
---|---|---|
R2 IB3 within Rack 2 |
R2-IB3-P8A to R2-U1-P3A |
3 meters |
R2 IB3 to Rack 1 |
R2-IB3-P10A to R1-U1-P7A |
5 meters |
R2 IB2 within Rack 2 |
R2-IB2-P8A to R2-U1-P3B |
3 meters |
R2 IB2 to Rack 1 |
R2-IB2-P10A to R1-U1-P7B |
5 meters |