Eight-Rack Cabling for X8M Racks
This section provides the cabling tables to connect eight (8) X8M racks together, all of which use RoCE Network Fabric.
Note:
The following conventions were used in the cabling notation for connecting multiple racks together.
-
The spine switch (also referred to as SS) is in U1 for all racks.
-
The leaf switches are referred to as Lower Leaf (LL) and Upper Leaf (UL).
-
The leaf switches are located as follows:
-
At rack unit 20 (U20) and 22 (U22) in Recovery Appliance X8M or Storage Expansion Rack X8M
-
At rack unit 21 (U21) and rack unit 23 (U23) in Recovery Appliance X8M-8
-
-
The cable lengths shown in the following lists assume that the racks are adjacent to each other, the cables are routed through a raised floor, and there are no obstacles in the routing between the racks. If the racks are not adjacent, or use overhead cabling trays, then they may require longer cable lengths. Cable lengths up to 100 meters are supported.
-
Only optical cables (with additional transceivers) are supported for lengths greater than 5 meters.
-
For X8M-8 systems with three database servers and 11 storage servers only, you must adjust the following multi-rack cabling information. On such systems only, port 30 on the leaf switches is connected to a database server and is not used as an inter-switch link. Consequently, in the following tables, any connection to port number 30 on any leaf switch must instead connect to port number 34 on the same leaf switch. For example, R1-UL-P30 must change to R1-UL-P34, R1-LL-P30 must change to R1-LL-P34, and so on. These changes only apply to X8M-8 systems with three database servers and 11 storage servers.
The following diagrams show the cable connections for the 8 spine switches when cabling eight racks together:
Description of the illustration x8m-8-rack-part1.png
Description of the illustration x8m-8-rack-part2.png
The following table describes the cable connections for the first spine switch (R1-SS) when cabling eight racks together:
Table 23-28 Leaf Switch Connections for the First Rack in a Eight-Rack System
| Leaf Switch | Connection | Cable Length |
|---|---|---|
| R1 UL within Rack 1 | R1-UL-P5 to R1-SS-P5 | 3 meters |
| R1 UL to Rack 2 | R1-UL-P31 to R2-SS-P5 | 5 meters |
| R1 UL to Rack 3 | R1-UL-P6 to R3-SS-P5 | 5 meters |
| R1 UL to Rack 4 | R1-UL-P4 to R4-SS-P5 | 10 meters |
| R1 UL to Rack 5 | R1-UL-P30 to R5-SS-P5 | 10 meters |
| R1 UL to Rack 6 | R1-UL-P32 to R6-SS-P5 | 10 meters |
| R1 UL to Rack 7 | R1-UL-P33 to R7-SS-P5 | 10 meters |
| R1 UL to Rack 8 | R1-UL-P7 to R8-SS-P5 | 10 meters |
| R1 LL within Rack 1 | R1-LL-P5 to R1-SS-P13 | 3 meters |
| R1 LL to Rack 2 | R1-LL-P31 to R2-SS-P13 | 5 meters |
| R1 LL to Rack 3 | R1-LL-P4 to R3-SS-P13 | 5 meters |
| R1 LL to Rack 4 | R1-LL-P30 to R4-SS-P13 | 10 meters |
| R1 LL to Rack 5 | R1-LL-P7 to R5-SS-P13 | 10 meters |
| R1 LL to Rack 6 | R1-LL-P32 to R6-SS-P13 | 10 meters |
| R1 LL to Rack 7 | R1-LL-P6 to R7-SS-P13 | 10 meters |
| R1 LL to Rack 8 | R1-LL-P33 to R8-SS-P13 | 10 meters |
The following table describes the cable connections for the second spine switch (R2-SS) when cabling eight full racks together:
Table 23-29 Leaf Switch Connections for the Second Rack in a Eight-Rack System
| Leaf Switch | Connection | Cable Length |
|---|---|---|
| R2 UL to Rack 1 | R2-UL-P5 to R1-SS-P6 | 5 meters |
| R2 UL within Rack 2 | R2-UL-P31 to R2-SS-P6 | 3 meters |
| R2 UL to Rack 3 | R2-UL-P6 to R3-SS-P6 | 5 meters |
| R2 UL to Rack 4 | R2-UL-P4 to R4-SS-P6 | 5 meters |
| R2 UL to Rack 5 | R2-UL-P7 to R5-SS-P6 | 10 meters |
| R2 UL to Rack 6 | R2-UL-P33 to R6-SS-P6 | 10 meters |
| R2 UL to Rack 7 | R2-UL-P32 to R7-SS-P6 | 10 meters |
| R2 UL to Rack 8 | R2-UL-P30 to R8-SS-P6 | 10 meters |
| R2 LL to Rack 1 | R2-LL-P5 to R1-SS-P14 | 5 meters |
| R2 LL within Rack 2 | R2-LL-P31 to R2-SS-P14 | 3 meters |
| R2 LL to Rack 3 | R2-LL-P6 to R3-SS-P14 | 5 meters |
| R2 LL to Rack 4 | R2-LL-P4 to R4-SS-P14 | 5 meters |
| R2 LL to Rack 5 | R2-LL-P33 to R5-SS-P14 | 10 meters |
| R2 LL to Rack 6 | R2-LL-P7 to R6-SS-P14 | 10 meters |
| R2 LL to Rack 7 | R2-LL-P32 to R7-SS-P14 | 10 meters |
| R2 LL to Rack 8 | R2-LL-P30 to R8-SS-P14 | 10 meters |
The following table describes the cable connections for the third spine switch (R3-SS) when cabling eight full racks together:
Table 23-30 Leaf Switch Connections for the Third Rack in a Eight-Rack System
| Leaf Switch | Connection | Cable Length |
|---|---|---|
| R3 UL to Rack 1 | R3-UL-P5 to R1-SS-P11 | 5 meters |
| R3 UL to Rack 2 | R3-UL-P6 to R2-SS-P11 | 5 meters |
| R3 UL within Rack 3 | R3-UL-P30 to R3-SS-P11 | 3 meters |
| R3 UL to Rack 4 | R3-UL-P4 to R4-SS-P11 | 5 meters |
| R3 UL to Rack 5 | R3-UL-P32 to R5-SS-P11 | 5 meters |
| R3 UL to Rack 6 | R3-UL-P7 to R6-SS-P11 | 10 meters |
| R3 UL to Rack 7 | R3-UL-P31 to R7-SS-P11 | 10 meters |
| R3 UL to Rack 8 | R3-UL-P33 to R8-SS-P11 | 10 meters |
| R3 LL to Rack 1 | R3-LL-P4 to R1-SS-P20 | 5 meters |
| R3 LL to Rack 2 | R3-LL-P31 to R2-SS-P20 | 5 meters |
| R3 LL within Rack 3 | R3-LL-P32 to R3-SS-P20 | 3 meters |
| R3 LL to Rack 4 | R3-LL-P33 to R4-SS-P20 | 5 meters |
| R3 LL to Rack 5 | R3-LL-P5 to R5-SS-P20 | 5 meters |
| R3 LL to Rack 6 | R3-LL-P30 to R6-SS-P20 | 10 meters |
| R3 LL to Rack 7 | R3-LL-P7 to R7-SS-P20 | 10 meters |
| R3 LL to Rack 8 | R3-LL-P6 to R8-SS-P20 | 10 meters |
The following table describes the cable connections for the fourth spine switch (R4-SS) when cabling eight full racks together:
Table 23-31 Leaf Switch Connections for the Fourth Rack in a Eight-Rack System
| Leaf Switch | Connection | Cable Length |
|---|---|---|
| R4 UL to Rack 1 | R4-UL-P7 to R1-SS-P10 | 10 meters |
| R4 UL to Rack 2 | R4-UL-P6 to R2-SS-P10 | 5 meters |
| R4 UL to Rack 3 | R4-UL-P33 to R3-SS-P10 | 5 meters |
| R4 UL within Rack 4 | R4-UL-P32 to R4-SS-P10 | 3 meters |
| R4 UL to Rack 5 | R4-UL-P5 to R5-SS-P10 | 5 meters |
| R4 UL to Rack 6 | R4-UL-P30 to R6-SS-P10 | 5 meters |
| R4 UL to Rack 7 | R4-UL-P31 to R7-SS-P10 | 10 meters |
| R4 UL to Rack 8 | R4-UL-P4 to R8-SS-P10 | 10 meters |
| R4 LL to Rack 1 | R4-LL-P5 to R1-SS-P17 | 10 meters |
| R4 LL to Rack 2 | R4-LL-P4 to R2-SS-P17 | 5 meters |
| R4 LL to Rack 3 | R4-LL-P31 to R3-SS-P17 | 5 meters |
| R4 LL within Rack 4 | R4-LL-P30 to R4-SS-P17 | 3 meters |
| R4 LL to Rack 5 | R4-LL-P7 to R5-SS-P17 | 5 meters |
| R4 LL to Rack 6 | R4-LL-P6 to R6-SS-P17 | 5 meters |
| R4 LL to Rack 7 | R4-LL-P32 to R7-SS-P17 | 10 meters |
| R4 LL to Rack 8 | R4-LL-P33 to R8-SS-P17 | 10 meters |
The following table describes the cable connections for the fifth spine switch (R5-SS) when cabling eight full racks together:
Table 23-32 Leaf Switch Connections for the Fifth Rack in a Eight-Rack System
| Leaf Switch | Connection | Cable Length |
|---|---|---|
| R5 UL to Rack 1 | R5-UL-P5 to R1-SS-P8 | 10 meters |
| R5 UL to Rack 2 | R5-UL-P4 to R2-SS-P8 | 10 meters |
| R5 UL to Rack 3 | R5-UL-P6 to R3-SS-P8 | 5 meters |
| R5 UL to Rack 4 | R5-UL-P33 to R4-SS-P12 | 5 meters |
| R5 UL within Rack 5 | R5-UL-P30 to R5-SS-P7 | 3 meters |
| R5 UL to Rack 6 | R5-UL-P7 to R6-SS-P7 | 5 meters |
| R5 UL to Rack 7 | R5-UL-P32 to R7-SS-P7 | 5 meters |
| R5 UL to Rack 8 | R5-UL-P31 to R8-SS-P7 | 10 meters |
| R5 LL to Rack 1 | R5-LL-P7 to R1-SS-P18 | 10 meters |
| R5 LL to Rack 2 | R5-LL-P6 to R2-SS-P18 | 10 meters |
| R5 LL to Rack 3 | R5-LL-P31 to R3-SS-P16 | 5 meters |
| R5 LL to Rack 4 | R5-LL-P33 to R4-SS-P19 | 5 meters |
| R5 LL within Rack 5 | R5-LL-P30 to R5-SS-P15 | 3 meters |
| R5 LL to Rack 6 | R5-LL-P4 to R6-SS-P15 | 5 meters |
| R5 LL to Rack 7 | R5-LL-P5 to R7-SS-P15 | 5 meters |
| R5 LL to Rack 8 | R5-LL-P32 to R8-SS-P15 | 10 meters |
The following table describes the cable connections for the sixth spine switch (R6-SS) when cabling eight full racks together:
Table 23-33 Leaf Switch Connections for the Sixth Rack in a Eight-Rack System
| Leaf Switch | Connection | Cable Length |
|---|---|---|
| R6 UL to Rack 1 | R6-UL-P7 to R1-SS-P16 | 10 meters |
| R6 UL to Rack 2 | R6-UL-P4 to R2-SS-P16 | 10 meters |
| R6 UL to Rack 3 | R6-UL-P6 to R3-SS-P19 | 10 meters |
| R6 UL to Rack 4 | R6-UL-P31 to R4-SS-P15 | 5 meters |
| R6 UL to Rack 5 | R6-UL-P33 to R5-SS-P18 | 5 meters |
| R6 UL within Rack 6 | R6-UL-P30 to R6-SS-P16 | 3 meters |
| R6 UL to Rack 7 | R6-UL-P5 to R7-SS-P16 | 5 meters |
| R6 UL to Rack 8 | R6-UL-P32 to R8-SS-P16 | 5 meters |
| R6 LL to Rack 1 | R6-LL-P5 to R1-SS-P12 | 10 meters |
| R6 LL to Rack 2 | R6-UL-P4 to R2-SS-P12 | 10 meters |
| R6 LL to Rack 3 | R6-LL-P31 to R3-SS-P7 | 10 meters |
| R6 LL to Rack 4 | R6-LL-P33 to R4-SS-P9 | 5 meters |
| R6 LL to Rack 5 | R6-LL-P30 to R5-SS-P8 | 5 meters |
| R6 LL within Rack 6 | R6-LL-P32 to R6-SS-P8 | 3 meters |
| R6 LL to Rack 7 | R6-LL-P6 to R7-SS-P8 | 5 meters |
| R6 LL to Rack 8 | R6-LL-P7 to R8-SS-P8 | 5 meters |
The following table describes the cable connections for the seventh spine switch (R7-SS) when cabling eight full racks together:
Table 23-34 Leaf Switch Connections for the Seventh Rack in a Eight-Rack System
| Leaf Switch | Connection | Cable Length |
|---|---|---|
| R7 UL to Rack 1 | R7-UL-P5 to R1-SS-P9 | 10 meters |
| R7 UL to Rack 2 | R7-UL-P7 to R2-SS-P7 | 10 meters |
| R7 UL to Rack 3 | R7-UL-P4 to R3-SS-P9 | 10 meters |
| R7 UL to Rack 4 | R7-UL-P6 to R4-SS-P8 | 10 meters |
| R7 UL to Rack 5 | R7-UL-P31 to R5-SS-P9 | 5 meters |
| R7 UL to Rack 6 | R7-UL-P33 to R6-SS-P9 | 5 meters |
| R7 UL within Rack 7 | R7-UL-P30 to R7-SS-P9 | 3 meters |
| R7 UL to Rack 8 | R7-UL-P32 to R8-SS-P9 | 5 meters |
| R7 LL to Rack 1 | R7-LL-P5 to R1-SS-P19 | 10 meters |
| R7 LL to Rack 2 | R7-LL-P7 to R2-SS-P19 | 10 meters |
| R7 LL to Rack 3 | R7-LL-P4 to R3-SS-P15 | 10 meters |
| R7 LL to Rack 4 | R7-LL-P6 to R4-SS-P18 | 10 meters |
| R7 LL to Rack 5 | R7-LL-P31 to R5-SS-P16 | 5 meters |
| R7 LL to Rack 6 | R7-LL-P33 to R6-SS-P18 | 5 meters |
| R7 LL within Rack 7 | R7-LL-P30 to R7-SS-P18 | 3 meters |
| R7 LL to Rack 8 | R7-LL-P32 to R8-SS-P12 | 5 meters |
The following table describes the cable connections for the eighth spine switch (R8-SS) when cabling eight full racks together:
Table 23-35 Leaf Switch Connections for the Eighth Rack in a Eight-Rack System
| Leaf Switch | Connection | Cable Length |
|---|---|---|
| R8 UL to Rack 1 | R8-UL-P5 to R1-SS-P15 | 10 meters |
| R8 UL to Rack 2 | R8-UL-P7 to R2-SS-P15 | 10 meters |
| R8 UL to Rack 3 | R8-UL-P4 to R3-SS-P18 | 10 meters |
| R8 UL to Rack 4 | R8-UL-P6 to R4-SS-P16 | 10 meters |
| R8 UL to Rack 5 | R8-UL-P31 to R5-SS-P19 | 10 meters |
| R8 UL to Rack 6 | R8-UL-P33 to R6-SS-P19 | 5 meters |
| R8 UL to Rack 7 | R8-UL-P30 to R7-SS-P19 | 5 meters |
| R8 UL within Rack 8 | R8-UL-P32 to R8-SS-P19 | 3 meters |
| R8 LL to Rack 1 | R8-LL-P5 to R1-SS-P7 | 10 meters |
| R8 LL to Rack 2 | R8-LL-P7 to R2-SS-P9 | 10 meters |
| R8 LL to Rack 3 | R8-LL-P4 to R3-SS-P12 | 10 meters |
| R8 LL to Rack 4 | R8-LL-P6 to R4-SS-P7 | 10 meters |
| R8 LL to Rack 5 | R8-LL-P31 to R5-SS-P12 | 10 meters |
| R8 LL to Rack 6 | R8-LL-P33 to R6-SS-P12 | 5 meters |
| R8 LL to Rack 7 | R8-LL-P30 to R7-SS-P12 | 5 meters |
| R8 LL within Rack 8 | R8-LL-P32 to R8-SS-P18 | 3 meters |