5 Multi-Rack Cabling Tables for Oracle Engineered System Rack X2 to X8

This appendix contains the multi-rack cabling tables for Oracle Engineered System Racks X2 to X8, which use a InfiniBand Transport Layer systems based on an InfiniBand Network Layer network.

5.1 Understanding Multi-Rack Cabling

Up to eight racks can be cabled together without external RDMA Network Fabric switches.

Three RDMA Network Fabric switches are needed when cabling racks together. If any rack does not include a third switch, then order the expansion switch kit. These switches attach to standard RDMA Network Fabric connectors at the end of the cables. The procedures in this section assume the racks are adjacent to each other. If they are not, then longer cables may be required for the connections.

Note:

Only for InfiniBand Transport Layer systems based on an InfiniBand Network Layer (X8 and earlier).
  • For Eighth or Quarter Racks, which are the smallest Elastic Configurations, follow the instructions in "Cabling Oracle Exadata Quarter Racks and Oracle Exadata Eighth Racks" for direct connection without spine switch.

  • For other racks (Half Rack, Full Rack, Elastic Configurations larger than Eighth or Quarter Rack) install a spine switch and follow the standard N-rack cabling using fat tree topology for two or more interconnected racks, for example "Two-Rack Cabling."

The switch at rack unit 1 (U1) is referred to as the spine switch.

The following switches are referred to as leaf switches:

  • The switches at rack unit 20 (U20) and rack unit 24 (U24) in Oracle Engineered System X2-2 racks and Oracle Exadata Storage Expansion Racks

  • The switches at rack unit 21 (U21) and rack unit 23 (U23) in Oracle Engineered System X2-8 and later.

  • The switches at rack unit 20 (U20) and 22 (U22) in Oracle Engineered System X3-2 and later racks or Storage Expansion Rack X3-2 and later.

    Note:

    Oracle Exadata Database Machine X4-2 and later racks or Oracle Exadata Database Machine X3-8 Full Racks with Exadata Storage Server X4-2L Servers do not include spare cables or a third Sun Datacenter InfiniBand Switch 36 switch. To extend Oracle Exadata Database Machine X4-2 and later racks or Oracle Exadata Database Machine X3-8 Full Racks with Exadata Storage Server X4-2L Servers, you must order cables and a Sun Datacenter InfiniBand Switch 36 switch.

In a single rack, the two leaf switches are interconnected using seven connections. In addition, each leaf switch has one connection to the spine switch. The leaf switches connect to the spine switch as shown in the following graphic:

Figure 5-1 Connections Between Spine Switch and Leaf Switches in a Single Rack

Description of Figure 5-1 follows
Description of "Figure 5-1 Connections Between Spine Switch and Leaf Switches in a Single Rack"

The Oracle Database servers and Exadata Storage Servers connect to the leaf switches as shown in the following graphic:

Figure 5-2 Connections Between Database Servers and Storage Servers and Leaf Switches

Description of Figure 5-2 follows
Description of "Figure 5-2 Connections Between Database Servers and Storage Servers and Leaf Switches"

When connecting up to eight racks together, remove the seven existing inter-switch connections between each leaf switch, as well as the two connections between the leaf switches and the spine switch. From each leaf switch, distribute eight connections over the spine switches in all racks. In multi-rack environments, the leaf switches inside a rack are no longer directly interconnected, as shown in the following graphic:

Figure 5-3 Connections Between Spine Switches and Leaf Switches Across Two Racks

Description of Figure 5-3 follows
Description of "Figure 5-3 Connections Between Spine Switches and Leaf Switches Across Two Racks"

As shown in the preceding graphic, each leaf switch in rack 1 connects to the following switches:

  • Four connections to its internal spine switch

  • Four connections to the spine switch in rack 2

The spine switch in rack 1 connects to the following switches:

  • Eight connections to both internal leaf switches

  • Eight connections to both leaf switches in rack 2

As the number of racks increases from two to eight, the pattern continues as shown in the following graphic:

Figure 5-4 Connections Between Spine Switches and Leaf Switches for up to 8 Racks

Description of Figure 5-4 follows
Description of "Figure 5-4 Connections Between Spine Switches and Leaf Switches for up to 8 Racks"

As shown in the preceding graphic, each leaf switch has eight inter-switch connections distributed over all spine switches. Each spine switch has 16 inter-switch connections distributed over all leaf switches. The leaf switches are not directly interconnected with other leaf switches, and the spine switches are not directly interconnected with the other spine switches.

Note:

Cable lengths are specified for racks 1 through 8.

5.1.1 Preparing to Cable Racks Together

Racks can be added together to increase system capacity and performance. When cabling racks together, note the following:

  • Oracle recommends that the names for the servers include the rack unit number. This helps identify the server during diagnostics.

  • When completing Oracle Exadata Deployment Assistant for the additional rack, you are prompted for SCAN addresses. However, these SCAN addresses are not used because the SCAN address from the original rack are used. Manually remove the new SCAN addresses from the generated installation files.

  • The software owner account names and group names, as well as their identifiers, must match the names and identifiers of the original rack.

  • If the additional grid disks are used with existing disk groups, then ensure the grid disk sizes for the new rack are the same as the original rack.

  • If the InfiniBand network consists of four or more racks cabled together, then disable the Subnet Manager on the leaf switches.

  • Verify the Master Subnet Manager is located on the spine switch.

  • Oracle Exadata Database Machine Quarter Racks can be extended as follows:

    • Connect two Oracle Exadata Database Machine Quarter Racks together. At least four of the six ports reserved for external connectivity are open on each leaf switch. The six ports are 5B, 6A, 6B, 7A, 7B, and 12A in each leaf switch. Maintain the existing seven inter-switch links between the leaf switches within each rack. Connect the leaf switches between the racks with two links each, using the ports reserved for external connectivity.

    • Connect one Oracle Exadata Database Machine Quarter Rack with one Oracle Exadata Database Machine Half Rack or one Oracle Exadata Database Machine Full Rack. At least four ports reserved for external connectivity are open on each leaf switch. The spine switch in the Oracle Exadata Database Machine Half Rack or Oracle Exadata Database Machine Full Rack remains as the spine switch. Maintain the existing seven inter-switch links between the leaf switches within each rack. Connect the leaf switches between the racks with two links each, using the ports reserved for external connectivity.

    • Connect one Oracle Exadata Database Machine Quarter Rack with two or more Oracle Exadata Database Machine Half Racks or Oracle Exadata Database Machine Full Racks. The racks are interconnected using a fat-tree topology. Connect each leaf switch in the quarter rack to the spine switch of each half rack or full rack using two links each. If there are more than four racks, then use one link instead of two. The seven inter-switch links between the leaf switches in the quarter rack are removed.

      Note:

      To connect more than one quarter rack to additional racks, it is necessary to purchase Sun Datacenter InfiniBand Switch 36 switches for the quarter racks.

  • If you are extending Oracle Exadata Database Machine X4-2 or later, or Oracle Exadata Database Machine X3-8 Full Rack, or Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) half rack, then order the expansion kit that includes a Sun Datacenter InfiniBand Switch 36 switch.

Perform the following tasks before cabling racks together:

  1. Determine the number of racks that will be cabled together.

  2. Count the spare cables from the kit, and existing inter-switch cables.

    Note:

    Oracle Exadata Database Machine X4-2 and later racks or Oracle Exadata Database Machine X3-8 Full Racks with Exadata Storage Server X4-2L Servers do not include spare cables or a third Sun Datacenter InfiniBand Switch 36 switch. To extend Oracle Exadata Database Machine X4-2 and later racks or Oracle Exadata Database Machine X3-8 Full Racks with Exadata Storage Server X4-2L Servers, you must order cables and a Sun Datacenter InfiniBand Switch 36 switch.

    For Oracle Exadata Racks earlier than Oracle Exadata Database Machine X4-2, no additional InfiniBand cables need to be purchased when connecting up to three Oracle Exadata Database Machine Full Racks. The following table lists the spare cables for the switch:

    Cable Spare Inter-switch Total

    InfiniBand 2 meter cable

    0

    7

    7

    InfiniBand 3 meter cable

    6

    2

    8

    InfiniBand 5 meter cable

    10

    0

    10

    When connecting four or more racks, it is necessary to purchase additional 10 meter optical QSFP-to-QSFP QDR InfiniBand cables. Two QSFP SR transceivers are needed for each optical cable. The following table lists the number of racks, and cables needed to interconnect the racks.

    Number of Racks 3 Meter Cables Needed 5 Meter Cables Needed 10 Meter Cables Needed

    2

    16

    16

    0

    3

    18

    30

    0

    4

    16

    40

    8

    5

    20

    42

    18

    6

    24

    46

    26

    7

    28

    44

    40

    8

    16

    52

    60

  3. Gather the following tools:

    • Pen

    • Note pad

    • Flashlight

    • Labels for cables and rack

    • Cable ties (Oracle recommends hook and loop straps)

    See Also:

    Oracle Exadata Database Machine Maintenance Guide for information about cable labels

  4. Determine a naming method for the rack prefixes. For example, if the original rack has the prefix dm01, then use the prefix dm02 for the second rack, the prefix dm03 for the third rack, and so on.

  5. Verify the racks have unique host names and IP addresses. All servers interconnected in the racks must have unique names and IP addresses.

    Server names and IP addresses conventions may differ in the following cases:

    • Initial installation of all Oracle Exadata Database Machine Full Racks: System address assignments and host names should be complete.

    • New Oracle Exadata Database Machine Full Racks are added to an existing cluster: The new rack configuration should require unique host names and IP addresses for the new Oracle Exadata Database Machines. The IP addresses on the same subnet cannot conflict with the existing systems.

    • Two existing Oracle Exadata Database Machine Full Racks are clustered together: You can assign host names and IP addresses only if Oracle Exadata Database Machines are already assigned unique host names and IP addresses, or the entire cluster must be reconfigured. The machines must be on the same subnet and not have conflicting IP addresses.

  6. Ensure the IP addresses for the new servers are in the same subnet, and do not overlap with the currently-installed servers.

  7. Ensure the firmware on the original switches are at the same level as the new switches using the nm2version command. If the firmware is not at the same level, then apply a firmware patch.

5.1.2 Cabling Oracle Exadata Quarter Racks and Oracle Exadata Eighth Racks

Oracle Exadata Quarter Racks and Oracle Exadata Eighth Racks can be cabled as follows:

  • Oracle Exadata Quarter Rack to Oracle Exadata Quarter Rack

  • Oracle Exadata Quarter Rack to Oracle Exadata Half Rack, or multiple Oracle Exadata Half Racks or Oracle Exadata Full Racks

  • Oracle Exadata Quarter Rack to Oracle Exadata Full Rack, or multiple Oracle Exadata Full Racks or Oracle Exadata Half Racks

  • Oracle Exadata Eighth Rack to Oracle Exadata Eighth Rack

  • Oracle Exadata Eighth Rack to Oracle Exadata Half Rack, or multiple Oracle Exadata Half Racks or Oracle Exadata Full Racks

  • Oracle Exadata Eighth Rack to Oracle Exadata Full Rack, or multiple Oracle Exadata Full Racks or Oracle Exadata Half Racks

Note:

  • To cable additional Oracle Exadata Quarter Racks or Oracle Exadata Eighth Racks, it is necessary to purchase additional Sun Datacenter InfiniBand Switch 36 switches.

  • It is assumed that four of the six ports dedicated for external connectivity in Oracle Exadata Half Rack or Oracle Exadata Full Rack are unused and available for switch interconnections with Oracle Exadata Quarter Rack or Oracle Exadata Eighth Rack. The six ports are 5B, 6A, 6B, 7A, 7B, and 12A in each leaf switch.

The following graphic shows the cable connections for two Oracle Exadata Quarter Racks. The leaf switches within each rack maintain their existing seven connections. The leaf switches interconnect between the racks with two links each using the ports reserved for external connectivity.

Figure 5-5 Leaf Switch Connections Between 2 Quarter Racks

Description of Figure 5-5 follows
Description of "Figure 5-5 Leaf Switch Connections Between 2 Quarter Racks"

The following graphic shows the cable connections from Oracle Exadata Quarter Rack to Oracle Exadata Half Rack or Oracle Exadata Full Rack. The leaf switches within each rack maintain their existing seven connections. The leaf switches interconnect between the racks with two links each using the ports reserved for external connectivity.

Figure 5-6 Leaf and Spine Switch Connections Between a Quarter Rack and a Half or Full Rack

Description of Figure 5-6 follows
Description of "Figure 5-6 Leaf and Spine Switch Connections Between a Quarter Rack and a Half or Full Rack"

The following graphic shows the cable connections from Oracle Exadata Quarter Rack to two or more racks. The racks that connect to Oracle Exadata Quarter Rack must be all Oracle Exadata Half Racks or Oracle Exadata Full Racks, interconnected using a fat-tree topology. Each leaf switch in Oracle Exadata Quarter Rack connects to the spine switches in the other half racks or full racks with two links each. If there are more than four racks, then use one link instead of two. The seven inter-switch links between the leaf switches in the quarter rack are removed.

Figure 5-7 Leaf and Spine Switch Connections for a Quarter Rack Connected to One or More Half or Full Racks

Description of Figure 5-7 follows
Description of "Figure 5-7 Leaf and Spine Switch Connections for a Quarter Rack Connected to One or More Half or Full Racks"

5.2 Two-Rack Cabling

Review this information before cabling two racks together.

  • The spine switch is in U1 (also referred to as IB1) for all racks.

  • In the following lists, the leaf switches are referred to as IB2 and IB3. Use the rack unit that is appropriate for your system.

  • In Oracle Exadata Database Machine X2-2 racks and Oracle Exadata Storage Expansion Racks with Exadata Storage Server with Sun Fire X4270 M2 Servers, the leaf switches are in U20 and U24, also referred to as IB2 and IB3, respectively.

  • In Oracle Exadata Database Machine 8-socket (X8-8, X7-8, X6-8, X5-8, X4-8, X3-8 and X2-8) Full Rack systems, the leaf switches are in U21 and U23, also referred to as IB2 and IB3, respectively.

  • In Oracle Exadata Database Machine X4-2 and later racks, Oracle Exadata Database Machine X3-2 Full Racks and Oracle Exadata Storage Expansion Rack X3-2 racks, the leaf switches are in U20 and U22, also referred to as IB2 and IB3, respectively.

  • The cable lengths shown in the following lists assume that the racks are adjacent to each other. If the racks are not adjacent or use overhead cabling trays, then they may require longer cables lengths. Cables lengths up to 100 meters are supported.

The following table shows the cable connections for the first spine switch (R1-U1) when cabling two full racks together.

Table 5-1 Leaf Switch Connections for the First Rack in a Two-Rack System

Leaf Switch Connection Cable Length
R1 IB3 within Rack 1

R1-IB3-P8A to R1-U1-P3A
R1-IB3-P8B to R1-U1-P4A
R1-IB3-P9A to R1-U1-P5A
R1-IB3-P9B to R1-U1-P6A

3 meters
R1 IB3 to Rack 2

R1-IB3-P10A to R2-U1-P7A
R1-IB3-P10B to R2-U1-P8A
R1-IB3-P11A to R2-U1-P9A
R1-IB3-P11B to R2-U1-P10A

5 meters
R1 IB2 within Rack 1

R1-IB2-P8A to R1-U1-P3B
R1-IB2-P8B to R1-U1-P4B
R1-IB2-P9A to R1-U1-P5B
R1-IB2-P9B to R1-U1-P6B

3 meters
R1 IB2 to Rack 2

R1-IB2-P10A to R2-U1-P7B
R1-IB2-P10B to R2-U1-P8B
R1-IB2-P11A to R2-U1-P9B
R1-IB2-P11B to R2-U1-P10B

5 meters

The following table shows the cable connections for the second spine switch (R2-U1) when cabling two full racks together.

Table 5-2 Leaf Switch Connections for the Second Rack in a Two-Rack System

Leaf Switch Connection Cable Length
R2 IB3 within Rack 2

R2-IB3-P8A to R2-U1-P3A
R2-IB3-P8B to R2-U1-P4A
R2-IB3-P9A to R2-U1-P5A
R2-IB3-P9B to R2-U1-P6A

3 meters
R2 IB3 to Rack 1

R2-IB3-P10A to R1-U1-P7A
R2-IB3-P10B to R1-U1-P8A
R2-IB3-P11A to R1-U1-P9A
R2-IB3-P11B to R1-U1-P10A

5 meters
R2 IB2 within Rack 2

R2-IB2-P8A to R2-U1-P3B
R2-IB2-P8B to R2-U1-P4B
R2-IB2-P9A to R2-U1-P5B
R2-IB2-P9B to R2-U1-P6B

3 meters
R2 IB2 to Rack 1

R2-IB2-P10A to R1-U1-P7B
R2-IB2-P10B to R1-U1-P8B
R2-IB2-P11A to R1-U1-P9B
R2-IB2-P11B to R1-U1-P10B

5 meters

5.3 Three-Rack Cabling

Note the following when cabling three racks together:

  • The spine switch is in U1 (also referred to as IB1) for all racks.

  • In the following tables, the leaf switches are referred to as IB2 and IB3. Use the rack unit that is appropriate for your system.

  • In Oracle Exadata Database Machine X2-2 racks and Oracle Exadata Storage Expansion Racks with Exadata Storage Server with Sun Fire X4270 M2 Servers, the leaf switches are in U20 and U24, also referred to as IB2 and IB3, respectively.

  • In Oracle Exadata Database Machine X2-8 and later, the leaf switches are in U21 and U23, also referred to as IB2 and IB3, respectively.

  • In Oracle Exadata Database Machine X4-2 and later racks, or Oracle Exadata Database Machine X3-2 Full Racks or Oracle Exadata Storage Expansion Rack X3-2 racks, the leaf switches are in U20 and U22, also referred to as IB2 and IB3, respectively.

  • The cable lengths shown in the tables assume the racks are adjacent to each other. If the racks are not adjacent or use overhead cabling trays, then they may require longer cables lengths. Up to 100 meters is supported.

  • Only optical cables are supported for lengths greater than 5 meters.

Table 5-3 shows the cable connections for the first spine switch (R1-U1) when cabling three racks together:

Table 5-3 Leaf Switch Connections for the First Rack in a Three-Rack System

Leaf Switch Connection Cable Length

R1 IB3 within Rack 1

R1-IB3-P8A to R1-U1-P3A

R1-IB3-P8B to R1-U1-P4A

R1-IB3-P9A to R1-U1-P5A

3 meters

R1 IB3 to Rack 2

R1-IB3-P9B to R2-U1-P6A

R1-IB3-P10A to R2-U1-P7A

R1-IB3-P10B to R2-U1-P8A

5 meters

R1 IB3 to Rack 3

R1-IB3-P11A to R3-U1-P9A

R1-IB3-P11B to R3-U1-P10A

5 meters

R1 IB2 within Rack 1

R1-IB2-P8A to R1-U1-P3B

R1-IB2-P8B to R1-U1-P4B

R1-IB2-P9A to R1-U1-P5B

3 meters

R1 IB2 to Rack 2

R1-IB2-P9B to R2-U1-P6B

R1-IB2-P10A to R2-U1-P7B

R1-IB2-P10B to R2-U1-P8B

5 meters

R1 IB2 to Rack 3

R1-IB2-P11A to R3-U1-P9B

R1-IB2-P11B to R3-U1-P10B

5 meters

Table 5-4 shows the cable connections for the second spine switch (R2-U1) when cabling three racks together:

Table 5-4 Leaf Switch Connections for the Second Rack in a Three-Rack System

Leaf Switch Connection Cable Length

R2 IB3 within Rack 2

R2-IB3-P8A to R2-U1-P3A

R2-IB3-P8B to R2-U1-P4A

R2-IB3-P9A to R2-U1-P5A

3 meters

R2 IB3 to Rack 1

R2-IB3-P11A to R1-U1-P9A

R2-IB3-P11B to R1-U1-P10A

5 meters

R2 IB3 to Rack 3

R2-IB3-P9B to R3-U1-P6A

R2-IB3-P10A to R3-U1-P7A

R2-IB3-P10B to R3-U1-P8A

5 meters

R2 IB2 within Rack 2

R2-IB2-P8A to R2-U1-P3B

R2-IB2-P8B to R2-U1-P4B

R2-IB2-P9A to R2-U1-P5B

3 meters

R2 IB2 to Rack 1

R2-IB2-P11A to R1-U1-P9B

R2-IB2-P11B to R1-U1-P10B

5 meters

R2 IB2 to Rack 3

R2-IB2-P9B to R3-U1-P6B

R2-IB2-P10A to R3-U1-P7B

R2-IB2-P10B to R3-U1-P8B

5 meters

Table 5-5 shows the cable connections for the third spine switch (R3-U1) when cabling three full racks together:

Table 5-5 Leaf Switch Connections for the Third Rack in a Three-Rack System

Leaf Switch Connection Cable Length

R3 IB3 within Rack 3

R3-IB3-P8A to R3-U1-P3A

R3-IB3-P8B to R3-U1-P4A

R3-IB3-P9A to R3-U1-P5A

3 meters

R3 IB3 to Rack 1

R3-IB3-P9B to R1-U1-P6A

R3-IB3-P10A to R1-U1-P7A

R3-IB3-P10B to R1-U1-P8A

5 meters

R3 IB3 to Rack 2

R3-IB3-P11A to R2-U1-P9A

R3-IB3-P11B to R2-U1-P10A

5 meters

R3 IB2 within Rack 3

R3-IB2-P8A to R3-U1-P3B

R3-IB2-P8B to R3-U1-P4B

R3-IB2-P9A to R3-U1-P5B

3 meters

R3 IB2 to Rack 1

R3-IB2-P9B to R1-U1-P6B

R3-IB2-P10A to R1-U1-P7B

R3-IB2-P10B to R1-U1-P8B

5 meters

R3 IB2 to Rack 2

R3-IB2-P11A to R2-U1-P9B

R3-IB2-P11B to R2-U1-P10B

5 meters

5.4 Four-Rack Cabling

Note the following when cabling four racks together:

  • The spine switch is in U1 (also referred to as IB1) for all racks.

  • In the following tables, the leaf switches are referred to as IB2 and IB3. Use the rack unit that is appropriate for your system.

  • In Oracle Exadata Database Machine X2-2 racks and Oracle Exadata Storage Expansion Racks with Exadata Storage Server with Sun Fire X4270 M2 Servers, the leaf switches are in U20 and U24, also referred to as IB2 and IB3, respectively.

  • In Oracle Exadata Database Machine X2-8 and later, the leaf switches are in U21 and U23, also referred to as IB2 and IB3, respectively.

  • In Oracle Exadata Database Machine X4-2 and later racks, or Oracle Exadata Database Machine X3-2 Full Racks or Oracle Exadata Storage Expansion Rack X3-2 racks, the leaf switches are in U20 and U22, also referred to as IB2 and IB3, respectively.

  • The cable lengths shown in the tables assume the racks are adjacent to each other. If the racks are not adjacent or use overhead cabling trays, then they may require longer cables lengths. Up to 100 meters is supported.

  • Only optical cables are supported for lengths greater than 5 meters.

Table 5-6 shows the cable connections for the first spine switch (R1-U1) when cabling four racks together:

Table 5-6 Leaf Switch Connections for the First Rack in a Four-Rack System

Leaf Switch Connection Cable Length

R1 IB3 within Rack 1

R1-IB3-P8A to R1-U1-P3A

R1-IB3-P8B to R1-U1-P4A

3 meters

R1 IB3 to Rack 2

R1-IB3-P9A to R2-U1-P5A

R1-IB3-P9B to R2-U1-P6A

5 meters

R1 IB3 to Rack 3

R1-IB3-P10A to R3-U1-P7A

R1-IB3-P10B to R3-U1-P8A

5 meters

R1 IB3 to Rack 4

R1-IB3-P11A to R4-U1-P9A

R1-IB3-P11B to R4-U1-P10A

10 meters

R1 IB2 within Rack 1

R1-IB2-P8A to R1-U1-P3B

R1-IB2-P8B to R1-U1-P4B

3 meters

R1 IB2 to Rack 2

R1-IB2-P9A to R2-U1-P5B

R1-IB2-P9B to R2-U1-P6B

5 meters

R1 IB2 to Rack 3

R1-IB2-P10A to R3-U1-P7B

R1-IB2-P10B to R3-U1-P8B

5 meters

R1 IB2 to Rack 4

R1-IB2-P11A to R4-U1-P9B

R1-IB2-P11B to R4-U1-P10B

10 meters

Table 5-7 shows the cable connections for the second spine switch (R2-U1) when cabling four full racks together:

Table 5-7 Leaf Switch Connections for the Second Rack in a Four-Rack System

Leaf Switch Connection Cable Length

R2 IB3 within Rack 2

R2-IB3-P8A to R2-U1-P3A

R2-IB3-P8B to R2-U1-P4A

3 meters

R2 IB3 to Rack 1

R2-IB3-P11A to R1-U1-P9A

R2-IB3-P11B to R1-U1-P10A

5 meters

R2 IB3 to Rack 3

R2-IB3-P9A to R3-U1-P5A

R2-IB3-P9B to R3-U1-P6A

5 meters

R2 IB3 to Rack 4

R2-IB3-P10A to R4-U1-P7A

R2-IB3-P10B to R4-U1-P8A

5 meters

R2 IB2 within Rack 2

R2-IB2-P8A to R2-U1-P3B

R2-IB2-P8B to R2-U1-P4B

3 meters

R2 IB2 to Rack 1

R2-IB2-P11A to R1-U1-P9B

R2-IB2-P11B to R1-U1-P10B

5 meters

R2 IB2 to Rack 3

R2-IB2-P9A to R3-U1-P5B

R2-IB2-P9B to R3-U1-P6B

5 meters

R2 IB2 to Rack 4

R2-IB2-P10A to R4-U1-P7B

R2-IB2-P10B to R4-U1-P8B

5 meters

Table 5-8 shows the cable connections for the third spine switch (R3-U1) when cabling four full racks together:

Table 5-8 Leaf Switch Connections for the Third Rack in a Four-Rack System

Leaf Switch Connection Cable Length

R3 IB3 within Rack 3

R3-IB3-P8A to R3-U1-P3A

R3-IB3-P8B to R3-U1-P4A

3 meters

R3 IB3 to Rack 1

R3-IB3-P10A to R1-U1-P7A

R3-IB3-P10B to R1-U1-P8A

5 meters

R3 IB3 to Rack 2

R3-IB3-P11A to R2-U1-P9A

R3-IB3-P11B to R2-U1-P10A

5 meters

R3 IB3 to Rack 4

R3-IB3-P9A to R4-U1-P5A

R3-IB3-P9B to R4-U1-P6A

5 meters

R3 IB2 within Rack 3

R3-IB2-P8A to R3-U1-P3B

R3-IB2-P8B to R3-U1-P4B

3 meters

R3 IB2 to Rack 1

R3-IB2-P10A to R1-U1-P7B

R3-IB2-P10B to R1-U1-P8B

5 meters

R3 IB2 to Rack 2

R3-IB2-P11A to R2-U1-P9B

R3-IB2-P11B to R2-U1-P10B

5 meters

R3 IB2 to Rack 4

R3-IB2-P9A to R4-U1-P5B

R3-IB2-P9B to R4-U1-P6B

5 meters

Table 5-9 shows the cable connections for the fourth spine switch (R4-U1) when cabling four full racks together:

Table 5-9 Leaf Switch Connections for the Fourth Rack in a Four-Rack System

Leaf Switch Connection Cable Length

R4 IB3 within Rack 4

R4-IB3-P8A to R4-U1-P3A

R4-IB3-P8B to R4-U1-P4A

3 meters

R4 IB3 to Rack 1

R4-IB3-P9A to R1-U1-P5A

R4-IB3-P9B to R1-U1-P6A

10 meters

R4 IB3 to Rack 2

R4-IB3-P10A to R2-U1-P7A

R4-IB3-P10B to R2-U1-P8A

5 meters

R4 IB3 to Rack 3

R4-IB3-P11A to R3-U1-P9A

R4-IB3-P11B to R3-U1-P10A

5 meters

R4 IB2 within Rack 4

R4-IB2-P8A to R4-U1-P3B

R4-IB2-P8B to R4-U1-P4B

3 meters

R4 IB2 to Rack 1

R4-IB2-P9A to R1-U1-P5B

R4-IB2-P9B to R1-U1-P6B

10 meters

R4 IB2 to Rack 2

R4-IB2-P10A to R2-U1-P7B

R4-IB2-P10B to R2-U1-P8B

5 meters

R4 IB2 to Rack 3

R4-IB2-P11A to R3-U1-P9B

R4-IB2-P11B to R3-U1-P10B

5 meters

5.5 Five-Rack Cabling

Note the following when cabling five racks together:

  • The spine switch is in U1 (also referred to as IB1) for all racks.

  • In the following tables, the leaf switches are referred to as IB2 and IB3. Use the rack unit that is appropriate for your system.

  • In Oracle Exadata Database Machine X2-2 racks and Oracle Exadata Storage Expansion Racks with Exadata Storage Server with Sun Fire X4270 M2 Servers, the leaf switches are in U20 and U24, also referred to as IB2 and IB3, respectively.

  • In Oracle Exadata Database Machine X2-8 and later racks, the leaf switches are in U21 and U23, also referred to as IB2 and IB3, respectively.

  • In Oracle Exadata Database Machine X4-2 and later racks, or Oracle Exadata Database Machine X3-2 Full Racks and Oracle Exadata Storage Expansion Rack X3-2 racks, the leaf switches are in U20 and U22, also referred to as IB2 and IB3, respectively.

  • The cable lengths shown in the tables assume the racks are adjacent to each other. If the racks are not adjacent or use overhead cabling trays, then they may require longer cables lengths. Up to 100 meters is supported.

  • Only optical cables are supported for lengths greater than 5 meters.

Table 5-10 shows the cable connections for the first spine switch (R1-U1) when cabling five racks together:

Table 5-10 Leaf Switch Connections for the First Rack in a Five-Rack System

Leaf Switch Connection Cable Length

R1 IB3 within Rack 1

R1-IB3-P8A to R1-U1-P3A

R1-IB3-P8B to R1-U1-P4A

3 meters

R1 IB3 to Rack 2

R1-IB3-P9A to R2-U1-P5A

R1-IB3-P9B to R2-U1-P6A

5 meters

R1 IB3 to Rack 3

R1-IB3-P10A to R3-U1-P7A

R1-IB3-P10B to R3-U1-P8A

5 meters

R1 IB3 to Rack 4

R1-IB3-P11A to R4-U1-P9A

10 meters

R1 IB3 to Rack 5

R1-IB3-P11B to R5-U1-P10A

10 meters

R1 IB2 within Rack 1

R1-IB2-P8A to R1-U1-P3B

R1-IB2-P8B to R1-U1-P4B

3 meters

R1 IB2 to Rack 2

R1-IB2-P9A to R2-U1-P5B

R1-IB2-P9B to R2-U1-P6B

5 meters

R1 IB2 to Rack 3

R1-IB2-P10A to R3-U1-P7B

R1-IB2-P10B to R3-U1-P8B

5 meters

R1 IB2 to Rack 4

R1-IB2-P11A to R4-U1-P9B

10 meters

R1 IB2 to Rack 5

R1-IB2-P11B to R5-U1-P10B

10 meters

Table 5-11 shows the cable connections for the second spine switch (R2-U1) when cabling five full racks together:

Table 5-11 Leaf Switch Connections for the Second Rack in a Five-Rack System

Leaf Switch Connection Cable Length

R2 IB3 within Rack 2

R2-IB3-P8A to R2-U1-P3A

R2-IB3-P8B to R2-U1-P4A

3 meters

R2 IB3 to Rack 1

R2-IB3-P11B to R1-U1-P10A

5 meters

R2 IB3 to Rack 3

R2-IB3-P9A to R3-U1-P5A

R2-IB3-P9B to R3-U1-P6A

5 meters

R2 IB3 to Rack 4

R2-IB3-P10A to R4-U1-P7A

R2-IB3-P10B to R4-U1-P8A

5 meters

R2 IB3 to Rack 5

R2-IB3-P11A to R5-U1-P9A

10 meters

R2 IB2 within Rack 2

R2-IB2-P8A to R2-U1-P3B

R2-IB2-P8B to R2-U1-P4B

3 meters

R2 IB2 to Rack 1

R2-IB2-P11B to R1-U1-P10B

5 meters

R2 IB2 to Rack 3

R2-IB2-P9A to R3-U1-P5B

R2-IB2-P9B to R3-U1-P6B

5 meters

R2 IB2 to Rack 4

R2-IB2-P10A to R4-U1-P7B

R2-IB2-P10B to R4-U1-P8B

5 meters

R2 IB2 to Rack 5

R2-IB2-P11A to R5-U1-P9B

10 meters

Table 5-12 shows the cable connections for the third spine switch (R3-U1) when cabling five full racks together:

Table 5-12 Leaf Switch Connections for the Third Rack in a Five-Rack System

Leaf Switch Connection Cable Length

R3 IB3 within Rack 3

R3-IB3-P8A to R3-U1-P3A

R3-IB3-P8B to R3-U1-P4A

3 meters

R3 IB3 to Rack 1

R3-IB3-P11A to R1-U1-P9A

5 meters

R3 IB3 to Rack 2

R3-IB3-P11B to R2-U1-P10A

5 meters

R3 IB3 to Rack 4

R3-IB3-P9A to R4-U1-P5A

R3-IB3-P9B to R4-U1-P6A

5 meters

R3 IB3 to Rack 5

R3-IB3-P10A to R5-U1-P7A

R3-IB3-P10B to R5-U1-P8A

5 meters

R3 IB2 within Rack 3

R3-IB2-P8A to R3-U1-P3B

R3-IB2-P8B to R3-U1-P4B

3 meters

R3 IB2 to Rack 1

R3-IB2-P11A to R1-U1-P9B

5 meters

R3 IB2 to Rack 2

R3-IB2-P11B to R2-U1-P10B

5 meters

R3 IB2 to Rack 4

R3-IB2-P9A to R4-U1-P5B

R3-IB2-P9B to R4-U1-P6B

5 meters

R3 IB2 to Rack 5

R3-IB2-P10A to R5-U1-P7B

R3-IB2-P10B to R5-U1-P8B

5 meters

Table 5-13 shows the cable connections for the fourth spine switch (R4-U1) when cabling five full racks together:

Table 5-13 Leaf Switch Connections for the Fourth Rack in a Five-Rack System

Leaf Switch Connection Cable Length

R4 IB3 within Rack 4

R4-IB3-P8A to R4-U1-P3A

R4-IB3-P8B to R4-U1-P4A

3 meters

R4 IB3 to Rack 1

R4-IB3-P10A to R1-U1-P7A

R4-IB3-P10B to R1-U1-P8A

10 meters

R4 IB3 to Rack 2

R4-IB3-P11A to R2-U1-P9A

5 meters

R4 IB3 to Rack 3

R4-IB3-P11B to R3-U1-P10A

5 meters

R4 IB3 to Rack 5

R4-IB3-P9A to R5-U1-P5A

R4-IB3-P9B to R5-U1-P6A

5 meters

R4 IB2 within Rack 4

R4-IB2-P8A to R4-U1-P3B

R4-IB2-P8B to R4-U1-P4B

3 meters

R4 IB2 to Rack 1

R4-IB2-P10A to R1-U1-P7B

R4-IB2-P10B to R1-U1-P8B

10 meters

R4 IB2 to Rack 2

R4-IB2-P11A to R2-U1-P9B

5 meters

R4 IB2 to Rack 3

R4-IB2-P11B to R3-U1-P10B

5 meters

R4 IB2 to Rack 5

R4-IB2-P9A to R5-U1-P5B

R4-IB2-P9B to R5-U1-P6B

5 meters

Table 5-14 shows the cable connections for the fifth spine switch (R5-U1) when cabling five full racks together:

Table 5-14 Leaf Switch Connections for the Fifth Rack in a Five-Rack System

Leaf Switch Connection Cable Length

R5 IB3 within Rack 5

R5-IB3-P8A to R5-U1-P3A

R5-IB3-P8B to R5-U1-P4A

3 meters

R5 IB3 to Rack 1

R5-IB3-P9A to R1-U1-P5A

R5-IB3-P9B to R1-U1-P6A

10 meters

R5 IB3 to Rack 2

R5-IB3-P10A to R2-U1-P7A

R5-IB3-P10B to R2-U1-P8A

10 meters

R5 IB3 to Rack 3

R5-IB3-P11A to R3-U1-P9A

5 meters

R5 IB3 to Rack 4

R5-IB3-P11B to R4-U1-P10A

5 meters

R5 IB2 within Rack 5

R5-IB2-P8A to R5-U1-P3B

R5-IB2-P8B to R5-U1-P4B

3 meters

R5 IB2 to Rack 1

R5-IB2-P9A to R1-U1-P5B

R5-IB2-P9B to R1-U1-P6B

10 meters

R5 IB2 to Rack 2

R5-IB2-P10A to R2-U1-P7B

R5-IB2-P10B to R2-U1-P8B

10 meters

R5 IB2 to Rack 3

R5-IB2-P11A to R3-U1-P9B

5 meters

R5 IB2 to Rack 4

R5-IB2-P11B to R4-U1-P10B

5 meters

5.6 Six-Rack Cabling

Note the following when cabling six racks together:

  • The spine switch is in U1 (also referred to as IB1) for all racks.

  • In the following tables, the leaf switches are referred to as IB2 and IB3. Use the rack unit that is appropriate for your system.

  • In Oracle Exadata Database Machine X2-2 racks and Oracle Exadata Storage Expansion Racks with Exadata Storage Server with Sun Fire X4270 M2 Servers, the leaf switches are in U20 and U24, also referred to as IB2 and IB3, respectively.

  • In Oracle Exadata Database Machine X2-8 and later 8-socket racks, the leaf switches are in U21 and U23, also referred to as IB2 and IB3, respectively.

  • In Oracle Exadata Database Machine X4-2 and later two-socket racks, or Oracle Exadata Database Machine X3-2 Full Racks and Oracle Exadata Storage Expansion Rack X3-2 racks, the leaf switches are in U20 and U22, also referred to as IB2 and IB3, respectively.

  • The cable lengths shown in the tables assume the racks are adjacent to each other. If the racks are not adjacent or use overhead cabling trays, then they may require longer cables lengths. Up to 100 meters is supported.

  • Only optical cables are supported for lengths greater than 5 meters.

Table 5-15 shows the cable connections for the first spine switch (R1-U1) when cabling six racks together:

Table 5-15 Leaf Switch Connections for the First Rack in a Six-Rack System

Leaf Switch Connection Cable Length

R1 IB3 within Rack 1

R1-IB3-P8A to R1-U1-P3A

R1-IB3-P8B to R1-U1-P4A

3 meters

R1 IB3 to Rack 2

R1-IB3-P9A to R2-U1-P5A

R1-IB3-P9B to R2-U1-P6A

5 meters

R1 IB3 to Rack 3

R1-IB3-P10A to R3-U1-P7A

5 meters

R1 IB3 to Rack 4

R1-IB3-P10B to R4-U1-P8A

10 meters

R1 IB3 to Rack 5

R1-IB3-P11A to R5-U1-P9A

10 meters

R1 IB3 to Rack 6

R1-IB3-P11B to R6-U1-P10A

10 meters

R1 IB2 within Rack 1

R1-IB2-P8A to R1-U1-P3B

R1-IB2-P8B to R1-U1-P4B

3 meters

R1 IB2 to Rack 2

R1-IB2-P9A to R2-U1-P5B

R1-IB2-P9B to R2-U1-P6B

5 meters

R1 IB2 to Rack 3

R1-IB2-P10A to R3-U1-P7B

5 meters

R1 IB2 to Rack 4

R1-IB2-P10B to R4-U1-P8B

10 meters

R1 IB2 to Rack 5

R1-IB2-P11A to R5-U1-P9B

10 meters

R1 IB2 to Rack 6

R1-IB2-P11B to R6-U1-P10B

10 meters

Table 5-16 shows the cable connections for the second spine switch (R2-U1) when cabling six full racks together:

Table 5-16 Leaf Switch Connections for the Second Rack in a Six-Rack System

Leaf Switch Connection Cable Length

R2 IB3 within Rack 2

R2-IB3-P8A to R2-U1-P3A

R2-IB3-P8B to R2-U1-P4A

3 meters

R2 IB3 to Rack 1

R2-IB3-P11B to R1-U1-P10A

5 meters

R2 IB3 to Rack 3

R2-IB3-P9A to R3-U1-P5A

R2-IB3-P9B to R3-U1-P6A

5 meters

R2 IB3 to Rack 4

R2-IB3-P10A to R4-U1-P7A

5 meters

R2 IB3 to Rack 5

R2-IB3-P10B to R5-U1-P8A

10 meters

R2 IB3 to Rack 6

R2-IB3-P11Ato R6-U1-P9A

10 meters

R2 IB2 within Rack 2

R2-IB2-P8A to R2-U1-P3B

R2-IB2-P8B to R2-U1-P4B

3 meters

R2 IB2 to Rack 1

R2-IB2-P11B to R1-U1-P10B

5 meters

R2 IB2 to Rack 3

R2-IB2-P9A to R3-U1-P5B

R2-IB2-P9B to R3-U1-P6B

5 meters

R2 IB2 to Rack 4

R2-IB2-P10A to R4-U1-P7B

5 meters

R2 IB2 to Rack 5

R2-IB2-P10B to R5-U1-P8B

10 meters

R2 IB2 to Rack 6

R2-IB2-P11Ato R6-U1-P9B

10 meters

Table 5-17 shows the cable connections for the third spine switch (R3-U1) when cabling six full racks together:

Table 5-17 Leaf Switch Connections for the Third Rack in a Six-Rack System

Leaf Switch Connection Cable Length

R3 IB3 within Rack 3

R3-IB3-P8A to R3-U1-P3A

R3-IB3-P8B to R3-U1-P4A

3 meters

R3 IB3 to Rack 1

R3-IB3-P11A to R1-U1-P9A

5 meters

R3 IB3 to Rack 2

R3-IB3-P11B to R2-U1-P10A

5 meters

R3 IB3 to Rack 4

R3-IB3-P9A to R4-U1-P5A

R3-IB3-P9B to R4-U1-P6A

5 meters

R3 IB3 to Rack 5

R3-IB3-P10A to R5-U1-P7A

5 meters

R3 IB3 to Rack 6

R3-IB3-P10B to R6-U1-P8A

10 meters

R3 IB2 within Rack 3

R3-IB2-P8A to R3-U1-P3B

R3-IB2-P8B to R3-U1-P4B

3 meters

R3 IB2 to Rack 1

R3-IB2-P11A to R1-U1-P9B

5 meters

R3 IB2 to Rack 2

R3-IB2-P11B to R2-U1-P10B

5 meters

R3 IB2 to Rack 4

R3-IB2-P9A to R4-U1-P5B

R3-IB2-P9B to R4-U1-P6B

5 meters

R3 IB2 to Rack 5

R3-IB2-P10A to R5-U1-P7B

5 meters

R3 IB2 to Rack 6

R3-IB2-P10B to R6-U1-P8B

10 meters

Table 5-18 shows the cable connections for the fourth spine switch (R4-U1) when cabling six full racks together:

Table 5-18 Leaf Switch Connections for the Fourth Rack in a Six-Rack System

Leaf Switch Connection Cable Length

R4 IB3 within Rack 4

R4-IB3-P8A to R4-U1-P3A

R4-IB3-P8B to R4-U1-P4A

3 meters

R4 IB3 to Rack 1

R4-IB3-P10B to R1-U1-P8A

10 meters

R4 IB3 to Rack 2

R4-IB3-P11A to R2-U1-P9A

5 meters

R4 IB3 to Rack 3

R4-IB3-P11B to R3-U1-P10A

5 meters

R4 IB3 to Rack 5

R4-IB3-P9A to R5-U1-P5A

R4-IB3-P9B to R5-U1-P6A

5 meters

R4 IB3 to Rack 6

R4-IB3-P10A to R6-U1-P7A

5 meters

R4 IB2 within Rack 4

R4-IB2-P8A to R4-U1-P3B

R4-IB2-P8B to R4-U1-P4B

3 meters

R4 IB2 to Rack 1

R4-IB2-P10B to R1-U1-P8B

10 meters

R4 IB2 to Rack 2

R4-IB2-P11A to R2-U1-P9B

5 meters

R4 IB2 to Rack 3

R4-IB2-P11B to R3-U1-P10B

5 meters

R4 IB2 to Rack 5

R4-IB2-P9A to R5-U1-P5B

R4-IB2-P9B to R5-U1-P6B

5 meters

R4 IB2 to Rack 6

R4-IB2-P10A to R6-U1-P7B

5 meters

Table 5-19 shows the cable connections for the fifth spine switch (R5-U1) when cabling six full racks together:

Table 5-19 Leaf Switch Connections for the Fifth Rack in a Six-Rack System

Leaf Switch Connection Cable Length

R5 IB3 within Rack 5

R5-IB3-P8A to R5-U1-P3A

R5-IB3-P8B to R5-U1-P4A

3 meters

R5 IB3 to Rack 1

R5-IB3-P10A to R1-U1-P7A

10 meters

R5 IB3 to Rack 2

R5-IB3-P10B to R2-U1-P8A

10 meters

R5 IB3 to Rack 3

R5-IB3-P11A to R3-U1-P9A

5 meters

R5 IB3 to Rack 4

R5-IB3-P11B to R4-U1-P10A

5 meters

R5 IB3 to Rack 6

R5-IB3-P9A to R6-U1-P5A

R5-IB3-P9B to R6-U1-P6A

5 meters

R5 IB2 within Rack 5

R5-IB2-P8A to R5-U1-P3B

R5-IB2-P8B to R5-U1-P4B

3 meters

R5 IB2 to Rack 1

R5-IB2-P10A to R1-U1-P7B

10 meters

R5 IB2 to Rack 2

R5-IB2-P10B to R2-U1-P8B

10 meters

R5 IB2 to Rack 3

R5-IB2-P11A to R3-U1-P9B

5 meters

R5 IB2 to Rack 4

R5-IB2-P11B to R4-U1-P10B

5 meters

R5 IB2 to Rack 6

R5-IB2-P9A to R6-U1-P5B

R5-IB2-P9B to R6-U1-P6B

5 meters

Table 5-20 shows the cable connections for the sixth spine switch (R6-U1) when cabling six full racks together:

Table 5-20 Leaf Switch Connections for the Sixth Rack in a Six-Rack System

Leaf Switch Connection Cable Length

R6 IB3 within Rack 6

R6-IB3-P8A to R6-U1-P3A

R6-IB3-P8B to R6-U1-P4A

3 meters

R6 IB3 to Rack 1

R6-IB3-P9A to R1-U1-P5A

R6-IB3-P9B to R1-U1-P6A

10 meters

R6 IB3 to Rack 2

R6-IB3-P10A to R2-U1-P7A

10 meters

R6 IB3 to Rack 3

R6-IB3-P10B to R3-U1-P8A

10 meters

R6 IB3 to Rack 4

R6-IB3-P11A to R4-U1-P9A

5 meters

R6 IB3 to Rack 5

R6-IB3-P11B to R5-U1-P10A

5 meters

R6 IB2 within Rack 6

R6-IB2-P8A to R6-U1-P3B

R6-IB2-P8B to R6-U1-P4B

3 meters

R6 IB2 to Rack 1

R6-IB2-P9A to R1-U1-P5B

R6-IB2-P9B to R1-U1-P6B

10 meters

R6 IB2 to Rack 2

R6-IB2-P10A to R2-U1-P7B

10 meters

R6 IB2 to Rack 3

R6-IB2-P10B to R3-U1-P8B

10 meters

R6 IB2 to Rack 4

R6-IB2-P11A to R4-U1-P9B

5 meters

R6 IB2 to Rack 5

R6-IB2-P11B to R5-U1-P10B

5 meters

5.7 Seven-Rack Cabling

Note the following when cabling five racks together:

  • The spine switch is in U1 (also referred to as IB1) for all racks.

  • In the following tables, the leaf switches are referred to as IB2 and IB3. Use the rack unit that is appropriate for your system.

  • In Oracle Exadata Database Machine X2-2 racks and Oracle Exadata Storage Expansion Racks with Exadata Storage Server with Sun Fire X4270 M2 Servers, the leaf switches are in U20 and U24, also referred to as IB2 and IB3, respectively.

  • In Oracle Exadata Database Machine X2-8 and later racks, the leaf switches are in U21 and U23, also referred to as IB2 and IB3, respectively.

  • In Oracle Exadata Database Machine X4-2 and later racks, or Oracle Exadata Database Machine X3-2 Full Racks and Oracle Exadata Storage Expansion Rack X3-2 racks, the leaf switches are in U20 and U22, also referred to as IB2 and IB3, respectively.

  • The cable lengths shown in the tables assume the racks are adjacent to each other. If the racks are not adjacent or use overhead cabling trays, then they may require longer cables lengths. Up to 100 meters is supported.

  • Only optical cables are supported for lengths greater than 5 meters.

Table 5-21 shows the cable connections for the first spine switch (R1-U1) when cabling seven racks together:

Table 5-21 Leaf Switch Connections for the First Rack in a Seven-Rack System

Leaf Switch Connection Cable Length

R1 IB3 within Rack 1

R1-IB3-P8A to R1-U1-P3A

R1-IB3-P8B to R1-U1-P4A

3 meters

R1 IB3 to Rack 2

R1-IB3-P9A to R2-U1-P5A

5 meters

R1 IB3 to Rack 3

R1-IB3-P9B to R3-U1-P6A

5 meters

R1 IB3 to Rack 4

R1-IB3-P10A to R4-U1-P7A

10 meters

R1 IB3 to Rack 5

R1-IB3-P10B to R5-U1-P8A

10 meters

R1 IB3 to Rack 6

R1-IB3-P11A to R6-U1-P9A

10 meters

R1 IB3 to Rack 7

R1-IB3-P11B to R7-U1-P10A

10 meters

R1 IB2 within Rack 1

R1-IB2-P8A to R1-U1-P3B

R1-IB2-P8B to R1-U1-P4B

3 meters

R1 IB2 to Rack 2

R1-IB2-P9A to R2-U1-P5B

5 meters

R1 IB2 to Rack 3

R1-IB2-P9B to R3-U1-P6B

5 meters

R1 IB2 to Rack 4

R1-IB2-P10A to R4-U1-P7B

10 meters

R1 IB2 to Rack 5

R1-IB2-P10B to R5-U1-P8B

10 meters

R1 IB2 to Rack 6

R1-IB2-P11A to R6-U1-P9B

10 meters

R1 IB2 to Rack 7

R1-IB2-P11B to R7-U1-P10B

10 meters

Table 5-22 shows the cable connections for the second spine switch (R2-U1) when cabling seven full racks together:

Table 5-22 Leaf Switch Connections for the Second Rack in a Seven-Rack System

Leaf Switch Connection Cable Length

R2 IB3 within Rack 2

R2-IB3-P8A to R2-U1-P3A

R2-IB3-P8B to R2-U1-P4A

3 meters

R2 IB3 to Rack 1

R2-IB3-P11B to R1-U1-P10A

5 meters

R2 IB3 to Rack 3

R2-IB3-P9A to R3-U1-P5A

5 meters

R2 IB3 to Rack 4

R2-IB3-P9B to R4-U1-P6A

5 meters

R2 IB3 to Rack 5

R2-IB3-P10A to R5-U1-P7A

10 meters

R2 IB3 to Rack 6

R2-IB3-P10B to R6-U1-P8A

10 meters

R2 IB3 to Rack 7

R2-IB3-P11A to R7-U1-P9A

10 meters

R2 IB2 within Rack 2

R2-IB2-P8A to R2-U1-P3B

R2-IB2-P8B to R2-U1-P4B

3 meters

R2 IB2 to Rack 1

R2-IB2-P11B to R1-U1-P10B

5 meters

R2 IB2 to Rack 3

R2-IB2-P9A to R3-U1-P5B

5 meters

R2 IB2 to Rack 4

R2-IB2-P9B to R4-U1-P6B

5 meters

R2 IB2 to Rack 5

R2-IB2-P10A to R5-U1-P7B

10 meters

R2 IB2 to Rack 6

R2-IB2-P10Bto R6-U1-P8B

10 meters

R2 IB2 to Rack 7

R2-IB2-P11A to R7-U1-P9B

10 meters

Table 5-23 shows the cable connections for the third spine switch (R3-U1) when cabling seven full racks together:

Table 5-23 Leaf Switch Connections for the Third Rack in a Seven-Rack System

Leaf Switch Connection Cable Length

R3 IB3 within Rack 3

R3-IB3-P8A to R3-U1-P3A

R3-IB3-P8B to R3-U1-P4A

3 meters

R3 IB3 to Rack 1

R3-IB3-P11A to R1-U1-P9A

5 meters

R3 IB3 to Rack 2

R3-IB3-P11B to R2-U1-P10A

5 meters

R3 IB3 to Rack 4

R3-IB3-P9A to R4-U1-P5A

5 meters

R3 IB3 to Rack 5

R3-IB3-P9B to R5-U1-P6A

5 meters

R3 IB3 to Rack 6

R3-IB3-P10A to R6-U1-P7A

10 meters

R3 IB3 to Rack 7

R3-IB3-P10B to R7-U1-P8A

10 meters

R3 IB2 within Rack 3

R3-IB2-P8A to R3-U1-P3B

R3-IB2-P8B to R3-U1-P4B

3 meters

R3 IB2 to Rack 1

R3-IB2-P11A to R1-U1-P9B

5 meters

R3 IB2 to Rack 2

R3-IB2-P11B to R2-U1-P10B

5 meters

R3 IB2 to Rack 4

R3-IB2-P9A to R4-U1-P5B

5 meters

R3 IB2 to Rack 5

R3-IB2-P9B to R5-U1-P6B

5 meters

R3 IB2 to Rack 6

R3-IB2-P10A to R6-U1-P7B

10 meters

R3 IB2 to Rack 7

R3-IB2-P10B to R7-U1-P8B

10 meters

Table 5-24 shows the cable connections for the fourth spine switch (R4-U1) when cabling seven full racks together:

Table 5-24 Leaf Switch Connections for the Fourth Rack in a Seven-Rack System

Leaf Switch Connection Cable Length

R4 IB3 within Rack 4

R4-IB3-P8A to R4-U1-P3A

R4-IB3-P8B to R4-U1-P4A

3 meters

R4 IB3 to Rack 1

R4-IB3-P10B to R1-U1-P8A

10 meters

R4 IB3 to Rack 2

R4-IB3-P11A to R2-U1-P9A

5 meters

R4 IB3 to Rack 3

R4-IB3-P11B to R3-U1-P10A

5 meters

R4 IB3 to Rack 5

R4-IB3-P9A to R5-U1-P5A

5 meters

R4 IB3 to Rack 6

R4-IB3-P9B to R6-U1-P6A

5 meters

R4 IB3 to Rack 7

R4-IB3-P10A to R7-U1-P7A

10 meters

R4 IB2 within Rack 4

R4-IB2-P8A to R4-U1-P3B

R4-IB2-P8B to R4-U1-P4B

3 meters

R4 IB2 to Rack 1

R4-IB2-P10B to R1-U1-P8B

10 meters

R4 IB2 to Rack 2

R4-IB2-P11A to R2-U1-P9B

5 meters

R4 IB2 to Rack 3

R4-IB2-P11B to R3-U1-P10B

5 meters

R4 IB2 to Rack 5

R4-IB2-P9A to R5-U1-P5B

5 meters

R4 IB2 to Rack 6

R4-IB2-P9B to R6-U1-P6B

5 meters

R4 IB2 to Rack 7

R4-IB2-P10A to R7-U1-P7B

10 meters

Table 5-25 shows the cable connections for the fifth spine switch (R5-U1) when cabling seven full racks together:

Table 5-25 Leaf Switch Connections for the Fifth Rack in a Seven-Rack System

Leaf Switch Connection Cable Length

R5 IB3 within Rack 5

R5-IB3-P8A to R5-U1-P3A

R5-IB3-P8B to R5-U1-P4A

3 meters

R5 IB3 to Rack 1

R5-IB3-P10A to R1-U1-P7A

10 meters

R5 IB3 to Rack 2

R5-IB3-P10B to R2-U1-P8A

10 meters

R5 IB3 to Rack 3

R5-IB3-P11A to R3-U1-P9A

5 meters

R5 IB3 to Rack 4

R5-IB3-P11B to R4-U1-P10A

5 meters

R5 IB3 to Rack 6

R5-IB3-P9A to R6-U1-P5A

5 meters

R5 IB3 to Rack 7

R5-IB3-P9B to R7-U1-P6A

5 meters

R5 IB2 within Rack 5

R5-IB2-P8A to R5-U1-P3B

R5-IB2-P8B to R5-U1-P4B

3 meters

R5 IB2 to Rack 1

R5-IB2-P10A to R1-U1-P7B

10 meters

R5 IB2 to Rack 2

R5-IB2-P10B to R2-U1-P8B

10 meters

R5 IB2 to Rack 3

R5-IB2-P11A to R3-U1-P9B

5 meters

R5 IB2 to Rack 4

R5-IB2-P11B to R4-U1-P10B

5 meters

R5 IB2 to Rack 6

R5-IB2-P9A to R6-U1-P5B

5 meters

R5 IB2 to Rack 7

R5-IB2-P9B to R7-U1-P6B

5 meters

Table 5-26 shows the cable connections for the sixth spine switch (R6-U1) when cabling seven full racks together:

Table 5-26 Leaf Switch Connections for the Sixth Rack in a Seven-Rack System

Leaf Switch Connection Cable Length

R6 IB3 within Rack 6

R6-IB3-P8A to R6-U1-P3A

R6-IB3-P8B to R6-U1-P4A

3 meters

R6 IB3 to Rack 1

R6-IB3-P9B to R1-U1-P6A

10 meters

R6 IB3 to Rack 2

R6-IB3-P10A to R2-U1-P7A

10 meters

R6 IB3 to Rack 3

R6-IB3-P10B to R3-U1-P8A

10 meters

R6 IB3 to Rack 4

R6-IB3-P11A to R4-U1-P9A

5 meters

R6 IB3 to Rack 5

R6-IB3-P11B to R5-U1-P10A

5 meters

R6 IB3 to Rack 7

R6-IB3-P9A to R7-U1-P5A

5 meters

R6 IB2 within Rack 6

R6-IB2-P8A to R6-U1-P3B

R6-IB2-P8B to R6-U1-P4B

3 meters

R6 IB2 to Rack 1

R6-IB2-P9B to R1-U1-P6B

10 meters

R6 IB2 to Rack 2

R6-IB2-P10A to R2-U1-P7B

10 meters

R6 IB2 to Rack 3

R6-IB2-P10B to R3-U1-P8B

10 meters

R6 IB2 to Rack 4

R6-IB2-P11A to R4-U1-P9B

5 meters

R6 IB2 to Rack 5

R6-IB2-P11B to R5-U1-P10B

5 meters

R6 IB2 to Rack 7

R6-IB2-P9A to R7-U1-P5B

5 meters

Table 5-27 shows the cable connections for the seventh spine switch (R7-U1) when cabling seven full racks together:

Table 5-27 Leaf Switch Connections for the Seventh Rack in a Seven-Rack System

Leaf Switch Connection Cable Length

R7 IB3 within Rack 7

R7-IB3-P8A to R7-U1-P3A

R7-IB3-P8B to R7-U1-P4A

3 meters

R7 IB3 to Rack 1

R7-IB3-P9A to R1-U1-P5A

10 meters

R7 IB3 to Rack 2

R7-IB3-P9B to R2-U1-P6A

10 meters

R7 IB3 to Rack 3

R7-IB3-P10A to R3-U1-P7A

10 meters

R7 IB3 to Rack 4

R7-IB3-P10B to R4-U1-P8A

10 meters

R7 IB3 to Rack 5

R7-IB3-P11A to R5-U1-P9A

5 meters

R7 IB3 to Rack 6

R7-IB3-P11B to R6-U1-P10A

5 meters

R7 IB2 within Rack 7

R7-IB2-P8A to R7-U1-P3B

R7-IB2-P8B to R7-U1-P4B

3 meters

R7 IB2 to Rack 1

R7-IB2-P9A to R1-U1-P5B

10 meters

R7 IB2 to Rack 2

R7-IB2-P9B to R2-U1-P6B

10 meters

R7 IB2 to Rack 3

R7-IB2-P10A to R3-U1-P7B

10 meters

R7 IB2 to Rack 4

R7-IB2-P10B to R4-U1-P8B

10 meters

R7 IB2 to Rack 5

R7-IB2-P11A to R5-U1-P9B

5 meters

R7 IB2 to Rack 6

R7-IB2-P11B to R6-U1-P10B

5 meters

5.8 Eight-Rack Cabling

Note the following when cabling five racks together:

  • The spine switch is in U1 (also referred to as IB1) for all racks.

  • In the following tables, the leaf switches are referred to as IB2 and IB3. Use the rack unit that is appropriate for your system.

  • In Oracle Exadata Database Machine X2-2 racks and Oracle Exadata Storage Expansion Racks with Exadata Storage Server with Sun Fire X4270 M2 Servers, the leaf switches are in U20 and U24, also referred to as IB2 and IB3, respectively.

  • In Oracle Exadata Database Machine X2-8 and later racks, the leaf switches are in U21 and U23, also referred to as IB2 and IB3, respectively.

  • In Oracle Exadata Database Machine X4-2 and later racks, or Oracle Exadata Database Machine X3-2 Full Racks and Oracle Exadata Storage Expansion Rack X3-2 racks, the leaf switches are in U20 and U22, also referred to as IB2 and IB3, respectively.

  • The cable lengths shown in the tables assume the racks are adjacent to each other. If the racks are not adjacent or use overhead cabling trays, then they may require longer cables lengths. Up to 100 meters is supported.

  • Only optical cables are supported for lengths greater than 5 meters.

Table 5-28 shows the cable connections for the first spine switch (R1-U1) when cabling eight racks together:

Table 5-28 Leaf Switch Connections for the First Rack in a Eight-Rack System

Leaf Switch Connection Cable Length

R1 IB3 within Rack 1

R1-IB3-P8A to R1-U1-P3A

3 meters

R1 IB3 to Rack 2

R1-IB3-P8B to R2-U1-P4A

5 meters

R1 IB3 to Rack 3

R1-IB3-P9A to R3-U1-P5A

5 meters

R1 IB3 to Rack 4

R1-IB3-P9B to R4-U1-P6A

10 meters

R1 IB3 to Rack 5

R1-IB3-P10A to R5-U1-P7A

10 meters

R1 IB3 to Rack 6

R1-IB3-P10B to R6-U1-P8A

10 meters

R1 IB3 to Rack 7

R1-IB3-P11A to R7-U1-P9A

10 meters

R1 IB3 to Rack 8

R1-IB3-P11B to R8-U1-P10A

10 meters

R1 IB2 within Rack 1

R1-IB2-P8A to R1-U1-P3B

3 meters

R1 IB2 to Rack 2

R1-IB2-P8B to R2-U1-P4B

5 meters

R1 IB2 to Rack 3

R1-IB2-P9A to R3-U1-P5B

5 meters

R1 IB2 to Rack 4

R1-IB2-P9B to R4-U1-P6B

10 meters

R1 IB2 to Rack 5

R1-IB2-P10A to R5-U1-P7B

10 meters

R1 IB2 to Rack 6

R1-IB2-P10B to R6-U1-P8B

10 meters

R1 IB2 to Rack 7

R1-IB2-P11A to R7-U1-P9B

10 meters

R1 IB2 to Rack 8

R1-IB2-P11B to R8-U1-P10B

10 meters

Table 5-29 shows the cable connections for the second spine switch (R2-U1) when cabling eight full racks together:

Table 5-29 Leaf Switch Connections for the Second Rack in a Eight-Rack System

Leaf Switch Connection Cable Length

R2 IB3 within Rack 2

R2-IB3-P8A to R2-U1-P3A

3 meters

R2 IB3 to Rack 1

R2-IB3-P11B to R1-U1-P10A

5 meters

R2 IB3 to Rack 3

R2-IB3-P8B to R3-U1-P4A

5 meters

R2 IB3 to Rack 4

R2-IB3-P9A to R4-U1-P5A

5 meters

R2 IB3 to Rack 5

R2-IB3-P9B to R5-U1-P6A

10 meters

R2 IB3 to Rack 6

R2-IB3-P10A to R6-U1-P7A

10 meters

R2 IB3 to Rack 7

R2-IB3-P10B to R7-U1-P8A

10 meters

R2 IB3 to Rack 8

R2-IB3-P11A to R8-U1-P9A

10 meters

R2 IB2 within Rack 2

R2-IB2-P8A to R2-U1-P3B

3 meters

R2 IB2 to Rack 1

R2-IB2-P11B to R1-U1-P10B

5 meters

R2 IB2 to Rack 3

R2-IB2-P8B to R3-U1-P4B

5 meters

R2 IB2 to Rack 4

R2-IB2-P9A to R4-U1-P5B

5 meters

R2 IB2 to Rack 5

R2-IB2-P9B to R5-U1-P6B

10 meters

R2 IB2 to Rack 6

R2-IB2-P10A to R6-U1-P7B

10 meters

R2 IB2 to Rack 7

R2-IB2-P10B to R7-U1-P8B

10 meters

R2 IB2 to Rack 8

R2-IB2-P11A to R8-U1-P9B

10 meters

Table 5-30 shows the cable connections for the third spine switch (R3-U1) when cabling eight full racks together:

Table 5-30 Leaf Switch Connections for the Third Rack in a Eight-Rack System

Leaf Switch Connection Cable Length

R3 IB3 within Rack 3

R3-IB3-P8A to R3-U1-P3A

3 meters

R3 IB3 to Rack 1

R3-IB3-P11A to R1-U1-P9A

5 meters

R3 IB3 to Rack 2

R3-IB3-P11B to R2-U1-P10A

5 meters

R3 IB3 to Rack 4

R3-IB3-P8B to R4-U1-P4A

5 meters

R3 IB3 to Rack 5

R3-IB3-P9A to R5-U1-P5A

5 meters

R3 IB3 to Rack 6

R3-IB3-P9B to R6-U1-P6A

10 meters

R3 IB3 to Rack 7

R3-IB3-P10A to R7-U1-P7A

10 meters

R3 IB3 to Rack 8

R3-IB3-P10B to R8-U1-P8A

10 meters

R3 IB2 within Rack 3

R3-IB2-P8A to R3-U1-P3B

3 meters

R3 IB2 to Rack 1

R3-IB2-P11A to R1-U1-P9B

5 meters

R3 IB2 to Rack 2

R3-IB2-P11B to R2-U1-P10B

5 meters

R3 IB2 to Rack 4

R3-IB2-P8B to R4-U1-P4B

5 meters

R3 IB2 to Rack 5

R3-IB2-P9A to R5-U1-P5B

5 meters

R3 IB2 to Rack 6

R3-IB2-P9B to R6-U1-P6B

10 meters

R3 IB2 to Rack 7

R3-IB2-P10A to R7-U1-P7B

10 meters

R3 IB2 to Rack 8

R3-IB2-P10B to R8-U1-P8B

10 meters

Table 5-31 shows the cable connections for the fourth spine switch (R4-U1) when cabling eight full racks together:

Table 5-31 Leaf Switch Connections for the Fourth Rack in a Eight-Rack System

Leaf Switch Connection Cable Length

R4 IB3 within Rack 4

R4-IB3-P8A to R4-U1-P3A

3 meters

R4 IB3 to Rack 1

R4-IB3-P10B to R1-U1-P8A

10 meters

R4 IB3 to Rack 2

R4-IB3-P11A to R2-U1-P9A

5 meters

R4 IB3 to Rack 3

R4-IB3-P11B to R3-U1-P10A

5 meters

R4 IB3 to Rack 5

R4-IB3-P8B to R5-U1-P4A

5 meters

R4 IB3 to Rack 6

R4-IB3-P9A to R6-U1-P5A

5 meters

R4 IB3 to Rack 7

R4-IB3-P9B to R7-U1-P6A

10 meters

R4 IB3 to Rack 8

R4-IB3-P10A to R8-U1-P7A

10 meters

R4 IB2 within Rack 4

R4-IB2-P8A to R4-U1-P3B

3 meters

R4 IB2 to Rack 1

R4-IB2-P10B to R1-U1-P8B

10 meters

R4 IB2 to Rack 2

R4-IB2-P11A to R2-U1-P9B

5 meters

R4 IB2 to Rack 3

R4-IB2-P11B to R3-U1-P10B

5 meters

R4 IB2 to Rack 5

R4-IB2-P8B to R5-U1-P4B

5 meters

R4 IB2 to Rack 6

R4-IB2-P9A to R6-U1-P5B

5 meters

R4 IB2 to Rack 7

R4-IB2-P9B to R7-U1-P6B

10 meters

R4 IB2 to Rack 8

R4-IB2-P10A to R8-U1-P7B

10 meters

Table 5-32 shows the cable connections for the fifth spine switch (R5-U1) when cabling eight full racks together:

Table 5-32 Leaf Switch Connections for the Fifth Rack in a Eight-Rack System

Leaf Switch Connection Cable Length

R5 IB3 within Rack 5

R5-IB3-P8A to R5-U1-P3A

3 meters

R5 IB3 to Rack 1

R5-IB3-P10A to R1-U1-P7A

10 meters

R5 IB3 to Rack 2

R5-IB3-P10B to R2-U1-P8A

10 meters

R5 IB3 to Rack 3

R5-IB3-P11A to R3-U1-P9A

5 meters

R5 IB3 to Rack 4

R5-IB3-P11B to R4-U1-P10A

5 meters

R5 IB3 to Rack 6

R5-IB3-P8B to R6-U1-P4A

5 meters

R5 IB3 to Rack 7

R5-IB3-P9A to R7-U1-P5A

5 meters

R5 IB3 to Rack 8

R5-IB3-P9B to R8-U1-P6A

10 meters

R5 IB2 within Rack 5

R5-IB2-P8A to R5-U1-P3B

3 meters

R5 IB2 to Rack 1

R5-IB2-P10A to R1-U1-P7B

10 meters

R5 IB2 to Rack 2

R5-IB2-P10B to R2-U1-P8B

10 meters

R5 IB2 to Rack 3

R5-IB2-P11A to R3-U1-P9B

5 meters

R5 IB2 to Rack 4

R5-IB2-P11B to R4-U1-P10B

5 meters

R5 IB2 to Rack 6

R5-IB2-P8B to R6-U1-P4B

5 meters

R5 IB2 to Rack 7

R5-IB2-P9A to R7-U1-P5B

5 meters

R5 IB2 to Rack 8

R5-IB2-P9B to R8-U1-P6B

10 meters

Table 5-33 shows the cable connections for the sixth spine switch (R6-U1) when cabling eight full racks together:

Table 5-33 Leaf Switch Connections for the Sixth Rack in a Eight-Rack System

Leaf Switch Connection Cable Length

R6 IB3 within Rack 6

R6-IB3-P8A to R6-U1-P3A

3 meters

R6 IB3 to Rack 1

R6-IB3-P9B to R1-U1-P6A

10 meters

R6 IB3 to Rack 2

R6-IB3-P10A to R2-U1-P7A

10 meters

R6 IB3 to Rack 3

R6-IB3-P10B to R3-U1-P8A

10 meters

R6 IB3 to Rack 4

R6-IB3-P11A to R4-U1-P9A

5 meters

R6 IB3 to Rack 5

R6-IB3-P11B to R5-U1-P10A

5 meters

R6 IB3 to Rack 7

R6-IB3-P8B to R7-U1-P4A

5 meters

R6 IB3 to Rack 8

R6-IB3-P9A to R8-U1-P5A

5 meters

R6 IB2 within Rack 6

R6-IB2-P8A to R6-U1-P3B

3 meters

R6 IB2 to Rack 1

R6-IB2-P9B to R1-U1-P6B

10 meters

R6 IB2 to Rack 2

R6-IB3-P10A to R2-U1-P7B

10 meters

R6 IB2 to Rack 3

R6-IB2-P10B to R3-U1-P8B

10 meters

R6 IB2 to Rack 4

R6-IB2-P11A to R4-U1-P9B

5 meters

R6 IB2 to Rack 5

R6-IB2-P11B to R5-U1-P10B

5 meters

R6 IB2 to Rack 7

R6-IB2-P8B to R7-U1-P4B

5 meters

R6 IB2 to Rack 8

R6-IB2-P9A to R8-U1-P5B

5 meters

Table 5-34 shows the cable connections for the seventh spine switch (R7-U1) when cabling eight full racks together:

Table 5-34 Leaf Switch Connections for the Seventh Rack in a Eight-Rack System

Leaf Switch Connection Cable Length

R7 IB3 within Rack 7

R7-IB3-P8A to R7-U1-P3A

3 meters

R7 IB3 to Rack 1

R7-IB3-P9A to R1-U1-P5A

10 meters

R7 IB3 to Rack 2

R7-IB3-P9B to R2-U1-P6A

10 meters

R7 IB3 to Rack 3

R7-IB3-P10A to R3-U1-P7A

10 meters

R7 IB3 to Rack 4

R7-IB3-P10B to R4-U1-P8A

10 meters

R7 IB3 to Rack 5

R7-IB3-P11A to R5-U1-P9A

5 meters

R7 IB3 to Rack 6

R7-IB3-P11B to R6-U1-P10A

5 meters

R7 IB3 to Rack 8

R7-IB3-P8B to R8-U1-P4A

5 meters

R7 IB2 within Rack 7

R7-IB2-P8A to R7-U1-P3B

3 meters

R7 IB2 to Rack 1

R7-IB2-P9A to R1-U1-P5B

10 meters

R7 IB2 to Rack 2

R7-IB2-P9B to R2-U1-P6B

10 meters

R7 IB2 to Rack 3

R7-IB2-P10A to R3-U1-P7B

10 meters

R7 IB2 to Rack 4

R7-IB2-P10B to R4-U1-P8B

10 meters

R7 IB2 to Rack 5

R7-IB2-P11A to R5-U1-P9B

5 meters

R7 IB2 to Rack 6

R7-IB2-P11B to R6-U1-P10B

5 meters

R7 IB2 to Rack 8

R7-IB2-P8B to R8-U1-P4B

5 meters

Table 5-35 shows the cable connections for the eighth spine switch (R8-U1) when cabling eight full racks together:

Table 5-35 Leaf Switch Connections for the Eighth Rack in a Eight-Rack System

Leaf Switch Connection Cable Length

R8 IB3 within Rack 8

R8-IB3-P8A to R8-U1-P3A

3 meters

R8 IB3 to Rack 1

R8-IB3-P8B to R1-U1-P4A

10 meters

R8 IB3 to Rack 2

R8-IB3-P9A to R2-U1-P5A

10 meters

R8 IB3 to Rack 3

R8-IB3-P9B to R3-U1-P6A

10 meters

R8 IB3 to Rack 4

R8-IB3-P10A to R4-U1-P7A

10 meters

R8 IB3 to Rack 5

R8-IB3-P10B to R5-U1-P8A

10 meters

R8 IB3 to Rack 6

R8-IB3-P11A to R6-U1-P9A

5 meters

R8 IB3 to Rack 7

R8-IB3-P11B to R7-U1-P10A

5 meters

R8 IB2 within Rack 8

R8-IB2-P8A to R8-U1-P3B

3 meters

R8 IB2 to Rack 1

R8-IB2-P8B to R1-U1-P4B

10 meters

R8 IB2 to Rack 2

R8-IB2-P9A to R2-U1-P5B

10 meters

R8 IB2 to Rack 3

R8-IB2-P9B to R3-U1-P6B

10 meters

R8 IB2 to Rack 4

R8-IB2-P10A to R4-U1-P7B

10 meters

R8 IB2 to Rack 5

R8-IB2-P10B to R5-U1-P8B

10 meters

R8 IB2 to Rack 6

R8-IB2-P11A to R6-U1-P9B

5 meters

R8 IB2 to Rack 7

R8-IB2-P11B to R7-U1-P10B

5 meters