InfiniBand Fabric

The following tables show InfiniBand (IB) connections for these switches.

• IB Spine switch, see Table 14

• IB leaf switch No. 1

  • Full rack, see Table 15

  • Half rack, see Table 16

• IB leaf switch No. 2

  • Full rack, see Table 17

  • Half rack, see Table 18

IB Spine Switch

Table 14  IB Spine Switch (U1) Connections

Spine Switch Port To Device Device Location Device Port U1 1B Leaf switch 1 U18 8B U1 0B Leaf switch 2 U24 8B

IB Leaf Switch No. 1

Table 15  Full Rack Connections for IB Leaf Switch 1 (U18)

Leaf Switch 1 Port To Device Device Location Device Port U18 0A T4-4 No. 4 U32 PCIE-12 P2 U18 0B T4-4 No. 4 U32 PCIE-9 P2 U18 1A T4-4 No. 4 U32 PCIE-7 P2 U18 1B T4-4 No. 4 U32 PCIE-1 P2 U18 2A — — — U18 2B — — — U18 3A Sun ZFS Storage 7320 storage controllers No. 2 U26 PCIE-0 P1 U18 3B T4-4 No. 2 U13 PCIE-12 P1 U18 4A T4-4 No. 2 U13 PCIE-9 P1 U18 4B T4-4 No. 2 U13 PCIE-7 P1 U18 5A T4-4 No. 2 U13 PCIE-1 P1 U18 5B T4-4 No. 3 U27 PCIE-12 P2 U18 6A T4-4 No. 3 U27 PCIE-9 P2 U18 6B T4-4 No. 3 U27 PCIE-7 P2 U18 7A T4-4 No. 3 U27 PCIE-1 P2 U18 7B — — — U18 8A IB leaf switch 2 U24 8A U18 8B IB spine switch U1 1B U18 9A IB leaf switch 2 U24 9B U18 9B IB leaf switch 2 U24 9A U18 10A IB leaf switch 2 U24 10B U18 10B IB leaf switch 2 U24 10A U18 11A IB leaf switch 2 U24 11B U18 11B IB leaf switch 2 U24 11A U18 12A — — — U18 12B T4-4 No. 1 U8 PCIE-12 P1 U18 13A T4-4 No. 1 U8 PCIE-9 P1 U18 13B T4-4 No. 1 U8 PCIE-7 P1 U18 14A T4-4 No. 1 U8 PCIE-1 P1 U18 14B Sun ZFS Storage 7320 storage controller No. 1 U25 PCIE-0 P1 U18 15A Exadata Storage Server No. 6 U41 PCIE-3 P2 U18 15B Exadata Storage Server No. 5 U39 PCIE-3 P2 U18 16A Exadata Storage Server No. 4 U37 PCIE-3 P2 U18 16B Exadata Storage Server No. 3 U6 PCIE-3 P1 U18 17A Exadata Storage Server No. 2 U4 PCIE-3 P1 U18 17B Exadata Storage Server No. 1 U2 PCIE-3 P1

Table 16  Half Rack Connections for IB Leaf Switch No. 1 (U18)

Leaf Switch 1 Port To Device Device Location Device Port U18 0A — — — U18 0B — — — U18 1A — — — U18 1B — — — U18 2A — — — U18 2B — — — U18 3A Sun ZFS Storage 7320 storage controllers No. 2 U26 PCIE-0 P1 U18 3B T4-4 No. 2 U13 PCIE-12 P1 U18 4A T4-4 No. 2 U13 PCIE-9 P1 U18 4B T4-4 No. 2 U13 PCIE-7 P1 U18 5A T4-4 No. 2 U13 PCIE-1 P1 U18 5B — — — U18 6A — — — U18 6B — — — U18 7A — — — U18 7B — — — U18 8A IB leaf switch No. 2 U24 8A U18 8B IB spine switch U1 1B U18 9A IB leaf switch No. 2 U24 9B U18 9B IB leaf switch No. 2 U24 9A U18 10A IB leaf switch No. 2 U24 10B U18 10B IB leaf switch No. 2 U24 10A U18 11A IB leaf switch No. 2 U24 11B U18 11B IB leaf switch No. 2 U24 11A U18 12A — — — U18 12B T4-4 No. 1 U8 PCIE-12 P1 U18 13A T4-4 No. 1 U8 PCIE-9 P1 U18 13B T4-4 No. 1 U8 PCIE-7 P1 U18 14A T4-4 No. 1 U8 PCIE-1 P1 U18 14B Sun ZFS Storage 7320 storage controller No. 1 U25 PCIE-0 P1 U18 15A — — — U18 15B — — — U18 16A — — — U18 16B Exadata Storage Server No. 3 U6 PCIE-3 P1 U18 17A Exadata Storage Server No. 2 U4 PCIE-3 P1 U18 17B Exadata Storage Server No. 1 U2 PCIE-3 P1

IB Leaf Switch No. 2

Table 17  Full Rack Connections for IB Leaf Switch No. 2 (U24)

Leaf Switch 2 Port To Device Device Location Device Port U24 0A T4-4 No. 4 U32 PCIE-12 P1 U24 0B T4-4 No. 4 U32 PCIE-9 P1 U24 1A T4-4 No. 4 U32 PCIE-7 P1 U24 1B T4-4 No. 4 U32 PCIE-1 P1 U24 2A — — — U24 2B — — — U24 3A Sun ZFS Storage 7320 storage controller No. 2 U26 PCIE-0 P2 U24 3B T4-4 No. 2 U13 PCIE-12 P2 U24 4A T4-4 No. 2 U13 PCIE-9 P2 U24 4B T4-4 No. 2 U13 PCIE-7 P2 U24 5A T4-4 No. 2 U13 PCIE-1 P2 U24 5B T4-4 No. 3 U27 PCIE-12 P1 U24 6A T4-4 No. 3 U27 PCIE-9 P1 U24 6B T4-4 No. 3 U27 PCIE-7 P1 U24 7A T4-4 No. 3 U27 PCIE-1 P1 U24 7B — — — U24 8A IB leaf switch No. 1 U18 8A U24 8B IB spine switch U1 0B U24 9A IB leaf switch No. 1 U18 9B U24 9B IB leaf switch No. 1 U18 9A U24 10A IB leaf switch No. 1 U18 10B U24 10B IB leaf switch No. 1 U18 10A U24 11A IB leaf switch No. 1 U18 11B U24 11B IB leaf switch No. 1 U18 11A U24 12A — — — U24 12B T4-4 No. 1 U8 PCIE-12 P2 U24 13A T4-4 No. 1 U8 PCIE-9 P2 U24 13B T4-4 No. 1 U8 PCIE-7 P2 U24 14A T4-4 No. 1 U8 PCIE-1 P2 U24 14B Sun ZFS Storage 7320 storage controller No. 1 U25 PCIE-0 P2 U24 15A Exadata Storage Server No. 6 U41 PCIE 3 P1 U24 15B Exadata Storage Server No. 5 U39 PCIE 3 P1 U24 16A Exadata Storage Server No. 4 U37 PCIE 3 P1 U24 16B Exadata Storage Server No. 3 U6 PCIE-3 P2 U24 17A Exadata Storage Server No. 2 U4 PCIE-3 P2 U24 17B Exadata Storage Server No. 1 U2 PCIE-3 P2

Table 18  Half Rack Connections for IB Leaf Switch No. 2 (U24)

Leaf Switch 2 Port To Device Device Location Device Port U24 0A — — — U24 0B — — — U24 1A — — — U24 1B — — — U24 2A — — — U24 2B — — — U24 3A Sun ZFS Storage 7320 storage controller No. 2 U26 PCIE-0 P2 U24 3B T4-4 No. 2 U13 PCIE-12 P2 U24 4A T4-4 No. 2 U13 PCIE-9 P2 U24 4B T4-4 No. 2 U13 PCIE-7 P2 U24 5A T4-4 No. 2 U13 PCIE-1 P2 U24 5B — — — U24 6A — — — U24 6B — — — U24 7A — — — U24 7B — — — U24 8A IB leaf switch No. 1 U18 8A U24 8B IB spine switch U1 0B U24 9A IB leaf switch No. 1 U18 9B U24 9B IB leaf switch No. 1 U18 9A U24 10A IB leaf switch No. 1 U18 10B U24 10B IB leaf switch No. 1 U18 10A U24 11A IB leaf switch No. 1 U18 11B U24 11B IB leaf switch No. 1 U18 11A U24 12A — — — U24 12B T4-4 No. 1 U8 PCIE-12 P2 U24 13A T4-4 No. 1 U8 PCIE-9 P2 U24 13B T4-4 No. 1 U8 PCIE-7 P2 U24 14A T4-4 No. 1 U8 PCIE-1 P2 U24 14B Sun ZFS Storage 7320 storage controller No. 1 U25 PCIE-0 P2 U24 15A — — — U24 15B — — — U24 16A — — — U24 16B Exadata Storage Server No. 3 U6 PCIE-3 P2 U24 17A Exadata Storage Server No. 2 U4 PCIE-3 P2 U24 17B Exadata Storage Server No. 1 U2 PCIE-3 P2