InfiniBand Private Network Physical Connections (SPARC T5-8 Servers)

Each SPARC T5-8 server contains several dual-ported Sun QDR InfiniBand PCIe Low Profile host channel adapters (HCAs). The number of InfiniBand HCAs and their locations in the SPARC T5-8 servers varies, depending on the configuration of Oracle SuperCluster T5-8:

• Full Rack: Eight InfiniBand HCAs, installed in these PCIe slots:

  • PCIe slot 3

  • PCIe slot 4

  • PCIe slot 7

  • PCIe slot 8

  • PCIe slot 11

  • PCIe slot 12

  • PCIe slot 15

  • PCIe slot 16

• Half Rack: Four InfiniBand HCAs, installed in these PCIe slots:

  • PCIe slot 3

  • PCIe slot 8

  • PCIe slot 11

  • PCIe slot 16

See Card Locations (SPARC T5-8 Servers) for more information on the location of the InfiniBand HCAs.

The two ports in each InfiniBand HCA (ports 1 and 2) connect to a different leaf switch to provide redundancy between the SPARC T5-8 servers and the leaf switches. The following figures show how redundancy is achieved with the InfiniBand connections between the SPARC T5-8 servers and the leaf switches in the Full Rack and Half Rack configurations.

Figure 8  InfiniBand Connections for SPARC T5-8 Servers, Full Rack

Figure 9  InfiniBand Connections for SPARC T5-8 Servers, Half Rack

Note that only the physical connections for the InfiniBand private network are described in this section. Once the logical domains are created for each SPARC T5-8 server, the InfiniBand private network will be configured differently depending on the type of domain created on the SPARC T5-8 servers. The number of IP addresses needed for the InfiniBand network will also vary, depending on the type of domains created on each SPARC T5-8 server. For more information, see Understanding the Software Configurations.