Cable Indirect-Attached Rack Servers (QDR)
To create a merged IB fabric for rack servers through indirect connections, connect one or more ports on each Oracle Fabric Interconnect to one or more ports on intervening IB switches.
When connected correctly, the Oracle Fabric Interconnects are meshed through the connections to the IB switches. The IB switches then enable the network to “fan out” to support the hosts.
To create a merged fabric of rack servers through indirect connections, follow this procedure:
- Make sure that the subnet manager is shut down and the LID table is removed.
- Connect a QDR IB cable from one port on the first Oracle Fabric Interconnect (1) to an IB port on the first IB switch.
- Connect a QDR IB cable from one port on the first Oracle Fabric Interconnect (1) to an IB port on the second IB switch.
- Connect a QDR IB cable from one port on the second Oracle Fabric Interconnect (2) to an IB port on the first IB switch.
- Connect a QDR IB cable from one port on the second Oracle Fabric Interconnect (2) to an IB port on the second IB switch.
- On each host server, connect a QDR IB cable from one server HCA port to an IB port on the first IB switch.
-
On each host server, connect a QDR IB cable from one server HCA port to an
IB port on the second IB switch.
At the completion of this step, the Oracle Fabric Interconnects and servers are connected as shown.
No.Description1, 2Oracle Fabric Interconnect F1-15s (or F1-4s) indirectly connected through intermediary QDR IB switches.3QDR speed connections (interswitch links, ISLs) connect the Oracle Fabric Interconnects together through the IB switches.4, 5Intermediary QDR speed IB switches provide IB fan out, and connect host servers to the IB fabric.6, 7 8, 9Host servers with QDR-capable HCAs directly connected to the intermediary IB switches over QDR links. - Complete merging the fabric.