Cable Indirect-Attached Rack Servers (QDR)
To create a merged IB fabric for rack servers through indirect connections,
connect one or more ports on each Oracle Fabric Interconnect to one or more
ports on intervening IB switches.
When connected correctly, the Oracle Fabric Interconnects are meshed through
the connections to the IB switches. The IB switches then enable the network to
“fan out” to support the hosts.
To create a merged fabric of rack servers through indirect connections, follow
this procedure:
-
Make sure that the subnet manager is shut down and the LID table is
removed.
See Preparing to Merge an IB Fabric (QDR).
-
Connect a QDR IB cable from one port on the first Oracle Fabric
Interconnect (1) to an IB port on the first IB switch.
-
Connect a QDR IB cable from one port on the first Oracle Fabric
Interconnect (1) to an IB port on the second IB switch.
-
Connect a QDR IB cable from one port on the second Oracle Fabric
Interconnect (2) to an IB port on the first IB switch.
-
Connect a QDR IB cable from one port on the second Oracle Fabric
Interconnect (2) to an IB port on the second IB switch.
-
On each host server, connect a QDR IB cable from one server HCA port to an
IB port on the first IB switch.
-
On each host server, connect a QDR IB cable from one server HCA port to an
IB port on the second IB switch.
At the completion of this step, the Oracle Fabric Interconnects and
servers are connected as shown.
|
|
1, 2
|
Oracle Fabric Interconnect F1-15s (or F1-4s)
indirectly connected through intermediary QDR IB
switches.
|
3
|
QDR speed connections (interswitch links, ISLs)
connect the Oracle Fabric Interconnects together through
the IB switches.
|
4, 5
|
Intermediary QDR speed IB switches provide IB fan out,
and connect host servers to the IB fabric.
|
6, 7 8, 9
|
Host servers with QDR-capable HCAs directly connected
to the intermediary IB switches over QDR links.
|
|
-
Complete merging the fabric.
See Complete the Merged Fabric (QDR).