Cable Indirect-Attached Servers (EDR)
Use this procedure if your host servers will be indirectly attached through an intervening layer of leaf switches. This configuration is also called an IB "fan out" in which hosts are connected to a pair of leaf switches, which are in turn connected to spine switches. It is a best practice to always connect to redundant switches to ensure HA.
This procedure assumes you will use the 12x IB ports as the Inter-Switch Links (ISLs). You can use 4x IB ports for ISLs, but as your fabric grows, be aware that you might saturate the 4x ISLs and might need to recable the ISLs to the 12x ports to enable higher throughput.
-
Verify the cable types, plugs, and color codes with the ports and color
codes on the switch.
If there is a mismatch, determine the correct type of cable and port before proceeding. See EDR Cables Overview.
- Connect one IB cable from one 12x port on one leaf switch to a 12x port on the first spine switch.
- Connect one IB cable from one 12x port on leaf switch to a 12x port on the second spine switch.
- Connect one IB cable from one 12x port on the other leaf switch to a 12x port on the first spine switch.
-
Connect one IB cable from one 12x port on the other leaf switch to an 12x
IB port on the spine switch.
At this point of the procedure, the leaf switches should have redundant cross-connections to each of the spine switches. Each of these connections should be on the 12x ports.
-
Attach the hosts to the remaining 4x ports on each leaf switch as needed.
At the completion of this task, the switches are connected as shown.
No.Description1Spine switches.2Leaf switches.3Host servers with EDR-capable HCAs. Hosts must be running a supported OS.412x EDR ports on the spine switches, which provide connections to the leaf switches.54x EDR ports on the leaf switches, which provide connections to the hosts.612x EDR ports on the leaf switches, which provide connections to the spine switches.74x EDR ports on the host's EDR HCA, which provide connections to the switches.