Connect the Datacenters

To support switch mode's longer distances, the switches must be fully meshed, except for the leaf switches which should not directly connect to each other. As you perform this task, you will find it helpful to refer to the diagram in Switch-Mode Supported Topology.

To connect the datacenters, you connect the switches within both datacenters, then connect the datacenters together over the IB LR links, which are single-mode fiber cables.

1. Before beginning this procedure, read the configuration considerations:

   • See Switch-Mode Topology Considerations

   • See Switch-Mode Configuration Considerations

2. In one datacenter, connect the virtualization switches together:
   1. Connect two magenta 4x IB cables to ports 5 and 6 on the virtualization switch.
   2. Connect the other end of the cables to the same ports on the other virtualization switch.
3. Connect the leaf switches to the virtualization switches:
   1. Connect two black 12x IB cables to any two ports (1 through 4) on the leaf switch.
   2. Connect the other end of the cables to any 12x IB ports on the virtualization switch.
   3. Repeat these steps until each leaf switch is cross-connected to both virtualization switches.

      ---

      Caution  -  Do not directly connect the leaf switches to each other.

      ---

4. Connect each host to a 4x IB port on one leaf switch.
5. Connect each host to a 4x IB port on the other leaf switch.
6. In the other datacenter, repeat Step 2 through Step 5.
7. Connect the two datacenters:
   1. Connect one single-mode fiber cable to one of the ports on the IB LR module.
   2. Connect the other end of the cable to a port on an IB LR Module in one of the virtualization switches in the other datacenter.
   3. Connect a second single-mode fiber cable to one of the ports on the IB LR module.
   4. Connect the other end of the second cable to a port on an IB LR module in the other virtualization switch in the other datacenter.
   5. Repeat these steps to connect the IB LR links until each virtualization switch has two links connected, one link to each of the other virtualization switches in the other datacenter.

      ---

      Note -  At this point, each virtualization switch should be connected to the other two virtualization switches in the other datacenter.

      ---

8. In each datacenter, connect to the respective management network.

   When connecting to datacenter management networks:

   • Each leaf and virtualization switch must be connected to the datacenter management network in both data centers.

   • It must be possible to communicate between the two data centers through the management network.