5.1.2 Cabling Multiple Oracle Exadata X8M Racks

A spine switch must exist in the Oracle Exadata rack, regardless of size, to multi-rack with other racks.

Note:

To extend Oracle Exadata racks with RoCE Network Fabric, you must order cables, transceivers if needed, and a RoCE Network Fabric switch, if they are not already available.

In the following steps, the number in parentheses also indicates the number of cables required.

1. Split each leaf switch uplink bundle by the number of spine switches (or racks) in such a way that the total count adds up to 8. This split is represented in parentheses for each example.

   Example 1: For two racks, take 8 uplinks from each leaf switch and split evenly by 2. Four uplinks from each leaf switch go to rack1-spine switch, and four uplinks from each leaf switch go to rack2-spine switch ( 4 + 4 for each leaf switch).

   Example 2: For three racks, take 8 uplinks from each leaf switch and split evenly by 3. Three uplinks go to rack1-spine switch, three uplinks go to rack2-spine switch, two uplinks go to rack3-spine switch ( 3 + 3 + 2 for each leaf switch).

2. Starting from the first available port on a different spine switch, round-robin the above split scheme for each leaf switch and spine switch.

   For example, for three racks:

   • rack1-leaf1 switch starts from rack1-spine switch for first split, rack2-spine switch for second split, rack3-spine switch for third split
   • rack2-leaf1 switch starts from rack2-spine switch for first split, rack3-spine switch for second split, rack1-spine for third split
   • rack3-leaf1 switch starts from rack3-spine switch for first split, rack1-spine switch for second split, rack2-spine switch for third split
   • and so on...

3. After walking through all leaf switch uplinks in each case, you will have used all spine switch ports between port 5 and 20 inclusive.

The remaining topics in this section provide detailed cabling information for cabling up to 8 racks together.