14.6.1 RDMA Network Fabric Cabling for Oracle Exadata X9M-2
This topic contains RDMA Network Fabric cabling details for Oracle Exadata X9M-2.
The RDMA Network Fabric switches are located in rack units 20 and 22 for Oracle Exadata X9M-2.
In the following tables, Un denotes the slot in the rack, where n is the number, such as U20. Pn denotes the RDMA Network Fabric port n, where n is port number, such as P1. The cables used for the RDMA Network Fabric switch are black in color.
Note:
Not all the slots shown in the tables below will be used in a single configuration. Match the rack component with the appropriate slot in the tables below to determine how to cable that particular component.The following tables are in this section:
- Table 14-9 RDMA Network Fabric for Oracle Exadata Database Machine X9M-2 Database Servers
- Table 14-10 RDMA Network Fabric Cabling for Oracle Exadata Database Machine X9M-2 Storage Servers
- Table 14-11 RDMA Network Fabric Inter-Switch Cabling for Oracle Exadata Database Machine X9M-2
The following table lists the location, ports and cables for the RDMA Network Fabric connections for database servers in an Oracle Exadata X9M-2 rack.
Table 14-9 RDMA Network Fabric Cabling for Oracle Exadata Database Machine X9M-2 Database Servers
| From Rack Unit | Port | To RDMA Network Fabric Switch | Port | Description |
|---|---|---|---|---|
|
U39 |
System Board QSFP28 Port 2 |
U22 |
08 |
3 meters |
|
U39 |
System Board QSFP28 Port 1 |
U20 |
08 |
3 meters |
|
U37 |
System Board QSFP28 Port 2 |
U22 |
09 |
3 meters |
|
U37 |
System Board QSFP28 Port 1 |
U20 |
09 |
3 meters |
|
U35 |
System Board QSFP28 Port 2 |
U22 |
10 |
3 meters |
|
U35 |
System Board QSFP28 Port 1 |
U20 |
10 |
3 meters |
|
U33 |
System Board QSFP28 Port 2 |
U22 |
11 |
3 meters |
|
U33 |
System Board QSFP28 Port 1 |
U20 |
11 |
3 meters |
|
U31 |
System Board QSFP28 Port 2 |
U22 |
12 |
3 meters |
|
U31 |
System Board QSFP28 Port 1 |
U20 |
12 |
3 meters |
|
U29 |
System Board QSFP28 Port 2 |
U22 |
13 |
3 meters |
|
U29 |
System Board QSFP28 Port 1 |
U20 |
13 |
3 meters |
|
U27 |
System Board QSFP28 Port 2 |
U22 |
14 |
3 meters |
|
U27 |
System Board QSFP28 Port 1 |
U20 |
14 |
3 meters |
|
U26 |
System Board QSFP28 Port 2 |
U22 |
15 |
3 meters |
|
U26 |
System Board QSFP28 Port 1 |
U20 |
15 |
3 meters |
|
U25 |
System Board QSFP28 Port 2 |
U22 |
16 |
3 meters |
|
U25 |
System Board QSFP28 Port 1 |
U20 |
16 |
3 meters |
|
U24 |
System Board QSFP28 Port 2 |
U22 |
17 |
3 meters |
|
U24 |
System Board QSFP28 Port 1 |
U20 |
17 |
3 meters |
|
U23 |
System Board QSFP28 Port 2 |
U22 |
18 |
3 meters |
|
U23 |
System Board QSFP28 Port 1 |
U20 |
18 |
3 meters |
|
U19 |
System Board QSFP28 Port 2 |
U22 |
19 |
3 meters |
|
U19 |
System Board QSFP28 Port 1 |
U20 |
19 |
3 meters |
|
U18 |
System Board QSFP28 Port 2 |
U22 |
20 |
3 meters |
|
U18 |
System Board QSFP28 Port 1 |
U20 |
20 |
3 meters |
|
U17 |
System Board QSFP28 Port 2 |
U22 |
21 |
3 meters |
|
U17 |
System Board QSFP28 Port 1 |
U20 |
21 |
3 meters |
|
U16 |
System Board QSFP28 Port 2 |
U22 |
22 |
3 meters |
|
U16 |
System Board QSFP28 Port 1 |
U20 |
22 |
3 meters |
|
U14 |
System Board QSFP28 Port 2 |
U22 |
23 |
3 meters |
|
U14 |
System Board QSFP28 Port 1 |
U20 |
23 |
3 meters |
|
U12 |
System Board QSFP28 Port 2 |
U22 |
24 |
3 meters |
|
U12 |
System Board QSFP28 Port 1 |
U20 |
24 |
3 meters |
|
U10 |
System Board QSFP28 Port 2 |
U22 |
25 |
3 meters |
|
U10 |
System Board QSFP28 Port 1 |
U20 |
25 |
3 meters |
|
U08 |
System Board QSFP28 Port 2 |
U22 |
26 |
3 meters |
|
U08 |
System Board QSFP28 Port 1 |
U20 |
26 |
3 meters |
The following table lists the location, ports and cables for the RDMA Network Fabric connections for storage servers in an Oracle Exadata X9M-2 rack.
Note:
On Exadata X9M-2 High Capacity (HC) and Extreme Flash (EF) storage servers, the RDMA Network Fabric ports are located at PCIe slot 3. On Extreme (XT) storage servers, the RDMA Network Fabric ports are located at PCIe slot 10.
Table 14-10 RDMA Network Fabric for Oracle Exadata Database Machine X9M-2 Storage Servers
| From Rack Unit | Port | To RDMA Network Fabric Switch | Port | Description |
|---|---|---|---|---|
|
U41 |
PCIe QSFP28 Port 2 |
U22 |
15 |
3 meters |
|
U41 |
PCIe QSFP28 Port 1 |
U20 |
15 |
3 meters |
|
U39 |
PCIe QSFP28 Port 2 |
U22 |
08 |
3 meters |
|
U39 |
PCIe QSFP28 Port 1 |
U20 |
08 |
3 meters |
|
U37 |
PCIe QSFP28 Port 2 |
U22 |
09 |
3 meters |
|
U37 |
PCIe QSFP28 Port 1 |
U20 |
09 |
3 meters |
|
U35 |
PCIe QSFP28 Port 2 |
U22 |
10 |
3 meters |
|
U35 |
PCIe QSFP28 Port 1 |
U20 |
10 |
3 meters |
|
U33 |
PCIe QSFP28 Port 2 |
U22 |
11 |
3 meters |
|
U33 |
PCIe QSFP28 Port 1 |
U20 |
11 |
3 meters |
|
U31 |
PCIe QSFP28 Port 2 |
U22 |
12 |
3 meters |
|
U31 |
PCIe QSFP28 Port 1 |
U20 |
12 |
3 meters |
|
U29 |
PCIe QSFP28 Port 2 |
U22 |
13 |
3 meters |
|
U29 |
PCIe QSFP28 Port 1 |
U20 |
13 |
3 meters |
|
U27 |
PCIe QSFP28 Port 2 |
U22 |
14 |
3 meters |
|
U27 |
PCIe QSFP28 Port 1 |
U20 |
14 |
3 meters |
|
U25 |
PCIe QSFP28 Port 2 |
U22 |
16 |
3 meters |
|
U25 |
PCIe QSFP28 Port 1 |
U20 |
16 |
3 meters |
|
U23 |
PCIe QSFP28 Port 2 |
U22 |
18 |
3 meters |
|
U23 |
PCIe QSFP28 Port 1 |
U20 |
18 |
3 meters |
|
U18 |
PCIe QSFP28 Port 2 |
U22 |
20 |
3 meters |
|
U18 |
PCIe QSFP28 Port 1 |
U20 |
20 |
3 meters |
|
U14 |
PCIe QSFP28 Port 2 |
U22 |
23 |
3 meters |
|
U14 |
PCIe QSFP28 Port 1 |
U20 |
23 |
3 meters |
|
U12 |
PCIe QSFP28 Port 2 |
U22 |
24 |
3 meters |
|
U12 |
PCIe QSFP28 Port 1 |
U20 |
24 |
3 meters |
|
U10 |
PCIe QSFP28 Port 2 |
U22 |
25 |
3 meters |
|
U10 |
PCIe QSFP28 Port 1 |
U20 |
25 |
3 meters |
|
U08 |
PCIe QSFP28 Port 2 |
U22 |
26 |
3 meters |
|
U08 |
PCIe QSFP28 Port 1 |
U20 |
26 |
3 meters |
|
U06 |
PCIe QSFP28 Port 2 |
U22 |
27 |
3 meters |
|
U06 |
PCIe QSFP28 Port 1 |
U20 |
27 |
3 meters |
|
U04 |
PCIe QSFP28 Port 2 |
U22 |
28 |
3 meters |
|
U04 |
PCIe QSFP28 Port 1 |
U20 |
28 |
3 meters |
|
U02 |
PCIe QSFP28 Port 2 |
U22 |
29 |
3 meters |
|
U02 |
PCIe QSFP28 Port 1 |
U20 |
29 |
3 meters |
The following table lists the location, ports and cables for the RDMA Network Fabric connections for the leaf switches in an Oracle Exadata X9M-2 rack.
Table 14-11 RDMA Network Fabric Inter-Switch Cabling for Oracle Exadata Database Machine X9M-2
| From Rack Unit | Port | To RDMA Network Fabric Switch | Port | Description |
|---|---|---|---|---|
|
U20 |
04 |
U22 |
04 |
1 meter |
|
U20 |
05 |
U22 |
05 |
1 meter |
|
U20 |
06 |
U22 |
06 |
1 meter |
|
U20 |
07 |
U22 |
07 |
1 meter |
|
U20 |
30 |
U22 |
30 |
1 meter |
|
U20 |
31 |
U22 |
31 |
1 meter |
|
U20 |
32 |
U22 |
32 |
1 meter |
|
U20 |
33 |
U22 |
33 |
1 meter |
Parent topic: RDMA Network Fabric Network Cabling