14.6.2 RDMA Network Fabric Cabling for Oracle Exadata X9M-8

This topic contains RDMA Network Fabric cabling details for Oracle Exadata X9M-8.

The RDMA Network Fabric switches are located in rack units 21 and 23 for Oracle Exadata X9M-8.

In the following tables, Un denotes the slot in the rack, where n is the number, such as U23. Pn denotes the RDMA Network Fabric port n, where n is port number, such as P1. The cables used for the RDMA Network Fabric switch are black in color.

Note:

Not all the slots shown in the tables below will be used in a single configuration. Match the rack component with the appropriate slot in the tables below to determine how to cable that particular component.

The following tables are in this section:

• Table 14-12 RDMA Network Fabric Cabling for Oracle Exadata Database Machine X9M-8 Database Servers
• Table 14-13 RDMA Network Fabric Cabling for Oracle Exadata Database Machine X9M-8 Storage Servers
• Table 14-14 RDMA Network Fabric Inter-Switch Cabling for Oracle Exadata Database Machine X9M-8

The following table lists the rack location, ports and cables for the RDMA Network Fabric connections for database servers in an Oracle Exadata X9M-8 rack.

Table 14-12 RDMA Network Fabric Cabling for Oracle Exadata Database Machine X9M-8 Database Servers

From Rack Unit	Port	To RDMA Network Fabric Switch	Port	Description
U24	PCIe 5, P2	U23	15	3 meters
U24	PCIe 5, P1	U21	15	3 meters
U24	PCIe 7, P2	U23	16	3 meters
U24	PCIe 7, P1	U21	16	3 meters
U24	PCIe 9, P2	U23	17	3 meters
U24	PCIe 9, P1	U21	17	3 meters
U24	PCIe 11, P2	U23	18	3 meters
U24	PCIe 11, P1	U21	18	3 meters
U16	PCIe 5, P2	U23	19	3 meters
U16	PCIe 5, P1	U21	19	3 meters
U16	PCIe 7, P2	U23	20	3 meters
U16	PCIe 7, P1	U21	20	3 meters
U16	PCIe 9, P2	U23	21	3 meters
U16	PCIe 9, P1	U21	21	3 meters
U16	PCIe 11, P2	U23	22	3 meters
U16	PCIe 11, P1	U21	22	3 meters
U10	PCIe 5, P2	U23	23	3 meters
U10	PCIe 5, P1	U21	23	3 meters
U10	PCIe 7, P2	U23	24	3 meters
U10	PCIe 7, P1	U21	24	3 meters
U10	PCIe 9, P2	U23	25	3 meters
U10	PCIe 9, P1	U21	25	3 meters
U10	PCIe 11, P2	U23	30 or 8 Note: Port 30 is the target only in a configuration with 3 database servers and 11 storage servers. Otherwise, port 8 is the target.	3 meters
U10	PCIe 11, P1	U21	30 or 8 Note: Port 30 is the target only in a configuration with 3 database servers and 11 storage servers. Otherwise, port 8 is the target.	3 meters

The following table lists the location, ports and cables for the RDMA Network Fabric connections for storage servers in an Oracle Exadata X9M-8 rack.

Note:

On Exadata X9M-2 High Capacity (HC) and Extreme Flash (EF) storage servers, the RDMA Network Fabric ports are located at PCIe slot 3. On Extreme (XT) storage servers, the RDMA Network Fabric ports are located at PCIe slot 10.

Table 14-13 RDMA Network Fabric Cabling for Oracle Exadata Database Machine X9M-8 Storage Servers

From Rack Unit	Port	To RDMA Network Fabric Switch	Port	Description
U41	PCIe QSFP28 Port 2	U23	08	3 meters
U41	PCIe QSFP28 Port 1	U21	08	3 meters
U39	PCIe QSFP28 Port 2	U23	09	3 meters
U39	PCIe QSFP28 Port 1	U21	09	3 meters
U37	PCIe QSFP28 Port 2	U23	10	3 meters
U37	PCIe QSFP28 Port 1	U21	10	3 meters
U35	PCIe QSFP28 Port 2	U23	11	3 meters
U35	PCIe QSFP28 Port 1	U21	11	3 meters
U33	PCIe QSFP28 Port 2	U23	12	3 meters
U33	PCIe QSFP28 Port 1	U21	12	3 meters
U31	PCIe QSFP28 Port 2	U23	13	3 meters
U31	PCIe QSFP28 Port 1	U21	13	3 meters
U29	PCIe QSFP28 Port 2	U23	14	3 meters
U29	PCIe QSFP28 Port 1	U21	14	3 meters
U14	PCIe QSFP28 Port 2	U23	23	3 meters
U14	PCIe QSFP28 Port 1	U21	23	3 meters
U12	PCIe QSFP28 Port 2	U23	24	3 meters
U12	PCIe QSFP28 Port 1	U21	24	3 meters
U10	PCIe QSFP28 Port 2	U23	25	3 meters
U10	PCIe QSFP28 Port 1	U21	25	3 meters
U08	PCIe QSFP28 Port 2	U23	26	3 meters
U08	PCIe QSFP28 Port 1	U21	26	3 meters
U06	PCIe QSFP28 Port 2	U23	27	3 meters
U06	PCIe QSFP28 Port 1	U21	27	3 meters
U04	PCIe QSFP28 Port 2	U23	28	3 meters
U04	PCIe QSFP28 Port 1	U21	28	3 meters
U02	PCIe QSFP28 Port 2	U23	29	3 meters
U02	PCIe QSFP28 Port 1	U21	29	3 meters

The following table lists the location, ports and cables for the RDMA Network Fabric connections for the leaf switches in an Oracle Exadata X9M-8 rack.

Table 14-14 RDMA Network Fabric Inter-Switch Cabling for Oracle Exadata Database Machine X9M-8

From Rack Unit	Port	To RDMA Network Fabric Switch	Port	Description
U21	04	U23	04	1 meter
U21	05	U23	05	1 meter
U21	06	U23	06	1 meter
U21	07	U23	07	1 meter
U21	30 or 34 Note: Port 34 is used only in a configuration with 3 database servers and 11 storage servers. Otherwise, port 30 is used.	U23	30 or 34 Note: Port 34 is used only in a configuration with 3 database servers and 11 storage servers. Otherwise, port 30 is used.	1 meter
U21	31	U23	31	1 meter
U21	32	U23	32	1 meter
U21	33	U23	33	1 meter