4.7.6.2 Scanning the RoCE Network Fabric with rocelinkinfo
rocelinkinfo scans the RoCE Network Fabric and presents real-time status information for each link. The
output is presented using the same format as the output of the Linux
iblinkinfo command.
If you run rocelinkinfo without any inputs, the utility
automatically discovers the RoCE Network Fabric
switches connected to the host running the utility. Alternatively, you can specify a
comma-separated list of switch names on the command line identifying specific RoCE Network Fabric switches you want to scan. In all
cases, you must also have SSH equivalency configured between the host running the
utility and the RoCE Network Fabric switches being
scanned.
Following is an example of the output from
rocelinkinfo. In the example, the target Exadata system contains 2
database servers, 3 storage servers, and 2 RoCE Network Fabric leaf switches. The example output also contains link descriptions
created earlier by running configure_roce_hostinfo.sh.
[root@dbm01adm01 ~]# /opt/oracle.SupportTools/ibdiagtools/rocelinkinfo
Switch: 4006.d5b5.4a98 Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-roceb0.example.com 192.0.2.132:
1[ ] ==( auto Down/xcvrAbsent)==> [ ] "" ( )
2[ ] ==( auto Down/xcvrAbsent)==> [ ] "" ( )
3[ ] ==( auto Down/xcvrAbsent)==> [ ] "" ( )
4[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 4[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-rocea0.example.com 192.0.2.131" ( )
5[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 5[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-rocea0.example.com 192.0.2.131" ( )
6[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 6[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-rocea0.example.com 192.0.2.131" ( )
7[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 7[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-rocea0.example.com 192.0.2.131" ( )
8[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
9[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
10[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
11[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
12[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
13[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
14[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
15[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
16[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
17[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
18[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
19[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
20[ ] ==( 4X 25.0 Gbps Down/notconnect)==> [ ] "" ( )
21[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
22[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 2[ ] "dbm01adm02.example.com S 192.168.0.3,192.168.0.4 HCA-1" ( )
23[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 2[ ] "dbm01adm01.example.com S 192.168.0.1,192.168.0.2 HCA-1" ( )
24[ ] ==( 4X 25.0 Gbps Down/notconnect)==> [ ] "" ( )
25[ ] ==( 4X 25.0 Gbps Down/notconnect)==> [ ] "" ( )
26[ ] ==( 4X 25.0 Gbps Down/notconnect)==> [ ] "" ( )
27[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 2[ ] "dbm01celadm03.example.com C 192.168.0.9,192.168.0.10 HCA-1" ( )
28[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 2[ ] "dbm01celadm02.example.com C 192.168.0.7,192.168.0.8 HCA-1" ( )
29[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 2[ ] "dbm01celadm01.example.com C 192.168.0.5,192.168.0.6 HCA-1" ( )
30[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 30[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-rocea0.example.com 192.0.2.131" ( )
31[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 31[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-rocea0.example.com 192.0.2.131" ( )
32[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 32[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-rocea0.example.com 192.0.2.131" ( )
33[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 33[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-rocea0.example.com 192.0.2.131" ( )
34[ ] ==( auto Down/xcvrAbsent)==> [ ] "" ( )
35[ ] ==( auto Down/xcvrAbsent)==> [ ] "" ( )
36[ ] ==( auto Down/xcvrAbsent)==> [ ] "" ( )
Switch: 4006.d5b5.49fc Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-rocea0.example.com 192.0.2.131:
1[ ] ==( auto Down/xcvrAbsent)==> [ ] "" ( )
2[ ] ==( auto Down/xcvrAbsent)==> [ ] "" ( )
3[ ] ==( auto Down/xcvrAbsent)==> [ ] "" ( )
4[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 4[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-roceb0.example.com 192.0.2.132" ( )
5[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 5[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-roceb0.example.com 192.0.2.132" ( )
6[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 6[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-roceb0.example.com 192.0.2.132" ( )
7[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 7[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-roceb0.example.com 192.0.2.132" ( )
8[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
9[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
10[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
11[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
12[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
13[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
14[ ] ==( auto Down/xcvrAbsent)==> [ ] "" ( )
15[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
16[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
17[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
18[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
19[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
20[ ] ==( 4X 25.0 Gbps Down/notconnect)==> [ ] "" ( )
21[ ] ==( 4X 25.0 Gbps Down/xcvrAbsent)==> [ ] "" ( )
22[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 1[ ] "dbm01adm02.example.com S 192.168.0.3,192.168.0.4 HCA-1" ( )
23[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 1[ ] "dbm01adm01.example.com S 192.168.0.1,192.168.0.2 HCA-1" ( )
24[ ] ==( 4X 25.0 Gbps Down/notconnect)==> [ ] "" ( )
25[ ] ==( 4X 25.0 Gbps Down/notconnect)==> [ ] "" ( )
26[ ] ==( 4X 25.0 Gbps Down/notconnect)==> [ ] "" ( )
27[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 1[ ] "dbm01celadm03.example.com C 192.168.0.9,192.168.0.10 HCA-1" ( )
28[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 1[ ] "dbm01celadm02.example.com C 192.168.0.7,192.168.0.8 HCA-1" ( )
29[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 1[ ] "dbm01celadm01.example.com C 192.168.0.5,192.168.0.6 HCA-1" ( )
30[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 30[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-roceb0.example.com 192.0.2.132" ( )
31[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 31[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-roceb0.example.com 192.0.2.132" ( )
32[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 32[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-roceb0.example.com 192.0.2.132" ( )
33[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 33[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-roceb0.example.com 192.0.2.132" ( )
34[ ] ==( auto Down/xcvrAbsent)==> [ ] "" ( )
35[ ] ==( auto Down/xcvrAbsent)==> [ ] "" ( )
36[ ] ==( auto Down/xcvrAbsent)==> [ ] "" ( )
CA: dbm01adm01.example.com S 192.168.0.1,192.168.0.2 HCA-1:
e8eb.d394.c168 1[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 23[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-rocea0.example.com 192.0.2.131" ( )
e8eb.d394.c169 2[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 23[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-roceb0.example.com 192.0.2.132" ( )
CA: dbm01adm02.example.com S 192.168.0.3,192.168.0.4 HCA-1:
e8eb.d394.bf08 1[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 22[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-rocea0.example.com 192.0.2.131" ( )
e8eb.d394.bf09 2[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 22[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-roceb0.example.com 192.0.2.132" ( )
CA: dbm01celadm03.example.com C 192.168.0.9,192.168.0.10 HCA-1:
e8eb.d394.bee8 1[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 27[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-rocea0.example.com 192.0.2.131" ( )
e8eb.d394.bee9 2[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 27[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-roceb0.example.com 192.0.2.132" ( )
CA: dbm01celadm02.example.com C 192.168.0.7,192.168.0.8 HCA-1:
1070.fd87.12ea 1[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 28[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-rocea0.example.com 192.0.2.131" ( )
1070.fd87.12eb 2[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 28[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-roceb0.example.com 192.0.2.132" ( )
CA: dbm01celadm01.example.com C 192.168.0.5,192.168.0.6 HCA-1:
1070.fda3.79e0 1[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 29[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-rocea0.example.com 192.0.2.131" ( )
1070.fda3.79e1 2[ ] ==( 4X 25.0 Gbps Active/ LinkUp)==> 29[ ] "Cisco Nexus 9336C-FX2 QSFP28 dbm01sw-roceb0.example.com 192.0.2.132" ( )
Parent topic: Discovering the RoCE Network Fabric