Documentation, Support, and Training
Understanding Hardware Commands
Performs InfiniBand fabric diagnostic.
ibdiagnet [-c count][-v][-r][-o outputdir][-t topology][-s system][-i device][-p port][-wt topology][-pm][-pc][-P PM = value][-lw 1x|4x|12x][-ls 2.5|5|10][-skip checks][-load_db file][-h][-V]
where:
count is the number of packets.
outputdir is the output directory.
topology is the topology file.
system is the local system name.
device is the index of the device connecting to the InfiniBand fabric.
port is the port of the device.
PM is the Performance Manager counter number.
value is the threshold of the Performance Manager counter.
checks is one or more strings that identifies the checks made:
dup_guids
zero_guids
pm
logical_state
part
ipoib
all
file is the subnet database .db file.
This InfiniBand command scans the InfiniBand fabric using directed route packets, extracting all the available information regarding the connectivity and devices. This command produces a set of files in the output directory. By default, the output directory is /tmp. The following table describes the files.
|
During the discovery phase, the command also checks for duplicate node/port GUIDs in the InfiniBand fabric. If such an error is detected, it is displayed on the standard output.
After the discovery phase is completed, directed route packets are sent multiple times to detect possible problematic paths on which packets might be lost. A report of suspected bad links is displayed on the standard output.
If requested with the -r option, a full report of fabric qualities is displayed, including:
Subnet Manager report
Number of nodes and systems
Hop-count information containing maximal hop-count, an example path, and a hop-count histogram
All CA-to-CA paths traced
Credit loop report
MGID-MLID-HCAs multicast group and report
Partitions report
IPoIB report
Note - If the InfiniBand fabric includes only one CA, then CA-to-CA paths are not reported. Additionally, if a topology file is provided, the ibdiagnet command uses the names defined in the topology file for the output reports.
The following table describes the options to the ibdiagnet command and their purposes:
|
The following example shows how to test the InfiniBand fabric with the ibdiagnet command. The command checks for 4x link width and 10 Gbyte/sec speed, and then dumps the Performance Manager counters and then clears them.
# ibdiagnet -lw 4x -ls 10 -pm -pc Loading IBDIAGNET from: /usr/lib64/ibdiagnet1.2 -W- Topology file is not specified. Reports regarding cluster links will use direct routes. Loading IBDM from: /usr/lib/ibdm1.2 -I- Using port 0 as the local port. -I- Discovering ... 24 nodes (23 Switches & 1 CA-s) discovered. -I--------------------------------------------------- -I- Bad Guids/LIDs Info -I--------------------------------------------------- -I- No bad Guids were found -I--------------------------------------------------- -I- Links With Logical State = INIT -I--------------------------------------------------- -I- No bad Links (with logical state = INIT) were found -I--------------------------------------------------- -I- PM Counters Info -I--------------------------------------------------- -I- No illegal PM counters values were found -I--------------------------------------------------- -I- Links With links width != 4x (as set by -lw option) -I--------------------------------------------------- -I- No unmatched Links (with width != 4x) were found -I--------------------------------------------------- -I- Links With links speed != 10 (as set by -ls option) -I--------------------------------------------------- -I- No unmatched Links (with speed != 10) were found -I--------------------------------------------------- -I- Fabric Partitions Report (see ibdiagnet.pkey for a full hosts list) -I--------------------------------------------------- -I--------------------------------------------------- -I- IPoIB Subnets Check -I--------------------------------------------------- -I- Subnet: IPv4 PKey:0x7fff QKey:0x00000b1b MTU:2048Byte rate:10Gbps SL:0x00 -W- No members found for group -I--------------------------------------------------- -I- Bad Links Info -I- No bad link were found -I--------------------------------------------------- ---------------------------------------------------------------- -I- Stages Status Report: STAGE Errors Warnings Bad GUIDs/LIDs Check 0 0 Link State Active Check 0 0 Performance Counters Report 0 0 Specific Link Width Check 0 0 Specific Link Speed Check 0 0 Partitions Check 0 0 IPoIB Subnets Check 0 1 Please see /tmp/ibdiagnet.log for complete log ---------------------------------------------------------------- -I- Done. Run time was 68 seconds. #
ibdiagnet man page