ibdiagnet Command - Sun Datacenter InfiniBand Switch 648 Topic Set

`ibdiagnet` Command

Performs InfiniBand fabric diagnostic. Issued on the Linux InfiniBand host.

Syntax

ibdiagnet [-c count][-v][-r][-o outputdir][-t topology][-s system][-i device][-p port][-wt topology][-pm][-pc][-P PM = value][-lw 1x|4x|12x][-ls 2.5|5|10][-skip checks][-load_db file][-h][-V]

where:

count is the number of packets.
outputdir is the output directory.
topology is the topology file.
system is the local system name.
device is the index of the device connecting to the InfiniBand fabric.
port is the port of the device.
PM is the Performance Manager counter number.
value is the threshold of the Performance Manager counter.
checks is one or more strings that identifies the checks made:
- dup_guids
- zero_guids
- pm
- logical_state
- part
- ipoib
- all
file is the subnet database .db file.

Description

This InfiniBand software command scans the InfiniBand fabric using directed route packets, extracting all the available information regarding the connectivity and devices. This command produces a set of files in the output directory. By default, the output directory is /tmp. The following table describes the files.

File Name	Description
`ibdiagnet.log`	Dump of all the application reports generated according to the provided flags.
`ibdiagnet.lst`	List of all the nodes, ports, and links in the fabric.
`ibdiagnet.fdbs`	Dump of the unicast forwarding tables of the fabric switches.
`ibdiagnet.mcfdbs`	Dump of the multicast forwarding tables of the fabric switches.
`ibdiagnet.masks`	In case of duplicate port/node GUIDs, these file include the map between masked GUIDs and real GUIDs.
`ibdiagnet.sm`	List of all the Subnet Manager (state and priority) in the fabric.
`ibdiagnet.pm`	Dump of the Performance Manager counters values, for the fabric links.
`ibdiagnet.pkey`	Dump of the existing partitions and their member host ports.
`ibdiagnet.mcg`	Dump of the multicast groups, their properties, and member host ports.
`ibdiagnet.db`	Dump of the internal subnet database. This file can be loaded in later runs using the `-load_db` option.

During the discovery phase, the command also checks for duplicate node/port GUIDs in the InfiniBand fabric. If such an error is detected, it is displayed on the standard output.

After the discovery phase is completed, directed route packets are sent multiple times to detect possible problematic paths on which packets might be lost. A report of suspected bad links is displayed on the standard output.

If requested with the -r option, a full report of fabric qualities is displayed, including:

Subnet Manager report
Number of nodes and systems
Hop-count information containing maximal hop-count, an example path, and a hop-count histogram
All CA-to-CA paths traced
Credit loop report
MGID-MLID-HCAs multicast group and report
Partitions report
IPoIB report

Note - If the InfiniBand fabric includes only one CA, then CA-to-CA paths are not reported. Additionally, if a topology file is provided, the ibdiagnet command uses the names defined in the topology file for the output reports.

Options

The following table describes the options to the ibdiagnet command and their purposes:

Option	Purpose
`-c`	Sets the minimum number of packets sent across each link.
`-v`	Provides verbose output.
`-r`	Provides a report of fabric qualities.
`-t`	Specifies the topology file name.
`-s`	Specifies the local system name.
`-i`	In the case of multiple devices on the local system, this option specifies the index of the device of the port used to connect to the InfiniBand fabric.
`-p`	Specifies the local device port number used to connect to the InfiniBand fabric.
`-o`	Specifies the output directory.
`-lw`	Specifies the expected link width.
`-ls`	Specifies the expected link speed.
`-pm`	Dumps all the fabric link Performance Manager counters into `ibdiagnet.pm`.
`-pc`	Resets all the fabric link Performance Manager counters.
`-P`	Use the Performance Manager counter of PM set to the threshold of value.
`-skip`	Skips the executions of the selected checks. One or more checks can be specified.
`-wt`	Writes out the discovered topology into the given file.
`-load_db`	Loads subnet data from the given `.db` file and skips the subnet discovery stage. Note - Some checks require actual subnet discovery, and are disabled if `load_db` is specified. Those checks are duplicate/zero GUIDs, link state, and Subnet Manager status.
`-h`	Provides help.
`-V`	Displays the version information.

Example

The following example shows how to test the InfiniBand fabric with the ibdiagnet command. The command checks for 4x link width and 10 Gbyte/sec speed and dumps the performance manager counters and then clears them.

# ibdiagnet -lw 4x -ls 10 -pm -pc
Loading IBDIAGNET from: /usr/lib64/ibdiagnet1.2
-W- Topology file is not specified.
     Reports regarding cluster links will use direct routes.
Loading IBDM from: /usr/lib64/ibdm1.2
-I- Using port 2 as the local port.
-I- Discovering ... 55 nodes (54 Switches & 1 CA-s) discovered.
-I---------------------------------------------------
-I- Bad Guids/LIDs Info
-I---------------------------------------------------
-I- No bad Guids were found
-I---------------------------------------------------
-I- Links With Logical State = INIT
-I---------------------------------------------------
-I- No bad Links (with logical state = INIT) were found
-I---------------------------------------------------
-I- PM Counters Info
-I---------------------------------------------------
-I- No illegal PM counters values were found
-I---------------------------------------------------
-I- Links With links width != 4x (as set by -lw option)
-I---------------------------------------------------
-I- No unmatched Links (with width != 4x) were found
-I---------------------------------------------------
-I- Links With links speed != 10 (as set by -ls option)
-I- No unmatched Links (with speed != 10) were found
-I---------------------------------------------------
-I- Fabric Partitions Report (see ibdiagnet.pkey for a full hosts list)
-I---------------------------------------------------
-I-    PKey:0x7fff Hosts:1 full:1 partial:0
-I---------------------------------------------------
-I- IPoIB Subnets Check
-I---------------------------------------------------
-I- Subnet: IPv4 PKey:0x7fff QKey:0x00000b1b MTU:2048Byte rate:10Gbps SL:0x00
-W- Suboptimal rate for group. Lowest member rate:40Gbps > group-rate:10Gbps
-I---------------------------------------------------
-I- Bad Links Info
-I- No bad link were found
-I---------------------------------------------------
----------------------------------------------------------------
-I- Stages Status Report:
     STAGE                                    Errors Warnings
     Bad GUIDs/LIDs Check                     0      0
     Link State Active Check                  0      0
     Performance Counters Report              0      0
     Specific Link Width Check                0      0
     Specific Link Speed Check                0      0
     Partitions Check                         0      0
     IPoIB Subnets Check                      0      1
Please see /tmp/ibdiagnet.log for complete log
----------------------------------------------------------------
-I- Done. Run time was 42 seconds.
#

Related Information

ibdiagnet man page
ibdiagpath Command

Exit Print View
Sun Datacenter InfiniBand Switch 648 Topic Set