man pages section 1M: System Administration Commands

Exit Print View

Updated: July 2014
 
 

ibnetdiscover (1m)

Name

ibnetdiscover - discover InfiniBand topology

Synopsis

ibnetdiscover    [-d(ebug)]    [-e(rr_show)]    [-v(erbose)]
[-s(how)]     [-l(ist)]     [-g(rouping)]      [-H(ca_list)]
[-S(witch_list)]  [-R(outer_list)] [-C ca_name] [-P ca_port]
[-t(imeout) timeout_ms] [-V(ersion)] [--node-name-map <node-
name-map>]  [--cache  <filename>]  [--load-cache <filename>]
[-p(orts)] [-m(ax_hops)] [-h(elp)] [<topology-file>]

Description




OpenIB Diagnostics                              IBNETDISCOVER(1m)



NAME
     ibnetdiscover - discover InfiniBand topology


SYNOPSIS
     ibnetdiscover    [-d(ebug)]    [-e(rr_show)]    [-v(erbose)]
     [-s(how)]     [-l(ist)]     [-g(rouping)]      [-H(ca_list)]
     [-S(witch_list)]  [-R(outer_list)] [-C ca_name] [-P ca_port]
     [-t(imeout) timeout_ms] [-V(ersion)] [--node-name-map <node-
     name-map>]  [--cache  <filename>]  [--load-cache <filename>]
     [-p(orts)] [-m(ax_hops)] [-h(elp)] [<topology-file>]


DESCRIPTION
     ibnetdiscover performs IB subnet  discovery  and  outputs  a
     human  readable  topology  file. GUIDs, node types, and port
     numbers are displayed as well as port LIDs and  NodeDescrip-
     tions.  All nodes (and links) are displayed (full topology).
     Optionally, this utility can be used  to  list  the  current
     connected nodes by nodetype.  The output is printed to stan-
     dard output unless a topology file is specified.


OPTIONS
     -l, --list
          List of connected nodes

     -g, --grouping
          Show grouping.  Grouping correlates IB nodes by differ-
          ent  vendor  specific  schemes.   It  may also show the
          switch external ports correspondence.

     -H, --Hca_list
          List of connected CAs

     -S, --Switch_list
          List of connected switches

     -R, --Router_list
          List of connected routers

     -s, --show
          Show progress information during discovery.

     --node-name-map <node-name-map>
          Specify a node name map.  The node name map  file  maps
          GUIDs  to  more  user  friendly names.  See file format
          below.

     --cache <filename>
          Cache the ibnetdiscover network data in  the  specified
          filename.   This  cache  may be used by other tools for



OpenIB              Last change: May 13, 2009                   1






OpenIB Diagnostics                              IBNETDISCOVER(1m)



          later analysis.

     --load-cache <filename>
          Load and use the cached ibnetdiscover  data  stored  in
          the  specified  filename.  May be useful for outputting
          and learning about other fabrics or a previous state of
          a fabric.

     --diff <filename>
          Load cached ibnetdiscover data and do a diff comparison
          to the current network or  another  cache.   A  special
          diff  output for ibnetdiscover output will be displayed
          showing differences between the old and current fabric.
          By default, the following are compared for differences:
          switches, channel adapters, routers, and  port  connec-
          tions.

     --diffcheck <key(s)>
          Specify  what  diff checks should be done in the --diff
          option  above.   Comma  separate  multiple  diff  check
          key(s).   The available diff checks are: sw = switches,
          ca = channel adapters, router = routers,  port  =  port
          connections,  lid = lids, nodedesc = node descriptions.
          Note that port, lid, and nodedesc are checked only  for
          the  node  types  that  are  specified  (e.g.  sw,  ca,
          router).  If port is specified alongside  lid  or  nod-
          edesc, remote port lids and node descriptions will also
          be compared.



     -p, --ports
          Obtain a ports report which  is  a  list  of  connected
          ports  with  relevant  information  (like LID, portnum,
          GUID, width, speed, and NodeDescription).

     -m, --max_hops
          Report max hops discovered.


COMMON OPTIONS
     Most OpenIB diagnostics take the following common flags. The
     exact  list  of  supported flags per utility can be found in
     the usage message and can be shown using  the  util_name  -h
     syntax.

     # Debugging flags

     -d      raise the IB debugging level.
             May be used several times (-ddd or -d -d -d).





OpenIB              Last change: May 13, 2009                   2






OpenIB Diagnostics                              IBNETDISCOVER(1m)



     -e      show send and receive errors (timeouts and others)

     -h      show the usage message

     -v      increase the application verbosity level.
             May be used several times (-vv or -v -v -v)

     -V      show the version info.

     # Other common flags:

     -C <ca_name>    use the specified ca_name.

     -P <ca_port>    use the specified ca_port.

     -t   <timeout_ms>  override  the  default  timeout  for  the
     solicited mads.

     Multiple CA/Multiple Port Support

     When no IB device or port is specified, the port to  use  is
     selected by the following criteria:

     1. the first port that is ACTIVE.

     2.  if  not  found, the first port that is UP (physical link
     up).

     If a port and/or CA name is specified, the user  request  is
     attempted to be fulfilled, and will fail if it is not possi-
     ble.


TOPOLOGY FILE FORMAT
     The topology file format is human readable and largely intu-
     itive.  Most identifiers are given textual names like vendor
     ID (vendid), device ID (device ID), GUIDs of  various  types
     (sysimgguid, caguid, switchguid, etc.).  PortGUIDs are shown
     in parentheses ().  For  switches,  this  is  shown  on  the
     switchguid  line.   For  CA and router ports, it is shown on
     the connectivity lines.  The IB node is identified  followed
     by  the  number of ports and a quoted the node GUID.  On the
     right of this line is a comment (#)  followed  by  the  Nod-
     eDescription  in quotes.  If the node is a switch, this line
     also contains whether switch port 0 is base or enhanced, and
     the  LID  and LMC of port 0.  Subsequent lines pertaining to
     this node show the connectivity.   On the left is  the  port
     number  of  the current node.  On the right is the peer node
     (node at other end of link). It is identified in quotes with
     nodetype  followed  by  - followed by NodeGUID with the port
     number in square brackets.  Further on the right is  a  com-
     ment (#).  What follows the comment is dependent on the node



OpenIB              Last change: May 13, 2009                   3






OpenIB Diagnostics                              IBNETDISCOVER(1m)



     type.  If it it a switch node, it is followed  by  the  Nod-
     eDescription  in quotes and the LID of the peer node.  If it
     is a CA or router node, it is followed by the local LID  and
     LMC  and  then followed by the NodeDescription in quotes and
     the LID of the peer node.  The active link width  and  speed
     are then appended to the end of this output line.

     An example of this is:
     #
     # Topology file: generated on Tue Jun  5 14:15:10 2007
     #
     # Max of 3 hops discovered
     # Initiated from node 0008f10403960558 port 0008f10403960559

     Non-Chassis Nodes

     vendid=0x8f1
     devid=0x5a06
     sysimgguid=0x5442ba00003000
     switchguid=0x5442ba00003080(5442ba00003080)
     Switch  24 "S-005442ba00003080"         # "ISR9024 Voltaire" base port 0 lid 6 lmc 0
     [22]    "H-0008f10403961354"[1](8f10403961355)         # "MT23108 InfiniHost Mellanox Technologies" lid 4 4xSDR
     [10]    "S-0008f10400410015"[1]         # "SW-6IB4 Voltaire" lid 3 4xSDR
     [8]     "H-0008f10403960558"[2](8f1040396055a)         # "MT23108 InfiniHost Mellanox Technologies" lid 14 4xSDR
     [6]     "S-0008f10400410015"[3]         # "SW-6IB4 Voltaire" lid 3 4xSDR
     [12]    "H-0008f10403960558"[1](8f10403960559)         # "MT23108 InfiniHost Mellanox Technologies" lid 10 4xSDR

     vendid=0x8f1
     devid=0x5a05
     switchguid=0x8f10400410015(8f10400410015)
     Switch  8 "S-0008f10400410015"          # "SW-6IB4 Voltaire" base port 0 lid 3 lmc 0
     [6]     "H-0008f10403960984"[1](8f10403960985)         # "MT23108 InfiniHost Mellanox Technologies" lid 16 4xSDR
     [4]     "H-005442b100004900"[1](5442b100004901)        # "MT23108 InfiniHost Mellanox Technologies" lid 12 4xSDR
     [1]     "S-005442ba00003080"[10]                # "ISR9024 Voltaire" lid 6 1xSDR
     [3]     "S-005442ba00003080"[6]         # "ISR9024 Voltaire" lid 6 4xSDR

     vendid=0x2c9
     devid=0x5a44
     caguid=0x8f10403960984
     Ca      2 "H-0008f10403960984"          # "MT23108 InfiniHost Mellanox Technologies"
     [1](8f10403960985)     "S-0008f10400410015"[6]         # lid 16 lmc 1 "SW-6IB4 Voltaire" lid 3 4xSDR

     vendid=0x2c9
     devid=0x5a44
     caguid=0x5442b100004900
     Ca      2 "H-005442b100004900"          # "MT23108 InfiniHost Mellanox Technologies"
     [1](5442b100004901)     "S-0008f10400410015"[4]         # lid 12 lmc 1 "SW-6IB4 Voltaire" lid 3 4xSDR

     vendid=0x2c9
     devid=0x5a44
     caguid=0x8f10403961354
     Ca      2 "H-0008f10403961354"          # "MT23108 InfiniHost Mellanox Technologies"



OpenIB              Last change: May 13, 2009                   4






OpenIB Diagnostics                              IBNETDISCOVER(1m)



     [1](8f10403961355)     "S-005442ba00003080"[22]                # lid 4 lmc 1 "ISR9024 Voltaire" lid 6 4xSDR

     vendid=0x2c9
     devid=0x5a44
     caguid=0x8f10403960558
     Ca      2 "H-0008f10403960558"          # "MT23108 InfiniHost Mellanox Technologies"
     [2](8f1040396055a)     "S-005442ba00003080"[8]         # lid 14 lmc 1 "ISR9024 Voltaire" lid 6 4xSDR
     [1](8f10403960559)     "S-005442ba00003080"[12]                # lid 10 lmc 1 "ISR9024 Voltaire" lid 6 1xSDR

     When  grouping  is used, IB nodes are organized into chasses
     which are numbered. Nodes which cannot be determined  to  be
     in a chassis are displayed as "Non-Chassis Nodes".  External
     ports are also shown on the connectivity lines.



NODE NAME MAP FILE FORMAT
     The node name map is used to specify user friendly names for
     nodes  in the output.  GUIDs are used to perform the lookup.


     Generically:

          # comment
          <guid> "<name>"


     Example:

          # IB1
          # Line cards
          0x0008f104003f125c   "IB1   (Rack   11   slot   1     )
          ISR9288/ISR9096 Voltaire sLB-24D"
          0x0008f104003f125d   "IB1   (Rack   11   slot   1     )
          ISR9288/ISR9096 Voltaire sLB-24D"
          0x0008f104003f10d2   "IB1   (Rack   11   slot   2     )
          ISR9288/ISR9096 Voltaire sLB-24D"
          0x0008f104003f10d3   "IB1   (Rack   11   slot   2     )
          ISR9288/ISR9096 Voltaire sLB-24D"
          0x0008f104003f10bf   "IB1   (Rack   11   slot   12    )
          ISR9288/ISR9096 Voltaire sLB-24D"
          # Spines
          0x0008f10400400e2d  "IB1  (Rack  11 spine 1   ) ISR9288
          Voltaire sFB-12D"
          0x0008f10400400e2e "IB1 (Rack 11 spine  1    )  ISR9288
          Voltaire sFB-12D"
          0x0008f10400400e2f  "IB1  (Rack  11 spine 1   ) ISR9288
          Voltaire sFB-12D"
          0x0008f10400400e31 "IB1 (Rack 11 spine  2    )  ISR9288
          Voltaire sFB-12D"
          0x0008f10400400e32  "IB1  (Rack  11 spine 2   ) ISR9288
          Voltaire sFB-12D"



OpenIB              Last change: May 13, 2009                   5






OpenIB Diagnostics                              IBNETDISCOVER(1m)



          # GUID   Node Name
          0x0008f10400411a08 "SW1   (Rack   3)  ISR9024  Voltaire
          9024D"
          0x0008f10400411a28  "SW2   (Rack   3)  ISR9024 Voltaire
          9024D"
          0x0008f10400411a34 "SW3   (Rack   3)  ISR9024  Voltaire
          9024D"
          0x0008f104004119d0  "SW4   (Rack   3)  ISR9024 Voltaire
          9024D"


AUTHORS
     Hal Rosenstock
          <halr@voltaire.com>

     Ira Weiny
          <weiny2@llnl.gov>



ATTRIBUTES
     See  attributes(5)  for  descriptions   of   the   following
     attributes:

     +---------------+----------------------+
     |ATTRIBUTE TYPE |   ATTRIBUTE VALUE    |
     +---------------+----------------------+
     |Availability   | network/open-fabrics |
     +---------------+----------------------+
     |Stability      | Volatile             |
     +---------------+----------------------+
NOTES
     This   software   was   built   from   source  available  at
     https://java.net/projects/solaris-userland.   The   original
     community  source was downloaded from  ['http://www.openfab-
     rics.org/downloads/ibutils/ibutils-1.5.7.tar.gz',
     'http://www.openfabrics.org/downloads/libibverbs/libib-
     verbs-1.1.4-1.22.g7257cd3.tar.gz',      'http://www.openfab-
     rics.org/down-
     loads/libmlx4/libmlx4-1.0.1-1.18.gb810a27.tar.gz',
     'http://www.openfabrics.org/downloads/libsdp/lib-
     sdp-1.1.108-0.15.gd7fdb72.tar.gz',      'http://www.openfab-
     rics.org/downloads/management/infiniband-
     diags-1.5.8.tar.gz',       'http://www.openfabrics.org/down-
     loads/management/libibmad-1.3.7.tar.gz',   'http://www.open-
     fabrics.org/downloads/management/libibumad-1.3.7.tar.gz',
     'http://www.openfabrics.org/downloads/manage-
     ment/opensm-3.3.9.tar.gz', 'http://www.openfabrics.org/down-
     loads/perftest/perftest-1.3.0-0.42.gf350d3d.tar.gz',
     'http://www.openfabrics.org/down-
     loads/qperf/qperf-0.4.6-0.1.gb81434e.tar.gz',
     'http://www.openfabrics.org/downloads/rdmacm/librd-



OpenIB              Last change: May 13, 2009                   6






OpenIB Diagnostics                              IBNETDISCOVER(1m)



     macm-1.0.14.1.tar.gz',     'http://www.openfabrics.org/down-
     loads/rds-tools/rds-tools-2.0.4.tar.gz']

     Further information about this software can be found on  the
     open   source   community   website  at  http://www.openfab-
     rics.org/.

















































OpenIB              Last change: May 13, 2009                   7