JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Sun Network QDR InfiniBand Gateway Switch HTML Document Collection for Firmware Version 2.0
search filter icon
search icon

Document Information

Using This Documentation

Product Notes

Related Documentation

Feedback

Support and Accessibility

Installation

Understanding the Gateway

Gateway Overview

Physical Specifications

Environmental Requirements

Acoustic Noise Emissions

Electrical Specifications

Network Management Connector and Pins

USB Management Connector and Pins

Data QSFP Connector and Pins

Understanding Cabling

Routing Service Cables

Understanding Data Cabling

Preparing to Install the Gateway

Installation Preparation

Suggested Tools for Installation

Antistatic Precautions for Installation

Installation Responsibilities

Installation Sequence

Verify Shipping Carton Contents

Assemble the Ethernet Splitter Data Cables

Route the Data Cables

Installing the Gateway

Install the Gateway in the Rack

Powering On the Gateway

Connecting Data cables

Verifying the InfiniBand Fabric

Administration

Troubleshooting the Gateway

Gateway Hardware Problems

InfiniBand Fabric Problems

Identifying LEDs

Understanding Routing Through the Gateway

Switch GUIDs Overview

Understanding Administrative Commands

Configuration Overview

Hardware Command Overview

InfiniBand Command Overview

ILOM Command Overview

Administering the Chassis

Monitoring the Chassis

Controlling the Chassis

Administering the I4 Switch Chip

Monitoring the I4 Switch Chip

Controlling the I4 Switch Chip

Administering the Subnet Manager

Monitoring the Subnet Manager

Controlling the Subnet Manager

Administering the InfiniBand Fabric

Monitoring the InfiniBand Fabric

Controlling the InfiniBand Fabric

Configuring the Fabric Director Node List

Partitioning the InfiniBand Fabric

Administering Gateway Resources

Installing Gateway Supportive Software (Linux)

Creating VNICs Under Gateway Manual Mode (Linux)

Creating VNICs Under Host Manual Mode (Linux)

Creating Virtual IO Adapters (Oracle Solaris)

Monitoring Gateway Resources

Controlling LAGs

Controlling VLANs and VNICs

Controlling Gateway Ports and Parameters

Remote Management

Understanding Oracle ILOM on the Gateway

Oracle ILOM Overview

Supported Features

Understanding Oracle ILOM Targets

Administering Oracle ILOM (CLI)

CLI Overview

Accessing Oracle ILOM From the CLI

Switching Between the Oracle ILOM Shell and the Linux Shell

Monitoring Oracle ILOM Targets (CLI)

Controlling Oracle ILOM Targets (CLI)

Upgrading the Gateway Firmware Through Oracle ILOM (CLI)

Administering Oracle ILOM (Web)

Web Interface Overview

Access Oracle ILOM From the Web Interface

Monitoring Oracle ILOM Targets (Web)

Controlling Oracle ILOM Targets (Web)

Upgrade the Gateway Firmware (Web)

Using the Fabric Monitor

Access the Fabric Monitor

Fabric Monitor Features

Accessing the Rear Panel Diagram

Accessing Status Pane Information

Control Panel Function

Monitoring Parameters and Status

Administering Oracle ILOM (SNMP)

SNMP Overview

Understanding SNMP Commands

Monitoring Oracle ILOM Targets (SNMP)

Controlling Oracle ILOM Targets (SNMP)

Administering Hardware (IPMI)

ipmitool Overview

Display the Sensor States (IPMI)

Display the Sensor Information (IPMI)

Display the System Event Log (IPMI)

Display FRU ID Information (IPMI)

Display Gateway Status LED States (IPMI)

Enable the Locator LED (IPMI)

Disable the Locator LED (IPMI)

Understanding Oracle ILOM Commands

cd Command

create Command

delete Command

dump Command

exit Command (ILOM)

help Command (ILOM)

load Command

reset Command

set Command

show Command

version Command (ILOM)

Service

Understanding Service Procedures

Replaceable Components

Suggested Tools for Service

Antistatic Precautions for Service

Servicing Power Supplies

Inspecting a Power Supply

Power Off a Power Supply

Remove a Power Supply

Install a Power Supply

Power On a Power Supply

Servicing Fans

Inspecting a Fan

Remove a Fan

Install a Fan

Servicing Data Cables

Inspecting the Data Cables

Remove a Data Cable

Install a Data Cable

Servicing the Battery

Remove the Gateway From the Rack

Replace the Battery

Reference

Understanding Hardware Commands

Linux Shells for Hardware Commands

addlagport Command

allowhostconfig Command

checkboot Command

checkpower Command

checktopomax Command

checkvoltages Command

connector Command

createlag Command

createvlan Command

createvnic Command

dcsport Command

deletelag Command

deletevlan Command

deletevnic Command

dellagport Command

disablecablelog Command

disablegwport Command

disablelagmode Command

disablelinklog Command

disablesm Command

disableswitchport Command

disablevnic Command

disallowhostconfig Command

enablecablelog Command

enablegwport Command

enablelagmode Command

enablelinklog Command

enablesm Command

enableswitchport Command

enablevnic Command

env_test Command

exit Command (Hardware)

fdconfig Command

fwverify Command

generatetopology Command

getfanspeed Command

getmaster Command

getportcounters Command

getportstatus Command

help Command (Hardware)

listlinkup Command

matchtopology Command

setcontrolledhandover Command

setdefaultgwdiscpkey Command

setgwethport Command

setgwinstance Command

setgwsl Command

setgwsystemname Command

setmsmlocationmonitor Command

setsmmkey Command

setsmpriority Command

setsubnetprefix Command

showfruinfo Command

showgwconfig Command

showgwports Command

showlag Command

showpsufru Command

showsmlog Command

showtemps Command

showtopology Command

showunhealthy Command

showvlan Command

showvnics Command

smconfigtest Command

smnodes Command

smpartition Command

version Command (Hardware)

Understanding InfiniBand Commands

Linux Shells for InfiniBand Commands

ibdiagnet Command

ibhosts Command

ibnetdiscover Command

ibnetstatus Command

ibnodes Command

ibportstate Command

ibroute Command

ibrouters Command

ibstat Command

ibswitches Command

ibtracert Command

perfquery Command

saquery Command

smpquery Command

Understanding SNMP MIB OIDs

OID Tables Overview

Understanding the SUN-DCS-IB-MIB MIB OIDs

Understanding the SUN-FABRIC-MIB MIB OIDs

Understanding the SUN-HW-TRAP-MIB MIB OIDs

Understanding the SUN-ILOM-CONTROL-MIB MIB OIDs

Understanding the SUN-PLATFORM-MIB MIB OIDs

Understanding the ENTITY-MIB MIB OIDs

Index

ibdiagnet Command

Performs InfiniBand fabric diagnostic.

Syntax

ibdiagnet [-c count][-v][-r][-o outputdir][-t topology][-s system][-i device][-p port][-wt topology][-pm][-pc][-P PM = value][-lw 1x|4x|12x][-ls 2.5|5|10][-skip checks][-load_db file][-h][-V]

where:

Description

This InfiniBand command scans the InfiniBand fabric using directed route packets, extracting all the available information regarding the connectivity and devices. This command produces a set of files in the output directory. By default, the output directory is /tmp. This table describes the files.

File Name
Description
ibdiagnet.log
Dump of all the application reports generated according to the provided flags.
ibdiagnet.lst
List of all the nodes, ports, and links in the fabric.
ibdiagnet.fdbs
Dump of the unicast forwarding tables of the fabric switches.
ibdiagnet.mcfdbs
Dump of the multicast forwarding tables of the fabric switches.
ibdiagnet.masks
In case of duplicate port/node GUIDs, this file includes the map between masked GUIDs and real GUIDs.
ibdiagnet.sm
List of all the Subnet Manager (state and priority) in the fabric.
ibdiagnet.pm
Dump of the Performance Manager counters values, for the fabric links.
ibdiagnet.pkey
Dump of the existing partitions and their member host ports.
ibdiagnet.mcg
Dump of the multicast groups, their properties, and member host ports.
ibdiagnet.db
Dump of the internal subnet database. You can load this file in later runs using the -load_db option.

During the discovery phase, the command also checks for duplicate node/port GUIDs in the InfiniBand fabric. If such an error is detected, it is displayed on the standard output.

After the discovery phase is completed, directed route packets are sent multiple times to detect possible problematic paths on which packets might be lost. A report of suspected bad links is displayed on the standard output.

If requested with the -r option, a full report of fabric qualities is displayed, including:


Note - If the InfiniBand fabric includes only one CA, then CA-to-CA paths are not reported. Additionally, if a topology file is provided, the ibdiagnet command uses the names defined in the topology file for the output reports.


The ibdiagnet command is available from the /SYS/Fabric_Mgmt Linux shell target of the Oracle ILOM CLI interface.

Options

This table describes the options to the ibdiagnet command and their purposes.

Option
Purpose
-c
Sets the minimum number of packets sent across each link.
-v
Provides verbose output.
-r
Provides a report of fabric qualities.
-t
Specifies the topology file name.
-s
Specifies the local system name.
-i
In the case of multiple devices on the local system, this option specifies the index of the device of the port used to connect to the InfiniBand fabric.
-p
Specifies the local device port number used to connect to the InfiniBand fabric.
-o
Specifies the output directory.
-lw
Specifies the expected link width.
-ls
Specifies the expected link speed.
-pm
Dumps all the fabric link Performance Manager counters into ibdiagnet.pm.
-pc
Resets all the fabric link Performance Manager counters.
-P
Uses the Performance Manager counter of PM set to the threshold of value.
-skip
Skips the executions of the selected checks. One or more checks can be specified.
-wt
Writes out the discovered topology into the given file.
-load_db
Loads subnet data from the given .db file and skips the subnet discovery stage.

Note - Some checks require actual subnet discovery and are disabled if load_db is specified. Those checks are for duplicate/zero GUIDs, link state, and Subnet Manager status.


-h
Provides help.
-V
Displays the version information.
Example

This example shows how to test the InfiniBand fabric with the ibdiagnet command. The command checks for 4x link width and 10 Gbyte/sec speed, dumps the Performance Manager counters, and then clears the counters.


Note - The output for your InfiniBand fabric will differ from that in the example.


# ibdiagnet -lw 4x -ls 10 -pm -pc -skip dup_guids
Loading IBDIAGNET from: /usr/lib/ibdiagnet1.2
-W- Topology file is not specified.
     Reports regarding cluster links will use direct routes.
Loading IBDM from: /usr/lib/ibdm1.2
-I- Using port 0 as the local port.
-I- Discovering ... 2 nodes (1 Switches & 1 CA-s) discovered.
-I---------------------------------------------------
-I- Bad Guids/LIDs Info
-I---------------------------------------------------
-I- No bad Guids were found
-I---------------------------------------------------
-I- Links With Logical State = INIT
-I---------------------------------------------------
-I- No bad Links (with logical state = INIT) were found
-I---------------------------------------------------
-I- PM Counters Info
-I---------------------------------------------------
-I- No illegal PM counters values were found
-I---------------------------------------------------
-I- Links With links width != 4x (as set by -lw option)
-I---------------------------------------------------
-I- No unmatched Links (with width != 4x) were found
-I---------------------------------------------------
-I- Links With links speed != 10 (as set by -ls option)
-I---------------------------------------------------
-I- No unmatched Links (with speed != 10) were found
-I---------------------------------------------------
-I- Fabric Partitions Report (see ibdiagnet.pkey for a full hosts list)
-I---------------------------------------------------
-I---------------------------------------------------
-I- IPoIB Subnets Check
-I---------------------------------------------------
-I- Subnet: IPv4 PKey:0x7fff QKey:0x00000b1b MTU:2048Byte rate:10Gbps SL:0x00
-W- No members found for group
-I---------------------------------------------------
-I- Bad Links Info
-I- No bad link were found
-I---------------------------------------------------
----------------------------------------------------------------
-I- Stages Status Report:
     STAGE                                    Errors Warnings
     Bad GUIDs/LIDs Check                     0      0
     Link State Active Check                  0      0
     Performance Counters Report              0      0
     Specific Link Width Check                0      0
     Specific Link Speed Check                0      0
     Partitions Check                         0      0
     IPoIB Subnets Check                      0      1
Please see /tmp/ibdiagnet.log for complete log
----------------------------------------------------------------
-I- Done. Run time was 2 seconds.
#
Related Information