Documentation, Support, and Training
Understanding Administrative Commands
Monitoring the InfiniBand Fabric
Controlling the InfiniBand Fabric
Understanding Signal Routing Through the Switch
Understanding Switch Startup and Component Addition
Understanding ILOM on the Switch
Switching Between the ILOM Shell and the Linux Shell
Controlling ILOM Targets (CLI)
Upgrading the Switch Firmware Through ILOM (CLI)
Administering ILOM (Web Interface)
Access ILOM From the Web Interface
Monitoring ILOM Targets (Web Interface)
Controlling ILOM Targets (Web Interface)
Upgrading the Switch Firmware Through ILOM (Web Interface)
Understanding InfiniBand Cabling
Understanding Switch Specifications
Understanding the Installation
Fabric Card and Line Card Fillers
Suggested Tools for Installation
Antistatic Precautions for Installation
Installing the InfiniBand Software Stack
Install the HPC Software Suite
Remove the Shipping Carton Contents
Unsecure the Chassis From the Pallet
Install the Rail Assemblies in the Rack
Install the Chassis in the Rack
Inspecting the Midplane Condition
Midplane Fabric Card Connector Nomenclature
Inspect the Pins on the Rear Side of the Midplane
Midplane Line Card Connector Nomenclature
Inspect the Pins on the Front Side of the Midplane
Inspecting the Fabric Cards Condition
Fabric Card Handling Guidelines
Inspect the Fabric Card Chassis Condition
Inspect the Fabric Card Retainer Bolts Condition
Inspect the Fabric Card XBOW Connectors Condition
Inspecting the Line Cards Condition
Inspect the Line Card Chassis Condition
Inspect the Line Card Retainer Bolts Condition
Inspect the Line Card XBOW Connectors Condition
Inspect the Line Card CXP Connectors Condition
Inspecting the InfiniBand Cables Condition
Inspect the InfiniBand Cables Hardware Condition
Inspect the InfiniBand Cables Connector Condition
Install the Midplane Stiffener
Install the Fabric Cards or Fabric Card Fillers
Access the CMC From the Serial Management Port
Set the Initial UBoot Parameters
Access the CMC From the Network Management Port
Display the CMC FRU ID Information
Verify the Power Supply Status
Display the Power Supply FRU ID Information
Display the Fabric Card FRU ID Information
Display the Line Card FRU ID Information
Attach Cables to the CXP Connectors
Display the InfiniBand Cable FRU ID Information
Verifying the InfiniBand Fabric
Discover the InfiniBand Fabric Topology
Perform Diagnostics on the InfiniBand Fabric
Validate the InfiniBand Fabric and Report Errors
Understanding Service Procedures
Servicing the Fabric Cards and Fabric Card Fillers
Servicing the InfiniBand Cables
Doing Supportive Software Tasks
Understanding Switch-Specific Commands
If your InfiniBand fabric uses the Fat-Tree routing algorithm, you must provide a modified listing of fabric card GUIDs, called the root node GUID file, for the Subnet Manager. The /etc/opensm/guid.txt file is such a file. This procedure describes how to create the guid.txt file.
Note - In this procedure, you issue commands to both the CMC and the Linux InfiniBand host.
See the table in Installation Sequence.
# getbaseguid fc slot
where slot is the number of the fabric card (0–8). For example:
# getbaseguid fc 0 Base GUID = 0x00bad0fc0009f000 #
Once you have recorded each fabric card’s GUID, you will have a list similar to the following on your Linux InfiniBand host:
0x00bad0fc0009f000 0x00bad0fc0033f100 0x00bad00017fcf200 0x00bad00005fcf300 0x00bad00003fcf400 0x00bad00007fcf500 0x00bad0fc0011f600 0x00bad0b00002f700 0x00bad0b00001f800
Note - If you have fewer than 9 fabric cards, your list will be smaller.
0xa2
0xb2
Note - For GUIDs that end with two zeros (00), you are effectively replacing the zeros with a2 and b2.
For example, 0x00bad0fc0009f000 becomes:
0x00bad0fc0009f0a2
0x00bad0fc0009f0b2
For example:
0x00bad0fc0009f0a2 0x00bad0fc0009f0b2 0x00bad0fc0033f1a2 0x00bad0fc0033f1b2 0x00bad00017fcf2a2 0x00bad00017fcf2b2 0x00bad00005fcf3a2 0x00bad00005fcf3b2 0x00bad00003fcf4a2 0x00bad00003fcf4b2 0x00bad00007fcf5a2 0x00bad00007fcf5b2 0x00bad0fc0011f6a2 0x00bad0fc0011f6b2 0x00bad0b00002f7a2 0x00bad0b00002f7b2 0x00bad0b00001f8a2 0x00bad0b00001f8b2
# chmod 0644 /etc/opensm/guid.txt #
For example:
# /etc/init.d/opensmd stop Stopping IB Subnet Manager..-. [ OK ] # /usr/sbin/opensm -R ftree -A -a /etc/opensm/guid.txt -s 120 -B
Switch Command Reference, getbaseguid command
Switch Command Reference, opensm command