C H A P T E R 2

Installing the Adapter

This chapter describes how to install the Sun Dual Port 4x DDR IB Host Channel Adapter PCIe ExpressModule and verify that it has been installed correctly. It contains the following sections:

Installing the Hardware

Verifying the Installation

Installing the Hardware

The following instructions describe the basic tasks required to install the ExpressModule. Refer to your system installation or service manual for detailed ExpressModule installation instructions.

To Install the ExpressModule

1. Halt and power off your system.

2. Install the ExpressModule into the ExpressModule slot, pushing the module’s edge connector into the connector on the chassis.

Ensure that the front plate on the ExpressModule card mounts flush with the chassis panel opening.

3. Connect the 4x end of the InfiniBand I/O cable(s) to the ExpressModule port connector(s).

Ensure that the connectors are properly engaged.

Caution - Avoid putting unnecessary stress on the connection. Do not bend or twist the cable near the connectors and avoid cable bends of more than 90 degrees.

4. If not already connected, connect the InfiniBand I/O cable(s) to the appropriate ports on the InfiniBand switch.

5. Turn power back on to the system and allow the server to reboot.

This completes the hardware installation. Proceed to the verification instructions in To Verify the Installation.

Verifying the Installation

Before you can verify the installation, you must install the ExpressModule in the chassis, power the server, and cable it to an operational InfiniBand switch. The InfiniBand switch should automatically recognize InfiniBand servers when they are connected to the fabric.

To Verify the Installation

1. Ensure that the cables are connected to the ExpressModule and switches.

2. Use the lspci command to verify that the IB Subnet Manager is running on the IB switch:

> lspci
...
80:0e.0 PCI bridge: nVidia Corporation CK804 PCIE Bridge (rev a3)
81:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20)

In this example, the IB ExpressModule (Mellanox InfiniHost III Lx HCA) is configured as PCI bus number 81, device 00, function 0 (81:00.0), which is EM slot 1 in a Sun Blade trademark 8000 P Series Modular System. On your system, you might see a different designation for the EM.

3. Check that the green LED is illuminated for each port that is connected to the switch.

If the green LED is not on, check the cable connections at the ExpressModule and at the switch.

4. Check that the amber LED is illuminated for each port that is connected to the switch.

If the amber LED is not on, refer to Appendix B, Diagnostic Software and Troubleshooting for more information.

5. Verify that the IB EM ports are up and the driver is attached, by typing:

> ibstat

The state of each port is displayed:

State	Comments
Active	The port is working.
Down	The port is not working or the cable is not connected. If the cable is connected, see Appendix B for troubleshooting information.
Initializing	If the initialization does not finish, the Subnet Manager might not be running for the subnet. If restarting the Subnet Manager has no effect, see Appendix B for troubleshooting information.