Hot-Swap the ExpressModule (Linux)

This procedure is for removing and installing an ExpressModule without powering off the system.

1. Verify that the IB status is okay:

   > ibstat
   CA 'mlx4_0'
           CA type: MT26428
           Number of ports: 2
           Firmware version: 2.6.0
           Hardware version: a0
           Node GUID: 0x0002c903000379a4
           System image GUID: 0x0002c903000379a7
           Port 1:
                   State: Active
                   Physical state: LinkUp
                   Rate: 20
                   Base lid: 26
                   LMC: 0
                   SM lid: 23
                   Capability mask: 0x02510868
                   Port GUID: 0x0002c903000379a5
           Port 2:
                   State: Down
                   Physical state: Polling
                   Rate: 10
                   Base lid: 28
                   LMC: 0
                   SM lid: 1
                   Capability mask: 0x02510868
                   Port GUID: 0x0002c903000379a6

2. Load the acpiphp module:

   ---

   Note - Do not use the acpiphp command for hot-plug operations in SUSE Linux SLES10SP2. Use the modprobe pciehp pciehp_poll_mode=1 command instead. The remaining steps and commands are the same for the SUSE Linux SLES10SP2 hot-plug procedure.

   ---

   > modprobe acpiphp

3. If IPoIB is configured, type:

   > ifconfig ibn down

   where n is the instance number of the ExpressModule.

4. Press the Hot-Swap button on the ExpressModule.

   Within eight seconds, the green LEDs will extinguish. The ExpressModule is now physically ready to be removed from the chassis slot.

5. Type:

   > /etc/init.d/openibd status
   HCA driver loaded
    
   Configured IPoIB devices:
    
   Currently active IPoIB devices:
    
   The following OFED modules are loaded:
    
     rdma_ucm
     ib_sdp
     rdma_cm
     ib_addr
     ib_ipoib
     mlx4_core
     mlx4_ib
     ib_mthca
     ib_uverbs
     ib_umad
     ib_ucm
     ib_sa
     ib_cm
     ib_mad
     ib_core

   The output displays the driver as loaded but not attached to any device, which indicates the driver is stable.

6. Insert the ExpressModule into the chassis slot.
7. After approximately one minute, after all the LEDS are extinguished, press the Hot-Swap button again.

   The LEDs start to blink and both the ExpressModule and the ports become active in nine seconds.

8. Verify that the InfiniBand ports are up:

   > ibstat
   CA 'mlx4_0'
           CA type: MT26428
           Number of ports: 2
           Firmware version: 2.6.0
           Hardware version: a0
           Node GUID: 0x0002c903000379a4
           System image GUID: 0x0002c903000379a7
           Port 1:
                   State: Active
                   Physical state: LinkUp
                   Rate: 40
                   Base lid: 26
                   LMC: 0
                   SM lid: 23
                   Capability mask: 0x02510868
                   Port GUID: 0x0002c903000379a5
           Port 2:
                   State: Active
                   Physical state: LinkUp
                   Rate: 40
                   Base lid: 28
                   LMC: 0
                   SM lid: 1
                   Capability mask: 0x02510868
                   Port GUID: 0x0002c903000379a6

9. Verify the installation.

   See Verify the Installation (Linux).

Related Information

• Install the ExpressModule

• Hot-Swap the ExpressModule (Oracle Solaris)