8.4 InfiniBand Connection Validation (Layer-2)

Once the cables are connected and you can see green LEDs on the ports, you must validate the InfiniBand connection.

1. Log in to the Oracle Big Data Appliance compute node or Exalytics machine.
2. From the Linux root shell, run the ibhosts command to verify if the newly connected (Exalytics/Oracle Big Data Appliance) machine is visible.
3. On the Exalytics machine, run the ibstat command to see the local ports' status. If an active Subnet Manager is present, then the ports should be Active.

   [root@exalytics ~]# ibstat
   
   CA 'mlx4_0'
   CA type: MT26428
   Number of ports: 2
   Firmware version: 2.7.8130
   Hardware version: b0
   Node GUID: 0x0021280001efe984
   System image GUID: 0x0021280001efe987
   Port 1:
   State: Active
   Physical state: LinkUp
   Rate: 40
   Base lid: 209
   LMC: 0
   SM lid: 187
   Capability mask: 0x02510868
   Port GUID: 0x0021280001efe985
   Link layer: IB
   Port 2:
   State: Active
   Physical state: LinkUp
   Rate: 40
   Base lid: 73
   LMC: 0
   SM lid: 187
   Capability mask: 0x02510868
   Port GUID: 0x0021280001efe986
   Link layer: IB
    
   [root@exalytics ~]# sminfo
   
   sminfo: sm lid 187 sm guid 0x10e00c16c0c0a0, activity count 124743184 priority 14 state 3 SMINFO_MASTER
   

4. Run the following command to make sure all the links are at 4X QDR:

   [root@exalytics ~]# ibdiagnet -ls 10 -lw 4x