1. Determine if there are errors or problems with the InfiniBand fabric.

   See:

   • Perform Comprehensive Diagnostics for the Entire Fabric

   • Find 1x or SDR or DDR Links in the Fabric

   • Determine Which Links Are Experiencing Significant Errors

2. Locate the affected nodes by the GUID provided in the output of the ibdiagnet command.

   See Locate a Switch Chip or Connector From the GUID.

3. If the problem is at a cable connection, swap the suspect cable with a known good cable or reconnect the cable to a known good remote port and repeat Step 1.

   See Switch Service, servicing an InfiniBand cable.

4. If the problem still remains at the cable connection, disable and re-enable that port on the line card and repeat Step 1.

   See Disable a Port and Enable a Port.

5. If the problem is within a line card or fabric card, disable and re-enable the respective port.

   See Disable a Port and Enable a Port.

6. If the problem still remains within a line card or fabric card, reduce the local deflection of the midplane.

   Unscrew the retainer bolts of the affected fabric cards or line cards or both by 3/4 turn. Alternate between retainer bolts, turning each 1/4 turn counter-clockwise each time and then reseat the fabric card or line card.

   See Switch Service, servicing a fabric card, servicing a line card.

Temporary solution:

• If the problem still remains, disable the affected port.

  See Disable a Port.

Permanent solution:

• If the problem still remains, replace the affected component.

  See Switch Service, replacing a fabric card, replacing a line card, replacing an InfiniBand cable.

  See remote port’s documentation for replacement procedures.

1. Disconnect and properly reconnect both ends of the respective InfiniBand cable.

   See Switch Service, servicing an InfiniBand cable.

2. If the LED is still blinking, determine the significance of the errors through use of the ibdiagnet command.

   See Determine Which Links Are Experiencing Significant Errors.

3. Determine which connectors map to the affected link.

   See Locate a Switch Chip or Connector From the GUID.

4. If some of the links are running at 1x or SDR, use that situation elsewhere in this table to rectify the problem.

5. Disable and re-enable the respective ports.

   See Disable a Port and Enable a Port.

6. If the errors are still significant, swap the cable with a known good one or reconnect the cable to a known good remote port, and repeat from 2.

7. Depending upon what does or does not rectify the problem, replace that component.

   See Switch Service, replacing a InfiniBand cable, replacing a line card.

   See remote port’s documentation for replacement procedures.

1. Clear the error counters.

   See Clear Error Counters.

2. Start a fabric stress test.

3. Identify the suspect links using the ibdiagnet command.

   See Determine Which Links Are Experiencing Significant Errors. Look for text like the following:

   -W- lid=0x0006 guid=0x0021283a8816c0a0 dev=48438 Port=34

   Performance Monitor counter : Value

   link_recovery_error_counter : 0x1

   symbol_error_counter : 0x25 (Increase by 3 during ibdiagnet)

4. For links that are experiencing recovery errors or substantial symbol errors, refer to other parts of this table to help identify the cause and rectify the problem.

1. Identify the suspect links using the ibdiagnet command.

   See Find 1x or SDR or DDR Links in the Fabric. Look for text like the following:

   -W- link with SPD=2.5 found at direct path "1,19"

   From: a Switch PortGUID=0x00066a00d80001dd Port=19

   To: a Switch PortGUID=0x00066a00d80001dd Port=24

2. Determine which connectors map to the affected link.

   See Locate a Switch Chip or Connector From the GUID.

3. Verify the cable connection at both ends.

   See Switch Service, servicing an InfiniBand cable.

4. Disable and re-enable the respective ports.

   See Disable a Port and Enable a Port.

5. If the previous steps do not rectify the problem, disable the port.

   See Disable a Port.

For a permanent solution:

1. Perform the steps for a temporary solution, steps 1 to step 5.

2. Swap the cable with a known good one or reconnect the cable to a known good remote port, and repeat from 1.

3. Depending upon what does or does not rectify the problem, replace that component.

   See Switch Service, replacing a InfiniBand cable, replacing a fabric card, replacing a line card.

   See remote port’s documentation for replacement procedures.

1. The findport switch-specific command can translate GUID port combinations to the location in the switch.

   See Locate a Switch Chip or Connector From the GUID.

2. If the port immediately links to a CXP connector, the findport command identifies that connector.

   See Switch Reference, findport command.