Go to main content

Oracle® Server X8-2L Service Manual

Exit Print View

Updated: October 2019
 
 

Verify NVMe Cable Connections

When NVMe cables are removed or replaced between the disk backplane and NVMe switch cards, you must perform the procedure in this section to confirm that all NVMe cable connections are correct. If NVMe cable connections are not correct, the server operating system should not be allowed to boot, as it could cause a problem with disk drive mapping.

Before beginning this procedure, ensure that the server has been returned to operation. See Step 4 in Install NVMe Cables. This procedure requires access to the Oracle ILOM service processor (SP). The BIOS Setup Utility is used to halt the system so NVMe cable connection tests can be run for NVMe storage drives.

You can access the BIOS Setup Utility screens from the following interfaces:

  • Use a terminal (or terminal emulator connected to a computer) through the serial port on the back panel of the server.

  • Connect to the server using the Oracle ILOM Remote System Console Plus application.

  1. Power on the server.

    For example, to power on the server:

    • From the local server, press the On/Standby button on the front bezel.

    • From the Oracle ILOM web interface, click Host Management → Power Control and select Power On from the Select Action list.

    • From the Oracle ILOM CLI, type start /System

    The power-on self-test (POST) sequence begins.

  2. To enter the BIOS Setup Utility, press the F2 key (Ctrl+E from a serial connection) when prompted and while the BIOS is running the power-on self-tests (POST).

    This step halts the system so NVMe cable connection tests can be run for the NVMe storage drives.

  3. Open a terminal window. Using SSH (secure shell), log in to the Oracle ILOM command-line interface (CLI) on the server SP.
  4. To access the Oracle ILOM Diagnostic Shell, type the start /sp/diag/shell command, and then click Enter.
  5. Type y to confirm that you want to start the Oracle ILOM Diagnostic Shell.

    The diag> prompt appears.

  6. To run the NVMe cable connection tests, type the hwdiag io nvme_test command, and then click Enter.

    Your output should be similar to the following example:

    diag> hwdiag io nvme_test
    HWdiag (Diag Shell) - Build Number 120817 (Aug 21 2017, 00:15:43)
             Current Date/Time: Oct 04 2017, 17:19:34
        Checking NVME drive fru contents...
            checking fru on drive NVMe 0              OK
            checking fru on drive NVMe 1              OK
            checking fru on drive NVMe 2              OK
            checking fru on drive NVMe 3              OK
            checking fru on drive NVMe 4              OK
            checking fru on drive NVMe 5              OK
            checking fru on drive NVMe 6              OK
            checking fru on drive NVMe 7              OK
            checking fru on drive NVMe 8              OK
            checking fru on drive NVMe 9              OK
            checking fru on drive NVMe 10             OK
            checking fru on drive NVMe 11             OK
        NVME drives fru check:                        PASSED
    
        Checking NVME drive pcie links...
            checking pcie link on drive NVMe 0        OK
            checking pcie link on drive NVMe 1        OK
            checking pcie link on drive NVMe 2        OK
            checking pcie link on drive NVMe 3        OK
            checking pcie link on drive NVMe 4        OK
            checking pcie link on drive NVMe 5        OK
            checking pcie link on drive NVMe 6        OK
            checking pcie link on drive NVMe 7        OK
            checking pcie link on drive NVMe 8        OK
            checking pcie link on drive NVMe 9        OK
            checking pcie link on drive NVMe 10       OK
            checking pcie link on drive NVMe 11       OK
        NVME drives pcie link check:                  PASSED
    
        Checking NVME drive DSN...
            checking DSN on drive NVMe 0              OK
            checking DSN on drive NVMe 1              OK
            checking DSN on drive NVMe 2              OK
            checking DSN on drive NVMe 3              OK
            checking DSN on drive NVMe 4              OK
            checking DSN on drive NVMe 5              OK
            checking DSN on drive NVMe 6              OK
            checking DSN on drive NVMe 7              OK
            checking DSN on drive NVMe 8              OK
            checking DSN on drive NVMe 9              OK
            checking DSN on drive NVMe 10             OK
            checking DSN on drive NVMe 11             OK
        NVME drives DSN check:                        PASSED 
    
        Checking NVME cabling...
            Cables associated with Switch Card 0 in PCIe Slot 3 verified
            Cables associated with Switch Card 1 in PCIe Slot 4 verified
            Cables associated with Switch Card 2 in PCIe Slot 8 verified
            Cables associated with Switch Card 3 in PCIe Slot 9 verified
        NVME cable check:                             PASSED
    
    NVME test PASSED
    diag>

    If any of the tests fail, there could be a problem with the cabling. For example:

    • If a cable is crossed (plugged into the incorrect port), then the NVMe drive DSN (Drive Serial Number) test should report two drives where the DSN does not match. The cabling for those drives should be inspected to ensure that it is correct.

    • If multiple NVMe drive PCIe links are not active, or multiple NVMe drive DSNs are not OK, then check the Port 2 connection on the NVMe switch card. If Port 2 is incorrectly cabled, it could affect all three drives that are connected to that NVMe switch card.

  7. Once issues are resolved, power cycle the system by removing the power cords from the power supplies, waiting 30 seconds, and then reattaching the power cords.
  8. Retest the system by repeating Step 1 through Step 6.

    Once all NVMe cable connection tests have passed, the server operating system can be booted.