When NVMe cables are removed or replaced between the disk backplane and NVMe switch cards, you must perform the procedure in this section to confirm that all NVMe cable connections are correct. If NVMe cable connections are not correct, the server operating system should not be allowed to boot, as it could cause a problem with disk drive mapping.
Before beginning this procedure, ensure that the server has been returned to operation. See Step 4 in Install NVMe Cables. This procedure requires access to the Oracle ILOM service processor (SP). The BIOS Setup Utility is used to halt the system so NVMe cable connection tests can be run for NVMe storage drives.
You can access the BIOS Setup Utility screens from the following interfaces:
Use a terminal (or terminal emulator connected to a computer) through the serial port on the back panel of the server.
Connect to the server using the Oracle ILOM Remote System Console Plus application.
For example, to power on the server:
From the local server, press the On/Standby button on the front bezel.
From the Oracle ILOM web interface, click Host Management → Power Control and select Power On from the Select Action list.
From the Oracle ILOM CLI, type start /System
The power-on self-test (POST) sequence begins.
This step halts the system so NVMe cable connection tests can be run for the NVMe storage drives.
The diag> prompt appears.
Your output should be similar to the following example:
diag> hwdiag io nvme_test HWdiag (Diag Shell) - Build Number 120817 (Aug 21 2017, 00:15:43) Current Date/Time: Oct 04 2017, 17:19:34 Checking NVME drive fru contents... checking fru on drive NVMe 0 OK checking fru on drive NVMe 1 OK checking fru on drive NVMe 2 OK checking fru on drive NVMe 3 OK checking fru on drive NVMe 4 OK checking fru on drive NVMe 5 OK checking fru on drive NVMe 6 OK checking fru on drive NVMe 7 OK checking fru on drive NVMe 8 OK checking fru on drive NVMe 9 OK checking fru on drive NVMe 10 OK checking fru on drive NVMe 11 OK NVME drives fru check: PASSED Checking NVME drive pcie links... checking pcie link on drive NVMe 0 OK checking pcie link on drive NVMe 1 OK checking pcie link on drive NVMe 2 OK checking pcie link on drive NVMe 3 OK checking pcie link on drive NVMe 4 OK checking pcie link on drive NVMe 5 OK checking pcie link on drive NVMe 6 OK checking pcie link on drive NVMe 7 OK checking pcie link on drive NVMe 8 OK checking pcie link on drive NVMe 9 OK checking pcie link on drive NVMe 10 OK checking pcie link on drive NVMe 11 OK NVME drives pcie link check: PASSED Checking NVME drive DSN... checking DSN on drive NVMe 0 OK checking DSN on drive NVMe 1 OK checking DSN on drive NVMe 2 OK checking DSN on drive NVMe 3 OK checking DSN on drive NVMe 4 OK checking DSN on drive NVMe 5 OK checking DSN on drive NVMe 6 OK checking DSN on drive NVMe 7 OK checking DSN on drive NVMe 8 OK checking DSN on drive NVMe 9 OK checking DSN on drive NVMe 10 OK checking DSN on drive NVMe 11 OK NVME drives DSN check: PASSED Checking NVME cabling... Cables associated with Switch Card 0 in PCIe Slot 3 verified Cables associated with Switch Card 1 in PCIe Slot 4 verified Cables associated with Switch Card 2 in PCIe Slot 8 verified Cables associated with Switch Card 3 in PCIe Slot 9 verified NVME cable check: PASSED NVME test PASSED diag>
If any of the tests fail, there could be a problem with the cabling. For example:
If a cable is crossed (plugged into the incorrect port), then the NVMe drive DSN (Drive Serial Number) test should report two drives where the DSN does not match. The cabling for those drives should be inspected to ensure that it is correct.
If multiple NVMe drive PCIe links are not active, or multiple NVMe drive DSNs are not OK, then check the Port 2 connection on the NVMe switch card. If Port 2 is incorrectly cabled, it could affect all three drives that are connected to that NVMe switch card.
Once all NVMe cable connection tests have passed, the server operating system can be booted.