When NVMe cables are removed or replaced between the storage drive backplane and NVMe switch cards, you must perform the procedure in this section to confirm that all NVMe cable connections are correct. If NVMe cable connections are not correct, the storage server operating system should not be allowed to boot, as it could cause a problem with disk drive mapping.
Before beginning this procedure, ensure that the storage server has been returned to operation. See Step 4 in Install NVMe Cables. This procedure requires access to the Oracle ILOM service processor (SP). The BIOS Setup Utility is used to halt the system so NVMe cable connection tests can be run for NVMe storage drives.
You can access the BIOS Setup Utility screens from the following interfaces:
Use a USB keyboard and VGA monitor connected directly to the server. (A mouse is not required to access the BIOS Setup Utility.)
Use a terminal (or terminal emulator connected to a computer) through the serial port on the back panel of the server.
Connect to the server using the Oracle ILOM Remote System Console Plus application.
For example, to power on the server:
From the local server, press the Power button on the front panel of the server to power on the server.
From the Oracle ILOM web interface, click Host Management → Power Control and select Power On from the Select Action list.
From the Oracle ILOM CLI, type start /System
The power-on self-test (POST) sequence begins.
This step halts the system so NVMe cable connection tests can be run for the NVMe storage drives.
The # prompt appears.
Your output should be similar to the following example:
hwdiag io nvme_test HWdiag - Build Number 92713 (Sep 18 2014, 18:11:57) Current Date/Time: Sep 26 2014, 11:38:30 Checking NVME drive fru contents... checking fru on drive NVMe 0 OK checking fru on drive NVMe 1 OK checking fru on drive NVMe 2 OK checking fru on drive NVMe 3 OK checking fru on drive NVMe 4 OK checking fru on drive NVMe 5 OK checking fru on drive NVMe 6 OK checking fru on drive NVMe 7 OK checking fru on drive NVMe 8 OK checking fru on drive NVMe 9 OK checking fru on drive NVMe 10 OK checking fru on drive NVMe 11 OK NVME drives fru check: PASSED Checking NVME drive pcie links... checking pcie link on drive NVMe 0 OK checking pcie link on drive NVMe 1 OK checking pcie link on drive NVMe 2 OK checking pcie link on drive NVMe 3 OK checking pcie link on drive NVMe 4 OK checking pcie link on drive NVMe 5 OK checking pcie link on drive NVMe 6 OK checking pcie link on drive NVMe 7 OK checking pcie link on drive NVMe 8 OK checking pcie link on drive NVMe 9 OK checking pcie link on drive NVMe 10 OK checking pcie link on drive NVMe 11 OK NVME drives pcie link check: PASSED Checking NVME drive DSN... checking DSN on drive NVMe 0 OK checking DSN on drive NVMe 1 OK checking DSN on drive NVMe 2 OK checking DSN on drive NVMe 3 OK checking DSN on drive NVMe 4 OK checking DSN on drive NVMe 5 OK checking DSN on drive NVMe 6 OK checking DSN on drive NVMe 7 OK checking DSN on drive NVMe 8 OK checking DSN on drive NVMe 9 OK checking DSN on drive NVMe 10 OK checking DSN on drive NVMe 11 OK NVME drives DSN check: PASSED Checking NVME cabling... Cables associated with Switch Card 3 in PCIe Slot 6 verified Cables associated with Switch Card 2 in PCIe Slot 5 verified Cables associated with Switch Card 1 in PCIe Slot 2 verified Cables associated with Switch Card 0 in PCIe Slot 1 verified NVME cable check: PASSED NVME test PASSED
If any of the tests fail, there could be a problem with the cabling. For example:
If a cable is crossed (plugged into the incorrect port), then the NVMe drive DSN (Drive Serial Number) test should report two drives where the DSN does not match. The cabling for those drives should be inspected to ensure that it is correct.
If multiple NVMe drive PCIe links are not active, or multiple NVMe drive DSNs are not OK, then check the Port 2 connection on the NVMe switch card. If Port 2 is incorrectly cabled, it could affect all three drives that are connected to that NVMe switch card.
Once all NVMe cable connection tests have passed, the storage server operating system can be booted.