1. Oracle Server X9-2L Service Manual
  2. Servicing PCIe Cards (CRU)

10 Servicing PCIe Cards (CRU)

This section describes how to service PCIe cards. PCIe cards are customer-replaceable units (CRUs) that require you to power off the server. For more information about CRUs, see Illustrated Parts Breakdown and Customer-Replaceable Units.

For complete software and cabling information about your card, refer to your PCIe card documentation.

Caution:

Do not use the procedures in this section to service the Oracle Storage 12 Gb SAS PCIe RAID HBA, Internal card or the Oracle Retimer card. Depending on your storage drive configuration, either card can be located in PCIe slot 10. The Oracle Storage 12 Gb SAS PCIe RAID HBA, Internal card and Oracle Retimer cards are field-replaceable units (FRUs) and should be serviced only by authorized Oracle Service personnel.

Caution:

These procedures require that you handle components that are sensitive to electrostatic discharge. This sensitivity can cause the components to fail. To avoid damage, ensure that you follow antistatic practices as described in Electrostatic Discharge Safety.

Caution:

Ensure that all power is removed from the server before removing or installing PCIe cards. You must disconnect all power cables from the system before performing these procedures.

Note:

For a complete list of supported PCIe cards, refer to the Oracle Server X9-2L Product Notes at Oracle Server X9-2L Documentation.

This section covers the following topics and procedures:

Servicing the Internal HBA Card and HBA Super Capacitor (FRU)

PCIe Slot Locations

There are eleven PCIe slots available for PCIe cards. The following figure shows the PCIe slot numbering.


Figure showing the PCIe slot numbering.
Callout Description

1

PCIe slot 1 (Nonfunctional in single-processor systems.)

2

PCIe slot 2 (Nonfunctional in single-processor systems.)

3

PCIe slot 3 (Nonfunctional in single-processor systems.)

4

PCIe slot 4 (Nonfunctional in single-processor systems.)

5

PCIe slot 5 (Nonfunctional in single-processor systems.)

6

(Oracle internal use only) Dedicated Root-of-Trust (RoT) card slot

7

PCIe slot 6

8

PCIe slot 7

9

PCIe slot 8

10

PCIe slot 9

11

PCIe slot 10 (This PCIe slot can contain the Oracle Storage 12 Gb SAS PCIe RAID HBA, Internal card for SAS storage drive configurations, or the Oracle Retimer card for NVMe storage drive configurations that include NVMe SSDs in PCIe slots 8 through 11.)

Note:

All of the PCIe slots comply with the PCI Express 4.0 specification and can accommodate 75 Watt PCIe cards.

Remove a PCIe Card

Note:

PCIe slots 1 through 5 are nonfunctional in single-processor systems.

Note:

Before replacing or installing a PCIe card, refer to the card documentation for specific installation and cabling instructions.
  1. Prepare the server for service.
    1. Power off the server and disconnect the power cords from the power supplies.
    2. Extend the server to the maintenance position.
    3. Attach an antistatic wrist strap.
    4. Remove the server top cover.
  2. Locate the PCIe card that you want to remove.
  3. If necessary, record where the PCIe cards are installed.
  4. Unplug all data cables from the PCIe card [1].

    Note the cable connections so that it's easier to properly reconnect the cables.

    Note:

    If it is difficult to disconnect the cables from the PCIe card when the card is inserted into the PCIe slot, consider performing the procedure in Step 5 before completing this step.
  5. Rotate the PCIe card locking mechanism in to an upright position [1], and then lift up on the PCIe card to disengage it from the motherboard connectors [2].

    Figure showing a PCIe card being removed from the server.
  6. Place the PCIe card on an antistatic mat.

    Caution:

    If you are not immediately inserting a replacement PCIe card into the empty slot, insert a PCIe filler panel in the slot to reduce the possibility of radiated electromagnetic interference (EMI). See Remove and Install Filler Panels.

Install a PCIe Card

Note:

PCIe slots 1 through 5 are nonfunctional in single-processor systems.
  1. Unpack the replacement PCIe card, and place it on an antistatic mat.
  2. Locate the proper PCIe slot for the card you are replacing.

    Note:

    PCIe slot 10 is the primary slot for the Oracle Storage 12 Gb SAS PCIe HBA, Internal card or the Oracle Retimer card. The Oracle Storage 12 Gb SAS PCIe HBA, Internal card manages SAS storage drives. The Oracle Retimer card manages NVMe storage drives in slots 8 through 11.
  3. If necessary, remove the PCIe filler panel from the slot.

    Note:

    Save this filler panel in case you need to remove the PCIe card from the system.
  4. Insert the PCIe card into the correct slot [1], and rotate the PCIe locking mechanism downward to secure the PCIe card in place [2].

    You hear an audible click when the PCIe card is secured into the slot.

    Note:

    If it is difficult to reconnect the cables to the PCIe card when the card is inserted into the PCIe slot, consider performing the procedure in Step 5 before completing this step.

    Figure showing a PCIe card being installed into the server.
  5. Reconnect the cables to the PCIe card that you unplugged during the removal procedure [2].

    Note:

    To ensure proper SAS cable connections for Oracle Storage 12 Gb SAS PCIe HBA, Internal card, see the SAS cabling reference table in Step 3 in Install SAS Storage Drive Cables. To ensure proper NVMe cable connections for the Oracle Retimer card, see the NVMe cabling reference table in Step 3 in Install NVMe Cables, and perform the procedures in Verify NVMe Cable Connections.
  6. Return the server to operation.
    1. Install the top cover.
    2. Return the server to the normal rack position.
    3. Connect any required data cables to the PCIe card.

      Route data cables through the cable management arm.

    4. Reconnect the power cords to the power supplies and power on the server.
    5. Verify that the System OK LED is lit.
  7. Use Oracle ILOM to clear any server PCIe card faults.

    If a PCIe card fault message in Oracle ILOM is not cleared under Open Problems, you must manually clear the fault using Oracle ILOM. refer to "Clear Faults for Undetected Replaced or Repaired Hardware Components" in the Oracle ILOM User's Guide for System Monitoring and Diagnostics Firmware Release 5.0.x at Oracle Integrated Lights Out Manager (ILOM) 5.0 Documentation.

  8. To determine whether additional steps are required to complete the installation of the PCIe card, refer to the server product notes for the type of PCIe card you installed.