1. Service Manual
  2. Servicing CRUs That Do Not Require System Power Off

3 Servicing CRUs That Do Not Require System Power Off

Perform tasks for servicing hot plug components in the Oracle Database Appliance.

Hot-pluggable components can be replaced with the system powered on.

Note:

Hot-pluggable components can also be replaced with the system powered off. If you choose this method, first power off the system as described in Preparing Oracle Database Appliance for Service.

Oracle Database Appliance X9-2 Server Hot-Pluggable Components

Provides instructions on how to perform hot plug tasks on an Oracle Database Appliance server.

This section describes how to service customer-replaceable components that do not require server power off for an Oracle Database Appliance X9-2S/X9-2L server or X9-2-HA server node.

Note:

In certain storage expansion scenarios (explained in the following procedures), adding optional storage might require that the appliance be powered off.

If you are servicing storage shelf hot plug components, see Oracle Database Appliance Storage Shelf Hot-Pluggable Components.

Hot-pluggable components are those that can be removed and installed while the system is running. These are designated as Customer Replaceable Units (CRUs).

Note:

After a physical insertion or removal action of a storage component (such as a disk or SAS cable), you should wait before performing the next action to allow the system software to fully process the event. This could take up to 3 minutes to complete.

For instructions on servicing NVMe front-mounted storage devices, see Servicing NVMe Storage Drives (CRU). Note that front mounted storage drives are not supported in the Oracle Database Appliance X9-2-HA server nodes.

Caution: An NVMe drive must first be powered off before removal. If you do not power off the NVMe drive before removal you can crash the operating system. Oracle Database Appliance uses special commands to remove or add NVMe storage.

Refer to the following table for hot-plug component instructions for your appliance.

Note:

Some of the links in the following table take you to instructions outside of the Oracle Database Appliance library to the Oracle Server X9-2L documentation library. Use your browser's Back button to return to the Oracle Database Appliance library.

Oracle Database Appliance X9-2S/X9-2L Oracle Database Appliance X9-2-HA

Servicing NVMe Storage Drives (CRU)

Caution: An NVMe drive must first be powered off before removal. If you do not power off the NVMe drive before removal you can crash the operating system. Oracle Database Appliance uses special commands to remove or add NVMe storage.

Front mounted storage drives are not supported in the server nodes for this product.

Servicing Internal M.2 Flash SSDs (CRU)

Servicing Internal M.2 Flash SSDs (CRU)

Servicing Fan Modules (CRU)

Servicing Fan Modules (CRU)

Servicing Power Supplies (CRU)

Servicing Power Supplies (CRU)
  • Servicing NVMe Storage Drives (CRU)
    Perform tasks for servicing front-mounted NVMe drives in an Oracle Database Appliance X9-2S/X9-2L system. The Oracle Database Appliance X9-2-HA does not support front-mounted drives in server nodes.

Parent topic: Servicing CRUs That Do Not Require System Power Off

Servicing NVMe Storage Drives (CRU)

Perform tasks for servicing front-mounted NVMe drives in an Oracle Database Appliance X9-2S/X9-2L system. The Oracle Database Appliance X9-2-HA does not support front-mounted drives in server nodes.

Parent topic: Oracle Database Appliance X9-2 Server Hot-Pluggable Components

Replacing an NVMe Storage Drive

Replace a front-mounted NVMe drive in an Oracle Database Appliance X9-2S/X9-2L system.

Before removing an NVMe drive it must be powered off. After installing an NVMe drive it must be powered on. Failed drives are automatically powered off after 5 minutes.

Caution:

If you remove an NVMe drive without first powering it off, you can crash the operating system. Oracle Database Appliance uses special commands to add or remove NVMe storage.

The instructions describe how to replace an NVMe drive using the odaadmcli power on/off commands for an Oracle Database Appliance server.

The odaadmcli commands listed here must be run as the root user.

  1. Identify the NVMe drive to be replaced. Type the following command at the host:
    # odaadmcli show disk

    For example, the drive names pd_00 and pd_01 are seen by the host as NVMe0 and NVMe1 and correspond to front drive slots 0 and 1, respectively.

  2. Unmount and power off the drive to be replaced. Do one of the following:
    1. If you are replacing a failed drive, check to see if its blue ready-to-remove light is lit.
      If a drive has failed, it should power off automatically and light its blue ready-to-remove indicator after 5 minutes.
    2. If the drive being replaced doesn't already have its blue ready-to-remove indicator lit, type the following command from the host to unmount and turn off power to the drive:
      # odaadmcli power disk off pd_xx
      Where pd_xx is the name of the disk to be replaced as described in step 1.
      Before proceeding to the next step confirm that the blue ready-to-remove indicator on the drive is lit.
  3. Push the latch release button on the front of the drive to open the drive latch [1], grasp the latch, and pull the drive out of the drive slot [2] as shown in the following figure.
  4. Align the replacement drive with the drive slot.
    The drive is physically addressed according to the slot in which it is installed. It is important to install a replacement drive in the same slot as the drive that was removed.
  5. With the drive latch in the open position, slide the drive into the slot until the drive is fully seated [1] and then close the drive latch to lock the drive in place [2] as shown in the following figure.
  6. Power on and remount the replacement drive. Type the following command from the host:
    # odaadmcli power disk on pd_xx

    Where pd_xx is the name of the disk being replaced as described in step 1.

    It can take a few minutes for the power-on operation to complete. Once complete, the drive's green OK indicator will light.

  7. To check the status of the replacement disk, type the following command from the host:
    # odaadmcli show disk

    For additional information, refer to the Oracle Database Appliance Deployment and User's Guide.

Adding NVMe Expansion Storage

Add NVMe storage to an Oracle Database Appliance X9-2L. Note that the Oracle Database Appliance X9-2S does not support additional storage.

  • The Oracle Database Appliance X9-2L supports ten expansion slots for NVMe drives. These are slots labeled 2 through 11 as seen from the front of the system (callouts 2 through 11 in the following illustration). Storage drives can be added in sets of two. For information about supported optional storage drives, refer to the Owner's Guide.


    Description of g7745_x9-2l-drive-expansion-slots.jpg follows
    Description of the illustration g7745_x9-2l-drive-expansion-slots.jpg

  • When expanding storage, populate slots starting with the lowest numbered available slot (for example, slot 2 as seen on the front panel).

  • The drive power-up and storage expansion odaadmcli commands listed here should only be run after all new disk drives have been physically installed. Once you expand storage, you cannot revert to a previous storage configuration (for example, the default installation with only two NVMe drives).

  1. Remove the drive filler panel of an unpopulated slot by using your finger to pry open the panel release latch and pulling the filler out of the drive slot.

    Note:

    Replace filler panels with drives one at a time to ensure proper system cooling.

  2. Open the latch of the new drive by pressing the release button on the drive.

    Caution:

    Do not over-rotate the latch outwards when opening. Doing so can damage it.

  3. Insert the new drive by sliding the drive into the slot until the drive is fully seated. Then close the drive latch to lock the drive in place.
  4. Repeat steps 1 - 3 for each new drive.

    For the Oracle Database Appliance X9-2L, the name for these drives to the host will be pd_02, pd_03, pd_04, etc., matching the numbering of their respective drive slots up to pd_11.

  5. Power on each of the new drives by entering the appropriate commands from the host using the following command format:

    # odaadmcli power disk on pd_##

    Where ## is the number of the drive being powered on (02 through 11).

    It can take a few minutes for the power-on operation to complete. Once complete, the green OK indicators will light for the newly installed drives.

    Note:

    Do not attempt to power on and expand storage on one new disk and then power on and expand storage on the next new disk. Before you can expand storage, all the new disks must be powered on with their green OK indicators lit.

  6. Expand the system storage to include the new drives. Type the following command from the host:
    # odaadmcli expand storage -ndisk number_of_disks

    Where number_of_disks is the number of disks being added. The system will then detect and use the new storage.

    For additional information, refer to the Oracle Database Appliance Deployment and User's Guide.

Oracle Database Appliance X9-2-HA Storage Shelf Hot Pluggable Components

Perform hot plug tasks on an Oracle Database Appliance storage shelf.

Under most circumstances, Oracle Database Appliance X9-2-HA storage shelf components can all be replaced while the storage shelf is powered-on and running. Any exceptions will be described in the following procedures.

Parent topic: Servicing CRUs That Do Not Require System Power Off

Adding Optional Storage Shelf Drives (CRU)

Add disk storage in an Oracle Database Appliance X9-2-HA storage shelf.

The Oracle Database Appliance X9-2-HA storage shelf partially-populated base configuration supports storage expansion with either 7.68 TB SSDs (available in six-packs) or 18 TB HDDs (available in an eighteen-pack). Depending on your initial base storage shelf configuration, there can be up to eighteen expansion slots. These slots are labeled 6-23 as seen from the front of the system (callouts 6 through 23 in the following illustration).


Description of gxxxx-x8-2ha_optional_shelf_drives_top.jpg follows
Description of the illustration gxxxx-x8-2ha_optional_shelf_drives_top.jpg

  • Upgrade options:

    • For a high performance shelf, you can add up to eighteen 7.68 TB SSDs (available in six-packs). You can add storage with the system running and use the appliance CLI software commands to expand the storage configuration.

    • For a high capacity shelf, you can add eighteen 18 TB HDDs (available in an eighteen-pack). You must power off the appliance, add storage, then re-image and redeploy the appliance software. This is required to reconfigure the appliance from the default high performance mode to high capacity mode.

  • Expanding the base configuration storage shelf with 7.68 TB SSDs for high performance. The base configuration has slots 0-5 with 7.68 TB drives for DATA/RECO. Expansion SSDs are available in six-packs and are populated in shelf storage slots from the bottom up to also be used for DATA/RECO, as follows:

    • The first six expansion drives go into slots 6-11.
    • The second six expansion drives go into slots 12-17.
    • The third six expansion drives go into slots 18-23.

    The following illustration shows the expanded drive configuration for 7.68 TB SSDs.


    Description of oda_x8-2-ha_shelf_ssds.jpg follows
    Description of the illustration oda_x8-2-ha_shelf_ssds.jpg

  • Expanding the base configuration storage shelf with 18 TB HDDs for high capacity. When expanding the storage shelf with HDDs, the base configurations SSDs will be used for FLASH and the HDDs will be used for DATA/RECO. Expansion HDDs are available in an eighteen-pack and populate the empty shelf storage slots (slots 6-23).

    The following illustration shows the expanded drive configuration for 18 TB HDDs.


    Description of g7746_x9-2-ha-shelf-hdd.jpg follows
    Description of the illustration g7746_x9-2-ha-shelf-hdd.jpg

  • Once you expand storage to support additional disks, you cannot revert to the base configuration (no drives in slots 6-23).

  • If you plan on adding another storage shelf, it must be populated identically to your first storage shelf.

  • The odaadmcli commands listed here must be run as the root user.

  1. Do one of the following:

    • If you are adding 7.68 TB SSDs to the storage shelf, leave the appliance powered on and proceed to the next step.

    • If you are adding 18 TB HDDs to the storage shelf, power off the appliance and proceed to the next step.

  2. To install the new drives into the storage shelf, do the following:
    1. Remove one of the empty slot drive filler panels by pressing its release button [1] and removing it from the drive slot [2] as shown in the following figure.

      Note:

      Replace filler panels with drives one at a time to ensure proper storage shelf chassis cooling.

      Description of g5703_de3-24c_drive_removal1.jpg follows
      Description of the illustration g5703_de3-24c_drive_removal1.jpg
    2. Open the drive lever of the new drive by pressing the release button on the drive.
    3. Slide the new drive into the drive slot [1] until it engages with the backplane connector and then close the lever to fully seat the drive and lock it into place [2] as shown in the following figure.

      Check that the drive is locked in place and flush with the other drives.

      Description of g5704_de3-24c_drive_replacement1.jpg follows
      Description of the illustration g5704_de3-24c_drive_replacement1.jpg
    4. Repeat steps a through c until all of the new drives are installed in the storage shelf.
  3. Do one of the following:

    • If you added 7.68 TB SSDs to the storage shelf, check that the OK indicator on each new drive is steady green to indicate that it is ready. Then expand the system storage to include the new drives by typing the following command from the primary server node (node 0):

      # odaadmcli expand storage -ndisk number_of_disks -enclosure 0

      Where number_of_disks is the number of disks being added. The system will then detect and use the new storage. It can take up to 12 minutes to update the configuration. For additional storage commands, refer to the Deployment and User's Guide.

    • If you added 18 TB HDDs to the storage shelf, power on the appliance, then re-image and redeploy the appliance software as described in the Deployment and User's Guide.

  4. If you are also adding a storage expansion shelf (can only be added to a system with an identically and fully populated storage shelf), you can do so after successfully completing the previous step.

    Refer to the Oracle Database Appliance Owner's Guide for information on installing a storage expansion shelf into a rack.

    Refer to the Oracle Database Appliance Deployment and User's Guide for information on cabling and storage expansion commands for a storage expansion shelf.

Parent topic: Oracle Database Appliance X9-2-HA Storage Shelf Hot Pluggable Components

Replace a Storage Shelf HDD or SSD (CRU)

Replace a storage disk in an Oracle Database Appliance storage shelf.

Note:

After a physical insertion or removal action of a storage component (such as a disk or SAS cable), you should wait before performing the next action to allow the system software to fully process the event. This could take several minutes to complete.

  1. Locate the failed drive on the front of the storage shelf.

    Look for the drive with its Service Required indicator and/or Ready-to-Remove indicator lit.

    Caution:

    Equipment damage. Do not remove a drive unless you have an immediate replacement.

  2. Remove the drive by pressing the release button [1] to release the drive lever and then, rotate the lever open and, grasping the drive body, slide the drive out of the storage shelf [2] as shown in the following figure.
  3. Install the replacement drive by opening the drive lever on the replacement drive. Slide the new drive into the drive slot [1] until it engages with the backplane connector and then close the lever to fully seat the drive and lock it into place [2] as shown in the following figure.

    Check that the drive is locked in place and flush with the other drives.

    Description of g5704_de3-24c_drive_replacement.jpg follows
    Description of the illustration g5704_de3-24c_drive_replacement.jpg
  4. Once installed, the system should automatically detect and use the replacement drive. Ensure that the green OK indicator for the replacement drive lights to show that the drive is ready.

    This can take a few minutes to complete.

Parent topic: Oracle Database Appliance X9-2-HA Storage Shelf Hot Pluggable Components

Replace a Storage Shelf Power Supply (CRU)

Replace a power supply in an Oracle Database Appliance storage shelf.

The storage shelf and the storage expansion shelf each have two power supplies. Indicators (LEDs) on the power supplies indicate their operation state.

The power supplies are redundant and one can be replaced while the appliance is running.

Example figure showing a DE3-24C power supply:

Description of g5498-x6-2ha_storage_shelf_psu_callouts.jpg follows
Description of the illustration g5498-x6-2ha_storage_shelf_psu_callouts.jpg
Callout Description

1

AC power fail indicator

2

Fan fail indicator

3

Power supply status indicator

4

DC power fail indicator

5

Release lever

6

AC connector

7

On/Off switch

If any of the indicators are amber, or if the power supply status indicator does not light, replace the power supply.

  1. Ensure the power supply on/off switch is in the "O" off position.
  2. Disconnect the power cord tie strap from the power cord, and unplug the power cord from the power supply.
  3. Remove the power supply unit by grasping the power supply handle, push the release button [1], and slide the power supply out of the chassis [2] as shown in the following figure.
  4. Slide the replacement power supply module into the chassis.

    When fully installed, you should hear a click of the release lever returning to the locked position.

  5. Plug the power cord into the new power supply and attach the power cord tie strap to the power cord.
  6. Place the power supply on/off switch to the "I" on position.

    The Power/OK status LED should be a steady green, and all other indicators should be off.

Parent topic: Oracle Database Appliance X9-2-HA Storage Shelf Hot Pluggable Components

Replace a Storage Shelf I/O Module (CRU)

Replace an IO module in an Oracle Database Appliance storage shelf.

Storage Shelf SAS I/O modules are fully redundant and can be replaced while the appliance is running.

Note:

After a physical insertion or removal action of a storage component (such as a disk or SAS cable), you should wait before performing the next action to allow the system software to fully process the event. This could take up to 3 minutes to complete.

Example figure showing the DE3-24C:

Description of g5499-x6-2ha_storage_shelf_iom_callouts.jpg follows
Description of the illustration g5499-x6-2ha_storage_shelf_iom_callouts.jpg
Callout Description

1

I/O module locking levers

2

I/O module 1 (top)

3

I/O module 0 (bottom)
  1. Disconnect the cables from the I/O module. Be sure to mark the cables so you can return them to their proper connectors.
  2. Remove the I/O module using your thumb and forefinger, pinch the release buttons [1] together to release the locking lever [2] as shown in the following figure. Grasp the lever and remove the I/O module from the chassis [3] as shown in the following figure.
  3. Open the lever in the new I/O module if it is not already open.
  4. Slide the new I/O module into the disk shelf and close the locking lever.

    Caution:

    Equipment damage. Be careful not to damage the connector pins at the back of the I/O module.

  5. Reconnect the cables.
  6. Verify that the new I/O module is working correctly.

    • After approximately 60 seconds, the power LED should be solid green and the Fault/Locate LED should be off.

    • For each port that has a cable connected, all four activity LEDs should be solid green.

Parent topic: Oracle Database Appliance X9-2-HA Storage Shelf Hot Pluggable Components