3 Servicing CRUs That Do Not Require System Power Off

This chapter describes how to remove and replace the hot-pluggable components in Oracle Database Appliance. Hot-pluggable means that these components can be replaced with the system powered on.

Note:

Hot-pluggable components can also be replaced with the system powered off. If you choose this method, first power off the system as described in Chapter 2, Preparing Oracle Database Appliance for Service.

The following topics are covered:

Oracle Database Appliance X6-2 Series Server Node Hot-Pluggable Components

This section describes replacing hot-pluggable components for the Oracle Database Appliance X6-2 series server node. If you have the Oracle Database Appliance X3-2, Oracle Database Appliance X4-2, or Oracle Database Appliance X5-2, go to Oracle Database Appliance X3-2/X4-2/X5-2 Server Node Hot-Pluggable Components. If you have Oracle Database Appliance (original version), go to Oracle Database Appliance (Original Version) Hot-Pluggable Components.

Hot-pluggable components are those that can be removed and installed while the system and the server nodes are running. These are designated as Customer Replaceable Units (CRUs).

Refer to the following table for hot-plug component instructions for your appliance.

Note:

The links in the following table take you to instructions outside of the Oracle Database Appliance library to the Oracle Server X6-2 documentation library. Use your browser's Back button to return to the Oracle Database Appliance library.
Oracle Database Appliance X6-2S Oracle Database Appliance X6-2M
Servicing Non-NVMe Storage Drives (CRU)
Caution  -  NVMe disks are not hot-swappable. If you remove an NVMe drive without first powering it off, you can crash the operating system. Oracle Database Appliance uses special commands to add or remove NVMe storage, see the link below.
Servicing Non-NVMe Storage Drives (CRU)
Caution  -  NVMe disks are not hot-swappable. If you remove an NVMe drive without first powering it off, you can crash the operating system. Oracle Database Appliance uses special commands to add or remove NVMe storage, see the link below.
Servicing NVMe Storage Drives (CRU)
Servicing NVMe Storage Drives (CRU)
Servicing Fan Modules (CRU)
Servicing Fan Modules (CRU)
Servicing Power Supplies (CRU)
Servicing Power Supplies (CRU)

Servicing NVMe Storage Drives (CRU)

This section describes servicing NVMe drives in an Oracle Database Appliance X6-2 series system.

Replacing an NVMe Storage Drive

NVMe solid state drives are not hot-swappable. Before removing a drive it must be powered off. After installing a drive it must be powered on. Failed drives are automatically powered off after 5 minutes.

Caution:

If you remove an NVMe drive without first powering it off, you can crash the operating system. Oracle Database Appliance uses special commands to add or remove NVMe storage.

These instructions describe how to replace an NVMe drive using the odaamcli power on/off commands for an Oracle Database X6-2 series system.

  1. Identify the NVMe drive to be replaced. Type the following command at the host:

    # odaadmcli show disk

    For example, the drive names pd_00 and pd_01 seen by the host, correspond to drive slot labels NMe0 and NVMe1, respectively.

  2. Unmount and power off the drive to be replaced. Do one of the following:
    • If you are replacing a failed drive, check to see if its blue ready-to-remove light is lit.

      If a drive has failed, it should power off automatically and light its blue ready-to-remove indicator after 5 minutes.

    • If the drive being replaced doesn't already have its blue ready-to-remove indicator lit, type the following command from the host to unmount and turn off power to the drive:

      # odaadmcli power disk off pd_xx

      Where pd_xx is the name of the disk to be replaced as described in step 1.

      Before proceeding to the next step confirm that the blue ready-to-remove indicator on the drive is lit.

  3. Push the latch release button on the front of the drive to open the drive latch, grasp the latch, and pull the drive out of the drive slot.
  4. Align the replacement drive with the drive slot.

    The drive is physically addressed according to the slot in which it is installed. It is important to install a replacement drive in the same slot as the drive that was removed.

  5. Slide the drive into the slot until the drive is fully seated. Then close the drive latch to lock the drive in place.
  6. Power on and remount the replacement drive. Type the following command from the host:

    # odaadmcli power disk on pd_xx

    Where pd_xx is the name of the disk being replaced as described in step 1.

    It can take a few minutes for the power-on operation to complete. Once complete, the drive's green OK indicator will light.

  7. To check the status of the replacement disk, type the following command from the host:

    # odaadmcli show disk

    For additional information, refer to the Deployment and User's Guide.

Adding NVMe Expansion Storage

Before You Begin

The Oracle Database Appliance X6-2S/X6-2M includes two expansion slots for NVMe drives (slots labeled NVMe2 and NVMe3 as seen from the front of the system). When adding NVMe storage, you must populate both of these slots.

If you have the default configuration of only two NVMe drives (seen as pd_00 and pd_01 from the host), then you can expand storage by adding two (2) NVMe drives.

Note:

If you expand storage, then you must add two (2) disks and expand storage on both disks at the same time. Once you expand storage to four (4) NVMe drives, you cannot revert to the default configuration of two (2) NVMe drives.
  1. Insert the new NVMe drives into the expansion disk slots.

    The name for these drives to the host will be pd_02 and pd_03, matching the numbering of their respective drive slots.

  2. Slide the drive into the slot until the drive is fully seated. Then close the drive latch to lock the drive in place. Repeat for the second drive.
  3. Power on both drives by typing the following commands from the host:

    $ odaadmcli power disk on pd_02

    $ odaadmcli power disk on pd_03

    It can take a few minutes for the power-on operation to complete. Once complete, the green OK indicator will light for both drives.

    Note:

    Do not run attempt to power on and expand storage on one disk and then power on and expand storage on the second disk. Before you expand storage, both disks must be powered on with their green OK indicator lit.
  4. Expand the system storage to include the new drives. Type the following command from the host:

    $ odaadmcli expand storage

    For additional information, refer to the Oracle Database Appliance Deployment and User's Guide.

Oracle Database Appliance X3-2/X4-2/X5-2 Server Node Hot-Pluggable Components

This section describes replacing hot-pluggable components for the Oracle Database Appliance X3-2, Oracle Database Appliance X4-2, or Oracle Database Appliance X5-2. If you have Oracle Database Appliance X6-2 series, go to Oracle Database Appliance X6-2 Series Server Node Hot-Pluggable Components. If you have Oracle Database Appliance (original version), go to Oracle Database Appliance (Original Version) Hot-Pluggable Components.

Hot-pluggable components are those that can be removed and installed while the system and the server nodes are running. These are designated as Customer Replaceable Units (CRUs).

Note:

For Oracle Database Appliance X3-2 and Oracle Database Appliance X4-2 bare metal installations, you must run the oakcli add disk -local command after replacing a failed disk. Refer to the Administration and Reference Guide for details.

Refer to the following table for hot-plug component instructions for your appliance.

Note:

The links in the following table take you to instructions outside of the Oracle Database Appliance library to the Sun Server X3-2, Sun Server X4-2, and Oracle Server X5-2 documentation libraries. Use your browser's Back button to return to the Oracle Database Appliance library.
Oracle Database Appliance X3-2 Oracle Database Appliance X4-2 Oracle Database Appliance X5-2
Servicing Storage Drives (CRU)
Servicing Storage Drives (CRU)
Servicing Storage Drives (CRU)
Servicing Fan Modules (CRU)
Servicing Fan Modules (CRU)
Servicing Fan Modules (CRU)
Servicing Power Supplies (CRU)
Servicing Power Supplies (CRU)
Servicing Power Supplies (CRU)

Oracle Database Appliance Storage Shelf Hot-Pluggable Components

This section describes replacing hot-pluggable components for the Oracle Database Appliance storage shelf. Only certain versions of Oracle Database Appliance support a storage shelf. If you have Oracle Database Appliance (original version), go to Oracle Database Appliance (Original Version) Hot-Pluggable Components.

A storage shelf, or expansion storage shelf, are only supported with the following Oracle Database Appliance products:

  • Oracle Database Appliance X5-2 supports the DE2-24C shelf.

  • Oracle Database Appliance X3-2 and X4-2 support the DE2-24P shelf.

Hot-pluggable components are those that can be removed and installed while the system and the server nodes are running. These are designated as Customer Replaceable Units (CRUs).

Caution:

Data loss. Do not remove power from both power supplies of the storage shelf or storage expansion shelf (if equipped) while the database is running. If you must power off the storage shelf or expansion storage shelf, stop the database first.

Note:

After a physical insertion or removal action of a storage shelf component (such as a disk or SAS cable), you should wait before performing the next action to allow the system software to fully process the event. This could take up to 3 minutes to complete.

Replacement Procedures

Oracle Database Appliance X3-2/X4-2 (DE2-24P) and Oracle Database Appliance X5-2 (DE2-24C) storage shelf components can all be replaced while the storage shelf is powered-on and running. They include:

Replace a Storage Shelf HDD or SSD (CRU)

  1. Locate the failed drive on the front of the chassis.

    Caution:

    Equipment damage. Do not remove a drive unless you have an immediate replacement.
  2. Remove the drive:

    • For the DE2-24C, press the button to release the drive lever, rotate the lever open and, grasping the drive body, slide the drive out of the chassis.


      image:Picture showing drive being removed from the DE2-24C storage shelf chassis.

    • For the DE2-24P, press the button to release the drive lever, rotate the lever open and, grasping the drive body, slide the drive out of the chassis.


      image:Picture showing drive being removed from the DE2-24P storage shelf chassis.
  3. Install the replacement drive:

    • For the DE2-24C, open the drive lever on the replacement drive and slide the replacement drive as far into the drive slot as possible and then close the lever to lock it into place. Check that the drive is locked in place and flush with the other drives.


      image:Picture showing drive being inserted into the DE2-24C storage shelf chassis.

    • For the DE2-24P, open the drive lever on the replacement drive and slide the replacement drive as far into the drive slot as possible and then close the lever to lock it into place. Check that the drive is locked in place and flush with the other drives.


      image:Picture showing drive being inserted into the DE2-24P storage shelf chassis.

Replace a Storage Shelf Power Supply (CRU)

The storage shelf and the storage expansion shelf each have two power supplies. Indicators (LEDs) on the power supplies indicate their operation state.

The power supplies are redundant and one can be replaced while the appliance is running.

Example showing a DE2-24C or DE2-24P power supply:

image:Picture of DE2-24P and DE2-24C storage shelf power supply with callouts.
Callout Description

1

AC power fail indicator

2

Fan fail indicator

3

Power supply status indicator

4

DC power fail indicator

5

Release lever

6

AC connector

7

On/Off switch

If any of the indicators are amber, or if the power supply status indicator does not light, replace the power supply.

  1. Ensure the power supply on/off switch is in the "O" off position.
  2. Disconnect the power cord tie strap from the power cord, and unplug the power cord from the power supply.
  3. Remove the power supply unit:

    • For the DE2-24C, push the ejector release button [1], grasp the power supply ejector handle and rotate it to the open position [2]. Slide the power supply out of the chassis.


      image:Picture of storage shelf power supply with levers open.

    • For the DE2-24P power supply, push the ejector release button [1], grasp the power supply ejector handle and rotate it to the open position [2]. Slide the power supply out of the chassis.


      image:Picture of storage shelf power supply with levers open.
  4. Open the lever/ejection arms on the replacement power supply.
  5. Slide the replacement power supply module into the chassis.

    Be careful to not bend the pins.

  6. Close the lever/ejection arms.

    Push the lever fully closed until you hear or feel a click.

Replace a Storage Shelf I/O Module (CRU)

Storage Shelf SAS I/O modules are fully redundant and can be replaced while the appliance is running.

  1. Disconnect the cables from the I/O module.

    Be sure to mark the cables so you can return them to their proper connectors.

    Example showing the DE2-24C:


    image:Picture showing the DE2-24C IO modules on the Oracle Database Appliance storage shelf.

    Example showing the DE2-24P:


    image:Picture showing the DE2-24P IO modules on the Oracle Database Appliance storage shelf.
    Callout Description

    1

    I/O module locking levers

    2

    I/O module 1 (top)

    3

    I/O module 0 (bottom)
  2. Remove the I/O module:

    • For the DE2-24C, using your thumb and forefinger, squeeze the release button [1] toward the locking lever hole to release the lever [2]. Grasp the lever and remove the I/O module from the chassis.


      image:Picture of I/O release latch for the DE2-24P and DE2-24C.

    • For the DE2-24P, using your thumb and forefinger, squeeze the release button [1] toward the lever hole to release the locking lever [2]. Grasp the lever and remove the I/O module from the chassis.


      image:Picture of I/O release latch for the DE2-24P.

    Caution:

    Equipment damage. Be careful not to damage the connector pins at the back of the I/O module.
  3. Open the lever in the new I/O module if it is not already open.
  4. Slide the new I/O module into the disk shelf and close the locking lever.
  5. Reconnect the cables.
  6. Verify that the new I/O module is working correctly.

    • After approximately 60 seconds, the power LED should be solid green and the Fault/Locate LED should be off.

    • For each port that has a cable connected, all four activity LEDs should be solid green.

Oracle Database Appliance (Original Version) Hot-Pluggable Components

This section describes replacing hot-pluggable components for the Oracle Database Appliance (original version). If you have Oracle Database Appliance X6-2 series, go to Oracle Database Appliance X6-2 Series Server Node Hot-Pluggable Components. If you have Oracle Database Appliance X3-2, Oracle Database Appliance X4-2, or Oracle Database Appliance X5-2, go to Oracle Database Appliance X3-2/X4-2/X5-2 Server Node Hot-Pluggable Components or Oracle Database Appliance X3-2/X4-2/X5-2 Storage Shelf Hot-Pluggable Components.

Hot-pluggable components are those that can be removed and installed while the system and the server nodes are running. These are designated as Customer Replaceable Units (CRUs).

In Oracle Database Appliance (original version), the following devices are hot-pluggable:

Note:

After a physical insertion or removal action of a storage component (such as a disk or SAS cable), you should wait before performing the next action to allow the system software to fully process the event. This could take up to 3 minutes to complete.

Note:

For the Oracle Database Appliance (original version), you must run the oakcli add disk -local command after replacing a failed disk. Refer to the Administration and Reference Guide for details.

Servicing Storage Drives and Boot Drives (CRU)

Oracle Database Appliance (original version) has 24 storage drives, located in the front of the chassis, and each server node has 2 boot drives, located in the rear. All storage drives and boot drives are hot-pluggable.

See Storage and Boot Drive Status Indicators for specific information about drive status LEDs.

This section includes:

Storage and Boot Drive Locations

Appliance storage drives. Appliance storage drives are installed at the front of Oracle Database Appliance (original version). Drive numbering is listed on the drive label on the right side of the appliance [1].

image:Graphic showing the storage drive locations.

The following table shows slot numbering and the type of the storage drive supported in each slot.

Note:

The appliance might include both mechanical hard disk (HDD) and solid state (SSD) drives. Both use the same 2.5 inch disk drive bracket. An SSD has a smaller footprint inside of the bracket than an HDD.
Drive Slot Numbers and Types SupportedDrive Slot Numbers and Types SupportedDrive Slot Numbers and Types SupportedDrive Slot Numbers and Types Supported

SSD20

SSD21

SSD22

SSD23

HDD16

HDD17

HDD18

HDD19

HDD12

HDD13

HDD14

HDD15

HDD8

HDD9

HDD10

HDD11

HDD4

HDD5

HDD6

HDD7

HDD0

HDD1

HDD2

HDD3

Server Node boot drives. Each appliance server node contains two boot drives that are installed in the back of Oracle Database Appliance and identified on the rear drive panel label.

image:Graphic showing the boot drive locations.

The following table shows the location and numbering of the boot drives.

Callout Drive Locations and Identification

1

Boot Drives at the back of server node 1. Location and identification of the server node 1 boot drives as listed in Oracle ILOM:

  • /SYS/MB/RHDD1

  • /SYS/MB/RHDD0

2

Boot Drives at the back of server node 0. Location and identification of the server node 0 boot drives as listed in Oracle ILOM:

  • /SYS/MB/RHDD1

  • /SYS/MB/RHDD0

Remove a Storage Drive

  1. Identify the storage drive that you want to remove.

    If a drive has failed, its amber Service Required LED will be lit. For information on drive status LEDs, see Storage and Boot Drive Status Indicators. For specific drive locations, see Storage and Boot Drive Locations.

  2. Remove a storage drive.
    1. On the drive you plan to remove, push the storage drive release button to open the latch [1].
    2. Grasp the latch and open it to a 45 degree angle [2].
    3. Using the latch, slowly pull the drive straight out of the drive slot, not sideways or vertically.

      Do not use much force on the latch when removing the drive.

      Caution:

      Equipment damage. Do not use the latch as an ejector. Do not bend it too far to the right.

      Caution:

      Equipment damage. Always replace a drive with another drive to maintain proper airflow.

      image:Figure showing how to remove a front storage drive. The release button is on the left, and the latch opens left-to-right.

Install Storage Drive

  1. Determine the drive slot location for the replacement drive.

    Drives are physically addressed according to the slot in which they are installed. See Storage and Boot Drive Locations for drive locations.

  2. Slide the drive into the drive slot until it is fully seated.
    image:Figure showing how to install a front storage drive.
  3. Close the latch to lock the drive in place.
  4. Run the fwupdate command and verify that it lists all the disks under the DISKS section.

Remove a Boot Drive

  1. Identify the boot drive that you want to remove.

    The amber Service Required LED might be lit. For specific drive locations, see Storage and Boot Drive Locations.

  2. Remove the boot drive.
    1. To open the drive door, slide the latch to the left.
    2. Pull the door open [2].
    3. On the drive you plan to remove, push the storage drive release button to open the latch [2].
    4. Grasp the latch and open it to a 45 degree angle [2].
    5. Using the latch, slowly pull the drive straight out of the drive slot, not sideways or vertically [2].

      Do not use much force on the latch when removing the drive.

      Caution:

      Equipment damage. Do not use the latch as an ejector. Do not bend it too far to the right.

      Caution:

      Equipment damage. Close the door after removing a boot drive to prevent overheating due to improper airflow.

      image:Figure showing how to remove a rear boot drive.

Install a Boot Drive

  1. Determine the drive slot location for the replacement drive.

    Drives are physically addressed according to the slot in which they are installed. See Storage and Boot Drive Locations for drive locations.

  2. Pull open the rear drive door.
  3. Slide the drive into the drive slot until it is fully seated [1].
    image:Figure showing how to install a rear boot drive.
  4. Close the drive latch to lock the drive in place.
  5. Close the drive door and ensure that the door latch engages [2].

    Caution:

    Equipment damage. Close the door after removing a boot drive to prevent overheating due to improper airflow.
  6. Run the following command to ensure that the OS sees the replacement disk.
    # lsscsi -v | grep ATA
[0:0:0:0]    disk    ATA      SEAGATE ST95001N n/a   /dev/sda
[1:0:0:0]    disk    ATA      SEAGATE ST95001N n/a   /dev/sdb

Servicing Fan Modules (CRU)

Each Oracle Database Appliance (original version) server node contains two fan modules. Each fan module contains a single, dual-motor, 80 mm, counter-rotating fan assembly that contains two fans. Each fan module is hot-pluggable. For specific information about fan status LEDs, see Server Node Fan Indicators.

When a server node is removed from the system, the fans provide sufficient cooling for all 24 front disks and the server node. A door automatically closes off the vacant server node slot to ensure proper cooling. When the server node is replaced, its two fans also cool the 24 front disks and provide cooling to the second server node.

Caution:

Equipment damage. If a fan module fails, replace it as soon as possible.

The following topics are covered in this section:

Detecting a Fan Module Failure

The following LEDs are lit when a fan module fault is detected:

  • Front and rear Service Required LEDs located on each server node

  • REAR HDD/PS/FAN LED located on the front of each server node

  • Fan Fault Status LED located on the faulty fan module

The system over-temperature LED for a server node might light if a fan fault causes an increase in server node operating temperature. See Server Node Status Indicators for more information about server node LEDs.

Remove a Fan Module

Caution:

Equipment damage. When removing a fan module from a powered on system, it must be replaced within two minutes of removal.
  1. Identify the faulty fan module with its amber LED.

    When a fan fails, the Service Required LED lights amber. For information about fan status LEDs, see Server Node Fan Indicators.

  2. To remove the faulty fan module, press down on the fan release lever [1].
  3. While holding the release lever down, use the handle on the fan module to pull it out of the server node [2].

    The fan door swings shut to ensure proper cooling.


    image:Picture of fan being removed from the chassis.

Install a Fan Module

  1. Align the fan module in the fan door and slide it into the server node.

    The extender on the side of the fan module pushes the fan door open, allowing the fan module to enter.

  2. Apply firm pressure to fully seat the fan module.

    The fan release lever engages when the fan is fully seated.


    image:Figure showing how to install a fan module
  3. Verify that the Fan Fault Status LED on the replacement fan module is lit green.
  4. Verify that the following fault LEDs on the applicable server node are off.
    • Front and rear Service Required LEDs located on the applicable server node
    • REAR HDD/PS/FAN LED located on the front of the applicable server node

    See Server Node Status Indicators for more information about system LEDs.

Servicing Power Supplies (CRU)

Oracle Database Appliance (original version) is equipped with two hot pluggable power supplies. They are redundant, meaning that the system can operate with a single power supply. This allows you to remove and replace a power supply without shutting the system down.

If a power supply fails and you do not have a replacement available, to ensure proper airflow, leave the failed power supply installed in the system.

For information about power supply status LEDs, see Power Supply Status Indicators.

This section includes:

Detecting a Power Supply Failure

The following LEDs light up when the system detects a power supply fault:

  • Front and rear Service Required LEDs located on each server node

  • REAR HDD/PS/FAN LED located on the front of each server node

  • Service Required LED located on the faulty power supply

See Server Node Status Indicators on page 1-6 for information about these LEDs.

Remove a Power Supply

Caution:

Hazardous voltages. To reduce the risk of electric shock and danger to personal health, follow the instructions.
  1. Identify which power supply (0 or 1) requires replacement.

    A lit Service Required LED on a power supply indicates that a failure was detected. For information about power supply status LEDs, see Power Supply Status Indicators.

  2. Disconnect the power cord from the faulty power supply.
  3. Grasp the power supply handle and press the release latch [1]).
  4. Pull the power supply out of the chassis [2].
    image:Figure showing how to remove a power supply.

Install a Power Supply

  1. Align the replacement power supply with the empty power supply chassis bay.
  2. Slide the power supply into the bay until it is fully seated.

    The release latch engages when the power supply is fully seated.


    image:Figure showing how to install a power supply.
  3. Reconnect the power cord to the power supply and verify the following:

    • The power supply AC Present LED is lit

    • The power supply Service Required LED is off.

  4. Verify that the following LEDs are off:

    • Service Required LEDs located on the front of both server nodes

    • REAR HDD/PS/FAN LED located on the front of both server nodes

    See Storage and Boot Drive Locations for more information about system LEDs.