Previous Next JavaScript must be enabled to correctly display this content

## 11 Managing Storage

Understand the storage options and how to manage storage for your Oracle Database Appliance deployment.

- About Managing Storage
  Understand Oracle Database Appliance storage options.
- About Managing Oracle ASM Disks
  Understand the Oracle ASM disk management features that Oracle Database Appliance supports.
- Managing Storage on Single-Node Systems
  Understand the storage options for your Oracle Database Appliance X11-S and X11-L systems.
- Managing Storage on High-Availability Systems
  Understand the storage for your Oracle Database Appliance X11-HA system.

### About Managing Storage

Understand Oracle Database Appliance storage options.

Oracle Database Appliance uses raw storage to protect data in the following ways:

- Fast Recovery Area (FRA) backup. FRA is a storage area (directory on disk or Oracle ASM diskgroup) that contains redo logs, control file, archived logs, backup pieces and copies, and flashback logs.

- Mirroring. Double or triple mirroring provides protection against mechanical issues.

The amount of available storage is determined by the location of the FRA backup (external or internal) and if double or triple mirroring is used. External NFS storage is supported for online backups, data staging, or additional database files.

Oracle Database Appliance X11-L and X11-HA models provide storage expansion options from the base configuration. In addition, on Oracle Database Appliance X11-HA multi-node platforms, you can add an optional storage expansion shelf.

The redundancy level for FLASH is based on the DATA and RECO selection. If you choose High redundancy (triple mirroring), then FLASH is also High redundancy.

Parent topic: Managing Storage

### About Managing Oracle ASM Disks

Understand the Oracle ASM disk management features that Oracle Database Appliance supports.

Oracle Database Appliance enables you to manage your Oracle ASM disks.

Bringing Oracle ASM Disk Groups Online Automatically

Oracle Database Appliance periodically checks the status of Oracle ASM disks in disk groups. If any Oracle ASM disk is OFFLINE due to transient disk errors, then Oracle Database Appliance attempts to bring the disk ONLINE.

Optimizing Oracle ASM Disk Group Rebalance Operations

Oracle Database Appliance ensures rebalancing of Oracle ASM disks to complete as quickly as possible, without overloading the system and the disks at the same time. This ensures that the system is in a steady state running with the appropriate redundancy. There are default thresholds for the values for rebalancing operations, and you can also set the custom threshold values. For example:

    odacli modify-agentconfig-parameters -n ASMRM_CPU_RQ -v 50 -d "CPU RUN QUEUE THRESHOLD" -u
    odacli modify-agentconfig-parameters -n ASMRM_MAX_HDD_DISK_RQ -v 2 -d "HDD DISK QUEUE THRESHOLD" -u
    odacli modify-agentconfig-parameters -n ASMRM_MAX_SSD_DISK_RQ -v 32 -d "SSD DISK QUEUE THRESHOLD" -u
    odacli modify-agentconfig-parameters -n ASMRM_MAX_NVME_DISK_RQ -v 50 -d "NVME DISK QUEUE THRESHOLD" -u

The above command options set custom threshold limits for rebalance monitoring of Oracle ASM disks.

You can monitor rebalance operations using the `odacli describe-schedule -i Schedule ID` and `odacli list-scheduled-executions` commands.

Parent topic: Managing Storage

### Managing Storage on Single-Node Systems

Understand the storage options for your Oracle Database Appliance X11-S and X11-L systems.

- About Storage on Oracle Database Appliance X11-S and X11-L
  Understand the storage for your Oracle Database Appliance single-node system.
- Adding Small Form Factor (SFF) NVMe Storage Disks
  Depending on the available drives, you can expand Oracle Database Appliance X11-L storage to add Small Form Factor (SFF) NVMe disks or replace existing NVMe disks.
- Adding Add-in-Card (AIC) NVMe Storage Disks
  You can expand Oracle Database Appliance X11-L storage with two or four Add-in-Card (AIC) NVMe disks. Oracle Database Appliance X11-L supports a maximum of four AICs.
- Replacing Small Form Factor (SFF) NVMe Storage Disks
  Understand how you can replace existing SFF NVMe disks on Oracle Database Appliance.

Parent topic: Managing Storage

#### About Storage on Oracle Database Appliance X11-S and X11-L

Understand the storage for your Oracle Database Appliance single-node system.

Oracle Database Appliance X11-S has two 6.8TB NVMe disks that host DATA and RECO disk groups. There are ten partitions that you can divide between DATA and RECO for Oracle ASM storage information. By default, DATA has an eight partition configuration and RECO has a two partition configuration. The storage capacity is fixed and cannot be expanded.

Oracle Database Appliance X11-L has two form factor-based NVMe disks namely Small Form Factor (SFF) and AIC (Add-in-Card). Both form factor NVMe disks are of 6.8TB storage capacity. SFF is a single disk of 6.8TB capacity whereas AIC has two NVMe disks each of 3.4TB capacity, with a combined storage capacity of 6.8TB. The default configuration for Oracle Database Appliance X11-L is two 6.8TB NVMe disks that host DATA and RECO disk groups.

When you first deploy and configure X11-L in this release, you can set the storage on X11-L in multiple of 2 packs of NVMe and AIC drives, such as 2, 4, and 6 disks, up to a maximum of 8 disks.

Oracle Database Appliance X11-L supports four SFF NVMe disks and four AIC NVMe disks. You must populate all four SFF NVMe disks before you add AIC NVMe disks to the system.

The table describes the NVMe storage configurations with expansion memory and storage options for single-node systems.

Table 11-1 Storage Options for Oracle Database Appliance X11-S and X11-L

Configuration | Oracle Database Appliance X11-S | Oracle Database Appliance X11-L
--- | --- | ---
Base Configuration | 2 x 6.8 TB NVMe = 13.6 TB NVMe | 2 x 6.8 TB NVMe = 13.6 TB NVMe
Storage addition options | None | 6x6.8TB NVMe storage drives for total storage of 54.4TB NVMe. You must populate all four SFF NVMe disks before you add AIC NVMe disks to the system. For the additional two SFF NVMe, order the following: Qty:1 (Two 6.8TB 2.5-inch NVMe PCIe SFF SSD with marlin bracket for Oracle Database Appliance X11-L) For the additional four NVMe AIC SSDs (these are PCIe NVMe Flash Cards that require cover removal to install), order the following: Qty:1 for two, Qty:2 for four: Qty:1 - (Two 6.8 TB NVMe PCIe Cards for Oracle Database Appliance X11-L)

Parent topic: Managing Storage on Single-Node Systems

#### Adding Small Form Factor (SFF) NVMe Storage Disks

Depending on the available drives, you can expand Oracle Database Appliance X11-L storage to add Small Form Factor (SFF) NVMe disks or replace existing NVMe disks.

Use the ODAADMCLI commands to perform appliance storage maintenance tasks, including perform storage diagnostics and collect diagnostic logs for storage components.

Preparing for a Storage Upgrade

Review and perform these best practices before adding storage.

1. Check the disk health of the existing storage disks.

       # odaadmcli show disk

2. Run the the `odaadmcli show disk` and `asmcmd lsdsk -p` commands to view and review the storage disk information in OAKD and Oracle Automatic Storage Management (Oracle ASM).

       # odaadmcli show disk

       # asmcmd lsdsk -p

3. Use ORAchk to confirm Oracle ASM and Oracle Clusterware health.

Adding Small Form Factor (SFF) NVMe Storage Disks

The default configuration for Oracle Database Appliance X11-S or X11-L includes two NVMe disks. You cannot expand storage for Oracle Database Appliance X11-S.

For Oracle Database Appliance X11-L, you can expand storage by adding two SFF NVMe disks followed by two or four Add-in-Cards (AIC).

Important:

You must populate all four SFF slots before adding AIC.

WARNING:

Pulling a drive before powering it off will crash the kernel, which can lead to data corruption. Do not pull the drive when the LED is an amber or green color. When you need to replace an NVMe drive, use the software to power off the drive before pulling the drive from the slot. If you have more than one disk to replace, complete the replacement of one disk before starting replacement of the next disk.

Follow all these steps to add SFF NVMe disks:

1. Before adding the NVMe disks, ensure that the current disks are online in `oakd` and Oracle ASM. Otherwise, the prechecks fail. For example, for 2-disks expansion to slots 2 and 3, the disks in slots 0 and 1 must be online in Oracle ASM and `oakd`.
2. Insert each disk one at a time in the appropriate slot and wait for the disk to power ON.
3. The disk automatically powers on when you insert the disk in the slot. Wait for one minute and then check disk status. If the disk is in the ON state, then you need not power on the disk manually. If the disk state is OFF as per the disk status, and then power on the disk manually, and then check the status again.

       # odaadmcli power disk status slot_number

       # odaadmcli power disk on slot_number

   For example, to add two (2) NVMe disks, insert the disks in slots 2 and 3.

       # odaadmcli power disk status pd_02
       # odaadmcli power disk on pd_02

       # odaadmcli power disk status pd_03
       # odaadmcli power disk on pd_03

4. Repeat steps 2 and 3 for each disk to be added.
5. Run the `odaadmcli expand storage` command to add the new storage disks. Note: You must run this step to add the storage disk. Otherwise, the newly-added disk is not visible to OAKD and hence does not display when you run the `odaadmcli show disk` or `odaadmcli show storage` commands. The newly-added disk is recognized by OAKD after the `odaadmcli expand storage` command completes running successfully.

       # odaadmcli expand storage -ndisk number_of_disks

   For example, to add two (2) NVMe drives:

       #odaadmcli expand storage -ndisk 2
       Running precheck, it may take a few minutes.
       Precheck passed.
       Initializing NVMe disks. It might take a few minutes.
       Initialization of NVMe disks completed successfully.

       Check the progress of expansion of storage by executing 'odaadmcli show disk'
       Waiting for expansion to finish. It may take several minutes to complete
       depending upon the number of disks being expanded

       odaadmcli expand storage command completed.

       Check status of newly expanded disk in OAK by executing 'odaadmcli show disk'
       All of them must be ONLINE status. Check status of newly expanded disk in ASM
       by executing 'asmcmd lsdsk -p', mount status must be CACHED and header status
       should be MEMBER for all the newly added disks. If the status of newly added
       disks in OAK and ASM are different then expand storage command failed.

       Expand Storage Operation Status :
         OAKD : Success
          ASM : Success

       If the command status reports a failure, manually verify the status by following
       the steps outlined above.

6. Check the status of the new disk in OAKD with the `odaadmcli show disk` command. The disk must have the status `Online` and `Good` in OAKD.

       # odaadmcli show disk
        NAME PATH TYPE STATE STATE_DETAILS

        pd_00 /dev/nvme0n1 NVD ONLINE Good
        pd_01 /dev/nvme1n1 NVD ONLINE Good
        pd_02 /dev/nvme3n1 NVD ONLINE Good
        pd_03 /dev/nvme2n1 NVD ONLINE Good
       #

7. Verify that the disks in slots 2 and 3 are added to Oracle Automatic Storage Management (Oracle ASM) as follows. The new disk in Oracle ASM must be in `CACHED MEMBER ONLINE NORMAL` state.
   1. Run `asm_script` to verify that the disks in slots 2 and 3 are added to Oracle ASM. Verify that both disks are successfully added (CACHED and MEMBER). Following is example of default configuration of 80:20 where eight partitions (p1 to p8) are part of the DATA disk group and two partitions (p9 and p10) are part of the RECO disk group.

          # su gridUser /opt/oracle/oak/bin/stordiag/asm_script.sh 0 6
          # su grid /opt/oracle/oak/bin/stordiag/asm_script.sh 0 6

          SQL*Plus: Release 19.0.0.0.0 - Production on Mon Jul 28 18:20:11 2025
          Version 19.28.0.0.0

          Copyright (c) 1982, 2025, Oracle.  All rights reserved.

          Connected to:
          Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
          Version 19.28.0.0.0

          SQL> SQL> SQL> SQL> SQL>
          PATH                                     NAME                                GROUP_NUMBER STATE    MODE_ST MOUNT_S HEADER_STATU
          ---------------------------------------- ----------------------------------- ------------ -------- ------- ------- ------------
          /dev/NVD_S01_PHCP43120001p9              NVD_S01_PHCP43120001P9                         2 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S02_PHCP424400MVp1              NVD_S02_PHCP424400MVP1                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S02_PHCP424400MVp10             NVD_S02_PHCP424400MVP10                        2 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S02_PHCP424400MVp2              NVD_S02_PHCP424400MVP2                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S02_PHCP424400MVp3              NVD_S02_PHCP424400MVP3                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S02_PHCP424400MVp4              NVD_S02_PHCP424400MVP4                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S02_PHCP424400MVp5              NVD_S02_PHCP424400MVP5                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S02_PHCP424400MVp6              NVD_S02_PHCP424400MVP6                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S02_PHCP424400MVp7              NVD_S02_PHCP424400MVP7                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S02_PHCP424400MVp8              NVD_S02_PHCP424400MVP8                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S02_PHCP424400MVp9              NVD_S02_PHCP424400MVP9                         2 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S03_PHCP4246004Yp1              NVD_S03_PHCP4246004YP1                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S03_PHCP4246004Yp10             NVD_S03_PHCP4246004YP10                        2 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S03_PHCP4246004Yp2              NVD_S03_PHCP4246004YP2                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S03_PHCP4246004Yp3              NVD_S03_PHCP4246004YP3                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S03_PHCP4246004Yp4              NVD_S03_PHCP4246004YP4                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S03_PHCP4246004Yp5              NVD_S03_PHCP4246004YP5                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S03_PHCP4246004Yp6              NVD_S03_PHCP4246004YP6                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S03_PHCP4246004Yp7              NVD_S03_PHCP4246004YP7                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S03_PHCP4246004Yp8              NVD_S03_PHCP4246004YP8                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S03_PHCP4246004Yp9              NVD_S03_PHCP4246004YP9                         2 NORMAL   ONLINE  CACHED  MEMBER

          SQL> Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
          Version 19.28.0.0.0
          #

   2. Use the `odaadmcli show validation storage errors` command to view hard storage errors. Hard errors include having the wrong type of disk inserted into a particular slot, an invalid disk model, or an incorrect disk size.

          # odaadmcli show validation storage errors

   3. Use the `odaadmcli show validation storage failures` command to view soft validation errors. A typical soft disk error would be an invalid version of the disk firmware.

          # odaadmcli show validation storage failures

   4. Confirm that the `oak_storage_conf.xml` file shows the number of disks added. For example, if you added 2 disks to the base configuration, then the `oak_storage_conf.xml` file must show `numberOfDisks` as 4.

          #cat /opt/oracle/oak/conf/oak_storage_conf.xml
          <!--
          This file is created by the Oracle Database Appliance software as part of system provisioning based on system provisioning requests.
          Values of element nodes can bechanged by OAK in response to storage configuration change operation.
          DO NOT EDIT THIS FILE.
          -->
          <CometConfiguration>
            <OakStorageConfigInfo type="string" dimension="vector" readonly="true" required="true" default="">
              <!--  Number of disks part of OAK -->
              <numberOfDisks>4</numberOfDisks>
              <!--   Number of partitions per disk part of DATA diskgroup in multiple partition scheme   -->
              <!--   Number of partitions per disk part of RECO diskgroup in multiple partition scheme   -->
              <!--   are derived from number of partitions per disk which are part of ASM DATA diskgroup -->
              <numOfDataDiskPartitionInAsm>8</numOfDataDiskPartitionInAsm>
            </OakStorageConfigInfo>
          </CometConfiguration>
          #

Parent topic: Managing Storage on Single-Node Systems

#### Adding Add-in-Card (AIC) NVMe Storage Disks

You can expand Oracle Database Appliance X11-L storage with two or four Add-in-Card (AIC) NVMe disks. Oracle Database Appliance X11-L supports a maximum of four AICs.

Use the ODAADMCLI commands to perform appliance storage maintenance tasks, including perform storage diagnostics and collect diagnostic logs for storage components.

Preparing for a Storage Upgrade

Review and perform these best practices before adding storage.

1. Update Oracle Database Appliance to the latest Patch Bundle before expanding storage.

       # odacli describe-component

2. Check the disk health of the existing storage disks.

       # odaadmcli show disk

3. Run the `odaadmcli show diskgroup` command to display and review Oracle Automatic Storage Management (Oracle ASM) disk group information.

4. Use the `asmcmd` command to verify that all the disks for the four SFF slots are part of Oracle ASM.

5. Use Oracle ORAchk to confirm Oracle ASM and Oracle Clusterware health.

Adding Add-in-Cards (AIC) NVMe Storage Disks

For Oracle Database Appliance X11-L, you can expand storage by adding two SFF NVMe disks followed by two or four Add-in-Cards (AIC). You can expand storage by first adding 2 AIC NVMe disks or 4 AIC, up to a maximum of 4 AIC disks. When you expand storage, adding odd numbers of AIC drives is not supported.

Important:

You must populate all four SFF slots before adding AIC.

WARNING:

Pulling a drive before powering it off will crash the kernel, which can lead to data corruption. Do not pull the drive when the LED is an amber or green color. When you need to replace an NVMe drive, use the software to power off the drive before pulling the drive from the slot. If you have more than one disk to replace, complete the replacement of one disk before starting replacement of the next disk.

See Also:

Chapter Installing Oracle Database Appliance Into a Rack in the Oracle Database Appliance X11 Owner's Guide in the Oracle Database Appliance Documentation Library for this release for requirements before adding any optional PCIe add-in card storage
Note that you must install the AIC NVMe disks in the correct slots as described in the Oracle Database Appliance X11 Owner's Guide and complete all configuration steps to add the AIC NVMe disks. Following are PCI slot for AIC:

- x16-PCIe slot 2:NVMe AIC (first)
- x8-PCIe slot 3: NVMe AIC (second)
- x8-PCIe slot 9: NVMe AIC (third)
- x16-PCIe slot 1: NVMe AIC (fourth)

Follow these steps to add AIC NVMe disks:

1. On successful installation of AIC, the system restarts. Check that Oracle Clusterware is up and running.

       crsctl check crs
       CRS-4638: Oracle High Availability Services is online
       CRS-4537: Cluster Ready Services is online
       CRS-4529: Cluster Synchronization Services is online
       CRS-4533: Event Manager is online

2. Verify that the `oakd` process is running.

       # odaadmcli show disk

   If the `oakd` process is not running, then start it:

       # odaadmcli start oak

3. Run the `odaadmcli show disk` command to check that `oakd` has discovered all AIC NVMe disks. For 2 AIC disks, there are four (4) NVMe disks of 3.4TB, and for four AIC disks, there are eight (8) 3.4TB NVME disks.
   For two AIC:

       # odaadmcli show storage
       ==== BEGIN STORAGE DUMP ========
       Host Description: Oracle Corporation:ORACLE SERVER E6-2L
       Total number of controllers: 8
               Id          = 0
               Pci Slot    = 100
               Serial Num  = PHCP431200027P6CGN
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PH27X076TOC
               FwVers      = G70YR112
               strId       = nvme:71:00.00
               Pci Address = 71:00.0

               Id          = 1
               Pci Slot    = 101
               Serial Num  = PHCP431200017P6CGN
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PH27X076TOC
               FwVers      = G70YR112
               strId       = nvme:72:00.00
               Pci Address = 72:00.0

               Id          = 3
               Pci Slot    = 103
               Serial Num  = PHCP4246004Y7P6CGN
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PH27X076TOC
               FwVers      = G70YR112
               strId       = nvme:73:00.00
               Pci Address = 73:00.0

               Id          = 2
               Pci Slot    = 102
               Serial Num  = PHCP424400MV7P6CGN
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PH27X076TOC
               FwVers      = G70YR112
               strId       = nvme:74:00.00
               Pci Address = 74:00.0

               Id          = 4
               Pci Slot    = 2
               Serial Num  = PHCR4286002Q6P8AGN-1
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PHC7E068TON
               FwVers      = G79YR112
               strId       = nvme:e1:00.00
               Pci Address = e1:00.0

               Id          = 5
               Pci Slot    = 22
               Serial Num  = PHCR4286002Q6P8AGN-2
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PHC7E068TON
               FwVers      = G79YR112
               strId       = nvme:e2:00.00
               Pci Address = e2:00.0

               Id          = 7
               Pci Slot    = 3
               Serial Num  = PHCR428000396P8AGN-1
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PHC7E068TON
               FwVers      = G79YR112
               strId       = nvme:f1:00.00
               Pci Address = f1:00.0

               Id          = 6
               Pci Slot    = 23
               Serial Num  = PHCR428000396P8AGN-2
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PHC7E068TON
               FwVers      = G79YR112
               strId       = nvme:f2:00.00
               Pci Address = f2:00.0

       Total number of expanders: 0
       Total number of PDs: 8
               /dev/nvme0n1    Solidigm          NVD  6801gb slot:  0  pci-addr : 71  SOLIDIGM SB5PH27X076TOC          SFF
               /dev/nvme1n1    Solidigm          NVD  6801gb slot:  1  pci-addr : 72  SOLIDIGM SB5PH27X076TOC          SFF
               /dev/nvme2n1    Solidigm          NVD  6801gb slot:  2  pci-addr : 74  SOLIDIGM SB5PH27X076TOC          SFF
               /dev/nvme3n1    Solidigm          NVD  6801gb slot:  3  pci-addr : 73  SOLIDIGM SB5PH27X076TOC          SFF
               /dev/nvme6n1    Solidigm          NVD  3400gb slot:  4  pci-addr : e1  SOLIDIGM SB5PHC7E068TON          AIC
               /dev/nvme7n1    Solidigm          NVD  3400gb slot:  5  pci-addr : e2  SOLIDIGM SB5PHC7E068TON          AIC
               /dev/nvme9n1    Solidigm          NVD  3400gb slot:  6  pci-addr : f1  SOLIDIGM SB5PHC7E068TON          AIC
               /dev/nvme8n1    Solidigm          NVD  3400gb slot:  7  pci-addr : f2  SOLIDIGM SB5PHC7E068TON          AIC
       ==== END STORAGE DUMP =========
       #

       # odaadmcli show disk
               NAME            PATH            TYPE            STATE           STATE_DETAILS

               pd_00           /dev/nvme0n1    NVD             ONLINE          Good
               pd_01           /dev/nvme1n1    NVD             ONLINE          Good
               pd_02           /dev/nvme2n1    NVD             ONLINE          Good
               pd_03           /dev/nvme3n1    NVD             ONLINE          Good
               pd_04_c1        /dev/nvme6n1    NVD             UNKNOWN         NewDiskInserted
               pd_04_c2        /dev/nvme7n1    NVD             UNKNOWN         NewDiskInserted
               pd_05_c1        /dev/nvme9n1    NVD             UNKNOWN         NewDiskInserted
               pd_05_c2        /dev/nvme8n1    NVD             UNKNOWN         NewDiskInserted
       [root@oak ~]#

   For four AIC:

       # odaadmcli show storage
       ==== BEGIN STORAGE DUMP ========
       Host Description: Oracle Corporation:ORACLE SERVER E6-2L
       Total number of controllers: 12
               Id          = 8
               Pci Slot    = 9
               Serial Num  = PHCR4286003Z6P8AGN-1
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PHC7E068TON
               FwVers      = G79YR112
               strId       = nvme:61:00.00
               Pci Address = 61:00.0

               Id          = 9
               Pci Slot    = 29
               Serial Num  = PHCR4286003Z6P8AGN-2
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PHC7E068TON
               FwVers      = G79YR112
               strId       = nvme:62:00.00
               Pci Address = 62:00.0

               Id          = 0
               Pci Slot    = 100
               Serial Num  = PHCP431200027P6CGN
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PH27X076TOC
               FwVers      = G70YR112
               strId       = nvme:71:00.00
               Pci Address = 71:00.0

               Id          = 5
               Pci Slot    = 101
               Serial Num  = PHCP431200017P6CGN
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PH27X076TOC
               FwVers      = G70YR112
               strId       = nvme:72:00.00
               Pci Address = 72:00.0

               Id          = 6
               Pci Slot    = 103
               Serial Num  = PHCP4246004Y7P6CGN
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PH27X076TOC
               FwVers      = G70YR112
               strId       = nvme:73:00.00
               Pci Address = 73:00.0

               Id          = 7
               Pci Slot    = 102
               Serial Num  = PHCP424400MV7P6CGN
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PH27X076TOC
               FwVers      = G70YR112
               strId       = nvme:74:00.00
               Pci Address = 74:00.0

               Id          = 3
               Pci Slot    = 1
               Serial Num  = PHCR4300000N6P8AGN-1
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PHC7E068TON
               FwVers      = G79YR112
               strId       = nvme:91:00.00
               Pci Address = 91:00.0

               Id          = 4
               Pci Slot    = 21
               Serial Num  = PHCR4300000N6P8AGN-2
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PHC7E068TON
               FwVers      = G79YR112
               strId       = nvme:92:00.00
               Pci Address = 92:00.0

               Id          = 1
               Pci Slot    = 2
               Serial Num  = PHCR4286002Q6P8AGN-1
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PHC7E068TON
               FwVers      = G79YR112
               strId       = nvme:e1:00.00
               Pci Address = e1:00.0

               Id          = 2
               Pci Slot    = 22
               Serial Num  = PHCR4286002Q6P8AGN-2
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PHC7E068TON
               FwVers      = G79YR112
               strId       = nvme:e2:00.00
               Pci Address = e2:00.0

               Id          = 10
               Pci Slot    = 3
               Serial Num  = PHCR428000396P8AGN-1
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PHC7E068TON
               FwVers      = G79YR112
               strId       = nvme:f1:00.00
               Pci Address = f1:00.0

               Id          = 11
               Pci Slot    = 23
               Serial Num  = PHCR428000396P8AGN-2
               Vendor      = Solidigm
               Model       = SOLIDIGM SB5PHC7E068TON
               FwVers      = G79YR112
               strId       = nvme:f2:00.00
               Pci Address = f2:00.0

       Total number of expanders: 0
       Total number of PDs: 12
               /dev/nvme0n1    Solidigm          NVD  6801gb slot:  0  pci-addr : 71  SOLIDIGM SB5PH27X076TOC          SFF
               /dev/nvme1n1    Solidigm          NVD  6801gb slot:  1  pci-addr : 72  SOLIDIGM SB5PH27X076TOC          SFF
               /dev/nvme3n1    Solidigm          NVD  6801gb slot:  2  pci-addr : 74  SOLIDIGM SB5PH27X076TOC          SFF
               /dev/nvme2n1    Solidigm          NVD  6801gb slot:  3  pci-addr : 73  SOLIDIGM SB5PH27X076TOC          SFF
               /dev/nvme10n1   Solidigm          NVD  3400gb slot:  4  pci-addr : e1  SOLIDIGM SB5PHC7E068TON          AIC
               /dev/nvme11n1   Solidigm          NVD  3400gb slot:  5  pci-addr : e2  SOLIDIGM SB5PHC7E068TON          AIC
               /dev/nvme8n1    Solidigm          NVD  3400gb slot:  6  pci-addr : f1  SOLIDIGM SB5PHC7E068TON          AIC
               /dev/nvme9n1    Solidigm          NVD  3400gb slot:  7  pci-addr : f2  SOLIDIGM SB5PHC7E068TON          AIC
               /dev/nvme6n1    Solidigm          NVD  3400gb slot:  8  pci-addr : 61  SOLIDIGM SB5PHC7E068TON          AIC
               /dev/nvme7n1    Solidigm          NVD  3400gb slot:  9  pci-addr : 62  SOLIDIGM SB5PHC7E068TON          AIC
               /dev/nvme12n1   Solidigm          NVD  3400gb slot: 10  pci-addr : 91  SOLIDIGM SB5PHC7E068TON          AIC
               /dev/nvme13n1   Solidigm          NVD  3400gb slot: 11  pci-addr : 92  SOLIDIGM SB5PHC7E068TON          AIC
       ==== END STORAGE DUMP =========
       #

       # odaadmcli show disk
               NAME            PATH            TYPE            STATE           STATE_DETAILS

               pd_00           /dev/nvme0n1    NVD             ONLINE          Good
               pd_01           /dev/nvme1n1    NVD             ONLINE          Good
               pd_02           /dev/nvme3n1    NVD             ONLINE          Good
               pd_03           /dev/nvme2n1    NVD             ONLINE          Good
               pd_04_c1        /dev/nvme7n1    NVD             UNKNOWN         NewDiskInserted
               pd_04_c2        /dev/nvme6n1    NVD             UNKNOWN         NewDiskInserted
               pd_05_c1        /dev/nvme9n1    NVD             UNKNOWN         NewDiskInserted
               pd_05_c2        /dev/nvme8n1    NVD             UNKNOWN         NewDiskInserted
               pd_06_c1        /dev/nvme10n1   NVD             UNKNOWN         NewDiskInserted
               pd_06_c2        /dev/nvme11n1   NVD             UNKNOWN         NewDiskInserted
               pd_07_c1        /dev/nvme13n1   NVD             UNKNOWN         NewDiskInserted
               pd_07_c2        /dev/nvme12n1   NVD             UNKNOWN         NewDiskInserted
       [root@oak ~]#

   For example, to add two (2) AIC disks, you must specify `ndisk` values as 4 because AIC has two NVMe disks, so the total value of two AIC disks is four NVMe disks.

       #odaadmcli expand storage -ndisk 4
       Running precheck, it may take a few minutes.
       Precheck passed.

       Check the progress of expansion of storage by executing 'odaadmcli show disk'
       Waiting for expansion to finish. It may take several minutes to complete
       depending upon the number of disks being expanded

       odaadmcli expand storage command completed.

       Check status of newly expanded disk in OAK by executing 'odaadmcli show disk'
       All of them must be ONLINE status. Check status of newly expanded disk in ASM
       by executing 'asmcmd lsdsk -p', mount status must be CACHED and header status
       should be MEMBER for all the newly added disks. If the status of newly added
       disks in OAK and ASM are different then expand storage command failed.

       Expand Storage Operation Status :
         OAKD : Success
          ASM : Success

       If the command reported status is not 'Success' then manually
       verify the status by following the steps outlined above.

4. Run the `odaadmcli show disk` command to ensure that all disks are listed, are online, and are in a good state.

       # odaadmcli show disk
               NAME PATH TYPE STATE STATE_DETAILS

               pd_00 /dev/nvme0n1 NVD ONLINE Good
               pd_01 /dev/nvme1n1 NVD ONLINE Good
               pd_02 /dev/nvme3n1 NVD ONLINE Good
               pd_03 /dev/nvme14n2 NVD ONLINE Good
               pd_04_c1 /dev/nvme10n1 NVD ONLINE Good
               pd_04_c2 /dev/nvme11n1 NVD ONLINE Good
               pd_05_c1 /dev/nvme8n1 NVD ONLINE Good
               pd_05_c2 /dev/nvme9n1 NVD ONLINE Good
       #

5. Verify that the two AIC disks are added to Oracle Automatic Storage Management (Oracle ASM) as follows:
   1. Run `asm_script` to verify that the disks in slots 3 and 4 are added to Oracle ASM. Verify that both disks are successfully added (CACHED and MEMBER). Following is an example of default configuration of 80:20 where eight partitions (p1 to p8) are part of the DATA disk group and two partitions (p9 and p10) are part of the RECO diskgroup.

          # su gridUser /opt/oracle/oak/bin/stordiag/asm_script.sh 0 6
          # su grid /opt/oracle/oak/bin/stordiag/asm_script.sh 0 6

          SQL*Plus: Release 19.0.0.0.0 - Production on Mon Feb 28 18:33:11 2026
          Version 19.30.0.0.0

          Copyright (c) 1982, 2025, Oracle.  All rights reserved.

          Connected to:
          Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
          Version 19.30.0.0.0

          SQL> SQL> SQL> SQL> SQL>
          PATH                                     NAME                                GROUP_NUMBER STATE    MODE_ST MOUNT_S HEADER_STATU
          ---------------------------------------- ----------------------------------- ------------ -------- ------- ------- ------------
          /dev/NVD_S04_C1_PHCR42860p1              NVD_S04_C1_PHCR42860P1                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S04_C1_PHCR42860p2              NVD_S04_C1_PHCR42860P2                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S04_C1_PHCR42860p3              NVD_S04_C1_PHCR42860P3                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S04_C1_PHCR42860p4              NVD_S04_C1_PHCR42860P4                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S04_C1_PHCR42860p5              NVD_S04_C1_PHCR42860P5                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S04_C2_PHCR42860p10             NVD_S04_C2_PHCR42860P10                        2 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S04_C2_PHCR42860p6              NVD_S04_C2_PHCR42860P6                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S04_C2_PHCR42860p7              NVD_S04_C2_PHCR42860P7                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S04_C2_PHCR42860p8              NVD_S04_C2_PHCR42860P8                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S04_C2_PHCR42860p9              NVD_S04_C2_PHCR42860P9                         2 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S05_C1_PHCR42800p1              NVD_S05_C1_PHCR42800P1                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S05_C1_PHCR42800p2              NVD_S05_C1_PHCR42800P2                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S05_C1_PHCR42800p3              NVD_S05_C1_PHCR42800P3                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S05_C1_PHCR42800p4              NVD_S05_C1_PHCR42800P4                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S05_C1_PHCR42800p5              NVD_S05_C1_PHCR42800P5                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S05_C2_PHCR42800p10             NVD_S05_C2_PHCR42800P10                        2 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S05_C2_PHCR42800p6              NVD_S05_C2_PHCR42800P6                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S05_C2_PHCR42800p7              NVD_S05_C2_PHCR42800P7                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S05_C2_PHCR42800p8              NVD_S05_C2_PHCR42800P8                         1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/NVD_S05_C2_PHCR42800p9              NVD_S05_C2_PHCR42800P9                         2 NORMAL   ONLINE  CACHED  MEMBER
          /dev/SSD_QRMDSK_p1                                                                      0 NORMAL   ONLINE  CLOSED  FORMER
          /dev/SSD_QRMDSK_p2                                                                      0 NORMAL   ONLINE  CLOSED  FORMER

          SQL> Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
          Version 19.30.0.0.0
          #

   2. Use the `odaadmcli show validation storage errors` command to view hard storage errors. Hard errors include having the wrong type of disk inserted into a particular slot, an invalid disk model, or an incorrect disk size.

          # odaadmcli show validation storage errors

   3. Use the `odaadmcli show validation storage failures` command to view soft validation errors. A typical soft disk error would be an invalid version of the disk firmware.

          # odaadmcli show validation storage failures

   4. Confirm that the `oak_storage_conf.xml` file shows the number of disks added. For example, if you added two AIC to four SFF, then the `oak_storage_conf.xml` file must show `numberOfDisks` as 8.

          #cat /opt/oracle/oak/conf/oak_storage_conf.xml
          <!--
          This file is created by the ODA software as part of system provisioning
          based on system provisioning requests. Values of element nodes can be
          changed by OAK in response to storage configuration change operation.
          DO NOT EDIT THIS FILE.
          -->
          <CometConfiguration>
            <OakStorageConfigInfo type="string" dimension="vector" readonly="true" required="true" default="">
              <!--  Number of disks part of OAK -->
              <numberOfDisks>8</numberOfDisks>
              <!--   Number of partitions per disk part of DATA diskgroup in multiple partition scheme   -->
              <!--   Number of partitions per disk part of RECO diskgroup in multiple partition scheme   -->
              <!--   are derived from number of partitions per disk which are part of ASM DATA diskgroup -->
              <numOfDataDiskPartitionInAsm>8</numOfDataDiskPartitionInAsm>
            </OakStorageConfigInfo>
          </CometConfiguration>
          #

      For example, to add four (4) AIC disks, you must specify `ndisk` values as 8 because AIC has two NVMe disks, so the total value of four AIC disks is eight NVMe disks.

          #odaadmcli expand storage -ndisk 8
          Running precheck, it may take a few minutes.
          Precheck passed.

          Check the progress of expansion of storage by executing 'odaadmcli show disk'
          Waiting for expansion to finish. It may take several minutes to complete
          depending upon the number of disks being expanded

          odaadmcli expand storage command completed.

          Check status of newly expanded disk in OAK by executing 'odaadmcli show disk'
          All of them must be ONLINE status. Check status of newly expanded disk in ASM
          by executing 'asmcmd lsdsk -p', mount status must be CACHED and header status
          should be MEMBER for all the newly added disks. If the status of newly added
          disks in OAK and ASM are different then expand storage command failed.

          Expand Storage Operation Status :
            OAKD : Success
             ASM : Success

          If the command reported status is not 'Success' then manually
          verify the status by following the steps outlined above.

   5. Run the `odaadmcli show disk` command to ensure that all disks are listed, are online, and are in a good state.

          # odaadmcli show disk
          pd_00 /dev/nvme0n1 NVD ONLINE Good
          pd_01 /dev/nvme1n1 NVD ONLINE Good
          pd_02 /dev/nvme2n1 NVD ONLINE Good
          pd_03 /dev/nvme3n1 NVD ONLINE Good
          pd_04_c1 /dev/nvme4n1 NVD ONLINE Good
          pd_04_c2 /dev/nvme5n1 NVD ONLINE Good
          pd_05_c1 /dev/nvme9n1 NVD ONLINE Good
          pd_05_c2 /dev/nvme8n1 NVD ONLINE Good
          pd_06_c1 /dev/nvme11n1 NVD ONLINE Good
          pd_06_c2 /dev/nvme10n1 NVD ONLINE Good
          pd_07_c1 /dev/nvme12n1 NVD ONLINE Good
          pd_07_c2 /dev/nvme13n1 NVD ONLINE Good

   6. Verify that the disks are added to Oracle Automatic Storage Management (Oracle ASM) as follows:
      1. Run `asm_script` to verify that the AIC disks are added to Oracle ASM. Verify that both disks are successfully added (CACHED and MEMBER). Following is example of default configuration of 80:20 where eight partitions (p1 to p8) are part of the DATA disk group and two partitions (p9 and p10) are part of the RECO diskgroup.

             # su gridUser /opt/oracle/oak/bin/stordiag/asm_script.sh 0 6
             # su grid /opt/oracle/oak/bin/stordiag/asm_script.sh 0 6

             SQL*Plus: Release 19.0.0.0.0 - Production on Mon Feb 28 18:46:15 2026
             Version 19.30.0.0.0

             Copyright (c) 1982, 2025, Oracle.  All rights reserved.

             Connected to:
             Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
             Version 19.30.0.0.0

             SQL> SQL> SQL> SQL> SQL>
             PATH                                     NAME                                GROUP_NUMBER STATE    MODE_ST MOUNT_S HEADER_STATU
             ---------------------------------------- ----------------------------------- ------------ -------- ------- ------- ------------
             /dev/NVD_S04_C1_PHCR42860p1              NVD_S04_C1_PHCR42860P1                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S04_C1_PHCR42860p2              NVD_S04_C1_PHCR42860P2                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S04_C1_PHCR42860p3              NVD_S04_C1_PHCR42860P3                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S04_C1_PHCR42860p4              NVD_S04_C1_PHCR42860P4                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S04_C1_PHCR42860p5              NVD_S04_C1_PHCR42860P5                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S04_C2_PHCR42860p10             NVD_S04_C2_PHCR42860P10                        2 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S04_C2_PHCR42860p6              NVD_S04_C2_PHCR42860P6                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S04_C2_PHCR42860p7              NVD_S04_C2_PHCR42860P7                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S04_C2_PHCR42860p8              NVD_S04_C2_PHCR42860P8                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S04_C2_PHCR42860p9              NVD_S04_C2_PHCR42860P9                         2 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S05_C1_PHCR42800p1              NVD_S05_C1_PHCR42800P1                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S05_C1_PHCR42800p2              NVD_S05_C1_PHCR42800P2                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S05_C1_PHCR42800p3              NVD_S05_C1_PHCR42800P3                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S05_C1_PHCR42800p4              NVD_S05_C1_PHCR42800P4                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S05_C1_PHCR42800p5              NVD_S05_C1_PHCR42800P5                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S05_C2_PHCR42800p10             NVD_S05_C2_PHCR42800P10                        2 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S05_C2_PHCR42800p6              NVD_S05_C2_PHCR42800P6                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S05_C2_PHCR42800p7              NVD_S05_C2_PHCR42800P7                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S05_C2_PHCR42800p8              NVD_S05_C2_PHCR42800P8                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S05_C2_PHCR42800p9              NVD_S05_C2_PHCR42800P9                         2 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S06_C1_PHCR42860p1              NVD_S06_C1_PHCR42860P1                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S06_C1_PHCR42860p2              NVD_S06_C1_PHCR42860P2                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S06_C1_PHCR42860p3              NVD_S06_C1_PHCR42860P3                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S06_C1_PHCR42860p4              NVD_S06_C1_PHCR42860P4                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S06_C1_PHCR42860p5              NVD_S06_C1_PHCR42860P5                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S06_C2_PHCR42860p10             NVD_S06_C2_PHCR42860P10                        2 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S06_C2_PHCR42860p6              NVD_S06_C2_PHCR42860P6                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S06_C2_PHCR42860p7              NVD_S06_C2_PHCR42860P7                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S06_C2_PHCR42860p8              NVD_S06_C2_PHCR42860P8                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S06_C2_PHCR42860p9              NVD_S06_C2_PHCR42860P9                         2 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S07_C1_PHCR43000p1              NVD_S07_C1_PHCR43000P1                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S07_C1_PHCR43000p2              NVD_S07_C1_PHCR43000P2                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S07_C1_PHCR43000p3              NVD_S07_C1_PHCR43000P3                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S07_C1_PHCR43000p4              NVD_S07_C1_PHCR43000P4                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S07_C1_PHCR43000p5              NVD_S07_C1_PHCR43000P5                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S07_C2_PHCR43000p10             NVD_S07_C2_PHCR43000P10                        2 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S07_C2_PHCR43000p6              NVD_S07_C2_PHCR43000P6                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S07_C2_PHCR43000p7              NVD_S07_C2_PHCR43000P7                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S07_C2_PHCR43000p8              NVD_S07_C2_PHCR43000P8                         1 NORMAL   ONLINE  CACHED  MEMBER
             /dev/NVD_S07_C2_PHCR43000p9              NVD_S07_C2_PHCR43000P9                         2 NORMAL   ONLINE  CACHED  MEMBER
             /dev/SSD_QRMDSK_p2                                                                      0 NORMAL   ONLINE  CLOSED  FORMER
             /dev/SSD_QRMDSK_p1                                                                      0 NORMAL   ONLINE  CLOSED  FORMER

             SQL> Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
             Version 19.30.0.0.0
             #

      2. Use the `odaadmcli show validation storage errors` command to view hard storage errors. Hard errors include having the wrong type of disk inserted into a particular slot, an invalid disk model, or an incorrect disk size.

             # odaadmcli show validation storage errors

      3. Use the `odaadmcli show validation storage failures` command to view soft validation errors. A typical soft disk error would be an invalid version of the disk firmware.

             # odaadmcli show validation storage failures

      4. Confirm that the `oak_storage_conf.xml` file shows the number of disks added. For example, if you added four AIC to four SFF, then the `oak_storage_conf.xml` file must show `numberOfDisks` as 12, that is, four SFF NVME disks and eight NVMe disks for four AIC.

             #cat /opt/oracle/oak/conf/oak_storage_conf.xml
             <!--
             This file is created by the ODA software as part of system provisioning
             based on system provisioning requests. Values of element nodes can be
             changed by OAK in response to storage configuration change operation.
             DO NOT EDIT THIS FILE.
             -->
             <CometConfiguration>
               <OakStorageConfigInfo type="string" dimension="vector" readonly="true" required="true" default="">
                 <!--  Number of disks part of OAK -->
                 <numberOfDisks>12</numberOfDisks>
                 <!--   Number of partitions per disk part of DATA diskgroup in multiple partition scheme   -->
                 <!--   Number of partitions per disk part of RECO diskgroup in multiple partition scheme   -->
                 <!--   are derived from number of partitions per disk which are part of ASM DATA diskgroup -->
                 <numOfDataDiskPartitionInAsm>12</numOfDataDiskPartitionInAsm>
               </OakStorageConfigInfo>
             </CometConfiguration>
             #

Parent topic: Managing Storage on Single-Node Systems

#### Replacing Small Form Factor (SFF) NVMe Storage Disks

Understand how you can replace existing SFF NVMe disks on Oracle Database Appliance.

Preparing for a Storage Upgrade

1. Check the disk health of the existing storage disks.

       # odaadmcli show disk

2. Run the the `odaadmcli show disk` and `asmcmd lsdsk -p` commands to view and review the storage disk information in OAKD and Oracle Automatic Storage Management (Oracle ASM).

       # odaadmcli show disk

       # asmcmd lsdsk -p

3. Use ORAchk to confirm Oracle ASM and Oracle Clusterware health.

Replacing NVMe Storage Disks

Follow all these steps to replace NVMe storage disks:

WARNING:

Pulling a drive before powering it off will crash the kernel, which can lead to data corruption. Do not pull the drive when the LED is an amber or green color. When you need to replace an NVMe drive, use the software to power off the drive before pulling the drive from the slot. If you have more than one disk to replace, complete the replacement of one disk before starting replacement of the next disk.

1. Power OFF the NVMe disk before removing it from the slot.
2. Wait for one minute for OAKD to complete the operation for disk removal.
3. Insert the new disk in the slot.
4. Wait for at least 2-3 minutes between inserting each disk for OAKD to complete the operation to add the disk to Oracle ASM and OAK.
5. Check the status of the new disk in OAKD with the `odaadmcli show disk` command. The disk must have the status `Online` and `Good` in OAKD. Check the status of the new disk in Oracle ASM with the `asmcmd lsdsk -p` command. The disk must be in `CACHED MEMBER ONLINE NORMAL` state.

       # odaadmcli show disk

       # asmcmd lsdsk -p

Parent topic: Managing Storage on Single-Node Systems

### Managing Storage on High-Availability Systems

Understand the storage for your Oracle Database Appliance X11-HA system.

- About Storage Options for Oracle Database Appliance X11-HA
  Oracle Database Appliance High-Availability systems have options for high performance and high capacity storage configurations.
- Adding Solid-State Drives (SSDs) for Data Storage
  Add a pack of solid-state drives (SSDs) for data storage into the existing Oracle Database Appliance X11-HA base configuration to fully populate the base storage shelf.
- Adding the Storage Expansion Shelf
  After the base storage shelf is fully populated, you can add the storage expansion shelf to expand your data storage on your high-availability platform.

Parent topic: Managing Storage

#### About Storage Options for Oracle Database Appliance X11-HA

Oracle Database Appliance High-Availability systems have options for high performance and high capacity storage configurations.

The base configuration of Oracle Database Appliance X11-HA hardware model has six slots (slots 0-5) with 7.68 TB drives of SSD raw storage. If you choose to order and deploy the full storage capacity, then you can fill the remaining 18 slots (slots 6-23) with either SSD or HDD drives. For even more storage, you can add a storage expansion shelf to double the storage capacity of your appliance.

In all configurations, the base storage and the storage expansion shelf each have six SSDs for DATA/RECO in the SSD option or FLASH in the HDD option.

Oracle Database Appliance X11-HA does not allocate dedicated SSD drives for REDO disk groups. Instead, the space for REDO logs is allocated on SSD drives as required.

For Oracle ASM storage, the REDO logs are stored in the available disk group space during database creation, based on the database shape selected. For Oracle ACFS storage, the space for REDO logs is allocated during the database storage creation assuming the minimum db shape (odb1s). If you create the database storage without database, then the space allocated for REDO logs is 4 GB, assuming the minimum db shape (odb1s). Subsequently, when you create a database with your required database shape on the existing database storage, the REDO logs space is extended based on shape of the database.

On Oracle Database Appliance X11-HA High Performance configurations, with only SSD drives, the DATA and RECO disk groups use all the SSD drives whether 6, 12, 18, 24, or 48 with storage expansion shelf. REDO logs are stored in the RECO disk group.

On Oracle Database Appliance X11-HA High Capacity configurations, with both HDD and SSD drives, the DATA and RECO disk groups use the HDD drives, and the SSD drives store the FLASH disk group. REDO logs are stored in the FLASH disk group.

On both High Performance and High Capacity configurations, REDO logs are always created on SSD drives, similar to earlier Oracle Database Appliance hardware models. REDO logs are always created with high redundancy irrespective of the redundancy level of the disk group, whether RECO or FLASH.

High Performance

A high performance configuration uses solid state drives (SSDs) for DATA and RECO storage. The base configuration has six disks, each with 7.68 TB SSD raw storage for DATA and RECO.

You can add up to three (3) 6-Pack SSDs on the base configuration, for a total of 184.32 TB SSD raw storage. If you need more storage, you can double the capacity by adding an expansion shelf of SSD drives. The expansion shelf provides an additional 24 SSDs, each with 7.68TB raw storage for DATA and RECO, for a total of another 184.32 TB SSD raw storage.

Adding an expansion shelf requires that the base storage shelf and expansion shelf are fully populated with SSD drives. When you expand the storage, there is no downtime.

A system fully configured for high performance has 368.64 TB SSD raw storage for DATA and RECO.

High Capacity

A high capacity configuration uses a combination of SSD and HDD drives.

The base configuration has six disks, each with 7.68 TB SSD raw storage for FLASH.

The following expansion options are available:

- Base shelf: additional 396 TB HDD raw storage for DATA and RECO (18 HDDs, each with 22 TB storage)

- Expansion Storage shelf: additional shelf storage configuration must be identical to the storage configuration of the base shelf.

A system fully configured for high capacity has a total of 884.16 TB raw storage for DATA, RECO, and FLASH, with 92.16 TB SSD and 792 TB HDD.

Table 11-2 Storage Options for Oracle Database Appliance X11-HA

Configuration | Oracle Database Appliance X11-HA SSD-Only Configuration for High Performance | Oracle Database Appliance X11-HA SSD and HDD Configuration for High Capacity
--- | --- | ---
Base configuration | Base storage shelf contains 6 SSDs of 7.68 TB. 6 x 7.68 TB SSD = 46 TB SSD | Base storage shelf is fully populated with 6-pack SSDs of 7.68 TB and 18-drives of HDDs with 22 TB. 6 x 7.68 TB SSD = 46 TB SSD 18 x 22TB HDD = 396 TB HDD Total storage on the first JBOD = 442 TB, with 46 TB SSD and 396 TB HDD
Storage addition options | Base shelf contains 6 SSDs. Additional 18 SSDs must be added in packs of 6. Base system: 6 x 7.68 TB SSD = 46 TB SSD Adding 6 SSDs: 12x 7.68 TB SSD = 92 TB SSD Adding 12 SSDs: 18 x 7.68 TB SSD = 138 TB SSD Adding 18 SSDs: 24 x 7.68 TB SSD = 184 TB SSD (full shelf) | Not applicable. Base storage shelf is fully populated.
Storage shelf expansion options | The optional expansion storage shelf can only be installed after the base storage shelf is fully populated, and it must have the same configuration as the base storage shelf. Total storage on the base storage shelf = 184 TB SSD Storage on the expansion shelf = 24 x 7.68 TB SSD = 184 TB SSD Total storage including both JBODs = 368.6 TB SSD | The optional expansion storage shelf can only be installed after the base storage shelf is fully populated, and it must have the same configuration as the base storage shelf. Total storage on the base storage shelf = 442 TB, with 46 TB SSD and 396 TB HDD Total storage including both JBODs = 884 TB, with 92 TB SSD and 792 TB HDD

Converting High Performance to High Capacity system

You can order a fully populated base storage shelf of 6 SSD + 18 HDD for high capacity configuration. If you initially ordered only the default 6 SSD base storage shelf and deployed high performance system, then you can later convert this to high capacity system configuration. Order 18 HDD and populate the base storage shelf as follows:

1. Takea backup of your database on the default storage of 6 SSD configuration on the provisioned system.
2. Run the cleanup utility with the `--erasedata` option on both nodes to erase all SSD disks of OAK and Oracle ASM headers. The `--erasedata` option completely erases the SSD disk so any data on the disks is lost and becomes unrecoverable.
3. After cleanup of both nodes, add 18 HDD disks to the base storage shelf. These 18 HDDs must be brand new disks from Oracle without any OAK and Oracle ASM header.
4. Reimage both nodes with the required OAK version.
5. Reprovision the system again.

Parent topic: Managing Storage on High-Availability Systems

#### Adding Solid-State Drives (SSDs) for Data Storage

Add a pack of solid-state drives (SSDs) for data storage into the existing Oracle Database Appliance X11-HA base configuration to fully populate the base storage shelf.

If you need to add storage to the base configuration, you can order one, two, or three 6-pack of SSDs to complete the base configuration on Oracle Database Appliance X11-HA.

You must fully populate the base configuration before you can add an expansion shelf to Oracle Database Appliance X11-HA. If you add an expansion shelf, the shelf must have the same disk storage configuration as the base configuration.

Note:

For a high-performance configuration, you can add SSDs to the base storage shelf or add a storage expansion shelf. For high-capacity base configuration with 6-SSDs, if you want to expand storage to use HDDs, then you must reimage and deploy the appliance.
Before adding the disks to the system, ensure that Oracle Database Appliance is on the latest release.

1. Insert disks one at a time in the slots.

   To add one 6-pack of SSDs, insert the disks in slots 6 to 11. To add two 6-pack of SSDs, insert the disks in slots 6 to 17. To add three 6-pack of SSDs, insert the disks in slots 6 to 23.

   Note:

   Allow at least one minute between inserting each disk.
   After all disks are added, go to Step 2.
2. Run the `odaadmcli expand storage` command on any node.

       # odaadmcli expand storage -ndisk number_of_disks_to_be_added -enclosure enclosure_number_of_the_disks_to_be_added

   The enclosure number is 0 when you add storage disks to the first storage enclosure or JBOD (just a bunch of disks).

   For example:

       # odaadmcli expand storage -ndisk 6 -enclosure 0
       Running precheck, it may take a few minutes.
       Precheck passed.

       Check the progress of expansion of storage by executing 'odaadmcli show disk'
       Waiting for expansion to finish. It may take several minutes to complete
       depending upon the number of disks being expanded

       odaadmcli expand storage command completed.

       Check status of newly expanded disk in OAK by executing 'odaadmcli show disk'
       All of them must be ONLINE status. Check status of newly expanded disk in ASM
       by executing 'asmcmd lsdsk -p', mount status must be CACHED and header status
       should be MEMBER for all the newly added disks. If the status of newly added
       disks in OAK and ASM are different then expand storage command failed.

       Expand Storage Operation Status :
         OAKD : Success
          ASM : Success

       If the command reported status is not 'Success' then manually
       verify the status by following the steps outlined above.

   It takes 10 to 12 minutes to add all of the disks to the configuration.
3. Run the `odaadmcli show disk` command to ensure that all disks are listed, are online, and are in a good state.

       # odaadmcli show disk

4. Verify that the disks in slots 6 to 11 are added to Oracle Automatic Storage Management (Oracle ASM).
   1. Run the `asm_script` to verify that the disks in slots 6 to 11 are added to Oracle Automatic Storage Management (Oracle ASM). If the 6 disks are successfully added (CACHED and MEMBER), then go to Step 7. Following is example of default configuration of 80:20 where eight partitions (p1 to p8) are part of the DATA disk group and two partitions (p9 and p10) are part of the RECO diskgroup.

          # su gridUser /opt/oracle/oak/bin/stordiag/asm_script.sh 1 6

      For example:

          # su gridUser /opt/oracle/oak/bin/stordiag/asm_script.sh 1 6
          # su grid /opt/oracle/oak/bin/stordiag/asm_script.sh 1 6

          SQL*Plus: Release 19.0.0.0.0 - Production on Wed Feb 11 09:26:05 2026
          Version 19.30.0.0.0
          Copyright (c) 1982, 2025, Oracle.  All rights reserved.
          Connected to:
          Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
          Version 19.30.0.0.0

          SQL> SQL> SQL> SQL> SQL>
          PATH                                             NAME                                 GROUP_NUMBER STATE    MODE_ST MOUNT_S HEADER_STATUS
          ---------------------------------------- -----------------------------------         -------------------- ------- ------- ---------------
          /dev/mapper/SSD_E0_S06_46724304p1                SSD_E0_S06_46724304P1                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S06_46724304p10               SSD_E0_S06_46724304P10                         2 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S06_46724304p2                SSD_E0_S06_46724304P2                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S06_46724304p3                SSD_E0_S06_46724304P3                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S06_46724304p4                SSD_E0_S06_46724304P4                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S06_46724304p5                SSD_E0_S06_46724304P5                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S06_46724304p6                SSD_E0_S06_46724304P6                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S06_46724304p7                SSD_E0_S06_46724304P7                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S06_46724304p8                SSD_E0_S06_46724304P8                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S06_46724304p9                SSD_E0_S06_46724304P9                          2 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S07_47182256p1                SSD_E0_S07_47182256P1                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S07_47182256p10               SSD_E0_S07_47182256P10                         2 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S07_47182256p2                SSD_E0_S07_47182256P2                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S07_47182256p3                SSD_E0_S07_47182256P3                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S07_47182256p4                SSD_E0_S07_47182256P4                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S07_47182256p5                SSD_E0_S07_47182256P5                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S07_47182256p6                SSD_E0_S07_47182256P6                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S07_47182256p7                SSD_E0_S07_47182256P7                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S07_47182256p8                SSD_E0_S07_47182256P8                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S07_47182256p9                SSD_E0_S07_47182256P9                          2 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S08_46724288p1                SSD_E0_S08_46724288P1                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S08_46724288p10               SSD_E0_S08_46724288P10                         2 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S08_46724288p2                SSD_E0_S08_46724288P2                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S08_46724288p3                SSD_E0_S08_46724288P3                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S08_46724288p4                SSD_E0_S08_46724288P4                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S08_46724288p5                SSD_E0_S08_46724288P5                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S08_46724288p6                SSD_E0_S08_46724288P6                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S08_46724288p7                SSD_E0_S08_46724288P7                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S08_46724288p8                SSD_E0_S08_46724288P8                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S08_46724288p9                SSD_E0_S08_46724288P9                          2 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S09_47180592p1                SSD_E0_S09_47180592P1                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S09_47180592p10               SSD_E0_S09_47180592P10                         2 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S09_47180592p2                SSD_E0_S09_47180592P2                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S09_47180592p3                SSD_E0_S09_47180592P3                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S09_47180592p4                SSD_E0_S09_47180592P4                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S09_47180592p5                SSD_E0_S09_47180592P5                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S09_47180592p6                SSD_E0_S09_47180592P6                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S09_47180592p7                SSD_E0_S09_47180592P7                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S09_47180592p8                SSD_E0_S09_47180592P8                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S09_47180592p9                SSD_E0_S09_47180592P9                          2 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S10_47182288p1                SSD_E0_S10_47182288P1                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S10_47182288p10               SSD_E0_S10_47182288P10                         2 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S10_47182288p2                SSD_E0_S10_47182288P2                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S10_47182288p3                SSD_E0_S10_47182288P3                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S10_47182288p4                SSD_E0_S10_47182288P4                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S10_47182288p5                SSD_E0_S10_47182288P5                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S10_47182288p6                SSD_E0_S10_47182288P6                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S10_47182288p7                SSD_E0_S10_47182288P7                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S10_47182288p8                SSD_E0_S10_47182288P8                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S10_47182288p9                SSD_E0_S10_47182288P9                          2 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S11_46722912p1                SSD_E0_S11_46722912P1                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S11_46722912p10               SSD_E0_S11_46722912P10                         2 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S11_46722912p2                SSD_E0_S11_46722912P2                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S11_46722912p3                SSD_E0_S11_46722912P3                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S11_46722912p4                SSD_E0_S11_46722912P4                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S11_46722912p5                SSD_E0_S11_46722912P5                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S11_46722912p6                SSD_E0_S11_46722912P6                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S11_46722912p7                SSD_E0_S11_46722912P7                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S11_46722912p8                SSD_E0_S11_46722912P8                          1 NORMAL   ONLINE  CACHED  MEMBER
          /dev/mapper/SSD_E0_S11_46722912p9                SSD_E0_S11_46722912P9                          2 NORMAL   ONLINE  CACHED  MEMBER

          SQL> Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
          Version 19.30.0.0.0
          #

   2. Use the `odaadmcli show validation storage errors` command to view hard storage errors. Hard errors include having the wrong type of disk inserted into a particular slot, an invalid disk model, or an incorrect disk size.

          # odaadmcli show validation storage errors

   3. Use the `odaadmcli show validation storage failures` command to view soft validation errors. A typical soft disk error would be an invalid version of the disk firmware.

          # odaadmcli show validation storage failures

   4. Confirm that the `oak_storage_conf.xml` file shows the number of disks added. For example, if you added 6 disks to the base configuration, then the `oak_storage_conf.xml` file must show `numberOfDisks` as 12. If you added 12 disks to the base configuration, then the `oak_storage_conf.xml` file must show 18.

          #cat /opt/oracle/oak/conf/oak_storage_conf.xml
          <!--
          This file is created by the ODA software as part of system provisioning
          based on system provisioning requests. Values of element nodes can be
          changed by OAK in response to storage configuration change operation.
          DO NOT EDIT THIS FILE.
          -->
          <CometConfiguration>
            <OakStorageConfigInfo type="string" dimension="vector" readonly="true" required="true" default="">
              <!--  Number of disks part of OAK -->
              <numberOfDisks>12</numberOfDisks>
              <!--   Number of partitions per disk part of DATA diskgroup in multiple partition scheme   -->
              <!--   Number of partitions per disk part of RECO diskgroup in multiple partition scheme   -->
              <!--   are derived from number of partitions per disk which are part of ASM DATA diskgroup -->
              <numOfDataDiskPartitionInAsm>8</numOfDataDiskPartitionInAsm>
            </OakStorageConfigInfo>
          </CometConfiguration>
          #

Parent topic: Managing Storage on High-Availability Systems

#### Adding the Storage Expansion Shelf

After the base storage shelf is fully populated, you can add the storage expansion shelf to expand your data storage on your high-availability platform.

The expansion shelf is available on Oracle Database Appliance high-availability platforms, such as Oracle Database Appliance X11-HA. The addition of the storage expansion shelf includes checks across both nodes. It is important to confirm that SSH does work across the nodes and all users can connect as expected using their shared password.

You must fully populate the base configuration before you can add an expansion shelf. If you add an expansion shelf, the shelf must have the same disk storage configuration as the base storage shelf.

Note:

Oracle recommends that you add a storage expansion shelf when you have relatively little activity on your databases. When the system discovers the new storage, Oracle Automatic Storage Management (Oracle ASM) automatically rebalances the disk groups. The rebalance operation may degrade database performance until the operation completes.

1. Install and cable the storage expansion shelf, but do not power on the expansion shelf.

   Caution:

   Review cabling instructions carefully to ensure that you have carried out cabling correctly. Incorrect connections can cause data loss when adding a storage expansion shelf to Oracle Database Appliance with existing databases.

2. Verify that both nodes plus the base storage shelf are up and running. Log into each server node and run the`odacli validate-storagetopology` command to confirm that the base configuration cabling is correct.

       # odacli validate-storagetopology
        ...
             INFO  : Check if JBOD powered on
         SUCCESS   : JBOD : Powered-on
             INFO  : Check for correct number of EBODS(2 or 4)
         SUCCESS   : EBOD found : 2                                                                                                                                                                     INFO       : Check for overall status of cable validation on Node0
         SUCCESS   : Overall Cable Validation on Node0
         SUCCESS   : JBOD Nickname set correctly : Oracle Database Appliance - E0

   Run the command to confirm that the two server nodes are properly cabled to the base storage shelf and all disks are online, with a good status, and added to the existing disk groups on both nodes. If there any failures, then fix the cabling before proceeding to the next step.

   Note:

   If the output shows that `EBOD found` is 2, then you only have the base storage shelf. If `EBOD found` is 4, then you have a base storage shelf and an expansion shelf.

   Note:

   If you add a new JBOD fresh from the factory, then the output of the `odacli validate-storagetopology` command is:

       # odacli validate-storagetopology
        ...
       WARNING : JBOD Nickname is incorrectly set to :

3. Power on the storage expansion shelf and wait for 20 minutes before issuing the CLI command for storage expansion.
4. Log in to each server node and run the `odacli validate-storagetopology` command to validate the storage cabling and confirm that the new storage shelf is recognized.

       # odacli validate-storagetopology

         INFO    : Check if JBOD powered on
         SUCCESS : 2JBOD : Powered-on
         INFO    : Check for correct number of EBODS(2 or 4)
         SUCCESS : EBOD found : 4
          ...
          ...

          INFO    : Check for overall status of cable validation on Node0
          SUCCESS : Overall Cable Validation on Node0
          SUCCESS : JBOD0 Nickname set correctly : Oracle Database Appliance - E0
          SUCCESS : JBOD1 Nickname set correctly : Oracle Database Appliance - E1

   If you add a new JBOD fresh from the factory, then the output of the `odacli validate-storagetopology` command is:

       # odacli validate-storagetopology
        ...
       WARNING : JBOD Nickname is incorrectly set to :

   Look for the following indicators that both storage shelves are recognized:
   - When there are two shelves, the JBOD is numbered. For example:

         SUCCESS : 2JBOD : Powered-on

   - When both shelves are recognized, the `EBOD found` value is 4.

         SUCCESS : EBOD found : 4

   - When the expansion shelf is cabled properly, the nickname is `E1`. For example:

                 SUCCESS : JBOD0 Nickname set correctly : Oracle Database Appliance - E0
                 SUCCESS : JBOD1 Nickname set correctly : Oracle Database Appliance - E1

   Fix any errors before proceeding.

5. Run the `odaadmcli show disk` command to ensure that all disks in the expansion shelf are listed, are online, and are in a good state.

       # odaadmcli show disk

   When all disks are online and in a good state, proceed to the next step.
6. Run the `odaadmcli show enclosure` command to check the health of components in expansion shelf.

       # odaadmcli show enclosure

7. Run the `odaadmcli expand storage` command.

       # odaadmcli expand storage -ndisk 24 -enclosure 1
       Running precheck, it may take a few minutes.
       Precheck passed.

       Check the progress of expansion of storage by executing 'odaadmcli show disk'
       Waiting for expansion to finish. It may take several minutes to complete
       depending upon the number of disks being expanded

       odaadmcli expand storage command completed.

       Check status of newly expanded disk in OAK by executing 'odaadmcli show disk'
       All of them must be ONLINE status. Check status of newly expanded disk in ASM
       by executing 'asmcmd lsdsk -p', mount status must be CACHED and header status
       should be MEMBER for all the newly added disks. If the status of newly added
       disks in OAK and ASM are different then expand storage command failed.

       Expand Storage Operation Status :
         OAKD : Success
          ASM : Success

       If the command reported status is not 'Success' then manually
       verify the status by following the steps outlined above.

   It takes approximately 30 to 40 minutes to add all of the disks to the configuration.
8. Check the status of the newly-expanded disk in OAK by running the `odaadmcli show disk` command. All disks must have status ONLINE. Check the status of the newly-expanded disk in Oracle ASM by running the `asmcmd lsdsk -p` command. The mount status must be CACHED and header status must be MEMBER for all the newly-added disks. If the status of the newly-added disks in OAK and Oracle ASM is different, then the `odaadmcli expand storage` command failed.
   Example of an Oracle Database Appliance X11-HA High Capacity System:

       # odaadmcli show disk
               NAME            PATH            TYPE            STATE           STATE_DETAILS

               e0_pd_00        /dev/sda        SSD             ONLINE          Good
               e0_pd_01        /dev/sdb        SSD             ONLINE          Good
               e0_pd_02        /dev/sdc        SSD             ONLINE          Good
               e0_pd_03        /dev/sdab       SSD             ONLINE          Good
               e0_pd_04        /dev/sdad       SSD             ONLINE          Good
               e0_pd_05        /dev/sdac       SSD             ONLINE          Good
               e0_pd_06        /dev/sdae       HDD             ONLINE          Good
               e0_pd_07        /dev/sdaf       HDD             ONLINE          Good
               e0_pd_08        /dev/sdag       HDD             ONLINE          Good
               e0_pd_09        /dev/sdai       HDD             ONLINE          Good
               e0_pd_10        /dev/sdah       HDD             ONLINE          Good
               e0_pd_11        /dev/sdaj       HDD             ONLINE          Good
               e0_pd_12        /dev/sdak       HDD             ONLINE          Good
               e0_pd_13        /dev/sdam       HDD             ONLINE          Good
               e0_pd_14        /dev/sdal       HDD             ONLINE          Good
               e0_pd_15        /dev/sdan       HDD             ONLINE          Good
               e0_pd_16        /dev/sdap       HDD             ONLINE          Good
               e0_pd_17        /dev/sdar       HDD             ONLINE          Good
               e0_pd_18        /dev/sdaq       HDD             ONLINE          Good
               e0_pd_19        /dev/sdao       HDD             ONLINE          Good
               e0_pd_20        /dev/sdat       HDD             ONLINE          Good
               e0_pd_21        /dev/sdas       HDD             ONLINE          Good
               e0_pd_22        /dev/sdau       HDD             ONLINE          Good
               e0_pd_23        /dev/sdav       HDD             ONLINE          Good
               e1_pd_00        /dev/sdaw       SSD             ONLINE          Good
               e1_pd_01        /dev/sdax       SSD             ONLINE          Good
               e1_pd_02        /dev/sday       SSD             ONLINE          Good
               e1_pd_03        /dev/sdaz       SSD             ONLINE          Good
               e1_pd_04        /dev/sdba       SSD             ONLINE          Good
               e1_pd_05        /dev/sdbb       SSD             ONLINE          Good
               e1_pd_06        /dev/sdbc       HDD             ONLINE          Good
               e1_pd_07        /dev/sdbd       HDD             ONLINE          Good
               e1_pd_08        /dev/sdbe       HDD             ONLINE          Good
               e1_pd_09        /dev/sdbf       HDD             ONLINE          Good
               e1_pd_10        /dev/sdbg       HDD             ONLINE          Good
               e1_pd_11        /dev/sdbh       HDD             ONLINE          Good
               e1_pd_12        /dev/sdbi       HDD             ONLINE          Good
               e1_pd_13        /dev/sdbj       HDD             ONLINE          Good
               e1_pd_14        /dev/sdbk       HDD             ONLINE          Good
               e1_pd_15        /dev/sdbl       HDD             ONLINE          Good
               e1_pd_16        /dev/sdbm       HDD             ONLINE          Good
               e1_pd_17        /dev/sdbn       HDD             ONLINE          Good
               e1_pd_18        /dev/sdbo       HDD             ONLINE          Good
               e1_pd_19        /dev/sdbp       HDD             ONLINE          Good
               e1_pd_20        /dev/sdbq       HDD             ONLINE          Good
               e1_pd_21        /dev/sdbr       HDD             ONLINE          Good
               e1_pd_22        /dev/sdbs       HDD             ONLINE          Good
               e1_pd_23        /dev/sdbt       HDD             ONLINE          Good
       #

       # su grid /opt/oracle/oak/bin/stordiag/asm_script.sh 1 6

       SQL*Plus: Release 19.0.0.0.0 - Production on Wed Jun 11 11:28:05 2025
       Version 19.28.0.0.0
       Copyright (c) 1982, 2025, Oracle.  All rights reserved.
       Connected to:
       Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
       Version 19.28.0.0.0

       SQL> SQL> SQL> SQL> SQL>
       PATH                                             NAME                                 GROUP_NUMBER STATE    MODE_ST MOUNT_S HEADER_STATU
       ---------------------------------------- -----------------------------------  -------------------- ------- ------- ------------
       /dev/mapper/HDD_E0_S06_134748696p1               HDD_E0_S06_134748696P1                         1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/HDD_E0_S06_134748696p10              HDD_E0_S06_134748696P10                        2 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/HDD_E0_S06_134748696p2               HDD_E0_S06_134748696P2                         1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/HDD_E0_S06_134748696p3               HDD_E0_S06_134748696P3                         1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/HDD_E0_S06_134748696p4               HDD_E0_S06_134748696P4                         1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/HDD_E0_S06_134748696p5               HDD_E0_S06_134748696P5                         1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/HDD_E0_S06_134748696p6               HDD_E0_S06_134748696P6                         1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/HDD_E0_S06_134748696p7               HDD_E0_S06_134748696P7                         1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/HDD_E0_S06_134748696p8               HDD_E0_S06_134748696P8                         1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/HDD_E0_S06_134748696p9               HDD_E0_S06_134748696P9                         2 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/HDD_E0_S07_134749392p1               HDD_E0_S07_134749392P1                         1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/HDD_E0_S07_134749392p10              HDD_E0_S07_134749392P10                        2 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/HDD_E0_S07_134749392p2               HDD_E0_S07_134749392P2                         1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/HDD_E0_S07_134749392p3               HDD_E0_S07_134749392P3                         1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/HDD_E0_S07_134749392p4               HDD_E0_S07_134749392P4                         1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/HDD_E0_S07_134749392p5               HDD_E0_S07_134749392P5                         1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/HDD_E0_S07_134749392p6               HDD_E0_S07_134749392P6                         1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/HDD_E0_S07_134749392p7               HDD_E0_S07_134749392P7                         1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/HDD_E0_S07_134749392p8               HDD_E0_S07_134749392P8                         1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/HDD_E0_S07_134749392p9               HDD_E0_S07_134749392P9                         2 NORMAL   ONLINE  CACHED  MEMBER
       ''
       ''
       ''
       SQL> Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
       Version 19.28.0.0.0
       #

   Example of an Oracle Database Appliance X11-HA High Performance System:

       # odaadmcli show disk
               NAME            PATH            TYPE            STATE           STATE_DETAILS

               e0_pd_00        /dev/sda        SSD             ONLINE          Good
               e0_pd_01        /dev/sdc        SSD             ONLINE          Good
               e0_pd_02        /dev/sdaa       SSD             ONLINE          Good
               e0_pd_03        /dev/sdab       SSD             ONLINE          Good
               e0_pd_04        /dev/sdac       SSD             ONLINE          Good
               e0_pd_05        /dev/sdad       SSD             ONLINE          Good
               e0_pd_06        /dev/sdag       SSD             ONLINE          Good
               e0_pd_07        /dev/sdae       SSD             ONLINE          Good
               e0_pd_08        /dev/sdah       SSD             ONLINE          Good
               e0_pd_09        /dev/sdaf       SSD             ONLINE          Good
               e0_pd_10        /dev/sdai       SSD             ONLINE          Good
               e0_pd_11        /dev/sdaj       SSD             ONLINE          Good
               e0_pd_12        /dev/sdak       SSD             ONLINE          Good
               e0_pd_13        /dev/sdam       SSD             ONLINE          Good
               e0_pd_14        /dev/sdal       SSD             ONLINE          Good
               e0_pd_15        /dev/sdan       SSD             ONLINE          Good
               e0_pd_16        /dev/sdap       SSD             ONLINE          Good
               e0_pd_17        /dev/sdao       SSD             ONLINE          Good
               e0_pd_18        /dev/sdaq       SSD             ONLINE          Good
               e0_pd_19        /dev/sdar       SSD             ONLINE          Good
               e0_pd_20        /dev/sdas       SSD             ONLINE          Good
               e0_pd_21        /dev/sdau       SSD             ONLINE          Good
               e0_pd_22        /dev/sdat       SSD             ONLINE          Good
               e0_pd_23        /dev/sdav       SSD             ONLINE          Good
               e1_pd_00        /dev/sdaw       SSD             ONLINE          Good
               e1_pd_01        /dev/sdax       SSD             ONLINE          Good
               e1_pd_02        /dev/sday       SSD             ONLINE          Good
               e1_pd_03        /dev/sdaz       SSD             ONLINE          Good
               e1_pd_04        /dev/sdba       SSD             ONLINE          Good
               e1_pd_05        /dev/sdbb       SSD             ONLINE          Good
               e1_pd_06        /dev/sdbc       SSD             ONLINE          Good
               e1_pd_07        /dev/sdbd       SSD             ONLINE          Good
               e1_pd_08        /dev/sdbe       SSD             ONLINE          Good
               e1_pd_09        /dev/sdbf       SSD             ONLINE          Good
               e1_pd_10        /dev/sdbg       SSD             ONLINE          Good
               e1_pd_11        /dev/sdbh       SSD             ONLINE          Good
               e1_pd_12        /dev/sdbi       SSD             ONLINE          Good
               e1_pd_13        /dev/sdbj       SSD             ONLINE          Good
               e1_pd_14        /dev/sdbk       SSD             ONLINE          Good
               e1_pd_15        /dev/sdbl       SSD             ONLINE          Good
               e1_pd_16        /dev/sdbm       SSD             ONLINE          Good
               e1_pd_17        /dev/sdbn       SSD             ONLINE          Good
               e1_pd_18        /dev/sdbo       SSD             ONLINE          Good
               e1_pd_19        /dev/sdbp       SSD             ONLINE          Good
               e1_pd_20        /dev/sdbq       SSD             ONLINE          Good
               e1_pd_21        /dev/sdbr       SSD             ONLINE          Good
               e1_pd_22        /dev/sdbs       SSD             ONLINE          Good
               e1_pd_23        /dev/sdbt       SSD             ONLINE          Good
       #
       #su grid /opt/oracle/oak/bin/stordiag/asm_script.sh 1 6

       SQL*Plus: Release 19.0.0.0.0 - Production on Wed Jun 11 11:46:05 2025
       Version 19.28.0.0.0
       Copyright (c) 1982, 2025, Oracle.  All rights reserved.
       Connected to:
       Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
       Version 19.28.0.0.0

       SQL> SQL> SQL> SQL> SQL>
       PATH                                            NAME                                GROUP_NUMBER STATE    MODE_ST MOUNT_S HEADER_STATU
       ---------------------------------------- ----------------------------------- ------------ -------- ------- ------- ------------
       /dev/mapper/SSD_E0_S00_51829200p1                SSD_E0_S00_51829200P1                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S00_51829200p10               SSD_E0_S00_51829200P10                         2 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S00_51829200p2                SSD_E0_S00_51829200P2                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S00_51829200p3                SSD_E0_S00_51829200P3                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S00_51829200p4                SSD_E0_S00_51829200P4                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S00_51829200p5                SSD_E0_S00_51829200P5                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S00_51829200p6                SSD_E0_S00_51829200P6                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S00_51829200p7                SSD_E0_S00_51829200P7                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S00_51829200p8                SSD_E0_S00_51829200P8                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S00_51829200p9                SSD_E0_S00_51829200P9                          2 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S01_51829664p1                SSD_E0_S01_51829664P1                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S01_51829664p10               SSD_E0_S01_51829664P10                         2 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S01_51829664p2                SSD_E0_S01_51829664P2                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S01_51829664p3                SSD_E0_S01_51829664P3                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S01_51829664p4                SSD_E0_S01_51829664P4                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S01_51829664p5                SSD_E0_S01_51829664P5                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S01_51829664p6                SSD_E0_S01_51829664P6                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S01_51829664p7                SSD_E0_S01_51829664P7                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S01_51829664p8                SSD_E0_S01_51829664P8                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S01_51829664p9                SSD_E0_S01_51829664P9                          2 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S02_51849168p1                SSD_E0_S02_51849168P1                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S02_51849168p10               SSD_E0_S02_51849168P10                         2 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S02_51849168p2                SSD_E0_S02_51849168P2                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S02_51849168p3                SSD_E0_S02_51849168P3                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S02_51849168p4                SSD_E0_S02_51849168P4                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S02_51849168p5                SSD_E0_S02_51849168P5                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S02_51849168p6                SSD_E0_S02_51849168P6                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S02_51849168p7                SSD_E0_S02_51849168P7                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S02_51849168p8                SSD_E0_S02_51849168P8                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S02_51849168p9                SSD_E0_S02_51849168P9                          2 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S03_51828144p1                SSD_E0_S03_51828144P1                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S03_51828144p10               SSD_E0_S03_51828144P10                         2 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S03_51828144p2                SSD_E0_S03_51828144P2                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S03_51828144p3                SSD_E0_S03_51828144P3                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S03_51828144p4                SSD_E0_S03_51828144P4                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S03_51828144p5                SSD_E0_S03_51828144P5                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S03_51828144p6                SSD_E0_S03_51828144P6                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S03_51828144p7                SSD_E0_S03_51828144P7                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S03_51828144p8                SSD_E0_S03_51828144P8                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S03_51828144p9                SSD_E0_S03_51828144P9                          2 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S04_51829536p1                SSD_E0_S04_51829536P1                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S04_51829536p10               SSD_E0_S04_51829536P10                         2 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S04_51829536p2                SSD_E0_S04_51829536P2                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S04_51829536p3                SSD_E0_S04_51829536P3                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S04_51829536p4                SSD_E0_S04_51829536P4                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S04_51829536p5                SSD_E0_S04_51829536P5                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S04_51829536p6                SSD_E0_S04_51829536P6                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S04_51829536p7                SSD_E0_S04_51829536P7                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S04_51829536p8                SSD_E0_S04_51829536P8                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S04_51829536p9                SSD_E0_S04_51829536P9                          2 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S05_51841136p1                SSD_E0_S05_51841136P1                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S05_51841136p10               SSD_E0_S05_51841136P10                         2 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E0_S05_51841136p2                SSD_E0_S05_51841136P2                          1 NORMAL   ONLINE  CACHED  MEMBER
       /dev/mapper/SSD_E1_S23_51847424p9                SSD_E1_S23_51847424P9                          2 NORMAL   ONLINE  CACHED  MEMBER
       ''
       ''
       ''

       SQL> Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
       Version 19.28.0.0.0
       #

9. Use the `odaadmcli show validation storage errors` command to show hard storage errors.
   Hard errors include having the wrong type of disk inserted into a particular slot, an invalid disk model, or an incorrect disk size.

       # odaadmcli show validation storage errors

10. Use the `odaadmcli show validation storage failures` command to show soft validation errors.
    A typical soft disk error would be an invalid version of the disk firmware.

        # odaadmcli show validation storage failures

11. Run the `odacli describe-component` command to verify that all firmware components in the storage expansion are current.

        # odacli describe-component

12. If needed, update the storage shelf and then run the `odacli describe-component` command to confirm that the firmware is current.

        # odacli update
        # odacli describe-component

Parent topic: Managing Storage on High-Availability Systems