B Storage on Oracle Database Appliance

Review this section to understand Oracle Database Appliance storage architecture and options and how to determine usable storage.

About Oracle Database File Storage

Use Oracle Automatic Storage Management (Oracle ASM) or Oracle Automatic Storage Management Cluster File System (Oracle ACFS) for database files storage.

Database file systems are used exclusively for storing database files, and they include a DATA file system for database data files and a RECO file system for storing archive files and backups. Oracle Database Appliance supports Oracle ACFS and Oracle ASM database file storage. You determine the type of database storage when you create the database. Beginning with Oracle Database Appliance 12.2, if you do not select a storage type, the default storage is Oracle ASM.

About Oracle ASM Database Storage

Use Oracle ASM with Oracle Database 12c release 2 (12.2.0.1) and 12c release 1 (12.1.0.2).

With Oracle ASM, database data files are stored in DATA diskgroup. Redo and archive files are in RECO diskgroup.

Reserved storage is the amount of Oracle Automatic Storage Management (Oracle ASM) storage required to maintain redundancy in the event of a disk failure. If you use the reserve storage capacity, then the system continues to run, and it is protected through Oracle ASM mirroring. However, in the event of a second disk failure, the system is then running in a non-protected and degraded mode. In this event, you must replace disks immediately.

About Oracle ACFS Database Storage

Oracle ACFS is supported with Oracle Database 12c release 2 (12.2.0.1), 12c release 1 (12.1.0.2), and Oracle Database 11g release 2 (11.2.0.4). Version 11.2.0.4 databases must use ACFS for storage.

With Oracle ACFS, an Oracle ACFS file system is created from DATA diskgroup for each database to store data files, and an Oracle ACFS file system is created from RECO diskgroup for REDO and fast recovery area for all databases.

Each database created under ACFS has it own mount points, /u02/app/oracle/oradata/dbid. When you delete a database, you should clean up the files.

About Database File Storage

Use Oracle Automatic Storage Management (Oracle ASM) or Oracle Automatic Storage Management Cluster File System (Oracle ACFS) for database files storage.

Database file systems are used exclusively for storing database files, and they include a DATA file system for database data files and a RECO file system for storing archive files and backups. Oracle Database Appliance supports Oracle Automatic Storage Management Cluster File System (Oracle ACFS) or Oracle Automatic Storage Management (Oracle ASM) database file storage. You determine the type of database storage when you create the database.

About Oracle ASM Database Storage

Use Oracle ASM with Oracle Database 12c release 2 (12.2.0.1) and 12c release 1 (12.1.0.2).

With Oracle ASM, database datafiles are stored in DATA diskgroup. Redo and archive files are in RECO diskgroup.

Reserved storage is the amount of Oracle Automatic Storage Management (Oracle ASM) storage required to maintain redundancy in the event of a disk failure. If you use the reserve storage capacity, then the system continues to run, and it is protected through Oracle ASM mirroring. However, in the event of a second disk failure, the system is then running in a non-protected and degraded mode. In this event, you must replace disks immediately.

About Oracle ACFS Database Storage

Oracle ACFS is supported with Oracle Database 12c release 2 (12.2.0.1), 12c release 1 (12.1.0.2), and Oracle Database 11g release 2 (11.2.0.4). Version 11.2.0.4 databases must use ACFS for storage.

With Oracle ACFS, an Oracle ACFS file system is created from DATA diskgroup for each database to store datafiles, and an Oracle ACFS file system is created from RECO diskgroup for redo and fast recovery area for all databases.

Each database created under ACFS has it own mount points, /u02/app/oracle/oradata/dbid. When you delete a database, you should clean up the files.

Storage Configuration Options

When Oracle Database Appliance is deployed, you can configure how the storage capacity is shared between DATA diskgroup and RECO diskgroup. You can choose anywhere from 10% to 90% for DATA and the remainder for RECO. The amount of usable storage is determined by the percentage configured for DATA. You can run the command lsdg to determine the usable storage on the DATA disk group.

Oracle ACFS Mount Points and Storage Space

Review Oracle ASM Cluster file system (ACFS) mount points for Oracle Database Appliance.

If you select Oracle Automatic Storage Management (Oracle ASM) for database storage when you create a database, then an Oracle ASM Cluster file system (ACFS) is not created. All files are in an Oracle ASM diskgroup.

If you select Oracle ACFS for database storage, then each database has its own Oracle ACFS mount point:

  • DATA diskgroup: /u02/app/oracleuser/oradata/db_name

  • RECO diskgroup: /u03/app/oracleuser.

With Oracle ACFS, the following are created:

  • A 100G ACFS is created from +DATA diskgroup for each database. This Oracle ACFS automatically extends the space on demand.

  • A common Oracle ACFS with 25% of +RECO diskgroup is created with auto extension on. This file system is used for fast recovery area and redo logs for all databases.

Table B-1 Oracle ACFS Mount Points and Related Oracle ASM Disk Groups and Volume Information

File System Oracle ASM Disk Group Oracle ASM Dynamic Volume Mount Point

DATA

+DATA

/dev/asm/datdbname-nnn

For example: /dev/asm/datodacn-123

/u02/app/oracleuser/oradata/dbname

For example: /u02/app/example/oradata/odacn

RECO

+RECO

/dev/asm/reco-nn

/u03/app/oracleuser

This mount point is shared by all databases for fast_recovery_area and redo logs.

For fast_recovery_area, the path is: /u03/app/oracleuser/fast_recovery_area/db_name

For redo logs, the path is: /u03/app/oracleuser/redo/db_name

Example B-1 Oracle ACFS Storage Space

When the Oracle ACFS file systems are created, they do not initially consume all of the storage in the appliance. Space is preserved for additional repositories, or in some cases, database files stored directly in Oracle ASM. You can check for available storage space in your file systems by running the operating system command df -k as shown in the following example.

# df -k 
Filesystem                            1K-blocks   Used         Available  Use%   Mounted on 
/dev/mapper/VolGroupSys-LogVolRoot    30963708    14203568     15187276   49%     / 
tmpfs                                 65952292      647800     65304492    1%     /dev/shm 
/dev/sda1                               495844       43872       426372   10%     /boot 
/dev/mapper/VolGroupSys-LogVolOpt     61927420    18594420     40187272   32%     /opt 
/dev/mapper/VolGroupSys-LogVolU01    103212320    49621560     48347880   51%     /u01 
/dev/asm/reco-62                      76546048     1469676     75076372    2%     /u03/app/oracle
/dev/asm/datrdb2-268                 104857600     3872368    100985232    4%     /u02/app/oracle/oradata/rdb2
/dev/asm/datndb11-268                104857600      247160    104610440    1%     /u02/app/oracle/oradata/ndb11
/dev/asm/datndb12-268                104857600      247160    104610440    1%     /u02/app/oracle/oradata/ndb12

Displaying Mounted Disk Details

Use the Oracle Automatic Storage Management lsdg command to display mounted disk groups and their information for Oracle Database Appliance.

To display information about a specific disk group, specify the disk group in the command.
  1. Log in as a grid user.
  2. Run the Oracle Automatic Storage Management lsdg command.

Example B-2 Determining Storage on the DATA Disk Group

ASMCMD [+] > lsdg data

State    Type    Rebal  Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB
MOUNTED  NORMAL  N         512   4096  4194304     12288     8835             1117            3859

(continued)
Offline_disks  Voting_files  Name
            0             N  DATA

Usable Space on Oracle Database Appliance X7-2S and X7-2M

Review the table for the approximate amount of usable space for Oracle Database Appliance X7-2S and X7-2M.

Oracle Database Appliance X7-2S and X7-2M use 6.4TB NVMe. The usable data capacity varies because it is derived by converting disk hardware terabytes (based on 1 kilobyte equals 1,000 bytes) into software storage terabytes (based on 1 kilobyte equals 1,024 bytes) and splitting the usable capacity into Oracle Automatic Storage Management (Oracle ASM) disk groups.

The estimated usable space is calculated based on the number of drives, where 90% is allocated to data.

Table B-2 Usable Disk Capacity on Oracle Database Appliance X7-2S and X7-2M

Number of Drives (90% allocated to data) Normal Redundancy High Redundancy

X7-2S

2 drives (12.8TB raw storage)

  • DATA: 5.24 TB

  • RECO: 0.58 TB

Not applicable

X7-2M

2 drives (12.8TB raw storage)

  • DATA: 5.24 TB

  • RECO: 0.58 TB

Not applicable

X7-2M

5 drives (32TB raw storage)

  • DATA: 10.48 TB

  • RECO: 1.16 TB

  • Reservation: 2.91 TB

  • DATA: 5.24 TB

  • RECO: 0.58 TB

  • Reservation: 5.82 TB

X7-2M

8 drives (51.2TB raw storage)

  • DATA: 18.34 TB

  • RECO: 2.04 TB

  • Reservation: 2.91 TB

  • DATA: 10.48 TB

  • RECO: 1.16 TB

  • Reservation: 5.82 TB

The Reservation value represents the amount of storage required to maintain full redundancy in case of disk failure.

Usable Space on Oracle Database Appliance X7-2-HA

Review the table for the approximate amount of usable space for high performance or high capacity on Oracle Database Appliance X7-2-HA.

Usable Space for High Performance

Oracle Database Appliance X7-2-HA use 3.2 TB solid state drives (SSDs). The usable data capacity varies because it is derived by converting disk hardware terabytes (based on 1 kilobyte equals 1,000 bytes) into software storage terabytes (based on 1 kilobyte equals 1,024 bytes) and splitting the usable capacity into Oracle Automatic Storage Management (Oracle ASM) disk groups.

The estimated usable space is calculated based on the number of drives, where 90% is allocated to data.

Table B-3 Usable Disk Capacity for High Performance on Oracle Database Appliance X7-2-HA

Number of Drives (90% allocated to data) Normal Redundancy High Redundancy

5 drives (16 TB raw storage)

  • DATA: 5.24 TB

  • RECO: 0.58 TB

  • REDO: 0.97 TB

  • Reservation: 0.97 TB

  • DATA: 2.62 TB

  • RECO: 0.29 TB

  • REDO: 0.97 TB

  • Reservation: 1.94 TB

10 drives (32 TB raw storage)

  • DATA: 11.79 TB

  • RECO: 1.31 TB

  • REDO: 0.97 TB

  • Reservation: 0.97 TB

  • DATA: 6.98 TB

  • RECO: 0.78 TB

  • REDO: 0.97 TB

  • Reservation: 1.94 TB

15 drives (48 TB raw storage)

  • DATA: 18.34 TB

  • RECO: 2.04 TB

  • REDO: 0.97 TB

  • Reservation: 0.97 TB

  • DATA: 11.35 TB

  • RECO: 1.26 TB

  • REDO: 0.97 TB

  • Reservation: 1.94 TB

20 drives (64 TB raw storage)

  • DATA: 24.88 TB

  • RECO: 2.76 TB

  • REDO: 0.97 TB

  • Reservation: 0.97 TB

  • DATA: 15.72 TB

  • RECO: 1.75 TB

  • REDO: 0.97 TB

  • Reservation: 1.94 TB

40 drives (128 TB raw storage)

  • DATA: 51.08 TB

  • RECO: 5.68 TB

  • REDO: 1.94 TB

  • Reservation: 0.97 TB

  • DATA: 33.18 TB

  • RECO: 3.69 TB

  • REDO: 1.94 TB

  • Reservation: 1.94 TB

The Reservation value represents the amount of storage required to maintain full redundancy in case of disk failure.

The REDO disk group uses 800 GB raw Solid-State Drives or 0.8 TB total usable space. The REDO disk group uses high redundancy that provides approximately 0.3 TB usable space for database redo logs.

Note:

For an Oracle Database Appliance virtualized platform:

  • Nn-CDB databases, the REDO disk group has 50% free disk space.

  • CDB databases, an Oracle ASM Cluster file system (ACFS) mount point is created per CDB database, based on the template log file size. If this is a Single Instance (SI) database, then multiply by 3. If the database is an Oracle RAC database, then multiply by 4.

Usable Space for High Capacity

The usable space for high capacity assumes that 90% is allocated to data and that you use external backup.

Table B-4 Usable Disk Capacity for High Capacity on Oracle Database Appliance X7-2-HA

Number of Drives (90% allocated to data) Normal Redundancy High Redundancy

15 drives (150 TB raw storage)

  • DATA: 57.30 TB

  • RECO: 6.37 TB

  • REDO: 0.97 TB

  • Flash: 5.52 TB

  • Reservation: 4.5 TB

  • DATA: 35.47 TB

  • RECO: 3.94 TB

  • REDO: 0.97 TB

  • Flash: 2.61 TB

  • Reservation: 9.0 TB

30 drives (300 TB raw storage)

  • DATA: 118.69 TB

  • RECO: 13.19 TB

  • REDO: 1.94 TB

  • Flash: 12.50 TB

  • Reservation: 4.5 TB

  • DATA: 76.40 TB

  • RECO: 8.49 TB

  • REDO: 1.94 TB

  • Flash: 7.16 TB

  • Reservation: 9.0 TB

Usable Free Space with Oracle ASM

When Oracle ASM calculates usable Free Space, it determines the amount of space to reserve in the case of a disk failure.

For Oracle Database Appliance with 2 or 4 NVMe drives, this reserved space is not required. When you query Oracle ASM or Oracle Database Appliance commands to view the amount of storage available, the usable_File_MB value may report a negative number.

Table B-5 Oracle ASM Calculations

Number of Drives Redundancy Total_MB Free_MB Req_mir_free_MB Usable_file_MB Name

2

NORMAL

4894016

4893372

0

1220644

RECO/

4

NORMAL

1231176

1230996

610468

305150

RECO/

Note:

Note: 1TB = MB divided by 10242

How Oracle ASM and Oracle Database Appliance Define Capacity

The following table describes how capacity terms are defined by Oracle ASM and Oracle Database Appliance.

Table B-6 Definition of Terminology

Term Oracle ASM Definition Oracle Database Appliance Definition

Total_MB

Size of the disk group in MB

Total usable storage. For example, for 2 NVMe drives, total usable storage is 5.8TB.

Free_MB

Free space in the disk group in MB, without regard to redundancy.

Total usable storage after formatting to Oracle ASM disk groups. For example, for 2 NVMe drives, total usable storage is 5.8TB.

Req_mir_free_MB

Amount of space that must be available in the disk group to restore full redundancy after the worst failure that can be tolerated by the disk group.

Total usable storage after formatting to Oracle ASM disk groups. For example, for 2 NVMe drives, total usable storage is 5.8TB.

Usable_file_MB

Amount of free space, adjusted for mirroring, that is available for new files.

Total usable space taking into consideration the mirroring level. Oracle ASM also calculates the amount of space required.