Review this section to understand Oracle Database Appliance storage architecture and options and how to determine usable storage.
Topics:
Oracle Database Appliance uses the Oracle Automatic Storage Management Cluster File System (Oracle ACFS) for storage of database and virtual machine files.
Oracle ACFS provides both servers with concurrent access to some or all of the shared storage on Oracle Database Appliance. Oracle ACFS supports space-efficient storage snapshots, which provides fast provisioning databases and virtual machines within Oracle Database Appliance.
Storage Systems Used With Oracle Database Appliance
Three types of Oracle ACFS file systems are used in Oracle Database Appliance:
Database File Systems
Shared repositories
General-purpose storage
About Database File Systems
Database file systems are used exclusively for storing database files, and they include a FLASH file system for storing database data files and flash cache files, a DATA file system for database data files, a RECO file system for storing archive files and backups, and a REDO file system for storing redo log files.
About Shared Repositories
Shared repositories are file systems created on Oracle Database Appliance Virtualized Platform, and they are used to store virtual machine templates, runtime images, and virtual disks.
About General Purpose Storage
Every Oracle Database Appliance has a general-purpose cluster file system created by default That cluster file system is named cloudfs. You can use the cloudfs file system for general-purpose storage of files that must be shared between the servers. For example, you can use the cloudfs file system for staging data loads.
All Oracle ACFS file systems are created on Oracle ASM Dynamic Volumes provisioned from disk groups that are created in the shared disk storage pool. In a bare-metal deployment, these file systems are mounted directly in the operating system hosting the databases. In a virtualized deployment, these file systems are managed and mounted directly in ODA_BASE.
lsdg command to display mounted disk groups and their information for Oracle Database Appliance.Review Oracle ASM Cluster file system (ACFS) mount points for Oracle Database Appliance.
The storage shelf in the base Oracle Database Appliance X6-2-HA system is half populated with 10 solid-state drives (SSDs) for data storage, totaling 12TB of raw storage capacity. The storage shelf in the base system also has four 200 GB high endurance SSDs for database redo logs to improve performance and reliability. A separate ACFS file system is created from the REDO diskgroup for redo log files. Add 10 more SSDs on the base system for a total of 20 SSDs and 24 TB of raw storage capacity.
If you need additional storage, an optional storage expansion shelf is available. With the optional storage expansion shelf, the raw data storage capacity increases to a total of 48TB. The expansion shelf contains four 200 GB SSDs to expand the storage capacity for the database REDO logs. In addition, you can use external NFS storage outside of the appliance for online backups, data staging, or additional database files.
Table B-1 Oracle ACFS Mount Points and Related Oracle ASM Disk Groups and Volume Information
| File System | Oracle ASM Disk Group | Oracle ASM Dynamic Volume | Mount Point |
|---|---|---|---|
|
DATA |
+DATA |
For example: |
For example: |
|
RECO |
+RECO |
/dev/asm/reco-nn |
This mount point is shared by all databases for For For redo logs, the path is: |
Example B-1 Oracle ACFS Storage Space
When the Oracle ACFS file systems are created, they do not initially consume all of the storage in the appliance. Space is preserved for additional repositories, or in some cases, database files stored directly in Oracle ASM. You can check for available storage space in your file systems by running the operating system command df -k as shown in the following example.
# df -k Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/VolGroupSys-LogVolRoot 30963708 14203568 15187276 49% / tmpfs 65952292 647800 65304492 1% /dev/shm /dev/sda1 495844 43872 426372 10% /boot /dev/mapper/VolGroupSys-LogVolOpt 61927420 18594420 40187272 32% /opt /dev/mapper/VolGroupSys-LogVolU01 103212320 49621560 48347880 51% /u01 /dev/asm/reco-62 76546048 1469676 75076372 2% /u03/app/oracle /dev/asm/datrdb2-268 104857600 3872368 100985232 4% /u02/app/oracle/oradata/rdb2 /dev/asm/datndb11-268 104857600 247160 104610440 1% /u02/app/oracle/oradata/ndb11 /dev/asm/datndb12-268 104857600 247160 104610440 1% /u02/app/oracle/oradata/ndb12
Use the Oracle Automatic Storage Management lsdg command to display mounted disk groups and their information for Oracle Database Appliance.
grid user.lsdg command.Example B-2 Determining Storage on the DATA Disk Group
ASMCMD [+] > lsdg data
State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB
MOUNTED NORMAL N 512 4096 4194304 12288 8835 1117 3859
(continued)
Offline_disks Voting_files Name
0 N DATA
Review the usable disk capacity available for Oracle Database Appliance X6-2-HA and how capacity is derived.
Oracle Database Appliance X6-2-HA use 1.6TB raw Solid-State Drives, formatted to 1.2TB for performance. The usable data capacity varies because it is derived by converting disk hardware terabytes (based on 1 kilobyte equals 1,000 bytes) into software storage terabytes (based on 1 kilobyte equals 1,024 bytes) and splitting the usable capacity into Oracle Automatic Storage Management (Oracle ASM) disk groups.
Each of the SSD usable storage is approximately 1.1TB. This is calculated by the storage usable capacity of the drive converted to TB (1.2 TB divided by 1.0244= 1.1TB.)
The following tables provide the approximate amount of usable space for the Oracle Database Appliance X6-2-HA for the DATA and RECO disk groups.
Usable Disk Capacity on Oracle Database Appliance X6-2-HA
The following table provides the approximate amount of usable space for the Oracle Database Appliance X6-2-HA.
Table B-2 Usable Disk Capacity on Oracle Database Appliance X6-2-HA
| Description | Sizing for X6-2-HA with 10 SSD Drives | Sizing for X6-2-HA with 20 SSD Drives | Sizing for X6-2-HA with 40 SSD Drives |
|---|---|---|---|
| Number of SSD Drives |
10 |
20 |
40 |
|
Total usable Space |
10.9TB |
22TB |
44TB |
|
Reserved Space Normal Redundancy |
1.1TB |
1.1TB |
1.1TB |
|
Total Usable Normal (Double Mirror) Oracle ASM Redundancy |
4.9TB |
10.5TB |
21.5TB |
|
DATA Disk Group Normal Redundancy, External Backup (80% usable) |
3.9TB |
8.4TB |
17.2TB |
|
RECO Disk Group Normal Redundancy, External Backup (20% usable) |
1.0TB |
2.1TB |
4.3TB |
|
DATA Disk Group Normal Redundancy, Internal Backup (40% Usable) |
2.0TB |
4.2TB |
8.6TB |
|
RECO Disk Group Normal Redundancy, Internal Backup (60% usable) |
2.9TB |
6.3TB |
12.9TB |
|
Reserved Space High Redundancy |
2.2TB |
2.2TB |
2.2TB |
|
Total usable High (Triple Mirror) Oracle ASM Redundancy |
2.9TB |
6.6TB |
13.9TB |
|
DATA Disk Group High Redundancy, External Backup(80% usable) |
2.3TB |
5.3TB |
11.1TB |
|
RECO Disk Group High Redundancy, External Backup (20% usable) |
0.6TB |
1.3TB |
2.8TB |
|
DATA Disk Group High Redundancy, Internal Backup(40% usable) |
1.2TB |
2.6TB |
5.6TB |
|
RECO Disk Group (10% usable) |
1.7TB |
4.0TB |
8.3TB |
The Reserved Space values represent the amount of storage required to maintain full redundancy in case of disk failure.
The REDO disk group uses 800 GB raw Solid-State Drives or 0.8TB total usable space. The REDO disk group uses high redundancy that provides approximately 0.3TB usable space for database redo logs.
Note:
For non-CDB databases, the REDO disk group has 50% free disk space.
For CDB databases, an Oracle ASM Cluster file system (ACFS) mount point is created per CDB database, based on the template log file size. If this is a Single Instance (SI) database, then multiply by 3. If the database is an Oracle RAC database, then multiply by 4.
Oracle ASM Calculations
When Oracle ASM calculates Usable Free Space, it determines the amount of space to reserve in the case of a disk failure. When you query Oracle ASM or Oracle Database Appliance commands to view the amount of storage available, the usable_File_MB value may report a negative number.
Table B-3 Oracle ASM Calculations
| Number of Drives | Redundancy | Total_MB | Free_MB | Req_mir_free_MB | Usable_file_MB | Name |
|---|---|---|---|---|---|---|
|
2 |
NORMAL |
4894016 |
4893372 |
2441888 |
1220644 |
DATA/ |
|
4 |
NORMAL |
1231176 |
1230996 |
610468 |
305150 |
RECO/ |
Note:
Note: 1TB = MB divided by 10242The following table describes how capacity terms are defined by Oracle ASM and Oracle Database Appliance.
Table B-4 Definition of Terminology
| Term | Oracle ASM Definition | Oracle Database Appliance Definition |
|---|---|---|
Total_MB |
Size of the disk group in MB |
Total usable storage. For example, for 2 NVMe drives, total usable storage is 5.8TB. |
Free_MB |
Free space in the disk group in MB, without regard to redundancy. |
Total usable storage after formatting to Oracle ASM disk groups. For example, for 2 NVMe drives, total usable storage is 5.8TB. |
Req_mir_free_MB |
Amount of space that must be available in the disk group to restore full redundancy after the worst failure that can be tolerated by the disk group. |
Usable space required to restore full redundancy. When storage is consumed higher than the |
Usable_file_MB |
Amount of free space, adjusted for mirroring, that is available for new files. |
Total usable space taking into consideration the mirroring level. Oracle ASM also calculates the amount of space required |
The Oracle ACFS file systems are automatically created when you create a database on Oracle Database Appliance.
When you use Oracle ACFS for database storage, the following is created:
A 100G ACFS is created from +DATA diskgroup for each database. This Oracle ACFS automatically extends the space on demand.
A common Oracle ACFS with 25% of +RECO diskgroup is created with auto extension on. This file system is used for fast recovery area and redo logs for all databases.
When the Oracle ACFS file systems are created, they do not initially consume all of the storage in the appliance. Space is preserved for additional repositories, or in some cases, database files stored directly in Oracle ASM. You can check for available storage space in your file systems by running the operating system command df -k as shown in the following example.
# df -k Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/VolGroupSys-LogVolRoot 30963708 14203568 15187276 49% / tmpfs 65952292 647800 65304492 1% /dev/shm /dev/sda1 495844 43872 426372 10% /boot /dev/mapper/VolGroupSys-LogVolOpt 61927420 18594420 40187272 32% /opt /dev/mapper/VolGroupSys-LogVolU01 103212320 49621560 48347880 51% /u01 /dev/asm/reco-62 76546048 1469676 75076372 2% /u03/app/oracle /dev/asm/datrdb2-268 104857600 3872368 100985232 4% /u02/app/oracle/oradata/rdb2 /dev/asm/datndb11-268 104857600 247160 104610440 1% /u02/app/oracle/oradata/ndb11 /dev/asm/datndb12-268 104857600 247160 104610440 1% /u02/app/oracle/oradata/ndb12