Learn About the Shared Storage Options for Oracle Cloud at Customer

In Oracle Cloud at Customer, block storage volumes can’t be shared across instances. You also can’t easily provision a floating IP address that can be switched to another instance, if the instance that the block volumes are attached to becomes unavailable. Due to these limitations, you can’t implement many of the commonly-used shared storage options, such as Red Hat Global File System (GFS), Oracle Cluster File System (OCFS2), and so on. You do, however, have a few options to consider if you want to set up a shared storage solution in Oracle Cloud at Customer.

Network File System

One of the key considerations for shared storage is, if a hardware failure occurs, can you tolerate a brief outage. If you can, then you should consider setting up a Network File System (NFS) server on one of your compute instances. If the compute instance becomes unavailable due to a hardware issue, then the instance is re-created automatically. This process can take a few minutes. If that’s acceptable, then you can use this simple approach to share storage in Oracle Cloud at Customer.

An improvement over the single NFS server approach is to set up two NFS servers with failover. While this approach would reduce the time taken for your storage server to become available after an outage, it won't entirely eliminate it.

Oracle Database Exadata Cloud Service

Another option is to use an existing Oracle Database Exadata Cloud at Customer rack. If you have an existing rack with spare capacity, then you can use Oracle Automatic Storage Management Cluster File System (ACFS) to implement shared storage.

Gluster

If you don’t have spare capacity in an existing Oracle Database Exadata Cloud at Customer rack, but you can’t tolerate an outage in your shared storage availability, then you can consider shared storage solutions that don’t require shared block storage, such as Gluster.

About Network File System

Network File System (NFS) is a client/server application that lets you interact with files on a remote computer as though they are locally mounted. NFS is supported on Oracle Linux and on other UNIX-like operating systems. Using NFS allows you to consolidate your storage resources and make them available to all your compute instances from a dedicated file server instance.

Using NFS in Oracle Cloud at Customer

When you set up an instance to serve as an NFS server, you attach the block volumes to the instance during instance creation. After you install the NFS server on the instance, you can mount all or part of the file system on the NFS server. NFS clients can access that part of the file system that is mounted.

NFS clients use the mount command to access the shared file system. Files and directories are mounted on the client and you can use them as if they were locally mounted. You can also ensure that a client mounts the specified NFS shares automatically whenever it starts by updating the /etc/fstab file. Clients’ access to shared files depends on whether the files are designated as read-only or read-write. You can use NFS to centralize your storage provisioning and help improve data consistency and reliability.

Oracle Linux 7 supports NFSv3 and NFSv4.

To use NFS, you’d first set up an instance as an NFS server. You can install the nfs-utils package by using yum. Next, you specify the files that you want to make available for clients to mount and start the NFS server. You must also ensure that you allow access through the firewall for NFS clients.

On the instances that need access to the shared files, set up the NFS client. You can use the showmount command to discover the file systems that the NFS server exports. Then use the mount command to mount an exported file system on an available mount point. To make a mount persistent, add an entry to the /etc/fstab file.

Why NFS might not work for you

Although setting up an NFS server can allow you to quickly and easily get started sharing files across compute instances, it doesn’t provide built-in resilience. For example, if the instance on which your NFS server is running becomes unreachable for any reason, you won’t have access to the shared file system. The Compute service can detect if an instance fails and it can automatically re-create the instance and attach the storage volumes to the instance. However, this process can take a few minutes, during which time your file system is unavailable.

One way to guard against this kind of outage is to set up two NFS servers with data replication by using the rsync utility.

You can use the rsync utility to seamlessly move files from one server to another. With the rsync utility, you can efficiently synchronize the files and directories from one location to another. This utility compares the files in the source and the destination and transfers only the data that has changed. A benefit of using the rsync utility is that mirroring occurs with only a single transmission in each direction.

With a second NFS server in place and with data replicated by using the rsync utility, if the instance that hosts the primary NFS server becomes unreachable, you can quickly switch to the backup instance. For this to work, you’d have to ensure that the IP address that is associated with the NFS server instance can be quickly and easily switched to the backup instance.

Using multiple NFS servers helps to reduce, but doesn’t eliminate, the time that your shared storage is unavailable in the event of an outage. It also doesn’t guarantee zero data loss.

Another limitation with this approach is the volume of data that you can make available on a single NFS server is limited, because the number and size of block volumes that can be attached to a compute instance is limited. So this approach isn’t ideal if you require very high volumes of shared storage.

About Gluster File System

Gluster is an open source, distributed shared file system. 

Using Gluster

Gluster doesn’t depend on shared block storage, because it uses a replication model to make files highly available from multiple servers. You can scale up to a large number of servers, with multiple block volumes attached to each server. Gluster manages the replication of files across servers in the cluster as applicable, and ensures that data is synchronized. Gluster also manages failover from one server to another in case of an outage. When you set up a Gluster client, the client is aware of every server in the cluster. If any server becomes unavailable, the client automatically switches to another server. Because Gluster manages this switch, you don’t require a floating IP address that must be reassigned from one file server to another.

Why Gluster might not work for you

The Gluster server isn’t supported on Oracle Linux. This lack of support doesn’t mean that you can’t install the Gluster server on an Oracle Linux instance. You can install the Gluster server on an Oracle Linux instance from the CentOS YUM repository. Alternatively, for the Gluster servers, you can set up CentOS instances. Note that you won’t be entitled to Oracle Linux Support for Gluster even when Gluster is running on an Oracle Linux instance.

Although Gluster servers require CentOS, the Gluster client can be installed on Oracle Linux instances seamlessly.

About Oracle Automatic Storage Management Cluster File System

If you have an Oracle Database Exadata Cloud Service subscription, you can consider using Oracle Automatic Storage Management Cluster File System to share the files stored in your Exadata Storage Servers. Oracle Database Exadata Cloud Service enables you to leverage the power of Exadata in a service-based consumption model. You have full access to the features and operations that are available with Oracle Database, but with Oracle owning and managing the Exadata infrastructure. Each Oracle Database Exadata Cloud Service instance contains a predefined number of compute nodes (database servers) and a predefined number of Exadata Storage Servers, all tied together by a high-speed, low-latency InfiniBand network and intelligent Exadata Storage Server Software.

Oracle Automatic Storage Management Cluster File System is an industry standard, POSIX, X/OPEN, and Windows compliant cluster file system that supports multiple operating systems and server platforms. It includes features such as file system snapshot, role reversal replication, compression, tagging, security, encryption, auditing, file system freezing and thawing, and highly available Network File System (NFS) and Server Message Block (SMB) services. Oracle Automatic Storage Management Cluster File System is integrated with Oracle Automatic Storage Management, extending its functionality to support general-purpose files as well as database files.

Using Oracle Automatic Storage Management Cluster File System

When Oracle Automatic Storage Management is set up in an Oracle Database Exadata Cloud Service deployment, the storage in Oracle Database Exadata Cloud Service is configured with five Oracle Automatic Storage Management disk groups: DATA, RECO, DBFS, ACFS1 and ACFS2. These file systems are mounted on the compute nodes. The volumes that belong to Oracle Automatic Storage Management Cluster File System reside on the Oracle Automatic Storage Management Cluster File System disk group. One of these file systems can be used as a staging area for files that need to be loaded into the Oracle database in the Oracle Database Exadata Cloud Service instance. These could be files that are linked to external tables, Oracle Data Pump files, and so on. The second file system is used to store software binaries. If additional staging space is needed, file systems can be created on one of the bigger disk groups for the duration that they are needed and then dropped to release space. If the staging area volume is not of sufficient size, a new volume can be created that uses the DATA disk group. However, this will reduce the space available to your database.

When you use Oracle Automatic Storage Management Cluster File System in an Oracle Database Exadata Cloud Service instance to provide shared storage, you don’t have to worry about switching the IP address for the file storage server in case the storage server becomes unreachable and you have to switch to a failover server. By default, a floating IP address is available as part of this solution.

Typical use case for Oracle Automatic Storage Management Cluster File System in Oracle Database Exadata Cloud Service

Using your Oracle Database Exadata Cloud Service instance to provide shared storage works best when you already have an Oracle Database Exadata Cloud Service subscription and you have sufficient excess capacity that you can use to provide shared storage. The expense of an Oracle Database Exadata Cloud Service subscription is unlikely to be justified if this service is used exclusively to provide a shared storage solution in the cloud.

Select Your Shared Storage Option

Use this decision tree to select the shared storage option that best fits your requirements.

Description of the illustration select_shared_storage_ocicc.png