Best Practices for Using Storage Software Appliance

Topics

Best Practices – Configuring Cache Storage

Oracle Cloud Infrastructure Storage Software Appliance uses local storage attached to the server (or virtual server) for hosting the filesystems and cache. Files written to a filesystem in the appliance are uploaded to the associated container or bucket, with a portion of the file set maintained locally in the filesystem as a warm cache.

For optimal performance, reliability, and fault tolerance, consider the following guidelines when configuring the local appliance storage:

Allocate a dedicated volume for the appliance filesystem and metadata.
Multiple disks (hard disk drives or solid state drives) in a RAID10 set provide an optimal balance of performance, reliability, and fault tolerance. Alternatively, RAID6 may be used. Avoid RAID0 or single disk (no RAID) due to the potential for data loss due to disk failure
Enable read-ahead on the volume.
Provision a volume that can accommodate the local cache and ingest new files (upload buffer) without ever becoming more than 80% full.
A general guideline is to use a volume that is at least 1.5 times the size of the data set that you want to hold in local cache. For example, if the expected size of the entire file set is 50 TB and if 10% (5 TB) of that file set will be accessed frequently, then the cache storage volume should have at least 7.5 TB of usable capacity.
Note:
- If the cache size reaches a near-full threshold, any data ingest will result in out of space error
  in the appliance.

Best Practices – Determining the Cache Size

The local cache of Oracle Cloud Infrastructure Storage Software Appliance serves two roles: ingest cache (upload/write buffer) and read cache. You can specify the maximum size for the read cache. The write buffer will use any remaining available space on the local storage volume and does not have a cache size setting.

The maximum size of the write buffer is an important criterion to determine the cache size. The write buffer size increases when data is uploaded in the appliance. And the write buffer size decreases after the data is transferred to cloud. Write buffer cannot be removed from local cache. When the write buffer uses all the available local cache space, any data ingest will result in out of space error in the appliance.

Use the following guidelines to determine the appropriate setting for the ingest cache:

Identify the amount of data to be uploaded in the appliance. If a large amount of data must be uploaded, the appliance write buffer may reach its maximum. This will lead to I/O failure as the local cache has no space. If the data transfer can be regulated, for example, by pausing after a certain amount of data is transferred or allow the uploads to complete periodically, the local cache space can be increased and I/O failure can be avoided. You can also follow this approach for backup/cron jobs if the local cache space is lesser than the amount of data to be uploaded.
Calculate the amount of data that would be uploaded on any typical day or a week in the appliance. Also, calculate the amount of data that can be uploaded over a time period, based on the available bandwidth or historical data. The difference between former and latter data quantities should not exceed the write buffer size.
If the application can handle I/O failure and resume the data transfer, set the write buffer size with the amount of data that you’d like to upload before the cache size decreases.

Configuring the read cache size is necessary only when the appliance must retrieve a significant amount of data from the cloud. Alternatively, you can preserve frequently accessed files in the local cache. Setting aside a large portion of the local cache for read cache might impact the amount of data that can be uploaded before the cache size decreases. Configuring the read cache is optional and depends on the appliance workload.

Use the following guidelines to determine the appropriate setting for the read cache:

The default limit of the read cache size is the lower of 300 GB or the storage volume size.
Do not set the read cache maximum to the size of the local storage volume. Doing so would allocate 100% of the volume for read cache and would not leave available capacity for ingest. If there is no available space for new file ingest, then the appliance might stop the data ingest and begin evicting files from the read cache to create space. This severely degrades ingest performance.
Start with a read cache setting that is 50% of the size of the local storage volume (leaving 50% for ingest). Monitor the available capacity on the local storage volume over time, especially after periods of very high or sustained ingest activity. If the available capacity remains above 30% consistently, consider increasing the read cache size. If the available capacity is consistently below 20%, then consider decreasing the read cache size.
The general strategy is to set the read cache size to equal the amount of data that you anticipate to be accessed frequently, while leaving enough capacity on the volume for the ingest cache (write buffer).

After you size the cache, you can choose to configure the read cache either while creating the filesystem or later. See Adding a FileSystem and Changing the Properties of a FileSystem.

Best Practices – Encrypting Data Using Keys

You can provide your own RSA asymmetric keys if you’ve enabled encryption for a filesystem. The symmetric key converts the data to a readable form called cleartext. If you lose the keys, you lose the data.

Asymmetric keys: There’s a single key pair for every instance of the appliance. The same key pair is used to encrypt information related to local configuration. If you provide an asymmetric key pair, then the key pair is used to encrypt or decrypt the specified filesystem database configuration items. Ensure that the asymmetric keys are backed up.

Symmetric keys: The symmetric key is stored within the local filesystem database. Each filesystem can have its own unique symmetric encryption key. The symmetric key is encrypted using the asymmetric key that’s stored locally on the disk.

At any time, you can download a tar.gz file containing the details of all the keys stored on the disk.

Key rotation enables data recovery if the appliance fails at any time.

Rotating Keys in the Appliance

Log in to the management console and select the filesystem.
Provide your asymmetric key pair. The appliance ensures that the keys are valid by encrypting and decrypting randomly generated sample data.
If the keys are valid, then they are saved in a temporary location on the disk. The old keys are moved to a backup location on the disk.
The database’s encrypted configuration items are encrypted again in the filesystem by using the asymmetric key pair.
The new keys are saved to a permanent location on the disk.
Download the compressed key archive of the encryption keys. The compressed archive includes the new key details as well as the backup keys.

Best Practices – Scalability Recommendations

Ensure that the number of objects stored in an appliance filesystem doesn’t exceed 10 million (10000000). For data sets that consist of more than 10 million objects, ensure that the objects are distributed across multiple filesystems.
The minimum amount of memory required for any appliance filesystem is 16 GB.
- For filesystems with the number of files up to 5 million, the required amount of memory is 32 GB.
- For large filesystems with the number of files up to 10 million, the required amount of memory is 64 GB
To improve the efficiency of file ingest and cloud upload operations, and to reduce the number of objects in the namespace, bin-pack or zip small files before writing them to the appliance.
Multiple filesystems can be created on a single appliance. However, for optimal performance, ensure that each filesystem is hosted on a dedicated appliance.

Best Practices – Recommended Workloads and Use Cases

Oracle Cloud Infrastructure Storage Software Appliance is an effective cloud gateway for many workloads. Use the following guidelines to determine whether the appliance is appropriate for your specific use cases and workloads:

The appliance supports NFSv4 in asynchronous mode and POSIX Sync mode. The POSIX Sync mode is enabled in the appliance by default.
In the asynchronous mode, there is scope for data loss in the event of a sudden server failure. Avoid using the appliance for workloads and use cases that require synchronous write behavior.
The appliance is ideal for backup and archive use cases that require the replication of infrequently accessed data to cloud containers.
Carefully consider use cases that involve frequent changes to existing files. Each time a file is modified and closed, the appliance creates a new version of the file, which is then uploaded to the container or bucket in your service instance, replacing the previous version. The appliance will be less efficient and may not perform optimally for this type of workload.
Don't run applications and executables directly from the appliance mount points, particularly if the appliance cache is not large enough for all the files that the applications will access. Applications typically create temporary files and modify them often, affecting the operational efficiency of the appliance.