File System Tiering with Object Storage
File system tiering automatically archives cold data to Object Storage from a Lustre file system, based on policies that determine which files are eligible to be moved and when. With this automated archiving, File system tiering improves capacity utilization for more expensive Lustre storage tiers by moving cold data to an Object Storage tier.
After data is archived to Object Storage and released from Lustre, those files remain visible in the file system. If needed, it's easy to return those files back to Lustre: when an application reads released data, the data is automatically restored to Lustre.
During archiving, file data is copied to the Object Storage Standard tier. Other Object Storage tiers aren't supported.
Enabling Tiering
You enable tiering when creating a file system or updating an existing one. When you enable tiering, you select either the default tiering policy or define your own custom policy. See File System Tiering Policies.
Unlike Object copy, File system tiering doesn't require the creation of links to Object Storage buckets. Instead, when you enable tiering, File Storage with Lustre creates the associated buckets and also manages their deletion when you later delete the file system.
Before enabling tiering: Ensure that access-time updates are enabled on clients that mount the file system. File system tiering uses each file's last access time (atime) to determine whether it meets the policy's archiving criteria. For example, a policy can archive files that haven't been accessed for seven days. Access-time updates are enabled by default. Don't use the noatime mount option, because disabling these updates can prevent tiering policies from accurately identifying inactive files.
Disabling Tiering
Disabling tiering stops policy-driven archiving. It does not, however, remove data already stored in tiering buckets, and Object Storage usage charges continue. See Storage Costs.
While tiering is disabled:
- You can still view and edit the file system's tiering policy configuration. If you make changes, the updated policy takes effect when you enable tiering again.
- You can still access released files, which restores their data to Lustre.
- You can still delete files as needed.
Accessing Released Files
Released data is file data that has been archived to Object Storage and removed from local Lustre storage to free space. Files remain visible in the Lustre file system and data is automatically restored when accessed (read).
When accessing released files:
- Ensure that the file system has enough free space for the data being restored.
A restore can fail with
ENODATAif the file system lacks space. - Before you can read released data, it must be restored to Lustre. This means that reading released data can take longer than reading data already stored on Lustre.
- Before starting a task that uses released files, read those files to give their data time to restore to Lustre.
For example, if a training job needs released files, reading those files beforehand can reduce the time the job spends waiting for restoration.
Many simultaneous restores can increase read latency and affect other workloads. For existing capacity monitoring guidance, see Monitoring File System Usage.
Storage Costs
You're charged for Object Storage usage for tiered data, even when tiering is disabled.
When you delete the file system, the service cleans up the associated tiering buckets and charges for tiered storage stop.
Objects exported through separate Object Storage links using Object copy remain in your buckets and continue to incur applicable Object Storage charges.
Data Encryption
By default, tiered data is protected by Oracle-managed encryption keys. If you use a customer-managed encryption key, configure the required key permissions for Object Storage and File Storage with Lustre. See File Storage with Lustre Policies.
If you use a customer-managed encryption key and later change it, existing tiered data must be re-encrypted. The file system remains in the UPDATING lifecycle state until re-encryption completes.