File System Tiering Policies

A tiering policy contains rules that can archive files based on time since last access and specific directory paths. File Storage with Lustre provides you with a default tiering policy, or you can create your own custom tiering policies.

  • Use the default tiering policy to use the generic archiving and release rule. This rule applies to all files greater than 1 MB in your Lustre file system.
  • Create a custom tiering policy with its own rules if you want greater control over which file paths are archived and when their local data is released.
Each rule in a tiering policy includes an action:
  • An ARCHIVE rule defines eligibility for archiving and release.
  • A SKIP rule excludes matching paths from policy-driven archiving.

You configure the tiering policy you want when you create a file system or update an existing one.

Default Policy

The default policy applies to all files in the file system. Its rule uses the following values:

  • Action: ARCHIVE
  • Minimum file size: 1,048,576 bytes (1 MiB)
  • Archive eligibility: 30 days since last access
  • Release eligibility: 31 days since last access

Lustre checks files against these criteria in the background. Meeting a threshold makes a file eligible for archiving or release. Lustre then schedules eligible files for processing. This means that the threshold itself doesn't indicate when the operation completes.

The default policy's rules table is read-only. To customize rules, switch to a custom policy.

Custom Policies and Rules

To control tiering for specific file paths, use a custom policy.

Each file system's custom tiering policy must contain at least one rule and supports up to 50 custom rules. Across all rules in that policy, you can specify up to 100 unique path prefixes.

Each rule must include at least one path prefix. A path prefix identifies the location the rule applies to, such as /projects. When evaluating files for archiving, Lustre applies only the rule whose specified path prefix most closely matches the file's path.

All prefixes count toward the policy's shared limit of 100 unique prefixes. You can put all 100 prefixes in one rule or distribute them across multiple rules, provided that the policy stays within both limits. For example, one rule with 100 prefixes is valid, as are 50 rules with two prefixes each. The 100-prefix limit doesn't permit 100 rules.

The supported configuration fields are:

  • Action: ARCHIVE or SKIP
  • Archive after days since last access: Archive-age criterion
  • Release after days since last access: Release-age criterion
  • Minimum file size: Minimum size criterion for archiving (at least 1 MB)
  • Minimum file unit: Select the unit, such as MB
  • Path prefix: One or more paths to which the rule applies. Unique prefixes count toward the policy-wide total of 100.