How HSM Works

HSM automatically moves data between your local disk and another storage media based on a set of policies specified by an administrator. Migration is the process of moving files from a client filesystem to the migration store, which is a remote migration storage device; recall is the process of moving files from the remote storage device back to the original location on the client filesystem. The purpose of HSM is to manage a network's storage resources more effectively by keeping newer data available for fast access without compromising the availability of older or less frequently accessed data. Except for a relatively longer access time for migrated files, the entire migration and recall process is transparent to the user.

HSM moves files between the migration client and the migration store and is managed by the migration server. The migration client is any system on the network containing data to be migrated. The migration server is a system on the network providing migration services. The migration store is attached to the migration server and can consist of disks, tapes, or optical storage media.

This data management strategy relies on the administrator's definition of a high water mark, which defines the threshold condition that determines when automatic migration begins, and a low water mark, which defines the threshold condition that determines when automatic migration stops. Migration continues until all eligible files are migrated or until the low water mark is reached.

File Migration

File migration is a "sweeping" operation determined by the criteria you define. Backup generates lists of files that are candidates for migration according to the assigned criteria. Access time is the most frequently used parameter to determine these candidates. You can enable or disable migration services for each migration client. Certain files are always excluded from migration. These files include system files, shared libraries, and all executables and data files used by Backup.

File migration can be either automatic or manual, depending on the requirements of your system. All you have to do is define your criteria and assign the appropriate criteria to each migration client. Backup automatically migrates each client's files that meet those criteria. Backup automatically recalls a migrated file when a user or application accesses it.

When a file is migrated, the original file on the client computer is replaced with a stub file that points to the location of the migrated file on storage media. The stub file is a UNIX symbolic link and contains information about the file that serves two purposes:

• As a place holder for the migrated file, making it appear as though the file is still resident on the local disk

• As a pointer to the new location, allowing the HSM software to find the migrated file and recall it to the local disk

After a file migrates and is replaced with a stub file, the user can perform the same actions on the stub file as on any other file in the filesystem. The stub file can be moved, renamed, or have any other action applied to it that does not require read or write access.

Migration Policies

You can set the values in the Migration resource that HSM uses to determine what the client filesystem capacity should be for migration to start and stop. From the nwadmin GUI, select Migration Setup from the Clients menu. For each migration client, you determine the following:

• High water mark - specifies the percentage of disk space filled. When this value is reached, migration starts automatically.

• Low water mark - specifies the percentage of disk space filled after migration. When this value is reached, migration stops.

When the client filesystem reaches the specified high water mark, the Backup HSM application automatically migrates the files that meet the defined criteria.

In addition to the high and low water marks, you must set one or more criteria that files must meet to become candidates for migration. If you set more that one criteria, files must meet all the specified criteria to become candidates for migration. For example, you can set a policy that specifies when the client filesystem exceeds 70% full, files in the /home directory that have not been accessed in over 60 days and are at least 2 KB or larger in size are automatically migrated. You can set the following migration criteria:

• Last access time - specifies the length of time since a file was last accessed.

• Minimum file size - specifies the minimum file size to consider for migration. Files smaller than this entry do not provide enough available disk space after being replaced with a stub file to warrant migrating them.

• File owner - specifies the name of the owner of the file you want considered for migration. If you want all owners allowed, leave this text box blank. If you want all owners allowed except for owner_name, enter -owner_name in the field.

• File group - specifies the name of the group with access to the files to be migrated. If you want all groups allowed except for group_name, enter -group_name in this field.

• Preserve - specifies the files you do not want migrated. These entries must be full pathnames and may contain UNIX shell wildcard characters.

How Files Are Migrated

After you have specified migration policies for your migration client in the Migration resource, Backup migrates files in the following way:

1. When the Backup server conducts a regularly scheduled backup, it checks each client in the backup group for files that are candidates for migration. During a scheduled backup, the premigration command, nsrpmig, searches the migration client filesystem for files that meet the migration criteria.

   Premigration is a resource-intensive activity. The group containing migration clients should start its scheduled backup at a time of low system use.

2. Files that meet the migration criteria are premigrated. During premigration, the file is copied to a Backup storage location (a migration volume), but the original file remains on the client machine.

3. When the client filesystem reaches the high water mark, the nsrexecd daemon starts the migration command, nsrmig, and migration occurs automatically. The nsrmig command checks the premigrated files to ensure that they still meet the migration criteria. If the premigrated files are still candidates for migration, the nsrmig command does the following:

   • Renames the original file on the client filesystem with a temporary name.

   • Creates a stub file on the client filesystem to point to the migrated file on the migration media.

   • Deletes the original file from the client filesystem.

4. Migration continues until the low water mark is reached. If not enough files meet the migration criteria, the migration process might not meet the low water mark.

5. A migration report is emailed to the administrator.

   Backup makes entries for migrated files in the client index. These entries, however, are not visible to a user through the recovery GUI. Backup uses these entries to track the link between the migrated file and the stub file in the client filesystem as well as for recall purposes. Because a migrated file must be available for a user to recall, the index entries for migrated data are exempt from the automatic data recycling policies set for a Backup client. See "How HSM Handles Renamed or Deleted Files" for details on how Backup handles files that have been deleted from the client filesystem.

Files That Are Not Migrated

Certain files are always excluded from migration. These files include system files, shared libraries, and all executables and data files used by Backup. The following files are excluded from migration:

• All files in the /, /usr, /opt, and /var filesystems

• All files that end with .so

• All files (executables and data files) used by Backup

• Files that are larger than 2 GB

Additionally, you can choose certain files or groups of files to exclude from migration. For example, you can exclude files owned by root from automatic migration.

File Recall

When a user or application accesses a migrated file to read, write, or change attributes, Backup automatically recalls the file to the location of the stub file. After the recall starts, the file begins to open. The recall operation finishes before control return to the user program. Therefore, the user might notice a delay in reading and writing until the file is completely recalled to its original position. Other than a slower access time, however, the entire recall process is transparent to the user. Access time depends on the availability of the migration media, device speed, and network speed.

If the local hard disk has insufficient free space to recall the file, Backup issues the appropriate notifications. Backup provides preconfigured HSM notifications. See "Event Notification" for details on using notifications.

How HSM Complements Backup Operations

HSM is a complementary solution to backup, archiving, and save set staging operations. HSM allows system administrators to manage network resources more effectively, often resulting in lower cost for hardware storage. All Backup features store data on media; each one, however, has a specific purpose. Table 8-1 compares the goals of backup, HSM, save set staging, and archiving to demonstrate how these features work together to provide a complete storage management solution.

Table 8-1 Comparison of Backup, HSM, Save Set Staging, and Archive

	Backup	HSM	Save Set Staging	Archive
Goal	Protects data from accidental loss or damage	Conserves network storage resources	Moves data from one storage medium to another	Conserves online storage space
Files stored	Entire file system	Infrequently accessed files	Any backed up, migrated, or archived file	Rarely accessed files
Frequency	Regularly	Policy-based	Policy-based	At project's end
Method	Automatically	Automatically or manually	Automatically or manually	Manually
Original File	Left in place	Stub remains (can recall file)	Moved to new storage medium	Usually deleted
Method used to return file to client	Restored by administrator if data online is corrupted or accidentally deleted	Recalled automatically and transparently whenever user tries to access file	Restored by administrator if data online is corrupted or accidentally deleted	Retrieved by administrator if needed by users

Using HSM with Network Filesystem (NFS) Clients

When Backup migrates a file, it leaves a stub file on the original client filesystem. The stub file is a UNIX symbolic link that points to the new location of the migrated file. Because the stub files that Backup creates are symbolic links, NFS (network filesystem) clients cannot premigrate or migrate files on an NFS-mounted directory.

An NFS client, however, might need to recall previously migrated files from an NFS-mounted directory. Backup allows this operation under the following configuration:

• The NFS server must be a Solaris computer running the Backup client software and have a Migration Setup configured for it.

• The NFS client must be a Solaris computer running the Backup client software and have a Migration Setup configured for it.

• Both the NFS server and the NFS client must be configured as clients to the same Backup server.

• The NFS client must have the correct user/group available and have write privileges on the NFS-mounted directory.

• The NFS server must list the NFS client as a Remote Access User in its Backup client resource.

Figure 8-1 illustrates this configuration scenario.

Figure 8-1 Configuration Scenario For NFS and Migration Clients

In this scenario, the host Oak is a Backup server with the HSM module enabled and Elm and Pine on its list of clients. The host Elm is an NFS server that has the Backup client software running on it and Migration Setup configured for it, which makes it a Backup migration client. Pine is an NFS client that receives NFS services from Elm. Pine also has the Backup client software running on it and a Migration Setup configured for it, which makes it a Backup migration client.

For Pine to recall a file that has been migrated from Elm, Pine must be listed in the Remote Access attribute in Elm's Backup client resource. The recall operation recalls the migrated file from the migration store to the location of the stub file on Elm. This operation is transparent to the user.

If an NFS client does not meet these configuration criteria, you can use the rlogin command to log on to the NFS server and recall the file by performing any read or write operation on it. The migrated file is automatically recalled to its original location.