7 Migrating to New Storage Media

To migrate archive files from old to new media, you need to identify the files that will migrate, stage them to the disk cache, and then write them to the new media, without interfering with normal SAM-QFS file-system operations. This chapter covers the following tasks:

Planning for Data Migration
Edit the archiver.cmd file
Migrating Data to Replacement Media

Planning for Data Migration

Before you start moving data, you need to carry out the following tasks.

Estimate Available Resources
Plan Post-Migration Disposition of Old Media

Estimate Available Resources

The details of the actual data-migration process depend largely on two factors: the amount of disk storage available and the number of tape drives available. The migration works by staging files from tapes in the old media format to the disk cache and then rearchiving the migrating files from disk cache to tapes in the new media format.

Ideally, you would stage all the files on any given tape volume to disk at once and then immediately archive them to new media. But, to do this, you would have to dedicate significant resources for the duration of the migration:

disk space equivalent to the capacity of a full tape
exclusive use of a drive that reads the old tape format
exclusive use of a drive that writes the new format.

The above is not a problem if you can quiesce the file system for the duration of the migration. But migrating data in a production setting, without unduly interfering with ongoing file system and archive operations, requires some thought. If disk space or tape drives are in short supply, you need to identify the resources that you can reasonably spare for migration and then adjust the migration process. So proceed as follows:

Estimate the amount of disk cache that you can use for migration without impeding normal file-system operations.
Estimate the number of tape drives that you can afford to dedicate to migration.

If only a limited number of tape drives are available, plan on throttling the staging and archiving processes so that the migration process does not impede normal operations.
Based on your estimates above, decide on staging and archiving parameters. Determine the maximum number of migrating files that the available disk space will hold at any one time and the maximum rate at which files can be moved out of cache and on to new media.
Once you have estimated the resources required, proceed with Plan Post-Migration Disposition of Old Media.

Plan Post-Migration Disposition of Old Media

Following data migration, you may or may not need to retain the old media. So identify your requirements and plan for media disposition now, before you begin the data-migration process.

If you have recovery-point files that were created with samfsdump prior to the migration, the file-system metadata in the dump file still refers to the old tape volumes. You could not restore the file system to these recovery-points without access to the data on the original media. So, at a minimum, you should retain the old media until you are sure that you will not need to recover the file system to the state it was in at the time the older recovery-point files were created. Once you have rerun samfsdump enough times to create recovery points that reference the new volumes, you can dispose of the old volumes.

Once all current recovery-point files point to new media, you can export the old volumes from the robotic library or update and relabel them for re-use, depending on the media type. For example, you might simply export DLT media for disposal. But you could relabel media for an older Oracle StorageTek T10000C drive and use it with a newer T10000D drive.

Once you have decided how you will dispose of the old media, you are ready to start Configuring the Archiving Process for Migration.

Configuring the Archiving Process for Migration

During the migration process, you modify the archiver configuration file, archiver.cmd, to include the new, replacement media type alongside the type that you are replacing.

Edit the `archiver.cmd` file

Modify the archiver.cmd file so that the second archive copy is sent to the new media. There is no reason to add an additional archive copy.

Open the /etc/opt/SUNWsamfs/archiver.cmd file in a text editor.

The archiving policy specifies two copies, both of which are written to the media type that we want to replace. In the example, we open the file in the vi editor. We want to replace the DLT cartridges (type lt):

root@solaris: vi /etc/opt/SUNWsamfs/archiver.cmd
# =============================================
# /etc/opt/SUNWsamfs/archiver.cmd
# ---------------------------------------------
...
# ---------------------------------------------
# VSN Directives
vsns
samqfsms.1 lt .*
all.1 lt .*
all.2 lt .*
endvsns

In the directives for copy 2, change the specified media type to the identifier for the new media, save the file, and close the text editor.

In the example, we want to migrate data from the old DLT tapes to new LTO cartridges. So, in copy 2, we change the old media type, lt (DLT), to li (LTO):

root@solaris: vi /etc/opt/SUNWsamfs/archiver.cmd
# =============================================
# /etc/opt/SUNWsamfs/archiver.cmd
# ---------------------------------------------
...
# ---------------------------------------------
# VSN Directives
vsns
samqfsms.1 lt .*
all.1 lt .*
all.2 li .*
~
:wq
root@solaris:~#

Check the archiver.cmd file for syntax errors. Run the command archiver -lv, and correct errors until no errors are found.

The archiver -lv command will print out the file line-by-line. If it encounters an error, it will stop running at the point where the error occurred.

root@solaris:~# archiver -lv
Reading '/etc/opt/SUNWsamfs/archiver.cmd'.
1: # =============================================
2: # /etc/opt/SUNWsamfs/archiver.cmd
3: # ---------------------------------------------
4: # Global Directives
5: logfile = /var/opt/SUNWsamfs/archiver.log
6: # ---------------------------------------------
7: # File System Directives:
8: fs = samqfsms
9: all .
10: 1 5m ...

Once the modified archiver.cmd file is error-free, load it into the current configuration using the command samd config:
```
root@solaris:~# samd config
```
Next, Migrate Data from One Cartridge to Another.

Migrating Data to Replacement Media

The recommended procedure for migrating data uses sfind, the SAM-QFS archival file-system extension of the GNU find command. The sfind command is used to locate files on a specified tape volume and launch the stage and rearchive commands against all files found. (If you are unfamiliar with the sfind, stage, and/or rearchive commands, you should review their respective man pages now.)

For each tape cartridge that holds data that must be migrated, proceed as follows:

Migrate Data from One Cartridge to Another

Log in to the file-system host as root.
```
root@solaris:~# 
```
Move to the mount-point directory for the file system that holds the files that you are migrating.

In the example, we are migrating archived copies of files stored in the samqfs1 file system mounted at /samqfs1:
```
root@solaris:~# cd /samqfs1
```
Select a tape volume.

When migrating data from media type to media type, work with one volume at a time. The examples below, we work with volume serial number VOL008.
First, search the selected volume for damaged files that cannot be successfully staged. Use the SAM-QFS command sfind . -vsn volume-serial-number -damaged, where volume-serial-number is the alphanumeric string that uniquely identifies the volume in the library.

In the example, we start the search from the current working directory (.). The -vsn parameter limits the search to files that are found on our current tape, VOL008. The -damaged flag limits the search to files that cannot be successfully staged:
```
root@solaris:~# sfind . -vsn VOL008 -damaged 
```
If the sfind search for damaged files returns any results, try to fix the file. Use the command undamage -m media-type -vsn volume-serial-number file, where media-type is one of the two-character media type codes listed in Appendix A, "Glossary of Equipment Types", volume-serial-number is the alphanumeric string that uniquely identifies the volume, and file is the path and name of the damaged file.

Sometimes a transitory I/O error causes a copy to be marked damaged. The SAM-QFS undamage command clears this condition. In the example, the archive file copy /samqfs1/data0008/20131025DAT is reported damaged. So we undamage it and retry the search for damaged files:
```
root@solaris:~# sfind . -vsn VOL008 -damaged
/samqfs1/data0008/20131025DAT
root@solaris:~# undamage -m lt -vsn VOL008 /samqfs1/data0008/20131025DAT
root@solaris:~# sfind . -vsn VOL008 -damaged 
root@solaris:~# 
```

If the sfind command again lists the file as damaged, the copy is unusable. See if the archive contains another, undamaged copy of the file. To list the available copies, use the command sls -d file, where file is the path and name of the file. To check the status of any copies found, use the command sfind file -vsn volume-serial-number

In the example, we list all copies of the file /samqfs1/data0008/20131025DAT and find:

root@solaris:~# sls -d /samqfs1/data0008/20131025DAT
20131025DAT:
mode: -rw-r--r--  links:   1  owner: root      group: other
            length:    319279  admin id:      7  inode: 1407.5
            project: system(0)
            offline;  archdone;  stage -n;
            copy 1: ---- May 21 07:12     1e4b1.1    lt VOL008
            copy 2: ---- May 21 10:29     109c6.1    lt VOL022
...
root@solaris:~# sfind file -vsn volume-serial-number
/samqfs1/data0008/20131025DAT
root@solaris:~#

If a copy is unusable and another undamaged copy of the file exists, archive the file so that two good copies exist in the archive before you proceed further. Then unarchive the damaged copy.

In the example, copy 1 of file /samqfs1/data0008/20131025DAT on volume VOL008 is unusable, so we issue the archive command with the -c option to create a valid copy 1 before unarchiving the damaged copy on volume VOL008:
```
root@solaris:~# archive -c 1 /samqfs1/data0008/20131025DAT
root@solaris:~# unarchive -m lt -vsn VOL008 /samqfs1/data0008/20131025DAT
```
If no usable copies exist, see if the file is resident in cache. If it is, archive the file so that two good copies exist in the archive before you proceed. Once two good copies exist, unarchive damaged copy.

In the example, both copy 1 on volume VOL008 and copy 2 on volume VOL022 are unusable, so we issue the archive command to create two valid copies before unarchiving the damaged copy on volume VOL008:
```
root@solaris:~# archive /samqfs1/data0008/20131025DAT
root@solaris:~# unarchive -m lt -vsn VOL008 /samqfs1/data0008/20131025DAT
```
If no usable copies exist and if the file is not resident in the disk cache, the data has probably been lost. If the data is critical, consult a specialist data recovery firm for assistance. Otherwise, unarchive the damaged copy.

In the example, both copy 1 on volume VOL008 and copy 2 on volume VOL022 are unusable and the file is no longer in the disk cache. The data is not critical, so we unarchive the damaged copy on volume VOL008:
```
root@solaris:~# archive /samqfs1/data0008/20131025DAT
root@solaris:~# unarchive -m lt -vsn VOL008 /samqfs1/data0008/20131025DAT
```
When the sfind search for damaged files returns no results, stage the all files from the current tape to the disk cache. Use the command sfind . -vsn volser -offline -exec stage {}\;, where volser is the volume serial number (VSN) of the tape volume.

The -exec parameter runs the command line stage {}\; against each result of the sfind search, replacing the paired braces {} with each file name found.

In the example, the -vsn parameter again limits the search to files that are found on our current tape, VOL008. The -offline parameter further restricts the sfind output to files that are on tape and not already resident in cache. This precaution keeps files from being damaged during staging.
```
root@solaris:~# sfind . -vsn VOL008 -offline -exec stage {} \;
```
Once files have been staged from tape, selectively rearchive them. Use the command sfind . -vsn volser -online -exec rearch -r -m media-type {}\; where media-type is the type of media from which you are migrating.

The -exec parameter runs the command line rearch -r -m media-type {}\; against each result of the sfind search, replacing the paired braces {} with each file name found. The -vsn parameter again limits the search to files that are found on our current tape. The -online parameter further restricts the sfind output to files that are currently resident in cache rather than those still on tape. This precaution keeps files from being damaged during archiving. The -r parameter tells rearchive to recurse through directories and subdirectories.

In the example, the -vsn parameter value is VOL008, and the value of the -m parameter specifies lt, for DLT media:
```
root@solaris:~# sfind . -vsn VOL008 -online -exec rearch -r -m lt {} \;
```
Repeat the preceding step until the sfind search finds no more files.
When all files have been rearchived, dispose of the tape as planned (see "Plan Post-Migration Disposition of Old Media").
Repeat this procedure until data has been migrated from all old media to new media.