7 Migrating to New Storage Media

To migrate archive files from old to new media, you need to identify the files that will migrate, stage them to the disk cache, and then write them to the new media, without interfering with normal Oracle Hierarchical Storage Manager file-system operations. This chapter covers the following stages in the process:

Planning for Data Migration
Edit the archiver.cmd file
Migrating Data to Replacement Media.

Planning for Data Migration

Before you start moving data, you need to carry out the following tasks.

Estimate Available Resources
Plan Post-Migration Disposition of Old Media.

Estimate Available Resources

The details of the actual data-migration process depend largely on two factors: the amount of disk storage available and the number of removable media drives available. During media migration, the Oracle HSM stager loads the old removable volumes into drives that can read the old media format and restores archived files to the disk cache. Then the Oracle HSM archiver rearchives the files to new removable volumes using drives that can write the new media format. So, ideally, you would stage all the files on any given tape volume to disk at once and then immediately archive them to new media.

To do this, you would have to dedicate significant resources for the duration of the migration:

disk space equivalent to the capacity of a full tape
exclusive use of a drive that reads the old tape format
exclusive use of a drive that writes the new format.

The above is not a problem if you can quiesce the file system until migration is complete. But migrating data in a production setting, without unduly interfering with ongoing file system and archive operations, requires some thought. If disk space or tape drives are in short supply, you need to identify the resources that you can reasonably spare for migration and then adjust the migration process. So proceed as follows:

Estimate the amount of disk cache that you can use for migration without impeding normal file-system operations.
Estimate the number of tape drives that you can afford to dedicate to migration.

If only a limited number of tape drives are available, plan on throttling the staging and archiving processes so that the migration process does not impede normal operations.
Based on your estimates above, decide on staging and archiving parameters. Determine the maximum number of migrating files that the available disk space will hold at any one time and the maximum rate at which files can be moved out of cache and on to new media.
Once you have estimated the resources, Plan Post-Migration Disposition of Old Media.

Plan Post-Migration Disposition of Old Media

Following data migration, you may or may not need to retain the old media. So identify your requirements and plan for media disposition now, before you begin the data-migration process.

If you have recovery-point files that were created with samfsdump prior to the migration, the file-system metadata in the dump file still refers to the old tape volumes. You could not restore the file system to these recovery-points without access to the data on the original media. So, at a minimum, you should retain the old media until you are sure that you will not need to recover the file system to the state it was in at the time the older recovery-point files were created. Once you have rerun samfsdump enough times to create recovery points that reference the new volumes, you can dispose of the old volumes.

Once all current recovery-point files point to new media, you can export the old volumes from the robotic library or update and relabel them for re-use, depending on the media type. For example, you might simply export DLT media for disposal. But you could relabel media for an older Oracle StorageTek T10000C drive and use it with a newer T10000D drive.

Once you have decided how you will dispose of the old media, you are ready to start Configuring the Archiving Process for Migration.

Configuring the Archiving Process for Migration

During the migration process, you modify the archiver configuration file, archiver.cmd, to include the new, replacement media type alongside the type that you are replacing.

Edit the `archiver.cmd` file

Modify the archiver.cmd file so that the second archive copy is sent to the new media. There is no reason to add an additional archive copy.

Open the /etc/opt/SUNWsamfs/archiver.cmd file in a text editor.

The archiving policy specifies two copies, both of which are written to the media type that we want to replace. In the example, we open the file in the vi editor. We want to replace the DLT cartridges (type lt):

root@solaris: vi /etc/opt/SUNWsamfs/archiver.cmd
# =============================================
# /etc/opt/SUNWsamfs/archiver.cmd
# ---------------------------------------------
...
# ---------------------------------------------
# VSN Directives
vsns
allfiles.1 lt .*
allfiles.2 lt .*
endvsns

In the directives for copy 2, change the specified media type to the identifier for the new media, save the file, and close the text editor.

In the example, we want to migrate data from the old DLT tapes to new LTO cartridges. So, in copy 2, we change the old media type, lt (DLT), to li (LTO):

root@solaris: vi /etc/opt/SUNWsamfs/archiver.cmd
# =============================================
# /etc/opt/SUNWsamfs/archiver.cmd
# ---------------------------------------------
...
# ---------------------------------------------
# VSN Directives
vsns
allfiles.1 lt .*
allfiles.2 li .*
endvsns
:wq
root@solaris:~#

Check the archiver.cmd file for syntax errors. Run the command archiver -lv, and correct errors until no errors are found.

The archiver -lv command will print out the file line-by-line. If it encounters an error, it will stop running at the point where the error occurred.

root@solaris:~# archiver -lv
Reading '/etc/opt/SUNWsamfs/archiver.cmd'.
1: # =============================================
2: # /etc/opt/SUNWsamfs/archiver.cmd
3: # ---------------------------------------------
4: # Global Directives
5: logfile = /var/opt/SUNWsamfs/archiver.log
6: # ---------------------------------------------
7: # File System Directives:
8: fs = samqfsms
9: all .
10: 1 5m ...
root@solaris:~#

Once the modified archiver.cmd file is error-free, load it into the current configuration using the command samd config:
```
root@solaris:~# samd config
Configuring SAM-FS
root@solaris:~# 
```
Next, Migrate Data from One Cartridge to Another.

Migrating Data to Replacement Media

The recommended procedure for migrating data uses sfind, the Oracle HSM extension of the GNU find command. The sfind command is used to locate files on a specified tape volume and launch the stage and rearchive commands against all files found.

If you are unfamiliar with the sfind, stage, and/or rearchive commands, you should review their respective man pages now. Then, for each tape cartridge that holds data that must be migrated, proceed as follows:

Migrate Data from One Cartridge to Another

Log in to the file-system host as root.
```
root@solaris:~# 
```
Move to the mount-point directory of the file system that holds the files that you are migrating.

In the example, we are migrating archived copies of files stored in the samqfs1 file system mounted at /samqfs1:
```
root@solaris:~# cd /samqfs1
root@solaris:~# 
```
Select a tape volume.

When migrating data from media type to media type, work with one volume at a time. In the examples below, we work with volume serial number VOL008.
First, search the selected volume for damaged files that cannot be successfully staged. Use the Oracle HSM command sfind . -vsn volume-serial-number -damaged, where volume-serial-number is the alphanumeric string that uniquely identifies the volume in the library.

In the example, we start the search from the current working directory (.). The -vsn parameter limits the search to files that are found on our current tape, VOL008. The -damaged flag limits the search to files that cannot be successfully staged:
```
root@solaris:~# sfind . -vsn VOL008 -damaged 
```
If the sfind search for damaged files returns any results, try to fix the file. Use the command undamage -m media-type -vsn volume-serial-number file, where:
- media-type is one of the two-character media type codes listed in Appendix A, "Glossary of Equipment Types".
- volume-serial-number is the alphanumeric string that uniquely identifies the volume.
- file is the path and name of the damaged file.
Sometimes a transitory I/O error causes a copy to be marked damaged. The Oracle HSM undamage command clears this condition. In the example, the archive file copy /samqfs1/data0008/20131025DAT is reported damaged. So we undamage it, and retry the search for damaged files:
```
root@solaris:~# sfind . -vsn VOL008 -damaged
/samqfs1/data0008/20131025DAT
root@solaris:~# undamage -m lt -vsn VOL008 /samqfs1/data0008/20131025DAT
root@solaris:~# sfind . -vsn VOL008 -damaged 
```

If the sfind command again lists the file as damaged, the copy is unusable. See if the archive contains another, undamaged copy of the file. To list the available copies, use the command sls -D file, where file is the path and name of the file. To check the status of any copies found, use the command sfind file -vsn volume-serial-number.

In the example, the undamage command could not fix the copy. So we use sls to list all copies of the file /samqfs1/data0008/20131025DAT:

root@solaris:~# undamage -m lt -vsn VOL008 /samqfs1/data0008/20131025DAT
root@solaris:~# sfind . -vsn VOL008 -damaged
/samqfs1/data0008/20131025DAT
root@solaris:~# sls -D /samqfs1/data0008/20131025DAT
20131025DAT:
mode: -rw-r--r--  links:   1  owner: root      group: other
            length:    319279  admin id:      7  inode: 1407.5
            project: system(0)
            offline;  archdone;  stage -n;
            copy 1: ---- May 21 07:12     1e4b1.1    lt VOL008
            copy 2: ---- May 21 10:29     109c6.1    lt VOL022
...

Tape volume VOL022 holds a second copy of the file. So we check the second copy with sfind:

root@solaris:~# sfind /samqfs1/data0008/20131025DAT -vsn VOL022 -damaged

If a copy is unusable and one undamaged copy of the file exists, rearchive the file. Then, once the archive holds two good copies, unarchive the damaged copy.

In the example, copy 1 of file /samqfs1/data0008/20131025DAT on volume VOL008 is unusable, but the sfind command did not find damage to copy 2. So we issue the archive command with the -c option to create a valid copy 1 before unarchiving the damaged copy on volume VOL008:
```
root@solaris:~# sfind /samqfs1/data0008/20131025DAT -vsn VOL022 -damaged
root@solaris:~# archive -c 1 /samqfs1/data0008/20131025DAT
...
root@solaris:~# unarchive -m lt -vsn VOL008 /samqfs1/data0008/20131025DAT
```

If no usable copies exist, see if the file is resident in cache. Use the command sfind . -vsn volume-serial-number -online.

In the example, both copy 1 on volume VOL008 and copy 2 on volume VOL022 are damaged and unusable. So we see if the file is available online, in the disk cache:

root@solaris:~# undamage -m lt -vsn VOL008 /samqfs1/data0008/20131025DAT
root@solaris:~# sfind . -vsn VOL008 -damaged
/samqfs1/data0008/20131025DAT
root@solaris:~# undamage -m lt -vsn VOL022 /samqfs1/data0008/20131025DAT
root@solaris:~# sfind /samqfs1/data0008/20131025DAT -vsn VOL022 -damaged
/samqfs1/data0008/20131025DAT
root@solaris:~# sfind /samqfs1/data0008/20131025DAT -online

If no usable copies exist, but the file is resident in cache, archive the file. Then, once the archive holds two good copies, unarchive the damaged copy.

In the example, both copy 1 on volume VOL008 and copy 2 on volume VOL022 are unusable, so we issue the archive command to create two valid copies before unarchiving the damaged copy on volume VOL008:

root@solaris:~# undamage -m lt -vsn VOL008 /samqfs1/data0008/20131025DAT
root@solaris:~# sfind . -vsn VOL008 -damaged
/samqfs1/data0008/20131025DAT
root@solaris:~# undamage -m lt -vsn VOL022 /samqfs1/data0008/20131025DAT
root@solaris:~# sfind /samqfs1/data0008/20131025DAT -vsn VOL022 -damaged
/samqfs1/data0008/20131025DAT
root@solaris:~# sfind /samqfs1/data0008/20131025DAT -online
/samqfs1/data0008/20131025DAT
root@solaris:~# archive /samqfs1/data0008/20131025DAT
root@solaris:~# unarchive -m lt -vsn VOL008 /samqfs1/data0008/20131025DAT

If no usable copies exist and if the file is not resident in the disk cache, the data has probably been lost. If the data is critical, consult a specialist data recovery firm for assistance. Otherwise, unarchive the damaged copy.

In the example, both copy 1 on volume VOL008 and copy 2 on volume VOL022 are unusable. The sfind command could not find the file the disk cache. The data is not critical. So we unarchive the damaged copy on volume VOL008:

root@solaris:~# undamage -m lt -vsn VOL008 /samqfs1/data0008/20131025DAT
root@solaris:~# sfind . -vsn VOL008 -damaged
/samqfs1/data0008/20131025DAT
root@solaris:~# undamage -m lt -vsn VOL022 /samqfs1/data0008/20131025DAT
root@solaris:~# sfind /samqfs1/data0008/20131025DAT -vsn VOL022 -damaged
/samqfs1/data0008/20131025DAT
root@solaris:~# sfind /samqfs1/data0008/20131025DAT -online
root@solaris:~# archive /samqfs1/data0008/20131025DAT
root@solaris:~# unarchive -m lt -vsn VOL008 /samqfs1/data0008/20131025DAT

When the sfind search for damaged files returns no results, stage the files from the current tape to the disk cache. Use the command sfind . -vsn volume-serial-number -offline -exec stage {}\;

The -vsn parameter limits the search to files that are found on the current tape (always migrate data one tape at a time.).

The -offline parameter further restricts the sfind output to files that are not already resident in cache, so that data is not overwritten.

The -exec stage {}\; argument takes each path and file name that sfind returns and uses it as the argument to an Oracle HSM stage command. The stage command then restores the specified file to disk cache. The process repeats until all eligible files have been staged.

In the example, the sfind -vsn VOL008 -damaged command returns no output. So we use sfind to stage all files that are found on VOL008 and are not already in cache:
```
root@solaris:~# sfind . -vsn VOL008 -damaged
root@solaris:~# sfind . -vsn VOL008 -offline -exec stage {}\;
```
Once files have been staged from tape, selectively rearchive them. Use the command sfind . -vsn volume-serial-number -online -exec rearch -r -m media-type {}\; where media-type is the type of media from which you are migrating.

The -vsn parameter limits the search to files that are also found on the current tape (always migrate data one tape at a time.).

The -online parameter further restricts the sfind output to files that are resident in cache, so that data is not overwritten.

The -exec rearch -r -m media-type {}\; argument takes each path and file name that sfind returns and uses it as the argument to an Oracle HSM rearch -r -m media-type command. The -r argument runs the process recursively through subdirectories. The -m argument rearchives only files that reside on the source media.

In the example, the -vsn parameter value is VOL008, and the value of the -m parameter specifies lt, for DLT media:
```
root@solaris:~# sfind . -vsn VOL008 -online -exec rearch -r -m lt {}\;
```
Repeat the preceding step until the sfind search finds no more files.
When all files have been rearchived, dispose of the tape as planned (see "Plan Post-Migration Disposition of Old Media").
Repeat this procedure until data has been migrated from all old media to new media.