Skip Headers
Oracle® Hierarchical Storage Manager and StorageTek QFS Software File System Recovery Guide
Release 6.0
E42065-03
  Go To Documentation Library
Library
Go To Table Of Contents
Contents

Previous
Previous
 
Next
Next
 

5 Recovering Lost and Damaged Files

This chapter outlines procedures for restoring individual files to the file system. It covers the following topics:

Recovering Files Using a Recovery Point File

A recovery point file is the fastest, most reliable, most thorough, and least labor-intensive way of recovering lost or damaged files. So, if a recovery point file is available, proceed as follows:

  1. Log in to the file-system metadata server as root.

    root@solaris:~# 
    
  2. If you have not already done so, stop archiving and recycling using the procedures in "Stopping Archiving and Recycling Processes"

  3. In the target file system, create a temporary recovery directory to hold the recovered files.

    In the example, we create the temporary directory restore under the mount point for the recreated file system, /samqfs1:

    root@solaris:~# mkdir /samqfs1/restore
    
  4. Keep the archiver from archiving from the temporary directory. Use the command archive -r -n directory, where:

    • -r -n recursively disable archiving of files that reside in or under the specified directory.

    • directory is the path and directory name of the temporary recovery directory.

    root@solaris:~# archive -r -n /samqfs1/restore
    
  5. Change to the temporary recovery directory.

    root@solaris:~# cd /samqfs1/restore
    
  6. Identify the most recent available recovery point file.

    In the example, we have been creating dated recovery point files for the file system samqfs1 in a well-known location, the subdirectory samqfs1_recovery on the independent file system /zfs1. So the latest file, 20150324, is easy to find:

    root@solaris:~# dir /zfs1/samqfs1_recovery/
    20150321    20150322    20150323    20150324
    root@solaris:~# 
    
  7. Make sure that the file that you need to recover is in the recovery point file. Search for the required file in the output of the command samfsrestore -t -f recovery-point, where:

    • -t displays a table of contents.

    • -f recovery-point-file specifies the path and file name of the selected recovery point file.

    In the example, we are trying to recover the file genw445. So we run the command samfsrestore -t with the recovery point file /zfs1/samqfs1_recovery/20150324. To simplify the search, we pipe the output of samfsrestore -t to the Solaris grep command and the regular expression "genw445" (note that the command below is entered as a single line—the line break is escaped by the backslash character):

    root@solaris:~# samfsrestore -t -f /zfs1/samqfs1_recovery/20150324 | \
    grep "genw445"
    ./genfiles/genw445
    root@solaris:~# 
    
  8. Restore the file's inode information to the current directory. Use the command samfsrestore -f recovery-point file, where:

    • -f recovery-point-file specifies the path and file name of the selected recovery point file.

    • file specifies the exact path and name that the recovery point file lists for the file that you want to recover.

    In the example, we recover ./genfiles/genw445 from the recovery point file /zfs1/samqfs1_recovery/20150324 (note that the command below is entered as a single line—the line break is escaped by the backslash character):

    root@solaris:~# samfsrestore -f /zfs1/samqfs1_recovery/20150324 \
    ./genfiles/genw445
    root@solaris:~# 
    
  9. Make sure that the file has been restored correctly. Use the command sls -D file, where file specifies the path and name of the file relative to the temporary recovery directory.

    In the example, the file genfiles/genw445 has been recovered to the temporary directory /samqfs1/restore/:

    root@solaris:~# sls -D genfiles/genw445
    genfiles/genw445:
      mode: -rw-r--r--    links:   1  owner: data        group: samqfs1      
      length:     14975  inode:    25739.1
    offline; archdone;
    copy 1: ---- Mar  4 11:55 8ae.1 xt 000000
    copy 2: ---- Mar  4 15:51 cd3.7f57 xt 000000
      access:      Mar  4 11:55  modification: Mar  4 21:50
      changed:     Mar  4 11:50  attributes:   Mar  4 21:50
      creation:    Mar  4 11:50  residence:    Mar  4 21:50
    root@solaris:~# 
    
  10. If the file has been correctly restored, move it to the correct location in the file system.

    In the example, we move the file genw445 from the temporary, working directory /samqfs1/restore/genfiles/ to its original location in /samqfs1/genfiles/:

    root@solaris:~# mv -f genfiles/genw445 /samqfs1/genfiles/genw445
    root@solaris:~# 
    
  11. Repeat this procedure until all missing files have been recovered.

  12. Finish the recovery procedure. Go to "Restoring Archiving File Systems to Normal Operation".

Recovering Files Using Archiver Log Entries

Recovering files with an archive log is always a tedious and labor-intensive process, if more than a few files are involved. So, whenever possible, use the procedures in this section only when a recovery point cannot restore the file that you need.

While the overall process for recovering files from archival media is essentially the same in all cases, details can differ for different types of file. So select the procedure intended for the type of file that you are restoring:

Note that files may not be restored to the precise location that you expect when you recover a copy from the media. Files are restored to their location at the time when the archive copy was made. So files that were subsequently moved are not restored to the directory where they were when they were lost.

Restore Lost and Damaged Regular Files

For each file that you need to recover, proceed as follows:

  1. If you have not already done so, log in to the file-system metadata server as root.

    root@solaris:~# 
    
  2. If you have not already done so, stop archiving and recycling using the procedures in "Stopping Archiving and Recycling Processes"

  3. Change to the root directory of the file system that you are restoring.

    Oracle HSM archive files store copies relative to the file-system root directory. So to restore them to their original locations, we want to restore them from the root directory.

    In the example, we change to the root of the samqfs1 file system:

    root@solaris:~# cd /samqfs1
    root@solaris:~# 
    
  4. If you have an archiver log for the period when the regular file was last archived, find the most recent entry for the file.

    In the first example, we look for an entry for the regular (type f) file genA0:

    A 2015/03/03 13:09:05 li VOL004 all.1 212.1 samqfs1 1535.2 1971 genfiles/genA0 f 0 0
    

    In the second example, we look for an entry for the regular (type f) file spcC4:

    A 2015/03/03 21:49:15 dk DISKVOL1/f2 all.1 2.2e9 samqfs1 1511.2 8971 socfiles/spcC4 f 0 0
    
  5. Once you have located a log entry for a required file, note the media type, the volume serial number of the media, and the path and name of the file relative to the root directory of the file system.

    In the first example, the file genA0 resides on an LTO (li) tape volume with the Volume Serial Number (VSN) VOL004. The file was originally stored in the file system directory /samqfs1/genfiles/:

    A 2015/03/03 13:09:05 li VOL004 all.1 212.1 samqfs1 1535.2 1971 genfiles/genA0 f 0 0
    

    In the second example, the file spcC4 resides in a disk archive (dk) with the volume serial number DISKVOL1. The file was originally stored in the file system directory /samqfs1/socfiles/:

    A 2015/03/03 21:49:15 dk DISKVOL1/f2 all.1 2.2e9 samqfs1 1511.2 8971 socfiles/spcC4 f 0 0
    
  6. If a required file resides on sequential-access media, such as magnetic tape, also note the hexadecimal value that represents the starting position of the archive (tar) file.

    In the example, file genA0 resides on tape starting at position 0x212 (212):

    A 2015/03/03 13:09:05 li VOL004 all.1 212.1 samqfs1 1535.2 1971 genfiles/genA0 f 0 0
    
  7. If a required file resides on random-access media, such as archival disk, also note the path and file name of the tar file relative to the volume serial number.

    In the example, file spcC4 resides in the f2 subdirectory immediately under the root directory of disk volume DISKVOL1:

    A 2015/03/03 21:49:15 dk DISKVOL1/f2 all.1 2.2e9 samqfs1 1511.2 8971 socfiles/spcC4 f 0 0
    
  8. If the file that you are restoring is archived on disk media, extract the archive copy of the missing or damaged file from the tar file on the disk volume. Use the command star -xv -f tarfile file, where:

    • tarfile is the name of the archive file

    • file is the path—relative to the file-system root directory—and name of the file that you need to restore.

    The star command is an enhanced Oracle HSM version of GNU tar that restores specified files from the archive file.

    In the example, we extract the data file socfiles/spcC4 from the tar file DISKVOL1/f2. The file is restored to /samqfs1/socfiles/spcC4:

    root@solaris:~# star -xvf DISKVOL1/f2 socfiles/spcC4
    
  9. If you have restored the required file from a disk archive, continue to Restore Lost and Damaged Regular Files until all required files have been restored.

  10. If the file that you are restoring is archived on removable media, such as magnetic tape, create a directory in the restored file system to hold temporary archive files.

    In the example, we create the directory /samqfs1/tars

    root@solaris:~# mkdir /samqfs1/tars
    
  11. Position the media at the beginning of the tar header for the archive file that holds the archived copy, and read the archive from the media into memory. Use the command request -m media-type -v volume-serial-number -p 0xposition path/requestfile, where:

    • -m media-type species one of the two-character media type codes listed in Appendix B.

    • -v volume-serial-number specifies the six-character, alphanumeric code that identifies the media volume.

    • -p 0xposition specifies the hexadecimal starting position that you noted in the archiver log entry.

    • path is the path to the temporary recovery directory.

    • requestfile is the name that you want to use for the in-memory tar file that the request command reads from the media.

    In the example, we create a request file, /samqfs1/tars/currentrequest starting from position 0x78 on LTO (li) volume VOL012:

    root@solaris:~# request -m li -v VOL012 -p 0x78 /samqfs1/tars/currentrequest
    
  12. Extract the archive copy of the missing or damaged file from the in-memory tar file that you created in the preceding step. Use the command star -xv -f requestfile, where:

    • requestfile is the name of the in-memory tar file.

    • file is the path—relative to the file-system root directory—and name of the file that you need to restore.

    The star command is an enhanced Oracle HSM version of GNU tar that restores specified files from the request file (the in-memory copy of the archive file).

    In the example, we extract the data file genfiles/genA0 from the request file tars/currentrequest. The file is restored to /samqfs1/genfiles/genA0:

    root@solaris:~# star -xvf tars/currentrequest genfiles/genA0
    
  13. Set any required file attributes.

    When you restore a file from a tar file, without a samfsdump or qfsdump recovery point file, the original file attributes are lost. An .inodes file has to be created for the file from scratch, using default attribute values.

  14. Repeat this procedure. Restore Lost and Damaged Regular Files until all required files have been recovered.

  15. If necessary, Restore Lost and Damaged Segmented Files or Restore Lost and Damaged Volume Overflow Files.

  16. Otherwise, finish the recovery procedure. Go to "Restoring Archiving File Systems to Normal Operation".

Restore Lost and Damaged Segmented Files

Restoring a segmented file is much like restoring a regular file. However, you recover the individual segments rather than the file itself. So, to restore the file, you must reassemble the segments into a single file and then re-segment the result. For each file that you need to recover, proceed as follows:

  1. If you have not already done so, log in to the file-system metadata server as root.

    root@solaris:~# 
    
  2. If you have not already done so, stop archiving and recycling using the procedure in "Stopping Archiving and Recycling Processes"

  3. If you have an archiver log for the period when the segmented file was last archived, search the entries for segmented (type S) files. Select the most recent entries for segments of the required file.

    A 2015/03/03 14:01:47 li VOL013 all.1 76a.1 samqfs1 14.5 10485760 bf/dat011/1 S 0 51
    A 2015/03/03 14:04:11 li VOL013 all.1 2476f.5002 samqfs1 15.5 10485760 bf/dat011/2 S 0 51
    A 2015/03/03 14:06:24 li VOL013 all.1 1409aa4.1 samqfs1 16.5 184 bf/dat011/3 S 0 51
    
  4. Once you have located the latest entries for the segments, note the following details:

    • the media type

    • the volume serial numbers of the media volumes that store file segments

    • the hexadecimal starting positions of the archive (tar) files that hold the segments

    • the path and name of the segmented file relative to the root directory of the file system

    • the number of segments in the file.

    In the example, the file dat011 is divided into three segments (1, 2, and 3). The three segments are stored in three archive files, all on a single LTO (li) tape volume, volume serial number VOL013. The three archive files start at positions 0x76a (76a), 0x2476f (2476f), and 0x1409aa4 (1409aa4)

    A 2015/03/03 14:01:47 li VOL013 all.1 76a.1 samqfs1 14.5 10485760 bf/dat011/1 S 0 51
    A 2015/03/03 14:04:11 li VOL013 all.1 2476f.5002 samqfs1 15.5 10485760 bf/dat011/2 S 0 51
    A 2015/03/03 14:06:24 li VOL013 all.1 1409aa4.1 samqfs1 16.5 184 bf/dat011/3 S 0 51
    
  5. Change to the root directory of the file system that you are restoring.

    Oracle HSM archive files store copies relative to the file-system root directory. So to restore them to their original locations, we want to restore them from the root directory.

    In the example, we change to the root of the samqfs1 file system.

    root@solaris:~# cd /samqfs1
    
  6. Create a directory in the restored file system to hold temporary archive files.

    In the example, we create the directory /samqfs1/tars

    root@solaris:~# mkdir /samqfs1/tars
    
  7. Position the media at beginning of each archive file that holds an archived copy of one or more of the file segments, and read the archive from the media into memory. Use the command request -m media-type -v volume-serial-number -p 0xposition path/requestfile, where:

    • -m media-type specifies one of the two-character media type codes listed in Appendix B.

    • -v volume-serial-number specifies the six-character alphanumeric code that identifies the media volume.

    • -p 0xposition specifies the hexadecimal starting position that you noted in the archiver log entry.

    • path is the path to the temporary recovery directory.

    • requestfile is the name that you want to use for the in-memory tar file that the request command reads from the media.

    In the example, we need to create two request files. The first, /samqfs1/tars/request76a, loads the archive file that starts at position 0x76a on LTO (li) VOL013. This archive contains both of the first two segments. The second request file, /samqfs1/tars/request1409aa4, loads the archive file at position 0x1409aa4, in this case on the same volume (segments can reside on any volume in the library):

    root@solaris:~# request -m li -v VOL013 -p 0x76a /samqfs1/tars/request76a
    root@solaris:~# request -m li -v VOL013 -p 0x1409aa4 \
    /samqfs1/tars/request1409aa4
    
  8. Extract each segment of the backup copy of the missing or damaged file from the in-memory tar file that you created in the preceding step. Use the command star -xv -f requestfile segment, where requestfile is the name of the in-memory tar file and segment is the path—relative to the file-system root directory—and name of each segment of the file that you need to restore.

    The star command is an enhanced Oracle HSM version of GNU tar that restores specified files from the archive file that you are pointing to with the request file.

    In the example, we extract two of the three segments of the data file bf/dat011 from the request file (in-memory tar files) tars/request76a and one from the request file tars/request1409aa4. The file is restored to a directory, /samqfs1/bf/dat011/, in three separate pieces:

    root@solaris:~# star -xvf tars/request76a bf/dat011/1
    root@solaris:~# star -xvf tars/request76a bf/dat011/2
    root@solaris:~# star -xvf tars/request1409aa4 bf/dat011/3
    

    When we list the contents of /samqfs1/bf/dat011, we see one sequentially numbered file for each restored segment:

    root@solaris:~# ls /samqfs/bf/dat011
    total 40968
    -rw-rw---- 1 root other 10485760 Mar  5 17:06 1
    -rw-rw---- 1 root other 10485760 Mar  5 17:06 2
    -rw-rw---- 1 root other      184 Mar  5 17:07 3
    root@solaris:~# 
    
  9. Re-assemble the restored segments into a single, unsegmented, temporary file.

    In the example, we concatenate the three segments in the /samqfs1/bf/dat011/ directory to create the file /samqfs1/bf/dat011file:

    root@solaris:~# cat /samqfs/bf/dat011/1 /samqfs/bf/dat011/2 \
    /samqfs/bf/dat011/3 > /samqfs/bf/dat011file
    root@solaris:~# 
    

    When we list the contents of /samqfs1/bf/, the new file appears alongside the directory containing the segments.

    root@solaris:~# ls -l /samqfs/bf/dat011*
    drwxr-xr-x 2 root root      4096 Mar  5 17:06 dat011
    -rw-rw---- 1 root other 20971704 Mar  5 17:14 dat011file
    root@solaris:~# 
    
  10. Remove the segments and the directory that contains them.

    root@solaris:~# rm -r /samqfs/bf/dat011/ 
    root@solaris:~# 
    
  11. Create an empty file using the original path and name of the segmented file. Use the command touch file, where file is the original path and file name.

    In the example, we create the empty file /samqfs/bf/dat011, the original name of the segmented file that we are restoring:

    root@solaris:~# touch /samqfs/bf/dat011 
    root@solaris:~# 
    
  12. Set the Oracle HSM segment attribute on the newly created, empty file. Use the command segment -l segment-length file, where segment-length is the segment length that you noted in the archiver log entry and file is the original path and name of the segmented file.

    In the example, the archiver log shows that the segment length for the file dat011 is 10485760 (the file ends in the third segment, so the length of the data on the media is less than the segment length):

    A 2015/03/03 14:01:47 li VOL013 all.1 76a.1 samqfs1 14.5 10485760 bf/dat011/1 S 0 51
    A 2015/03/03 14:04:11 li VOL013 all.1 76a.5002 samqfs1 15.5 10485760 bf/dat011/2 S 0 51
    A 2015/03/03 14:06:24 li VOL013 all.1 1409aa4.1 samqfs1 16.5 184 bf/dat011/3 S 0 51
    

    So we set the segment length for the empty file to 10485760:

    root@solaris:~# segment -l 10485760 /samqfs/bf/dat011 
    root@solaris:~# 
    
  13. Copy the unsegmented temporary file to the empty segmented file.

    In the example, we copy dat011file to dat011:

    root@solaris:~# cp /samqfs/bf/dat011file /samqfs/bf/dat011
    root@solaris:~# 
    

    When we use the command sls -2K samqfs/bf/dat011 to list the segments, they are listed as shown. So the file has been restored.

    root@solaris:~# sls -2K /samqfs/bf/dat011
    -rw-rw---- 1 root other        20971704     Mar  5 17:12 samqfs/bf/dat011
    ---------- ----- sI {3,0,0,0}
    -rw-rw---- 1 root other        10485760     Mar  5 17:12 samqfs/bf/dat011/1
    ---------- ----- sS
    -rw-rw---- 1 root other        10485760     Mar  5 17:12 samqfs/bf/dat011/2
    ---------- ----- sS
    -rw-rw---- 1 root other             184     Mar  5 17:12 samqfs/bf/dat011/3
    ---------- ----- sS
    
  14. Set any other required file attributes.

    When you restore a file from a tar file, without a samfsdump or qfsdump recovery point file, the original file attributes are lost. An .inodes file has to be created for the file from scratch, using default attribute values.

  15. The file has now been restored. Delete the unsegmented temporary file.

    In the example, we delete dat011file:

    root@solaris:~# rm /samqfs/bf/dat011file
    root@solaris:~# 
    
  16. Repeat this procedure until all required files have been recovered.

  17. Finish the recovery procedure. Go to "Restoring Archiving File Systems to Normal Operation".

Restore Lost and Damaged Volume Overflow Files

A volume overflow file is a regular file that spans media volumes. Restoring a volume overflow file is thus much like restoring any other regular file. However, you must combine sections of an archive file that resides on multiple volumes into a single archive file on disk before you extract the data file from the archive. So, for each file that you need to recover, proceed as follows:

  1. If you have not already done so, log in to the file-system metadata server as root.

    root@solaris:~# 
    
  2. If you have not already done so, stop archiving and recycling using the procedure in "Stopping Archiving and Recycling Processes"

  3. If you have an archiver log for the period when the volume overflow file was last archived, find the most recent entry for the file. Note the volume serial number(s) of the media, the length of each section of the file, the path and name of the file relative to the root directory of the file system, and the number of sections in the file.

    In the example, we know that the file /samqfs1/rf/rf81 is a volume overflow because it is a regular, type f file that resides on two volumes, VOL036 and VOL034, and has two sections, 0 and 1:

    A 2015/03/03 18:28:51 li VOL036 all.1 12d.1 samqfs1 11731.1 89128448  rf/rf81 f 0 210
    A 2013/08/23 18:28:51 li VOL034 all.1 15f.0 samqfs1 11731.1 525271552 rf/rf81 f 1 220
    
  4. Change to the root directory of the file system that you are restoring.

    Oracle HSM archive files store copies relative to the file-system root directory. So to restore them to their original locations, we want to restore from to the root directory.

    In the example, we change to the root of the samqfs1 file system.

    root@solaris:~# cd /samqfs1
    
  5. Before proceeding, make sure that the file system contains enough free space accommodate a file at least twice the size of the file that you are recovering.

    For the file in the example, rf/rf81, we will need about 1.2 gigabytes of free space, based on the sizes of the two sections of the file: 2 x (89128448 + 525271552) = 1228800000 bytes.

  6. Create a directory in the restored file system to hold temporary archive files.

    In the example, we create the directory /samqfs1/tars

    root@solaris:~# mkdir /samqfs1/tars
    
  7. Position the media at beginning of each archive file that holds an archived copy of one or more of the file segments, and read the archive from the media into memory. Use the command request -m media-type -v volume-serial-number -p 0xposition path/requestfile, where:

    • -m media-type specifies one of the two-character media type codes listed in Appendix B.

    • -v volume-serial-number specifies the six-character alphanumeric code that identifies the media volume.

    • -p 0xposition is the hexadecimal starting position that you noted in the archiver log entry.

    • path is the path to the temporary recovery directory.

    • requestfile is the name that you want to use for the in-memory tar file that the request command reads from the media.

    In the example, we create two request files. The first request file, /samqfs1/tars/requestVOL036, loads the archive file that starts at position 0x12d on LTO (li) VOL036. The second request file, /samqfs1/tars/requestVOL034, loads the archive file at position 0x15f on LTO (li) VOL034:

    root@solaris:~# request -m li -v VOL036 -p 0x12d /samqfs1/tars/requestVOL036
    root@solaris:~# request -m li -v VOL034 -p 0x15f /samqfs1/tars/requestVOL034
    
  8. Save each of the in-memory tar files that you created to disk as a section of the archive file. Use the command dd if= requestfile of=archive_section, where requestfile is the path and name of the in-memory tar file and archive_section is the path and name of each section of the archive file.

    In the example, we save the request files (in-memory tar files), tars/requestVOL036 and tars/requestVOL034 as tars/archive_part1 and tars/archive_part2:

    root@solaris:~# dd if=tars/requestVOL036 of=tars/archive_part1
    root@solaris:~# dd if=tars/requestVOL034 of=tars/archive_part2
    root@solaris:~# 
    
  9. Re-assemble the sections into a single archive file.

    In the example, we concatenate the two sections, tars/archive_part1 and tars/archive_part2, to create a single archive file, /tars/archive_complete:

    root@solaris:~# cat tars/archive_part1 tars/archive_part2 > \
    tars/archive_complete
    root@solaris:~# 
    
  10. Extract the backup copy of the missing or damaged volume overflow file from the archive (tar) file that you created in the preceding step. Use the command star -xv -f tarfile file, where tarfile is the name of the archive file and file is the path—relative to the file-system root directory—and name of the volume overflow file that you need to restore.

    The star command is an enhanced Oracle HSM version of GNU tar that restores specified files from the archive file that you are pointing to with the request file.

    In the example, we extract the volume overflow file rf/rf81 from the tar file tars/archive_complete:

    root@solaris:~# star -xvf tars/archive_complete rf/rf81
    
  11. Set any other required file attributes.

    When you restore a file from a tar file, without a samfsdump or qfsdump recovery point file, the original file attributes are lost. An .inodes file has to be created for the file from scratch, using default attribute values.

  12. The volume overflow file has now been restored. Delete the temporary file.

    In the example, we delete dat011file:

    root@solaris:~# rm tars/archive_*
    root@solaris:~# 
    
  13. Repeat this procedure until all required files have been recovered.

  14. Finish the recovery procedure. Go to "Restoring Archiving File Systems to Normal Operation".

Recovering Damaged Archive Copies

A damaged archive copy is a copy of a file that cannot be staged back to the disk cache. Sometimes, the file has merely failed to stage due to an intermittent, hardware-related I/O problem and can be easily resolved. At other times, the damaged copy is corrupt and the data is irrecoverable. Your only option in such cases is to recover the file from an alternate copy.

To identify and manage damaged copies, proceed as follows:

  1. Identify files that have damaged archive copies. Use the command sfind mountpoint -any_copy_d, where mountpoint is the directory where the recovered file system is mounted.

    In the example, we start the search in the directory /samqfs1 and find three files that have damaged copies:

    root@solaris:~# sfind /samqfs1 -any_copy_d
    ./genfiles/ab09
    ./genfiles/ab11
    ./genfiles/ay12
    root@solaris:~# 
    
  2. For each file that you identified, identify the damaged copies. Use the command sls -D file, where file is the path and file name that you want to check.

    Damaged copies are flagged with a D. In the example, copy 2 of /samqfs1/genfiles/ab09 and copy 1 of /samqfs1/genfiles/ab11 are damaged:

    root@solaris:~# sls -D /samqfs1/genfiles/ab09
    /samqfs1/genfiles/ab09:
      mode: -rw-r-----  links:   1  owner: root group: other
      length:    306581  admin id: 0  inode:    11748.11
      project: system(0)
      copy 1: ---- Mar 11 13:52       76f.421bc li VOL011
      copy 2: ---D Mar 31 14:02       286.1324f li VOL021
      access:   Mar 11 13:50  modification: Mar 11 13:50
      changed:  Mar 11 13:50  attributes:   Mar 11 13:50
      creation: Mar 11 13:50  residence:    Mar 11 13:50
    root@solaris:~# sls -D /samqfs1/genfiles/ab11
    /samqfs1/genfiles/ab11:
      mode: -rw-r-----  links:   1  owner: root group: other
      length:    380051  admin id: 0  inode:    1460.1
      project: system(0)
      copy 1: ---D Mar 01 10:21       431.21bc6 li VOL024
      access:   Mar 01 10:21  modification: Mar 01 10:21
      changed:  Mar 01 10:21  attributes:   Mar 01 10:21
      creation: Mar 01 10:21  residence:    Mar 01 10:21
    root@solaris:~# 
    
  3. If there is an alternate copy, unarchive the damaged copy. Use the command unarchive -c CopyNumber file, where CopyNumber is an integer representing the copy number and file is the path and file name of the damaged file. Stop here.

    When you unarchive the damaged copy, Oracle HSM stages from the remaining copy and creates an additional archive copy the next time that the archiver process runs. In the example, we have another, undamaged copy of /samqfs1/genfiles/ab09, so we unarchive the damaged copy, copy 2:

    root@solaris:~# unarchive -c 2 /samqfs1/genfiles/ab09
    root@solaris:~# 
    
  4. If you do not have another copy, undamage the damaged copy. Use the command undamage -cCopyNumber file, where CopyNumber is an integer representing the copy number and file is the path and file name of the damaged file.

    Sometimes a file fails to stage due to an intermittent, hardware-related I/O error. Clearing the damage flag and restaging may solve the problem. In the example, there is only one copy of /samqfs1/genfiles/ab11:

    root@solaris:~# undamage -c1 /samqfs1/genfiles/ab11
    
  5. Try to stage the copy. Use the command stage -c CopyNumber -I file, where CopyNumber is an integer representing the copy number and file is the path and file name of the file.

    The optional -I (immediate) parameter pushes the staging operation to the head of the queue:

    root@solaris:~# stage -c 1 -I /samqfs1/genfiles/ab11
    
  6. If staging succeeds, stop here.

  7. If staging failed again, Oracle HSM again sets the damaged flag. Note the major inode number in the output of the sls -D command for the damaged copy.

    In the example, the inode number of the file, /samqfs1/genfiles/ab11, is 1460:

    root@solaris:~# sls -D /samqfs1/genfiles/ab11
    /samqfs1/genfiles/ab11:
      mode: -rw-r-----  links:   1  owner: root group: other
      length:    380051  admin id: 0  inode:    1460.1
      project: system(0)
      copy 1: ---D Mar 01 10:21       431.21bc6 li VOL024
      ...
    root@solaris:~# 
    
  8. Look for possible causes. First, examine the Oracle HSM /var/adm/sam-log file for staging related messages that pertain to the inode of the file with the damaged copy.

    The search can be carried out in various ways. In the example, we list the contents of log file using the Solaris cat command and pipe the output to grep and a regular expression that matches the inode number. We find two messages. Both indicate an I/O error and one explicitly implicates equipment (eq) ordinal number 804, one of our tape drives:

    root@solaris:~# cat /var/adm/sam-log | grep "inode 1460"
    Mar 11 15:35:44 server1 genu-20[8899]: Stage request canceled for inode 1460 (eq 804): I/O error.
    Jan 11 15:35:44 server1 samfs[8894]: /sam4 inode 1460.1 - Archive copy 1 marked damaged: I/O error
    
  9. If the /var/adm/sam-log file implicates a specific Oracle HSM equipment ordinal number, examine the device log, /var/opt/SUNWsamfs/devlog/drive-equipment-number, where drive-equipment-number is the ordinal number listed in the /var/adm/sam-log file.

  10. If the problem appears to be specific to a particular drive, make the implicated drive unavailable to the staging process using the command samcmd unavail drive-equipment-number. Then undamage the copy, and try to stage it again.

    root@solaris:~# samcmd unavail 804
    root@solaris:~# stage -c 1 -I /samqfs1/genfiles/ab11
    root@solaris:~# undamage -c1 /samqfs1/genfiles/ab11
    root@solaris:~# 
    
  11. If staging fails again or if no single drive appears to be at fault, try to recover the copy using the request and star commands, as described in "Recovering Files Using Archiver Log Entries", or Solaris utilities such as tar and dd.

  12. If you still cannot recover the file and if the value of the data warrants it, engage a data recovery service. For assistance with Oracle StorageTek tape media, engage Oracle StorageTek Enterprise Tape Data Recovery services. Log in to My Oracle Support at support.oracle.com. Open a Service Request, select the tape drive model from the list under the request category, and select Media Issues from the list under subcategory.

  13. If the file proves to be irrecoverable, unarchive the damaged copy. Use the command unarchive -c CopyNumber file, where CopyNumber is an integer representing the copy number and file is the path and file name of the damaged file.

    root@solaris:~# unarchive -c 1 /samqfs1/genfiles/ab11
    root@solaris:~# 
    
  14. Resolve any drive or media issues that are revealed by the log files.

  15. If you disabled archiving, staging, and recycling processes in a previous step, re-enable them now. Go to "Restoring Archiving File Systems to Normal Operation".

  16. Otherwise, stop here.