Skip Headers
StorageTek Storage Archive Manager and StorageTek QFS Software File System Recovery Guide
Release 5.4
E42065-02
  Go To Documentation Library
Library
Go To Table Of Contents
Contents

Previous
Previous
 
Next
Next
 

5 Recovering Lost and Damaged Files

This chapter outlines procedures for restoring individual files to the file system. It covers the following topics:

Recovering Files Using a Recovery Point File

A recovery point file is the fastest, most reliable, most thorough, and least labor-intensive way of recovering lost or damaged files. So, if a recovery point file is available, proceed as follows:

  1. Log in to the file-system metadata server as root.

    root@solaris:~# 
    
  2. If you have not already done so, stop archiving and recycling using the procedure in "Stopping Archiving and Recycling Processes"

  3. In the target file system, create a temporary recovery directory to hold the recovered files.

    In the example, we create the temporary directory restore under the mount point for the recreated file system, /samqfs1:

    root@solaris:~# mkdir /samqfs1/restore
    
  4. Keep the archiver from archiving from the temporary directory. Use the command archive -r -n directory, where directory is the path and directory name of the temporary directory.

    root@solaris:~# archive -r -n /samqfs1/restore
    
  5. Change to the temporary recovery directory.

    root@solaris:~# cd /samqfs1/restore
    
  6. Identify the most recent available recovery point file.

    In the example, we have been creating dated recovery point files for the file system samqfs1 in a well-known location, the subdirectory samqfs1_recovery on the independent file system /zfs1. So the latest file is easy to find:

    root@solaris:~# dir /zfs1/samqfs1_recovery/
    20140321    20140322    20140323    20140324
    root@solaris:~# 
    
  7. Make sure that the file that you need to recover is in the recovery point file. Use the command samfsrestore -t -f recovery-point, where -t specifies the table of contents and recovery-point-file specifies the path and file name of the selected recovery point file. Search the output for the required file.

    In the example, we are trying to recover the file genw445. So we specify the recovery point file /zfs1/samqfs1_recovery/20140324 and pipe the output of samfsrestore -t to the Solaris grep command and the regular expression "genw445":

    root@solaris:~# samfsrestore -t -f /zfs1/samqfs1_recovery/20140324 | \
    grep "genw445"
    ./genfiles/genw445
    root@solaris:~# 
    
  8. Restore the file's inode information to the current directory. Use the command samfsrestore -f recovery-point file, where file specifies the exact path and name that the recovery point file lists for the file that you want to recover.

    In the example, we recover ./genfiles/genw445:

    root@solaris:~# samfsrestore -f /zfs1/samqfs1_recovery/20140324 \
    ./genfiles/genw445
    root@solaris:~# 
    
  9. Make sure that the file has been restored correctly. Use the command sls -D file, where file specifies the path and name of the file relative to the temporary recovery directory.

    In the example, the file genfiles/genw445 has been recovered:

    root@solaris:~# sls -D genfiles/genw445
    genfiles/genw445:
      mode: -rw-r--r--    links:   1  owner: data        group: samqfs1      
      length:     14975  inode:    25739.1
    offline; archdone;
    copy 1: ---- Mar  4 11:55 8ae.1 xt 000000
    copy 2: ---- Mar  4 15:51 cd3.7f57 xt 000000
      access:      Mar  4 11:55  modification: Mar  4 21:50
      changed:     Mar  4 11:50  attributes:   Mar  4 21:50
      creation:    Mar  4 11:50  residence:    Mar  4 21:50
    root@solaris:~# 
    
  10. Move the recovered file to the correct location in the file system.

    root@solaris:~# mv -f genfiles/genw445 /samqfs1/genfiles/genw445
    root@solaris:~# 
    
  11. Repeat this procedure until all missing files have been recovered.

  12. Finish the recovery procedure. Go to "Restoring Archiving File Systems to Normal Operation".

Recovering Files Using Archiver Log Entries

Using an archive log for guidance when recovering files from media is always a somewhat tedious and labor-intensive process, if more than a few files are involved. So, whenever possible, use the procedures in this section only when a recovery point cannot restore the file that you need.

While the process of recovering files from archival media is essentially the same in all cases, some details differ significantly, depending on the type of file that you are recovering. So select the procedure intended for the type of file that you are restoring:

Note that files may not be restored to the precise location that you expect when you recover a copy from the media. Files are restored to their location at the time when the archive copy was made. So files that were subsequently moved are not restored to the directory where they were when they were lost.

Restore Lost and Damaged Regular Files

For each file that you need to recover, proceed as follows:

  1. If you have not already done so, log in to the file-system metadata server as root.

    root@solaris:~# 
    
  2. If you have not already done so, stop archiving and recycling using the procedure in "Stopping Archiving and Recycling Processes"

  3. Change to the root directory of the file system that you are restoring.

    SAM-QFS archive files store copies relative to the file-system root directory. So to restore them to their original locations, we want to restore them from the root directory.

    In the example, we change to the root of the samqfs1 file system.

    root@solaris:~# cd /samqfs1
    
  4. If you have an archiver log for the period when the regular file was last archived, find the most recent entry for the file. Note the media type, the volume serial number of the media, and the path and name of the file relative to the root directory of the file system. If the copy is stored on sequential-access media, such as magnetic tape, also note the hexadecimal value that represents the starting position of the archive (tar) file. If the copy is stored on random-access media, such as archival disk, note the path and file name of the tar file relative to the volume serial number.

    A 2014/03/03 13:09:05 li VOL004 all.1 212.1 samqfs1 1535.2 1971 genfiles/genA0 f 0 0
    A 2014/03/03 21:49:15 dk DISKVOL1/f2 all.1 2.2e9 samqfs1 1511.2 8971 socfiles/spcC4 f 0 0
    
  5. If the file that you are restoring is archived on disk media, extract the archive copy of the missing or damaged file from the tar file on the disk volume. Use the command star -xv -f tarfile file, where tarfile is the name of the archive file and file is the path—relative to the file-system root directory—and name of the file that you need to restore. Stop here, and go on to the next file.

    The star command is an enhanced SAM-QFS version of GNU tar that restores specified files from the archive file that you are pointing to with the request file.

    In the example, we extract the data file genfiles/genC4 from the tar file DISKVOL1/f2. The file is restored to /samqfs1/genfiles/genC4:

    root@solaris:~# star -xvf DISKVOL1/f2 genfiles/genC4
    
  6. If the file that you are restoring is archived on removable media, such as magnetic tape, create a directory in the restored file system to hold temporary archive files.

    In the example, we create the directory /samqfs1/tars

    root@solaris:~# mkdir /samqfs1/tars
    
  7. Position the media at the beginning of the tar header for the archive file that holds the archived copy, and read the archive from the media into memory. Use the command request -m media-type -v volume-serial-number -p 0xposition path/requestfile, where:

    • media-type is one of the two-character media type codes listed in Appendix B

    • volume-serial-number is the six-character alphanumeric code that identifies the media volume

    • position is the hexadecimal starting position that you noted in the archiver log entry

    • path is the path to the temporary recovery directory.

    • requestfile is the name that you want to use for the in-memory tar file that the request command reads from the media.

    In the example, we create a request file, /samqfs1/tars/currentrequest starting from position 0x78 on LTO (li) VOL012:

    root@solaris:~# request -m li -v VOL012 -p 0x78 /samqfs1/tars/currentrequest
    
  8. Extract the archive copy of the missing or damaged file from the in-memory tar file that you created in the preceding step. Use the command star -xv -f requestfile, where requestfile is the name of the in-memory tar file and file is the path—relative to the file-system root directory—and name of the file that you need to restore.

    The star command is an enhanced SAM-QFS version of GNU tar that restores specified files from the archive file that you are pointing to with the request file.

    In the example, we extract the data file genfiles/genA0 from the request file (in-memory tar file) currentrequest which is in the temporary directory tars/. The file is restored to /samqfs1/genfiles/genA0:

    root@solaris:~# star -xvf tars/currentrequest genfiles/genA0
    
  9. Set any required file attributes.

    When you restore a file from a tar file, without a samfsdump or qfsdump recovery point file, the original file attributes are lost. An .inodes file has to be created for the file from scratch, using default attribute values.

  10. Repeat this procedure until all required files have been recovered.

  11. Finish the recovery procedure. Go to "Restoring Archiving File Systems to Normal Operation".

Restore Lost and Damaged Segmented Files

Restoring a segmented file is much like restoring a regular file. However, you recover the individual segments rather than the file itself. So, to restore the file, you must reassemble the segments into a single file and then re-segment the result. For each file that you need to recover, proceed as follows:

  1. If you have not already done so, log in to the file-system metadata server as root.

    root@solaris:~# 
    
  2. If you have not already done so, stop archiving and recycling using the procedure in "Stopping Archiving and Recycling Processes"

  3. If you have an archiver log for the period when the segmented file was last archived, find the most recent entry for the file. Note the media type, the volume serial number(s) of the media, the hexadecimal position where the archive (tar) file starts, file, the path and name of the segmented file relative to the root directory of the file system, and the number of segments in the file.

    A 2014/03/03 14:01:47 li VOL013 all.1 76a.1 samqfs1 14.5 10485760 bf/dat011/1 S 0 51
    A 2014/03/03 14:04:11 li VOL013 all.1 76a.5002 samqfs1 15.5 10485760 bf/dat011/2 S 0 51
    A 2014/03/03 14:06:24 li VOL013 all.1 1409AA4.1 samqfs1 16.5 184 bf/dat011/3 S 0 51
    
  4. Change to the root directory of the file system that you are restoring.

    SAM-QFS archive files store copies relative to the file-system root directory. So to restore them to their original locations, we want to restore them from the root directory.

    In the example, we change to the root of the samqfs1 file system.

    root@solaris:~# cd /samqfs1
    
  5. Create a directory in the restored file system to hold temporary archive files.

    In the example, we create the directory /samqfs1/tars

    root@solaris:~# mkdir /samqfs1/tars
    
  6. Position the media at beginning of each archive file that holds an archived copy of one or more of the file segments, and read the archive from the media into memory. Use the command request -m media-type -v volume-serial-number -p 0xposition path/requestfile, where:

    • media-type is one of the two-character media type codes listed in Appendix B

    • volume-serial-number is the six-character alphanumeric code that identifies the media volume

    • position is the hexadecimal starting position that you noted in the archiver log entry

    • path is the path to the temporary recovery directory.

    • requestfile is the name that you want to use for the in-memory tar file that the request command reads from the media.

    In the example, we need to create two request files. The first, /samqfs1/tars/request76a, loads the archive file that starts at position 0x76a on LTO (li) VOL013. This archive contains both of the first two segments. The second request file, /samqfs1/tars/request1409aa4, loads the archive file at position 0x1409aa4, in this case on the same volume (segments can reside on any volume in the library):

    root@solaris:~# request -m li -v VOL013 -p 0x76a /samqfs1/tars/request76a
    root@solaris:~# request -m li -v VOL013 -p 0x1409aa4 \
    /samqfs1/tars/request1409aa4
    
  7. Extract each segment of the backup copy of the missing or damaged file from the in-memory tar file that you created in the preceding step. Use the command star -xv -f requestfile segment, where requestfile is the name of the in-memory tar file and segment is the path—relative to the file-system root directory—and name of each segment of the file that you need to restore.

    The star command is an enhanced SAM-QFS version of GNU tar that restores specified files from the archive file that you are pointing to with the request file.

    In the example, we extract two of the three segments of the data file bf/dat011 from the request file (in-memory tar files) tars/request76a and one from the request file tars/request1409aa4. The file is restored to a directory, /samqfs1/bf/dat011/, in three separate pieces:

    root@solaris:~# star -xvf tars/request76a bf/dat011/1
    root@solaris:~# star -xvf tars/request76a bf/dat011/2
    root@solaris:~# star -xvf tars/request1409aa4 bf/dat011/3
    

    When we list the contents of /samqfs1/bf/dat011, we see one sequentially numbered file for each restored segment:

    root@solaris:~# ls /samqfs/bf/dat011
    total 40968
    -rw-rw---- 1 root other 10485760 Mar  5 17:06 1
    -rw-rw---- 1 root other 10485760 Mar  5 17:06 2
    -rw-rw---- 1 root other      184 Mar  5 17:07 3
    root@solaris:~# 
    
  8. Re-assemble the restored segments into a single, unsegmented, temporary file.

    In the example, we concatenate the three segments in the /samqfs1/bf/dat011/ directory to create the file /samqfs1/bf/dat011file:

    root@solaris:~# cat /samqfs/bf/dat011/1 /samqfs/bf/dat011/2 \
    /samqfs/bf/dat011/3 > /samqfs/bf/dat011file
    root@solaris:~# 
    

    When we list the contents of /samqfs1/bf/, the new file appears alongside the directory containing the segments.

    root@solaris:~# ls -l /samqfs/bf/dat011*
    drwxr-xr-x 2 root root      4096 Mar  5 17:06 dat011
    -rw-rw---- 1 root other 20971704 Mar  5 17:14 dat011file
    root@solaris:~# 
    
  9. Remove the segments and the directory that contains them.

    root@solaris:~# rm -r /samqfs/bf/dat011/ 
    root@solaris:~# 
    
  10. Create an empty file using the original path and name of the segmented file. Use the command touch file, where file is the original path and file name.

    In the example, we create the empty file /samqfs/bf/dat011, the original name of the segmented file that we are restoring:

    root@solaris:~# touch /samqfs/bf/dat011 
    root@solaris:~# 
    
  11. Set the SAM-QFS segment attribute on the newly created, empty file. Use the command segment -l segment-length file, where segment-length is the segment length that you noted in the archiver log entry and file is the original path and name of the segmented file.

    In the example, the archiver log shows that the segment length for the file dat011 is 10485760 (the file ends in the third segment, so the length of the data on the media is less than the segment length):

    A 2014/03/03 14:01:47 li VOL013 all.1 76a.1 samqfs1 14.5 10485760 bf/dat011/1 S 0 51
    A 2014/03/03 14:04:11 li VOL013 all.1 76a.5002 samqfs1 15.5 10485760 bf/dat011/2 S 0 51
    A 2014/03/03 14:06:24 li VOL013 all.1 1409aa4.1 samqfs1 16.5 184 bf/dat011/3 S 0 51
    

    So we set the segment length for the empty file to 10485760:

    root@solaris:~# segment -l 10485760 /samqfs/bf/dat011 
    root@solaris:~# 
    
  12. Copy the unsegmented temporary file to the empty segmented file.

    In the example, we copy dat011file to dat011:

    root@solaris:~# cp /samqfs/bf/dat011file /samqfs/bf/dat011
    root@solaris:~# 
    

    When we use the command sls -2K samqfs/bf/dat011 to list the segments, they are listed as shown. So the file has been restored.

    root@solaris:~# sls -2K /samqfs/bf/dat011
    -rw-rw---- 1 root other        20971704     Mar  5 17:12 samqfs/bf/dat011
    ---------- ----- sI {3,0,0,0}
    -rw-rw---- 1 root other        10485760     Mar  5 17:12 samqfs/bf/dat011/1
    ---------- ----- sS
    -rw-rw---- 1 root other        10485760     Mar  5 17:12 samqfs/bf/dat011/2
    ---------- ----- sS
    -rw-rw---- 1 root other             184     Mar  5 17:12 samqfs/bf/dat011/3
    ---------- ----- sS
    
  13. Set any other required file attributes.

    When you restore a file from a tar file, without a samfsdump or qfsdump recovery point file, the original file attributes are lost. An .inodes file has to be created for the file from scratch, using default attribute values.

  14. The file has now been restored. Delete the unsegmented temporary file.

    In the example, we delete dat011file:

    root@solaris:~# rm /samqfs/bf/dat011file
    root@solaris:~# 
    
  15. Repeat this procedure until all required files have been recovered.

  16. Finish the recovery procedure. Go to "Restoring Archiving File Systems to Normal Operation".

Restore Lost and Damaged Volume Overflow Files

A volume overflow file is a regular file that spans media volumes. Restoring a volume overflow file is thus much like restoring any other regular file. However, you must combine sections of an archive file that resides on multiple volumes into a single archive file on disk before you extract the data file from the archive. So, for each file that you need to recover, proceed as follows:

  1. If you have not already done so, log in to the file-system metadata server as root.

    root@solaris:~# 
    
  2. If you have not already done so, stop archiving and recycling using the procedure in "Stopping Archiving and Recycling Processes"

  3. If you have an archiver log for the period when the volume overflow file was last archived, find the most recent entry for the file. Note the volume serial number(s) of the media, the length of each section of the file, the path and name of the file relative to the root directory of the file system, and the number of sections in the file.

    In the example, we know that the file /samqfs1/rf/rf81 is a volume overflow because it is a regular, type f file that resides on two volumes, VOL036 and VOL034, and has two sections, 0 and 1:

    A 2014/03/03 18:28:51 li VOL036 all.1 12d.1 samqfs1 11731.1 89128448  rf/rf81 f 0 210
    A 2013/08/23 18:28:51 li VOL034 all.1 15f.0 samqfs1 11731.1 525271552 rf/rf81 f 1 220
    
  4. Change to the root directory of the file system that you are restoring.

    SAM-QFS archive files store copies relative to the file-system root directory. So to restore them to their original locations, we want to restore from to the root directory.

    In the example, we change to the root of the samqfs1 file system.

    root@solaris:~# cd /samqfs1
    
  5. Before proceeding, make sure that the file system contains enough free space accommodate a file at least twice the size of the file that you are recovering.

    For the file in the example, rf/rf81, we will need about 1.2 gigabytes of free space, based on the sizes of the two sections of the file: 2 x (89128448 + 525271552) = 1228800000 bytes.

  6. Create a directory in the restored file system to hold temporary archive files.

    In the example, we create the directory /samqfs1/tars

    root@solaris:~# mkdir /samqfs1/tars
    
  7. Position the media at beginning of each archive file that holds an archived copy of one or more of the file segments, and read the archive from the media into memory. Use the command request -m media-type -v volume-serial-number -p 0xposition path/requestfile, where:

    • media-type is one of the two-character media type codes listed in Appendix B

    • volume-serial-number is the six-character alphanumeric code that identifies the media volume

    • position is the hexadecimal starting position that you noted in the archiver log entry

    • path is the path to the temporary recovery directory.

    • requestfile is the name that you want to use for the in-memory tar file that the request command reads from the media.

    In the example, we create two request files. The first request file, /samqfs1/tars/requestVOL036, loads the archive file that starts at position 0x12d on LTO (li) VOL036. The second request file, /samqfs1/tars/requestVOL034, loads the archive file at position 0x15f on LTO (li) VOL034:

    root@solaris:~# request -m li -v VOL036 -p 0x12d /samqfs1/tars/requestVOL036
    root@solaris:~# request -m li -v VOL034 -p 0x15f /samqfs1/tars/requestVOL034
    
  8. Save each of the in-memory tar files that you created to disk as a section of the archive file. Use the command dd if= requestfile of=archive_section, where requestfile is the path and name of the in-memory tar file and archive_section is the path and name of each section of the archive file.

    In the example, we save the request files (in-memory tar files), tars/requestVOL036 and tars/requestVOL034 as tars/archive_part1 and tars/archive_part2:

    root@solaris:~# dd if=tars/requestVOL036 of=tars/archive_part1
    root@solaris:~# dd if=tars/requestVOL034 of=tars/archive_part2
    root@solaris:~# 
    
  9. Re-assemble the sections into a single archive file.

    In the example, we concatenate the two sections, tars/archive_part1 and tars/archive_part2, to create a single archive file, /tars/archive_complete:

    root@solaris:~# cat tars/archive_part1 tars/archive_part2 > \
    tars/archive_complete
    root@solaris:~# 
    
  10. Extract the backup copy of the missing or damaged volume overflow file from the archive (tar) file that you created in the preceding step. Use the command star -xv -f tarfile file, where tarfile is the name of the archive file and file is the path—relative to the file-system root directory—and name of the volume overflow file that you need to restore.

    The star command is an enhanced SAM-QFS version of GNU tar that restores specified files from the archive file that you are pointing to with the request file.

    In the example, we extract the volume overflow file rf/rf81 from the tar file tars/archive_complete:

    root@solaris:~# star -xvf tars/archive_complete rf/rf81
    
  11. Set any other required file attributes.

    When you restore a file from a tar file, without a samfsdump or qfsdump recovery point file, the original file attributes are lost. An .inodes file has to be created for the file from scratch, using default attribute values.

  12. The volume overflow file has now been restored. Delete the temporary file.

    In the example, we delete dat011file:

    root@solaris:~# rm tars/archive_*
    root@solaris:~# 
    
  13. Repeat this procedure until all required files have been recovered.

  14. Finish the recovery procedure. Go to "Restoring Archiving File Systems to Normal Operation".

Recovering Damaged Archive Copies

A damaged archive copy is a copy of a file that cannot be staged back to the disk cache. Sometimes, the file has merely failed to stage due to an intermittent, hardware-related I/O problem and can be easily resolved. At other times, the damaged copy is corrupt and the data is irrecoverable. Your only option in such cases is to recover the file from an alternate copy.

To identify and manage damaged copies, proceed as follows:

  1. Identify files that have damaged archive copies. Use the command sfind mountpoint -any_copy_d, where mountpoint is the directory where the recovered file system is mounted.

    In the example, we start the search in the directory /samqfs1 and find three files that have damaged copies:

    root@solaris:~# sfind /samqfs1 -any_copy_d
    ./genfiles/ab09
    ./genfiles/ab11
    ./genfiles/ay12
    root@solaris:~# 
    
  2. For each file that you identified, identify the damaged copies. Use the command sls -D file, where file is the path and file name that you want to check.

    Damaged copies are flagged with a D. In the example, copy 2 of /samqfs1/genfiles/ab09 and copy 1 of /samqfs1/genfiles/ab11 are damaged:

    root@solaris:~# sls -D /samqfs1/genfiles/ab09
    /samqfs1/genfiles/ab09:
      mode: -rw-r-----  links:   1  owner: root group: other
      length:    306581  admin id: 0  inode:    11748.11
      project: system(0)
      copy 1: ---- Mar 11 13:52       76f.421bc li VOL011
      copy 2: ---D Mar 31 14:02       286.1324f li VOL021
      access:   Mar 11 13:50  modification: Mar 11 13:50
      changed:  Mar 11 13:50  attributes:   Mar 11 13:50
      creation: Mar 11 13:50  residence:    Mar 11 13:50
    root@solaris:~# sls -D /samqfs1/genfiles/ab11
    /samqfs1/genfiles/ab11:
      mode: -rw-r-----  links:   1  owner: root group: other
      length:    380051  admin id: 0  inode:    1460.1
      project: system(0)
      copy 1: ---D Mar 01 10:21       431.21bc6 li VOL024
      access:   Mar 01 10:21  modification: Mar 01 10:21
      changed:  Mar 01 10:21  attributes:   Mar 01 10:21
      creation: Mar 01 10:21  residence:    Mar 01 10:21
    root@solaris:~# 
    
  3. If there is an alternate copy, unarchive the damaged copy. Use the command unarchive -c CopyNumber file, where CopyNumber is an integer representing the copy number and file is the path and file name of the damaged file. Stop here.

    When you unarchive the damaged copy, SAM-QFS stages from the remaining copy and creates an additional archive copy the next time that the archiver process runs. In the example, we have another, undamaged copy of /samqfs1/genfiles/ab09, so we unarchive the damaged copy, copy 2:

    root@solaris:~# unarchive -c 2 /samqfs1/genfiles/ab09
    root@solaris:~# 
    
  4. If you do not have another copy, undamage the damaged copy. Use the command undamage -cCopyNumber file, where CopyNumber is an integer representing the copy number and file is the path and file name of the damaged file.

    Sometimes a file fails to stage due to an intermittent, hardware-related I/O error. Clearing the damage flag and restaging may solve the problem. In the example, there is only one copy of /samqfs1/genfiles/ab11:

    root@solaris:~# undamage -c1 /samqfs1/genfiles/ab11
    
  5. Try to stage the copy. Use the command stage -c CopyNumber -I file, where CopyNumber is an integer representing the copy number and file is the path and file name of the file.

    The optional -I (immediate) parameter pushes the staging operation to the head of the queue:

    root@solaris:~# stage -c 1 -I /samqfs1/genfiles/ab11
    
  6. If staging succeeds, stop here.

  7. If staging failed again, SAM-QFS again sets the damaged flag. Note the major inode number in the output of the sls -D command for the damaged copy.

    In the example, the inode number of the file, /samqfs1/genfiles/ab11, is 1460:

    root@solaris:~# sls -D /samqfs1/genfiles/ab11
    /samqfs1/genfiles/ab11:
      mode: -rw-r-----  links:   1  owner: root group: other
      length:    380051  admin id: 0  inode:    1460.1
      project: system(0)
      copy 1: ---D Mar 01 10:21       431.21bc6 li VOL024
      ...
    root@solaris:~# 
    
  8. Look for possible causes. First, examine the SAM-QFS /var/adm/sam-log file for staging related messages that pertain to the inode of the file with the damaged copy.

    The search can be carried out in various ways. In the example, we list the contents of log file using the Solaris cat command and pipe the output to grep and a regular expression that matches the inode number. We find two messages. Both indicate an I/O error and one explicitly implicates equipment (eq) ordinal number 804, one of our tape drives:

    root@solaris:~# cat /var/adm/sam-log | grep "inode 1460"
    Mar 11 15:35:44 server1 genu-20[8899]: Stage request canceled for inode 1460 (eq 804): I/O error.
    Jan 11 15:35:44 server1 samfs[8894]: /sam4 inode 1460.1 - Archive copy 1 marked damaged: I/O error
    
  9. If the /var/adm/sam-log file implicates a specific SAM-QFS equipment ordinal number, examine the device log, /var/opt/SUNWsamfs/devlog/drive-equipment-number, where drive-equipment-number is the ordinal number listed in the /var/adm/sam-log file.

  10. If the problem appears to be specific to a particular drive, make the implicated drive unavailable to the staging process using the command samcmd unavail drive-equipment-number. Then undamage the copy, and try to stage it again.

    root@solaris:~# samcmd unavail 804
    root@solaris:~# stage -c 1 -I /samqfs1/genfiles/ab11
    root@solaris:~# undamage -c1 /samqfs1/genfiles/ab11
    root@solaris:~# 
    
  11. If staging fails again or if no single drive appears to be at fault, try to recover the copy using the request and star commands, as described in "Recovering Files Using Archiver Log Entries", or Solaris utilities such as tar and dd.

  12. If you still cannot recover the file and if the value of the data warrants it, engage a data recovery service. For assistance with Oracle StorageTek tape media, engage Oracle StorageTek Enterprise Tape Data Recovery services. Log in to My Oracle Support at support.oracle.com. Open a Service Request, select the tape drive model from the list under the request category, and select Media Issues from the list under subcategory.

  13. If the file proves to be irrecoverable, unarchive the damaged copy. Use the command unarchive -c CopyNumber file, where CopyNumber is an integer representing the copy number and file is the path and file name of the damaged file.

    root@solaris:~# unarchive -c 1 /samqfs1/genfiles/ab11
    root@solaris:~# 
    
  14. Resolve any drive or media issues that are revealed by the log files.

  15. If you disabled archiving, staging, and recycling processes in a previous step, re-enable them now. Go to "Restoring Archiving File Systems to Normal Operation".

  16. Otherwise, stop here.