C H A P T E R  6

Recycling

Recycling is the process of reclaiming space on archive volumes. The recycler works with the archiver to reclaim the space occupied by unused archive copies. As users modify files, the archive copies associated with the old versions can be purged from the system. The recycler identifies the volumes with the largest proportions of expired archive copies and directs the movement of unexpired copies to different volumes. If only expired copies exist on a given volume, a site-defined action is taken. For example, such a volume can be relabeled for immediate reuse or exported to offsite storage, thus keeping a separate historical record of file changes. Users are unaware of the recycling process as it relates to their data files.

This chapter describes the recycling process and directives. It includes the following topics:


Recycling Process Overview

The recycler keeps the amount of space consumed by expired archive copies to a minimum as defined by site-specified parameters. At any time, the space on a given archive volume consists of the following:

The capacity of a volume is the total amount of space for data on a volume. For example, a 10-gigabyte tape volume with 3 gigabytes written to it has a capacity of 10 gigabytes and 7 gigabytes of free space.

New or newly labeled archive media starts out with all its capacity as free space. As data is archived to the media, the amount of free space decreases and the amount of current data increases.

As archived files in the file system are changed or removed, their archive images expire and they move from the current data classification to the expired data classification. The physical space used by these images remains the same; there is simply no longer a file in the file system pointing to that space. Only when space is recycled can these images be removed and the space they occupy become free. The goal of the recycler is to transform space used by expired data into free space without losing any current data.

The recycler and the archiver work together, as follows:

1. The recycler marks all the current (valid) archive images that are present on a volume with the rearchive attribute.

2. If you are archiving to removable media, the recycler marks the selected archive volume with the recycle attribute. This prevents the archiver from writing any more archive images to the volume.

3. The archiver moves all the marked images to another volume. This operation is called rearchiving. After the archiver moves the current archive images from the old volume to the new volume, the old volume contains only free space and expired space. If you are archiving to removable media cartridges, you can relabel and reuse the cartridge. If you are archiving to disk, the recycler removes the file that contains the expired archive images.

The recycler is designed to run periodically. It performs as much work as it can each time it is invoked. The recycler has to finish marking copies for rearchiving before the archiver can rearchive the files.

Sometimes expired archive images, with the rearchive attribute set, remain on media. This can happen under the following conditions:

Between executions, the recycler keeps state information in the library catalogs and the inodes. During the recycling process, you can use the sls(1) command and its -D option to display information about a file. The output from the sls(1) command shows whether or not a file is scheduled for rearchiving.

Before configuring the recycler, note the following:



Caution - Extreme care must be taken when configuring the recycler if you are using disk archiving in an environment with multiple SAM-QFS servers. The diskvols.conf file for each SAM-QFS server must point to a unique set of disk volume resource specifications (disk archiving target directories). If any of these are shared between different SAM-QFS servers, then running the recycler from one SAM-QFS server will destroy the disk archive data that is being managed by the other SAM-QFS server.



The recycler is not enabled by default. You must initiate recycling by entering the sam-recycler(1M) command.


Recycling Tape Libraries

You initiate recycling for tape libraries by entering the sam-recycler(1M) command either manually or through a cron(1) job. TABLE 6-1 shows recycling methods.


TABLE 6-1 Recycling Methods and Media Types

Recycling Method

Media and Notes

By automated library

Removable media cartridges.

When you archive by library, you put recycling directives in the recycler.cmd file.

By archive set

Removable media cartridges and disk.

When you archive by archive set, you put all recycling directives in the archiver.cmd file.


As TABLE 6-1 shows, you can recycle either by library or by archive set. If you are archiving to disk, you can recycle only by archive set.

Using Recycling Directives

The recycler.cmd file accepts the directives described in the following sections:

Specifying a Log File: The logfile Directive

The logfile directive specifies a recycler log file. This directive has the following format:


logfile = filename

For filename, specify the path to the log file.

The following is an example of a logfile= directive line:


logfile=/var/adm/recycler.log

Preventing Recycling: The no_recycle Directive

The no_recycle directive enables you to prevent recycling of volumes. To specify the VSNs, you use regular expressions and one or more specific media types.

This directive has the following format:


no_recycle media-type VSN-regexP [ VSN-regexP ... ]

TABLE 6-2 Arguments for the no_recycle Directive

Argument

Meaning

media-type

A media type from the mcf(4) man page.

VSN-regexp

One or more space-separated regular expressions to describe the volumes. For information, see the regexp(5) man page or see File Name search-criterion Using Pattern Matching: -name regex.


 

By specifying a media type, you can prevent the recycling of volumes stored on a particular type of media. One or more VSN-regexp specifications enable you to identify specific cartridges to be excluded from recycling.

For example, the following directive line excludes from recycling any tape volumes whose VSN identifiers begin with DLT:


no_recycle lt DLT.*

Specifying Recycling for an Entire Automated Library: The Library Directive

The library directive enables you to specify various recycling parameters for the VSNs associated with a specific library.

This directive has the following format:


library parameter [ parameter ... ]

For library, specify the library's name as specified in the family set field of the mcf file.

For parameter, specify one or more space-separated parameter keywords from TABLE 6-3.


TABLE 6-3 Library Directive parameter Values

parameter Value

Meaning

-dataquantity size

Maximum amount of data that the recycler can schedule for rearchiving in its efforts to clear volumes of useful data. Default is 1 gigabyte.

-hwm percent

Library high-water mark. Default is 95.

-ignore

Directive that prevents volumes in this library from being recycled. This directive is useful during testing of the recycler.cmd file.

-mail email-address

Email addresses to which recycling email messages are to be sent. By default, no email is sent.

-mingain value

Minimum VSN gain. Default is 50.

-vsncount count

Maximum number of recycled volumes to be counted. Default is 1.


For example, consider the following directive line:


gr47 -hwm 85 -ignore -mail root -mingain 40

It specifies the following for library gr47:

In addition, no more than 1 gigabyte is to be rearchived. This is the default, so it is not specified in the recycler.cmd file.

Configuring the Recycler

When the recycler is initiated, the default recycler settings specified in Specifying Recycling for an Entire Automated Library: The Library Directive take effect. For more information on the recycler, see the sam-recycler(1M) man page.

Configuring the recycler for a tape library involves the tasks described in the following sections, in this order:

1. Creating a recycler.cmd File

2. Editing the archiver.cmd File (This step is optional for recycling tape libraries)

3. To Run the Recycler

4. Creating a crontab File for the Recycler

5. To Remove -recycle_ignore and ignore Parameters

6. Creating a recycler.sh File

Creating a recycler.cmd File

Create a recycler.cmd file if you are recycling archive copies on cartridges in a library.



Note - Even if you are recycling by archive set, you still should configure each library in the recycler.cmd file. This ensures that VSNs that do not fall into an archive set can be recycled if needed.



The recycler.cmd file contains general recycling directives. It can also contain directives for each library in the Sun StorageTek SAM environment. A typical recycler.cmd file contains the following directive lines:

Because you are creating the recycler.cmd line and it has not yet been tested, use the ignore keyword. You remove the ignore keyword in a later step in this process.


procedure icon  To create a recycler.cmd file

1. Become superuser.

2. Use vi(1) or another editor to open file /etc/opt/SUNWsamfs/recycler.cmd.

3. Add one or more directives described in this chapter to control recycler activity.

4. Save and close the file.

Alternatively, you can create a recycler.cmd file by using File System Manager software. For more information, see the File System Manager online help.

Example recycler.cmd File

CODE EXAMPLE 6-1 shows an example of a recycler.cmd file.


CODE EXAMPLE 6-1 Example recycler.cmd File
logfile = /usr/tmp/recycler.log
stk30  -hwm 51  -mingain 60  -ignore  -mail  root

The following sections describe the parameters specified in CODE EXAMPLE 6-1.

The -hwm 51 Parameter

By specifying a high-water mark, you can set the percentage of media usage below which recycling cannot occur. This percentage is the ratio of the used space in the library to its total capacity. As an example, a library that holds ten 20-gigabyte tapes, three of them 100 percent full and the remaining seven each 30 percent full, has the following media utilization percentage:


((3* 1.00 + 7 * 0.30) * 20G ) / ( 10 * 20G ) * 100%= 51%

Note that this calculation does not distinguish between current data and expired data. It only addresses the amount of media used.

In this example, if the utilization percentage is 51 percent or less, the recycler does not automatically select any of the automated library's VSNs for recycling.



Note - You can force a VSN to be recycled by using the following command to set the recycling flag:

# chmed +c lt.VSN

When the +c flag is set, the archiver does not write any more archive images to the volume. The +c flag can be viewed through the samu(1M) utility. For more information, see the chmed(1M) and samu(1M) man pages. For information on using the samu(1M) operator utility, see the Sun StorageTek QFS File System Configuration and Administration Guide.



The -mingain 60 Parameter

The minimum VSN gain percentage sets a lower limit on the amount of space to be gained by recycling a cartridge. For example, if a cartridge in an automated library is 95 percent current data and 5 percent expired data, the gain obtained by recycling the cartridge would be only 5 percent. It might not be worth moving the other 95 percent to retrieve this space. Setting the minimum gain to 6 percent or more inhibits the recycler from automatically selecting this VSN.

The -ignore Parameter

The -ignore parameter keeps the recycler from recycling a particular library. Use it when you are configuring the recycler.

The -mail Parameter

The -mail parameter specifies that the recycler send email when recycling occurs on a given library. The email message has the following subject line:


Robot robot-name recycle

TABLE 6-2 shows sample message bodies.


CODE EXAMPLE 6-2 Sample Recycling Messages
I will recycle VSN vsn.
Cannot find any candidate VSN in this media changer.
Previously selected VSN vsn is not yet finished recycling.
Previously selected VSN vsn is now finished recycling. It will now be post-recycled.


procedure icon  To Run the Recycler

Follow these instructions to manually run the recycler.

1. Issue the sam-recycler(1M) command.

The recycler reads the recycler.cmd file.

2. Examine the standard output log, Sun StorageTek SAM log, and /var/adm/messages for any error messages from the recycler.

Correct your files if errors appear.

CODE EXAMPLE 6-3 shows a sample recycler log file for recycling removable media cartridges.


CODE EXAMPLE 6-3 Recycler Log File Example for Removable Media Cartridges
========== Recycler begins at Wed Dec 12 14:05:21 2001 ===========
Initial 2 catalogs:
0  Family: m160                 Path: /var/opt/SUNWsamfs/catalog/m160
   Vendor: ADIC                 Product: Scalar 100
   SLOT                  ty    capacity         space vsn
      0                  at        25.0G        25.0G CLN005
      1                  at        48.5G         6.1G 000003
      2                  at        48.5G        32.1G 000004
      3                  at        48.5G        35.1G 000005
      4                  at        48.5G        44.6G 000044
      5                  at        48.5G        45.1G 000002
      6                  at        48.5G        45.9G 000033
      7                  at        48.5G        48.5G 000001
   Total Capacity:  364.8G bytes, Total Space Available: 282.3G bytes
   Volume utilization 22%, high 95% VSN_min 50%
   Recycling is ignored on this robot.
1  Family: hy                   Path: /var/opt/SUNWsamfs/catalog/historian
   Vendor: Sun SAM-FS           Product: Historian
   SLOT                  ty    capacity         space vsn
      (no VSNs in this media changer)
   Total Capacity:  0    bytes, Total Space Available: 0    bytes
   Volume utilization 0%, high 95% VSN_min 50%
   Recycling is ignored on this robot.
8 VSNs:
                    ---Archives---   -----Percent-----   m160
 ----Status-----    Count    Bytes   Use Obsolete Free   Library:Type:VSN
no-data VSN            0      0        0    87     13    m160:at:000003
no-data VSN            0      0        0    33     67    m160:at:000004
no-data VSN            0      0        0    27     73    m160:at:000005
no-data VSN            0      0        0     8     92    m160:at:000044
no-data VSN            0      0        0     7     93    m160:at:000002
no-data VSN            0      0        0     5     95    m160:at:000033
empty VSN              0      0        0     0    100    m160:at:CLN005
empty VSN              0      0        0     0    100    m160:at:000001
Recycler finished.
========== Recycler ends at Wed Dec 12 14:05:32 2001 ===========

Creating a crontab File for the Recycler

If the system is performing as expected, you can make a crontab entry for the superuser to run the recycler periodically. The frequency you choose depends on your site's conditions.

For instructions on creating a crontab entry, see the cron(1M) man page.

The following example entry in root's crontab file specifies that the cron daemon run the recycler every five minutes after the hour for every odd-numbered hour:


5 1,3,5,7,9,11,13,15,17,19,21,23   * * * /opt/SUNWsamfs/sbin/sam-recycler


procedure icon  To Remove -recycle_ignore and ignore Parameters

If you have used the -recycle_ignore or ignore parameter to disable recycling during configuration, use this procedure to remove them. As an alternative to the following procedure, you can use the File System Manager. For more information, see the File System Manager online help.

1. Use vi(1) or another editor to remove the -recycle_ignore parameters from the archiver.cmd file.

2. Use vi(1) or another editor to remove the ignore parameters from the recycler.cmd files.

You are now recycling.

Creating a recycler.sh File

Create a recycler.sh file if you are recycling archive copies on removable media cartridges. If you are archiving only to disk, do not perform this step.

The recycler executes the recycler.sh script when all the current images from a VSN have been rearchived to another VSN. You should determine your site requirements for dispensing with recycled cartridges. Some sites choose to relabel and reuse the cartridges; others choose to remove the cartridges from the automated library to use later for accessing historical files.

The recycler calls the /opt/SUNWsamfs/scripts/recycler.sh script with the following arguments:


Media type: $1  VSN: $2  Slot: $3  Eq: $4

For examples of the script, see the recycler.sh(1M) man page or /opt/SUNWsamfs/examples/recycler.sh. The latter shows how to relabel a recycled VSN and send mail to the superuser.

For more information, see the recycler(1M) and recycler.sh(1M) man pages.


Recycling Disk Archive Volumes

Configuring the recycler for recycling disk archive volumes involves the tasks described in the following sections, in this order:

1. Editing the archiver.cmd File

2. To Run the Recycler

3. Creating a crontab File for the Recycler

4. To Remove -recycle_ignore and ignore Parameters

Editing the archiver.cmd File

If you are archiving to disk, you must edit the archiver.cmd file in order to recycle.

If you are recycling by library, this step is optional.

To edit the archiver.cmd file, follow the steps described in To Create or Modify an archiver.cmd File and Propagate Your Changes.

Alternatively, you can edit the archiver.cmd file by using the File System Manager. For more information, see the File System Manager online help.

To enable recycling by archive set, you must add archive set recycling directives between the params and endparams directives. TABLE 6-4 shows the archive set recycling directives that you can use.


TABLE 6-4 Archive Set Recycling Directives

Directive

Function

-recycle_dataquantity size

Limits the amount of data the recycler schedules for rearchiving in order to clear a disk volume of useful data. By default, a limit is ignored for disk archive recycling.

-recycle_ignore

Prevents the archive set from being recycled.

-recycle_mailaddr mail-address

Sends recycler messages to the specified email address.

-recycle_mingain percent

Limits recycling of volumes in the archive set by setting the mingain mark for a disk volume. The mingain is expressed as a percentage of the expired data associated with the volume. When the expired data of the volume exceeds the mingain percentage, the recycler begins to recycle the volume. The default is 50%.

-recycle_minobs percent

Limits the recycler's selection of tar files in volume by setting a threshold for the recycler's rearchiving process of disk archive volumes. When the percentage of expired files within an archived tar file on the disk reaches this threshold, the recycler begins moving the current files from the archive into a new tar file. Once all the current files have been moved, the original tar file is marked as a candidate to be removed from the disk archive. The default is 50%.

-rearch_stage_copy copy-number

Allows staging for rearchiving to take place from selected (faster) copies.


For more information about archiver directives, see Chapter 3 or see the archiver.cmd(4) man page.


procedure icon  To Run the Recycler

Follow these instructions to manually run the recycler.

1. Issue the sam-recycler(1M) command.

The recycler reads the recycler.cmd file.

2. Examine the standard output log, Sun StorageTek SAM log, and /var/adm/messages for any error messages from the recycler.

Correct your files if errors appear.

CODE EXAMPLE 6-4 shows a sample recycler log file for recycling disk archive files.


CODE EXAMPLE 6-4 Recycler Log File Example for Disk Archive Files
---Archives---   -----Percent-----
 ----Status-----    Count    Bytes   Use Obsolete Free   Library:Type:VSN
new candidate          0      0        0    41     59  <none>:dk:disk01
677 files recycled from VSN disk01 (mars:/sam4/copy1)
0 directories recycled from VSN disk01 (mars:/sam4/copy1)

Creating a crontab File for the Recycler

If the system is performing as expected, you can make a crontab entry for the superuser to run the recycler periodically. The frequency depends on your site's conditions.

For instructions on creating a crontab entry, see the cron(1M) man page.

The following example entry in root's crontab file specifies that the cron daemon run the recycler every five minutes after the hour for every odd-numbered hour:


5 1,3,5,7,9,11,13,15,17,19,21,23   * * * /opt/SUNWsamfs/sbin/sam-recycler


procedure icon  To Remove -recycle_ignore and ignore Parameters

If you have used the -recycle_ignore or ignore parameter to disable recycling during configuration, use this procedure to remove them. As an alternative to the following procedure, you can use the File System Manager. For more information, see the File System Manager online help.

1. Use vi(1) or another editor to remove the -recycle_ignore parameters from the archiver.cmd file.

2. Use vi(1) or another editor to remove the ignore parameters from the recycler.cmd files.

You are now recycling.


Recycling for Archive Copy Retention

As an alternate to the normal recycling process, the sam-nrecycler(1M) tool can be used to work in conjunction with the File System Manager's backup and recovery point features. This tool removes expired archive copies and frees up archive volumes to aid in the ability to use Sun StorageTek SAM dump files for archive retention. If you want to take advantage of this functionality, you must use this recycler in place of the existing sam-recycler command.

The sam-nrecycler(1M) tool scans file system metadata and Sun StorageTek SAM dump files to determine which removable media and disk archive volumes contain archive images. It can be invoked through the crontab(1) file at an off-peak time, or invoked at any time using the sam-nrecycler command. The nrecycler identifies all archive images present on a removable media volume or in a disk archive tar file by scanning all file system .inodes files and specified Sun StorageTek SAM dump files. By scanning the file systems and Sun StorageTek SAM dump files, the nrecycler can determine if there are volumes that do not contain any archive images, and the space on these volumes can be reclaimed. If a removable media volume does not contain any archive images, it is safe to relabel the cartridge. If a disk archive tar file does not contain any archive images, it is safe to remove the tar file from the disk archive directory.

Directives for sam-nrecycler(1M) must be provided using the /etc/opt/SUNWsamfs/nrecycler.cmd file. You must specify a path to the directories that contain the Sun StorageTek SAM dump files to be searched. If no directories are specified in the command file, recycling does not occur. The list of directories must be complete and all Sun StorageTek SAM dump files must be contained in the directory list.

You can also include a logfile= directive line in the nrecycler.cmd file to specify an nrecycler log file. The system writes recycling messages and recycling reports to this file.

When sam-nrecycler(1M) detects that a removable media volume contains only free or expired space and is safe to relabel, it invokes the sam-nrecycler.sh script. The script can relabel the cartridge using either the original VSN or a new VSN. It can then export the cartridge from the library, or it can perform another user-defined action.

When sam-nrecycler detects that a disk archive volume contains only free or expired space, it unlinks the unused disk archive tar file.

For more information about sam-nrecycler(1M), see the sam-nrecycler(1M) man page.