A P P E N D I X  C

Compliance Archiving Software API

The NAS software supports strict compliance-archiving guidelines as a license-key-enabled software extension called "Sun StorageTek Compliance Archiving Software.

The Compliance Archiving Software is available in a stringent form (referred to as mandatory) and in a less stringent form (referred to as advisory). For overview information about the Compliance Archiving Software, see About Compliance Archiving Software.

This appendix is a technical overview of the features and application programming interface (API) for the strict Compliance Archiving Software. It contains the following sections:

For Sun StorageTek 5310 and Sun StorageTek 5320 NAS appliances and gateway systems, proper operation of the Compliance Archiving Software requires the correct physical configuration of the NAS appliance or gateway-system hardware. In particular, the redundant array of independent disks (RAID) controller must not be connected to any device or network, other than a private Fibre Channel connection to the NAS server, and (for non-gateway configurations), connections to any expansion units. There are no such requirements for Sun StorageTek 5210 NAS appliances.

To ensure the strongest possible enforcement of your data retention policies, you must also provide for the physical security of your NAS device. Software-controlled data retention can be no stronger than the physical safeguards used to control access to the system's hardware.


Compliance Features

The Compliance Archiving software operates on file volumes that have been created as compliance-enabled. Its functionality consists of these major features:

For test environments or for deployments with less stringent requirements, the Compliance Archiving Software provides the option of advisory enforcement that overrides some of these features.

With the standard mandatory enforcement, no one can delete a WORM file before its retention date, decrease a WORM file's retention time, or delete a compliance volume. Under the advisory enforcement option, authorized administrator can decrease a WORM file's retention time and delete a WORM file before its retention date. These operations are logged in the audit log.



Note - File volumes that are compliance-enabled might have slightly lower performance than volumes without this protection.




WORM Files

The term "WORM" means "write-once, read-many" and indicates that the file is archived in non-rewritable, non-erasable storage. A more accurate description is to call these files "permanent read-only" files.

A file can be created with the normal access controls and modified as needed, but after it becomes a WORM file, the Compliance Archiving software enforces stronger access controls than the traditional file access semantics provided by the NFS and CIFS protocols.

When a data management application designates a file as WORM, the file becomes permanently immutable. WORM files cannot be modified, extended, or renamed. A WORM file can be deleted only when its retention time has been met and in accordance with the file retention rules.

In addition to providing storage for WORM files, the Compliance Archiving System supports backup to immutable tape media, or WORM tape.



Note - Checkpoint files cannot be restored over write-once, read-many (WORM) files.




File Retention Periods

The Compliance Archiving Software associates a retention period with each WORM file. If you or the data management application that writes files to the volume does not set a retention period explicitly for each file, the default retention period is used.

When the retention period expires, you can delete a WORM file or extend its retention period. With the advisory compliance option, you can decrease the retention period for a file to allow it to be deleted. With the mandatory compliance option, you cannot decrease the retention period.



caution icon

Caution - If you or the data management application that writes files to the volume does not set a retention period explicitly for each file before making the file WORM, the default retention period for the volume is used. You can change the volume's default, but under mandatory compliance, this default retention period is permanent.




Administrative Lock-Down

Some system administration functions are disabled or restricted on compliance-enabled file volumes to ensure the retention and preservation assurances of WORM files and retention periods. These restrictions affect functions that could be used to circumvent a file's retention, for example, by deleting the file's volume.


Compliance Audit

The Compliance Archiving Software retains immutable records of all compliance-related activities that occur on the system. It maintains a text-based log file for attempted efforts to modify or delete data, with or without proper authority, and is enabled through the use of the Data Retention Audit Service (DRAS) API, which includes the following features:

The following events are audited:

A full description of the audit log provided by this service is in Chapter 9.


Accessing Compliance Functionality

To maintain compatibility with existing client operating systems and applications, the Compliance Archiving Software features are implemented as extensions to the existing file access protocols supported by the NAS appliance or gateway system (NFS and CIFS). In particular, the NAS device overloads existing file attributes to indicate the WORM status of a file and the end of its retention period. This simplifies the porting of existing document and record management applications because these metadata fields can be set and viewed using standard client APIs and utilities.


Compliance Volumes

Volumes must be designated as compliance-enabled at the time they are created; existing volumes cannot be converted into compliance volumes. It is possible to have multiple volumes on a single NAS appliance or gateway system, where only some of which are compliance-enabled.

Do not enable compliance archiving on volumes that will be used by applications (and users) that are not aware of the different data retention semantics enforced by the Compliance Archiving Software.


WORM Files

WORM files cannot be modified or updated. After a file becomes a WORM file, it is read-only until it is removed.

Creating WORM Files

The Compliance Archiving Software uses a WORM trigger to convert a normal file into a WORM file. When a client application or user executes the trigger action on a file, the Compliance Archiving Software interprets this to mean that the target file must be converted to a WORM file.

The WORM trigger for Unix clients is setting a file's permission mode to 4000. Client applications or users can invoke this WORM trigger using the chmod command or system call. On receiving this request, the Compliance Archiving Software converts the target file into a WORM file by doing the following:

Note: Executable files cannot be made into WORM files. For files created from Windows clients, this means that a file cannot be made into a WORM file if its access control list (ACL) has any access control entries (ACEs) granting execute permission on the file.

In the following example, a file with an access mode of 640 is converted to a WORM file. After the WORM trigger is issued, the file's access mode is 4440.

$ ls -l testfile
-rw-r-----   1 smith    staff      12139 Dec  2 13:18 testfile
$ chmod 4000 testfile
$ ls -l testfile
-r-Sr-----   1 smith    staff      12139 Dec  2 13:18 testfile

The Compliance Archiving Software uses this WORM trigger because it is an operation that is unlikely to be used by existing applications.

The WORM trigger for Windows clients is setting both the read-only and the system bit on a file. Setting these bits will only trigger WORM if neither the archive nor hidden bits are set on the file. The WORM trigger sets the file's read-only bit, but does not change its system bit. Use the following command to enable the WORM trigger:

hostname> fsctl compliance wte on

After a file becomes WORM, it cannot be changed back. From Windows clients, the read-only bit cannot be cleared and the system bit cannot be changed. From Unix clients, the setuid bit cannot be cleared nor can execute or write permissions be added to the file's access mode.

Compliance-enabled volumes translate these WORM settings between CIFS and NFS. For example, if a Unix client views a WORM file created by a Windows client, it sees a WORM access mode as described above.

Behavior of WORM Files

WORM files cannot be modified, overwritten, or extended. Any attempt to write to a WORM file will fail and return an error regardless of the client user's identity and access privileges.

Neither the owner of a WORM file nor a user with administrative privileges (even root privileges) can modify a WORM file. WORM files cannot be renamed or changed back to regular (non-WORM) files.

Metadata of WORM Files

The Compliance Archiving Software doesn't allow metadata that contains, protects, describes, or names client data to be modified. Only a restricted subset of metadata fields are allowed to change, depending on operating system, as shown in TABLE C-1.


TABLE C-1 WORM File Metadata That Can and Cannot Be Modified

Operating System

Can

Cannot

Unix

  • Set or clear read permission bits
  • Change file and group owner
  • Enable write and execute bits
  • Clear setuid bit
  • Modify size or modification time (mtime)

Windows

  • Set or clear read permission bits
  • Change archive bit
  • Create and modify access control lists (although a WORM file can never be modified regardless of ACL settings)
  • Change the read-only, system, or hidden bits
  • Modify size or modification time (mtime)

WORM Restrictions

The Compliance Archiving Software does not allow WORM files to be renamed. Furthermore, non-empty directories cannot be renamed. This rule guarantees that the full pathname of a WORM file cannot change for the lifetime of the file.

When a Unix client sets a file mode to 4000 (invoking the WORM trigger), the resulting access mode on the file will not be 4000. This violates the standard semantics of the chmod command and system call. As a result, the GNU version of the chmod(1) command used by many Linux distributions generates a warning message when it is used to issue the WORM trigger. You can ignore this message.


File Retention Periods

Each WORM file has a retention period during which it cannot be deleted. Any attempt to remove a WORM file prior to the end of its retention period will fail.

Compliance Archiving retention timestamps are stored in the access time (atime) attribute of WORM files.

Note: Because the access time (atime) attribute is used by the Compliance Archiving Software to store retention timestamps, that attribute is not updated as a side-effect of standard file-system operations, regardless of whether a file is a WORM file.

Note: Retention periods only govern the ability to remove files. A WORM file can never be modified, regardless of whether its retention period has expired.

Clients typically set the atime attribute prior to changing a file to be read-only. When a file becomes a WORM file, its atime value is rounded down to the nearest number of seconds to determine the retention timestamp.

If the client user or application does not specify a retention period, or the atime attribute represents a time in the past, the Compliance Archiving Software uses the default retention period specified for the volume when it was created.

To specify that a permanent retention, set the file's atime to the maximum legal value for a signed 32-bit integer (0x7fffffff). On Unix systems, this value is defined as INT_MAX in the limits.h header file. and translates to a timestamp of 03:14:07 GMT, Jan 19, 2038.

You can extend retention periods, and set new retention periods for files whose retentions are expired, as long as the new value represents a time later than the old retention timestamp. To extend the retention, reset the atime attribute on the WORM file.

Client applications and users can determine the retention status of a file by reading the file's attributes, using standards tools and APIs. Unix clients, for example, can read a file's attributes using the stat(2) system call. Unix users can view a file's attributes using the ls-lu command, which lists files with their access permissions and atime timestamps.


Unix System Calls with Compliance Archiving

Unix client applications access the Compliance Archiving Software through their local system call interface. These calls invoke the client NFS implementation, which translates system calls into standard NFS protocol requests. Because compliance-enabled file systems behave differently than standard NAS file systems, there are corresponding differences in the behavior of the client system calls.

This section describes the standard Unix system calls that behave differently when a client executes them on a compliance-enabled NAS share. System calls not listed here behave as normal.

It is important to remember that the interfaces to the NAS appliance or gateway system are the NFS and CIFS file access protocols. Thus, this section incorporates both the compliance-related behavior of the NAS device in response to standard protocol requests, and the mapping from system calls to NFS requests. The behavior of these calls has been verified on Solaris clients and must be the same on other Unix clients.


access(2)

Any check for write permission on a WORM file (that is, a call to access(2) where the amode argument includes the W_OK bit) fails and returns an error (EPERM).


chmod(2), fchmod(2)

If the target file is a regular, non-WORM file with none of the execute permission bits set, and the new access permission is 4000 (S_ISUID), then the target file becomes a WORM file. When this happens, the file receives a new access mode that is computed by adding the setuid bit to any existing read bits in the file's access mode. More specifically, given an old access mode, oldmode, a file's new access mode after receiving the WORM trigger can be computed as:

newmode = S_ISUID | (oldmode & 0444)

Executable files cannot be converted to WORM. Applying the WORM trigger (mode 4000) to a file with one or more execute permission bits fails and returns an error (EACCES).

Read access bits can be set or cleared on WORM files. Any attempt to enable write or execute permission on a WORM file, to set the setgid bit (S_ISGID) or sticky bit (S_ISVTX), or to clear the setuid bit on a WORM file fails and returns an error (EPERM).


chown(2), fchown(2)

These calls behave the same on WORM files as on non-WORM files.


link(2)

Clients can create new hard links to WORM files. Hard links to a WORM file cannot be removed until the file's retention period ends. (See unlink(2), on "Invalid Cross-Reference Format".)


read(2), readv(2)

Clients can read WORM files. Because retention timestamps are stored in the atime attribute, this value is not updated to reflect read access to WORM files.


rename(2)

Any attempt to rename a WORM file or a non-empty directory on a compliance-enabled file system fails and returns an error (EPERM).


stat(2), fstat(2)

When these calls are used to obtain information about regular files, the returned stat structure contains compliance-related values. The st_mode field contains (as always) the file's mode and permissions. A WORM file has the setuid bit set and no write or execute bits. The st_atime field contains a timestamp indicating the end of the file's retention period. If this value is equal to INT_MAX, as defined in limits.h, then the file is retained permanently.


unlink(2)

WORM files can only be unlinked if the current time, reflected by the NAS appliance or gateway system secure clock, is later than the date stored in the file's atime attribute (that is, the retention timestamp). If this condition does not hold, unlink(2) fails and returns an error (EPERM).


utime(2), utimes(2)

These calls are used to set a file's access time (atime) and modification time (mtime) attributes. When used on a non-WORM file, they behave normally and provide a mechanism for specifying the retention timestamp before a file is converted to WORM.

When invoked on a WORM file, these calls can be used to extend the file's retention period or to assign a new retention period to a file with expired retention. These calls succeed on a WORM file if the new atime value is greater than (that is, after) the file's existing atime value. If the new atime value is less than or equal to the current atime value, these calls fail and return an error (EPERM). When used on a WORM file, the mtime argument is ignored.


write(2), writev(2)

Any attempt to write to a WORM file fails and returns an error (EPERM).


Behavior of Windows Clients

This section describes the differences in compliance-enabled files for Windows clients.


Creating WORM Files

A regular, non-WORM file can be converted to a WORM file from Windows only if its archive and hidden bits are not set. If these bits are cleared, a Windows client converts
the file to a WORM file by setting its read-only and system bits. This WORM trigger will result in setting the file's read-only bit, but will not change the state of the file's system bit.


Metadata Restrictions on WORM Files

Windows clients can change the archive bit on a WORM file but they cannot change the read-only, hidden, or system bits. Windows clients can change ACLs on WORM files, but any write permissions in the ACL of a WORM file is ignored. Any attempt to modify the data in a WORM file fails, regardless of the permissions in the ACL.


WORM File's Read-Only Bit

It is especially important that compliance-enabled file volumes only be used by Windows applications and users that are aware of the special behavior of WORM files. Many standard Windows utilities for copying files include the read-only and system bits on a file. If these tools are used to make copies of WORM files on a compliance-enabled volume, the resulting files might become WORM files because their read-only and system bits set.


Compliance and Antivirus Software

When you enable antivirus protection on a compliance-enabled volume, the following cases are handled in a special manner:

For more information about virus scanning, see Chapter 4.

Many virus-checking programs attempt to preserve the access time on the files they examine. These programs read a file's atime before checking it for viruses, and afterwards, reset the atime to the value it had before the scan. This can lead to a race condition if the virus-checking program scans a file at the same time that another application is setting a retention time on the file. As a result, the file might have the wrong retention time.

To avoid this problem, make sure that virus-checking software does not run at the same time as applications that create WORM files.

Custom applications can also avoid this issue by using a short default retention period and setting a file's true retention period after applying the WORM trigger.


Other APIs

The Compliance Archiving Software can be accessed through many other client APIs, such as Java, Perl, and C++. All of these languages rely on the same underlying system calls to access shares mounted through NFS or CIFS.