JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Sun ZFS Storage 7000 System Administration Guide
search filter icon
search icon

Document Information

Preface

1.  Introduction

2.  Status

3.  Configuration

4.  Services

Services

Introduction

Data Services

Directory Services

System Settings

Remote Access

Security

BUI

Viewing a Specific Service Screen

Enabling a Service

Disabling a Service

Defining Properties

Viewing Service Logs

CLI

Selecting a Service

Viewing a Service's State

Enabling a Service

Disabling a Service

Setting Properties

Viewing Service Logs

Service Help

NFS

Introduction

Properties

Kerberos Realms

Logs

Analytics

CLI

Tasks

NFS Tasks

iSCSI

Introduction

Properties

Authentication

Authorization

Targets and Initiators

CLI

Tips

Troubleshooting

SMB

Introduction

Properties

Share Properties

NFS/SMB Interoperability

DFS Namespaces

Autohome Rules

Local Groups

Local Accounts

MMC Integration

Event Viewer

Share Management

Users, Groups and Connections

Services

CLI

Adding autohome rules

Adding a user to a local group

Tasks

SMB Tasks

FTP

Introduction

Properties

FTP Properties

General Settings

Security Settings

Logs

Tasks

FTP Tasks

HTTP

Introduction

Properties

Authentication and Access Control

Logs

Tasks

HTTP Tasks

NDMP

Introduction

Local vs. Remote Configurations

Backup Formats and Types

Backing up with "dump" and "tar"

Backing up with "zfs"

Incremental backups

Properties

Logs

SFTP

Introduction

Properties

SFTP Port

Logs

Tasks

SFTP Tasks

Virus Scan

Introduction

Properties

File Extensions

Scanning Engines

Logs

Tasks

Virus Scan Tasks

NIS

Introduction

Properties

Logs

Tasks

NIS Tasks

LDAP

Introduction

Properties

Custom Mappings

Logs

Tasks

LDAP Tasks

Active Directory

Introduction

Properties

Join Domain

Join Workgroup

Domains and Workgroups

LDAP Signing

Windows Server 2008 Support

Section A: Kerberos issue (KB951191)

Section B: NTLMv2 issue (KB957441)

Section C: Note on NTLMv2

BUI

CLI

Tasks

Active Directory Tasks

Identity Mapping

Concepts

Identity Mapping Concepts

Mapping Modes

IDMU

Directory-based Mapping

Identity Mapping Directory-based Mapping

Properties

Name-based Mapping

Identity Mapping Name-based Mapping

Name-based Mapping Rules

Case Sensitivity

Mapping Persistence

Domain-Wide Rules

Deny Mappings

Mapping Rule Directional Symbols

Ephemeral Mapping

Best Practices

Testing Mappings

Examples

Tasks

Identity Mapping Tasks

DNS

Introduction

Properties

CLI

Logs

Active Directory and DNS

Non-DNS Resolution

DNS-Less Operation

IPMP

Introduction

Properties

Logs

Tasks

NTP

Introduction

Properties

Validation

Authentication

BUI

CLI

BUI Clock

Tips

Tasks

NTP Tasks

Remote Replication

Introduction

Dynamic Routing

RIP and RIPng Dynamic Routing Protocols

Logs

Phone Home

Introduction

Oracle Single Sign-On Account

Properties

Web Proxy

Registration

Status

Service state

Logs

SNMP

Introduction

Properties

MIBs

Sun FM MIB

Sun AK MIB

Tasks

SNMP Tasks

SMTP

Introduction

Properties

Logs

Service Tags

Introduction

Properties

System Identity

Introduction

Properties

Logs

SSH

Introduction

Properties

Logs

Tasks

SSH Tasks

Shadow Migration

Introduction

Properties

Managing Shadow Migration

Syslog

Introduction

Properties

Classic Syslog: RFC 3164

Updated Syslog: RFC 5424

Message Format

Alert Message Format

Receiver Configuration Examples

Configuring a Solaris Receiver

Configuring a Linux Receiver

5.  Shares

6.  Analytics

7.  Integration

Glossary

NDMP

Introduction

The NDMP (Network Data Management Protocol) service enables the system to participate in NDMP-based backup and restore operations controlled by a remote NDMP client called a Data Management Application (DMA). Using NDMP, appliance user data (i.e., data stored in administrator-created shares on the appliance) can be backed up and restored to both locally attached tape devices and remote systems. Locally-attached tape devices can also be exposed to the DMA for backing up and restoring remote systems.

NDMP cannot be used to backup and restore system configuration data. Use the [[Maintenance:System:ConfigurationBackup|Configuration Backup/Restore]] feature for that.

Local vs. Remote Configurations

The appliance supports backup and restore using both a local configuration, in which tape drives are physically attached to the appliance, and a remote configuration, in which data is streamed to another system on the same network. In both cases, the backup must be managed by a supported DMA.

In local configurations, supported tape devices, including both drives and changers (robots), are physically connected to the system using a supported SCSI or Fibre Channel (FC) card configured in Initiator mode. These devices can be viewed on the NDMP status screen. The NDMP service presents these devices to a DMA when the DMA scans for devices. Once configured in the DMA, these devices are available for backup and restore of the appliance or other systems on the same network. After adding tape drives or changers to the system or removing such devices from the system, a reboot may be required before the changes will be recognized by the NDMP service. After that, the DMA may need to be reconfigured because tape device names may have changed.

In remote configurations, the tape devices are not physically connected to the system being backed up and restored (the data server) but rather to the system running the DMA or a separate system (the tape server). These are commonly called "3-way configurations" because the DMA controls two other systems. In these configurations the data stream is transmitted between the data server and the tape server over an IP network.

Backup Formats and Types

The NDMP protocol does not specify a backup data format. The appliance supports three backup types corresponding to different implementations and on-tape formats. DMAs can select a backup type using the following values for the NDMP environment variable "TYPE":

Backup type
Details
dump
File-based for filesystems only. Supports file history and direct access recovery (DAR).
tar
File-based for filesystems only. Supports file history and direct access recovery (DAR).
zfs
Share-based for both filesystems and volumes. Does not support file history or direct access recovery (DAR), but may be faster for some datasets. Only supported with NDMPv4.

There is no standard NDMP data stream format, so backup streams generated on the appliance can only be restored on 7000-series appliances running compatible software. Future versions of appliance software can generally restore streams backed up from older versions of the software, but the reverse is not necessarily true. For example, the "zfs" backup type is new in 2010.Q3 and systems running 2010.Q1 or earlier cannot restore backup streams created using type "zfs" under 2010.Q3.

Backing up with "dump" and "tar"

When backing up with "dump" and "tar" backup types, administrators specify the data to backup by a filesystem path, called the backup path. For example, if the administrator configures a backup of /export/home, then the share mounted at that path will be backed up. Similarly, if a backup stream is restored to /export/code, then that's the path where files will be restored, even if they were backed up from another path.

Only paths which are mountpoints of existing shares or contained within existing shares may be specified for backup. If the backup path matches a share's mountpoint, only that share is backed up. Otherwise the path must be contained within a share, in which case only the portion of that share under that path is backed up. In both cases, other shares mounted inside the specified share under the backup path will not be backed up; these shares must be specified separately for backup.

Snapshots

If the backup path specifies a live filesystem (e.g., /export/code) or a path contained within a live filesystem (e.g., /export/code/src), the appliance immediately takes a new snapshot and backs up the given path from that snapshot. When the backup completes, the snapshot is destroyed. If the backup path specifies a snapshot (e.g., /export/code/.zfs/snapshot/mysnap), no new snapshot is created and the system backs up from the specified snapshot.

Share metadata

To simplify backup and restore of complex share configurations, "dump" and "tar" backups include share metadata for projects and shares associated with the backup path. This metadata describes the share configuration on the appliance, including protocol sharing properties, quota properties, and other properties configured on the Shares screen. (This is not to be confused with filesystem metadata like directory

structure and file permissions, which is also backed up and restored with NDMP.)

For example, if you back up /export/proj, the share metadata for all shares whose mountpoints start with /export/proj will be backed up, as well as the share metadata for their parent projects. Similarly, if you back up /export/someshare/somedir, and a share is mounted at /export/someshare, that share and its project's share metadata will be backed up.

When restoring, if the destination of the restore path is not contained inside an existing share, projects and shares in the backup stream will be recreated as needed with their original properties as stored in the backup. For example, if you back up /export/foo, which contains project proj1 and shares share1 and share2, and then destroy the project and restore from the backup, then these two shares and the project will be recreated with their backed-up properties as part of the restore operation.

During a restore, if a project exists that would have been automatically recreated, the existing project is used and no new project is automatically created. If a share exists that would have been automatically recreated, and if its mountpoint matches what the appliance expects based on the original backup path and the destination of the restore, then the existing share is used and no new share is automatically created. Otherwise, a new share is automatically created from the metadata in the backup. If a share with the same name already exists (but has a different mountpoint), then the newly created share will be given a unique name starting with "ndmp-" and with the correct mountpoint.

It is recommended that you either restore a stream whose datasets no longer exist on the appliance, allowing the appliance to recreate datasets as specified in the backup stream, or precreate a destination share for restores. Either of these practices avoids surprising results related to the automatic share creation described above.

Backing up with "zfs"

When backing up with type "zfs", administrators specify the data to backup by its canonical name on the appliance. This can be found underneath the name of the share in the BUI: image:Image

or in the CLI as the value of the canonical_name property. Canonical names do not begin with a leading '/', but when configuring the backup path the canonical name must be prefixed with '/'.

Both projects and shares can be specified for backup using type "zfs". If the canonical name is specified as-is, then a new snapshot is created and used for the backup. A specific snapshot can be specified for backup using the '@snapshot' suffix, in which case no new snapshot is created and the

specified snapshot is backed up. For examples:

Canonical name
Shares backed up
pool-0/local/default
New snapshot of the local project called "default" and all of its shares.
pool-0/local/default@yesterday
Named snapshot "yesterday" of local project "default", and all of its shares having snapshot "yesterday".
pool-0/local/default/code
New snapshot of share "code" in local project "default". "code" could be a filesystem or volume.
pool-0/local/default/code@yesterday
Named snapshot "yesterday" of share "code" in local project "default". "code" could be a filesystem or volume.

Because level-based incremental backups using the "zfs" backup type require a base snapshot from the previous incremental, the default behavior for level backups for which a new snapshot is created is to keep the new snapshot so that it can be used for subsequent incremental backups. If the DMA indicates that the backup will not be used for subsequent incremental backups by setting UPDATE=n, the newly created snapshot is destroyed after the backup. Existing user snapshots are never destroyed after a backup. See "Incremental backups" below for details.

Share metadata

Share metadata (i.e., share configuration) is always included in "zfs" backups. When restoring a full backup with type "zfs", the destination project or share must not already exist. It will be recreated from the metadata in the backup stream. When restoring an incremental backup with type "zfs", the destination project or share must already exist. Its properties will be updated from the metadata in the backup stream. See "Incremental backups" below for details.

Incremental backups

The appliance supports level-based incremental backups for all of the above backup types. To specify a level backup, DMAs typically specify the following three environment variables:

Variable
Details
LEVEL
Integer from 0 to 9 identifying the backup level.
DMP_NAME
Specifies a particular incremental backup set. Multiple sets of level incremental backups can be used concurrently by specifying different values for DMP_NAME.
UPDATE
Indicates whether this backup can be used as the base for subsequent incremental backups

By definition, a level-N backup includes all files changed since the previous backup of the same backup set (specified by "DMP_NAME") of the same share using LEVEL less than N. Level-0 backups always include all files. If UPDATE has value "y" (the default), then the current backup is recorded so that future backups of level greater than N will use this backup as a base. These variables are typically managed by the DMA and need not be configured directly by administrators.

Below is a sample incremental backup schedule:

Day
Details
First of month
Level-0 backup. Backup contains all files in the share.
Every 7th, 14th, 21st of month
Level-1 backup. Backup contains all files changed since the last full (monthly) backup
Every day
Level-2 backup. Backup contains all files changed since the last level-1 backup

To recover the filesystem's state as it was on the 24th of the month, an administrator typically restores the Level-0 backup from the 1st of the month to a new share, then restores the Level-1 backup from the 21st of the month, and then restores the Level-2 backup from the 24th of the month.

To implement level-based incremental backups the appliance must keep track of the level backup history for each share. For "tar" and "dump" backups, the level backup history is maintained in the share metadata. Incremental backups traverse the filesystem and include files modified since the time of the previous level backup. At restore time, the system simply restores all the files in the backup stream. In the above example, it would therefore be possible to restore the Level-2 backup from the 24th onto any filesystem and the files contained in that backup stream will be restored even though the target filesystem may not match the filesystem where the files were backed up. However, best practice suggests using a procedure like the above which starts from an empty tree restores the previous level backups in order to recover the original filesystem state.

To implement efficient level-based incremental backups for type "zfs", the system uses a different approach. Backups that are part of an incremental set do not destroy the snapshot used for the backup but rather leave it on the system. Subsequent incremental backups use this snapshot as a base to quickly identify the changed filesystem blocks and generate the backup stream. As a consequence, the snapshots left by the NDMP service after a backup must not be destroyed if you want to create subsequent incremental backups.

Another important consequence of this behavior is that in order to restore an incremental stream, the filesystem state must exactly match its state at the base snapshot of the incremental stream. In other words, in order to restore a level-2 backup, the filesystem must look exactly as it did when the previous level-1 backup completed. Note that the above commonly-used procedure guarantees this because when restoring the Level-2 backup stream from the 24th, the system is exactly as it was when the Level-1 backup from the 21st completed because that backup has just been restored.

The NDMP service will report an error if you attempt to restore an incremental "zfs" backup stream to a filesystem whose most recent snapshot doesn't match the base snapshot for the incremental stream, or if the filesystem has been changed since that snapshot. You can configure the NDMP service to rollback to the base snapshot just before the restore begins by specifying the NDMP environment variable "ZFS_FORCE" with value "y" or by configuring the "Rollback datasets" property of the NDMP service (see Properties below).

Properties

The NDMP service configuration consists of the following properties:

Property
Description
DMA username and password
Used to authenticate the DMA (Data Management Application)
Enable DAR
Enables the system to locate files by position rather than by sequential search during restore operations. Enabling this option reduces the time it takes to recover a small number of files from many tapes. You must specify this option at backup time in order to be able to recover individual files later
Ignore file metadata changes for incremental backups
Directs the system to backup only files in which content has changed, ignoring files for which only metadata, such as permissions or ownership, has changed. This option only applies to incremental "tar" and "dump" backups and is disabled by default
Restore full absolute path for partial restore (v3 only)
Specifies that when a file is restored, the complete absolute path to that file is also restored (instead of just the file itself). This option is disabled by default.
NDMP version
The version of NDMP that your DMA supports
TCP port
The NDMP default connection port is 10000. NDMPv3 always uses this port. NDMPv4 allows a different port if needed.
Default restore pool(s)
When doing a full restore using types "tar" or "dump", the system will re-create datasets if there is not already a share mounted at the target. Because the NDMP protocol specifies only the mountpoint, the system will by default choose a pool in which to recreate any projects and shares. On a system with multiple pools, this property allows the user to explicitly specify one or more pools. Multiple pools need only be specified in a cluster with active pools on each head, and it is the responsibility of the user to make sure that this list is kept in sync with any storage configuration changes. If none of the pools exist or are online, then the system will select a default pool at random.
Rollback datasets before restore (ZFS backups only)
Only applies to backups with type "zfs". Determines whether when restoring an incremental backup the system rolls back the target project and share to the snapshot used as the base for the incremental restore. If the project and shares are rolled back, then any changes made since that snapshot will be lost. This setting is normally controlled by the DMA via the "ZFS_FORCE" environment variable (see "Incremental Backups" above) but this property can be used to override the DMA setting to always rollback these datasets or never roll them back. Not rolling them back will cause the restore to fail unless they have already been manually rolled back. This property is intended for use with DMAs that do not allow administrators to configure custom environment variables like ZFS_FORCE.
DMA tape mode (for locally attached drives)
Specifies whether the DMA expects SystemV or BSD semantics. The default is SystemV, which is recommended for most DMAs. This option is only applicable for locally attached tape drives exported via NDMP. Consult your DMA documentation for which mode your DMA expects. Changing this option only changes which devices are exported when the DMA scans for devices, so you will need to reconfigure the tape devices in your DMA after changing this setting.

Changing services properties is documented in the BUI and CLI sections of Services.

Logs

Log
Description
system-ndmpd:default
NDMP service log

To view service logs, refer to the Logs section from Services.