JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Sun ZFS Storage 7000 System Administration Guide
search filter icon
search icon

Document Information

Preface

1.  Introduction

2.  Status

3.  Configuration

4.  Services

5.  Shares

Shares

Introduction

Concepts

Storage Pools

Projects

Shares

Properties

Snapshots

Clones

Shadow Migration

Shadow Data Migration

Traditional Data Migration

Migration via synchronization

Migration via external interposition

Shadow Migration

Shadow migration behavior

Restrictions on shadow source

Shadow filesystem semantics during migration

Identity and ACL migration

Shadow Migration Management

Creating a shadow filesystem

Managing background migration

Handling errors

Monitoring progress

Canceling migration

Snapshots of shadow filesystems

Backing up shadow filesystems

Replicating shadow filesystems

Shadow migration analytics

Shadow migration requests

Shadow migration bytes

Shadow migration operations

Migration of local filesystems

Tasks

Testing potential shadow migration

Migrating data from an active NFS server

Space Management

Introduction

Terms

Space Management Terms

Physical Data

Logical Data

Referenced Data

Snapshot Data

Quota

Reservation

Understanding snapshots

Filesystem and project settings

Data quotas

Data reservations

User and group settings

Viewing current usage

BUI

CLI

User or group quotas

BUI

CLI

Identity management

Filesystem Namespace

Filesystem namespace

Nested mountpoints

Protocol access to mountpoints

NFSv2 / NFSv3

NFSv4

SMB

FTP / FTPS / SFTP

HTTP / HTTPS

Shares

BUI

List of Shares

Editing a Share

Usage Statistics

Available space

Referenced data

Snapshot data

Unused Reservation

Total space

Static Properties

Compression ratio

Case sensitivity

Reject non UTF-8

Normalization

Volume block size

Origin

Data Migration Source

Project Panel

Creating Shares

CLI

Navigation

Share Operations

Properties

General

General Share Properties

Space Usage

Volume size

Thin provisioned

Properties

Mountpoint

Read only

Update access time on read

Non-blocking mandatory locking

Data deduplication

Data compression

Checksum

Cache device usage

Synchronous write bias

Database record size

Additional replication

Virus scan

Prevent destruction

Restrict ownership change

Custom Properties

Protocols

Shares Protocols

NFS

CLI Considerations

Security Modes

Character set encodings

SMB

SCSI

HTTP

FTP

SFTP

Access

Access Control

Root Directory Access

User

Group

Permissions

ACL Behavior

ACL behavior on mode change

ACL inheritance behavior

Root Directory ACL

Snapshots

Introduction

Snapshot Properties

.zfs/snapshot visible

BUI

Listing Snapshots

Taking Snapshots

Renaming a Snapshot

Destroying a Snapshot

Rolling back to a Snapshot

Cloning a Snapshot

Scheduled Snapshots

CLI

Listing Snapshots

Taking Snapshots

Renaming a Snapshot

Destroying a Snapshot

Rolling back to a Snapshot

Cloning a Snapshot

Scheduled Snapshots

Projects

BUI

List of Projects

Editing a Project

Usage Statistics

Available space

Referenced data

Snapshot data

Unused Reservation

Unused Reservation of shares

Total space

Static Properties

Compression ratio

Creating Projects

CLI

Navigation

Project Operations

Selecting a pool in a cluster

Properties

General

General Project Properties

Space Usage

Quota

Reservation

Inherited Properties

Custom Properties

Filesystem Creation Defaults

LUN Creation Defaults

Protocols

Project Protocols

NFS

SMB

iSCSI

HTTP

FTP

Access

Access Control

Inherited ACL Behavior

Snapshots

Introduction

Snapshot Properites

.zfs/snapshot visible

BUI

CLI

Replication

Remote Replication Introduction

Concepts

Terminology

Targets

Actions and Packages

Storage Pools

Project-level vs Share-level Replication

Configuring Replication

Creating and Editing Targets

Creating and Editing Actions

Modes: Manual, Scheduled, or Continuous

Including Intermediate Snapshots

Sending and Cancelling Updates

Managing Replication Packages

BUI

CLI

Cancelling Replication Updates

Disabling a Package

Cloning a Package or Individual Shares

Exporting Replicated Filesystems

Severing Replication

Reversing the Direction of Replication

Destroying a Replication Package

Examples

Remote Replication Details

Authorizations

Alerts

Replication and Clustering

Snapshots and Data Consistency

Snapshot Management

Replicating iSCSI Configuration

Replicating Clones

Observing Replication

Replication Failures

Upgrading From 2009.Q3 and Earlier

Schema

Customized Share Properties

BUI

CLI

Tasks

Create a property to track contact info

6.  Analytics

7.  Application Integration

Glossary

Index

Replication

Remote Replication Introduction

Sun Storage 7000 appliances support snapshot-based replication of projects and shares from a source appliance to any number of target appliances manually, on a schedule, or continuously. The replication includes both data and metadata. Remote replication (or just "replication") is a general-purpose feature optimized for the following use cases:

The remote replication feature has several important properties:

Concepts

Terminology
Targets

Before a source appliance can replicate to a target, the two systems must set up a replication peer connection that enables the appliances to identify each other securely for future communications. Administrators setup this connection by creating a new replication target on the Configuration > Services > Remote Replication screen on the source appliance. To create a new target, administrators specify three fields:

The appliances then exchange keys used to securely identify each other in subsequent communications. These keys are stored persistently as part of the appliance's configuration and persist across reboots and upgrades. They will be lost if the appliance is factory reset or reinstalled. The root password is never stored persistently, so changing the root password on either appliance does not require any changes to the replication configuration. The password is never transmitted in the clear either because this initial identity exchange (like all replication control operations) is protected with SSL.

By default, the replication target connection is not bidirectional. If an administrator configures replication from a source A to a target B, B cannot automatically use A as a target. However, the system supports reversing the direction of replication, which automatically creates a target for A on B (if it does not already exist) so that B can replicate back to A.

To configure replication targets, see Configuring Replication below.

Actions and Packages

Targets represent a connection between appliances that enables them to communicate securely for the purpose of replication, but targets do not specify what will be replicated, how often, or with what options. For this, administrators must define replication actions on the source appliance. Actions are the primary administrative control point for replication, each one specifying:

The group is specified implicitly by the project or share on which the action is configured (see Project-level versus Share-level Replication below). The target appliance and storage pool cannot be changed after the action is created, but the other options can be modified at any time. Generally, if a replication update is in progress when an option is changed, then the new value only takes effect when the next update begins.

Actions are the primary unit of replication configuration on the appliance. Each action corresponds to a package on the target appliance that contains an exact copy of the source projects and shares on which the action is configured as of the start time of the last replication update. Administrators configure the frequency and other options for replication updates by modifying properties of the corresponding action. Creating the action on the source appliance creates the package on the target appliance in the specified storage pool, so the source must be able to contact the target when the action is initially created.

The first update for each replication action sends a full sync (or full update): the entire contents of the action's project and shares are sent to the target appliance. Once this initial sync completes, subsequent replication updates are incremental: only the changes since the previous update are sent. The action (on the source) and package (on the target) keep track of which changes have been replicated to the target through named replication snapshots (see below). Generally, as long as at least one full sync has been sent for an action and the action/package connection has not been corrupted due to a software failure or administrative action, replication updates will be incremental.

The action and package are bound to each other. If the package is somehow corrupted or destroyed, the action will not be able to send replication updates, even if the target still has the data and snapshots associated with the action. Similarly, if the action is destroyed, the package will be unable to receive new replication updates (even if the source still has the same data and snapshots). The BUI and CLI warn administrators attempting to perform operations that would destroy the action-package connection. If an error or explicit administrative operation breaks the action-package connection such that an incremental update is no longer possible, administrators must sever or destroy the package and action and create a new action on the source.

One special case of this needs explicit mention. The appliance avoids destroying data on the target unless explicitly requested by the administrator. As a result, if the initial replication update for an action fails for any reason after having replicated some data (thus leaving incomplete data inside the package), subsequent replication updates using the same action will fail because the appliance will not overwrite the already-received data. To resolve this, administrators should destroy the existing action and package and create a new action and package and start replication again.

In software releases prior to 2010.Q1, action and replica configuration (like target configuration) was stored on the controller rather than as part of the project and share configuration in the storage pool. As a result, factory reset caused all such configuration to be destroyed. In 2010.Q1 and later releases, the action and package configuration is stored in the storage pool with the corresponding projects and shares and so will be available even after factory reset. However, target information will still be lost, and actions with missing targets currently cannot be configured to point to a new target.

Storage Pools

When the action is initially configured, the administrator is given a choice of which storage pool on the target should contain the replicated data. The storage pool containing an action cannot be changed once the action has been created. Creating the action creates the empty package on the target in the specified storage pool, and after this operation the source has no knowledge of the storage configuration on the target. It does not keep track of which pool the action is being replicated to, nor is it updated with storage configuration changes on the target.

When the target is a clustered system, the chosen storage pool must be one owned by same head which owns the IP address used by the source for replication because only those pools are always guaranteed to be accessible when the source contacts the target using that IP address. This is exactly analogous to the configuration of NAS clients (NFS and SMB), where the IP address and path requested in a mount operation must obey the same constraint. When performing operations that change the ownership of storage pools and IP addresses in a cluster, administrators must consider the impact to sources replicating to the cluster. There is currently no way to move packages between storage pools or change the IP address associated with an action.

Project-level vs Share-level Replication

The appliance allows administrators to configure remote replication on both the project or share level. Like other properties configurable on the Shares screen, each share can either inherit or override the configuration of its parent project. Inheriting the configuration means not only that the share is replicated on the same schedule to the same target with the same options as its parent project is, but also that the share will be replicated in the same stream using the same project-level snapshots as other shares inheriting the project's configuration. This may be important for applications which require consistency between data stored on multiple shares. Overriding the configuration means that the share will not be replicated with any project-level actions, though it may be replicated with its own share-level actions that will include the project. It is not possible to override part of the project's replication configuration and inherit the rest.

More precisely, the replication configuration of a project and its shares define some number of replication groups, each of which is replicated with a single stream using snapshots taken simultaneously. All groups contain the project itself (which essentially just includes its properties). One project-level group includes all shares inheriting the replication configuration of the parent project. Any shares which override the project's configuration form a new group consisting of only the project and share themselves.

For example, suppose we have the following:

This configuration defines the following replication groups, each of which is replicated as a single stream per action using snapshots taken simultaneously on the project and shares:

It is strongly recommended that project- and share-level replication be avoided within the same project because it can lead to surprising results (particularly when reversing the direction of replication). See the documentation for Managing Replication Packages for more details.

Configuring Replication

Be sure to read and understand the above sections on replication targets, actions, and packages before configuring replication.

Creating and Editing Targets

Targets are configured under Configuration > Services > Remote Replication. In the BUI, click the Targets tab:

Image

In the CLI, navigate to the targets node to set or unset the target hostname, root_password, and label .

knife:> configuration services replication targets

From this context, administrators can:

Targets should not be destroyed while actions are using it. Such actions will be permanently broken. The system makes a best effort to enforce this but cannot guarantee that no actions exist in exported storage pools that are using a given target.

Creating and Editing Actions

After at least one replication target has been configured, administrators can configure actions on a local project or share by navigating to it in the BUI and clicking the Replication tab or navigating to it in the CLI and selecting the "replication" node. These interfaces show the status of existing actions configured on the project or share and allow administrators to create new actions:

Image

Replication actions have the following properties, which are presented slightly differently in the BUI and CLI:

Image
Property (CLI name)
Description
Target
Unique identifier for the replication target system. This property is specified when an action is initially configured and immutable thereafter.
Pool
Storage pool on the target where this project will be replicated. This property is specified when an action is initially configured and not shown thereafter.
Enabled
Whether the system will send updates for this action.
Mode (CLI: continuous) and schedule
Whether this action is being replicated continuously or at manual or scheduled intervals. See below for details.
Include Snapshots
Whether replication updates include non-replication snapshots. See below for details.
Limit bandwidth
Specifies a maximum speed for this replication update (in terms of data transferred over the network per second).
Use SSL
Whether to encrypt data on the wire using SSL. Using this feature can have a significant impact on per-action replication performance.
State
Read-only property describing whether the action is currently idle, sending an update, or cancelling an update.
Last sync
Read-only property describing the last time an update was successfully sent. This value may be unknown if the system has not sent a successful update since boot.
Last attempt
Read-only property describing the last time an update was attempted. This value may be unknown if the system has not attempted to send an update since boot.
Next update
Read-only property describing when the next attempt will be made. This value could be a date (for a scheduled update), "manual," or "continuous."
Modes: Manual, Scheduled, or Continuous

Replication actions can be configured to send updates manually, on a schedule, or continuously. The replication update process itself is the same in all cases. This property only controls the interval.

Because continuous replication actions send updates as frequently as possible, they essentially result in sending a constant stream of all filesystem changes to the target system. For filesystems with a lot of churn (many files created and destroyed in short intervals), this can result in replicating much more data than actually necessary. However, as long as replication can keep up with data changes, this results in the minimum data lost in the event of a data-loss disaster on the source system.

Note that continuous replication is still asynchronous. Sun Storage appliances do not currently support synchronous replication, which does not consider data committed to stable storage until it's committed to stable storage on both the primary and secondary storage systems.

Including Intermediate Snapshots

When the "Include Snapshots" property is true, replication updates include the non-replication snapshots created after the previous replication update (or since the share's creation, in the case of the first full update). This includes automatic snapshots and administrator-created snapshots.

This property can be disabled to skip these snapshots and send only the changes between replication snapshots with each update.

Sending and Cancelling Updates

For targets that have been configured with scheduled or manual replication, administrators can choose to immediately send a replication update by clicking the Image button in the BUI or using the sendupdate command in the CLI. This is not available (or will not work) if an update is actively being sent. Make sure there is enough disk space on the target to replicate the entire project before sending an update.

If an update is currently active, the BUI will display a barber-pole progress bar and the CLI will show a state of sending. To cancel the update, click the Cancel button or use the cancelupdate command. It may take several seconds before the cancellation completes.

Managing Replication Packages

Packages are containers for replicated projects and shares. Each replication action on a source appliance corresponds to one package on the target appliance as described above. Both the BUI and CLI enable administrators to browse replicated projects, shares, snapshots, and properties much like local projects and shares. However, because replicated shares must exactly match their counterparts on the source appliance, many management operations are not allowed inside replication packages, including creating, renaming, and destroying projects and shares, creating and renaming snapshots, and modifying most properties of projects and shares. Snapshots other than those used as the basis for incremental replication can be destroyed in replication packages. This practice is not recommended but can be used when additional free space is necessary.

In 2009.Q3 and earlier software versions, properties could not be changed on replicated shares. The 2010.Q1 release (with associated deferred upgrades) adds limited support for modifying properties of replicated shares to implement differing policies on the source and target appliances. Such property modifications persist across replication updates. Only the following properties of replicated projects and shares may be modified:

The BUI and CLI don't allow administrators to change immutable properties. For shares, a different icon is used to indicate that the property's inheritance cannot be changed:

Image

Note that the deferred updates provided with the 2010.Q1 release must be applied on replication targets in order to modify properties on such targets. The system will not allow administrators to modify properties inside replication packages on systems which have not applied the 2010.Q1 deferred updates.

Note that the current release does not support configuration of "chained" replication (that is, replicating replicated shares to another appliance).

BUI

Replication packages are displayed in the BUI as projects under the "Replica" filter:

Image

Selecting a replication package for editing brings the administrator to the Shares view for the package's project. From here, administrators can manage replicated shares much like local shares with the exceptions described above. Package properties (including status) can be modified under the Replication tab (see below):

Image

The status icon on the left changes when replication has failed:

Image

Packages are only displayed in the BUI after the first replication update has begun. They may not appear in the list until some time after the first update has completed.

CLI

Replication packages are organized in the CLI by source under shares replication sources. Administrators first select a source, then a package. Package-level operations can be performed on this node (see below), or the project can be selected to manage project properties and shares just like local projects and shares with the exceptions described above. For example:

loader:> shares replication sources
loader:shares replication sources> show
Sources:

source-000 ayu
            PROJECT    STATE       LAST UPDATE
package-000 oldproj    idle        unknown
package-001 aproj1     receiving   Sun Feb 21 2010 22:04:35 GMT+0000 (UTC)

loader:shares replication sources> select source-000
loader:shares replication source-000> select package-001
loader:shares replication source-000 package-001> show
Properties:
                      enabled = true
                        state = receiving
            state_description = Receiving update
                    last_sync = Sun Feb 21 2010 22:04:40 GMT+0000 (UTC)
                     last_try = Sun Feb 21 2010 22:04:40 GMT+0000 (UTC)

Projects:
                       aproj1

loader:shares replication source-000 package-001> select aproj1
loader:shares replication source-000 package-001 aproj1> get mountpoint
                    mountpoint = /export
loader:shares replication source-000 package-001 aproj1> get sharenfs
                      sharenfs = on
Cancelling Replication Updates

To cancel in-progress replication updates on the target using the BUI, navigate to the replication package (see above), then click the Replication tab. If an update is in progress, you will see a barber pole progress bar with a cancel button (Cancel) next to it as shown here:

Image

Click this button to cancel the update.

To cancel in-progress replication updates on the target using the CLI, navigate to the replication package (see above) and use the cancelupdate command.

It is not possible to initiate updates from the target. Administrators must login to the source system to initiate a manual update.

Disabling a Package

Replication updates for a package can be disabled entirely, cancelling any ongoing update and causing new updates from the source appliance to fail.

To toggle whether a package is disabled from the BUI, navigate to the package (see above), then click the Replication tab, and then click the Power icon. The status icon on the left should change to indicate the package's status (enabled, disabled, or failed). The package remains disabled until explicitly enabled by an administrator using the same button or the CLI.

To toggle whether a package is disabled from the CLI, navigate to the package (see above), modify the enabled property, and commit your changes.

Cloning a Package or Individual Shares

A clone of a replicated package is a local, mutable project that can be managed like any other project on the system. The clone's shares are clones of the replicated shares at the most recently received snapshot. These clones share storage with their origin snapshots in the same way as clones of share snapshots do (see Cloning a Snapshot). This mechanism can be used to failover in the case of a catastrophic problem at the replication source, or simply to provide a local version of the data that can be modified.

Use the Clone button in the BUI or the clone CLI command (in the package's context) to create a package clone based on the most recently received replication snapshot. Both the CLI and BUI interface require the administrator to specify a name for the new clone project and allow the administrator to override the mountpoint of the project or its shares to ensure that they don't conflict with those of other shares on the system.

In 2009.Q3 and earlier, cloning a replicated project was the only way to access its data and thus the only way to implement disaster-recovery failover. In 2010.Q1 and later, individual filesystems can be exported read-only without creating a clone (see below). Additionally, replication packages can be directly converted into writable local projects as part of a failover operation. As a result, cloning a package is no longer necessary or recommended, as these alternatives provide similar functionality with simpler operations and without having to manage clones and their dependencies.

In particular, while a clone exists, its origin snapshot cannot be destroyed. When destroying the snapshot (possibly as a result of destroying the share, project, or replication package of which the snapshot is a member), the system warns administrators of any dependent clones which will be destroyed by the operation. Note that snapshots can also be destroyed on the source at any time and such snapshots are destroyed on the target as part of the subsequent replication update. If such a snapshot has clones, the snapshot will instead be renamed with a unique name (typically recv-XXX).

Administrators can also clone individual replicated share snapshots using the normal BUI and CLI interfaces.

Exporting Replicated Filesystems

Replicated filesystems can be exported read-only to NAS clients. This can be used to verify the replicated data or to perform backups or other intensive operations on the replicated data (offloading such work from the source appliance).

The filesystem's contents always matches the most recently received replication snapshot for that filesystem. This may be newer than the most recently received snapshot for the entire package, and it may not match the most recent snapshot for other shares in the same package. See "Snapshots and Data Consistency" below for details.

Replication updates are applied atomically at the filesystem level. Clients looking at replicated files will see replication updates as an instantaneous change in the underlying filesystem. Clients working with files deleted in the most recent update will see errors. Clients working with files changed in the most recent update will immediately see the updated contents.

Replicated filesystems are not exported by default. They are exported by modifying the "exported" property of the project or share using the BUI or CLI:

Image

This property is inherited like other share properties. This property is not shown for local projects and shares because they are always exported. Additionally, severing replication (which converts the package into a local project) causes the package's shares to become exported.

Replicated LUNs currently cannot be exported. They must be first cloned or the replication package severed in order to export their contents.

Severing Replication

A replication package can be converted into a local, writable project that behaves just like other local projects (i.e. without the management restrictions applied to replication packages) by severing the replication connection. After this operation, replication updates can no longer be received into this package, so subsequent replication updates of the same project from the source will need to send a full update with a new action (into a new package). Subsequent replication updates using the same action will fail because the corresponding package no longer exists on the target.

This option is primarily useful when using replication to migrate data between appliances or in other scenarios that don't involve replicating the received data back to the source as part of a typical two-system disaster recovery plan.

Replication can be severed from the BUI by navigating to the replication package (see above), clicking the Replication tab, and clicking the Image button. The resulting dialog allows the administrator to specify the name of the new local project.

Replication can be severed from the CLI by navigating to the replication package (see above), and using the sever command. This command takes an optional argument specifying the name of the new local project. If no argument is specified, the original name is used.

Because all local shares are exported, all shares in a package are exported when the package is severed, whether or not they were previously exported (see above). If there are mountpoint conflicts between replicated filesystems and other filesystems on the system, the sever operation will fail. These conflicts must be resolved before severing by reconfiguring the mountpoints of the relevant shares.

Reversing the Direction of Replication

The direction of the replication can be reversed to support typical two-system disaster recovery plans. This operation is similar to the sever operation described above, but additionally configures a replication action on the new local project for incremental replication back to the source system. No changes are made on the source system when this operation is completed, but the first update attempt using this action will convert the original project on the source system into a replication package and rollback any changes made since the last successful replication update from that system. This feature does not automatically redirect production workloads, failover IP addresses, or perform other activities related to the disaster-recovery failover besides modifying the read-write status of the primary and secondary data copies.

As part of the conversion of the original source project into a replication package on the original source system (now acting as the target), the shares that were replicated as part of the action/package currently being reversed are moved into a new replication package and unexported. The original project remains in the local collection but may end up empty if the action/package included all of its shares. When share-level replication is reversed, any other shares in the original project remain unchanged.

As mentioned above, this feature is typically used to implement a two-system disaster recovery configuration in which a primary system serves production data and replicates it to a secondary or DR system (often in another datacenter) standing by to take over the production traffic in the event of a disaster at the primary site. In the event of a disaster at the primary site, the secondary site's copy must be made "primary" by making it writable and redirecting production traffic to the secondary site. When the primary site is repaired, the changes accumulated at the secondary site can be replicated back to the primary site and that site can resume servicing the production workload.

A typical sequence of events under such a plan is as follows:

  1. The primary system is serving the production workload and replicating to the secondary system.

  2. A disaster occurs, possibly representing a total system failure at the primary site. Administrators reverse the direction of replication on the secondary site, exporting the replicated shares under a new project configured for replication back to the primary site for when primary service is restored. In the meantime, the production workload is redirected to the secondary site.

  3. When the primary site is brought back online, an administrator initiates a replication update from the secondary site to the primary site. This converts the primary's copy into a replication package, rolling back any changes made since the last successful update to the target (before the failure). When the primary site's copy is up-to-date, the direction of replication is reversed again, making the copy at the primary site writable. Production traffic is redirected back to the primary site. Replication is resumed from the primary to the secondary, restoring the initial relationship between the primary and secondary copies.

When reversing the direction of replication for a package, it is strongly recommended that administrators first stop replication of that project from the source. If a replication update is in progress when an administrator reverses the direction of replication for a project, administrators cannot know which consistent replication snapshot was used to create the resulting project on the former target appliance (now source appliance).

Replication can be reversed from the BUI by navigating to the replication package (see above), clicking the Replication tab, and clicking the Reverse direction button. The resulting dialog allows the administrator to specify the name of the new local project.

Replication can be severed from the CLI by navigating to the replication package (see above), and using the reverse command. This command takes an optional argument specifying the name of the new local project. If no argument is specified, the original name is used.

Because all local shares are exported, all shares in a package are exported when the package is reversed, whether or not they were previously exported (see above). If there are mountpoint conflicts between replicated filesystems and other filesystems on the system, the reverse operation will fail. These conflicts must be resolved before severing by reconfiguring the mountpoints of the relevant shares. Because this operation is typically part of the critical path of restoring production service, it is strongly recommended to resolve these mountpoint conflicts when the systems are first setup rather than at the time of DR failover.

Destroying a Replication Package

The project and shares within a package cannot be destroyed without destroying the entire package. The entire package can be destroyed from the BUI by destroying the corresponding project. A package can be destroyed from the CLI using the destroy command at the shares replication sources node.

When a package is destroyed, subsequent replication updates from the corresponding action will fail. To resume replication, the action will need to be recreated on the source to create a new package on the target into which to receive a new copy of the data.

Examples

Below is an example of cloning a received replication project, overriding both the project's and one share's mountpoint:

perch:> shares
perch:shares> replication
perch:shares replication> sources
perch:shares replication sources> select source-000
perch:shares replication source-000> select package-000
perch:shares replication source-000 package-000> clone
perch:shares replication source-000 package-000 clone> set target_project=my_clone
                target_project = my_clone
perch:shares replication source-000 package-000 clone> list
CLONE PARAMETERS
                target_project = my_clone
           original_mountpoint = /export
           override_mountpoint = false
                    mountpoint =

    SHARE                           MOUNTPOINT
    bob                             (inherited)
    myfs1                           (inherited)
perch:shares replication source-000 package-000 clone> set override_mountpoint=true
           override_mountpoint = true
perch:shares replication source-000 package-000 clone> set mountpoint=/export/my_clone
                    mountpoint = /export/my_clone
perch:shares replication source-000 package-000 clone bob> select bob
perch:shares replication source-000 package-000 clone bob> set override_mountpoint=true
           override_mountpoint = true
perch:shares replication source-000 package-000 clone bob> set mountpoint=/export/bob
                    mountpoint = /export/bob
perch:shares replication source-000 package-000 clone bob> done
perch:shares replication source-000 package-000 clone> commit
CLONE PARAMETERS
                target_project = my_clone
           original_mountpoint = /export
           override_mountpoint = true
                    mountpoint = /export/my_clone

    SHARE                           MOUNTPOINT
    bob                             /export/bob (overridden)
    myfs1                           (inherited)
Are you sure you want to clone this project?
There are no conflicts.
perch:shares replication source-000 package-000 clone>

Remote Replication Details

Authorizations

In addition to the Remote Replication filter under the Services scope that allows administrators to stop, start, and restart the replication service, the replication subsystem provides two authorizations under the "Projects and Shares" scope:

Authorization
Details
rrsource
Allows administrators to create, edit, and destroy replication targets and actions and send and cancel updates for replication actions.
rrtarget
Allows administrators to manage replicated packages, including disabling replication at the package level, cloning a package or its members, modifying properties of received datasets, and severing or reversing replication. Other authorizations may be required for some of these operations (like setting properties or cloning individual shares). See the available authorizations in the Projects and Shares scope for details.

Note that the rrsource authorization is required to configure replication targets on an appliance, even though this is configured under the Remote Replication service screen.

For help with authorizations, see the Authorizations documentation.

Alerts

The system posts alerts when any of the following events occur:

Replication and Clustering

Replication can be configured from any SS7000 appliance to any other SS7000 appliance regardless of whether each is part of a cluster and whether the appliance's cluster peer has replication configured in either direction, except for the following constraints:

The following rules govern the behavior of replication in clustered configurations:

For details on clustering and cluster terminology, review the Clustering documentation.

Snapshots and Data Consistency

The appliance replicates snapshots and each snapshot is received atomically on the target, so the contents of a share's replica on the target always matches the share's contents on the source at the time the snapshot was taken. Because the snapshots for all shares sent in a particular group are taken at the same time (see above), the entire package contents after the completion of a successful replication update exactly matches the group's content when the snapshot was created on the source (when the replication update began).

However, each share's snapshots are replicated separately (and serially), so it's possible for some shares within a package to have been updated with a snapshot more recent than those of other shares in the same package. This is true during a replication update (after some shares have been updated but before others have) and after a failed replication update (after which some shares may have been updated but others may not have been).

To summarize:

Snapshot Management

Snapshots are the basis for incremental replication. The source and target must always share a common snapshot in order to continue replicating incrementally, and the source must know which is the most recent snapshot that the target has. To facilitate this, the replication subsystem creates and manages its own snapshots. Administrators generally need not be concerned with them, but the details are described here since snapshots can have significant effects on storage utilization.

Each replication update for a particular action consists of the following steps:

  1. Determine whether this is an incremental or full update based on whether we've tried to replicate this action before and whether the target already has the necessary snapshot for an incremental update.

  2. Take a new project-level snapshot.

  3. Send the update. For a full update, send the entire group's contents up to the new snapshot. For an incremental update, send the difference between from the previous (base) snapshot and the new snapshot.

  4. Record the new snapshot as the base snapshot for the next update and destroy the previous base snapshot (for incremental updates).

This has several consequences for snapshot management:

Replicating iSCSI Configuration

As described above, replication updates include most of the configuration specified on the Shares screen for a project and its shares. This includes any target groups and initiator groups associated with replicated LUNs. When using non-default target groups and initiator groups, administrators must ensure that the target groups and initiator groups used by LUNs within the project also exist on the replication target. It is only required that groups exist with the same name, not that they define the same configuration. Failure to ensure this can result in failure to clone and export replicated LUNs.

The SCSI GUID associated with a LUN is replicated with the LUN. As a result, the LUN on the target appliance will have the same SCSI GUID as the LUN on the source appliance. Clones of replicated LUNs, however, will have different GUIDs (just as clones of local LUNs have different GUIDs than their origins).

Replicating Clones

Replication in 2009.Q3 and earlier was project-level only and explicitly disallowed replicating projects containing clones whose origin snapshots resided outside the project. With share-level replication in 2010.Q1 and later, this restriction has been relaxed, but administrators must still consider the origin snapshots of clones being replicated. In particular, the initial replication of a clone requires that the origin snapshot have already been replicated to the target or is being replicated as part of the same update. This restriction is not enforced by the appliance management software, but attempting to replicate a clone when the origin snapshot does not exist on the target will fail.

In practice, there are several ways to ensure that replication of a clone will succeed:

In all cases, the "include snapshots" property should be true on the origin's action to ensure that the origin snapshot is actually sent to the target.

Observing Replication

While replication-specific analytics are not currently available, administrators can use the advanced TCP analytics to observe traffic by local port. Replication typically uses port 216 on the source appliance.

The status of individual replication actions and packages can be monitored using the BUI and CLI. See "Configuring Replication" above.

Replication Failures

Individual replication updates can fail for a number of reasons. Where possible, the appliance reports the reason for the failure in alerts posted on the source appliance or target appliance, or on the Replication screen for the action that failed. You may be able to get details on the failure by clicking the orange alert icon representing the action's status. The following are the most common types of failures:

Failure
Details
Cancelled
The replication update was cancelled by an administrator. Replication can be cancelled on the source or target and it's possible for one peer not to realize that the other peer has cancelled the operation.
Network connectivity failure
The appliance was unable to connect to the target appliance due to a network problem. There may be a misconfiguration on the source, target, or the network.
Peer verification failed
The appliance failed to verify the identity of the target. This occurs most commonly when the target has been reinstalled or factory reset. A new replication target must be configured on the source appliance for a target which has been reinstalled or factory reset in order to generate a new set of authentication keys. See Targets above.
Peer RPC failed
A remote procedure call failed on the target system. This occurs most commonly when the target appliance is running incompatible software. See "Migrating configuration from 2009.Q3 and earlier" below for more details.
No package
Replication failed because no package exists on the target to contain the replicated data. Since the package is created when configuring the action, this error typically happens after an administrator has destroyed the package on the target. It's also possible to see this error if the storage pool containing the package is not imported on the target system, which may occur if the pool is faulted or if storage or networking has been reconfigured on the target appliance.
Non-empty package exists
Replication failed because the target package contains data from a previous, failed replication update. This error occurs when attempting to send a replication update for an action whose first replication update failed after replicating some data. The target appliance will not destroy data without explicit administrative direction, so it will not overwrite the partially received data. The administrator should remove the existing action and package and create a new action on the source and start replication again.
Disabled
Replication failed because it is disabled on the target. Either the replication service is disabled on the target or replication has been disabled for the specific package being replicated.
Target busy
Replication failed because the target system has reached the maximum number of concurrent replication updates. The system limits the maximum number of ongoing replication operations to avoid resource exhaustion. When this limit is reached, subsequent attempts to receive updates will fail with this error, while subsequent attempts to send updates will queue up until resources are available.
Out of space
Replication failed because the source system had insufficient space to create a new snapshot. This may be because there is no physical space available in the storage pool or because the project or one of its shares would be over quota because of reservations that don't include snapshots.
Incompatible target
Replication failed because the target system is unable to receive the source system's data stream format. This can happen as a result of upgrading a source system and applying deferred updates without having upgraded and applied the same updates on the target. Check the release notes for the source system's software version for a list of deferred updates and whether any have implications for remote replication.
Misc
Replication failed, but no additional information is available on the source. Check the alert log on the target system and if necessary contact support for assistance. Some failure modes that currently fall into this category include insufficient disk space on the target to receive the update and attempting to replicate a clone whose origin snapshot does not exist on the target system.

A replication update fails if any part of the update fails. The current implementation replicates the shares inside a project serially and does not rollback changes from failed updates. As a result, when an update fails, some shares on the target may be up-to-date while others are not. See "Snapshots and Data Consistency" above for details.

Although some data may have been successfully replicated as part of a failed update, the current implementation resends all data that was sent as part of the previous (failed) update. That is, failed updates will not pick up where they left off, but rather will start where the failed update started.

When manual or scheduled updates fail, the system does not automatically try again until the next scheduled update (if any). When continuous replication fails, the system waits several minutes and tries again. The system will continue retrying failed continuous replications indefinitely.

When a replication update is in progress and another update is scheduled to occur, the latter update is skipped entirely rather than started immediately after the previous update completes. The next update will be sent only when the next update is scheduled to occur. The system posts an alert when an update is skipped for this reason.

Upgrading From 2009.Q3 and Earlier

The replication implementation has changed significantly between the 2009.Q3 and 2010.Q1 releases. It remains highly recommended to suspend replication to and from an appliance before initiating an upgrade from 2009.Q3 or earlier. This is mandatory in clusters using rolling upgrade.

There are three important user-visible changes related to upgrade to 2010.Q1 or later: