| Sun StorageTek Availability Suite 4.0 Remote Mirror Software Administration Guide |    
|
| Sun StorageTek Availability Suite 4.0 Remote Mirror Software Administration Guide |
|
| C H A P T E R 2 |
|
Replication and Synchronization Modes |
This chapter describes the following:
The Remote Mirror software supports two modes of data replication:
The replication mode is a user-selectable parameter for each Remote Mirror volume set. The volumes can be updated synchronously in real time or asynchronously using a store-and-forward technique. Typically, a primary volume is first copied explicitly to a designated secondary volume to establish matching contents. As applications write to the primary volume, the software replicates changes to the secondary volume, keeping the two volumes consistent.
In the event of planned or unplanned outages, the software maintains per-device bitmap volumes that are marked to indicate changed blocks with a granularity of 32 Kbytes per segment. This technique optimizes resynchronization by allowing the software to resynchronize only the blocks that have changed since the last synchronization.
Use the sndradm enable command and select the volume set's sync or async parameter to select a replication mode. Use the sndradm -R m command to change the replication mode thereafter.
In synchronous replication mode, a write operation is not confirmed as complete until the remote volume has been updated. Synchronous mirroring forces the Remote Mirror software to wait until the primary volume receives an acknowledgment of the receipt of the data from the secondary volume before returning to the application. The application is not acknowledged until the write operation at the secondary site is complete.
The advantage of synchronous replication is that the primary volume and the secondary volume are in sync after an acknowledgment of a write by the secondary site. One disadvantage might be an increase in write response time, especially for large data sets or long-distance replication (where write operations can incur additional latency because of the time required to transfer data and return acknowledgments).
In asynchronous replication mode, a write operation is confirmed as complete before the remote volume has been updated. Asynchronous replication enables the Remote Mirror software to return to the host as soon as the write operation has been completed on the primary volume and has been placed on a per-volume queue for the secondary site. The secondary site receives the queued requests in the order that they were queued. Once the I/O has been completed at the secondary site and the bitmap has been updated to reflect the volume state, notification is sent to the primary site.
The advantages of asynchronous replication are that it provides fast response and has the least impact on the response time of the primary application. A disadvantage is that data loss might occur at the secondary site after primary site or network failures.
For more information about asynchronous replication and adjusting the asynchronous queue, see Setting the Asynchronous Queue and Tuning the Asynchronous Queue.
The software synchronizes data in a forward (from the primary to the secondary) or reverse (from the secondary to the primary) direction. The software synchronizes data in four modes:
Using one of the Remote Mirror synchronization modes ensures that the primary and secondary volumes contain the same data and that they are identical at a clearly defined time. Synchronization is driven by the software through the sndradm command and progresses to completion.
When a volume set is enabled using the sndradm -e command, you must synchronize the primary and secondary volumes in the set initially. (Use the sndradm -E command if the volumes are already identical.)
Once a volume set has been synchronized, the software ensures that the primary and secondary volumes contain the same data through replication. Replication is driven by user-layer application write operations. Remote mirror replication is an ongoing process.
Full synchronization starts a full copy operation from the primary volume to the secondary volume. It also enables replication concurrently from the primary volume to the secondary volume so that any new write operations to the primary volume are also replicated to the secondary volume. After the operation is complete, the Remote Mirror software maintains the normal replication mode for the volume: either synchronous or asynchronous replication.
FIGURE 2-1 shows the full forward synchronization process.
1. The Remote Mirror software on the primary system (host1) requests disk blocks from the active primary volume. The data might already be resident in the primary system data cache or might require a local disk access.
2. The software transmits the disk blocks, with destaging instructions, over the connection to a cache region on the secondary system.
The software on the secondary system updates its remote volume and acknowledges the update to the primary system.
An update resynchronization initiates replication of only the changed primary site volume data to the secondary site, based on the bitmap. Only the blocks marked "dirty" (changed) in the bitmap are copied to the target volume. After the update is complete, the software maintains the normal replicating mode. The software can also be placed in logging mode. See Logging.
Logging and update resynchronization serve as a safety net if one of your replication processes is interrupted. The software monitors the network connections between the primary and secondary hosts. Any link failures or remote system failures are detected by the transport interface and are passed to the Remote Mirror software.
The software resynchronizes the secondary volume from the primary volume. It uses the changes reported in the log files during the interruption to update the secondary volume. It also enables concurrent replication between the primary and secondary volumes so that any new write operations to the primary volumes are also replicated on the secondary volumes.
If the interruption lasts many hours and the updates are widespread, the advantages of logging and performing the update resynchronization diminish. As time passes, the proportion of bits set to true in the bitmap volumes of a volume set might reach 100 percent. The overhead of logging and update resynchronization must then be balanced against that of a full synchronization.
FIGURE 2-2 shows an update resynchronization from the primary system to its secondary system when the secondary volumes are stale from the interruption.
1. The Remote Mirror software on host1 examines a bitmap from the primary and secondary hosts for the volumes.
2. The software on host1 requests the blocks that were updated during the interruption from the up-to-date volume. The data might already reside in the host1 data cache or on the local disk.
3. The software on host1 transmits the update blocks 3R to the host2 Remote Mirror software.
4. The software on host2 refreshes its stale image with the updated blocks and acknowledges the action to host1.
5. The software revises the bitmap to track the remote update.
All steps repeat until the remote replicated image is updated.
During reverse full synchronization, the Remote Mirror software replicates the volume data from the secondary site to the primary site. After you issue the sndradm -m -r command, the software starts a full reverse copy operation from the secondary volume to the primary volume. It also enables concurrent replication from the primary volume to the secondary volume so that any new write operations to the primary volume are also replicated to the secondary volume.
|
Note - It is necessary to unmount the primary volume before the reverse synchronization begins. The primary volume can then be remounted as soon as the reverse synchronization begins. |
You can use the primary volume during reverse synchronization. The primary volume shows a consistent volume image of the latest data as soon as the reverse synchronization starts. If your application had been writing to the secondary volume as part of a failure or disaster rehearsal, move the application to the primary volume when the reverse synchronization starts.
FIGURE 2-3 shows the full reverse synchronization process.
1. The data might already be resident in host2 data cache, or it might require a secondary disk access. If so, the Remote Mirror software on host1 requests blocks from the up-to-date secondary volume on host2.
2. The software on host2 transmits the cache blocks 2R over the link to a software region on host1 with destaging instructions.
3. The software on host1 updates its disk.
During reverse update synchronization, the Remote Mirror software compares the bitmaps between the primary and secondary sites and replicates only the changed blocks from the secondary site to the primary site.
The software resynchronizes the primary volume from the secondary volume. It uses the changes reported in the log files while replication was interrupted to update the primary volume. It also enables concurrent replication between the primary volume and secondary volumes so that any new write operations to the primary volume are also replicated on the secondary volumes.
|
Note - It is necessary to unmount the primary volume before the reverse update synchronization begins. The primary volume can then be remounted as soon as the reverse update synchronization begins. |
You can use the primary volume during reverse update synchronization. The primary volume shows a consistent volume image of the latest data as soon as the reverse update synchronization starts. If your application had been writing to the secondary volume as part of a failure or disaster rehearsal, move the application to the primary volume when the reverse update synchronization starts.
FIGURE 2-4 shows a reverse update resynchronization from the secondary system to its primary system.
1. The Remote Mirror software on host1 retrieves the secondary bitmap 1R from host2 for one of the volumes affected by the interruption.
2. The software on host1 requests the blocks updated during the interruption from the up-to-date secondary volume of host2. The data might already be resident in host2's data cache or it might require secondary disk access.
3. The software on host2 transmits the updated blocks 3R to host1 Remote Mirror software region of cache using the intersite link.
4. The software on host1 refreshes its stale image with the updated blocks.
5. The software on host1 revises the bitmap to track the remote update.
All steps repeat until the primary volume is up-to-date.
During logging, the Remote Mirror software updates only the bitmaps at the primary site. No replication occurs. At a later time, the bitmaps at the primary and secondary sites are compared and the changed blocks in the primary site volume are mirrored by update resynchronization to the secondary site. You can use logging to save on telecommunications or connection costs. However, you risk the cost of data loss. If you lose the primary site, you do not have the data at the secondary site that was written to the primary site.
If all volume sets in an I/O group are replicating (that is, each secondary volume contains a valid point-in-time copy of the corresponding primary volume) and one volume set enters logging mode, all other sets in the I/O group enter logging mode automatically. This scheme ensures that each secondary volume contains a valid copy of the data.
You can also perform logging on the secondary site before a failover. You can then update the primary site using a reverse synchronization or reverse update synchronization command.
With synchronous and asynchronous replication, the Remote Mirror software automatically switches to logging mode at the primary site if a break in the network occurs or if the primary site is down. The secondary site does not automatically switch to logging mode if a break in the network occurs or if the primary site is down. Instead, it changes to a state called "need sync". To identify this condition, issue the sndradm -P command on the secondary site. This "need sync" state protects the data at the secondary site from unwanted write operations. After the primary site is repaired, an administrator must cause the software to replicate over to the secondary site by first issuing the sndradm -l command from the primary site.
To resume Remote Mirror software operations from the primary site, use the sndradm -m command to perform a full resynchronization or the sndradm -u command to perform an update resynchronization.
When issued from the secondary site, the sndradm -l command does not work on any volume that is currently synchronizing.
An update resynchronization updates the secondary site with changes that occurred at the primary site while replication was suspended. The primary site can also be updated from the secondary, if desired.
A full synchronization is a complete disk-to-disk copy. This operation is the most time-consuming of the synchronization operations. A full synchronization is typically performed only when the Remote Mirror software volume set is:
The volume set data might be in question if, for example, a double disk failure on a RAID-5 set occurred or if the Remote Mirror software was shut down manually and write operations occurred to either the primary or secondary volumes when logging was not active. If the integrity of the volume data is questioned, the only way to get the volume to a synchronized state is to perform a full volume copy. The full copy can be performed from the primary site to the secondary site or, if appropriate, from the secondary to the primary.
|
Tip - If a synchronization initiated by the sndradm -m or sndradm -m -r commands is interrupted, use the sndradm -u update command to complete the synchronization. |
The Remote Mirror software provides two methods for synchronization after a scheduled or unscheduled link failure:
See also Using the Remote Mirror Software With the Point-in-Time Copy Software. Before you start a resynchronization operation, ensure that you have an appropriate Point-in-Time Copy of the target volume.
In a Sun
Cluster environment, consider the following when using autosynchronization:
To ensure a high level of data integrity on both sites during normal operations or data recovery, use the Remote Mirror software with the Point-in-Time Copy software. Use the Point-in-Time Copy software just before you perform a resynchronization to ensure that a consistent copy of the data exists. If a failure occurs, you have the Point-in-Time data copy to restore your data.
During the resynchronization process of updating the local and remote sites, the data on a secondary Remote Mirror volume is temporarily inconsistent with the primary volume. The secondary volume cannot be relied on for data recovery. Consistency is restored when the resynchronization is complete. To ensure data integrity, use Point-in-Time Copy software regularly to create a point-in-time copy of data at both sites. See the Point-in-Time Copy software documentation listed in Related Documentation.
The software returns the following error message in several circumstances when the shadow volume of a Point-in-Time Copy volume set is not the same size as the master volume.
Whenever a Point-in-Time Copy snapshot is taken, the volume that is used to create the snapshot, the shadow volume, is made to look exactly like the master volume, including matching the number of blocks. If the master volume is larger or smaller than the shadow volume's physical size, the shadow volume appears to be resized at the moment the snapshot is taken. Physically, the shadow volume has not changed size, but the Point-in-Time Copy kernel module always reports its size to be the same size as the master volume. This can present several problems with the Remote Mirror software, which does not expect the size of the volume to change:
The /usr/lib/sndrsyncd daemon automates update resynchronization after a network link or machine failure. If the Point-in-Time Copy software is also installed and you have added Point-in-Time Copy volume groups, the daemon invokes Point-in-Time copies when necessary to protect the data volumes being updated during a resynchronization.
When a network link in use by the Remote Mirror software becomes unavailable, the daemon attempts to execute Remote Mirror software update commands to resynchronize all volume sets that have autosynchronization enabled and are using the network link.
Use the sndradm -I command to create configuration entries marked with the ndr_ii key. The ndr_ii entries contain an additional state field that the kernel uses to determine when point-in-time copies must be made. The kernel notifies the Remote Mirror software synchronization daemon on the target system whenever a synchronization is started and waits for sndrsyncd to perform any necessary copies before allowing the synchronization to proceed.
The daemon is also notified when any Remote Mirror software resynchronization starts or finishes. The daemon performs Point-in-Time Copy operations on the secondary or target host, if configured.
On a secondary host, the daemon checks whether a file system is currently mounted on the secondary volume and informs the kernel not to allow the synchronization to start if the file system is currently mounted.
See the command description for Adding and Deleting Point-in-Time Copy Software Volumes.
This section describes three example scenarios:
|
Note - Consistency groups across interfaces are not supported in a one-to-many configuration because write ordering cannot be guaranteed. |
In a one-to-many volume set, you can replicate data from one primary volume to many secondary volumes residing on one or more hosts. The primary volume plus each secondary site volume forms a single volume set. Each volume set requires its own unique bitmap volume.
With one primary and three secondary host volumes, you need to configure three volume sets: primary A and secondary B1, primary A and secondary B2, and primary A and secondary B3. FIGURE 2-5 shows one primary and three secondary host volumes and therefore three volume sets: A and B1, A and B2, and A and B3.
|
Caution - When creating volume sets, do not create secondary or bitmap volumes using partitions that include cylinder 0. Data loss might occur. See VTOC Information. |
|
Note - Do not reverse synchronize the primary volume from more than one secondary volume or host at a time. |
When one-to-many replication is performed in synchronous mode, I/O from the primary volume is sent to the first secondary volume in the configuration (A+B1). The software does not wait for any I/O acknowledgment before starting to send the I/O to the second secondary volume in the configuration (B2). Writes are queued and processed in parallel. This pattern is repeated until I/O is acknowledged on all secondary volumes in the one-to-many configuration.
In a synchronous one-to-many configuration, the latency at the primary host is the combined I/O latency for each connection to a secondary host and for each disk access on the secondary hosts.
When one-to-many replication is performed in asynchronous mode, I/O is queued at the primary host for later transmission and acknowledgment for every secondary host. This scheme allows replication to proceed in parallel during one-to-many asynchronous replications.
The Remote Mirror software also supports the replication of volumes located on different hosts to volumes on a single host. The terminology differs from the one-to-many configuration terminology, where the "one" and the "many" referred to are volumes. Many-to-one configuration refers to the ability to replicate volumes across more than two hosts through more than one network connection. An example of a many-to-one configuration is shown in FIGURE 2-6.
FIGURE 2-6 shows a simple use of the many-to-one configuration. Host A backs up volumes on both Host B and Host C. As the Remote Mirror software does not place restrictions on many-to-one configurations, Host A can be configured as the primary host for some replicated volumes and as the secondary host for others.
In a multihop set, the secondary host volume of one volume set can act as the primary host volume of another volume set (it is still the secondary volume of the first volume set). In the case of one primary A and one secondary host volume B, the secondary host volume B appears as primary host volume A1 to the secondary host volume B1.
FIGURE 2-7 shows one primary volume A and its secondary host volume B. The secondary host volume B also acts as the primary host volume A1 to the secondary host volume B1.
Multihop configurations can become complex and the use and administration of multihop sets must be carefully considered. Consider what happens if resynchronization operations for every volume set in the multihop chain are performed in synchronous mode. The I/O proceeds along each link of the chain and the I/O acknowledgment is not confirmed until the last link is reached, at which point the process is complete.
If both sets were configured to replicate synchronously in the example in FIGURE 2-7:
In a multihop configuration where every set in the chain is configured to replicate synchronously, the I/O latency at the primary node (assuming a forward replication) is the combined latency of every link and disk access along the chain.
Conversely, when volume sets are part of a multihop configuration where all sets replicate asynchronously, the contents of any given non-primary volume is unpredictable with respect to its neighbor until the resynchronization is complete on all nodes.
These examples are for illustration only. The Remote Mirror software does not place any restrictions on the configurations between sets along the chain. A mix of synchronous and asynchronous sets is most useful.
As another example, configure the A+B volume set as a synchronous set running over a dark fiber in the same room (to ensure a consistent copy of the volume without adversely affecting performance on the primary site). Make the A1+B1 volume set an asynchronous set, running across a network to a remote location (to replicate the volume to a remote location at a comparatively fast rate).
Multihop configurations can be expanded and the performance of these configurations improved when the Point-in-Time Copy software and the Remote Mirror software are used together.
| Sun StorageTek Availability Suite 4.0 Remote Mirror Software Administration Guide | 819-6148-10 |
|
Copyright © 2006, Sun Microsystems, Inc. All Rights Reserved.
