Introduction to Replication

Replication is a topology wide feature that always involves more than one participant.

Replication works as follows:

1. A master receives a change. Once the change has been applied to the entry in the database, then because the server is a master, it stores the change in the change log database.

2. The master updates its Replica Update Vector (RUV).

3. The master notifies the replication threads that a new change has been recorded in the change log.

4. These replication threads contact replication partners to propagate the information.

For example, Master 1 receives a change, applies it to the entry and updates its change log. When master 1 contacts the consumer, the consumer shows that master replica its RUV. The master looks at the RUV and compares it with its own RUV to see if it contains more recent changes than the consumer. If, for example, it sees that the consumer contains a higher RUV, it does not send changes. If it contains a more recent change, it sends another request to the consumer asking for a lock on replica ID 1 so that it can make updates. If the lock is unavailable, the update will be made later. If the lock is available, then the master can proceed to make the change.

This introduction to replication addresses the following topics:

• Suppliers and Consumers

• Unit of Replication

• Replica Identity

• Replication Agreements

• Types of Replica

• Replication Authentication

• Replication Change Log

• Change Sequence Number

• Replica Update Vector

• Deleted Entries: Tombstones

• Consumer Initialization and Incremental Updates

• Referrals and Replication

Suppliers and Consumers

A Directory Server that replicates to other servers is called a supplier. A Directory Server that is updated by other servers is called a consumer. The supplier replays all updates on the consumer through specially designed LDAP v3 extended operations. In terms of performance, a supplier is therefore likely to be a demanding client application for the consumer.

A server can be both a supplier and a consumer, as in the following situations:

• In multi-master replication, a master replica is mastered on two different Directory Servers. Each server acts as a supplier and a consumer of the other server.

• When the server contains a hub replica, the server receives updates from a supplier and replicates the changes to consumers.

A server that plays the role of a consumer only is called a dedicated consumer.

For a master replica, the server must do the following:

• Respond to update requests from directory clients

• Maintain historical information and a change log

• Initiate replication to consumers

The server that contains the master replica is responsible for recording any changes made to the master replica and for replicating these changes to consumers.

For a hub replica, the server must do the following:

• Respond to read requests

• Refer update requests to the servers that contain a master replica

• Maintain historical information and a change log

• Initiate replication to consumers

For a consumer replica, the server must do the following:

• Respond to read requests

• Maintain historical information

• Refer update requests to the servers that contain a master replica

Unit of Replication

The smallest logical unit of replication is a suffix, also known as a naming context. The term suffix arises from the way the base DN for the naming context is a suffix for all DNs in that context. For example, the suffix dc=example,dc=com contains all directory entries in the Example.com naming context.

The replication mechanism requires one suffix to correspond to one database. The unit of replication applies to both suppliers and consumers. Therefore, two suffixes on a master replica cannot be replicated to one suffix on a consumer replica, and vice versa.

Replica Identity

Master replicas require a unique replica identifier that is a 16-bit integer between 1 and 65534. Consumer and hub replicas all have the replica ID of 65535. The replica ID identifies the replica on which changes are made.

If multiple suffixes are configured on one master, you can use the same replica ID for each suffix on the master. In this way, when a change is made on that replica ID, it is possible to identify the server on which change was made.

Types of Replica

A suffix that participates in replication is called a replica. There are three kinds of replica:

• A master replica is a read-write database that contains a master copy of the directory data. A master replica can perform the following tasks:

  • Respond to update requests and read requests from directory clients

  • Maintain historical information and a change log for the replica

  • Initiate replication to consumers or hubs

• A consumer replica is a read-only database that contains a copy of the information held in a master replica. A consumer replica can perform the following tasks:

  • Respond to read requests

  • Maintain historical information for the replica

  • Refer update requests to servers that contain a master replica

• A hub replica is a read-only database, like a consumer replica, but stored on a directory server that supplies one or more consumer replicas. A hub replica can perform the following tasks:

  • Respond to read requests

  • Maintain historical information and a change log for the replica

  • Initiate replication to consumers

  • Refer update requests to servers that contain a master replica

A single instance of Directory Server can be configured to manage several replicas.

A replica can act as a supplier of updates, or a consumer of updates, or both.

• A supplier is a replica that copies information to another replica.

  A master replica can be a supplier to a hub replica and a consumer replica. A hub replica can be a supplier to a consumer replica. In multi-master replication, one master replica can be a supplier to another master replica.

• A consumer is a replica that receives updates from another replica.

  A hub replica and a consumer replica can be consumers of a master replica. A consumer replica can be a consumer of a hub replica. In multi-master replication, one master replica can be a consumer of another master replica.

A replica can be promoted or demoted to change its behavior with respect to other replicas. Dedicated consumers can be promoted to hubs, and hubs can be promoted to masters. Masters can be demoted to hubs, and hubs can be demoted to dedicated consumers.

A server that contains a consumer replica only is called a dedicated consumer.

Replication Agreements

Replication agreements define the relationships between a supplier and a consumer. The replication agreement is configured on the supplier. A replication agreement contains the following replication parameters:

• The suffix to replicate.

• The consumer server to which the data is pushed.

• The replication schedule.

• The bind DN and credentials the master must use to bind to the consumer.

• How the connection is secured.

• Which attributes to exclude or include in fractional replication, if fractional replication is configured.

• The group and window sizes to configure the number of changes you can group into one request and the number of requests that can be sent before consumer acknowledgement is required.

• Information about the replication status for this agreement.

• The level of compression used in replication on Solaris and Linux systems.

Replication Authentication

Before a master can update a consumer, the consumer authenticates the master by using a special entry called the Replication Manager entry. The master uses the Replication Manager entry to bind to the consumer.

The Replication Manager entry has a special user profile that bypasses all access control rules defined on the consumer server. The special user profile is only valid in the context of replication.

The Replication Manager entry has the following characteristics.

• On a consumer server, the Replication Manager is the user who is allowed to perform updates. The entry for Replication Manager must be present for all replicas.

• The bind DN of the Replication Manager entry is set in the replication agreement. The bind DN must be configured for hubs, or masters to point to an existing Replication Manager entry.

• For initialization and security reasons, the Replication Manager entry cannot be part of the replicated data.

The Replication Manager entry is created by default when you configure replication through the browser-based interfaceDirectory Service Control Center. You can also create your own Replication Manager entry. For information about how to create a Replication Manager entry, see Using a Non-Default Replication Manager in Oracle Directory Server Enterprise Edition Administration Guide.

Authentication can be performed in the following ways for SSL with replication.

• For SSL server authentication, you must have a Replication Manager entry, and its associated password, in the server you are authenticating to.

• For SSL client authentication, you must have an entry that contains a certificate in the server you are authenticating to. This entry may or may not be mapped to the Replication Manager entry.

Replication Change Log

All modifications received by a master replica are recorded in a change log. A change log is maintained on all master replicas and hub replicas.

If your application needs to read the change log, use the retro change log plug-in for backward compatibility. For more information about the retro change log plug-in, see Replication and the Retro Change Log Plug-In.

Change Sequence Number

Each change to a master replica is identified by a change sequence number, CSN. The CSN is generated by the master server and is not visible to the client application. The CSN contains the timestamp, a sequence number, the replica ID, and a subsequence number. The change log is ordered by the CSN.

Replication is sequential, meaning that entries are replicated in an orderly way. Because replication is orderly, any change generated by a master is labeled by a change sequence number (CSN) that is unique for any change inside a multi-master topology. The CSN is a hexadecimal string that appears in the logs as follows:

41e6ee93000e00640000

The first 8 hexa-digits represent the time when the change was generated in the master. The time is represented in seconds since January 1, 1970.

The next four digits are the sequence number, or the order in the current second in which the change occurred. For example, multiple changes occur in second 41e6ee93. The sequence number tells us the progressive numbering of the change.

The next four digits specify the replica ID of the master that received the change in the first place.

The last four digits are reserved. Most of the time, they are 0000.

CSNs are generated only when local traffic introduces a new change to a replica. So only masters that receive updates generate CSNs. Consumers always refer to masters, because all the updates they receive are through replication.

Replica Update Vector

The replica update vector, RUV, identifies the state of each replica in a topology. Stored on the supplier and on the consumer, the RUV is used to establish which changes need to be replicated. The RUV stores the URL of the supplier, the ID of the supplier, the minimum CSN, and the maximum CSN.

Any replica in a replication topology stores its current replication state in a replica update vector (RUV). The RUV is stored in memory by a process that is running and provides the exact knowledge this replica has of itself and every other participant in the replication topology. The RUV entry on a given server contains a line for each master participating in a replication topology. Each line contains an identifier of one of the masters, the URL of the replica, and the CSN of the first and last changes made on the server. The CSN records only the first and last changes known by the server, not necessarily the most recent changes made by the master.

The RUV is mainly in memory and can be accessed using ldapsearch on the cn=replica,cn=suffix,cn=mapping tree,cn=config entry. For example, an ldapsearch for the ou=people suffix might yield the following results:

# ldapsearch -h host1 -p 1389 -D "cn=Directory Manager" -w secret \
-b "cn=replica,cn=ou=people,cn=mapping tree,cn=config" \
-s base objectclass=* nsds50ruv

nsds50ruv: {replicageneration} 45e8296c000000010000
nsds50ruv: {replica 1 ldap://server1:1389} 45ed8751000000010000 4600f252000000010000
nsds50ruv: {replica 2 ldap://server1:2389} 45eec0e1000000020000 45f03214000000020000

For clarity, we will simplify the RUV syntax to CSNchangenumber-replicaid. The change-number shows which change the RUV corresponds to in the successive changes that occurred on the master. For example, 45ed8751000000010000 can be written as CSN05-1. In the previous illustration, master 1 contains the following RUVs:

replica 1: CSN05-1 CSN43-1
replica 2: CSN05-2 CSN40-2

The first line provides information about the first change and the last change that this replica knows about from itself, master 1, as indicated by the replica ID 1. The second line provides information about the first change and the last change that it knows about from master 2. The information that is most interesting to us is the last change. In normal operations, master 1 should know more about the updates it received than master 2. We confirm this by looking at the RUV for master 2:

replica 2: CSN05-2 CSN50-2
replica 1: CSN01-1 CSN35-1

Looking at the last change, we see that master 2 knows more about the last change it received (CSN50-2) than master 1 (which shows the last change as having occurred at CSN40-2). By contrast, master 1 knows more about its last change (CSN43-1) than master 2 (CSN35-1).

When troubleshooting problems with replication, the CSNs can be useful in identifying the problem. Master 1 should always know at least as much about its own replica ID as any other participant in the replication topology because the change was first applied on master 1 and then replicated. So, CSN43-1 should be the highest value attributed to replica ID 1 in the topology.

A problem is identified if, for example, after 30 minutes the RUV on master 1 is still CSN40-2 but on master 2 the RUV has increased significantly to CSN67-2. This indicates that replication is not happening from master 2 to master 1.

If a failure occurs and you need to reinitialize the topology while saving as much data as possible, you can use the RUV picture to determine which machine contains the most recent changes. For example, in the replication topology described previously you have a hub that contains the following RUV:

2: CSN05-2 CSN50-2
1: CSN05-1 CSN43-1

In this case, server 1 seems like a good candidate for providing the most recent changes.

RUVs can be read through nsds50ruv(5dsconf) and ds6ruv(5dsconf) attributes.

Deleted Entries: Tombstones

Directory entries deleted on one replica are maintained by Directory Server until no longer needed for replication. Such deleted entries are called tombstones, as they have objectclass: nsTombstone. In rare cases, you might need to remove tombstones manually over LDAP.

Tombstones are visible only to Directory Manager. Furthermore, tombstones show up only in a search with filter (objectclass=nsTombstone). The following ldapsearch command returns tombstone entries under dc=example,dc=com.

$ ldapsearch -D "cn=Directory Manager" -b dc=example,dc=com "(objectclass=nsTombstone)"

Consumer Initialization and Incremental Updates

During consumer initialization, or total update, all data is physically copied from a master to a consumer. When you have created a replication agreement, the consumer defined by that agreement must be initialized. When a consumer has been initialized, the master can begin to replay, or replicate, update operations to the consumer. Under normal circumstances, the consumer should not require further initialization. However, if the data on a master is restored from a backup, it might be necessary to reinitialize the consumers that depend on that master.

In a multi-master replication topology, the default behavior of a read-write replica that has been re-initialized from a backup or from an LDIF file, is to refuse client update requests. By default, the replica remains in read-only mode until it is configured to accept updates again. You set the suffix property repl-accept-client-update-enabled to on using the dsconf set-suffix-prop command when the oldest updates are on the read-only replica.

When a consumer has been initialized, replication updates are sent to the consumer when the modifications are made on the supplier. These updates are called incremental updates. A consumer can be incrementally updated by several suppliers at once, provided that the updates originate from different replica IDs.

The binary copy feature can be used to clone master replicas or consumer replicas by using the binary backup files of one server to restore another server. For information about how to use binary copy for replication, see Initializing a Replicated Suffix by Using Binary Copy in Oracle Directory Server Enterprise Edition Administration Guide.

Referrals and Replication

When a consumer receives a request to modify data, it does not forward the request to the server that contains the master replica. Instead, it returns to the client a list of the URLs of the masters that can satisfy the request. These URLs are called referrals.

The replication mechanism automatically configures consumers to return referrals for all known masters in the replication topology. However, you can also add your own referrals and overwrite the referrals set automatically by the server. The ability to control referrals helps enables you to perform the following tasks:

• Point referrals to secure ports only

• Point to a Directory Proxy Server instead for load balancing

• Redirect to local servers only in the case of servers separated by a WAN

• Limit referrals to a subset of masters in four-way multi-master topologies

Directory Proxy Server is able to follow referrals.