Detecting and Resolving Replication Inconsistencies

Directory server replication has been designed to ensure that replicated databases remain consistent, even in the case of hardware faults, directory server restarts, or network failures. Despite these efforts, however, it is possible that hardware failures (disk errors, memory errors) or software errors (causing memory corruption) might lead to inconsistent databases.

These topics explain how to detect replication inconsistencies, and how to resolve them when they are identified.

Types of Replication Inconsistencies

When inconsistencies occur, they might remain hidden for some time or they might trigger replication or application errors. Examples of inconsistencies include the following:

• An entry is present on all but one directory server in the replication topology.

• An entry has a DN on one directory server that is different to its DN on all other directory servers.

• An entry has different attributes on one directory server than on other directory servers in the replication topology.

Detecting Inconsistencies

Use the following methods to check for replication inconsistencies:

• Check for information in the replication log file. The replication log file is configured by default and lists inconsistencies that are detected by the replication mechanism. Imagine, for example, that a modify operation is performed on an entry that is missing from one directory server in the topology. When replication attempts to replay this operation to that server, it will detect the problem and produce an error in the logs/replication error log. This kind of error will not stop replication, but the operation will not be replayed and the administrator will need to repair the inconsistency.

• Pay attention to errors reported by client applications or users. Client applications or users might experience errors when accessing the directory server that might be due to replication inconsistencies.

• Make regular checks for database consistency. With the current directory server release, these checks must be performed manually, using searches or database exports.

Resolving Inconsistencies

If a replication inconsistency is found on a single directory server in the topology, it is not possible to fix this inconsistency using regular LDAP operations. This is because the LDAP operation itself would be replicated to the other directory servers in the topology and might cause damage on those servers. In addition, the fix might involve modifying attributes that are generated by the directory server, such as the entryuuid or modifyTimestamp attributes. Such attributes cannot be modified by regular LDAP operations.

Replication repair operations must therefore be done using LDAP operations that specify the Replication Repair Control (OID: 1.3.6.1.4.1.26027.1.5.2).

---

Caution - Because the replication repair control allows you to skip several controls usually done by the directory server, it should be used with great care and only when consistency problems have been detected and asserted.

---

The repair control alters the regular processing of an operation as follows:

• The operation can modify attributes that might not normally be modified or added (NO-USER-MODIFICATION), such as entryuuid and ds-sync-hist.

• No replication change number is associated with the operation.

• The operation is not published to the replication server and is therefore a local-only operation.

• Replication does not try to resolve conflicts or to generate historical information for this operation.

• Most of the schema checks are not performed for this operation.

For example, the following ldapmodify operation repairs an entry on host1 only, with the changes contained in the file changes.ldif:

$ ldapmodify -J 1.3.6.1.4.1.26027.1.5.2 -h localhost -p 1389 \
  -D "cn=Directory Manager" -w password -f changes.ldif

When you repair an entry, you must repair all of its regular attributes as well as the attributes generated by the directory server, such as modifyTimestamp, modifiersName, createTimestamp, creatorsName, and ds-sync-hist. The values of these attributes should be read from a directory server that contains the correct values, and recreated on the server with faulty values.

The ds-sync-hist attribute contains historical information that replication uses to solve modify conflicts. This attribute can only be viewed by an administrator.

Solving Naming Conflicts

Entries with identical DNs can be created on separate directory servers if they are created before the servers replicate the changes to each other. When the remote operation is replicated to the local server, a naming conflict occurs. The naming conflict results in the creation of a conflict entry on the local server.

Conflict entries have a specific DN, of the form entryuuid=entryUid+oldRDN. Every conflict entry includes a ds-sync-conflict attribute, whose value is the DN of the conflicting regular entry.

For example, imagine that the entry cn=bjensen,ou=People,dc=example,dc=com is created simultaneously on two directory servers. The entry on server 1 is given a unique ID of uid1 and the entry on server 2 is given a unique ID of uid2. Both directory servers will have the following two entries after replication:

cn=bjensen,dc=example,dc=com
...
entryuuid=uid2+cn=bjensen,dc=example,dc=com
  ds-sync-conflict:cn=bjensen,dc=example,dc=com

When you have identified the conflicting entry, you can rename it so that it has a unique DN.

If the naming attribute in a conflicting entry is multi-valued, you can rename the conflicting entry as follows:

1. Rename the entry while keeping the old RDN value, for example:

   $ ldapmodify -h localhost -p 1389 -D "cn=Directory Manager" -w password
   dn: entryuuid=uid2+cn=bjensen,dc=example,dc=com
   changetype: modrdn
   newrdn: cn=bljensen
   deleteoldrdn: 0
   ^D

   You cannot delete the old RDN value in this step because it also contains the entryuuid operational attribute, which cannot be deleted.

2. Remove the old RDN value of the naming attribute and the conflict marker attribute, for example:

   $ ldapmodify -h localhost -p 1389 -D "cn=Directory Manager" -w password
   dn: cn=bljensen,dc=example,dc=com
   changetype: modify 
   delete: cn
   cn: bjensen
   delete: ds-sync-conflict
   ^D

If the naming attribute in a conflicting entry is single-valued, for example dc (domain component), you cannot simply rename the entry to another value of the same attribute. Instead, you must give the entry a temporary name, as follows:

1. Rename the entry by using a different naming attribute, and keep the old RDN, for example::

   $ ldapmodify -h localhost -p 1389 -D "cn=Directory Manager" -w password
   dn: entryuuid=uid2+dc=HR,dc=example,dc=com
   changetype: modrdn
   newrdn: o=TempHR
   deleteoldrdn: 0
   ^D

   You cannot delete the old RDN value in this step because it also contains the entryuuid operational attribute, which cannot be deleted.

2. Change the desired naming attribute to a unique value and remove the conflict marker attribute, for example:

   $ ldapmodify -h localhost -p 1389 -D "cn=Directory Manager" -w password
   dn: o=TempHR,dc=example,dc=com
   changetype: modify 
   replace: dc
   dc: NewHR
   delete: ds-sync-conflict
   ^D

3. Rename the entry back to the intended naming attribute and delete the temporary RDN, for example:

   $ ldapmodify -h localhost -p 1389 -D "cn=Directory Manager" -w password
   dn: dc=NewHR,dc=example,dc=com
   changetype: modrdn 
   newrdn: dc=NewHR
   deleteoldrdn: 1
   ^D