Using the External Change Log

The External Change Log (ECL) publicizes all changes that have occurred in a directory server database and is particularly useful for synchronizing the LDAP directory with other subsystems. The ECL logs the changes made in replicated suffixes only — changes made in non-replicated suffixes are not logged.

This topic describes how to enable the ECL in your directory service and how to configure client applications so that they can access the ECL.

For information about the architecture of the ECL, see External Change Log in Oracle Fusion Middleware Architecture Reference for Oracle Unified Directory. For a description of the ECL internals that will enable you to port applications relying on other change logs, see Porting Applications That Rely on Other Change Logs in Oracle Fusion Middleware Architecture Reference for Oracle Unified Directory.

This section covers the following topics:

• Enabling the External Change Log in Oracle Unified Directory

• External Change Log APIs

• How a Client Application Uses the External Change Log in Cookie Mode

• Format of External Change Log Entries

• Initializing Client Applications to Use the External Change Log

• Controlling Access to the External Change Log

• Purging the External Change Log

• To Disable the External Change Log for a Domain

Enabling the External Change Log in Oracle Unified Directory

The ECL is available by default when replication is configured in one of the following ways:

• By configuring a directory server as part of a replicated topology during installation. For more information, see Setting Up Replication During Installation in Oracle Fusion Middleware Installation Guide for Oracle Unified Directory.

  By using this method, you can conceivably set up replication on a standalone server, which will enable you to have access to an ECL on a standalone server.

• By configuring replication after installation, by using the dsreplication command. For more information, see Configuring Data Replication With dsreplication.

  ---

  Note - The ECL is not available if you configured replication with the --onlyReplicationServer or --noReplicationServer options.

  ---

To verify that the ECL is configured on a directory server instance, run the following search command:

$ ldapsearch -h localhost -p 1389 -D "cn=directory manager" -w password \
  -s base -b "" "objectclass=*" namingContexts
dn:  
namingContexts: cn=changelog 
namingContexts: dc=Europe,dc=com 
namingContexts: dc=us,dc=com

External Change Log APIs

The ECL can function supports two APIs, which enable two distinct modes of operation:

• Cookie mode. This is the recommended API that you should use to access the ECL.

  In cookie mode, the client application provides an ECL exchange control in its request to the server. In this mode, the DIT and schema provided in the entries that are returned by the server are not compatible with the LDAP change log draft (http://tools.ietf.org/html/draft-good-ldap-changelog-04).

• Draft-compatible mode. This mode should be used only by existing applications that rely on the LDAP change log draft.

  In this mode, the DIT and schema provided in the entries that are returned by the server are compatible with the LDAP change log draft.

  For improved performance and for simplicity, you should port client applications to use the cookie mode. For more information, see Porting Applications That Rely on Other Change Logs in Oracle Fusion Middleware Architecture Reference for Oracle Unified Directory.

How a Client Application Uses the External Change Log in Cookie Mode

Each entry in the ECL has an associated cookie. When a client application sends a SEARCH request, the application provides either the cookie of the last message that was read from the ECL (in a previous SEARCH), or an empty value. The server returns the ECL entries associated with that cookie.

Each entry is returned with its associated cookie. When the application disconnects, it stores the last cookie that it received, and provides this cookie to the server with its next SEARCH request.

This transmission of ECL cookies is illustrated in the following diagram.

The content of the cookie is not a public interface for the client application. The client application sends the cookie as a request control and the server sends the cookie as a response control.

The cookie exchange control has an OID of 1.3.6.1.4.1.26027.1.5.4. If the server identifies that the cookie provided by the application is corrupted, the request is rejected. The request is also rejected if the server identifies that the configuration of the ECL has changed since the server sent this cookie to the application, or that the ECL has been purged and the oldest change stored is newer than the cookie value. In this case, additional information is returned, indicating that a full resynchronization of the external application is recommended.

The following request and response examples indicate how the client application searches using the external change log and how the ECL responds.

Request One

To start reading the ECL, the client sends the first SEARCH request on cn=changelog, specifying an empty value in the cookie exchange control.

$ ldapsearch -h localhost -p 1389 -D "cn=directory manager" -w password \
  --control "1.3.6.1.4.1.26027.1.5.4:false:;" -b "cn=changelog" "(objectclass=*)" "*" +

Response One

The server sends each change to the client in a SearchResultEntry. The cookie attribute specifies the new cookie value. This value is also sent in a cookie exchange control, along with the entry.

# Public changelog exchange control(1.3.6.1.4.1.26027.1.5.4): 
  dc=europe,dc=com:0000012187eae081456200000001;o=example:;
dn: replicationcsn=0000012187eae081456200000001,dc=europe,dc=com,cn=changelog
objectClass: top
objectClass: changeLogEntry
replicationCSN: 0000012187eae081456200000001
replicaIdentifier: 17762
targetDN: cn=chek-piao chea,ou=unit1,o=people,dc=europe,dc=com
changeTime: 20090528155105Z
changes:: cmVwbGFjZTogc2VlQWxzbwpzZWVBbHNvOiBjbj1tY29uZmlnCi0KcmVwbGFjZTogbW9kaW
  ZpZXJzTmFtZQptb2RpZmllcnNOYW1lOiBjbj1EaXJlY3RvcnkgTWFuYWdlcixjbj1Sb290IEROcyxjb
  j1jb25maWcKLQpyZXBsYWNlOiBtb2RpZnlUaW1lc3RhbXAKbW9kaWZ5VGltZXN0YW1wOiAyMDA5MDUy
  ODE1NTEwNVoKLQo=
changeType: modify
changeLogCookie: dc=europe,dc=com:0000012187eae081456200000001;
targetEntryUUID: 08d1830c-02f1-34a6-9cf4-8d1270ec1db0
changeNumber: 0

Request Two

To read the ECL from the last returned entry, the client sends the SEARCH request on cn=changelog, specifying the last cookie value that it received in the cookie exchange control.

$ ldapsearch -h localhost -p 1389 -D "cn=directory manager" -w password
  --control "1.3.6.1.4.1.26027.1.5.4:false:dc=europe,dc=com:0000012187eae081456200000001;"
  -b "cn=changelog" "(objectclass=*)"

Format of External Change Log Entries

The DN for entries that are returned in the ECL is of the form:

replicationcsn=replicationCSN,replication-domain-DN,cn=changelog

For example:

dn: replicationcsn=0000012187eae081456200000001,dc=europe,dc=com,cn=changelog

The following attributes are returned for ECL entries:

targetDN / MUST
changeType / MUST
changeTime / MUST
changeNumber / MUST // used only for compatibility mode

changes / MAY, MUST for add, mod
newRDN / MAY, MUST for modrdn
deleteOldRDN / MAY, MUST for modrdn
newSuperior / MAY, MUST for modrdn

replicaIdentifier / MAY, OPERATIONAL / specific OUD value
replicationCSN / MAY, OPERATIONAL / specific OUD value
targetEntryuuid / MAY, OPERATIONAL / specific OUD value
changelogcookie / MAY, OPERATIONAL

To Specify the Attributes to be Included in the External Change Log

By default, attributes are included in the ECL only if they are affected by a change operation. So, for example, if the sn attribute of an entry is modified, only that attribute will appear in the ECL. You can, however, specify a list of attributes that will be included in the ECL regardless of whether they are affected by a change operation.

Use the following procedure to specify the list of attributes that must be included in the ECL.

• Use the dsconfig command to set the value of the ecl-include property.

  For example, to specify that the cn, and sn attributes always be included in the ECL if an entry is modified, run the following command:

  $ dsconfig -h localhost -p 4444 -D "cn=directory manager" -w password -Q -n -X \
    --provider-name "Multimaster Synchronization" --domain-name dc=example,dc=com \
    --add ecl-include:cn --add ecl-include:sn

  ---

  Note - In the ECL entry that is returned by the server, the attribute name is prefixed with target. For example, in the previous example, the ECL entries for changes on dc=example,dc=com will always contain the attributes targetcn and targetsn. The values of these attributes will be the values of the cn and sn attributes of the entry before it was modified or moved.

  ---

Initializing Client Applications to Use the External Change Log

No specific server configuration is required for clients to use the ECL. However, any client application that needs to use the ECL must be initialized as described in the following sections.

To Initialize a Client Application to Use the External Change Log

1. Read the last ECL cookie value from the LDAP server.

   This is the value of the lastExternalChangelogCookie attribute of the root DSE. For example:

   $ ldapsearch -h localhost -p 1389 -D "cn=directory manager" -w password \
     -s base -b "" "objectclass=*" lastExternalChangelogCookie
     dn:
     objectClass: top
     objectClass: ds-root-dse
     lastExternalChangelogCookie: dc=europe:00000121cea5221c04b100000005 \
       00000121cea5319e04b400000009;

2. Export the Oracle Unified Directory database.
3. Initialize the application from the exported database.

   The application can now start reading the ECL by providing the last cookie value as the value of the search control. For example:

   $ ldapsearch -h localhost -p 1389 -D "cn=directory manager" -w password
     --control "1.3.6.1.4.1.26027.1.5.4:false:dc=europe:00000121cea5221c04b100000005 \
       00000121cea5319e04b400000009" -b "cn=changelog" "(objectclass=*)"

Reinitializing a Client Application When a Domain is Added

When a new replication domain is added to a topology, the ECL is enabled on that domain by default. Client applications that use the ECL must be reinitialized for the new domain.

The server enforces this requirement by rejecting SEARCH operations if the cookie that is provided does not refer to the new domain. The operation result code is UNWILLING TO PERFORM. The server provides a detailed message that includes a list of the domains that are missing and a cookie value for a possible partial initialization.

The client application must be reinitialized using one of the following methods:

• Full reinitialization. The application is reinitialized for all domains.

  1. Read the value of the lastExternalChangelogCookie attribute. This value refers to all domains in the topology, including the new domain.

  2. Export the database for all domains, including the new domain.

  3. Initialize the application for all domains from the export output. For more information, see To Initialize a Client Application to Use the External Change Log.

  4. The application can now search the ECL using the last_cookie_from_dse_root.

• Partial reinitialization. The application is reinitialized only for the new domain.

  1. Export the database for the new domain only.

  2. Initialize the application from the export output, which contains only the entries in the new domain. For more information, see To Initialize a Client Application to Use the External Change Log.

  3. The application can now search the ECL, using the cookie value for a possible partial initialization that was returned by the server in its UNWILLING TO PERFORM error. Note that this might result in some updates that have already been processed being replayed, because the cookie value represents the initial state of the database.

Reinitializing a Client Application When a Domain is Removed or Disabled

When a replication domain is removed from a topology (or when the ECL is disabled for a specific domain), client applications must be alerted to the fact that no more changes will occur on that domain.

The server enforces this requirement by rejecting SEARCH operations if the cookie that is provided refers to the removed domain. The operation result code is UNWILLING TO PERFORM. The server provides a detailed message, that includes a list of the domains that are present in the cookie but have been removed (or for which the ECL has been disabled), and a cookie value for a possible continuation.

The client application can use one of the following methods to handle the removed domain:

• Smooth continuation. In this case, the application applies its own policy of what to do when a domain is removed. To assist with the formulation of this policy, the application can search the ECL by providing the cookie value for a possible continuation that is returned by the server in the error message.

• Full reinitialization. The application is reinitialized for all domains.

  1. Read the value of the lastExternalChangelogCookie attribute. This value refers to all domains in the topology, excluding the removed domain.

  2. Export the database for all domains.

  3. Initialize the application for all domains from the export output. For more information, see To Initialize a Client Application to Use the External Change Log.

  4. The application can now search the ECL using the lastExternalChangelogCookie.

Controlling Access to the External Change Log

Access to the ECL is ruled by global ACIs that can be configured on the server. By default, only the root user can access the ECL.

For information about configuring global ACIs, see Managing Global ACIs With dsconfig.

Purging the External Change Log

The ECL is purged simultaneously with the replication change log. For information about changing the interval at which the replication change log is purged, see Changing the Replication Purge Delay.

Sometimes, an application might submit a search request on the ECL, providing a cookie value that is older than the oldest change stored on the server (because a purge has occurred since the last request from that application). In this case, the server rejects the requests and indicates that the cookie is too old and that a full resync is required.

To Disable the External Change Log for a Domain

In certain situations, you might want to exclude changes on a specific domain from the external change log. You can disable the ECL for a specific replication domain, which prevents changes to this domain from being published in the ECL.

1. Obtain the domain name, as described in Retrieving the Replication Domain Name.
2. Set the external changelog domain properties for that domain.

   For example, to prevent changes to the schema from being published in the ECL, run the following dsconfig command:

   $ dsconfig -h localhost -p 4444 -D "cn=directory manager" -w password -n \
     set-external-changelog-domain-prop \
     --provider-name "Multimaster Synchronization" --domain-name cn=schema \
     --set enabled:false