Chapter 20
ldiffer.pl
The ldiffer.pl (LDAP difference) tool is a Perl script that detects differences between two LDAP directories and can perform modifications accordingly. Comparisons are based on attribute values so that directories with different DN structures and different schemas may be compared.
When the tool detects comparable entries in each directory, it can be configured to add or modify attributes in either entry. The value of new or modified attributes is given as a regular expression involving other attribute values. This flexibility can be used to migrate data to a new schema or to import specific data from one directory into another.
This chapter contains the following sections:
Command Usage
The ldiffer.pl tool interacts with two directories. One is called "sequential" and can be considered the source of information. The other is called "random" and can be considered the target to be modified. However, the tool also allows some modifications to be made in the source directory.
The script uses the ldapsearch tool (see Chapter 3) to extract a set of entries from both directories. The bind credentials, the base DN, and the filter for each search are specified separately, meaning that arbitrarily different sets of entries may be compared.
Entries from each set are matched based on equality between the value of one or more key attributes. This allows you to match entries that do not necessarily have the same DN. Because only values are compared, key attributes may also have different names in each directory.
Within a pair of matched entries, the tool then checks the values of one or more comparison attributes. A discrepancy between any of the comparison attribute values will trigger the tool to perform a set of update actions. An update action involves modifying an attribute in one of the entries based on the value of an attribute in its matched entry, after processing that value with a regular expression. The ldiffer.pl script uses the ldapmodify tool (see Chapter 4) to modify entries.
For example, an administrator may wish to migrate information from an old directory under the suffix o=Siroe.com,c=us to a new directory under dc=Siroe,dc=com. The key attribute might be employee login names that would be invariant. Then, ldiffer.pl could compare an attribute such as fax in a custom schema to facsimileTelephoneNumber in the standard schema of the new directory. Finally, the value of facsimileTelephoneNumber can be updated with the value of the fax attribute and with, for example, a new area code.
When matching entries are not found, the tool may also be configured to either delete the entry in the source directory, add it to the target directory, or both.
Due to the flexibility it offers, the ldiffer.pl script requires many parameters to describe the modifications of one or both target directories. These parameters are given in a configuration file that is named on the command line.
Syntax
The ldiffer.pl tool has the following syntax:
-
ldiffer.pl configFile
Where:
configFile
is the name of a file that contains all of the configuration parameters for binding to the directories, performing comparisons, and how to make any potential modifications. These parameters are described in "Configuration File Format".
All output from the script is written to log files with names and locations based on that of the configuration file:
configFile
.timestamp
.action.log - Displays the comparisons, additions, and modifications that were performed in a given run. Also, gives a count of the differences and lists the entries that were not successfully matched.
configFile
.timestamp
.debug.log - Records the commands that were executed while comparing and modifying the directories. This file is created only if the debugging parameter is specified.
The ldiffer.pl script requires Perl version 5.005_03 or later. See "Where to Find Additional Information" for links to Perl resources.
If you customize the ldiffer.pl script for added functionality, we encourage you to share your work with other LDAP users. Please post a message to the iplanet.server.idsrk public news group with your ideas or your code.
Configuration File Format
The configuration file uses the following syntax:
-
# line of comment
parameter
:value
...
The parameters may be given in any order, but are usually grouped according to the directory they affect. See the sample configuration file in "Configuration File Example".
You will need to specify values for all of the parameters in the configuration file before running ldiffer.pl. Due to the potential complexity of comparisons and updates, we recommend testing your configuration files on mock directories with the debugging parameter turned on.
The global parameters in the following table affect the behavior of the tool.
Table 20-1 Global Parameters in the ldiffer.pl Configuration File
|
Parameter
|
Purpose
|
|
ldapsearch
|
Specify the path for running the ldapsearch tool. If you installed the iPlanet DSRK in the default location, this parameter should be set to /opt/iPlanet/bin/idsrk/ldapsearch.
|
|
ldapmodify
|
Specify the path for running the ldapmodify tool. If you installed the iPlanet DSRK in the default location, this parameter should be set to /opt/iPlanet/bin/idsrk/ldapmodify.
|
|
debug
|
Indicate the use of the debugging log. The possible values are 0 for no logging or 1 for creating the log file.
|
|
delimiter
|
Set the character used internally to delimit key attribute values. You should set this parameter to a character that does not occur in the attribute values of your directories, for example !.
|
All other parameters have one of the following prefixes:
sqn_ for parameters that apply to the "sequential," or source, directory.
rnd_ for parameters that apply to the "random," or target, directory.
The connection parameters determine the LDAP directories to operate upon and give the bind credentials for accessing them.
Table 20-2 Connection Parameters in the ldiffer.pl Configuration File
|
Parameter
|
Purpose
|
|
sqn_h
sqn_p
rnd_h
rnd_p
|
Specify the host name or IP address and port number of each LDAP directory.
|
|
sqn_D
sqn_w
rnd_D
rnd_w
|
Give the bind DN and password needed to access each directory. These parameters may be left blank for anonymous binding. The bind credentials must have permissions for both searching the directories and modifying entries, if necessary. Also note that the bind DN may influence the results of searches when determining the set of entries to compare.
|
The comparison parameters determine the set of entries to compare, how to match entries from each directory, and the attribute values that will trigger actions.
Table 20-3 Comparison Parameters in the ldiffer.pl Configuration File
|
Parameter
|
Purpose
|
|
sqn_b
sqn_filter
rnd_b
rnd_filter
|
Specify the base DN and filter string used to search each directory. The filter string should be an RFC 2254-compliant LDAP search filter (see "LDAP Search Filters" in Appendix B of the iPlanet Directory Server Administrator's Guide), for example: uid=*
These parameters determine the set of entries that will be compared from each directory. These parameter values are not necessarily identical, as the ldiffer.pl tool allows you to match entries based on any attribute values, not only the DN of an entry.
|
|
sqn_key_attrs
rnd_key_attrs
|
Specify the names of the key attributes whose values must be equal in order to match an entry from each directory. These parameters may contain multiple attribute names in a comma separated list.
These lists may contain different attribute names, but they must contain the same number of names. In order for two entries to match, all attribute values must match in the order that the attribute names are given in their respective list. To match entries based on their DN, specify the dn attribute in both parameters.
|
|
sqn_comp_attrs
rnd_comp_attrs
|
Specify the names of the comparison attributes for entries from each directory. These are also comma separated lists containing an equal number of attribute names.
When two entries match according to the key attributes, the values of the comparison attributes are compared, again in the order that the attribute names are given in their respective list. If any one of the comparison attribute values differ in the pair, the two entries will be modified according to the action parameters. To perform a DN-based comparison, specify the dn attribute in both lists.
If any one of the attributes does not exist in its respective entry, the comparison will fail but no action will be triggered.
If you wish to trigger an action for every pair of matching entries, you must specify a single attribute name in each list parameter that you know will have a value and be different on each entry.
|
The action parameters in the following table determine what modifications will be made when matching entries fail the comparison or when entries fail to match. These parameters are complex lists of the following format:
-
Statement1
,Statement2
,...
Where each Statement
has the following syntax:
-
AttributeName
;otherAttribute
;regularExpression
The regularExpression
must follow the syntax for Perl regular expressions. Regular expressions may also be omitted, but the punctuation of the list must be respected, for example:
-
givenName;firstName;,sn;lastName;
Table 20-4 Action Parameters in the ldiffer.pl Configuration File
|
Parameter
|
Purpose
|
|
sqn_add_attrs
rnd_add_attrs
|
Specify the attributes to add to each of the matched entries and how to determine their initial value. These parameters are complex lists of the format described previously.
Each Statement
will create a new attribute with the given AttributeName
in the entry of the corresponding directory (either sqn or rnd). The initial value given to the attribute will be the value of the otherAttribute
in the matching entry (in the other directory), after applying the regularExpression
.
To create multivalued attributes, specify multiple add statements with the same AttributeName.
|
|
sqn_mod_attrs
rnd_mod_attrs
|
Specify the attribute values to modify and how to determine their new value. These parameters have the same complex list format as described previously.
Each Statement
will modify the value of the given AttributeName
in the entry of the corresponding directory (either sqn or rnd). The new value will be that of the otherAttribute
in the matching entry (in the other directory), after applying the regularExpression
.
|
All actions specified by parameters in the previous table will be triggered when a pair of matching entries fail the comparison. Thus, you may configure ldiffer.pl to add attributes or modify entries in both directories. Leave parameters blank if that specific action is not required. For example, if you do not wish to modify the source "sqn" directory, do not define the sqn_add_attrs or sqn_mod_attrs parameters.
The tool currently offers less flexibility for entries that do not match. After pairing up all entries in the chosen set from each directory, ldiffer.pl can only determine those entries from the source "sqn" directory that do have a match. These entries may optionally be removed from the "sqn" directory, added to the "rnd" directory, or both, according to the parameters in the following table.
Table 20-5 Unmatched Entry Parameters in the ldiffer.pl Configuration File
|
Parameter
|
Purpose
|
|
rnd_add_unmatched
|
Specify whether unmatched entries from the source "sqn" directory will be added to the target "rnd" directory. The value of this flag is either 0 for no additions or 1 for adding all unmatched entries.
When entries are added, there is no mechanism to modify their attributes or attribute values. Therefore, when this flag is true (1), the target "rnd" directory must support the same DN hierarchy and the same schema as the source "sqn" directory.
|
|
sqn_del_unmatched
|
Specify whether unmatched entries are deleted from the source "sqn" directory. The value of this flag is either 0 to leave the "sqn" directory unchanged or 1 to remove all unmatched entries.
|
Configuration File Example
The iPlanet DSRK installation includes a configuration file example named installDir
/perl/ldiffer-sample.config:
-
# Global parameters
ldapsearch:/opt/iPlanet/bin/idsrk51/ldapsearch
ldapmodify:/opt/iPlanet/bin/idsrk51/ldapmodify
debug:1
-
# the delimiter should be a character that will not
# occur in the values of key attributes to match
delimiter:!
-
# The "sequential" directory can be considered the source
sqn_h:legacy.siroe.com
sqn_p:389
sqn_D:cn=directory manager
sqn_w:bindPassword
sqn_b:o=Siroe.com,c=us
sqn_filter:login=*
sqn_key_attrs:login
sqn_comp_attrs:dn
sqn_add_attrs:branch;l;
sqn_mod_attrs:
sqn_del_unmatched:0
-
# The "random" directory can be considered the target
rnd_h:ldap.siroe.com
rnd_p:389
rnd_D:cn=directory manager
rnd_w:bindPassword
rnd_b:dc=siroe,dc=com
rnd_filter:uid=*
rnd_key_attrs:uid
rnd_comp_attrs:dn
rnd_add_attrs:
rnd_mod_attrs:mail;email;,cn;fullName;,sn;fullName;s/.*\s//
rnd_add_unmatched:0
This configuration example might be used in a hypothetical scenario where Siroe.com has undergone geographic expansion and has created a new LDAP directory using the standard schema. Some data has already been added to the new directory, such as employee uid and location (l). The rest of the needed information can be extracted from the legacy directory that has a custom schema.
Running the ldiffer.pl tool with this configuration file will match entries based on the uid attribute. Then, data from the legacy directory can be used to set attributes with standard names (mail, cn, and sn) in the new directory. If these attributes do not exist in the new directory, they will be created. The value of the surname (sn) attribute is assumed to be the second word of the legacy fullName attribute, as extracted by the Perl regular expression s/.*\s//.
However, the legacy system is still in use by the human resources department, and now that Siroe.com has several offices, it needs to update the legacy directory. The tool will copy the employees' new locations into a new attribute called branch.
After the migration, Siroe.com might also wish to keep information in the legacy directory up to date, assuming that the latest modifications occur only in the new directory. The ldiffer.pl tool can be used to automate the transfer of information back into the schema of the legacy directory. Another configuration file would be needed for this scenario, in which the attribute values in the new "rnd" directory would be used to modify attributes in the legacy "sqn" directory.
Updating or creating new directories can be highly automated through the use of the ldifxform (see Chapter 18) and ldiffer.pl tools. Together, they can modify extracted information for complex operations such as porting existing data to a new schema. However, careful planning is necessary to perform a complicated sequence of transformations. Directory modifications of this scope will require special considerations such as turning off schema checking and disallowing other read and write requests while the data is in an intermediate state.