37 Migrating Data from Other Data Repositories

This chapter describes how to migrate data from both LDAP Version 3-compatible directories and application-specific data repositories into Oracle Internet Directory.

This chapter includes the following sections:

• Section 37.1, "Introduction to Migrating Data from Other Data Repositories"

• Section 37.2, "Migrating Data from LDAP-Compliant Directories"

• Section 37.3, "Migrating User Data from Application-Specific Repositories"

37.1 Introduction to Migrating Data from Other Data Repositories

During an Oracle Internet Directory installation, Oracle Identity Management 11g Installer creates a default schema and directory information tree (DIT). Chapter 3, "Understanding Oracle Internet Directory Concepts and Architecture," and Chapter 34, "Planning, Deploying and Managing Realms," describe this default DIT framework. The framework is flexible and you can modify it to suit the needs of your deployment.

In Oracle Internet Directory, the following directory elements are created by default:

• Root Oracle Context (cn=OracleContext): This is the container where Oracle products store enterprise-wide configuration data.

• Default identity management realm (dc=dns_domain_of_host,dc=com): This is the container under which Oracle products expect to find enterprise users and groups. It approximates the enterprise DIT structure. For example, if Oracle Internet Directory is installed on a computer whose host name is: my_computer.us.my_company.com, then the default identity management realm created at installation of Oracle Internet Directory would be dc=us,dc=my_company,dc=com. Oracle products expect to find all users under the container cn=users,dc=us,dc=my_company,dc=com and all groups under cn=groups,dc=us,dc=my_company,dc=com. In addition to creating the default identity management realm entry, the Oracle Internet Directory Configuration Assistant stores a pointer to it in the Root Oracle Context so that other Oracle Internet Directory-enabled components can bootstrap themselves.

You can change this default identity management realm to suit your deployment requirements.

37.2 Migrating Data from LDAP-Compliant Directories

This section provides practical information for migrating data from an LDAP-compliant, third-party directory to Oracle Internet Directory. If you have a directory with an already-established structure, and you want to migrate the data from that directory into the default directory structure environment, then follow the instructions in this section.

Two tools that are commonly used for migrating data are bulkload and syncProfileBootstrap. Table 37-1, "Features of bulkload and syncProfileBootstrap" lists the features of bulkload, as compared with syncProfileBootstrap.

Table 37-1 Features of bulkload and syncProfileBootstrap

Feature	bulkload	syncProfileBootstrap
Speed	Fast	Slow
Data transfer method	SQL	LDAP
Input types accepted	LDIF file only	LDIF file, LDAP directory, tagged file, CSV file
Transforms data	No	Yes
Validates LDIF input	Yes	No

See Also:

• "Oracle Internet Directory Data Management Tools" in Oracle Fusion Middleware Reference for Oracle Identity Management for bulkload syntax information and examples,

• Oracle Fusion Middleware Administrator's Guide for Oracle Directory Integration Platform for more information about syncProfileBootstrap.

37.2.1 Migrating LDAP Data by Using an LDIF File and bulkload

The bulk loader, bulkload, is a command-line tool for loading a large number of entries into a directory server. It uses Oracle SQL*Loader to load the directory entries. The bulkload tool expects the input file to be in LDAP Data Interchange Format (LDIF). The bulkload tool can validate LDIF input for referential integrity, but it cannot perform any mapping or other transformation on the data.

When no translation is required and data is very large (500,000 or more), bulkload is the best choice for migrating data from a third-party directory to Oracle Internet Directory. It is fast and it can validate LDIF input.

To use this method, you must first export data from the third-party directory to an LDIF file, as shown in Figure 37-1, "Using an LDIF File and bulkload".

Figure 37-1 Using an LDIF File and bulkload

LDIF is the IETF-sanctioned ASCII interchange format for representing LDAP-compliant directory data as a file. All LDAP-compliant directories should have tools to export their contents into one or more LDIF files representing the DIT at the time of export.

See Also:

RFC 2849 of the IETF, available for download at: http://www.ietf.org

When using an LDIF file and bulkload to migrate data to Oracle Internet Directory, you must perform the following tasks.

Task 1: Export Data from the Non-Oracle Internet Directory Server into LDIF File Format

See the vendor-supplied documentation for instructions. If flags or options exist for exporting data from the foreign directory, be sure to select the method that:

• Produces LDIF output with the least amount of proprietary information included

• Provides maximum conformance to the IETF Request for Comments 2849 of the IETF, available for download at: http://www.ietf.org

Task 2: Analyze the LDIF User Data for Any Required Schema Additions Referenced in the LDIF Data

Any attributes not found in the Oracle Internet Directory base schema require extension of the Oracle Internet Directory base schema before the importation of the LDIF file. Some directories may support the use of configuration files for defining extensions to their base schema (Oracle Internet Directory does not). If you have a configuration file you can use it as a guideline for extending the base schema in Oracle Internet Directory in "Task 3: Extend the Schema in Oracle Internet Directory".

Task 3: Extend the Schema in Oracle Internet Directory

See Chapter 21, "Managing Directory Schema" for tips on how to extend the directory schema in Oracle Internet Directory. You can do this by using either Oracle Directory Services Manager or the SchemaSynch tool, which is documented in Oracle Fusion Middleware Reference for Oracle Identity Management.

If you have users who are using other Oracle products, you must create users with object class orclUserV2 and its required attributes. If you are integrating with Active Directory, you must create users with object class orclADUser and its required attributes. These object classes and their attributes are documented inOracle Fusion Middleware Reference for Oracle Identity Management.

Task 4: Remove Any Proprietary Directory Data from the LDIF File

Certain elements of the LDAP v3 standard have not yet been formalized, such as ACI attributes. As a result, various directory vendors implement ACI policy objects in ways that do not translate well across vendor installations.

After the basic entry data has been imported from the cleaned up LDIF file to Oracle Internet Directory, you must explicitly reapply security policies in the Oracle Internet Directory environment. You can do this by using either Oracle Directory Services Manager, or command-line tools and LDIF files containing the desired ACP information.

There may be other proprietary metadata unrelated to access control. You should remove this as well. Understanding the various IETF RFCs can help you determine which directory metadata is proprietary to a given vendor and which complies with the LDAP standards, and is thus portable by way of an LDIF file.

Task 5: Remove Operational Attributes from the LDIF File

Four of the standard LDAP v3 operational attributes, namely, creatorsName, createTimestamp, modifiersName, and modifyTimestamp are automatically generated by Oracle Internet Directory whenever entries are created or imported. It is not possible to instantiate these values from existing directory data, for example by using LDIF file importation. Therefore you should remove these attributes from the file before attempting to import.

Task 6: Remove Incompatible userPassword Attribute Values from the LDIF File

The userPassword attribute hash algorithms supported by Oracle Internet Directory are listed in the orclcryptoscheme entry in Section 9.1.5, "Attributes of the DSE."

The userPassword attribute hash values used by some vendor products are not compatible with Oracle Internet Directory. As a result, you must remove all lines corresponding to the userPassword attribute and value from the LDIF data file unless they are represented in plain text or contain no value. After importation of the LDIF data, you must manually reenter or upload hashed userPassword information separately into the directory. Be sure that the passwords comply with the Oracle Internet Directory password policies and are in clear text.

Task 7: Run the bulkload check =TRUE" Mode and Determine Any Remaining Schema Violations or Duplication Errors

Before generating and loading an LDIF file, always perform a check on it by using the bulkload utility check mode. The bulkload output reports any inconsistencies in the data.

See Also:

The bulkload command-line tool reference in Oracle Fusion Middleware Reference for Oracle Identity Management for instructions on how to use the bulkload check mode

37.2.2 Migrating LDAP Data by Using syncProfileBootstrap Directly

The Directory Integration Assistant, syncProfileBootstrap, is a command-line tool for administering the synchronization profiles scheduled by the Oracle directory integration server. An administrator can use the syncProfileBootstrap operation to perform the initial migration of data between a connected directory and Oracle Internet Directory when configuring the Oracle directory integration server to perform ongoing synchronization. You also use it for a one-time data migration, without ongoing synchronization.

The syncProfileBootstrap operation can take data either directly from a third-party LDAP-compliant directory or from an LDIF file, tagged file, or CSV file. You must provide mapping rules, either as a synchronization profile or in a configuration file.

For syncProfileBootstrap syntax information, configuration file properties, information about input file types, and examples, see "Oracle Directory Integration PlatformTools" in Oracle Fusion Middleware Reference for Oracle Identity Management and Oracle Fusion Middleware Administrator's Guide for Oracle Directory Integration Platform.

If you must perform mapping when migrating the data from the third-party directory to Oracle Internet Directory, and if the data is small in size, you can use syncProfileBootstrap. As shown in Figure 37-2, "Using syncProfileBootstrap Directly", you can use the third-party directory itself as input to syncProfileBootstrap.

Figure 37-2 Using syncProfileBootstrap Directly

37.2.3 Migrating LDAP Data by Using an LDIF File and syncProfileBootstrap

Scenario 3 is a variation on Scenario 2. If you do not have direct access to the third-party directory, you can have the administrator export the data to an LDIF file. As shown in Figure 37-3, "Using an LDIF File and syncProfileBootstrap", syncProfileBootstrap can take its input from an LDIF file. You could also use Oracle directory integration server to migrate the data.

Figure 37-3 Using an LDIF File and syncProfileBootstrap

Whenever you use an LDIF file and bulkload to migrate data to Oracle Internet Directory, you must perform certain tasks. In this scenario, you are using a mapping file with syncProfileBootstrap or Oracle Directory Integration Platform, so do not have to perform all the tasks listed in "Migrating LDAP Data by Using an LDIF File and bulkload". You only have to perform the following tasks:

• Task 1: Export Data from the Non-Oracle Internet Directory Server into LDIF File Format

• Task 2: Analyze the LDIF User Data for Any Required Schema Additions Referenced in the LDIF Data

• Task 3: Extend the Schema in Oracle Internet Directory

37.2.4 Migrating LDAP Data by Using syncProfileBootstrap, bulkload, and LDIF Files

If you have a large amount of data and you must perform mapping on the data, you can use a combination of tools. As shown in Figure 37-4, "Using syncProfileBootstrap, bulkload, and LDIF Files", you can export the data from the third-party directory to an LDIF file, then use syncProfileBootstrap to perform the mapping into another LDIF file, which you then load with bulkload.

Figure 37-4 Using syncProfileBootstrap, bulkload, and LDIF Files

As in Section 37.2.3, "Migrating LDAP Data by Using an LDIF File and syncProfileBootstrap," you only have to perform these tasks:

• Task 1: Export Data from the Non-Oracle Internet Directory Server into LDIF File Format

• Task 2: Analyze the LDIF User Data for Any Required Schema Additions Referenced in the LDIF Data

• Task 3: Extend the Schema in Oracle Internet Directory

37.2.5 Migrating LDAP Data by Using the Oracle Directory Integration Platform Server

Under some circumstances, an administrator might choose not to use syncProfileBootstrap when configuring the Oracle directory integration server. After it is configured, the Oracle directory integration server itself can migrate data from a connected directory to Oracle Internet Directory. You can also use the Oracle directory integration server for a one-time data migration. For more information, see Oracle Fusion Middleware Administrator's Guide for Oracle Directory Integration Platform.

The Oracle directory integration server enables you to configure bidirectional, ongoing integration between Oracle Internet Directory and a Third-party directory, as shown in Figure 37-5, "Using the Oracle Directory Integration Server". For more information, see Oracle Fusion Middleware Administrator's Guide for Oracle Directory Integration Platform.

Figure 37-5 Using the Oracle Directory Integration Server

37.3 Migrating User Data from Application-Specific Repositories

Migrating user data from an application-specific repository requires:

• Collecting the user data from the application-specific repository and formatting it in a way that the directory can read it

• Making that data available to the directory administrator who must then:

  • Specify where to place it in the directory

  • Import it into the directory

37.3.1 The Intermediate Template File

To enable this migration to happen, the Oracle Directory Provisioning Integration Service requires the application-specific repository to export its data to an intermediate template file. Records in this template file are not in pure LDIF; they contain substitution variables that have to do with, for example, the location in the directory where the information is finally to reside. The application leaves these variables undefined, so that you, the directory administrator can define them later on.

To convert the user data from this intermediate template file into proper LDIF, you use the OID Migration Tool (ldifmigrator). After the data is converted to LDIF, you can load it into the directory.

To summarize: Migrating data from application-specific repositories involves these general steps:

1. Exporting the application-specific data as an intermediate template file

2. You, the directory administrator, using the OID Migration Tool (ldifmigrator) to read these partial LDIF entries and convert them to pure LDIF entries based on the deployment choices

3. You, the directory administrator, loading the data, now in pure LDIF, into Oracle Internet Directory

4. The application completing the migration process according to its own specifications

37.3.2 Reconciling Data in Application Repository with Data Already in the Directory

The data you are migrating from an application-specific repository may already reside in Oracle Internet Directory. If this is the case, then you can reconcile differences between the two directories by using the reconciliation feature of the OID Migration Tool (ldifmigrator).

See Also:

The ldifmigrator command-line tool reference in Oracle Fusion Middleware Reference for Oracle Identity Management for information about the reconciliation feature of the OID Migration Tool

37.3.3 Tasks For Migrating Data from Application-Specific Repositories

To migrate data from application-specific repositories, you create an intermediate template file, then run the OID Migration Tool.

37.3.3.1 Task 1: Create an Intermediate Template File

Applications generating data in national languages must store that data in AL32UTF8 in the intermediate template file as specified in the IETF RFC 2849, "The LDAP Data Interchange Format (LDIF) - Technical Specification" available at http://www.ietf.org.

When generating the intermediate template file, migrating applications must list all user records sequentially with a record separator as defined in RFC 2849. The OID Migration Tool (ldifmigrator) assigns all of these users to the default identity management realm, which corresponds to the enterprise itself.

Figure 37-6 shows the overall structure of the intermediate template file containing user entries.

Figure 37-6 Structure of the Intermediate User File

Description of ''Figure 37-6 Structure of the Intermediate User File''

The intermediate template file uses the following format to generate a valid user entry. All of the strings in bold text are supplied from the application-specific repository.

dn: cn=UserID, %s_UserContainerDN%
sn: Last_Name
orclGlobalID: GUID_for_User
%s_UserNicknameAttribute%: UserID
objectClass: inetOrgPerson
objectClass: orclUserV2

In this template, the strings %s_UserContainerDN% and %s_UserNicknameAttribute% are substitution variables for which the OID Migration Tool provides values. The OID Migration Tool determines these values according to deployment-specific considerations. Either the application passes the arguments to the OID Migration Tool, or the tool retrieves them from the directory.

37.3.3.1.1 Example: User Entries in an Intermediate Template File

The following intermediate template file includes user entries generated by the application-specific migration logic. In this example, all of the data listed in bold text is from the application-specific user repository.

dn: cn=jdoe, %s_UserContainerDN%
sn: Doe
%s_UserNicknameAttribute%: jdoe
objectClass: inetOrgPerson
objectClass: orclUserV2
title: Member of Technical Staff
homePhone: 415-584-5670
homePostalAddress: 234 Lez Drive$ Redwood City$ CA$ 94402

dn: cn=jsmith, %s_UserContainerDN%
sn: Smith
%s_UserNicknameAttribute%: jsmith
objectClass: inetOrgPerson
objectClass: orclUserV2
title: Member of Technical Staff
homePhone: 650-584-5670
homePostalAddress: 232 Gonzalez Drive$ San Francisco$ CA$ 94404

dn: cn=lrider, %s_UserContainerDN%
sn: Rider
%s_UserNicknameAttribute%: lrider
objectClass: inetOrgPerson
objectClass: orclUserV2
title: Senior Member of Technical Staff
homePhone: 650-584-5670

After all of the user data is converted to the intermediate file format, the OID Migration Tool further converts it into a proper LDIF file that can be loaded into Oracle Internet Directory.

You can find examples of intermediate template files in $ORACLE_HOME/ldap/schema/oid.

37.3.3.1.2 Attributes in User Entries

Each user entry has mandatory and optional attributes.

Table 37-2 lists and describes the mandatory attributes in a user entry.

Table 37-2 Mandatory Attributes in a User Entry

Attribute	Description
dn	Distinguished name of the user entry with appropriate substitution variables. The relative distinguished name of the entry MUST contain the cn attribute.
sn	Surname—that is, the last name—of the user
objectclass	Object classes the entry should minimally belong to: inetOrgPerson and orclUserV2

See Also:

• IETF Request for Comments 2798: "Definition of the inetOrgPerson LDAP Object Class," available at http://www.ietf.org, for a description of each attribute in the inetOrgPerson object class

• "Object Class Reference for Oracle Identity Management" in Oracle Fusion Middleware Reference for Oracle Identity Management for information about optional attributes of the orclUserV2 object class

37.3.3.2 Task 2: Run the OID Migration Tool

After you set up the intermediate template file, the OID Migration Tool enables you to bring all pertinent data from the application-specific repository into Oracle Internet Directory. After you have migrated the data, you can update whatever portion of it is relevant to the application by synchronizing that application with Oracle Internet Directory. You synchronize by using the Oracle Directory Synchronization Service.

See Also:

The ldifmigrator command-line tool reference in Oracle Fusion Middleware Reference for Oracle Identity Management for instructions about using the OID Migration Tool