| Sun ONE Meta-Directory 5.1.1 Product Brief |
Chapter 3
Meta-Directory Principles at WorkThis chapter uses an example to show how to configure Meta-Directory to integrate three different external data sources into one unified Meta View. In order to make this example compliant with the Windows NT or Solaris operating system, the sample uses data sources that work with both. These include an LDIF data file imported into a Directory Server, a CSV data file using the Universal Text Parser (UTP), and an Oracle database using the Database Connector. These data samples are small but the principles of working with them using Meta-Directory are the same as they would be for large bodies of data. This chapter includes the following sections:
LDIF Data FileAn LDIF file is a text version of LDAP data. The following LDIF data is what will be imported into Directory Server. Once it is imported, the LDAP data sub-tree is directly accessible by Meta-Directory and would be considered a direct connection.
CSV Data FileA CSV (comma-separated values) file is a text file format that can be output by a standard spreadsheet program. (Its name is derived from the fact that each record is stored in a single line with the individual items of data separated by commas.) The following CSV data will be imported into Meta-Directory via the UTP. (The UTP is considered an indirect connector as it is necessary to map the CSV file data to the Directory Server’s LDAP schema before it can be read by Meta-Directory.)
Code Example 3-2 CSV Data
REALNAME,LASTNAME,EMAIL,PHONE,BUILDING,JOBTITLE,ALTERNATIVENAME,NICKNAM E
Frankie Shaden,Shaden,frankshaden@siroe.com,408 555 1234,15,Accountant,fshaden,Frank
Gary Parker,Parker,garyparker@siroe.com,408 555 5678,7,Writer,gparker,Gary
Hal Kent,Kent,haroldkent@siroe.com,408 555 9012,18,Designer,hkent,Harold
Indy Connor,Connor,indranilconnor@siroe.com,408 555 3456,7,Accountant,iconnor,Indranil
Joe Hayes,Hayes,josephhayes@siroe.com,408 555 7890,15,Writer,jhayes,Joseph
Oracle DatabaseData stored in an Oracle database can be viewed in Meta-Directory with a Database Connector. The following data will be accessed from an Oracle 8 database via Structured Query Language (SQL). The Database Connector is considered a direct connector because it plugs directly into the Join Engine.
Table 3-1 Oracle Data
Employee ID
First Name
Last Name
Salary
Department ID
aquinn
Al
Quinn
75,000
10
bmoore
Betty
Moore
85,000
43
cwayne
Cal
Wayne
80,000
24
dgordon
Debby
Gordon
95,000
10
ecrane
Eric
Crane
85,000
43
fshaden
Frankie
Shaden
80,000
24
gparker
Gary
Parker
95,000
10
hkent
Hal
Kent
85,000
43
iconnor
Indy
Connor
80,000
24
jhayes
Joe
Hayes
95,000
10
Background InformationSiroe, Inc. is a small company that outsources services. It currently has ten employees on its payroll. Personal and employment information on their employees is divided among three data sources. Siroe’s Directory Server contains names and email addresses of five employees. Similar information for five consultants is in an Excel spreadsheet program. An Oracle database contains the salary information, employee numbers, and departments of all ten employees. Siroe needs to integrate all three of these sources and identify the employees by job function: four writers, three designers, and three accountants.
The example assumes that iPlanet Directory Server and the Sun ONE Meta-Directory software have been installed and a Join Engine has been created.
Configuration of Connectors and Join Process RulesSiroe, Inc. has hired a systems administrator, Tracy, to help with the integration of their data. Tracy decides to flow the data on Siroe’s Directory Server first. The next steps are integrating the CSV file exported from Excel and then syncing the Oracle tables into Siroe’s unified employee database.
Flowing LDIF Data to the Connector View
Siroe has two directory servers: one is an iPlanet Directory Server and the other is Netscape Directory Server 1.0. The Directory Server 1.0 is not completely compatible with Meta-Directory so Tracy exports its LDAP data to an LDIF file in order to import it into the newer iPlanet Directory Server. (The Netscape Directory Server will no longer be used; its place is taken by the iPlanet Directory Server.) Tracy creates an instance of the Universal Connector (UTC) from Sun ONE Console which also adds a Connector View for the data on the newer iPlanet Directory Server by:
- Naming the instance: firstgroup
- Naming the view ID: fg
- Naming the base dn: o=firstgroup
The LDIF file is already configured as uid=variable,o=firstgroup therefore, the base DN of our connector is input as o=firstgroup.
- Entering the LDAP URL of the Directory Server on which the data will be stored
- Entering an authenticating Distinguished Name and password
When Tracy refreshes the Directory Server and Meta-Directory Consoles, the o=firstgroup LDAP sub-tree is visible in the navigation tree of the Directory Server and the firstgroup connector and Connector View are visible in the navigation tree of Meta-Directory.
Once the Connector View is created, Tracy imports the LDIF file into the Connector View sub-tree via the iPlanet Directory Server by using the Import Database command from the Directory Server Console.
Flowing LDAP Data to the Meta View
The LDIF data is now written to a LDAP Directory Server that is directly connected to the Join Engine. Therefore, Tracy can begin configuring the join process to flow the LDAP data through the Join Engine and into the Meta View.by creating the rules to define how Siroe would like the data to flow.
Constructing an Attribute
destinationIndicator is an allowed attribute of the organizationalPerson object class so Siroe will use this attribute to keep track of an employee’s off-site locations. Tracy, therefore, needs to put a value to this attribute. One way to do this is to take a value that is already present and break it into sub-tokens, enabling the use of these sub-tokens for other attributes. As their personnel’s email address changes with each job, Tracy constructs an attribute with a value by:
With this rule, Tracy deconstructs the email address field, variable@company.com into three tokens: variable, company and com. From these fields, %company% is chosen as the value for constructeddestinationIndicator that will sync with the LDAP destinationIndicator attribute indicating the employee’s off-site location.
Mapping Attribute Flow Rules
After constructing the attribute, Tracy configures flow tables to match up the attributes from the LDIF/LDAP data to the attributes in Meta-Directory’s schema. Attribute flow rules can be configured for two directions: from the Connector View to the Meta View and from the Meta View back to the Connector View. The LDIF/LDAP data contains the following attributes: objectclass, cn, sn, mail, title, uid, and givenname; in addition, Tracy needs to flow the constructed attribute constructeddestinationIndicator. Tracy creates attribute flow rules by:
- Naming the rule: fg2mv
- Choosing the direction: To Meta View
- Entering selection criteria: (%objectclass%==person)
This specification means that any entry with an %objectclass%==person attribute/value pair will be selected and allowed to map its attributes.
- Adding attribute mappings
objectclass, cn, sn, mail, title, uid, givenname and constructeddestinationIndicator map respectively to objectclass, cn, sn, mail, title, uid, givenname and destinationIndicator
All attributes from the LDIF/LDAP data will flow to the same-titled attribute in the Meta View (although Tracy could flow them differently if Siroe wanted different mappings). The constructed attributes are visible when Constructed Attributes is chosen from the source objectclass drop-down list. In this case, Tracy maps the constructeddestinationIndicator constructed attribute to the schema attribute destinationIndicator.
When the attribute mappings are complete, Tracy needs to create a rule set entitled affg2mvset and add the fg2mv rule to it. Since attribute flow rule sets can not contain rules configured for data that flows in opposite directions, Tracy must also make a separate rule and rule set to map the attributes from the Meta View back to the Connector View. He names them fg2cv and affg2cvset, respectively. (The difference between the two rules is that, in the latter case, Tracy does not want the constructed attribute to be flowed back to the Connector View from the Meta View and therefore does not map it.)
Creating the Entry’s Distinguished Name
Tracy needs to configure the distinguished name (DN) for each entry. DN mapping rules create a place for each entry in the destination view. The first entry in the LDIF/LDAP data has as its source DN uid=aquinn,o=firstgroup. Tracy wants the Meta View DN of this entry to be uid=aquinn,ou=Writers,o=firstgroup. The DN mapping rule is created by:
- Naming the rule: fgwriter2mv
- Entering the selection criteria: ((%objectclass%==person) AND (%title%==Writer))
This specification means that any entry with %objectclass%==person and %title%==Writer attribute/value pairs will be selected and assigned the following DN.
- Constructing the DN: uid=%uid%,ou=Writers
Tracy wants the unique identifier (in this case, aquinn) to be the unique name of each partial DN relative to the base DN. This DN construction creates the organization unit Writers in which all entries with title=Writer and objectclass=person will be placed.
Tracy must now configure a rule for Accountants and Designers. The three completed rules, fgaccountant2mv, fgdesigner2mv and fgwriter2mv, are then added to one rule set that Tracy calls DNrulesfg2mv because they are all for flowing data to the Meta View.
Tracy then makes three more rules which flow the information back to the Connector View, using as selection criteria cn in ou=Writers, ou=Accountants, and ou=Designers and constructing a DN of uid=%uid%, o=firstgroup. Tracy puts them in a rule set entitled DNrulesfg2cv. The two rule sets DNrulesfg2mv and DNrulesfg2cv will flow the LDAP data into and out of the Meta View. Next, Tracy flows the CSV data into the Meta View.
Mapping CSV Data to LDAP Using the UTP
The CSV data must be flowed into the Connector View using the Universal Text Parser (UTP), so Tracy configures an instance of the Universal Connector (UTC) from Sun ONE Console by:
- Naming the instance: secondgroup
- Naming the view ID: sg
- Naming the base dn: o=secondgroup
- Entering the LDAP URL of the Directory Server on which the data will be stored
- Entering an authenticating Distinguished Name and password
- Entering the path name to the template.pl script included with Meta-Directory: NETSITE_ROOT/md50/utc-secondgroup/utp/template.pl
The template.pl file is part of the UTP. It is a Perl script that tells the UTC what to do. It must be moved to the configured UTC’s directory (entered in the path name above) from its default location at NETSITE_ROOT/bin/utc50/install/templates/universalparser/template.pl.
Tracy refreshes both Console so that the o=secondgroup sub-tree is visible in the navigation tree of the Directory Server and the secondgroup Connector View is visible in the navigation tree of Meta-Directory. Now, Tracy needs to convert the data in the CSV file into LDAP data using the Universal Text Parser’s csv.cfg file.
Note
When using the Universal Text Parser, attributes are mapped twice. First, CSV data is mapped to LDAP attributes within the csv.cfg file and second, the CSV/LDAP attributes are mapped to the Meta-Directory schema by configuring indirect connector rules.
Mapping CSV Data to LDAP Attributes
The csv.cfg text file is used to map comma-separated values to LDAP attributes. Tracy finds the file and the task.cfg file in NETSITE_ROOT/bin/utc50/install/templates/universalparser. Tracy first copies the csv.cfg file to the task.cfg file. (task.cfg is called by template.pl but it needs to be configured specifically for CSV conversion into LDIF using the csv.cfg file.) Tracy edits the new version of task.cfg by editing the LineFormat and ImportLineFormat sections to map the CSV data attributes REALNAME, LASTNAME, EMAIL, PHONE, BUILDING, JOBTITLE, ALTERNATIVENAME and NICKNAME to the LDAP attributes cn, sn, mail, telephoneNumber, l, title, uid and givenName, respectively. Tracy also changes the InputFile section of csv.cfg to %ScriptBase%csv.data. Tracy then places the csv.data file and the modified task.cfg file in the directory with template.pl located at NETSITE_ROOT/md50/utc-secondgroup/utp/.
Mapping LDAP Attributes to the Connector View
Once the CSV data has been mapped to LDAP-recognized attributes within the csv.cfg file, these LDAP attributes need to be mapped to the LDAP attributes in the Connector View using indirect connector rules. First, Tracy specifies the attributes defined in csv.cfg as attributes that are available to the connector. For this, each attribute name is entered on the connector’s External Available Attributes panel. Once defined, the attributes are accessible for mapping data to and from the Connector View attributes. Tracy can now create attribute flow rules for the indirect connector by:
- Naming the rule: secondgroupflow
- Choosing the mapping type: Mappings for Locally Owned Objects
This choice identifies the rule as applying to data flow from the external source to the Connector View.
- Match the external attributes cn, sn, mail, telephoneNumber, l, title, uid and givenName To Connector View attributes cn, sn, mail, telephoneNumber, l, title, uid and givenName
- Match the external attributes cn, sn, mail, telephoneNumber, l, title, uid and givenName From Connector View attributes cn, sn, mail, telephoneNumber, l, title, uid and givenName
- Choosing the mapping type: Mappings for Connector View Owned Objects
This choice identifies the rule as applying to data flow from the Connector View to the external data source.
- Match the external attributes cn, sn, mail, telephoneNumber, l, title, uid and givenName To Connector View attributes cn, sn, mail, telephoneNumber, l, title, uid and givenName
- Match the external attributes cn, sn, mail, telephoneNumber, l, title, uid and givenName From Connector View attributes cn, sn, mail, telephoneNumber, l, title, uid and givenName
Tracy can now apply the indirect connector rule by selecting the utc-secondgroup indirect connector and its corresponding secondgroupflow rule.
Flowing LDAP-configured CSV Data to the Meta View
From this point, Tracy follows the same procedure established for the LDIF/LDAP data. First, DN mapping rules are configured, selecting the entries and linking them to the DN as laid out in the table below.
Table 3-2 Entries and their corresponding DN constructed rules
Select entries that contain
Link entries to DN constructed as
objectclass=person AND l=15
uid=%uid%, ou=Accountants
objectclass=person AND l=7
uid=%uid%, ou=Writers
objectclass=person AND l=18
uid=%uid%, ou=Designers
Tracy names the rules sgwriters2mv, sgdesigners2mv and sgaccountants2mv and puts them in a rule set called DNrulessg2mvset. Whereas the LDIF file had employees identified by job title, the CSV data has them identified by building location with each building occupied by accountants, writers, or designers exclusively. These DN mapping rules place people in the correct sub-trees based on their building location.
Mapping LDAP-configured CSV Data to the Meta View
Attribute flow rules are needed to map the CSV/LDAP attributes in the Connector View to the LDAP attributes in the Join Engine’s Meta View. Tracy creates attribute flow rules by:
- Naming the rule: sg2mv
- Choosing the direction: To Meta View
- Entering the selection criteria: (%objectclass%==person)
This specification means that any entry with an %objectclass%==person attribute/value pair will be selected and assigned the following mappings.
- Adding attribute mappings
uid, objectclass, telephoneNumber, sn, mail, title, givenName and l map respectively to uid, objectclass, telephonenumber, sn, mail, title, givenname and l
Tracy also creates a separate rule to map the attributes from the Meta View back to the Connector View; this is called sg2cv. Rule sets are created (afsg2mvset and afsg2cvset)to hold the two configured rules. With that done, all the rules needed to manage the flow of data from the CSV file are complete.
Flowing Oracle Data to the Meta View
Tracy must now flow the Oracle data to the Meta View. This data includes salary information and employee identification numbers. To make the flow possible, Tracy needs to configure the Oracle database as a Data Server that can be recognized by the Meta-Directory’s Join Engine.
Note
In order to bring Oracle data into Meta-Directory, the OCI libraries, available from Oracle, must be installed on the same machine in which the Join Engine is installed.
Configuring the Oracle Database as a Recognizable Data Server
In order to configure the Oracle database as a recognizable Data Server, Tracy accesses the Data Server panel from Meta-Directory Console and completes the general information by:
- Entering a name for the data server
- Entering an alias for the data server
This is the Oracle connect string.
- Entering a user name that does not already exist within the Oracle database
This name will be used by Meta-Directory’s Database Connector as the privileged user.
- Entering the path name to where the OCI libraries are installed
- Using the default NLS Language Environment Variable
Next, Tracy selects the tables and columns that will participate in the data flow by:
- Choosing the Oracle columns for syncing
The Uninstrumented Columns are the full list of Oracle columns that Tracy will choose from. The Instrumented Columns are the ones Tracy has selected: EMPLOYEE_ID, FIRST_NAME, LAST_NAME, SALARY and DEPARTMENT_ID.
- Choosing (or nominating) a primary key for Meta-Directory’s use
The primary key is unique and not null among all the entries. Tracy chooses EMPLOYEE_ID.
The name and location for the change log tables and triggers are also chosen. (Tuning and DCNS schedule use the default option.) When these selections are saved, a script is generated which, when applied to the database, enables change detection on the selected tables and columns. Tracy then runs the script on the command line via SQLplus (part of Oracle client software) and creates both the Connector View and participating view from Meta-Directory Console.
Note
The Oracle table data can never be seen through the Meta-Directory Console. Only when the data is synced in the Meta View can it be viewed.
Constructing an Attribute for the Oracle Database
When flowing data from an Oracle database, the objectclass attribute must always be constructed and flowed with the data as Oracle does not use object classes. Tracy creates an objectclass attribute by:
Tracy also constructs a commonname attribute by:
- Naming the constructed attribute: dbcommonname
- Entering selection criteria: (%cv.FIRST_NAME%=@)
This specification means that any entry with a Connector View value of FIRST_NAME will be selected.
- Entering attribute construction: %cv.FIRST_NAME% %cv.LAST_NAME%
The dbcommonname will be made from the Connector View’s FIRST_NAME and LAST_NAME attributes.
To complete the constructed attributes, Tracy puts each rule in a separate constructed attribute rule set entitled dbobjectclass and dbcommonname, respectively.
Creating Attribute Flow Rules for the Oracle Database
Tracy can now create the attribute flow rules to map the Oracle columns to their counterpart LDAP attributes. This is done by:
- Naming the rule: oracle2mv
- Choosing the direction: To Meta View
- Entering the selection criteria: (%cv.EMPLOYEE_ID%=@)
This specification means that any entry with a Connector View value of EMPLOYEE_ID will be selected for these mappings.
- Adding attribute mappings
EMPLOYEE_ID, FIRST_NAME, LAST_NAME, SALARY, DEPARTMENT_ID, dbobjectclass and dbcommonname map respectively to uid, givenName, sn, description, departmentnumber, objectclass and cn.
This attribute flow rule is then put in an attribute flow rule set entitled oracle2mvset. An attribute flow rule going back to the Oracle column is also created (using the selection criteria %uid%=@) and put in a rule set entitled oracle2cvset.
Creating DN Mapping Rules for the Oracle Database
Tracy can now configure rules that will create DNs for the Oracle data. Three rules are needed to flow the data into the Meta View’s three organizational units. A DN rule for Writers is created by:
- Naming the rule: oraclewriter2mv
- Entering the selection criteria: ((%cv.EMPLOYEE_ID%=@) AND (%cv.DEPARTMENT_ID%==10))
This specification means that any entry with a Connector View value of EMPLOYEE_ID and a department ID of 10 will be selected for this DN.
- Constructing an DN: %cv.EMPLOYEE_ID%, ou=Writers
This DN rule creates the organizational unit Writers in which all entries with an employee ID and a department ID of 10 will be placed. Tracy must now configure a different rule for accountants and designers. The three completed rules that apply to data flowing into the Meta View are put into one rule set entitled DNrulesoracle2mvset.
Tracy then makes three more rules which flow the information back to the Connector View, using as selection criteria %mv.EMPLOYEE_ID%=@ and constructing a DN of %mv.EMPLOYEE_ID%. Tracy puts them in a rule set entitled DNrulesoracle2cv. The two rule sets DNrulesoracle2mv and DNrulesoracle2cv will flow the LDAP data into and out of the Meta View.
Integration of the Data With Join RulesWith the bulk of the rules configured for the three data sources, Tracy is now able to flow the data into the Meta View. Join rules need to be configured to synchronize the data correctly as each data source has different information flowing to the same Meta View entry. Tracy needs to configure join rules that look at all source entries and link only those that have passed certain criteria. Tracy creates a join rule for the Oracle database by:
Tracy puts the Oracle join rule into a join rule set called oraclejrcv2mvset. For the LDIF data source the join rule is configured by:
Tracy creates a join rule set called ldifjrcv2mvset and the LDIF join rule is put into it. Tracy then realizes that the join rule created for LDIF data will also work for the CSV data source because they both have been converted into LDAP. Now Tracy is ready to apply the join process rules.
Applying Rules to the Participating View
Tracy applies the join process rules to the correct participating views. (The participating view allows the Connector View to ‘participate’ in the join process.) From Meta-Directory Console, a participating view for the firstgroup, secondgroup and oracle Connector Views is added. It is here that the join process rules are applied as follows:
Table 3-3 (Description of the join process rules and their corresponding rule set name
Participating View
Join Process Rule
Rule Set Name
firstgroup
Attribute Flow To Meta View
affg2mvset
Attribute Flow To Connector View
affg2cvset
DN Mapping To Meta View
DNrulesfg2mv
DN Mapping To Connector View
DNrulesfg2cv
Filters
none configured
Join Rules To Meta View
ldifjrcv2mvset
secondgroup
Attribute Flow To Meta View
afsg2mvset
Attribute Flow To Connector View
afsg2cvset
DN Mapping
DNrulessg2mvset
Filters
none configured
Join Rules To Meta View
ldifjrcv2mvset
oracle
Attribute Flow To Meta View
oracle2mvset
Attribute Flow To Connector View
oracle2cvset
DN Mapping
DNrulesoracle2mvset
Filters
none configured
Join Rules To Meta View
oraclejrcv2mvset
Once the Join Engine is started, the connectors are enabled and the views are refreshed, Siroe’s LDIF data, CSV data and Oracle data will flow to the Meta View forming one complete source of employee information.
At this point, Tracy checks the Meta-Directory error logs to see if any of the data has failed to flow or link. He does this by issuing the following grep command in the log directory:
grep -i fail *.log
Had any errors been found, Tracy could use the Fix-It Tool to manually link the orphaned entries. Fortunately for him no errors were found so Tracy steps outside for lunch.
