Chapter 2   Planning the Meta-Directory System


The Meta-Directory solution involves planning, installing, and configuring the iPlanet Directory Server and iPlanet Meta-Directory systems so they can produce the meta view you have designed. In addition to these software components, you might also need to consider the installation and configuration of any additional services that your external directory data sources might need to operate.

Successful integration of Meta-Directory in the enterprise occurs in three phases: Design, Deployment, and Maintenance:

• In the design phase, you analyze your existing system and establish the architecture and requirements of the Meta-Directory system you will deploy. In this phase, you must take a careful look at your existing directory data sources and plan how to incorporate these into a single meta view.

• In the deployment phase, you devise a systematic approach to the configuration of components and the implementation of servers and services in the enterprise. This phase deals with how to set up a Meta-Directory system, the different servers and services you need to install, and the topology of the hardware that you will be employing.

• In the maintenance phase, you provide a plan for maintaining the Meta-directory system once it has been deployed. This third and last phase deals with what needs to be done to keep your Meta-Directory system properly tuned and maintained.

This chapter contains the following sections:

• Designing Your Meta-Directory System

• Deploying Your Meta-Directory System

• Maintaining Your Meta-Directory System

Designing Your Meta-Directory System

---

Ideally, all of your directory data would be contained in one integrated iPlanet Directory Server system, and the system would be spread evenly through the entire enterprise. We know, however, that such an ideal system is rarely in place. More often than not, many different types of directory systems are implemented across a large corporate network. Although the ideal of a single LDAP directory system is often out of reach, you can use Meta-Directory to join the heterogeneous types of directory systems running across your organization, and thus create a single LDAP directory of the information.

This section addresses the different tasks that you need to perform to design a robust Meta-Directory system. Included in this section are the following subsections:

• Beginning the System Design

• Performing a Site Survey

• Determining the Data Design for Your Views

• Planning the LDAP Schema Used in Your Views

• Planning System Security

• Allocating the Necessary Resources

Beginning the System Design

When reviewing the systems that are implemented across your organization, it's likely that you will see iPlanet Directory Server already being used as part of the directory system mix. When you add Meta-Directory to the directory architecture, you will be adding another instance of iPlanet Directory Server to the mixture of directory systems already in place. When you begin to address Meta-Directory system design, it's a good idea to separate the Directory Server instances that host organizational data and the Directory Server instances that host any of the Meta-Directory views.

Before you can begin to design your Meta-Directory system, you must clearly define and design the Directory Server system(s) you will use to host your organization's user directories. To help with this task, the iPlanet Directory Server Deployment Guide details the planning steps that you should follow to successfully deploy Directory Server in your organization.

Although the entire iPlanet Directory Server Deployment Guide is devoted to designing and implementing your user directory system, most of the principles and concepts described in that guide are also valid for designing and deploying a Meta-Directory system (Meta-Directory is built on iPlanet Directory Server). This guide assumes that you understand the concepts presented in the iPlanet Directory Server Deployment Guide and that you are familiar with configuring and running Directory Server, as explained in the iPlanet Directory Server Administrator's Guide.

The design of Meta-Directory begins with a site survey to gather information about the existing data sources and how you want to incorporate them into a single LDAP directory (the meta view). The next steps are to determine the data flow (selecting the attributes you want to use from each database) and plan the physical design of the system.

When designing the system, keep in mind the impact the new system will have on existing directory structures.

Installing Directory Server to Support Meta-Directory

Meta-Directory stores its configuration data in a Directory Server directory information tree (DIT). Like other iPlanet servers and components, Meta-Directory stores its configuration under the suffix o=netscaperoot.

When you install both iPlanet Directory Server and iPlanet Meta-Directory, the following components store their configuration data under o=netscaperoot:

• iPlanet Console

• iPlanet Directory Server

• iPlanet Meta-Directory

• iPlanet Administration Server

To optimize system performance, it is best to use a dedicated Directory Server instance for the Meta-Directory configuration database and a separate dedicated Directory Server instance for the meta view (which contains the user data that you will be synchronizing with Meta-Directory). Often, the Directory Server instance that hosts the Meta-Directory configuration is called the "config server."

---

Note	If you are synchronizing data contained in an Oracle database, Meta-Directory will use o=netscaperoot to store link information generated for the Oracle entries. In this case, it is possible that o=netscaperoot will house a large number of entries.

---

In addition to using separate Directory Server instances, it is also recommended that you use a dedicated disk partition to store the directory information tree that hosts the meta view. In production scenarios where you plan to synchronize large amounts of user data in the meta view, using dedicated systems for the individual Directory Server instances can greatly increase system performance. For more information on setting up your Meta-Directory components and services, refer to "System Topology".

Directory Server Design and Deployment

Detailed information describing the design of LDAP directory services abounds. In addition to the iPlanet Directory Server documentation, refer to the following references for more information on deploying Directory Server in your organization:

• Understanding and Deploying LDAP Directory Services
  by Mark Smith, Tim Howes, and Gordon Good (Macmillan Technical Publishing, January 1999)

• Implementing LDAP, by Mark Wilcox (Wrox Press, March, 1999)

Performing a Site Survey

The first stage of planning your Meta-Directory system is to perform a site survey.

Determining Data Sources and Authoritative Attributes

In the site survey, you will, among other things, determine the following:

• The data sources you want to join in the meta view.

• Which data sources contain the "authoritative" user attributes you want to flow to the connector and meta views.

The first stage of planning your Meta-Directory system design is to determine the different data sources that you wish to link with Meta-Directory. While this might at first seem obvious, it is a good idea to poll the different organizations in your enterprise to see if they have implemented any directories or databases that contain information that you want to join with the meta view.

Once you have determined each of the data sources you want to join in the meta view, you will need to compile a list of the attributes or fields contained in each directory or database. From this list, you can begin the process of defining which attributes or fields you will use from each of the directories or databases in your organization.

Authoritative attributes are the attributes that you will use to populate your meta view. For example, suppose you have two external data sources that flow attributes to the meta view: a Human Resources database and an Information Systems (IS) directory DIT. Consider that both data sources contain information on the company's employees. You will need to decide which attributes (fields) will be the authoritative source for each of the attributes contained in the meta view. It is likely that both data sources will contain the first and last names of the employees. However, it's likely that the Human Resources database will contain the most up-to-date and correct names. On the other hand, it's likely that the IS directory will contain the correct network IDs for each of the users.

Determining the authoritative attributes in your Meta-Directory system is a bit like going through the normalization process when you design a relational database. You must make sure to keep track of the authoritative attributes as you flow them from different data sources and join them in the meta view. Be especially attentive when you design how modified data flows back out to the various external data repositories. When designing your system, anticipate events that could affect system performance and stability. Minimize problems that could occur if a database goes down and keep enough flexibility in your plan to allow for the changes that could occur when a new data source comes on line. When planning, be sure that modifications made to one data source will not adversely affect the data contained in another.

In the example shown in Table 2-1, several data sources provide the authoritative attribute data.

Table 2-1    Data Sources From the Site Survey  

Data Source	Purpose	Provide Changes?	Accept Changes?	Connector Type/Protocol
Human Resources database (Oracle)	Maintains all the people entries for regular employees and contractors, including location, employment status, and so on.	Yes	Yes	Direct database connector
PBX	Stores employee and contractor telephone numbers. Depends on HR to provide information about changes in employment status.	Yes	Yes	Customized text connector (UTP)
IS database 1 -- NT Domain	NT Domain stores user information for Windows users	Yes	No	NT Domain connector
IS database 2 -- NIS (Network Information Server)	NIS contains UNIX user account information.	Yes	No	Customized text connector (UTP)
Netscape Phone book	Contains location, phone number, and email address information. This is a Directory Server application that enables the searching, browsing, and editing, of user information.	No	Yes	Direct LDAP connector

Determining Data Flow

In addition to determining the data sources and attributes you will use to populate the connector and meta views, you will also need to define the flow of the attributes into and out of the different Meta-Directory views. To help you determine this, you should collect all of the following information during the Meta-Directory planning phase:

• The data sources you want to join into your meta view.

• The user attributes (or database fields) contained in each of the external data repositories.

• The user attributes that need to flow to and from the unified directory (the meta view). These are often the authoritative attributes from each data source.

• The frequency that attributes need to flow to and from the meta view.

  The following questions should be answered during your site survey:

  • How often are updates made to each external data repository?

  • How often will modifications be made to the attributes contained in the meta view?

  Determining the update frequency will help you schedule the frequency of the synchronization cycles for your connector and meta views.

• The mapping of user attributes from the external data source to the connector view, and vice-versa. Mapping specifies which attributes from the external data sources flow to the specific attributes in the connector views, and vice-versa.

• The direction of attribute flow. Attributes may flow from the external data source to the connector view, from the meta view to the connector view, or both.

• The data ownership. Data ownership defines who owns a data object: the meta view or the external data source that populates a connector view. Data objects can be deleted only by the "owner" of the object. While you might be able to temporarily delete an object from a Meta-Directory view own the object, the object will reappear when the data is refreshed.

Data Flow Worksheet
Once you have collected the data needed to determine the data flow, you should create a data flow worksheet. The worksheet will list all the data sources you plan to include in your Meta-Directory system, the attributes in each data source, the attributes that you will flow from each data source and their name mappings, any constructed attributes that you plan to incorporate in your join rules, and so forth.

Determining the Data Design for Your Views

You should determine the design of the directory information trees (DITs) that host your Meta-Directory views during the planning phase. Whether you use one or more Directory Server instances, best performance will be achieved if you dedicate a directory suffix (namespace) for each view that you create. This means that the meta view will have its own directory suffix, as well as the individual connector views that you instantiate in your Meta-Directory system.

If you use individual Directory Server instances for your meta and connector views, this recommendation is moot. However, if you are instantiating different views on a single instance of Directory Server, then you will want to pay close attention to how you set up the DIT.

System performance is not influenced much by whether you create a relatively flat tree structure or a more vertical tree. However, a flat tree structure is easier to manipulate as you administer the different branches in your directory tree. In addition, should the need arise, troubleshooting the system will be a bit easier if you have a flat tree structure rather than a vertical tree structure.

For example, consider the following flat tree structures for hosting a meta view and three connector views:

o=metaview
o=oracleCV1
o=ldapCV2
o=adCV3

Compare this flat tree structure to a tree structure that has more levels, as the one shown below:

ou=metaview, ou=metadir, o=siroe
ou=oracleCV1, ou=connectorviews, ou=metadir, o=siroe
ou=ldapCV2, ou=connectorviews, ou=metadir, o=siroe
ou=adCV3, ou=connectorviews, ou=metadir, o=siroe

When you determining the structure of the directory trees that host your different Meta-Directory views, need to determine the base dn for your meta view as well as the base dn for each of the connector views in your system.

Setting Up the Meta-Directory DIT

You set up the namespaces for your Meta-Directory views through the Directory Server console that hosts the views.

In Directory Server 5.0, do the following:

1. From the Directory Server console, select the Configuration tab.

2. In the navigation tree in the left pane, right-click the Data icon and choose Create New Root Suffix.

3. Enter the dn for the root entries for the directory tree. In the dn specified, make sure there are no leading spaces or spaces between commas and attribute types.

In Directory Server 4.1x, do the following:

1. From the Directory Server console, select the Configuration tab. The Configuration window appears.

2. Double-click the Database icon and select the Settings tab. The Settings window appears and displays the suffixes.

3. From the Settings tab, select Add.

4. Enter the dn of the root entries for the directory tree. In the dn specified, make sure there are no leading spaces or spaces between commas and attribute types.

Planning the LDAP Schema Used in Your Views

In the system you design, Directory Server and Meta-Directory will likely exist to support one or more directory-enabled applications. These applications have requirements concerning the data contained in the directory, its format, and how the data is interpreted.

It is always best to use an existing LDAP schema to represent the data you are hosting in your Meta-Directory views. Using an existing schema will allow your data to be more portable to other applications if the need arises. If you do need to customize an existing schema, be sure to consider carefully any custom attributes you will add. Once you move forward with a custom schema, it is difficult to undo your modifications and return to a standard set of LDAP attributes.

Planning System Security

Many Meta-Directory design decisions have security implications. Just as with all applications that deal with a company's proprietary data, you want to ensure that the Meta-Directory system you deploy provides a secure environment for the data it manipulates. System security for Meta-Directory includes (but is not limited to) physical access to the systems used by Meta-Directory, network access to those systems, binding under commonly used accounts (for example, cn=directory manager), non-Meta-Directory client access to the directory server instances, and so on.

You can use the user authentication and access control mechanisms that Directory Server provides to protect the data that you host in the different Meta-Directory views. Using Directory Server services, you should be able to meet the security and privacy requirements of your system administrators, as well as the users of the applications that access the Meta-Directory views.

Between instances of Directory Server, you can use LDAPS for secure communication. However, it's important to note that Meta-Directory components do not use LDAPS to communicate between the components and views.

Allocating the Necessary Resources

Allocating resources includes obtaining the hardware and software necessary to support Meta-Directory and the other services you will use with your Meta-Directory system (this might include messaging systems, relational databases, network monitoring tools, and any special programs or utilities). Choosing hardware and software that satisfies design requirements for scaling and performance is crucial to successful deployment.

In addition to hardware and software, consider the human resources that you might need to get your system up and running, and the resources that you will need to maintain the system that you design.

Deploying Your System and Creating a Maintenance Plan

At this point, the design phase is over and it is time that you begin to deploy the system you have planned. The remaining sections in this chapter cover deployment and what you need to plan for once your system is up and running.

As part of the Planning phase, you should create a maintenance plan. While some of the items to consider are discussed on page 59, you should make sure to properly plan for the growth and expansion of your Meta-Directory system.

Deploying Your Meta-Directory System

---

Deploying your Meta-Directory system consists of implementing the system you mapped out in the design phase, as described in the preceding section, "Designing Your Meta-Directory System."

Effective Meta-Directory deployment centers around the use of system resources (your system hardware) to set up the instances of iPlanet Directory Server needed to host the different Meta-Directory components. When sizing system needs, you must also consider the services provided by any external data repositories and the resources they use.

A careful, systematic approach to your Meta-Directory deployment is strongly advised. By deploying your system incrementally, you can make sure each discrete portion of the system works before moving on to the next part of the deployment phase.

The deployment phase of Meta-Directory consists of properly sizing your host systems and effectively locating them on the network. Once your systems are up and running, tuning issues, system maintenance, and scaling the system for growth become the system administrator's biggest concern.

Data Entry Size and RAM Considerations

The size of the data objects that you will be flowing through your Meta-Directory system plays an important part in the overall design of your system setup. For example, you might be flowing entries that are relatively small (for example, 1024 Kb per entry) or you may be flowing larger entries that contain binaries, such as photos or digital signatures. Entry size plays an important role in the resources that the join engine uses to synchronize entries between views. Throughput of the system depends on having the available system resources to meet the demands of your setup.

In general, you can calculate the size of an entry by exporting it to LDIF and counting the number of characters it takes to represent the entry in text form. Add two characters for each line in the entry to account for the carriage return and line feed.

If your data contains binary data, you can use a different method to calculate the size of entries. Binary data is base64 encoded, meaning that binary data is stored in the Directory Server database as text characters. To calculate the size of an entry, export the entry to LDIF, then check the size of the LDIF file. On a UNIX system, you can use the following command to report the size of an LDIF file:

ls -l LDIF_file

On Windows NT systems, use the DIR command to view file size.

If you are estimating the size of entries for sizing information, you will likely want to average the size of several different entries to get a better idea of your database needs.

RAM Sizing

Not surprisingly, join engine performance is directly related to system resources: the more system memory (RAM), the faster the join engine will be able to synchronize entries between the Meta-Directory views. While you can supplement system RAM with virtual memory (swap space), you must keep in mind that overloading virtual memory will impact the performance of other system processes, such as other applications running on the system.

This section offers some simple examples to help you judge the amount of system RAM you will need in the units you deploy in your system. The database entries used in the examples below are relatively small, around 1024 Kb per entry. While this is a normal size for most data contained in a DIT of object class inetorgPerson, the size of the entries synchronized in the system you are designing might be vastly different, especially if you are flowing binary data.

When sizing the RAM requirements for your system units, consider the following data points observed from system testing:

• When synchronizing 300,000 entries, the join engine used roughly 400 Mb of memory.

  Note that in this scenario, Directory Server was actually hosting 600,000 entries: 300,000 in the meta view and 300,000 in the connector view. Here, both the meta view and the connector view were hosted by a single Directory Server instance located on a single machine, however, the join engine was hosted on a different machine.

• When synchronizing 500,000 entries, the join engine used roughly 665 Mb of memory. In this scenario, there were 1 million data entries hosted by Directory Server, with each entry being 1024 Kb. Both the meta view and the connector view were hosted by a single Directory Server instance located on a single machine, however, the join engine was hosted on a different machine.

When you start the join engine, the memory consumption will increase rapidly until it reaches the load where it begins to level off.

The minimum hardware requirement for Meta-Directory is 256 Mb of RAM (or 512 Mb if you are hosting Directory Server on the same machine). However, keep in mind that a larger production system will require more RAM, and perhaps more system units, than the established minimum requirements. Also, it is recommended that you hose the Meta-Directory components (the join engine and any components you have installed) on a different machine that the ones that host your Directory Server instances.

Table 2-2 gives additional examples on the memory used to synchronize entries between a connector view and a meta view. In this table, the average entry size is 1024 Kb, or 1 mega-byte of data per entry, per view.

Table 2-2    Estimating System RAM Size  

System RAM	Number of Entries Synchronized	Notes
256 Mb	1000 -> 10,000	iPlanet Console, join engine, and Directory Server hosted on the same machine.
256 Mb	10,000 -> 100,000	iPlanet Console, join engine, and Directory Server hosted on the same machine. Directory Server heavily tuned for 100,000 entries (a total of 200,000 entries). Synchronization might be slow and may time out. You must adjust the time-outs of the Directory Server that hosts the join engine.
256 Mb	100,000 to 300,000	Use this to test-pilot a system only! Directory Server would have to be heavily tuned in favor of 300,000 Kb of data. Synchronization times are long and time-outs are likely.
512 Mb	> 100,000	iPlanet Console, join engine, and Directory Server hosted on the same machine. Comfortable performance, Directory Server settings need to be tuned, but time-outs are unlikely.
512 Mb	100,000	iPlanet Console, join engine, and Directory Server hosted on the same machine. To sync 100,000 entries, Directory Server must be heavily tuned for 100 Kb of data. This would be much better than a 256 Kb system, but the join engine would still need longer than the default time-outs.
1 Gb	500,000 (1 million hosted data entries)	iPlanet Console and join engine hosted on the same machine; Directory Server hosted on a separate system. The directory server needs to be heavily tuned in favor of this high volume of data entries.

There is a delicate balance that needs to be achieved when you have large amounts of data. For details on the adjustments that you can make to customize your Meta-Directory setup, see Chapter 4 "Meta-Directory Performance Tuning."

Directory Sizing for Windows NT Systems

Calculating the number of Windows NT Domain entries that will be synchronized is useful for determining the RAM and disk space required for the directory service (for optimal performance, the Directory Server needs to hold the entire directory data in RAM).

In this case, entry size is not the only consideration. It is equally important to determine how often the data will be changing in each of the supported Windows NT Domains, how many systems the changed data will have to flow to, and the complexity of the data transformation (for example, attribute mapping, DN mapping, and the filter and rule sets that need to be established).

System Topology

The Meta-Directory system topology is the physical layout and locations on the network of the machines that you use to host all the servers and services in your Meta-Directory system. When designing the system topology, you are concerned with the physical implementation of the Meta-Directory components and where you place them on the network. In designing the system, you want to strike a balance between resource use and growth potential.

When you perform a site survey during the Meta-Directory planning phase, you will determine (among other things) the network location of the applications that will provide user information to your connector views and, ultimately, the meta view. Into this network layout, you will add the Meta-Directory components and services (the join engine, the meta view, connectors, and the associated connector views; plus the servers and services that support these components).

When deploying Meta-Directory, you must plan the topology of the system by looking at the types and amounts of data you plan to flow through the Meta-Directory connectors and join engine. Best performance is achieved when you design the topology to ensure that the set of hardware hosting the Meta-Directory components is properly load balanced. Network latency can also have an impact on synchronization times. It is possible for a smaller number of system units to outperform a larger set of system units if the smaller system is sized correctly.

Operating System Configuration

The configurations of the operating systems involved in your deployment is an important factor to consider when designing the Meta-Directory topology. When designing the system, you must take into account the physical memory (RAM) of the systems used, operating system settings (such as the amount of swap space used), and hard disk partitioning and configuration.

The slapd Processes
In Meta-Directory, individual processes are used to manage the main Meta-Directory components:

• nsmds runs the join engine.

• nsperlconn runs the Universal connector (UTC).

• Perl scripts are used by the Active Directory and NT Domain connectors.

Depending on your needs, you can host some of these processes on separate machines.

Directory Server Instances

When you are synchronizing large databases, it is best to create separate iPlanet Directory Server instances for the connector views, as well as using a different Directory Server instance for the meta view. Because of this, it is likely that your Meta-Directory system will contain a number of different instances of iPlanet Directory Server.

Hosting connector views on separate Directory Server instances allows you to balance the load of processing across different servers and machines. When designing the layout of the Directory Sever instances, distinguish between connector views that are populated by indirect connectors and the connector views that are directly updated by the join engine through the LDAP direct connector. Making this distinction is important when you tune the different Directory Server instances incorporated in your Meta-Directory system—Directory Server instances that host indirect connector views should be tuned for write performance while those that host "external" data should be tuned for read performance.

In determining the number and location of the Directory Server instances, keep in mind the Meta-Directory configuration is normally stored in the same Directory Server instance and DIT as the configuration for all other iPlanet and Netscape servers used in your Meta-Directory setup.

When creating different Directory Server instances to be used in a single Meta-Directory system, be sure to give the different instances different names as you create them.

Data Servers
Within the Meta-Directory console, each Directory Server instance is configured through the Data Server tab. To view the currently configured Directory Server instances:

1. Open the Meta-Directory console and highlight Meta-Directory in the navigation tree. The right pane displays six tabs.

2. Select the Data Servers tab to view the currently configured Directory Server instances.

Refer to iPlanet Meta-Directory Configuration and Administration Guide for instructions on how to add a new Directory Server instance to the Meta-Directory configuration.

Planning the Layout of Your System

There are three basic scenarios for hosting Meta-Directory views:

• A Meta-Directory view (either a meta view or a connector view) can be hosted on the same Directory Server instance that hosts the configuration directory (all on one machine).

• A Meta-Directory view can be hosted by a Directory Server instance that is different from the one hosting the configuration, with both Directory Server instances being located on the same machine.

• A Meta-Directory view can be hosted by a Directory Server instance that is different from the one hosting the configuration, but located on a separate machine.

In a large deployment, it is likely you will employ a combination of these setups.

Locating Meta-directory views on separate machines can increase the performance of the services, but you must balance any gains with the loss incurred through the added network latency. Through system analysis and comparisons with projected loads, you must decide on the setup that is needed for your particular implementation. However, using separate systems for each Directory Server instance is usually the best choice. If your system resources are limited, consider hosting at least the meta view on a dedicated system.

Whether more than one Meta-Directory view should be hosted on a single machine is ultimately dependent on the machine's processing capacity, the memory allocated to each of the system units, availability of disk space, the load on disk controllers and file systems, and the volume of data flowing between connector views and the meta view. System profiling and piloting should give you a good idea of the performance of your particular Meta-Directory set up.

Requirements for Hosting the Meta View
The meta view presents the final "image" of the collected data to end users. Most likely, you will use client applications to retrieve the information stored in the meta view. Because of this, the meta view normally has a higher performance requirement than does the typical connector view. While search indexes must be built for the meta view to speed up the different types of search operations performed by the applications accessing the meta view, the same level of searches are not generally used in connector views.

See "Determining Which Indexes to Create" for a list of the different indexes you should create for the different views in your deployment.

Requirements for Hosting Connector Views
The main task of a connector view is to host an LDAP copy of the data contained in an external data source. The primary operations performed on the connector view are more heavily weighted on add, delete, and modify; searching through connector views is not a priority as it is with the meta view. With a lighter demand for searches, there is a lesser need for indexes in the connector view. This directly affects your design for system performance: the fewer indexes that need to be updated in the connector view directory, the better the performance will be.

Reasons for Choosing a Multiple-Instance Configuration
Hosting the meta view and connector views on different Directory Server instances will result in better performance and diminish the likelihood that the failure of one component will greatly affect the performance of another.

A multiple Directory Server configuration makes it easier to track log file data for a component. This also makes it easier to maximize the overall Meta-Directory performance since you will be able to optimize components individually. For more information about Directory Server optimization for Meta-Directory, refer to Chapter 4 "Meta-Directory Performance Tuning."

Using Different Directory Server Instances for the Configuration Database and the Meta View
It is often desirable to use one Directory Server instances for the Meta-Directory configuration database and a different one for the meta view. By default, Meta-Directory expects these two DITs to exists on the same Directory Server instance. To use separate Directory Server instances for these databases, do the following:

1. Install two iPlanet Directory Server instances: one for the meta view and one for the configuration database. These instances can be on the same machine or they can be on different machines. Be sure to give the different instances different names.

2. Install iPlanet Meta-Directory. Make sure you reference the Directory Server instance that you want to host the configuration database during the Meta-Directory installation.

3. Start the iPlanet Console from the Directory Server instance that will host the configuration database. Notice that the iPlanet Console shows several Server Groups.

4. Create an instance of the join engine:
   1. Select the Server Group that represents Meta-Directory.

   2. Right click the Server Group, then choose Create Instance of the Join Engine.

   3. When supplying information for the join engine, type in the information for the Directory Server instance that you want to host the join engine and the meta view (this Directory Server instance will not be in the drop down list of available Data Servers until you specify it when creating an instance of a Meta-Directory component).

      Once you have configured several Directory Server instances, you can view them in the Data Servers tab from the Meta-Directory console.

      
      

      ---

      Note	It is recommended that the join engine bind to the Directory Server as a Directory Manager client. This will give the join engine the permissions it needs to perform its tasks.

      ---

      
      

Server Groups

iPlanet Console uses "server groups" to represents the different sets of iPlanet and Netscape servers and services installed in your Meta-Directory system.

When you install different instances of Administration Server, each one is represented as a server group within iPlanet Console. In your Meta-Directory system, you will have at least one server group displayed in your iPlanet Console. However, it is possible to have several, depending on your Meta-Directory system setup.

For example, suppose you are using iPlanet Directory Server 5.0 to host your Meta-Directory views. In this case, you will first install iPlanet Directory Server then iPlanet Meta-Directory. Both of these components are supported by iPlanet Administration Server 5.0.

Suppose you then use Netscape Directory Server 4.15 to host a direct connector view. Netscape Directory Server 4.15 is supported by Netscape Administration Server 4.2.

In this scenario, there are two versions of Administration Server. When you open iPlanet Console to administer your Meta-Directory system, you will see two server groups, one for each Administration Server.

Figure 2-1 shows three server groups. In this setup, the configuration database and the meta view are hosted by different Directory Server instances, with each instance supported by its own Administration Server. A third server group was created for the Meta-Directory components because a third Administration Server was installed when Meta-Directory was installed. Here, the join engine uses the Directory Server instance in Server Group (2) to host the meta view.

Figure 2-1    Multiple Server Groups in iPlanet Console

Different versions of Administration Server must be installed into different file paths—you cannot install iPlanet or Netscape software into the same file path if the components use different versions of Administration Server.

---

Note

Do not attempt to start the Meta-Directory console or any of its components using Netscape Console 4.x or Netscape Administration Server. Also, do not use Netscape Administration Server to administer Meta-Directory components. However, it is possible (and recommended) to administer Netscape components using iPlanet Console 5.0.

---

Non-Participating Connector Views

In Meta-Directory, it is possible to populate a connector view, but you may choose to not synchronize the data in that view with the meta view. For example, you might wish to flow entries from an Active Directory data repository to those contained in an LDAP connector view for the sake of having the entries available in an LDAP directory. In this configuration, you would not configure the connector view to be a participating view of the meta view, since you would not be flowing entries from this view to the meta view.

If you find a need to populate a connector view, but do not wish to make it a participating view of the meta view, you should use a Directory Server instance other than the one that hosts the meta view as the host for the non-participating connector view.

The reason for this recommendation is that the join engine processes all the change log entries of the connector views that participate in the meta view. If you have a non-participating connector view sharing a Directory Server instance with the meta view, the join engine will waste time processing the change log entries generated by the non-participating view when there is no need to do so.

Locating the Log Files

During the initial phases of deployment, you will likely have verbose logging turned on to help you ensure that things are running smoothly. In these stages of deployment, Meta-Directory can generate copious log files. Setting up your system so that it writes log files to different file systems on separate partitions, can improve performance when system logging is tuned to a high level.

It's important to note that in addition to increased controller traffic on a partition, increasing the logging level of your Meta-Directory components can result in a performance hit on your synchronization times. A higher level of logging will generate more disk write operations. These extra disk write operations use processor cycles that could otherwise be spent on synchronization. Once your system is running smoothly, you might see a gain in performance when you reduce the level of system logging.

Multi-Master Replication

Meta-Directory does not currently support multi-master replication, a feature of iPlanet Directory Server 5.0. The join engine depends on the architecture of the change log facility that is implemented in Netscape Directory Server 4.1x. Multi-master replication uses a change log that is significantly different in design than the one used by the single-master version found in Netscape Directory Server. Currently, when using iPlanet Directory Server 5.0, you must enable the retro change log (which turns on the Netscape 4.1 version of the change log) instead of the newer change log implementation that supports multi-master replication.

Because of this restriction, when using iPlanet Directory Server 5.0, the meta view and connector views must be located in the master of a single-master deployment, and the retro change log plug-in must be enabled.

Bringing the System On Line

At this point you have gone through the process of planning and physically deploying your Meta-Directory system. You have finished buying system resources (for the time being, at least) and have installed all the software needed to bring the system on line. To open the iPlanet Console so you can begin instantiating and configuring the various components in your Meta-Directory system, you will need to start the following servers and services:

1. Start the slapd process that will host your Meta-Directory configuration, then start the Administration Server that administers the slapd process.

2. Start the iPlanet Console that will host your Meta-Directory components.

At this point, you can use iPlanet Console to configure your Meta-Directory system, as described in the iPlanet Meta-Directory Configuration and Administration Guide. However, before you begin the configuration process, consider piloting and tuning your system before going into production.

Piloting

The best way to determine how Meta-Directory will perform in your particular scenario is to run a pilot test of the proposed deployment and measure the system performance. By piloting the system you have designed, you will gain the information needed to properly gauge the production system you will need to put in place. In a pilot deployment, you set up a smaller scale version of the Meta-Directory system you have designed.

During the pilot run, you not only test system topology configurations, but you can also test the different sets of rules that you will use to synchronize data from the disparate data sources. Running and tuning the rules you set up is a key ingredient to achieving the best possible system performance in your production system.

During the pilot run, test one connector at a time before attempting to merge additional connectors into the mix. The purpose is to test your Meta-Directory design, including the data flow, the application of join rules and DN mapping rules, the attribute flow and filtering of entries, and the functionality of constructed attributes. Ideally, your pilot run will give you the confidence to move the system into the complete system that you will bring on line.

Tuning

By monitoring network status and join engine performance, you can look for bottlenecks and other inefficiencies in system performance. Are the system links to external directories working consistently? Are the links between Directory Server instances that host connector views operating optimally?

For more information about tuning Directory Server performance for Meta-Directory, refer to Chapter 4 "Meta-Directory Performance Tuning."

Going Production

When it is time to finally go into production, the best course of action is to begin by synchronizing your main source of data, first into a connector view, then into the meta view. From there, flow other external data sources into connector views, one at a time. Once you have populated a second connector view, go ahead and link the entries with the ones contained in the meta view.

Check your logs to ensure things are flowing correctly by looking for failed join rules. A join rule failure will result in an entry being orphaned; entries are missed when a join rule fails, when a join rule returns more than a single object, or when a join rule encounters bad data. Use the Query Fix-It Tool to search for any unlinked entries and fix them whenever needed. Also, you can search the logs files with using the following commands:

  grep -i fail *
  grep -i error *

If you move one view at a time, running the system and checking the logs after each incremental step, you can more easily identify the location of performance bottlenecks. More importantly, you can address and fix any potential problems as you bring up the system.

Maintaining Your Meta-Directory System

---

During the system design phase, a maintenance plan should be created for maintaining the system once it has been deployed. Meta-Directory system maintenance includes (but is not limited to) the following items:

• Performing Data Backups

• Monitoring the System

• Planning for System Expansion

Performing Data Backups

It should almost go without saying: Perform regular backups of Meta-Directory data. At a minimum, plans should be in place to back up the meta view on a regular basis. In a more elaborate system, you might also consider backing up any connector views that you have populated.

In Meta-Directory, all the views are hosted by Directory Server (except when you use Oracle to host a Direct connector view). The Directory Server provides inherent replication capabilities that can be used to fulfill the task of data backup. Refer to Directory Server documentation for details on this feature.

Monitoring the System

How will you assure reliability and availability in your Meta-Directory system? You should design into the maintenance plan the structure needed to insure that system resources are available and the throughput of the system can be maintained at the needed level. Your enterprise may have an existing structure in place that monitors system resources such as the network traffic and individual system loads. If possible, use this existing infrastructure to monitor your Meta-Directory deployment. In other cases, you may need to set up an individual monitoring system as part of the Meta-Directory maintenance plan.

Monitoring your system resources also encompasses the task of trimming old log files to free up disk space. Provide a schedule for monitoring the log file directories and a procedure for deleting the old, outdated files.

Network security might also be a concern. The maintenance plan should have an eye toward future needs and include a section on how the system security will be kept at required levels.

Planning for System Expansion

In the course of monitoring system loads, you might find that faster performance is needed. Plans should be in place to add more resources in case you need to scale the system.

When does CPU utilization reach the point where additional processing power is required? When does network bandwidth utilization reach the point where additional bandwidth is required? Through the analysis of the system load and performance, you should be able to gauge the resources that are necessary to support the needs of your system.

While it is the job of the system administrator to make sure the system is available, it is the job of the system planners to ensure that the proper resources are on line to fulfill the needs of the system they design. For more information on the requirements of Directory Server, refer to the iPlanet Directory Server Deployment Guide.