Chapter 4 Meta-Directory Performance Tuning

There are several ways to tune the processing performance of Meta-Directory. In tuning, the overall goal is always to speed the flow of data through the Meta-Directory system. However, there are several systems to consider when you tune performance. For example, there are various configuration settings that affect the flow of data through the Meta-Directory components. Also, there are configuration settings that you can adjust to fine tune the performance of the Directory Server instances that you use in conjunction with your Meta-Directory system.

This chapter focuses specifically on the settings you can configure to speed the flow of data through the Meta-Directory components. The next chapter, Chapter 5, "Tuning Directory Server,"discusses the particular things you can do to tune the performance of the different Directory Sever instances running in your Meta-Directory system.

When configuring the Meta-Directory components and the Directory Server instances that host them, keep in mind that, compared to other Directory Server clients, Meta-Directory is particularly write intensive (especially to the Meta View). Because of this, it’s particularly important to optimize the write performance of the Directory Server instance that’s hosting the Meta-Directory components. This is very different than tuning the performance of a Directory Server instance that is hosting a user directory. These are normally optimized for read performance.

Adding Meta-Directory Indexes

Bulk Loading Data

Tuning the Data Servers

Tuning the I/O Block Time-Out Setting

Adding Meta-Directory Indexes

To improve the performance of the Meta-Directory system, you should create indexes for certain attributes in the Meta-Directory object class. The Meta-Directory object class attributes are contained in the Meta-Directory schema—you must first load the Meta-Directory schema before you can index these attributes. This is discussed in the section "Loading Meta-Directory Schema".

The Meta-Directory indexes are created and maintained by Directory Server. You create the indexes on each Directory Server instance based on the Meta-Directory views that are hosted by that Directory Server instance. In all, you can create indexes on the following five Meta-Directory object class attributes:

hasSubordinates

mdsLinkToCV

mdsLinkToMV

mdsExcludedCVs

mdsExcludedMVs

You can add these indexes through either the Directory Server console or from the command line.

Determining Which Indexes to Create

You create the indexes only on Directory Server instances that host Meta-Directory components. For example, if a particular Directory Server instance hosts only a Meta View, then you should create and maintain the following Meta-Directory indexes on that Directory Server instance:

Table 4-1 Indexes for Directory Server Instances That Host a Meta View
Index Attribute	Index Settings
hasSubordinates	Equality
mdsLinkToCV	Presence and equality
mdsExcludedCVs	Presence and equality

Likewise, if a Directory Server instance hosts only Meta-Directory Connector Views, then you need to create the following indexes on that Directory Server instance:

Table 4-2 Indexes for Directory Server Instances That Host Connector Views
Index Attribute	Index Settings
hasSubordinates	Equality
mdsLinkToMV	Presence and equality
mdsExcludedMVs	Presence and equality

In addition to the above guidelines, you should also add the Connector View indexes listed in Table 4-2 to a Directory Server instance that hosts the Meta-Directory configuration (o=NetscapeRoot) if that Directory Server instance also hosts a database (Oracle) Connector View.

To Add Indexes From the Directory Server Console

Open the Directory Server console.

Navigate to the database index pane:

In the Sun ONE Directory Server 5.x console, choose the Configuration tab, expand the Data node, then select Database Settings in the navigation tree.

In the Netscape Directory Server 4.1x console, choose the Configuration tab and select the Database node in the navigation tree.

The right-hand pane will display the attributes that are currently indexed.

In the right-hand pane, click Add Attribute.

Select the attribute name you want to index and click OK.

Select the indexes you want to create for the attribute (see Figure 4-1).

Repeat Steps 3 through 5 for each attribute listed in Table 4-1 and Table 4-2.

Click Save to apply changes.

Stop and restart the server.

Figure 4-1 Adding Indexes for Meta-Directory Attributes
Figure shows the options to add the index attributes in the Directory console window.

Adding Indexes From the Command Line

In iDS 5.0, you load dse.ldif, indexes must be in LDIF, not as below. It’s highly recommended that you only do this from the Console.

To add the necessary index attributes from the command line, open and modify the file slapd.ldbm.conf:

From a command prompt, stop the Directory Server using the stop-slapd command.

Open the file slapd.ldbm.conf (located in the directory NETSITE_ROOT/sloped-ServerInstance/config) and add the following lines in accordance with the guidelines listed in “Determining Which Indexes to Create”:

“index hasSubordinates eq”

“index mdsLinkToCv pres,eq”

“index mdsLinkToMv pres,eq”

“index mdsExcludedCvs pres,eq”

“index mdsExcludedMvs pres,eq”

Save and close the file.

Restart the server.

Bulk Loading Data

There are certain adjustments that you can make to speed the time it takes to populate a Meta View with entries from a new data source.

In particular, you should adjust the database cache sizes and the All IDs Threshold before you begin to load your initial sets of data into Meta-Directory.

Adjusting the Database Cache Size

If you are importing a very large database from LDIF, set the Maximum Cache Size parameter as large as possible. The larger you can set this parameter, the faster your database will be imported.

Determine how much free memory you have on your system.

Divide the amount of free RAM by two and subtract 1 Mb from the subtotal.

For example, if you have 50 Mb of free memory on your system, divide 50 by 2 (giving a subtotal of 25 Mb) and subtract 1 MB, giving you a final total of 24 Mb. Here, you would set the Maximum Cache Size in Bytes parameter to 24000000.

To Set the Maximum Cache Size

Open the Sun ONE Directory Server console.

Choose the Configuration tab, expand the Data node, then select Database Settings in the navigation tree.

In the right pane, choose the LDBM Plug-in Setting tab.

Enter your new setting in the Maximum Cache Size input box.

Choose Save to save the new setting.

Open the Directory Server console.

Choose the Configuration tab and select the Database node in the navigation tree.

In the right pane, choose the Performance tab.

Enter your new setting in the Maximum Cache Size input box.

Choose Save to save the new setting.

Adjusting the All IDs Threshold

The All IDs Threshold is used to limit the size of Directory Server indexes (for a full description of this feature, see "Tuning the All IDs Threshold").

Directory Server sets the default value for All IDs Threshold to 4,000. For Meta-Directory, this number is usually too low. For example, if you are flowing 100,000 entries from a Connector View to an empty Meta View, the recommended All IDs Threshold setting is 100,100. If you are flowing 300,000 entries, then set the All IDs Threshold to 300,100.

You should not bulk-load data into a Meta-Directory view until you have adjusted this setting.



Note	The All IDs Threshold setting alters the way indexes are written, so you must rebuild the indexes after adjusting this setting. Re-importing the data is the fastest and most direct way to create a fresh set of indexes.

For information on tuning the All IDs Threshold, refer to the Sun ONE Directory Server Administrator’s Guide.

Tuning the Data Servers

Meta-Directory has tuning parameters that pertain to the data servers that Meta-Directory accesses. You can configure these settings through the Data Server > Tuning tab in the Meta-Directory console.

In particular, you should configure the Time of Search to fit your specific system needs.

DCNS Scheduler

Configuring the Time of Search

DCNS Scheduler

To configure and tune the Data Change Notification System (DCNS), use the DCNS Schedule tab in the Data Server window of the Sun ONE Console. The parameters in this window specify the change log polling schedule and the maximum number of entries to be obtained for each poll of the change log.

The default settings specify that a change log poll will be issued every 15 seconds with a maximum number of 1024 entries to be obtained per poll. To optimize these settings for Meta-Directory, you should adjust the change log settings so it is polled after the join engine has processed at least 50 to 60 percent of the entries that it obtained from the previous poll. This setting will reduce memory consumption because it reduces the number of unprocessed entries added to the queue.

Although you should calculate your system performance, you can form a rough estimate by assuming the average time it takes to move an entry from the change log to the destination Connector View is one second.

For example, if you pull 1024 entries from the change log with each poll, the poll interval should be 512 seconds or more. Another way to reduce the amount of entries queued (and thus reduce memory consumption) would be to increase the polling interval, giving the join engine more time to process entries in the queue before the next poll.

When tuning the polling interval, consider the number of entries being obtained per poll rather than something such as the change log size. Results from queries are maintained internally. Results can overflow the heap when relatively large numbers of entries are obtained per poll.

Configuring the Time of Search

From the Meta-Directory Console, you can specify several tuning parameters for the different Directory Server instances that you have running in your Meta-Directory setup. Of particular interest is the amount of time that a data server should spend before timing-out on a specific search.

By default, the search time is set to 3,600 seconds (one hour). While this should be enough for most situations, you might need to adjust this setting if your system is deprived of resources.

If a system time-out occurs, the Directory Server in question will stop processing. If the log level is set to dataaccess=3, the following message will be written to the log file: DA_USER_TIMELIMIT_EXCEEDED.

Open the Meta-Directory console.

In the left pane, select Meta-Directory in the navigation tree.

In the right pane, choose the Data Servers tab to view the list of Directory Server instances used in your Meta-Directory setup.

Select the Directory Server instance which you need to adjust. A list of tabs will appear in the pane below.

Select the Tuning tab, then specify the new time-out setting in the Maximum Operation Result Time input box.

Click Save.

Tuning the I/O Block Time-Out Setting

In Directory Server 4.1x, you should adjust the IO Block Time Out setting to prevent the Directory Server from timing out during the time it takes the join engine to synchronize Meta-Directory views. This setting is located in the Directory Server configuration file, found at the following location:

The IO Block Time Out setting controls the time-out period for stalled clients. The default setting for the Directory Server’s ioblocktimeout parameter (180,000,000 ticks on the computer’s system clock) may not be great enough to allow the synchronization of a large number of entries. As a result, the Directory Server may time out the synchronization process before your data is fully synchronized. You are more likely to see this on slower clients.

The default value of the ioblocktimeout parameter (stored in slapd.conf) of 180,000,000 ticks is usually enough. However, keep in mind that the number of ticks per second is computer and platform-dependent. If you see time-out errors, you may need to increase this value.

For more information about the ioblocktimeout parameter, refer to the Sun ONE Directory Server Administrator’s Guide.