Chapter 5   Tuning Directory Server


iPlanet Meta-Directory uses iPlanet Directory Server (or, alternatively, Netscape Directory Server 4.1x) to store its configuration settings. Meta-Directory also uses the Directory Server to store the entries contained in the meta view. In addition to these required uses of the Directory Server, it's likely you will also use Meta-Directory to synchronize one or more Directory Server user databases.

This chapter describes how to optimize the Directory Server performance when you use it in conjunction with Meta-Directory.

While there are several ways to configure and tune the performance of iPlanet Directory Server, performance tuning can be grouped into two broad categories:

• Write performance tuning

• Read performance tuning

The main goal of optimizing Directory Server performance with regard to Meta-Directory is to maximize the number of Directory Server operations per second. In light of this, consider the following:

• A single search (read) request takes about 50 to 250 Directory Server operations.

• A single modify (write) request takes about 200 to 1,000 Directory Server operations.

In addition to the number of operations it takes per request, take into consideration that disk access is thousands of times slower than physical memory access. The nature of write requests dictates that they access the disk for each request. This makes them many times slower than read operations, which can often be completed quickly by accessing physical memory caches.

Compared to other Directory Server clients, Meta-Directory is particularly write intensive. Because of this, it is important to optimize the write performance of the Directory Server that's being used in conjunction with Meta-Directory. This chapter begins with a section on how to optimize the performance of Directory Server write operations, then it discusses optimizing Directory Server read performance.

Directory Server Write Performance Tuning

---

Simply put, write performance is the amount of time it takes to make an update to the Directory Server data stored on disk. When optimizing write performance, you must consider not only the database entry containing you're modifying, but also the many different files that reference the entry. When you add or modify a Directory Server entry, the following write operations take place:

• The new data is written to the entry.

• Any indexes containing attributes in the entry are updated.

• The update is noted in the transaction log.

• The access and audit logs are updated.

• The replication change log is updated if replication is to be performed.

Since write performance is directly related to the amount of information that must be written to disk, anything that reduces or eliminates disk traffic will speed updates. This being the case, write-tuning efforts are centered around optimizing the traffic into and out of the disk subsystem. This optimization comes in the following two forms:

• Minimize the amount of information that must be written to disk.

• Increase the throughput of disk write operations by using faster disks or by spreading the load across multiple disks.

Managing the Information Written to Disk

When a directory modification takes place, the following files can be affected:

• The database file

• Any database index files

• The transaction log

• The access, error, and audit log files

• The replication change log

• Various back-end files

Although there is not much you can do about optimizing the actual database file (it must be modified whenever there is a change), there are ways that you can minimize the amount of disk writes by carefully selecting the files that are maintained by the Directory Server.

Optimizing Indexes

While indexing makes searches much faster, they must be updated by the directory for each write operation. When creating indexes, you must balance the needs of fast data access to directory write performance.

You can make the biggest gain by removing any indexes that are unneeded. If your applications never search on a particular attribute or do so infrequently, then putting an index on that attribute will only slow directory writes without providing any search performance gains. When you first install Directory Server, it is configured with a certain set of default indexes. You should always examine the list of default indexes and remove any that are unnecessary.

It is also important to consider the types of indexes the directory is maintaining. Each type of index, while useful for optimizing particular types of queries, is also associated with a particular cost whenever that index must be maintained. Table 5-1 illustrates the relative cost of maintaining certain types of indexes. The cost is expressed in terms of the number of logical database writes associated with maintaining the index for a given value.

Table 5-1    Relative Cost to Index a Value

Index Type	Cost of Indexing a Value	Example Value "first, middle, last"
Presence	1 (binary)	1
Equality	1 (binary)	1
Approximate	Number of words in value	3
Sub-string	Number of characters in value	17

From the table above you can see that substring indexes are far more expensive to maintain than equality or presence indexes.

To optimize indexes, always use the least expensive type of index based on the types of queries being made to an attribute. For example, if you rarely expect to execute a wildcard search on a "uid" attribute, then you should not maintain a substring index for that attribute.

For more information on indexes and their types, refer to the chapter "Introduction to Directory Server" in the iPlanet Directory Server Deployment Guide. In addition, the "Managing Indexes" chapter in the iPlanet Directory Server Administrator's Guide contains details on how to configure Directory Server indexes.

The Transaction Log

iPlanet Directory Server keeps a transaction log that tracks the operations of all the databases it manages. Whenever a database operation (such as a write) is performed, the server logs the operation to the transaction log. If the Directory Server experiences a failure, it uses the transaction log to recover any affected databases.

In a system that performs many writes (such as a with a Meta-Directory system), activity to the transaction log can become significant. To optimize system performance, you want to specify a location for the transaction log that is different from the user database accessed by the Directory Server. This will minimize conflicting disk seeks and writes between the two files.

You can also increase Directory Server write performance by specifying the log file's checkpoint tag. For details on configuring the transaction log, see the "Tuning Directory Server Performance" chapter of the iPlanet Directory Server Administrator's Guide.

The Access, Error, and Audit Log Files

The Directory Server also maintains several user log files. Unlike the transaction log, which is stored in DB2 format, the user logs are stored in ASCII text. Specifically, the Directory Server has the following user logs:

• The access log tracks front-end Directory Server events such as authentication and binding.

• The error log tracks back-end Directory Server events.

• The audit log is similar to the transaction log, except it is kept in user-readable ASCII text.

While you can configure each of these logs in various ways, the most import optimization concern is the placement of the logs. Similar to the transaction log, you should locate the user logs in a disk partition that is different from the partition containing the user database.

In addition to controlling the location of the logs, you can also control the size of the logs (based on number of entries per log file or length of time before a new log file is opened), level of verbosity in the logs, and sometimes the deletion of the log files.

For details on configuring the Directory Server user logs, refer to the chapter titled "Monitoring Server and Database Activity" in the iPlanet Directory Server Administrator's Guide.

The Replication Change Log

The Directory Server uses the change log with database replication. In replication, every supplier server maintains a change log. A change log is a record that describes the modifications that have occurred on a replica. When an entry is modified, a change record describing the LDAP operation is recorded in the change log for the respective replica. The supplier server uses the change log to pass modifications made on one replica to the other consumer servers (or in the case of multi-master replication, on other master servers).

Like the user log files, you can specify the location of the change log and how large it should grow before a new change log file is opened. For details on replication and the change log, refer to the chapter "Managing Replication" in the iPlanet Directory Server Administrator's Guide.

Summary of Write Tuning

While this section deals almost entirely with how to configure iPlanet Directory Server, there are some important considerations to keep in mind when tuning Directory Server with regard to iPlanet Meta-Directory. To summarize:

• As much as possible, keep the user directory information tree (DIT) on a disk partition that is separate from the partition storing configuration, index, and log files.

• Remove any unnecessary indexes. Many of the default indexes supplied in the directory might not be necessary for your specific database needs.

• Minimize or eliminate logging. Turn off the access or audit logs if you don't use them on a regular basis or require them for audit purposes.

Tuning Directory Server Read Performance

---

By design, iPlanet Directory Server is tuned to perform fast read (search) operations. However, by carefully adjusting some of the Directory Server settings, you can optimize the read performance. Read performance settings can be grouped into the following categories:

• Back-end (database) settings

• Front-end settings

Back-end settings affect the physical make up of the caches and indexes while front-end settings can be used to tune the ways in which clients access the Directory Server indexes and database.

Directory Server Back-End Settings

The iPlanet Directory Server read and search performance is based on two key back-end technologies: database caching and indexing.

Virtually all of your read and search performance efforts should be directed at optimizing the performance of the Directory Server caches and indexes. Tuning other aspects of the Directory Server will provide minimal benefits if the indexes and caches are not properly tuned.

• Caching speeds the processing of searches by storing indexes and entries in physical memory as opposed to on a disk. You tune the performance of the Directory Server by reducing the number of database reads the server has to perform in order to retrieve the desired entries. You can increase Directory Server performance by optimizing the index-to-entry cache ratio.

• Indexing speeds the processing of searches by storing the values of specified attributes. If you perform a search on an attribute that has been indexed, the Directory Server can quickly search the index for the entries that meet the search criteria. You can configure the indexes created, tuning the Directory Server based on the specific types of queries the directory has historically been asked to perform.

When optimizing the performance of the Directory Server back-end, you must balancing the configuration of the following cache and index settings:

• Cache settings
  • Database cache

  • Entry cache

• Index settings
  • All IDs threshold

Tuning the Directory Server Caches

The Directory Server utilizes caches to speed search performance by storing database information in physical memory. In addition to faster reads, caches increase the speed of query returns because entries stored in the entry cache have already been converted from DB2 database format to LDAP format.

iPlanet Directory Servers has two distinct types of caches:

• The database cache stores pages from the database, caching both indexes and data.

• The entry cache stores the most recently access entries form the directory. It uses a least-recently-used algorithm to ensure that the most frequently accessed directory entries are available in physical memory.

While the database cache is set to the size of the maximum amount of physical memory it should consume, the entry cache setting is based on the maximum number of entries it should hold.

To maximize directory read performance, you must cache as much directory data in physical memory as possible. By preventing the directory from having to read information from disk, you can increase read performance by eliminating the disk subsystem. When sizing the cache settings, there are three rules you must follow:

• Your database cache must always be large enough to hold the databases indexes.

  While the database cache should hold more than just the indexes, you must always ensure that it is at least big enough to hold the directory indexes. If it isn't, the directory will be forced to read indexes from disk for every search request, which will quickly bring directory throughput to a virtual halt.

• Your database and entry caches must always fit into available, physical memory.

  If the size of the two caches combined is bigger than the amount of available physical memory on the machine, the operating system will begin to use virtual memory, swapping the cache to and from disk. This can cause a significant amount of disk "thrashing" that will quickly bring not just the directory, but the entire system to a virtual halt.

• The database cache is more important than the entry cache.

  When given a choice between allocating memory to the database cache or to the entry cache, you should generally favor the database over the entry cache.

Sizing the Database and Entry Cache Settings

To maximize Directory Server read performance, you will want to allocate as much available memory as possible to the caches. As very general rule, efficient operation can be achieved by allocating memory between the caches in the ratio of 75 percent for the database cache and 25 percent for the entry cache.

Sizing the Database Cache
The size of the database cache can be set through the iPlanet Console. It is important to note, however, that the actual amount of memory used by the database cache can exceed the size you specify by up to 25 percent. This is due to additional memory required to manage the cache itself. Also, it is important that the database cache (with overhead) should not be set to consume over two gigabytes (2 Gb) of memory. It is not capable of using more memory than that.

Sizing the Entry Cache
Unlike the database cache, the entry cache size is set not by the amount of memory you would like it to consume but by the maximum number of entries you would like it to hold. The actual amount of memory it will consume is a function of the average entry size.

For example, if your average entry size is 1Kb, and you specify that the entry cache should hold a maximum of 10,000 entries, then the amount of memory the entry cache will consume will be 12.5 Mb, calculated as follows:

(1 Kb / entry * 10,000 entries) = 10 Mb + 25 percent for cache management overhead.

To determine the average entry size, you will need to use your best judgment of the data the directory will hold. You can determine the size of an entry by adding the number of characters needed to list the entry and adding a carriage-return and line-feed for each line in the entry. For example, the entry below for Lenny Riceman is 398 bytes.

dn: uid=lriceman, ou=People, o=siroe.com
cn: Lenny Riceman
sn: Riceman
givenname: Lenny
objectclass: top
objectclass: person
objectclass: organizationalPerson
objectclass: inetOrgPerson
ou: Accounting
ou: People
l: Sunnyvale
uid: lriceman
mail: lriceman@siroe.com
telephonenumber: +1 408 555 4798
facsimiletelephonenumber: +1 408 555 9751
roomnumber: 4612
userpassword: cocoloco

Computing the Database and Entry Cache Sizes

While bigger caches are usually better, performance can degrade if dbcachesize is too large. If you set the database cache size exceeds the available free memory, the process will begin to thrash virtual memory as it writes clean and dirty pages to disk, resulting in severe performance degradation. (You can detect this behavior by using one of the system monitoring tools such as vmstat that provides statistics for virtual memory usage.)

To size your database and entry caches, use the following technique:

1. Identify the amount of free memory available when your system configuration is operating normally (use system monitoring tools).

2. Take 75 percent of your free memory as the base of your database cache size.

3. Divide the base database cache size (computed above) by 1.25 to account for the cache overhead, and use the result as the database cache size value.

   If the result is larger than 1.6 Gb, reduce it to 1.6 Gb, the maximum dbcache size.

   If the result is smaller than the size of all database indexes, increase the database cache to cover the size of all the indexes without exceeding available memory. Calculate the total index size (add up the sizes of *.db2 files in the Directory Server database directory and subtract the size of user database file id2entry.db2).

4. Take 25 percent of your free memory and use this as the base for your entry cache size.

5. Divide the base entry cache size by 1.25 to account for cache overhead.

6. Divide the result by the average entry size and use the result as the entry cache size (maximum number of cache entries).

   Determine the size of an entry by adding up the size of all its attributes. Examine your data closely to determine average entry size.

A Short Example
As an example, assume your system has the following properties:

• 150 Mb of available physical memory (RAM)

• An estimated entry size of 1Kb / entry

To size the database cache:

150 Mb * .75 = 112 Mb to allocate to the base database cache

112 Mb / 1.25 = 90 Mb for the Database cache

To size the entry cache:

150 Mb - 112 Mb = 38 Mb for base entry cache

38 Mb / 1.25 = 30 Mb avail for entries

30 Mb / 1 Kb = max 30,000 cache entries

To configure Directory Serve cache sizes, do the following:

1. From the Directory Server console, select the Configuration tab.

2. Navigate to the database performance configuration settings:
   • In the iPlanet Directory Server console, expand the Data node in the navigation tree, then select Database Settings. In the right pane, choose the LDBM Plug-In Setting tab.

   • In the Netscape Directory Server console, select Database node in the navigation tree, then select the Performance tab in the right pane.

   The current database performance settings appear.

3. Enter the amount of memory you want to make available for open index files in the Maximum Cache Size text box.

4. In Directory Server 4.1x, enter the number of entries you want the server to keep in memory in the Maximum Entries in Cache text box (note that iPlanet Directory Server does not support this configuration setting).

Monitoring the Database Cache

Once you have set your initial cache sizes, you should monitor your cache utilization from time to time to ensure your caches are being used efficiently. You can determine the effectiveness of your caches by examining the Database Performance Counters available through the iPlanet Console. you can view the performance counters by selecting the Status tab in the Directory Serve console.

Under optimal conditions, both the entry and the database cache hit ratios will be above 95 percent. If either hit ratio is less than 95 percent, and you have additional available physical memory, you should consider increasing your cache size to increase the hit ratio.

If you have been running the Directory Server for several days and find that your entry cache is not filled to the maximum level, you might also consider lowering the size of your entry cache to just above the high-water mark and giving any freed memory to the database cache.

For information about changing entry cache settings using the iPlanet Console, refer to theiPlanet Directory Server Administrator's Guide.

Monitoring Performance From the Command Line

In Netscape Directory server 4.1x, you can also monitor the database cache settings through the entry "cn=monitor, cn=ldbm". To read the database monitor entry from the command line, use the following command:

ldapsearch -b "cn=monitor,cn=ldbm" -s base "objectclass=*"

The cache parameters are listed in Table 5-2:

Table 5-2    Cache Parameters for ldapsearch Command  

Console Label	Monitor Entry Attribute	Description
Entry cache hits	entrycachehits	Number of requests filled from the entry cache.
Entry cache tries	entrycachetries	Number of total requests to the entry cache.
Entry cache hit ratio	entrycachehitratio	Percentage of requests filled from the entry cache.
Current number of entries in entry	currententrycachesize	Current number of entries in the entry cache.
Hits (under the Database Cache heading)	dbcachehits	Number of requests filled from the database cache.
Tries (under the Database Cache heading)	dbcachetries	Number of total requests to the database cache.
Hit ratio (under the Database Cache heading)	dbcachehitratio	Percentage of requests filled from the database cache.

Sizing the Entry Cache to Prevent Over-Allocation of Memory

In some cases, the Directory Server entry cache settings may be set to a value that results in the over-allocation of memory to the ns-slapd process, resulting in the slow but consistent growth of the ns-slapd process until it is so large that it causes every operation of the machine to require an excessive amount of time, and the Directory Server needs to be restarted because of the memory over-allocation.

The entry cache, which is used to store a copy of LDAP entries as they are being read, is most useful when the same entries are being pulled up repeatedly within a reasonable period of time. In the case of an ISP, at a given time there are usually fewer repeats of previously searched LDAP entries than there are original searches. In this case, or in similar situations where there are large numbers of users performing searches, it may be desirable to lower the entry cache, especially if the size of the entry cache is occasionally contributing to the over allocation of memory to the ns-slapd process.

Tuning the Directory Server Indexes

Proper indexing is the most important thing you can do to improve the Directory Server read performance. To optimize the indexing of your database, you need to understand the types of searches that are being submitted to your Directory Server database. Knowing this, you can create indexes for the attributes that are used in the searches.

For example, if your only application is a mail server, it will most likely search only for an exact match on the UID attribute. You would therefore set an index on the UID attribute. Setting indexes on other attributes would be necessary only if your Directory Server is accessed by other types of applications.

You can determine the types of searches your directory is handling by examining the access log. The access log records search requests in SRCH records. For example, you might see the following log entry:

[05/Mar/2001:09:23:35 -0800] conn=4 op=6
  SRCH base="ou=People,o=siroe.com" scope=2 filter="(uid=jsmith)"

By looking through the access log for all search requests, you can determine the type and frequency of the queries being performed. Your goal is to ensure that all commonly performed searches are fully indexed, meaning all attributes used in a search has a corresponding index.

---

Note

While indexing makes searches much faster, the server must manage the indexes you create whenever it writes an entry. As a result, maintaining indexes causes adds, modifies, deletes, and imports to be slower than they would be without indexes. Because of this, you must strike a balance between read performance and write performance. Write performance, and how indexing affects it, is discussed in the section above, "Directory Server Write Performance Tuning."

---

It's important to keep in mind that any search on an unindexed attribute will require the server to physically look at each entry in the directory tree (DIT) to determine if it meets the search criteria. Searching the entire DIT is slow and expensive.

With iPlanet Directory Server, you can use the access log to directly locate searches that referenced unindexed attributes. Do this by searching the access log for RESULT records that contain a notes=U entry. For example:

[02/Mar/1999:11:42:49 -0800] conn=4 op=6 RESULT err=0 tag=101
  nentries=1 etime=0 notes=U

By matching the connection (conn) and operation number (op) fields with the corresponding SRCH record, you can determine which searches are being performed without indexes.

When configuring indexes, it is also important to use the correct type of index to match the types of searches being performed. Using incorrect index types will at best impact the write-performance of the directory; at worst, incorrect index types provide the equivalent of using no indexes at all.

Indexing and index types are described in depth in the iPlanet Directory Server documentation. Specifically, the "Introduction to Directory Server" chapter in the iPlanet Directory Server Deployment Guide contains an overview of indexing and index types, while the "Managing Indexes" chapter in the iPlanet Directory Server Administrator's Guide provides details of creating and maintaining indexes.

Tuning the All IDs Threshold

Each index that the directory server uses is comprised of a table of index keys and matching list of entry IDs. That is, for each index key there is a list of directory entry IDs that match the key. This entry ID list is used by Directory Server to build a list of candidate entries that may match a specified search filter.

Directory Server sets a size limit for each entry ID list. This size limit is globally applied to every index key managed by the server and it is called the All IDs Threshold. When the size of an individual entry ID list reaches this boundary, the server replaces that entry ID list with an All IDs token.

The All IDs token causes the server to assume that all directory entries match the index key. In effect, the All IDs token causes the server to behave as if no index was available for that particular key and a search on it will cause the server to search the entire directory database. The assumption is that some other aspect of the search filter will allow the server to narrow its candidate list before processing the list.

For more details on the All IDs threshold and how to calculate the best setting, refer to the chapter titled "Managing Indexes" in the iPlanet Directory Server Administrator's Guide.

Directory Server Front-End Settings

The front-end server settings let you manage your server's performance by limiting the amount of resources the Directory Server puts into client search requests. Although LDAP clients can specify that the server use smaller values for the Size Limit and Time Limit settings, you should review the following configuration settings to optimize system performance:

• Size limit specifies the maximum number of entries the server will return to the client in response to a search operation. If this limit is reached, the server returns any entries it has located that match the search request, as well as an exceeded size limit error. The default value for this parameter is 2,000. Decreasing this value could reduce your average search time but will also limit the number of results returned on very large searches.

• Time limit specifies the maximum amount of real time (in seconds) you want the server to spend performing a search request. If this limit is reached during a search, the server returns any entries it has located that match the search request, as well as an exceeded time limit error. The default value for this parameter is 3,600 (1 hour). Decreasing this value will produce similar results to decreasing Size Limit.

• Idle Time-out specifies the time (in seconds) you want the server to maintain an idle connection before terminating.

• Look-through limit specifies the maximum number of entries the server will check when seeking candidate entries in response to a search request. If this limit is reached, the server returns any entries it has located that match the search request as well as an exceeded size limit error.

  The default value for this parameter is 5,000. Decreasing this value could reduce the average search time per request but will also produce more exceeded size limit errors and empty search requests.

  As a guideline, set this parameter 10 percent above your value for Size Limit.

• Maximum file descriptors sets the maximum number of file descriptors available to the Directory Server. This setting effects the number of simultaneous connection that can be made to Directory Server.

To Modify the Directory Server Settings

1. Open the Directory Server console and select the Configuration tab.

2. From the Configuration tab, select the root entry in the navigation tree in the left pane.

   The server-wide configuration tabs appear in the right pane.

3. Select the Performance tab in the right pane.

   The current front-end server performance settings appear.

4. Specify the maximum number of entries the server will return to the client in response to a search operation by entering a new value in the Size Limit text box. If you do not want to set a limit, type -1 in this text box.

5. Specify the maximum amount of real time (in seconds) you want the server to spend performing a search request in the Time Limit text box. If you do not want to set a limit, enter a zero (0) in this text box.

6. Specify the time (in seconds) you want the server to maintain an idle connection it terminates in the Idle Timeout text box. If you do not want to set a limit, enter a zero (0) in this text box.

7. Set the maximum number of file descriptors available to the Directory Server in the Max Number of File Descriptors text box. (Note that this option is not available on Windows NT or IBM® AIX systems.)

While the values set on these parameters are hard limits on the resources that the server will apply to each request, they are not enforced on the root DN. For further details on how these parameters impact your server's searching performance, refer to the "Managing Indexes" chapter in the iPlanet Directory Server Administrator's Guide.