Previous Next Contents Index


Chapter 7 Managing Indexes

Netscape Directory Server uses index files to aid in searching the directory. Indexes greatly improve the performance of searches in the directory databases, but they do so at the cost of slower database modification and creation operations. Indexes also cost more in terms of system resources, especially disk space.

This chapter discusses how you can use indexes in the following sections:


The Searching Algorithm
The directory server uses the following process when performing a search:

  1. An LDAP client such as Netscape Communicator or the directory server gateway sends a search filter to the directory server.

  2. The directory server examines the incoming request to make sure that the specified base DN matches a suffix assigned to the local database.

  3. If the search filter criteria may be satisfied by a single index, then the server reads that index to generate a list of potential, or candidate, matches.

    4. If an appropriate index is not found, the directory server generates a candidate list that includes all entries in the database. (The directory server will also do this if the All IDs token is set for the index key that the server is using. For information on All IDs, see "Managing All IDs Threshold".)

    If a search filter containing multiple criteria is used, the directory server consults multiple indexes and then combines the resulting lists of candidate entries.

  4. The candidate matches are taken from the index files in the form of a series of entry ID numbers.

  5. The directory server uses the returned entry ID numbers to read the corresponding entries from the id2entry.db2 file. The directory server then examines each of the candidate entries to see if any match the search criteria. Matching entries are returned to the client as each is found.

    6. The directory server continues until it either has examined all candidate entries, or until it reaches the limit set on one of the following parameters:

    The client may further restrict the number of returned entries by specifying lower values for the Size Limit and the Time Limit parameters on the search request.


Types of Indexes
To improve searching efficiency, the directory server can maintain six index types:

pres—Presence Index. Improves searches for entries that contain the indexed attribute.

eq—Equality Index. Improves searches for entries that contain an attribute that is set to a specific value.

approx—Approximate Index. Used only for string values such as commonName or givenName. Improves phonetic, or "sounds-like," searching.

sub—Substring Index. Improves searches for entries that contain a specified substring.

matching rule—International Index. Allows for searches that return entries sorted according to a specified collation order.

browse—Browsing Index. This index is used to enhance the browsing speed of the Directory Server Console.

Note. The server stores indexes in files. The names of the files are based on the indexed attribute, not the type of index contained in the file. Each index file may contain multiple types of indexes if multiple indexes are maintained for the specific attribute. For example, all the indexes maintained for the common name attribute are contained in the cn.db2 file.

Presence Index

The presence (pres) index is the simplest of the indexes. It lets you efficiently search the directory for entries that contain a specific attribute. This is useful if, for example, you want to examine any entries that have access control information associated with them. Generating an aci.db2 file that includes a presence index lets you efficiently perform the search for ACI=* in order to generate the Access Control List for the server.

Equality Index

The equality (eq) index allows you to efficiently search for entries containing a specific attribute value. For example, an equality index on the cn attribute allows a user to efficiently perform the search for cn=Babs Jensen.

Approximate Index

The approximate (approx) index allows efficient approximate or "sounds-like" searches. For example, an entry may include the attribute value cn=Robert E Lee. An approximate search would return this value for searches against cn~=Robert Lee, cn~=Robert, or cn~=Lee. Similarly, a search against l~=San Fransico (note the misspelling) would return entries including l=San Francisco.

The directory server uses a variation of the metaphone phonetic algorithm to perform this type of searching. Each value is treated as a sequence of words, and a phonetic code is generated for each word.

Values entered on an approximate search are similarly translated into a sequence of phonetic codes. An entry is considered to match a query if both of the following are true:

For example:

Name in the directory
(Phonetic code)
Query string
(Phonetic code)
Match comments
Alice B Sarette
(ALS B SRT)
Alice Sarette
(ALS SRT)
Matches. Codes are specified in the correct order.

Alice Sarrette
(ALS SRT)
Matches. Codes are specified in the correct order despite the misspelling of Sarette.

Surette
(SRT)
Matches. The generated code exists in the original name despite the misspelling of Sarette.

Bertha Sarette
(BR0 SRT)
No match. The code BR0 does not exist in the original name.

Sarette, Alice
(SRT ALS)
No match. The codes are not specified in the correct order.

Substring Index

The substring (sub) index is a costly index to maintain, but it allows efficient searching against substrings within entries.

For example, searches of the form:

cn=*derson

would match the common names containing strings such as

Bill Anderson
Jill Anderson
Steve Sanderson

Similarly, the search for

telephonenumber= *555*

would return all the entries in your directory with telephone numbers that contain 555.

Note. Substring indexes are limited to a minimum of two characters for each entry.

International Index

The international index speeds up searches for information in international directories. The process for creating an international index is similar to the process for creating regular indexes, except that you apply a matching rule by associating a locale (OID) with the attributes to be indexed. For a listing of supported locales and their associated OIDs, see Table  B.1. If you want to configure the directory server to accept additional matching rules, see the Netscape Directory Server Plug-In Programmer's Guide.

For more information on creating an international index, see "Creating Indexes".

Browsing Index

The browsing index, or virtual list view index, speeds up the display of entries in the Directory Server Console. This index is particularly useful if you have a branch that contains hundreds of entries, for example, the ou=people branch. You can create a browsing index on any branchpoint in the directory tree to improve display performance. You do this through the Directory Server Console or by using the vlvindex command-line tool.

For information on creating and removing browsing indexes, see "Using Browsing Indexes".


The Cost of Indexing
Although indexes greatly improve the speed of searches within the directory server database, it helps to be aware of the costs related to indexing as described in the following sections:

Slower Database Modification and Creation Times

The more indexes you maintain, the longer it takes the directory server to update the database. This is especially true for substring indexes that cause the directory server to generate multiple index file entries every time an attribute value is created or changed. Remember that for substring indexes, the number of index entries created is proportional to the length of the string being indexed.

Consider the procedure for creating a specific attribute:

  1. The directory server receives an add or modify operation.

  2. The directory server examines the indexing parameters to determine whether an index is maintained for the attribute values.

  3. If the created attribute values are indexed, then the directory server generates the new index entries.

  4. Once the server completes the indexing, the actual attribute values are created according to the client request.
For example, suppose the directory server is asked to add the entry

dn: cn=Bill Pumice, ou=People, o=airius.com
objectclass: top
objectClass: person
objectClass: orgperson
objectClass: inetorgperson
cn: Bill Pumice
cn: Bill
sn: Pumice
ou: Manufacturing
ou: people
telephonenumber: 408 555 8834
description: Manufacturing lead for the Z238 line.

Further suppose that the directory server is maintaining these indexes:

Then to add this entry to the directory, the directory server must perform these steps:

  1. Create the common name equality index entry for "Bill" and "Bill Pumice".

  2. Create the appropriate common name approximate index entries for "Bill" and "Bill Pumice".

  3. Create the appropriate common name substring index entries for "Bill" and "Bill Pumice".

  4. Create the surname equality index entry for "Pumice".

  5. Create the appropriate surname approximate index entry for "Pumice".

  6. Create the appropriate surname substring index entries for "Pumice".

  7. Create the telephonenumber equality index entry for "408 555 8834".

  8. Create the appropriate telephonenumber substring index entries for "408 555 8834".

  9. Create the appropriate description substring index entries for "Manufacturing lead for the Z238 line of foobar widgets." A large number of substring entries are generated for this string.
Higher System Resource Use

One other cost to maintaining index files is the increased system resources they require. Although the impact to your system depends on how large a database you use and how many attributes exist within your database, consider these facts:


Creating Indexes
You create presence, equality, approximate, substring, and international indexes for specific attributes by using the "Attribute to be Indexed" parameter. (For instructions on managing browsing indexes, see "Using Browsing Indexes".) Before you create new indexes, balance the benefits of maintaining indexes against the cost of indexing as described in "The Cost of Indexing". Consider that:

System and Default Indexes

When you install the directory server, some indexes are created by default. In addition, some default indexes are system indexes that you cannot remove. All of these indexes help to improve directory server performance.

Directory-enabled Netscape servers also maintain a standard set of indexes in the directory. If you are not using the server for which these indexes are maintained, you can either leave them alone (they do not require any resources if you are not using the corresponding server), or you can delete them from your server. For more information, see "Removing Indexes".

System Indexes

The system indexes, in Table  7.1, cannot be removed.

Table 7.1 System indexes  

Attribute
Eq
Pres
Purpose
aci

X
Allows the directory server to quickly obtain the access control information maintained in the database.
changeNumber
X

Used to improve replication performance.
copiedFrom

X
Used to improve replication performance.
dnComp
X

Used to help accelerate subtree searches in the directory.
objectClass
X

Used to help accelerate subtree searches in the directory.
entryDN
X

Speeds up entry retrieval based on DN searches.
parentID
X

Enhances directory performance during one-level searches.
numSubordinates

X
Used by the Directory Server Console to enhance display performance on the Directory tab.

Default Indexes

Default indexes are created for your server during installation. Although you can remove the default indexes, you should ensure that no server plug-ins or other Netscape servers in your enterprise require the index before you remove it. For information on removing indexes, see "Removing Indexes".

Table  7.2 contains a list of default indexes and their purpose.

Table 7.2 Default indexes  

Attribute
Eq
Pres
Sub
Purpose
cn
X
X
X
Improves the performance of the most common types of user directory searches.
givenName
X
X
X
Improves the performance of the most common types of user directory searches.
mail
X
X
X
Improves the performance of the most common types of user directory searches.
mailAlternateAddress
X


Improves NT Synchronization Service performance, and has no impact on your directory server's performance if you are not using the NT Synchronization Service.
mailHost
X


Used by the Netscape Messaging Server.
member
X


Improves Netscape server performance. This index is also used by the referential integrity plug-in. See "Managing the Referential Integrity Plug-in" for more information.
nsCalXItemId
X
X
X
Used by the Netscape Calendar Server.
nsLIProfileName
X


Used by Netscape Communicator.
ntGroupDomainId
X
X
X
Improves NT Synchronization Service performance, and has no impact on your directory server's performance if you are not using the NT Synchronization Service.
ntUserDomainId
X
X
X
Improves NT Synchronization Service performance, and has no impact on your directory server's performance if you are not using the NT Synchronization Service.
owner
X


Improves Netscape server performance. This index is also used by the referential integrity plug-in. See "Managing the Referential Integrity Plug-in" for more information.
pipstatus
X


Used by the Netscape Calendar Server.
pipuid
X
X
X
Used by the Netscape Calendar Server.
seeAlso
X


Improves Netscape server performance. This index is also used by the referential integrity plug-in. See "Managing the Referential Integrity Plug-in" for more information.
sn
X
X
X
Improves the performance of the most common types of user directory searches.
telephoneNumber
X
X
X
Improves the performance of the most common types of user directory searches.
uid
X


Improves Netscape server performance.
uniquemember
X


Improves Netscape server performance. This index is also used by the referential integrity plug-in. See "Managing the Referential Integrity Plug-in" for more information.

Standard Index Files

Because of the need to maintain default indexes and other internal indexing mechanisms, the directory server maintains several standard index files:

Creating Indexes From the Server Console

During the process of creating an index from the server console, the server is placed in read-only mode. While the server is in read-only mode, you cannot make any configuration changes or modify the contents of the directory.

Follow these steps to create presence, equality, approximate, substring, and international indexes for specific attributes from the Directory Server Console. (For instructions on managing browsing indexes, see "Using Browsing Indexes".)

  1. On the Directory Server Console, select the Configuration tab and then the Database icon.

  2. Select the Indexes tab in the right pane.

    3. The System Indexes table contains indexes that you cannot delete. The Additional Indexes table contains the default indexes and any indexes you create. See "System and Default Indexes" for more information.

  3. If the attribute you want to index is listed in the Additional Indexes table on the tab, skip to Step  4, otherwise, click Add Attribute.

    4. A dialog box appears containing a list of all of the available attributes in the server schema. Select the attribute you want to index and click OK. The server adds the attribute to the Additional Indexes table.

  4. Select the checkbox for each type of index you want maintained for the attribute.

  5. If you want to create an index for a language other than English, enter the OID for the collation order you want to use in the Matching Rules field.

    6. You can index the attribute using multiple languages by listing multiple OIDs separated by commas (but no whitespace). For a list of languages and their associated OIDs, see Table  B.1.

  6. Click Save.

    7. A dialog box appears displaying the status of the index creation and warning you that the server is in read-only mode while the indexes are created. While the server is in read-only mode, you cannot make any configuration changes or modify the contents of the directory. The server will display a dialog box informing you when the indexes are created and the server is no longer in read-only mode.

    The new index is immediately active for any new data that you add and any existing data in your directory. You do not have to restart your server.

Creating Indexes From the Command-Line

Creating indexes for attributes using the command line is a two-step process. First, you need to add the corresponding index description to slapd.ldbm.conf.

If you have existing data in your database that was not previously indexed, you also need to complete one of the following additional tasks, in order for the server to create the indexes you specify in slapd.ldbm.conf:

Adding Index Descriptions to slapd.ldbm.conf

To add presence, equality, approximate, substring, or international index descriptions to slapd.ldbm.conf do the following:

  1. Stop the server. See "Starting and Stopping the Directory Server" for information.

  2. Open slapd.ldbm.conf. For a description of where the configuration files reside in your directory server installation, see "Directory Server Configuration Files".

  3. Add an index parameter of the form:

    4. index <attribute> [<list of indexes>][<list of OIDs>]

    where <attribute> is the attribute to be indexed, <list of indexes> can include pres, eq, sub, and/or approx, and <list of OIDs> is a list of international collation OIDs. If you omit <list of indexes> and instead you specify just <list of OIDs> you must denote the null list of indexes with double quotation marks ("").

    You should always use the attribute's primary name (not the attribute's alias) when you create the index. The attribute's primary name is the first name listed for the attribute in the schema. See the Netscape Directory Server Schema Reference Guide for more information about attribute names.

    For a definition of the index keywords, refer to "Types of Indexes". For a list of available OIDs, see Table  B.1.

    For example:

    index cn,telephoneNumber pres,eq,sub
    index cn "" 2.16.840.1.113730.3.3.2.15.1
    index cn eq,sub 2.16.840.1.113730.3.3.2.15.1

    Specifying an index parameter and attribute name with no indexes causes all indexes (except international) to be maintained for the specified attribute. For example:

    index cn

    causes all indexes to be maintained for the common name attribute. Use the keyword none to specify that no indexes are to be maintained for the attribute. For example:

    index cn none

    Use the keyword default to specify the list of all attributes not otherwise identified in an index parameter. For example:

    index default none

  4. Start the server.

  5. When you finish editing the slapd.ldbm.conf file and start the server, the server will index any new entries. If you have existing data in your database that was not previously indexed, you also need to complete one of the following additional tasks, in order for the server to create the indexes you specify in slapd.ldbm.conf:
Creating Indexes Using db2index

You can create new presence, equality, approximate, substring, and international indexes from the command line using the slapd (Windows NT) or ns-slapd (Unix) command-line utility with the db2index keyword.

For information on where you can find the command-line utilities in your directory server installation, see "Finding the Command-Line Utilities".

Before you can create an index using db2index, you need to add the index description to slapd.ldbm.conf. See  "Adding Index Descriptions to slapd.ldbm.conf" for more information.

To create an index from the command line:

  1. You can run db2index with the server shut down or running.

    2. Stopping the directory server and then running db2index is quicker. If you choose to do this, skip to the next step. However, if you want to continue to read information from the directory during the indexing process, you need to place the server in read-only mode. See  "Placing a Database in Read-Only Mode" for information.

  2. From the command line, change to <NSHOME>/bin/slapd/server, where <NSHOME> is the directory where you installed the directory server.

  3. Run the slapd (Windows NT) or ns-slapd (Unix) command-line utility as follows. Parameters in brackets [ ] are optional.

    4. Windows NT:

    slapd db2index -f <slapd.conf> [-d  <debug_level>] [-n  backend-number] -t  attributeName[:indexTypes[:matchingRules]]

    Unix:

    ns-slapd db2index -f <slapd.conf> [-d  <debug_level>] [-n  backend-number] -t  attributeName[:indexTypes[:matchingRules]]

    where

  4. If you kept the server running during the process, stop the directory server and take the database out of read-only mode.

  5. Start the server.
Parameters Used for Index Creation

The following db2index parameters are used to complete index creation:

-d. Optional. Specifies the debug level to use during index creation. Debug levels are defined in "Log Level".

-f. Specifies the slapd.conf configuration file to use for the index creation process. Use the full path to the slapd.conf file with this argument. For information on where to find directory server configuration files, see "Directory Server Configuration Files".

-n. Reserved for use by the directory server.

-t. Specifies the attribute to be indexed as well as the types of indexes to create and matching rules to apply (if any). If you want to specify a matching rule, you must specify an index type.

db2index Example

Windows NT:

slapd db2index -f  c:\Netscape\SuiteSpot4\slapd-dirserver\config\slapd.conf -t  uid:eq,pres,sub:2.16.840.1.113730.3.3.2.11.1

Unix:

ns-slapd db2index -f /usr/ns-home/slapd-dirserver/config/slapd.conf -t  uid:eq,pres,sub:2.16.840.1.113730.3.3.2.11.1


Removing Indexes
You can delete all presence, equality, approximate, substring, and international indexes that you have created and all default indexes that are not also system indexes. This process is outlined in the following sections:

For instructions on removing a browsing index, see "Using Browsing Indexes".

You cannot delete system indexes. These indexes are listed in Table  7.1. For more information on these indexes, see "System and Default Indexes".

Removing Indexes Using the Server Console

You can remove any indexes you have created, indexes used by other Netscape servers (such as Messaging or Calendar server), and default indexes that are not also system indexes using the Directory Server Console. To remove indexes using the Directory Server Console:

  1. On the Directory Server Console, select the Configuration tab.

  2. Select Database in the navigation tree, and then select the Indexes tab in the right pane.

  3. Remove the index by clearing the checkbox under the type of index you want to remove in the same row as the attribute name.

    4. That is, to remove the presence index for the commonName (cn) attribute, clear the Presence checkbox in the same row as the cn attribute in the left column.

    If you want to remove all indexes maintained for a particular attribute, select the attribute's cell under Attribute Name and click Delete Attribute.

    For example, to remove all commonName indexing, delete cn from the Attribute Name column by selecting the cell and then clicking Delete Attribute.

  4. Click Save.

    5. The index is no longer maintained for the server, although any index files that were created previous to the deletion are not deleted. To eliminate these old index files, export and reimport your database as described in "Managing Databases Using LDIF".

Removing Standard Indexes Using the Command Line

You can remove any indexes you have created, indexes used by other Netscape servers (such as Messaging or Calendar server), and default indexes that are not also system indexes using the command line, or by using ldapdelete. For information on deleting entries using the ldapdelete, see "Deleting Entries Using ldapdelete".

To remove indexes using the command line:

  1. Stop the server. See "Starting and Stopping the Directory Server" for information.

  2. Edit slapd.ldbm.conf to remove the index.

    3. For example, suppose you are maintaining an equality, substring, and presence index for the commonName (cn) attribute. Then slapd.ldbm.conf contains the following line:

    index cn pres,sub,eq

    If you wanted to remove the substring index, then edit the line so that it reads:

    index cn pres,eq

  3. Start your server.

    4. The index is no longer maintained for the server, although any index files that were created previous to the deletion are not deleted. To eliminate these old index files, export and reimport your database as described in "Managing Databases Using LDIF".


Using Browsing Indexes
Browsing indexes can significantly speed up the display of large lists of entries on the Directory Server Console. There are some performance drawbacks to creating any kind of index, so use this feature carefully. See "The Cost of Indexing" for information. You might want to create a browsing index on branch points in your directory that contain a large number of entries, for example, the ou=people branch. This section describes these topics:

Creating Browsing Indexes

You create browsing indexes through the Directory Server Console on the Directory tab. This process will place your database in read-only mode while the server creates the index. While the database is in read-only mode, you cannot add, modify, or delete entries in the directory.

To create a browsing index:

  1. Log in to the Netscape Console using a bind DN with sufficient access permissions to modify the directory, for example, the directory manager.

  2. On the Directory Server Console, select the Directory tab.

  3. Right-click the entry for which you want to create the index in the navigation tree, for example, ou=people, and select Create Browsing Index from the pop-up menu.

    4. The server places the database in read-only mode and creates the index. While the database is in read-only mode, you cannot add, modify, or delete entries in the directory. Once complete, the new index is immediately active for any new data that you might add to your directory. You do not have to restart your server.

Removing Browsing Indexes

You can remove browsing indexes by deleting the index entry either by using ldapdelete or through the Directory Server Console. For information on deleting entries using the command-line, see "Deleting Entries Using ldapdelete".

To remove a browsing index from the server console:

  1. On the Directory Server Console, select the Database tab.

  2. Right-click the entry from which you want to delete the index in the navigation tree, for example, ou=people, and select Delete Browsing Index from the pop-up menu.

  3. The index is no longer maintained for the server, although any index files that were created previous to the deletion are not deleted. To eliminate these old index files, export and reimport your database as described in "Managing Databases Using LDIF".

Managing All IDs Threshold
Each index that the directory server uses is composed of a table of index keys and matching entry ID lists. That is, for each index key there is a list of directory entry IDs that match the key. This entry ID list is used by the directory server to build a list of candidate entries that may match a specified search filter (see "The Searching Algorithm" for details).

For each entry ID list there is a size limit. This size limit is globally applied to every index key managed by the server, and it is called the All IDs threshold. When the size of an individual ID list reaches this boundary, the server replaces that ID list with an All IDs token.

The All IDs token causes the server to assume that all directory entries match the index key. In effect, the All IDs token causes the server to behave as if no index was available for that type of search. The assumption is that some other aspect of the search filter will allow the server to narrow its candidate list before processing the list.

The following sections examine the benefits and drawbacks of the All IDs mechanism. They also give advice for the tuning of the All IDs threshold.

Benefits of the All IDs Mechanism

The All IDs mechanism is an important mechanism for improving search performance in those cases where the search results would be most if not all directory entries (for example, searches such as cn=*). By assuming All IDs, the directory server

Drawbacks of the All IDs Mechanism

Performance problems can occur if the All IDs threshold is set either too low (this is the most common problem) or too high for your directory's size.

When All IDs Threshold is Too Low

If the All IDs threshold is set too low, then too many index keys will be set to All IDs. This can result in too many directory searches examining every entry in your directory. Since this scenario is most likely to occur when your directory grows to be very large (millions of entries), the performance hit on searches can be excessive.

For example, suppose you are managing an equality index on the common name attribute. One of the index keys stored in your cn index might therefore be cn=Pete. The corresponding entry ID list would then contain the ID number of every entry containing a common name attribute that is set to Pete.

This is a very useful index to maintain because only a small fraction of the entries in your directory will include cn=Pete. Performance for searches that use a cn=Pete filter will be vastly improved because only that small fraction of entry IDs needs to be examined when servicing the search request.

However, over time your directory may continue to grow. As it does, more and more Petes may be added, but at the same relatively small proportion of total directory entries. Eventually, the cn=Pete entry ID list can become quite large, but it will still be a list that is necessary for search performance. If your directory grows large enough that so many cn=Pete entries are added that the All IDs threshold is met, then the cn=Pete entry ID list is replaced with an All IDs token. Suddenly, every time you search for cn=Pete, the directory server will examine every single entry in the directory in response to the search request.

Of course, for a large enough database size, the All IDs token will be set for a large percentage of all index keys and your search performance will severely degrade.

When All IDs Threshold is Too High

Setting the All IDs threshold to be too high can also cause performance problems. An excessively high All IDs threshold results in large entry ID lists that must be maintained and loaded into memory when servicing search requests. An excessively high All IDs threshold can eliminate all of the benefits of the All IDs mechanism (see "Benefits of the All IDs Mechanism" for details).

All IDs Threshold Tuning Advice

Be careful when changing the default All IDs threshold value for your server. If you change the threshold to an inappropriate value, you can hurt your server performance far more than help it.

If your directory is reasonably stable in size, set the All IDs threshold to approximately 5 percent of the total number of entries stored in your directory. That is, if you have 1,000,000 entries in your directory, set the All IDs threshold to 50,000.

If, however, you are planning to add large numbers of entries to your directory in the near future, you probably need to give more thought to your All IDs threshold value. Consider the following:

The trick, therefore, is balancing your current directory needs against future expansion plans while trying to avoid changing the All IDs threshold somewhere down the road (which requires a database rebuild).

Suppose, for example, that your current directory is 50,000 entries in size. However, in the next few years you expect to grow your directory to 1,000,000 entries. If you set your All IDs threshold to 5 percent of 50,000 (2500), then when your directory grows to 1,000,000 entries you will have a performance problem because 2500 is far too low for a 1,000,000 entry database (the lower bound for a 1,000,000 entry database is .5 percent of 1,000,000, or 5000).

In this situation, you can do one of the following:

Which strategy you should choose depends on the situation at your site. You should consider the cost of rebuilding your database (and all associated consumer servers) versus potential performance impacts as your All IDs threshold value moves away from the ideal setting of 5 percent. While these impacts should be slight, they may be noticeable for extremely loaded directories. Also consider how quickly your directory will grow and how long you it will take you to increase your directory size. If it will take years to grow, then it may be worth planning on a database rebuild. If, however, it will take only months to increase your directory size by an order of magnitude or greater, then consider ways to set the All IDs threshold such that you minimize the time between database rebuilds.

Default All IDs Threshold Value

By default, the directory server is set to an All IDs threshold of 4000. This value is ideal for a directory size of 80,000 entries. According to the advice given in the previous section, this default value is acceptable for directories between 8000 and 800,000 entries in size. If your anticipated directory size falls outside this range, change your All IDs threshold before populating your database.

Symptoms of an Inappropriate All IDs Threshold Value

When your All IDs threshold is set incorrectly, you will see poor search performance. However, poor search performance can be caused by other factors. For example:

Carefully examine these possibilities first before changing your All IDs threshold value.

If you think that your server is suffering from an All IDs threshold that is too low, look in your access log. (See  "Access Log" for more information.) Any searches that resulted in an All IDs situation will contain the notes=U flag. notes=U will be returned for

To determine whether the search result belonged to a search that should have been indexed, you have to match the conn and op values from the RESULT line to a previous SRCH line in your access log file. The SRCH line will show the search filter that was used for the search request. If you have an index for the specified search filter, then the notes=U flag is caused by an All IDs flag that has been set for the index key.

[24/July/1998:15:12:20 -0800] conn=2 op=1 SRCH base="o=airius.com" scope=0 filter="(cn=pete)"

[24/July/1998:15:12:20 -0800] conn=2 op=1 RESULT err=0 tag=101 nentries=10000 notes=U

Changing the All IDs Threshold Value

To change the All IDs threshold value for your server:

  1. Shut down your directory server.

  2. Export your directory database to LDIF using the command line. For more information see "Exporting to LDIF From the Command Line".

  3. Edit <NSHOME>/slapd-<server ID>/config/slapd.ldbm.conf with the text editor of your choice.

  4. If the allidsthreshold parameter is in the file, change it to the desired setting. If allidsthreshold is not in the file, add it to the file anywhere after the line beginning with plugin database. For more information on the allidsthreshold parameter, see "All IDs Threshold".

  5. Create your database using the command-line. See "Importing LDIF From the Command Line" for information.

  6. If you are increasing your All IDs threshold value, examine your database cache size.

    7. Increasing your All IDs threshold can result in larger memory requirements due to larger entry ID lists. The increase in memory requirement will differ depending on the number and types of indexes that you are maintaining, but it will never be larger than the factor by which you increased allidsthreshold. That is, if you double allidsthreshold then you should not have to increase your database cache size to more than double its current value.

    Increasing your database cache size by the same factor as you increased allidsthreshold is an extreme measure. If you have the physical memory available, try increasing your database cache size by a factor that is 25 percent of your allidsthreshold increase (that is, if you doubled allidsthreshold, then increase your database cache size by 50 percent). Then, if necessary, slowly increase your cache size until you are satisfied with your server's performance.

    You set your database cache size using the dbcachesize slapd.ldbm.conf parameter. For more information, see "Maximum Cache Size".

  7. Restart your directory server.
 

© Copyright 1999 Netscape Communications Corporation, a subsidiary of America Online, Inc. All Rights Reserved.