Previous Contents Index Next |
iPlanet Directory Server 5.1 Administrator's Guide |
Chapter 10 Managing Indexes
The iPlanet Directory Server Deployment Guide guide introduced the concept of indexing, the costs and benefits and the different types of index shipped with iPlanet Directory Server. This chapter begins with a description of the searching algorithm itself, so as to place the indexing mechanism in context, and then describes how to create, delete and manage indexes. This chapter contains the following sections:
About Indexes
About Indexes
This section provides an overview of indexing in Directory Server. It contains the following topics:
"About Index Types"
"About Default, System, and Standard Indexes"
About Index Types
Indexes are stored in files in the directory's databases. The names of the files are based on the indexed attribute, not the type of index contained in the file. Each index file may contain multiple types of indexes if multiple indexes are maintained for the specific attribute. For example, all indexes maintained for the common name attribute are contained in the cn.db3 file.Directory Server supports the following types of index:
Presence index (pres)
The substring index is a costly index to maintain, but it allows efficient searching against substrings within entries.
Equality index (eq)
- The presence index contains a list of the entries that contain a particular attribute. This index is useful if, for example, you want to examine any entries that contain access control information. Generating an aci.db3 file that includes a presence index lets you efficiently perform the search for ACI=* to generate the Access Control List for the server.
- The presence index is not used for base object searches.
Approximate index (approx)
- The equality index allows you to search efficiently for entries containing a specific attribute value. For example, an equality index on the cn attribute allows a user to perform the search for cn=Babs Jensen far more efficiently.
Substring index (sub)
- The approximate index allows efficient approximate or "sounds-like" searches. For example, an entry may include the attribute value cn=Robert E Lee. An approximate search would return this value for searches against cn~=Robert Lee, cn~=Robert, or cn~=Lee. Similarly, a search against l~=San Fransisco (note the misspelling) would return entries including l=San Francisco.
For example, searches of the form:
would match the common names containing strings such as
Bill Anderson
Jill Anderson
Steve Sandersonwould return all the entries in your directory with telephone numbers that contain 555.
Note Substring indexes are limited to a minimum of two characters for each entry.
International index
Browsing (virtual list view) index
- The international index speeds up searches for information in international directories. The process for creating an international index is similar to the process for creating regular indexes, except that you apply a matching rule by associating a locale (OID) with the attributes to be indexed.
- For a listing of supported locales and their associated OIDs, refer to Appendix D, "Internationalization." If you want to configure the directory server to accept additional matching rules, contact iPlanet Professional Services.
- The browsing index, or virtual list view index, speeds up the display of entries in the Directory Server Console. This index is particularly useful if you branch of your directory contains hundreds of entries, for example, the ou=people branch. You can create a browsing index on any branchpoint in the directory tree to improve display performance. You do this through the Directory Server Console or by using the vlvindex command (directoryserver vlvindex on Solaris 9 platforms).
About Default, System, and Standard Indexes
When you install Directory Server a set of default and system indexes is created per database instance. To maintain these indexes, the directory uses standard indexes.
Overview of Default Indexes
The default indexes can be modified depending on your indexing needs, although you should ensure that no server plug-ins or other servers in your enterprise depend on this index before you remove it.The following tables lists the default indexes installed with the directory:
Table 10-1    Default indexes
Attribute
Eq
Pres
Sub
Purpose
Improves the performance of the most common types of user directory searches.
Improves the performance of the most common types of user directory searches.
Improves the performance of the most common types of user directory searches.
Improves iPlanet server performance. This index is also used by the referential integrity plug-in. See "Maintaining Referential Integrity," on page 70 for more information.
Improves iPlanet server performance. This index is also used by the referential integrity plug-in. See iPlanet Directory Server Administrator's Guide for more information.
Improves iPlanet server performance. This index is also used by the referential integrity plug-in. See "Maintaining Referential Integrity," on page 70 for more information.
Improves the performance of the most common types of user directory searches.
Improves the performance of the most common types of user directory searches.
Improves iPlanet server performance. This index is also used by the referential integrity plug-in. See "Maintaining Referential Integrity," on page 70 for more information.
Overview of System Indexes
System indexes are indexes that cannot be deleted or modified. They are required by the directory to function properly. The following table lists the system indexes included with the directory:
Overview of Standard Indexes
Because of the need to maintain default indexes and other internal indexing mechanisms, the Directory Server also maintains certain standard index files. The following standard indexes exist by default. You do not need to generate them:
id2entry.db3 - Contains the actual directory database entries. All other database files can be recreated from this one.
id2children.db3 - Restricts the scope of one-level searches, that is, searches that examine an entry's immediate children.
dn.db3 - Controls the scope of subtree searches; that is, searches that examine an entry and all the entries in the subtree beneath it.
dn2id.db3 - Begins all searches efficiently by mapping an entry's distinguished name to its ID number.
Overview of the Searching Algorithm
Indexes are used to speed up searches. To understand how the directory uses indexes, it helps to understand the searching algorithm. Each index contains a list of attributes (such as the cn, common name, attribute) and a pointer to the entries corresponding to each value. Directory Server processes a search request as follows:
An LDAP client application such as Netscape Communicator or Directory Server Console sends a search request to the directory.
See iPlanet Directory Server Configuration, Command, and File Reference for further information about these attributes.The directory examines the incoming request to make sure that the specified base DN matches a suffix contained by one or more of its databases or database links.
If they do match, the directory processes the request.
If the search request for each database attribute can be satisfied by a single index, then the server reads that index to generate a list of potential matches.If they do not match, the directory returns an error to the client indicating that the suffix does not match. If a referral has been specified in the nsslapd-referral attribute under cn=config, the directory also returns the LDAP URL where the client can attempt to pursue the request.
If there is an index for the attribute, the directory takes the candidate matches from the index files in the form of a series of entry ID numbers.
- If there is no index for the attribute, the directory generates a candidate list that includes all entries in the database which makes the search considerably slower. (The directory will also do this if the All IDs token is set for the index key that the server is using. For information on All IDs, see "Managing Indexes".)
- If a search request contains multiple attributes, the directory consults multiple indexes and then combines the resulting lists of candidate entries.
The directory uses the returned entry ID numbers to read the corresponding entries from the id2entry.db3 file. The directory server then examines each of the candidate entries to see if any match the search criteria. The directory returns matching entries to the client as each is found.
- The directory continues until it has either examined all candidate entries, or until it reaches the limit set in one of the following attributes:
nsSizeLimit which specifies the maximum number of entries to return from a search operation. If this limit is reached, the directory returns any entries it has located that match the search request, as well as an exceeded size limit error.
nsTimeLimit which specifies the maximum number of seconds allocated for a search request. If this limit is reached, the directory returns any entries it has located that match the search request, as well as an exceeded time limit error
nsLookthroughLimit which specifies the maximum number of entries that the directory will check when examining candidate entries in response to a search request.
In addition, the directory uses a variation of the metaphone phonetic algorithm to perform searches on an approximate index. Each value is treated as a sequence of words, and a phonetic code is generated for each word.
Note The metaphone phonetic algorithm in Directory Server 5.0 supports only US-ASCII letters. Therefore, use approximate indexing only with English values.
Values entered on an approximate search are similarly translated into a sequence of phonetic codes. An entry is considered to match a query if both of the following are true:
All of the query string codes match the codes generated in the entry string.
For example, the following table shows how several queries would match with the entry name Alice B. Sarette whose phonetic code is ALS B SRT.All of the query string codes are in the same order as the entry string codes.
Balancing the Benefits of Indexing
Before you create new indexes, balance the benefits of maintaining indexes against the costs. Keep in mind that:
Approximate indexes are not efficient for attributes commonly containing numbers, such as telephone numbers.
The following example illustrates exactly how time consuming indexes can become. Consider the procedure for creating a specific attribute:Substring indexes do not work for binary attributes. Equality indexes should be avoided if the value is big (such as attributes intended to contain photographs or passwords containing encrypted data).
Maintaining indexes for attributes not commonly used in a search increase overhead without improving global searching performance.
Attributes that are not indexed can still be specified in search requests, although the search performance may be degraded significantly, depending on the type of search.
Keep in mind that the more indexes you maintain, the more disk space you will require.
The directory server receives an add or modify operation.
For example, suppose the directory server is asked to add the entryThe directory server examines the indexing attributes to determine whether an index is maintained for the attribute values.
If the created attribute values are indexed, then the directory server generates the new index entries.
Once the server completes the indexing, the actual attribute values are created according to the client request.
dn: cn=Bill Pumice, ou=People, o=siroe.com
objectclass: top
objectClass: person
objectClass: orgperson
objectClass: inetorgperson
cn: Bill Pumice
cn: Bill
sn: Pumice
ou: Manufacturing
ou: people
telephonenumber: 408 555 8834
description: Manufacturing lead for the Z238 line.Further suppose that the directory server is maintaining the following indexes:
Equality, approximate, and substring indexes for common name and surname attributes
Then to add this entry to the directory, the directory server must perform these steps:Equality and substring indexes for the telephone number attribute
Create the common name equality index entry for "Bill" and "Bill Pumice".
The example shows that indexing can be costly.Create the appropriate common name approximate index entries for "Bill" and "Bill Pumice".
Create the appropriate common name substring index entries for "Bill" and "Bill Pumice".
Create the surname equality index entry for "Pumice".
Create the appropriate surname approximate index entry for "Pumice".
Create the appropriate surname substring index entries for "Pumice".
Create the telephonenumber equality index entry for "408 555 8834".
Create the appropriate telephonenumber substring index entries for "408 555 8834".
Create the appropriate description substring index entries for "Manufacturing lead for the Z238 line of widgets." A large number of substring entries are generated for this string.
Creating Indexes
This section describes how to create presence, equality, approximate, substring and international indexes for specific attributes using the Directory Server Console and the command line. It also describes the separate procedures for creating browsing indexes.
This section contains the following procedures:
Creating Indexes From the Server Console
Creating Indexes From the Command Line
Creating Indexes From the Server Console
Using the Directory Server Console you can create presence, equality, approximate, substring, and international indexes for specific attributes.
On the Directory Server Console, select the Configuration tab.
The new index is immediately active for any new data that you add and any existing data in your directory. You do not have to restart your server.Expand the Data node, then expand the suffix of the database you want to index and select the database.
Select the Indexes tab in the right pane.
Note Do not click on the Database Settings node because this will take you to the Default Index Settings window and not the window for configuring indexes per database.
If the attribute you want to index is listed in the Additional Indexes table, skip to Step 6. Otherwise, click Add Attribute.
Select the attribute you want to index and click OK.
- A dialog box appears containing a list of all of the available attributes in the server schema.
Select the checkbox for each type of index you want to maintain for each attribute.
- The server adds the attribute to the Additional Indexes table.
If you want to create an index for a language other than English, enter the OID of the collation order you want to use in the Matching Rules field.
Click Save.
- You can index the attribute using multiple languages by listing multiple OIDs separated by commas (but no whitespace). For a list of languages, their associated OIDs and further information regarding collation orders see Appendix D, "Internationalization."
- The Indexes dialog box appears displaying the status of the index creation and informing you when the indexes have been created. You can click on the Status Logs box to view the status of the indexes created. Once the indexing is complete, click Close to close the Indexes dialog box.
Creating Indexes From the Command Line
You can create presence, equality, approximate, substring and international indexes for specific attributes from the command line.Creating indexes from the command line involves two steps:
Using the ldapmodify command-line utility to add a new index entry or edit an existing index entry.
The following sections describe the steps involved in creating indexes.Running the db2index.pl perl script (directoryserver db2index-task on the Solaris 9 platform) to generate the new set of indexes to be maintained by the server.
Note You cannot create new system indexes because system indexes are hard coded in Directory Server.
Adding an Index Entry
Use ldapmodify to add the new index attributes to your directory. If you want to create a new index that will become one of the default indexes, add the new index attributes to the cn=default indexes,cn=config,cn=ldbm database, cn=plugins,cn=config entry.To create a new index for a particular database, add it to the cn=index,cn=instanceName,cn=ldbm database,cn=plugins,cn=config entry, where cn=instanceName corresponds to the name of the database.
For information on the LDIF update statements required to add entries see "LDIF Update Statements," on page 60.
For example, you want to create presence, equality and substring indexes for the sn (surname) attribute in the Siroe1 database.
First, change to the following default directory containing the utility:
Solaris 9 platform
# cd /usr/iplanet/ds5/shared/bin Other platforms
# cd /usr/iplanet/servers/shared/bin Run the ldapmodify command-line utility as follows:
ldapmodify -a -h server.siroe.com -p 389 \
-D "cn=Directory Manager" -w passwordThe ldapmodify utility binds to the server and prepares it to add an entry to the configuration file. For more information about the ldapmodify command-line utility, refer to the iPlanet Directory Server Configuration, Command, and File Reference.
Next, you add the following entry for the new indexes:
dn: cn=sn,cn=index,cn=Siroe1,cn=ldbm database,cn=plugins,cn=config
objectClass:top
objectClass:nsIndex
cn:sn
nsSystemIndex:false
nsIndexType:pres
nsIndexType:eq
nsIndexType:sub
nsMatchingRule:2.16.840.1.113730.3.3.2.3.1The cn attribute contains the name of the attribute you want to index, in this example the sn attribute. The entry is a member of the nsIndex object class. The nsSystemIndex attribute is false, indicating that the index is not essential to Directory Server operations. The multi-valued nsIndexType attribute specifies the presence (pres), equality (eq) and substring (sub) indexes. Note that each keyword has to be entered on a separate line. The nsMatchingRule attribute specifies the OID of the Bulgarian collation order.
Specifying an index entry with no value in the nsIndexType attribute results in all indexes (except international) being maintained for the specified attribute. For example, suppose instead that you specify the following entry for your new sn indexes:
dn: cn=sn,cn=index,cn=instance,cn=ldbm database,cn=plugins,cn=config
objectClass:top
objectClass:nsIndex
cn:sn
nsSystemIndex:false
nsIndexType:This new entry results in all indexes for the sn (surname) attribute.
You can use the keyword none in the nsIndexType attribute to specify that no indexes are to be maintained for the attribute. For example, suppose you want to temporarily disable the sn indexes you just created on the Siroe1 database, change the nsIndexType to none as follows:
dn: cn=sn,cn=index,cn=Siroe1,cn=ldbm database,cn=plugins,cn=config
objectClass:top
objectClass:nsIndex
cn:sn
nsSystemIndex:false
nsIndexType:noneFor a complete list of collation orders and their OIDs, refer to Appendix D, "Internationalization."
For more information about the index configuration attributes, see the iPlanet Directory Server Configuration, Command, and File Reference.
Note You should always use the attribute's primary name (not the attribute's alias) when creating indexes. The primary name of the attribute is the first name listed for the attribute in the schema, for example uid for the userid attribute. See Table 10-6 for a list of all primary and alias attribute names.
Running the db2index.pl Script
Once you have created an indexing entry or added additional index types to an existing indexing entry, run the db2index.pl script (directoryserver db2index-task on the Solaris 9 platform) to generate the new set of indexes to be maintained by the Directory Server. Once you run the script, the new set of indexes is active for any new data you add to your directory and any existing data in your directory.The command for this script is platform-dependent:
Solaris 9 platform
# /usr/sbin/directoryserver db2index-task Windows platforms
installDir> bin\slapd\admin\bin\perl slapd-serverID\db2index.pl Other platforms
# installDir/slapd-serverID/db2index.pl For more information about using this perl script, refer to iPlanet Directory Server Configuration, Command, and File Reference.
The following examples create an index using the db2index.pl script.
iPlanet\Servers\bin\slapd\admin\bin\perl.exe
iPlanet\Servers\slapd-siroe\db2index.pl
-D "cn=Directory Manager" -w password -n Database1 -t sn# use 'directoryserver db2index-task' on Solaris 9 plaforms
db2index.pl \
-D "cn=Directory Manager" -w password -n Database1 -t snThe following table describes the db2index.pl options used in the examples:
For more information about the db2index.pl perl script, see the iPlanet Directory Server Configuration, Command, and File Reference.
Creating Browsing Indexes From the Server Console
To create a browsing index using the Directory Server Console:
On the Directory Server Console, select the Directory tab.
Select the entry for which you want to create the index in the left navigation tree (for example, People) and select Create Browsing Index from the Object menu.
The Create Browsing Index dialog box appears displaying the status of the index creation. You can click on the Status Logs box to view the status of the indexes created.
- You can also select and right-click the entry for which you want to create the index in the navigation tree and choose Create Browsing Index from the pop-up menu.
Click Close to close the Create Browsing Index dialog box.
- The new index is immediately active for any new data that you add to your directory. You do not have to restart your server.
Creating Browsing Indexes from the Command Line
Creating a browsing index, or virtual list view (VLV) index from the command line involves two steps:
Using the ldapmodify to add new browsing index entries or edit existing browsing index entries.
The following sections describe the steps involved in creating browsing indexes.Running the vlvindex (directoryserver vlvindex on Solaris 9 platforms) script to generate the new set of browsing indexes to be maintained by the server.
Adding a Browsing Index Entry
The type of browsing index entry we want to create depends on the type of ldapsearch attribute sorting we want to accelerate. It is important to take the following items into account:
The scope of the search (base, one, sub). For more information on the ldapsearch -s option which allows you to specify the scope of searches, see the iPlanet Directory Server Configuration, Command, and File Reference.
For example, you want to create a browsing index to accelerate an ldapsearch on the entry "dc=siroe,dc=com" held in the Siroe1 database where the search base is "dc=siroe,dc=com", the search filter is (|(objectclass=*)(objectclass=ldapsubentry)), the scope is one and the sorting order for the returned attributes is cn, givenname, o, ou, and sn.The base of the search (the entry you want to use as a starting point for the search). For more information on the ldapsearch -b option which allows you to specify the base of searches, see the iPlanet Directory Server Configuration, Command, and File Reference.
The attributes you want to sort,
The filter of the search. For more information on specifying filters for searches see Appendix B, "Finding Directory Entries".
The ldbm database to which the entry that forms the base of the search belongs.
Note You can only create browsing indexes in ldbm databases.
First, change to the following default directory containing the utility:
Solaris 9 platform
# cd /usr/iplanet/ds5/shared/bin Other platforms
# cd /usr/iplanet/servers/shared/bin Run the ldapmodify command-line utility as follows:
ldapmodify -a -h server -p 389 -D "cn=directory manager" -w password
The ldapmodify utility binds to the server and prepares it to add an entry to the configuration file.
Next, you need to add two browsing index entries which define your browsing index.
The first entry you add specifies the base, scope and filter of the browsing index:
dn: cn="dc=siroe,dc=com",cn=Siroe1,cn=ldbm database,cn=plugins,cn=config
objectClass:top
objectClass:vlvSearch
cn:"dc=siroe,dc=com"
vlvbase:"dc=siroe,dc=com"
vlvscope:one
vlvfilter:(|(objectclass=*)(objectclass=ldapsubentry))The cn contains the browsing index identifier which specifies the entry on which you want to create the browsing index, in this example the "dc=siroe,dc=com" entry. We recommend you use the dn of the entry for your browsing index identifier, which is, the approach adopted by the Directory Server Console, to prevent identical browsing indexes from being created. The entry is a member of the vlvSearch object class. The vlvbase attribute value specifies the entry on which you want to create the browsing index, in this example the "dc=siroe,dc=com" entry (that is the browsing index identifier). The vlvscope attribute is one, indicating that the base for the search you want to accelerate is one. A search base of one means that only the immediate children of the entry specified in the cn attribute, and not the entry itself, will be searched. The vlvfilter specifies the filter to be used for the search, in this example (|(objectclass=*)(objectclass=ldapsubentry)).
The second entry specifies the sorting order you want for the returned attributes:
dn:cn=sort_cn_givenname_o_ou_sn,cn="dc=siroe,dc=com",cn=Siroe1,
cn=ldbm database,cn=plugins,cn=config
objectClass:top
objectClass:vlvIndex
cn:cn=sort_cn_givenname_o_ou_sn
vlvsort:cn givenname o ou snThe cn contains the browsing index sort identifier. We recommend you use a sort identifier which clearly identifies the search sorting order for the browsing index you create, such as the explicit sort identifier cn=sort_cn_givenname_o_ou_sn in this example. The entry is a member of the vlvIndex object class. The vlvsort attribute value specifies the order in which you want your attributes to be sorted, in this example cn, givenname, o, ou and then sn.
Note This first browsing index entry must be added to the cn=instanceName,cn=ldbm database,cn=plugins,cn=config directory node, and the second entry must be a child of the first entry.
Running the vlvindex Command
Once you have created the two browsing indexing entries or added additional attribute types to an existing indexing browsing entries, run the vlvindex command (directoryserver vlvindex on Solaris 9 platforms) to generate the new set of browsing indexes to be maintained by the Directory Server. Once you run the script, the new set of browsing indexes is active for any new data you add to your directory and any existing data in your directory.To generate browsing indexes, use the following command:
Solaris 9 platform
# /usr/sbin/directoryserver vlvindex Other platforms
# installDir/slapd-serverID/vlvindex For more information about using this script, refer to iPlanet Directory Server Configuration, Command, and File Reference.
The following example generates browsing indexes using the vlvindex command:
# vlvindex -n Database1 -T "dc=siroe,dc=com"
For more information about the vlvindex command, see the iPlanet Directory Server Configuration, Command, and File Reference.
Deleting Indexes
This section describes how to delete presence, equality, approximate, substring, international and browsing indexes for specific attributes.
As the procedure for deleting browsing indexes is different, it is covered in a separate section. This section contains the following procedures:
Deleting Indexes From the Server Console
Deleting Indexes From the Command Line
Deleting Browsing Indexes From the Server Console
Deleting Browsing Indexes From the Command Line
You must not delete system indexes as deleting them can significantly affect Directory Server performance. System indexes are located in the cn=index,cn=instance,cn=ldbm database,cn=plugins,cn=config entry and the cn=default indexes,cn=config,cn=ldbm database,cn=plugins,cn=config entry. Take care when deleting default indexes as this can also affect how Directory Server works.
For further information on system and default indexes see the iPlanet Directory Server Deployment Guide.
Deleting Indexes From the Server Console
Using the Directory Server Console you can delete indexes you have created, indexes used by other iPlanet servers (such as Messaging or Calendar server), and default indexes. You cannot delete system indexes.To delete indexes using the Directory Server Console:
On the Directory Server Console, select the Configuration tab.
Expand the Data node and expand the suffix associated with the database containing the index. Select the database from which you want to delete the index.
Locate the attribute containing the index you want to delete. Clear the checkbox under the index.
Click Save.
- If you want to delete all indexes maintained for a particular attribute, select the attribute's cell under Attribute Name and click Delete Attribute.
The Delete Browsing Index dialog box appears displaying the status of the index deletion. You can click on the Status Logs button to view the status of the indexes deleted. Once the indexing is complete, click on Close to close the Delete Browsing Index box.
- A Delete Index warning dialog box appears asking you to confirm that you want to delete the index. Click Yes to delete the index.
Deleting Indexes From the Command Line
You can delete indexes using the ldapdelete command-line utility as follows:
Delete an entire index entry or delete unwanted index types from an existing index entry using the ldapdelete command-line utility.
The following sections describe the steps involved in deleting an index.Regenerate the remaining indexes to be maintained by the server using the db2index.pl script (directoryserver db2index-task on Solaris 9 platforms).
Deleting an Index Entry
Use the ldapdelete command-line utility to delete either the entire indexing entry or the unwanted index types from an existing entry.If you want to delete the indexes for a particular database, you remove your index entry from the cn=index,cn=instanceName,cn=ldbm database,cn=plugins,cn=config entry, where cn=instanceName corresponds to the name of the database.
To delete a default index, remove it from the cn=default indexes,cn=config,cn=ldbm database,cn=plugins,cn=config entry.
For example, you want to delete presence, equality and substring indexes for the sn attribute on the database named Siroe1.
You want to delete the following entry:
dn: cn=sn,cn=index,cn=Siroe1,cn=ldbm database,cn=plugins,cn=config
objectClass:top
objectClass:nsIndex
cn:sn
nsSystemIndex:false
nsIndexType:pres
nsIndexType:eq
nsIndexType:sub
nsMatchingRule:2.16.840.1.113730.3.3.2.3.1Run the ldapdelete command-line utility from the following default directory:
Solaris 9 platform
# cd /usr/iplanet/ds5/shared/bin Other platforms
# cd /usr/iplanet/servers/shared/bin Perform the ldapdelete as follows:
ldapdelete -h server.siroe.com -p 389 \
-D "cn=Directory Manager" -w password \
"cn=sn,cn=index,cn=Siroe1,dn=ldbm database,cn=plugins,dn=config"For full information on ldapdelete options, refer to the iPlanet Directory Server Configuration, Command, and File Reference.
Once you have deleted this entry, the presence, equality, and substring indexes for the sn attribute will no longer be maintained by the Siroe1 database.
Regenerating the Remaining Indexes
Once you have deleted an indexing entry or deleted some of the index types from an indexing entry, you need to regenerate the set of remaining indexes to be maintained by the Directory Server.To regenerate your indexes, follow the procedure described in "Running the db2index.pl Script". Once you run the script, the new set of indexes is active for any new data you add to your directory and any existing data in your directory.
For more information about the db2index.pl perl script (directoryserver db2index-task on Solaris 9 platforms), see the iPlanet Directory Server Configuration, Command, and File Reference.
Deleting Browsing Indexes From the Server Console
To delete a browsing index using the Directory Server Console:
On the Directory Server Console, select the Database tab.
Select the entry from which you want to delete the index in the navigation tree, for example, People, and select Delete Browsing Index from the Object menu.You can also select and right-click the entry for which you want to create the index in the navigation tree and choose Delete Browsing Index from the pop-up menu.
A Delete Browsing Index dialog box appears asking you to confirm that you want to delete the index. Click Yes to delete.
The Delete Browsing Index dialog box appears displaying the status of the index deletion.
Deleting Browsing Indexes From the Command Line
Deleting a browsing index, or virtual list view (VLV) index, from the command line involves two steps:
Using the ldapdelete to delete browsing index entries or edit existing browsing index entries.
The following sections describe the steps involved in deleting browsing indexes.Regenerate the remaining indexes by running the vlvindex command (directoryserver vlvindex on Solaris 9 platforms).
Deleting a Browsing Index Entry
Use the ldapdelete command-line utility to either delete browsing indexing entries or edit existing browsing index entries.To delete browsing indexes for a particular database, you remove your browsing index entries from the cn=index,cn=instanceName,cn=ldbm database,cn=plugins,cn=config entry, where cn=instanceName corresponds to the name of the database.
For example, you want to delete a browsing index for accelerating ldapsearch operations on the entry dc=siroe,dc=com held in the Siroe1 database where the search base is dc=siroe,dc=com then the search filter is (|(objectclass=*)(objectclass=ldapsubentry)), the scope is one and the sorting order for the returned attributes is cn, givenname, o, ou, and sn.
To delete this browsing index you need to delete the two corresponding browsing index entries which follow:
dn: cn="dc=siroe,dc=com",cn=Siroe1,cn=ldbm database,cn=plugins,cn=config
objectClass:top
objectClass:vlvSearch
cn:"dc=siroe,dc=com"
vlvbase:"dc=siroe,dc=com
vlvscope:one
vlvfilter:(|(objectclass=*)(objectclass=ldapsubentry))dn:cn=sort_cn_givenname_o_ou_sn,cn="dc=siroe,dc=com",cn=Siroe1,
cn=ldbm database,cn=plugins,cn=config
objectClass:top
objectClass:vlvIndex
cn:cn=sort_cn_givenname_o_ou_sn
vlvsort:cn givenname o ou snRun the ldapdelete command-line utility from the following default directory:
Solaris 9 platform
# cd /usr/iplanet/ds5/shared/bin Other platforms
# cd /usr/iplanet/servers/shared/bin Perform the ldapdelete as follows:
ldapdelete -h siroe.server.com -p 389 -D "cn=Directory Manager" -w password \
"cn="dc=siroe,dc=com",cn=Siroe1,cn=ldbm database,cn=plugins,cn=config" \
"cn=sort_cn_givenname_o_ou_sn,cn="dc=siroe,dc=com",cn=Siroe1, \
cn=ldbm database,cn=plugins,cn=config"For full information on ldapdelete options, refer to the iPlanet Directory Server Configuration, Command, and File Reference.
Once you have deleted these two browsing index entries, the browsing index for accelerating ldapsearch operations on the entry dc=siroe,dc=com held in the Siroe1 database where the search base is dc=siroe,dc=com the search filter is (|(objectclass=*)(objectclass=ldapsubentry)), the scope is one and the sorting order for the returned attributes is cn, givenname, o, ou, and sn. will no longer be maintained by the Siroe1 database
Regenerating the Remaining Indexes
Once you have deleted browsing indexing entries or deleted unwanted attribute types from existing browsing indexing entries, you need to regenerate the set of remaining indexes to be maintained by the Directory Server.To regenerate your browsing indexes, follow the procedure described in "Running the vlvindex Command". Once you run the command, the new set of indexes is active for any new data you add to your directory and any existing data in your directory.
For more information about the vlvindex command (directoryserver vlvindex on Solaris 9 platforms), see the iPlanet Directory Server Configuration, Command, and File Reference.
Managing Indexes
Each index that the directory uses is composed of a table of index keys and matching entry ID lists. This entry ID list is used by the directory to build a list of candidate entries that may match a client application's search request (see "About Indexes" for details).For each entry ID list there is a size limit as specified in the nsslapd-allidsthreshold attribute. This size limit is globally applied to all index keys managed by the server and is logically called All IDs Threshold. When the size of an individual ID list reaches this limit, the server replaces that entry ID list with an All IDs token.
The All IDs token causes the server to assume that all directory entries match the index key. In effect, the All IDs token causes the server to behave as if no index was available for that type of search. The directory assumes that some other aspect of the search request will allow the server to narrow its candidate list before processing the request.
The following sections examine the benefits and drawbacks of the All IDs mechanism. They also give advice for the tuning of the All IDs Threshold.
Benefits of the All IDs Mechanism
The All IDs mechanism is an important mechanism for improving search performance in those cases where the search results would be most or all directory entries (for example, searches such as cn=*). By assuming that all entry IDs are returned Directory Server:
Does not have to maintain infinitely increasing entry ID lists, thus minimizing your Directory Server's disk space usage
Does not have to load unnecessarily large entry ID lists into memory in response to search requests that result in all directory entries anyway, thus increasing search performance by reducing large disk reads
Does not require large amounts of RAM to hold in memory unnecessarily large entry ID lists
Drawbacks of the All IDs Mechanism
Performance problems can occur if the All IDs threshold is set either too low (this is the most common problem) or too high for your directory's size.
When All IDs Threshold is Too Low
When you set the All IDs Threshold too low, too many index keys will contain the All IDs token. This can result in too many directory searches examining every entry in your directory. The performance hit on searches can be considerable.For example, suppose you are managing an equality index on the common name (cn) attribute. One of the index keys stored in your cn index is cn=James. The corresponding entry ID list contains the ID number of every entry containing an attribute that is set to James.
The equality index on the cn attribute is easy to maintain because only small fraction of the entries in your directory will include cn=James. Performance for searches that use a cn=James filter will be improved because only that small fraction of entry IDs needs to be examined when servicing the search request.
However, over time your directory may continue to grow. As it does, more and more James may be added, but at the same relatively small proportion of total directory entries. Eventually, the cn=James entry ID list can become quite large, but it will still be a list that is necessary for search performance. If your directory grows large enough that so many cn=James entries are added that the All IDs threshold is met, then the cn=James entry ID list is replaced with an All IDs token. Every time you search for cn=James, the directory server will examine every single entry in the directory in response to the search request.
When the database grows large, the All IDs threshold will be set for a large percentage of all index keys and your search performance will significantly degrade.
When All IDs Threshold is Too High
Setting the All IDs Threshold too high can also cause performance problems. An excessively high All IDs Threshold results in large entry ID lists that must be maintained and loaded into memory when servicing search requests. An excessively high All IDs Threshold can eliminate all of the benefits of the All IDs mechanism (see "Benefits of the All IDs Mechanism" for details).
All IDs Threshold Tuning Advice for Single- Enterprise Directories
Be careful when changing the default All IDs Threshold value for your server. If you change the threshold to an inappropriate value, you can compromise rather than improve your server performance. This tuning advice is intended primarily for single-enterprise directories of up to 80,000 entries.If your directory size is stable, set the All IDs Threshold to about 5 percent of the total number of entries stored in your directory. That is, if you have 50,000 entries in your directory, set the All IDs Threshold to 2,500.
If, you plan to add large numbers of entries to your directory in the near future, you should carefully consider your All IDs Threshold value. Consider the following:
Changing the All IDs Threshold means that you have to rebuild your database. This is a potentially expensive operation, especially for directories that contain millions of entries.
You need to balance your current directory needs against future expansion plans to avoid changing the All IDs Threshold at some later stage (which requires a database rebuild).While we recommend setting the All IDs Threshold to 5 percent of your directory size, you should not see serious performance problems if your All IDs Threshold is as little as 0.5 percent of your current database size or as great as 50 percent of your current database size. However, we nevertheless recommend you aim to stay as close to the 5 percent rule as possible.
Suppose, for example, that your current directory is 50,000 entries in size. However, in the next few years you expect your directory to grow 1,000,000 entries. If you set your All IDs Threshold to 5 percent of 50,000 (2,500), then when your directory grows to 1,000,000 entries you will have a performance problem. 2,500 entries is too low for a database containing 1,000,000 entries because the lower limit for a 1,000,000 entry database is 0.5 percent of 1,000,000, i.e.5,000 entries.
If you expect your directory to grow considerably in the future, you can do one of the following:
Set the All IDs Threshold to the current best value (2,500), and plan on rebuilding your database when your directory becomes large enough to warrant it. A database rebuild means shutting down your directory for however long the rebuild takes, or at least putting your directory into read-only mode. It also means reinitializing any consumer servers that your directory server is replicating entries to.
The strategy you should choose depends on your directory deployment needs. Consider the cost of rebuilding your databases (and all associated consumer servers) versus potential affects on performance as your All IDs Threshold value moves away from the ideal setting of 5 percent.Find a value that is a bit high for your current needs but that will work well for your future needs. For example, if your current directory contains 50,000 entries, try setting the All IDs Threshold to 20,000that is 40 percent of 50,000 (which puts it within range of your current directory needs) and 2 percent of 1,000,000 (which puts it within range of your future directory needs).
Note It may make sense for you to have a different All IDs Threshold on a consumer server as it can be tuned to service different searches.
Also consider how quickly your directory will grow and how long it will take you to increase your directory size. If your directory takes years to grow, then plan to do a database rebuild. If in a few months your directory increases in size by an order of magnitude or greater, consider ways to set the All IDs Threshold so as to minimize the time between database rebuilds.
All IDs Threshold Tuning Advice for Service Providers and Extranets
For hosting service providers, extranet directories and directories with over 80,000 entries, tuning advice is available from iPlanet Professional Services.
Default All IDs Threshold Value
By default, the directory server is set to an All IDs Threshold of 4000. This value is suitable for a database of up to 80,000 entries. If you expect your databases to be larger than 80,000 entries, we recommend that you change your all IDs Threshold to a large value before populating your databases.
Symptoms of an Inappropriate All IDs Threshold Value
When your All IDs Threshold is set incorrectly, you will see poor search performance. However, poor search performance can be caused by other factors. For example:
Your users are performing a lot of searches for which you are not maintaining an index.
Carefully examine these possibilities first before changing your All IDs Threshold value.Your database cache size and entry cache size may be set incorrectly. See , "Tuning Directory Server Performance" on page 429 for more information.
If you think that your server is suffering from an All IDs Threshold that is too low, look in your access log. See Chapter 12, "Monitoring Server and Database Activity." Any searches that resulted in all entry IDs being returned will contain the notes=U flag. The notes=U flag will be returned for:
Searches for which you are not maintaining an index
To determine whether the search result belongs to a search that should have been indexed, you have to match the conn and op values from the RESULT line to a previous SRCH line in your access log file. The SRCH line will show the search filter that was used for the search request. If you have an index for the specified search filter, then the notes=U flag results from the All IDs Threshold value being reached for the index key. For example, the access log looks as follows:Searches for which an ID list is not maintained because the All IDs Threshold value has been reached for that index key
[24/July/1998:15:12:20 -0800] conn=2 op=1 SRCH base="o=siroe.com" scope=0 filter="(cn=James)"
[24/July/1998:15:12:20 -0800] conn=2 op=1 RESULT err=0 tag=101 nentries=10000 notes=U
The presence of the notes=U flag indicates that the All IDs Threshold has been reached for the cn attribute index.
Changing the All IDs Threshold Value
To change the All IDs Threshold value for your server:
Shut down your Directory Server.
After you increase your All IDs Threshold value, examine your database cache size.Export all of your directory databases to LDIF using the command line.
Use the ldapmodify utility to edit the nsslapd-allidsthreshold entry or edit the following file with any text editor:
- For more information, see Chapter 4, "Populating Directory Databases."
Solaris 9 platform
/var/ds5/slapd-serverID/config/dse.ldif Other platforms
installDir/slapd-serverID/config/dse.ldif Locate the nsslapd-allidsthreshold attribute and change the value to the desired setting.
Initialize all your databases using ldif2db.
Restart your Directory Server.
- See Chapter 4, "Populating Directory Databases."
Increasing your All IDs Threshold can result in larger memory requirements caused by larger entry ID lists. The increased in memory requirements will differ depending on the number and types of indexes that you are maintaining, but the requirements will never be larger than the factor by which you increased the nsslapd-allidsthreshold attribute value. That is, if you double the nsslapd-allidsthreshold attribute value, then you should not have to increase your database cache size to more than double its current value.
Increasing your database cache size by the same factor as you increased the All IDs Threshold is an extreme measure. If you have the physical memory available, try increasing your database cache size by a factor that is 25 percent of your nsslapd-allidsthreshold value increase. For example, if you doubled the All IDs Threshold value, increase your database cache size by 50 percent. If necessary, slowly increase your cache size until you are satisfied with your server's performance.
Set your database cache size using the attribute nsslapd-dbcachesize attribute. For more information, see nsslapd-dbcachesize attribute in the iPlanet Directory Server Configuration, Command, and File Reference.
Attribute Name Quick Reference Table
The following table lists all attributes which have a primary or real name as well as an alias. When creating indexes be sure to use the primary name.
Previous Contents Index Next
Copyright © 2001 Sun Microsystems, Inc. Some preexisting portions Copyright © 2001 Netscape Communications Corp. All rights reserved.
Last Updated October 29, 2001