Skip Navigation Links | |
Exit Print View | |
Oracle Directory Server Enterprise Edition Administration Guide 11g Release 1 (11.1.1.5.0) |
Part I Directory Server Administration
2. Directory Server Instances and Suffixes
3. Directory Server Configuration
6. Directory Server Access Control
7. Directory Server Password Policy
8. Directory Server Backup and Restore
9. Directory Server Groups, Roles, and CoS
10. Directory Server Replication
Changing the Index List Threshold
To Change the Index List Threshold
Browsing Indexes for Client Searches
To Add or Modify Browsing Index Entries
To Regenerate Browsing Indexes
13. Directory Server Attribute Value Uniqueness
15. Directory Server Monitoring
Part II Directory Proxy Server Administration
16. Directory Proxy Server Tools
17. Directory Proxy Server Instances
19. Directory Proxy Server Certificates
20. Directory Proxy Server Load Balancing and Client Affinity
21. Directory Proxy Server Distribution
22. Directory Proxy Server Virtualization
23. Virtual Data Transformations
24. Connections Between Directory Proxy Server and Back-End LDAP Servers
25. Connections Between Clients and Directory Proxy Server
26. Directory Proxy Server Client Authentication
27. Directory Proxy Server Logging
28. Directory Proxy Server Monitoring and Alerts
Part III Directory Service Control Center Administration
This section describes how to manage indexes for specific attributes. The section includes information about creating, modifying, and deleting indexes. See Managing Browsing Indexes for procedures specific to virtual list view (VLV) operations.
You can also check the attributes that need to be indexed by running the dsconf info command.
You can use DSCC to perform this task. For information, see Directory Service Control Center Interface and the DSCC online help.
$ dsconf list-indexes -h host -p port -v suffix-DN
Note - You cannot create a new system index. Only the existing system indexes defined internally by Directory Server are maintained.
You can use DSCC to perform this task. For information, see Directory Service Control Center Interface and the DSCC online help.
Use the dsconf create-index command-line utility to configure the new index information by specifying the attribute that you want to index.
For example, to create an index entry for the preferredLanguage attribute, use this command:
$ dsconf create-index -h host -p port dc=example,dc=com preferredLanguage
Note - The command dsconf create-index sets the index configuration, but does not actually create the index files necessary for searches. Generating the index files can affect performance. To give you more control over the indexing procedure, generating the index files is done manually after the new index configuration has been created.
Always use the attribute’s primary name when creating indexes. Do not use the attribute’s alias. The primary name of the attribute is the first name listed for the attribute in the schema, for example, uid for the userid attribute.
The dsconf create-index command creates an index with default properties. If you want to modify these properties, use the dsconf set-index-prop command. For more information about modifying index properties, see To Modify Indexes.
Note - When the configuration of an index is modified, re-indexing is required for the changes to take effect and to use the index again.
See To Generate Indexes.
You can use DSCC to perform this task. For information, see Directory Service Control Center Interface and the DSCC online help.
$ dsconf set-index-prop -h host -p port suffix-DN attr-name property:value
For example, to enable the approximate index approx-enabled for the preferredLanguage index , use the command:
$ dsconf set-index-prop -h host -p port dc=example,dc=com \ preferredLanguage approx-enabled:on
You can modify the following properties for each index:
eq-enabled equality
pres-enabled presence
sub-enabled substring
One of the properties that you might want to modify is the optional nsMatchingRule attribute. This attribute contains the OID for any matching rule known by the server. It enables the OID of a language collation order for internationalized indexes, and other matching rules such as CaseExactMatch. For a list of supported locales and the OID of their associated collation order, see Oracle Directory Server Enterprise Edition Reference.
For more information about index configuration attributes, see Oracle Directory Server Enterprise Edition Reference.
Note - When the configuration of an index is modified, re-indexing is required for the changes to take effect and to use the index again.
Run dsconf info to display the attributes that need to be reindexed. For example, the following output shows the cn and uid attributes that need to be reindexed.
$ dsconf info Instance path : /local/dsInst Global State : read-write Host Name : hostname Port : port Secure port : secure-port Total entries : 160 Server version : 11.1.1.5.0 Suffixes : dc=example,dc=com No active tasks dc=example,dc=com ================= Attribute to reindex : cn uid
See To Generate Indexes.
This procedure generates index files so that new or modified indexes can be searchable. If you modify an index configuration for an attribute, all searches that include that attribute as a filter are not indexed. To ensure that searches including that attribute are successful, use this procedure command to regenerate existing indexes every time you create or modify an index configuration for an attribute.
You can use DSCC to perform this task. For information, see Directory Service Control Center Interface and the DSCC online help.
Generate your new index files online.
$ dsconf reindex -h host -p port [-t attr] suffix-DN
where -t specifies that only the specified attribute or attributes are to be reindexed, not all attributes.
For example, to regenerate the preferredLanguage index, type:
$ dsconf reindex -h host -p port -t preferredLanguage dc=example,dc=com
While the dsconf reindex command is running, the contents of the suffix remain available through the server. However, searches are not indexed until the command has completed. Reindexing is a resource-intensive task that can impact the performance of other operations on the server.
Generate your new index files offline.
$ dsadm reindex -t attr instance-path suffix-DN
For example, to regenerate the preferredLanguage index, type:
$ dsadm reindex -t preferredLanguage /local/dsInst dc=example,dc=com
Regenerate all of your indexes quickly offline by reinitializing your suffix.
When you reinitialize a suffix, all index files are automatically regenerated. Depending on the size of the directory, reinitializing the suffix is usually faster than reindexing two or more attributes. However, the suffix is unavailable during the initialization. For more information, see Reindexing a Suffix by Reinitialization.
Analyzing indexes requires gathering search filter usage patterns from user requests, especially for values as specified in the filters and subsequently looking the status of those values into the index files. When a search requests is processed, internal or user generated, a first phase uses indexes to find potential entries to be returned. Depending on the search filter, if the index of one of the specified attributes matches too many entries (ALLIDs), the search becomes non-indexed (notes=U in the access log). In any case, all entries thus gathered (or the entire DIT) are matched against the filter for actually returning the search result.
Note - Directory Server tries to optimize the search so that not all index values matching ALLID will force the search to be not indexed.
The dsconf analyze-index-filters command when enabled, displays the Directory Server search filters with their statistics.
$ dsconf enable-index-filter-analyzer [--max-entries INT] SUFFIX_DN
For example, to enable index filter analyzer on o=example.com, run the following command:
$ dsconf enable-index-filter-analyzer -p port-number o=example.com $ dsconf get-suffix-prop -p port-number o=example.com \ index-filter-analyzer-enabled index-filter-analyzer-max-entries index-filter-analyzer-enabled : on index-filter-analyzer-max-entries : 2000
$ dsconf analyze-index-filters -p port-number SUFFIX_DN
The output displays all the attributes that need to be reindexed, index filter usage statistics, number of hits, and number of allids hits. Based on the output, you can configure indexes differently by modifying the all-ids-threshold property or by creating indexes, to improve performance.
$ dsconf analyze-index-filters -p port-number o=example.com Observations started at Nov 13, 2008 12:01:29 PM Total number of search requests : 8 Total number of Allids : 7 filter Type #allids/#hits Threshold Max matching Additional info entries ------------------- ---- ------------- --------- ------------ --------------- (departmentNumber eq 1/1 2 "departmentNumber" =9415) is not indexed (objectClass eq 1/1 *4000 10000 To investigate =inetOrgPerson) (objectClass=*) pres 1/1 10006 "pres" type is disabled for "objectClass" system index (roomNumber=*) pres 1/1 10000 "roomNumber" is not indexed (roomNumber=1*) eq 1/1 4071 "roomNumber" is not indexed (telephoneNumber=*) pres 2/2 *4000 10000 To investigate (telephoneNumber=1*) eq 0/1 4000 10000 ## * indicates thresholds which have been crossed. No attributes need to be reindexed Use "dsconf set-index-prop o=example.com ATTR_NAME..." to set the allids threshold value and to take benefit of indexes.
The displayed filters are the basic filter elements along with the following information:
Type that is used during processing.
How many times the filter element is used and how many times the index value is Allids.
Note - Allids can occur because of unsufficient privileges or if the index type configuration is changed. In the latter case, data may be skewed.
Complex filters are broken down to these basic elements and will not appear in their entirety.
The example output displays the following information:
Some index values are Allids
Some attributes are not indexed, that is, roomNumber, departmentNumber.
Total 4071 entries matched the roomNumber=1* filter
Some indexes are well configured for some filter use, that is, telephoneNumber=1*.
$ dsconf create-index -p port-number o=example.com roomNumber
$ dsconf analyze-index-filters -p port-number o=example.com Observations started at Nov 13, 2008 12:01:29 PM Total number of search requests : 9 Total number of Allids : 8 filter Type #allids/#hits Threshold Max matching Additional info entries -------------------- ---- ------------- --------- ------------ ----------------- (departmentNumber eq 1/1 2 "departmentNumber" =9415) is not indexed (objectClass eq 1/1 *4000 10000 To investigate =inetOrgPerson) (objectClass=*) pres 1/1 10006 "pres" type is disabled for "objectClass" system index (roomNumber=*) pres 1/1 *4000 10000 To investigate (roomNumber=1*) eq 1/1 *4000 4071 To investigate (telephoneNumber=*) pres 2/2 *4000 10000 To investigate (telephoneNumber=1*) eq 0/1 4000 10000 ## * indicates thresholds which have been crossed. Attributes to reindex : roomNumber Use "dsconf reindex --attr ATTR_NAME... o=example.com" to reindex. Use "dsconf set-index-prop o=example.com ATTR_NAME..." to set the allids threshold value and to take benefit of indexes.
As required, follow the appropriate procedure as displayed at the end of the output.
To restart the index filter analyzer, disable the analyzer using the disable-index-filter-analyzer subcommand and then start the analyzer again using the enable-index-filter-analyzer subcommand.
Note - Monitoring affects performance. It also requires heavy memory resources, based on the configured maximum number of filters to monitor.
Run the dsconf info command to know when the analyzer was enabled. If you do not want to monitor indexes and analyze-index-filters, it is not recommended to keep the analyzer running.
Directory Server tries to optimize the search so that when complex filters are evaluated, not all elements could be processed. Do not expect a one-to-one relation with what appears in the access log, and a complex filter and its constituent elements.
See Also
For more information, see dsconf(1M).
Using the dsconf analyze-index-filters command, gather the most used filters and their behavior. On the other hand, to know the data as appears in the index files, use dsadm analyze-indexes to have a snapshot of index files.
$ dsadm stop INSTANCE_PATH
$ dsadm analyze-indexes [-bRi] [-o FILE] INSTANCE_PATH SUFFIX_DN
For example, to analyze the attribute indexes of suffix o=example.com, run the following command:
$ dsadm analyze-indexes /local/myinst o=example.com This operation may take a long time and generate important amounts of data Do you want to continue [y/n]? y Generating raw index data, please wait... Raw index data available in file '/local/myinst/logs/db_stat_example%2ecom' Index Type Total Keys ALLIDs 95% 90% 80% --------------- --------- ---------- ------ --- --- --- aci PRESENCE 1 0 0 0 0 ancestorid EQUALITY 6 6 0 0 0 cn EQUALITY 200000 0 0 0 0 cn SUBSTRING 14828 15 0 0 0 entrydn EQUALITY 100006 0 0 0 0 givenName EQUALITY 8605 0 0 0 0 givenName SUBSTRING 4762 4 0 0 0 givenName PRESENCE 1 1 0 0 0 mail EQUALITY 100000 0 0 0 0 mail SUBSTRING 14975 26 1 3 2 mail PRESENCE 1 1 0 0 0 nsuniqueid EQUALITY 100007 0 0 0 0 numsubordinates PRESENCE 1 0 0 0 0 objectclass EQUALITY 7 4 0 0 0 parentid EQUALITY 6 5 0 0 0 sn EQUALITY 100000 0 0 0 0 sn SUBSTRING 12993 0 0 0 0 telephoneNumber EQUALITY 99924 0 0 0 0 telephoneNumber SUBSTRING 1106 24 0 0 0 telephoneNumber PRESENCE 1 1 0 0 0 uid EQUALITY 200000 0 0 0 0 uid PRESENCE 1 1 0 0 0 aci PRESENCE ============ ALLIDs keys : 0 / 1 ancestorid EQUALITY =================== ALLIDs keys : 6 / 6 [1] [2] [3] [4] [5] [6] cn EQUALITY =========== ALLIDs keys : 0 / 200000 cn SUBSTRING ============ ALLIDs keys : 15 / 14828 [100] [101] [102] [103] [104] [105] [106] [107] [108] [109] [^us] [er1] [r10] [ser] [use] entrydn EQUALITY ================ ALLIDs keys : 0 / 100006 givenName EQUALITY ================== ALLIDs keys : 0 / 8605 givenName SUBSTRING =================== ALLIDs keys : 4 / 4762 [^ma] [ie$] [na$] [ne$] givenName PRESENCE ================== ALLIDs keys : 1 / 1 [pres] mail EQUALITY ============= ALLIDs keys : 0 / 100000 mail SUBSTRING ============== ALLIDs keys : 26 / 14975 [.co] [0@e] [1@e] [2@e] [3@e] [4@e] [5@e] [6@e] [7@e] [8@e] [9@e] [@ex] [^ma] [amp] [com] [ell] [exa] [e.c] [ie_] [le.] [mar] [mpl] [na_] [ne_] [om$] [ple] [xam] mail PRESENCE ============= ALLIDs keys : 1 / 1 [pres] nsuniqueid EQUALITY =================== ALLIDs keys : 0 / 100007 numsubordinates PRESENCE ======================== ALLIDs keys : 0 / 1 objectclass EQUALITY ==================== ALLIDs keys : 4 / 7 [inetorgperson] [organizationalperson] [person] [top] parentid EQUALITY ================= ALLIDs keys : 5 / 6 [2] [3] [4] [5] [6] sn EQUALITY =========== ALLIDs keys : 0 / 100000 sn SUBSTRING ============ ALLIDs keys : 0 / 12993 telephoneNumber EQUALITY ======================== ALLIDs keys : 0 / 99924 telephoneNumber SUBSTRING ========================= ALLIDs keys : 24 / 1106 [120] [121] [130] [140] [141] [151] [171] [180] [181] [206] [213] [303] [408] [415] [510] [714] [804] [818] [^12] [^13] [^14] [^15] [^17] [^18] telephoneNumber PRESENCE ======================== ALLIDs keys : 1 / 1 [pres] uid EQUALITY ============ ALLIDs keys : 0 / 200000 uid PRESENCE ============ ALLIDs keys : 1 / 1 [pres]
Following what explained in Chapter 9, Directory Server Indexing, in Oracle Directory Server Enterprise Edition Reference, dsadm analyze-indexes displays the status of the value keys as used by Directory Server. If most of the keys are Allid or 95% of Allid, the number of entries matching the key is at least equal to all-ids-threshold. The index most likely has to be configured with a higher value for all-ids-threshold.
Note - Too high an all-ids-threshold value can impact performance.
The dsadm analyze-indexes displays which keys are ALLID or close to be, so it can be matched with the output of dsconf analyze-index-filters. If a search specifies a filter whose value is an allid key, the search might not be indexed, depending on the entire search filter as mentioned above.
You can use DSCC to perform this task. For information, see Directory Service Control Center Interface and the DSCC online help.
$ dsconf delete-index -h host -p port suffix-DN attr-name
For example, the following command deletes all indexes for the preferredLanguage attribute:
$ dsconf delete-index -h host -p port dc=example,dc=com preferredLanguage
Take great care when deleting default indexes because it can affect Directory Server functioning.
Slow searches might be a result of your system index list size exceeding the index list threshold. The index list threshold is the maximum number of values for each index key. To determine whether the index list threshold size has been exceeded, examine the access log. The notes=U flag at the end of an access log RESULT message indicates that an unindexed search was performed. A previous SRCH message for the same connection and operation specifies the search filter that was used. The following two-line example traces an unindexed search for cn=Smith that returns 10,000 entries. Timestamps have been removed from the messages.
conn=2 op=1 SRCH base="o=example.com" scope=0 filter="(cn=Smith)" conn=2 op=1 RESULT err=0 tag=101 nentries=10000 notes=U
If your system often exceeds the index list threshold, consider increasing the threshold to improve performance. The following procedure uses the dsconf set-server-prop command to modify the all-ids-threshold property.
You can use DSCC to perform this task. For information, see Directory Service Control Center Interface and the DSCC online help.
You can adjust your index list threshold at any of the following levels:
At the instance level:
dsconf set-server-prop -h host -p port all-ids-threshold:value
At the suffix level:
dsconf set-suffix-prop -h host -p port suffix-DN all-ids-threshold:value
At the entry level:
dsconf set-index-prop -h host -p port suffix-DN all-ids-threshold:value
At the index level, by search type:
dsconf set-index-prop -h host -p port suffix-DN all-ids-threshold search-type:value
where search-type is one of the following:
eq-enabled equality
pres-enabled presence
sub-enabled substring
The all-ids-threshold property cannot be configured for the approximate index.
You can use DSCC to set the threshold at the index level, by search type. For more information, see the Directory Server online help.
See To Generate Indexes.
Increase the database cache size by 25 percent of the magnitude of the increase to the all IDs threshold.
In other words, if you increase the all IDs threshold from 4000 to 6000, you can increase the database cache size by about 12 ½ percent to account for the increase in index list size.
Database cache size is set using the attribute dbcachesize. Find the optimum size empirically before applying changes to production servers.
If your index files become corrupt, or if you change the index for an attribute, you must reindex the suffix to recreate the index files in the corresponding database directory. You can reindex a suffix while the directory server is running or by reinitializing the suffix.
When you reindex a suffix, the server examines all of the entries the suffix contains and rebuilds the index files. During reindexing, the contents of the suffix are read-only. Because the server must scan the entire suffix for every attribute that is reindexed, this process might take up to several hours for suffixes with millions of entries. The length of time also depends on the indexes you configure. In addition, while the suffix is being reindexed, it is not available.
You can use DSCC to perform this task. For information, see Directory Service Control Center Interface and the DSCC online help.
$ dsconf reindex -h host -p port suffix-DN
For example, to initialize all indexes on the dc=example,dc=com suffix, use this command:
$ dsconf reindex -h host -p port dc=example,dc=com
When you reinitialize a suffix, the new contents are imported, which means that the suffix contents are replaced and new index files are created.
You can use DSCC to perform this task. For information, see Directory Service Control Center Interface and the DSCC online help.
During the initialization, the suffix is unavailable. When the initialization is complete, all configured indexes are ready to be used.