Oracle® Fusion Middleware Administrator's Guide for Oracle Directory Server Enterprise Edition 11g Release 1 (11.1.1.7.0) Part Number E28972-01 |
|
|
PDF · Mobi · ePub |
Like a book index, Directory Server indexes speed up searches by associating search strings with references to the directory contents.
For information about the types of indexes and index tuning, see Chapter 9, Directory Server Indexing, in the Reference for Oracle Directory Server Enterprise Edition.
This chapter covers the following topics:
This section describes how to manage indexes for specific attributes. The section includes information about creating, modifying, and deleting indexes. See Managing Browsing Indexes for procedures specific to virtual list view (VLV) operations.
You can also check the attributes that need to be indexed by running the dsconf info
command.
You can use the web interface Directory Service Control Center (DSCC) to perform this task.
To list your existing indexes and their properties, use this command:
$ dsconf list-indexes -h host -p port -v suffix-DN
Note:
You cannot create a new system index. Only the existing system indexes defined internally by Directory Server are maintained.
You can use the web interface Directory Service Control Center (DSCC) to perform this task.
Create the new index configuration.
Use the dsconf create-index
command-line utility to configure the new index information by specifying the attribute that you want to index.
For example, to create an index entry for the preferredLanguage
attribute, use this command:
$ dsconf create-index -h host -p port dc=example,dc=com preferredLanguage
Note:
The command dsconf create-index
sets the index configuration, but does not actually create the index files necessary for searches. Generating the index files can affect performance. To give you more control over the indexing procedure, generating the index files is done manually after the new index configuration has been created.
Always use the attribute's primary name when creating indexes. Do not use the attribute's alias. The primary name of the attribute is the first name listed for the attribute in the schema, for example, uid
for the userid
attribute.
Set the index properties by using the dsconf set-index-prop
command.
The dsconf create-index
command creates an index with default properties. If you want to modify these properties, use the dsconf set-index-prop
command. For more information about modifying index properties, see To Modify Indexes.
Note:
When the configuration of an index is modified, re-indexing is required for the changes to take effect and to use the index again.
Generate the index files.
See To Generate Indexes.
Repeat the previous steps for all servers that you want to be indexed.
You can use the web interface Directory Service Control Center (DSCC) to perform this task.
Modify the index properties.
$ dsconf set-index-prop -h host -p port suffix-DN attr-name property:value
For example, to enable the approximate index approx-enabled
for the preferredLanguage
index, use the command:
$ dsconf set-index-prop -h host -p port dc=example,dc=com \ preferredLanguage approx-enabled:on
You can modify the following properties for each index:
eq-enabled
equality
pres-enabled
presence
sub-enabled
substring
One of the properties that you might want to modify is the optional nsMatchingRule
attribute. This attribute contains the OID for any matching rule known by the server. It enables the OID of a language collation order for internationalized indexes, and other matching rules such as CaseExactMatch
. For a list of supported locales and the OID of their associated collation order, see the Reference for Oracle Directory Server Enterprise Edition.
For more information about index configuration attributes, see the Reference for Oracle Directory Server Enterprise Edition.
Note:
When the configuration of an index is modified, re-indexing is required for the changes to take effect and to use the index again.
Run dsconf info
to display the attributes that need to be reindexed. For example, the following output shows the cn
and uid
attributes that need to be reindexed.
$ dsconf info Instance path : /local/dsInst Global State : read-write Host Name : hostname Port : port Secure port : secure-port Total entries : 160 Server version : 11.1.1.6 Suffixes : dc=example,dc=com No active tasks dc=example,dc=com ================= Attribute to reindex : cn uid
Regenerate your new indexes.
See To Generate Indexes.
Repeat the previous steps for all servers that include the modified attribute index.
This procedure generates index files so that new or modified indexes can be searchable. If you modify an index configuration for an attribute, all searches that include that attribute as a filter are not indexed. To ensure that searches including that attribute are successful, use this procedure command to regenerate existing indexes every time you create or modify an index configuration for an attribute.
You can use the web interface Directory Service Control Center (DSCC) to perform this task.
Generate your index files in one of the following ways:
Generate your new index files online.
$ dsconf reindex -h host -p port [-t attr] suffix-DN
where -t
specifies that only the specified attribute or attributes are to be reindexed, not all attributes.
For example, to regenerate the preferredLanguage
index, type:
$ dsconf reindex -h host -p port -t preferredLanguage dc=example,dc=com
While the dsconf reindex
command is running, the contents of the suffix remain available through the server. However, searches are not indexed until the command has completed. Reindexing is a resource-intensive task that can impact the performance of other operations on the server.
Generate your new index files offline.
$ dsadm reindex -t attr instance-path suffix-DN
For example, to regenerate the preferredLanguage
index, type:
$ dsadm reindex -t preferredLanguage /local/dsInst dc=example,dc=com
Regenerate all of your indexes quickly offline by reinitializing your suffix.
When you reinitialize a suffix, all index files are automatically regenerated. Depending on the size of the directory, reinitializing the suffix is usually faster than reindexing two or more attributes. However, the suffix is unavailable during the initialization. For more information, see Reindexing a Suffix by Reinitialization.
Analyzing indexes requires gathering search filter usage patterns from user requests, especially for values as specified in the filters and subsequently looking the status of those values into the index files. When a search requests is processed, internal or user generated, a first phase uses indexes to find potential entries to be returned. Depending on the search filter, if the index of one of the specified attributes matches too many entries (ALLIDs), the search becomes non-indexed (notes=U
in the access log). In any case, all entries thus gathered (or the entire DIT) are matched against the filter for actually returning the search result.
Note:
Directory Server tries to optimize the search so that not all index values matching ALLID
will force the search to be not indexed.
The dsconf analyze-index-filters
command when enabled, displays the Directory Server search filters with their statistics.
Enable index filter analyzer using the following command:
$ dsconf enable-index-filter-analyzer [--max-entries INT] SUFFIX_DN
For example, to enable index filter analyzer on o=example.com
, run the following command:
$ dsconf enable-index-filter-analyzer -p port-number o=example.com $ dsconf get-suffix-prop -p port-number o=example.com \ index-filter-analyzer-enabled index-filter-analyzer-max-entries index-filter-analyzer-enabled : on index-filter-analyzer-max-entries : 2000
Analyze index filters to know the indexes that need to be reindexed.
$ dsconf analyze-index-filters -p port-number SUFFIX_DN
The output displays all the attributes that need to be reindexed, index filter usage statistics, number of hits, and number of allids
hits. Based on the output, you can configure indexes differently by modifying the all-ids-threshold
property or by creating indexes, to improve performance.
$ dsconf analyze-index-filters -p port-number o=example.com
Observations started at Nov 13, 2008 12:01:29 PM
Total number of search requests : 8
Total number of Allids : 7
filter Type #allids/#hits Threshold Max matching Additional info
entries
------------------- ---- ------------- --------- ------------ ---------------
(departmentNumber eq 1/1 2 "departmentNumber"
=9415) is not indexed
(objectClass eq 1/1 *4000 10000 To investigate
=inetOrgPerson)
(objectClass=*) pres 1/1 10006 "pres" type is
disabled for
"objectClass"
system index
(roomNumber=*) pres 1/1 10000 "roomNumber" is
not indexed
(roomNumber=1*) eq 1/1 4071 "roomNumber" is
not indexed
(telephoneNumber=*) pres 2/2 *4000 10000 To investigate
(telephoneNumber=1*) eq 0/1 4000 10000
## * indicates thresholds which have been crossed.
No attributes need to be reindexed
Use "dsconf set-index-prop o=example.com ATTR_NAME..." to set the allids threshold
value and to take benefit of indexes.
The displayed filters are the basic filter elements along with the following information:
Type that is used during processing.
How many times the filter element is used and how many times the index value is Allids
.
Note:
Allids
can occur because of insufficient privileges or if the index type configuration is changed. In the latter case, data may be skewed.
Complex filters are broken down to these basic elements and will not appear in their entirety.
The example output displays the following information:
Some index values are Allids
Some attributes are not indexed, that is, roomNumber
, departmentNumber
.
Total 4071
entries matched the roomNumber=1*
filter
Some indexes are well configured for some filter use, that is, telephoneNumber=1*
.
Create the new index for the attribute roomNumber
.
$ dsconf create-index -p port-number o=example.com roomNumber
Run the analyze-index-filters
command again to check the status.
$ dsconf analyze-index-filters -p port-number o=example.com
Observations started at Nov 13, 2008 12:01:29 PM
Total number of search requests : 9
Total number of Allids : 8
filter Type #allids/#hits Threshold Max matching Additional info
entries
-------------------- ---- ------------- --------- ------------ -----------------
(departmentNumber eq 1/1 2 "departmentNumber"
=9415) is not indexed
(objectClass eq 1/1 *4000 10000 To investigate
=inetOrgPerson)
(objectClass=*) pres 1/1 10006 "pres" type is
disabled for
"objectClass"
system index
(roomNumber=*) pres 1/1 *4000 10000 To investigate
(roomNumber=1*) eq 1/1 *4000 4071 To investigate
(telephoneNumber=*) pres 2/2 *4000 10000 To investigate
(telephoneNumber=1*) eq 0/1 4000 10000
## * indicates thresholds which have been crossed.
Attributes to reindex : roomNumber
Use "dsconf reindex --attr ATTR_NAME... o=example.com" to reindex.
Use "dsconf set-index-prop o=example.com ATTR_NAME..." to set the allids
threshold value and to take benefit of indexes.
As required, follow the appropriate procedure as displayed at the end of the output.
Restart the index filter analyzer to consider the latest index updates.
To restart the index filter analyzer, disable the analyzer using the disable-index-filter-analyzer
subcommand and then start the analyzer again using the enable-index-filter-analyzer
subcommand.
Note:
Monitoring affects performance. It also requires heavy memory resources, based on the configured maximum number of filters to monitor.
Run the dsconf info
command to know when the analyzer was enabled. If you do not want to monitor indexes and analyze-index-filters, it is not recommended to keep the analyzer running.
Directory Server tries to optimize the search so that when complex filters are evaluated, not all elements could be processed. Do not expect a one-to-one relation with what appears in the access log, and a complex filter and its constituent elements.
See Also
For more information, see dsconf.
Using the dsconf analyze-index-filters
command, gather the most used filters and their behavior. On the other hand, to know the data as appears in the index files, use dsadm analyze-indexes
to have a snapshot of index files.
Your server must be stopped before analyzing the attribute indexes.
$ dsadm stop INSTANCE_PATH
Analyze attribute indexes using the following command:
$ dsadm analyze-indexes [-bRi] [-o FILE] INSTANCE_PATH SUFFIX_DN
For example, to analyze the attribute indexes of suffix o=example.com
, run the following command:
$ dsadm analyze-indexes /local/myinst o=example.com This operation may take a long time and generate important amounts of data Do you want to continue [y/n]? y Generating raw index data, please wait... Raw index data available in file '/local/myinst/logs/db_stat_example%2ecom' Index Type Total Keys ALLIDs 95% 90% 80% --------------- --------- ---------- ------ --- --- --- aci PRESENCE 1 0 0 0 0 ancestorid EQUALITY 6 6 0 0 0 cn EQUALITY 200000 0 0 0 0 cn SUBSTRING 14828 15 0 0 0 entrydn EQUALITY 100006 0 0 0 0 givenName EQUALITY 8605 0 0 0 0 givenName SUBSTRING 4762 4 0 0 0 givenName PRESENCE 1 1 0 0 0 mail EQUALITY 100000 0 0 0 0 mail SUBSTRING 14975 26 1 3 2 mail PRESENCE 1 1 0 0 0 nsuniqueid EQUALITY 100007 0 0 0 0 numsubordinates PRESENCE 1 0 0 0 0 objectclass EQUALITY 7 4 0 0 0 parentid EQUALITY 6 5 0 0 0 sn EQUALITY 100000 0 0 0 0 sn SUBSTRING 12993 0 0 0 0 telephoneNumber EQUALITY 99924 0 0 0 0 telephoneNumber SUBSTRING 1106 24 0 0 0 telephoneNumber PRESENCE 1 1 0 0 0 uid EQUALITY 200000 0 0 0 0 uid PRESENCE 1 1 0 0 0 aci PRESENCE ============ ALLIDs keys : 0 / 1 ancestorid EQUALITY =================== ALLIDs keys : 6 / 6 [1] [2] [3] [4] [5] [6] cn EQUALITY =========== ALLIDs keys : 0 / 200000 cn SUBSTRING ============ ALLIDs keys : 15 / 14828 [100] [101] [102] [103] [104] [105] [106] [107] [108] [109] [^us] [er1] [r10] [ser] [use] entrydn EQUALITY ================ ALLIDs keys : 0 / 100006 givenName EQUALITY ================== ALLIDs keys : 0 / 8605 givenName SUBSTRING =================== ALLIDs keys : 4 / 4762 [^ma] [ie$] [na$] [ne$] givenName PRESENCE ================== ALLIDs keys : 1 / 1 [pres] mail EQUALITY ============= ALLIDs keys : 0 / 100000 mail SUBSTRING ============== ALLIDs keys : 26 / 14975 [.co] [0@e] [1@e] [2@e] [3@e] [4@e] [5@e] [6@e] [7@e] [8@e] [9@e] [@ex] [^ma] [amp] [com] [ell] [exa] [e.c] [ie_] [le.] [mar] [mpl] [na_] [ne_] [om$] [ple] [xam] mail PRESENCE ============= ALLIDs keys : 1 / 1 [pres] nsuniqueid EQUALITY =================== ALLIDs keys : 0 / 100007 numsubordinates PRESENCE ======================== ALLIDs keys : 0 / 1 objectclass EQUALITY ==================== ALLIDs keys : 4 / 7 [inetorgperson] [organizationalperson] [person] [top] parentid EQUALITY ================= ALLIDs keys : 5 / 6 [2] [3] [4] [5] [6] sn EQUALITY =========== ALLIDs keys : 0 / 100000 sn SUBSTRING ============ ALLIDs keys : 0 / 12993 telephoneNumber EQUALITY ======================== ALLIDs keys : 0 / 99924 telephoneNumber SUBSTRING ========================= ALLIDs keys : 24 / 1106 [120] [121] [130] [140] [141] [151] [171] [180] [181] [206] [213] [303] [408] [415] [510] [714] [804] [818] [^12] [^13] [^14] [^15] [^17] [^18] telephoneNumber PRESENCE ======================== ALLIDs keys : 1 / 1 [pres] uid EQUALITY ============ ALLIDs keys : 0 / 200000 uid PRESENCE ============ ALLIDs keys : 1 / 1 [pres]
Following what explained in Chapter 9, Directory Server Indexing, in the Reference for Oracle Directory Server Enterprise Edition, dsadm analyze-indexes
displays the status of the value keys as used by Directory Server. If most of the keys are Allid
or 95% of Allid
, the number of entries matching the key is at least equal to all-ids-threshold
. The index most likely has to be configured with a higher value for all-ids-threshold
.
Note:
Too high an all-ids-threshold
value can impact performance.
The dsadm analyze-indexes
displays which keys are ALLID or close to be, so it can be matched with the output of dsconf analyze-index-filters
. If a search specifies a filter whose value is an allid key, the search might not be indexed, depending on the entire search filter as mentioned above.
You can use the web interface Directory Service Control Center (DSCC) to perform this task.
Remove all indexes that are configured for an attribute.
$ dsconf delete-index -h host -p port suffix-DN attr-name
For example, the following command deletes all indexes for the preferredLanguage
attribute:
$ dsconf delete-index -h host -p port dc=example,dc=com preferredLanguage
Take great care when deleting default indexes because it can affect Directory Server functioning.
Slow searches might be a result of your system index list size exceeding the index list threshold. The index list threshold is the maximum number of values for each index key. To determine whether the index list threshold size has been exceeded, examine the access log. The notes=U
flag at the end of an access log RESULT
message indicates that an unindexed search was performed. A previous SRCH
message for the same connection and operation specifies the search filter that was used. The following two-line example traces an unindexed search for cn=Smith
that returns 10,000 entries. Timestamps have been removed from the messages.
conn=2 op=1 SRCH base="o=example.com" scope=0 filter="(cn=Smith)" conn=2 op=1 RESULT err=0 tag=101 nentries=10000 notes=U
If your system often exceeds the index list threshold, consider increasing the threshold to improve performance. The following procedure uses the dsconf set-server-prop
command to modify the all-ids-threshold
property.
You can use the web interface Directory Service Control Center (DSCC) to perform this task.
Adjust the index list threshold.
You can adjust your index list threshold at any of the following levels:
At the instance level:
dsconf set-server-prop -h host -p port all-ids-threshold:value
At the suffix level:
dsconf set-suffix-prop -h host -p port suffix-DN all-ids-threshold:value
At the entry level:
dsconf set-index-prop -h host -p port suffix-DN all-ids-threshold:value
At the index level, by search type:
dsconf set-index-prop -h host -p port suffix-DN all-ids-threshold search-type:value
where search-type is one of the following:
eq-enabled
equality
pres-enabled
presence
sub-enabled
substring
The all-ids-threshold
property cannot be configured for the approximate index.
You can use DSCC to set the threshold at the index level, by search type. For more information, see the Directory Server online help.
Regenerate the suffix indexes.
See To Generate Indexes.
If the database cache size was tuned for the old all IDs
threshold value and the server has adequate physical memory, consider increasing the database cache size.
Increase the database cache size by 25 percent of the magnitude of the increase to the all IDs
threshold.
In other words, if you increase the all IDs
threshold from 4000 to 6000, you can increase the database cache size by about 12 ½ percent to account for the increase in index list size.
Database cache size is set using the attribute dbcachesize
. Find the optimum size empirically before applying changes to production servers.
If your index files become corrupt, or if you change the index for an attribute, you must reindex the suffix to recreate the index files in the corresponding database directory. You can reindex a suffix while the directory server is running or by reinitializing the suffix.
When you reindex a suffix, the server examines all of the entries the suffix contains and rebuilds the index files. During reindexing, the contents of the suffix are read-only. Because the server must scan the entire suffix for every attribute that is reindexed, this process might take up to several hours for suffixes with millions of entries. The length of time also depends on the indexes you configure. In addition, while the suffix is being reindexed, it is not available.
You can use the web interface Directory Service Control Center (DSCC) to perform this task.
Reindex all indexes on a suffix.
$ dsconf reindex -h host -p port suffix-DN
For example, to initialize all indexes on the dc=example,dc=com
suffix, use this command:
$ dsconf reindex -h host -p port dc=example,dc=com
When you reinitialize a suffix, the new contents are imported, which means that the suffix contents are replaced and new index files are created.
You can use the web interface Directory Service Control Center (DSCC) to perform this task.
Set the suffix to read-only, as described in Setting Referrals and Making a Suffix Read-Only.
Export the entire suffix to an LDIF file, as described in Backing Up to LDIF.
Import the same LDIF file to reinitialize the suffix, as described in Importing Data From an LDIF File.
During the initialization, the suffix is unavailable. When the initialization is complete, all configured indexes are ready to be used.
Make the suffix writable again, as described in Setting Referrals and Making a Suffix Read-Only.
Browsing indexes are special indexes used only for search operations that request server-side sorting of results. The Reference for Oracle Directory Server Enterprise Edition explains how browsing indexes work in Directory Server.
Customized browsing indexes for sorting client search results must be defined manually. To create a browsing index, or virtual list view (VLV) index, use the following procedure. This section also includes procedures for adding or modifying browsing index entries and for regenerating browsing indexes.
You can use the web interface Directory Service Control Center (DSCC) to perform this task.
Add new browsing index entries or edit existing browsing index entries by using the ldapmodify
command.
For instructions, see To Add or Modify Browsing Index Entries.
Run the dsconf reindex
command to generate the new set of browsing indexes to be maintained by the server.
For instructions, see To Regenerate Browsing Indexes.
A browsing index is specific to a given search on a given base entry and its subtree. The browsing index configuration is defined in the database configuration of the suffix that contains the entry.
Configure the vlvBase
, vlvScope
, and vlvFilter
attributes for each browsing index on a directory server.
These attributes configure the base of the search, the scope of the search, and a filter for the search. These attributes use the vlvSearch
object class.
Configure the vlvSort
attribute for each browsing index.
This attribute specifies the name of the attribute or attributes that sort the index. This entry is a child of the first entry and uses the vlvIndex
object class to specify which attributes to sort and in what order.
The following example uses the ldapmodify
command to create the browsing index configuration entries:
$ ldapmodify -a -h host -p port -D cn=admin,cn=Administrators,cn=config -w - Enter bind password: dn: cn=people_browsing_index, cn=database-name, cn=ldbm database,cn=plugins,cn=config objectClass: top objectClass: vlvSearch cn: Browsing ou=People vlvBase: ou=People,dc=example,dc=com vlvScope: 1 vlvFilter: (objectclass=inetOrgPerson) dn: cn=Sort rev employeenumber, cn=people_browsing_index, cn=database-name,cn=ldbm database,cn=plugins,cn=config objectClass: top objectClass: vlvIndex cn: Sort rev employeenumber vlvSort: -employeenumber ^D
The vlvScope
is one of the following:
0
for the base entry alone
1
for the immediate children of the base
2
for the entire subtree rooted at the base
The vlvFilter
is the same LDAP filter that is used in the client search operations. Because all browsing index entries are located in the same place, you should use descriptive cn
values to name your browsing indexes.
Each vlvSearch
entry must have at least one vlvIndex
entry. The vlvSort
attribute is a list of attribute names that defines the attribute to sort on and the sorting order. The dash ( -
) in front of an attribute name indicates reverse ordering. You can define more than one index for a search by defining several vlvIndex
entries. With the previous example, you could add the following entry:
$ ldapmodify -a -h host -p port -D cn=admin,cn=Administrators,cn=config -w - dn: cn=Sort sn givenname uid, cn=people_browsing_index, cn=database-name,cn=ldbm database,cn=plugins,cn=config objectClass: top objectClass: vlvIndex cn: Sort sn givenname uid vlvSort: sn givenname uid ^D
To modify a browsing index configuration, edit the corresponding vlvSearch
entry or the corresponding vlvIndex
entry.
To remove a browsing index so that the browsing index is no longer maintained by the server, remove the individual vlvIndex
entries.
Alternatively, if only one vlvIndex
entry exists, remove both the vlvSearch
entry and the vlvIndex
entry.
After you have created the browsing index entries, generate the new browsing indexes for the attributes specified.
$ dsadm reindex -l -t attr-index instance-path suffix-DN
The command scans the directory contents and creates a database file for the browsing index.
The following example generates the browsing index that you defined in the previous section:
$ dsadm reindex -l -b database-name -t Browsing /local/dsInst \
ou=People,dc=example,dc=com
For more information about the dsadm reindex
command, see the dsadm man page.