Chapter 11 The ldclt Stress Test Tool

The ldclt tool is a powerful and flexible client for creating and running stress tests on Lightweight Directory Access Protocol (LDAP) directory servers. The tool generates LDIF files and performs searches, additions, modifications, deletions, or renaming using complex randomizing capabilities to simulate real usage conditions. This chapter provides instructions on how to use the ldclt tool. It contains the following sections:

Overview

The ldclt (LDAP client) is a multithreaded application developed using the Sun™ ONE LDAP SDK for C. It is designed to be used in automated tests on a directory; its command-line interface is extensive and suitable for scripting. This will allow you to run complex tests automatically and compare performance over time for reliability. The DSRK and its updates include the latest version of the tool in the DSRK_base/bin/dsrk52 directory.


Note	To insure that scripts can be easily maintained, the ldclt tool continues to provide a high degree of backward compatibility.

ldclt uses a minimum of 8 MB of memory and may use much more depending upon the chosen options and operations. If necessary, it sets its own resource limits according to the options and data it processes from the directory.


Note	See the man pages for ulimit and getrlimit for more information.

The ldclt tool continuously displays its performance measurements on the standard output. By default, ldclt will display performance statistics cumulated over all threads every 10 seconds. And every 15 minutes (and also upon exit), the tool will display a cumulative count of operations performed (global statistics) and errors encountered. (See the -e noglobalstats option as detailed in General Options to suppress the global statistics output.) Several error management options allow the tool to handle most errors without interrupting the test. Fatal errors will be reflected in the tool’s exit status.

Command Usage

The ldclt tool performs repeated operations on a designated directory server and measures the average completion time for requests. The tool is multithreaded and can be configured to perform high load testing. All threads loop continuously to perform the same LDAP operation repeatedly but with different input values each time. The possible operations are binds, searches, additions, modifications (including RDNs whose modifications are also referred to as reparenting), and deletions.

Syntax

Options


Note	The ldclt tool is highly configurable through its command-line options, and it is rarely used in the simple forms presented here. See Sample Output, for an explanation of the options for controlling each feature and their possible combinations.

The many ldclt tool options allow you to randomize the values of most LDAP operation parameters in order to simulate realistic client requests. They also allow you to control how often, and for how long, multiple threads will run these operations, simulating a given load on your directory server. The ldclt tool has four types of options:

The following sections detail these options. The ldclt -H, -h, or -? options when run on the command-line will display brief descriptions of all the command-line options.

General Options


Tip	The ldclt tool has a special -e option for specifying multiple execution parameters. The -e option appears a number of times in the following tables to describe these parameters separately. When using the -e option on the command-line, you may specify its parameters in a comma separated list, for example: ldclt -e noglobalstats,delayedstartup=10,counteach

The general options listed in Table 11-1 control multi-threading, execution time, error handling and output.

Table 11-1 General Options for ldclt
Option	Parameter	Purpose
-h	hostname	Specify the hostname or IP address of the directory server. When this option is omitted, the default is the local host. The ldclt tool supports full IPv6 addressing.
-p	port number	Specify the port number for accessing the directory server host. The default is 389 when this option is omitted.
-n	number of threads	Specify the number of threads that will perform LDAP operations in parallel. The output displays the average performance of all threads combined. The default number of threads is 10. The maximum number is 4000, although most systems are limited to less by the maximum threads and file descriptors per process.
-e	number of seconds	Specify the delay time in seconds between the startup of each thread. By default, all threads start at the same time. With this parameter, each thread will sleep for threadNumber x seconds before starting.
-a	number of concurrent operations	Run the tool in asynchronous operation mode, with each thread having at most maxPending concurrent operations. A pending operation is one whose results have not yet been read from the server. Each thread will also maintain a minimum of maxPending / 2 concurrent operations.
-N	number of intervals	Specify the total number of intervals, and therefore the total time, for which the tool should run. For example, specify -N 360 to run for one hour with the 10 second intervals. By default, there is no limit to the number of intervals, and the tool will run indefinitely.
-T	number of operations	Specify the total number of operations, cumulated across all threads, to perform before exiting. The tool will exit after the interval where this number of operations is reached or exceeded, thus more than the total operations may be performed. By default, there is no limit to the number of operations, and the tool will run indefinitely.
-W	number of seconds	Specify the waiting time, in seconds, between each operation of each thread. By default there is no waiting time (0 seconds).
-t	number of seconds	Specify the server-side time-out period, in seconds, for all LDAP operations performed. When this option is omitted, the default is 30 seconds.
-e	dontsleeponserverdown	By default, an ldclt thread will sleep 1 second if it receives an error indicating that the server is down (code 81 or 91). This parameter will prevent threads from sleeping, allowing them to loop and repeatedly attempt operations. Due to the built-in resource management, the ldclt tool may use 100% of all CPU time and block the client machine when using this parameter.
-i	number of intervals	Specify the number of inactive intervals for a thread before a warning message is sent. A thread is inactive or starving if, for whatever reason, it performs no operation during an interval. By default, a warning message is sent on the interval after 3 inactive intervals (40 seconds total).
-E	number of allowed error occurrences	Specify the maximum number of allowed occurrences of any one error, and exit after the interval when that number is reached or exceeded. When this option is omitted, the default is 1000. This option only applies to errors not ignored by the -I option.
-I	numerical error code	Specify a numerical error code to ignore when returned by an operation. This option can be used multiple times, once for each error to ignore. See Error Handling for more information.
-e	counteach	Specify that the statistics include all operations attempted. For example, even a search that failed because a random value was not found will be reported. By default, only operations that succeed are counted. With this option, the output will more accurately represent thread activity and server response. The following errors will be counted: When performing searches: LDAP_NO_SUCH_OBJECT (code 32) When performing additions: LDAP_ALREADY_EXISTS (code 68) When performing deletions: LDAP_NO_SUCH_OBJECT (code 32) When renaming entries: LDAP_ALREADY_EXISTS (code 68) or LDAP_NO_SUCH_OBJECT (code 32) or LDAP_PROTOCOL_ERROR (code 2)
-e	v2	Perform all operations using LDAP v2 protocol version. LDAP v3 is the default; specify LDAP v2 only when connecting to servers that do not support v3.
-e	noglobalstats	Suppress the periodical global statistics output. By default, the global statistics will be printed every 90 intervals. See Sample Output for more information.
-v		(Small v) Specify a verbose output, which provides additional information about the operations that ldclt is performing. This option may be used in conjunction with -q to display a useful report of tool activity without too many error messages.
-V		(Capital V) Specify a VERY verbose output, which shows debugging information about the operations that ldclt is performing. Specify this option with -v, but not -q or -Q, to display all possible output.
-q		(Small q) Specify a quiet output. None of the bind errors will be reported, and intermediate node creation messages are not displayed. Use with the -I option to suppress output of ignored errors.
-Q		(Capital Q) Specify a VERY quiet output. Bind errors are not reported, intermediate node creation messages are not displayed, and all messages regarding thread activity and asynchronous, pending requests are suppressed.
-H		Display the usage help text that briefly describes all options.

Bind Options

The options in Table 11-2 determine how the ldclt tool will bind to the directory server and possibly establish a secure connection.

Table 11-2 Bind Options for ldclt
Option	Parameter	Purpose
-D	bind DN	Specify a bind DN for accessing your directory, usually in double quotes ("") for the shell. The bind DN should have sufficient access permissions for the test operations you wish to perform. If the bind DN and password are omitted, the tool will use anonymous authentication: it will not bind before performing operations. Usually, only search operations may be performed anonymously. To use a different bind DN with every operation, use X placeholders in the bindDN and specify the -e randombinddn options. For more information, see Random Binding.
-w	password	Specify the password for the bind DN. The password may also include X placeholders to include the same randomly generated value that is substituted into the corresponding bind DN.
-e	randombinddn	Use this option to replace X placeholders in the bind DN and password options with a random value. You must specify a range of random numbers with the randombinddnlow and randombinddnhigh options detailed in the following table cells.
-e	randombinddnlow=integer	Specify the low end of the possible range for random integers to be substituted into the bind DN and password options.
-e	randombinddnhigh=integer	Specify the high end of the possible range for random integers to be substituted into the bind DN and password options.
-e	randombinddnfromfile=filename	Specify the full path and filename to a file that contains bind DN and password pairs. The bind credentials will be randomly selected from this file each time the tool binds to the server. (See Bind Operations for more information.) Do not use -D, -w, and -e randombinddn with this option.
-e	referral=on\|off\|rebind	Specify the binding policy when following referrals: on - Follow referrals using anonymous binding; this is the default when this option is omitted. off - Do not follow referrals. rebind - Follow referrals by rebinding with the same credentials.
-e	smoothshutdown	Force each thread to finish the operation in progress and unbind before terminating for whatever reason. With this option, the tool will wait up to 20 seconds for operations to finish. Without this option, pending operations and connections will be dropped upon any forced exit.
-e	bindeach	Specify that the tool will rebind for each operation it performs. When using this option, the performance measurements in the output include the time required to establish the connection, bind to the directory and perform the given operation. This behavior simulates many clients performing quick operations.
-e	close	Specify how the tool will disconnect from the server. With this option, the tool will simply close the connection without unbinding. By default, the tool will perform the LDAP unbind operation. This option is often used with -e bindeach to simulate connections being dropped by clients.
-Z	path to certificate database file directory	Establish secure communications using SSL (Secure Socket Layer) client authentication for all operations. This allows you to test the performance of your server over SSL. The certPath parameter specifies the directory containing the cert7.db certificate database file. (See SSL Authentication.) You must also specify the bind DN and password options, as well as the following three parameters.
-e	cltcertname=certificate name	Specify the name of the certificate to be found in the directory given by the -Z option. This certificate will be used to identify the client when establishing an SSL connection with the server.
-e	keydbfile=keyDBfilename	Specify the full path and filename of the key database on the client machine. This file should only be used for testing because the corresponding password is exposed on the command-line.
-e	keydbpin=key database password	Specify the password needed to access the key database file given in the previous option. CAUTION: This option exposes the password to the SSL key database on the command-line. You should use a special certificate and key database, used only for testing purposes, to avoid revealing your actual password.

Target Object Options

The options in Table 11-3 determine the target object, either its base DN, its RDN (relative DN), its attribute values, or any combination thereof. Use these options to specify either specific or random entries and create random attribute values, depending upon the type of operation being performed. For example, you may want to add 100 sequentially-named entries with random contents or delete random entries (RDNs) under a given base DN. See Sample Output, for more information.

Table 11-3 Target Object Options for ldclt
Option	Parameter	Purpose
-b	base DN	Specify the base DN or suffix for all operations that require one, usually in double quotes ("") for the shell. To use a different base DN with every operation, use X placeholders in this option and specify the following -e randombase option. For more information see Random Base DNs.
-e	randombase	Use this option to replace X placeholders in the base DN with random values. You must specify a range of random numbers with the randombaselow and randombasehigh options detailed in the following two table cells.
-e	randombaselow=integer	Specify the low end of the possible range for random integers to be substituted into the base DN option.
-e	randombasehigh=integer	Specify the high end of the possible range for random integers to be substituted into the base DN option.
-f	patternString	Specify a filter string for search operations or the RDN pattern for other operations. The patternString has the format attribute=value, and the value may contain X placeholders for any of the substitution features described by the following options. See Random Filters and RDNs for further details. CAUTION: This option is mutually exclusive with -e rdn=patternString.
-e	incr[,noloop]	Activate the incremental counter feature to replace X placeholders with sequential integers. Use with the -r and -R options to specify the starting and ending values of the counter, respectively. Specify the noloop option to count the range only once. In this case, the corresponding thread will die upon reaching the upper bound.
-e	commoncounter	Specify that all threads use and increment the same thread-safe counters. This option applies to all counters in use, such as those for patternString substitutions and those for template substitutions.
-e	random	Activate the random value substitution feature to replace X placeholders. Specify the -r and -R options to define the range of random integers for substitution. Alternatively, use with the -e string[,ascii] options to substitute random strings.
-r	integer	Specify the lower bound of the incremental counter or random value range.
-R	integer	Specify the upper bound of the incremental counter or random value range.
-e	string[,ascii]	Activate the random string substitution feature to replace X placeholders. This option requires the -e random option. By default, UTF-8 characters will replace each of the X placeholders. Use the ascii option to generate only 7-bit characters (hexadecimal values less than or equal to 0x7f).
-e	rdn=’patternString’	The patternString may specify either a filter pattern for search operations or an RDN pattern for other operations. The patternString has the format attribute:value, and instead of using X placeholders, the value uses a special syntax described in Random Filters and RDNs. Use single quotes (’’) to use this special syntax with the shell. This option is mutually exclusive with the -f, -e incr[,noloop], -e random, -r, -R, and -e string[,ascii] options. However, it requires the -e object=filename option with a valid template file, even if this file is not used to generate entries.
-e	object=filename	Specify the path and name of a file containing an entry template for generating random entries (see Random Entry Generation). Requires the -e rdn=patternString option.

Operation-specific Options

The options in Table 11-4 are operation-specific options. All of the options for specifying operations, namely -e bindonly, -e esearch, -e add, -e attreplace, -e delete, -e rename, and -e genldif are mutually exclusive. See the corresponding section under Sample Output, for more information about each operation.

Table 11-4 Operation-Specific Options for ldclt
Option	Parameter	Purpose
-e	bindonly	Specify that only bind and unbind operations be performed repeatedly, with no other intervening LDAP operations. The -e close option will simulate client disconnects without unbinding. See Bind Operations for more information.
-e	esearch	Perform searches with -f patternString or -e rdn=patternString as the filter. In this release of the tool, searches with large results are very inefficient and should be avoided. Use wildcards or operators other than equality only if they return few entries. This restriction will be fixed in future versions. See Searches for more information.
-s	scope	Specify the scope of search operations. The scope parameter may be one of the following: 1. base - Search only the base entry itself. 2. one - Search only the children of the base entry. 3. subtree - Search the base entry and all its descendants. This is the default if the -s option is omitted.
-e	attrlist=attribute[:attribute]...	Specify the type names of the attributes to retrieve for entries in the search results. When this option is omitted, all attributes will be retrieved by default.
-e	randomattrlist=attribute[:attribute]...	Specify the type names of possible attributes to retrieve for entries in the search results. For every search, the tool will randomly select one of these attributes to be retrieved for all entries in the results.
-e	attrsonly=0\|1	Specify whether attribute values are returned (0) or not (1), along with the attribute names of the entries in the search results. The ldclt tool will use less memory when attribute values are not returned. By default, attribute values are returned (0).
-e	add	Perform LDAP add operations, using DNs determined by the base DN and by the RDN patternString of either the -f or -e rdn options. You must also specify the one of the following object class template options or create a custom template in a file given by the -e object parameter. See Adding Entries for more information.
-e	person\|emailperson\|inetorgperson	Specify one of the predefined object class templates for adding or generating entries. See Random Entry Generation for more information. Both emailperson and inetorperson require the following imagesdir option.
-e	imagesdir=path	Specify the full path of the directory containing JPEG image files. Only files with the .jpg extension in this directory will be used. The ldclt tool will take the files sequentially from this directory when generating emailperson and inetorperson entries. The files are interpreted as binary image files but their actual contents is unimportant. Files may be as large as needed to test your directory.
-e	genldif=filename[,append]	Specify the full path and name of a file to create with the generated entries in LDIF format. This operation is similar to an addition and uses the same options, except the new entries are written to the file (see Random Entry Generation). Unless you specify the append option, the tool will exit with an error if the file exists.
-e	attreplace=attribute:value	Perform LDAP modify operations on entries determined by the base DN and the RDN of the -f patternString option. Specify the attribute to modify and a value string containing X placeholders: the tool will replace each placeholder with a random character with every operation (no other options are needed). See Modify Operations.
-e	delete	Perform LDAP delete operations on entries determined by the base DN and the RDN patternString of either the -f or -e rdn options. See Delete Operations for more details.
-e	rename[,withnewparent]	Perform LDAP modRDN operations on entries determined by the base DN and the RDN patternString of either the -f or -e rdn options. The new RDN will be determined from the same patternString which therefore cannot rely on incremental value substitutions. Specify the withnewparent option to simulate a reparent operation if you are also using the random base DN options. See modRDN Operations for more details.

Exit Status

By default, the ldclt tool will run indefinitely, performing operations repeatedly. It will only exit in one of the following conditions:

Upon termination, the ldclt tool will return one of the following values to the shell. When possible, the display will provide further information about any errors.

Table 11-5 Exit Status of ldclt
Exit Status	Explanation
0	No problem during execution.
1	Serious error that should not occur (please report to idsrk@Sun.COM).
2	Error in the parameter usage.
3	Maximum limit of LDAP errors reached (see Exit Status 3).
4	Unable to bind to the directory server.
5	Cannot load libssl for secure binding.
6	Multithreading error (mutex problem).
7	Initialization problem, for example, if the tool cannot open or create a file.
8	Resource limitation such as a malloc error for memory.
99	Unknown error (please report to idsrk@Sun.COM).

Error Handling

The ldclt tool handles errors that occur as a result of LDAP operations. Each thread handles errors individually: for example, a connection error will cause only that thread to die. Some errors are expected when performing hundreds and thousands of operations with randomly generated parameters. Operational errors are not fatal, and each thread will continue performing operations with new parameters.

Allowing Error Occurences


Tip	See Troubleshooting for more information about specific errors and how to avoid them.

The tool will count the occurrences of each error code, and by default, it will terminate only if any one error code occurs more than 1000 times. This behavior allows occasional errors but detects server conditions or test configurations that will probably not give valid results. You may use the -E option to set a different occurrence limit, for example, to detect all errors when developing tests. See Table 11-1 for more information.

Ignoring Error Occurences

If you always expect a given error to occur, use the -I option to ignore it in all threads. This error will be exempt from the error occurrence limit and it will not be displayed if the -q option is specified. For example, specify -I 32 (LDAP_NO_SUCH_OBJECT) when performing random delete operations or -I 68 (LDAP_ALREADY_EXISTS) when performing random add operations.

Normally, connection errors are fatal to the thread in which they occur. The -I option may be used to ignore connection errors 81 (LDAP_SERVER_DOWN) and 91 (LDAP_CONNECT_ERROR) when they occur during a bind. In this case the thread will attempt to rebind instead of exiting. However, a connection error that occurs during an operation is always fatal.


Note	This tool will trap the following signals: SIGINT (^C or kill -2): display the global statistics report and exit the tool normally. SIGQUIT (^\ or kill -3): display the current statistics but do not exit.

Sample Output

Performance Statistics

The average operation rates in Code Example 11-1 provide the following information (from right to left):

By default, the statistics only take into account successful operations. This may misrepresent client and server activity when performing operations that may fail naturally; for example, when performing delete operations with random relative distinguished names (RDNs). Use the -e counteach option as detailed in General Options to count all operations in the statistics. In the asynchronous mode (the -a option also detailed in General Options), all operations are counted by default.

Global Statistics

The global statistics give the cumulated operation rates and the error counts for the entire elapsed runtime of the tool. In addition, when running the tool in asynchronous mode (again, the -a option detailed in General Options), the global statistics will include the status of pending operations for all threads. The global statistics detailed in Code Example 11-1 provide the following information:


Note	The output may also include the number of threads that have died, indicating either the end of a non-looping counter or a connection problem

You may request the global statistics at any time using the SIGQUIT signal, available on UNIX® systems via the ^\ (Ctrl-Backslash) key sequence. However, this may interrupt the LDAP operation in one or more threads and impact the current interval statistics.


Note	The ldclt statistics are not absolute measurements due to the counting mechanism and the overhead of interval timing. Measurements are more accurate for faster machines. However, unless the ldclt tool is running on an overloaded machine, it can submit more operations than a server can process. Therefore, it provides accurate overall results about the number of operations per second.

Command-Line Examples

The large number of options for the ldclt tool allows you to configure the behavior for many advanced features. This section explains some of these features and how the options interact to implement the desired behavior. Many of these examples show how to randomize values such as the bind DN or the target DN with each operation. Randomization simulates realistic usage of a directory server in performance tests. You should adjust the command-line parameters so that the range of randomization values most closely resembles your expected directory usage. Other examples of ldclt usage can be found in DSRK_base/examples/ldclt.

Bind Operations


Note	Due to requirements for backwards compatibility, the ldclt tool has several mechanisms for generating randomized values.

The following sections describe how to use these options detailed in Table 11-2 to randomize the bind credentials or to use secure binding.

Random Binding

Random binding uses different bind DN and password credentials to perform simple authentication with every LDAP bind operation. In order to use a different, numeric bind DN with every operation, you must specify the -e randombinddn options and use X character placeholders in the -D bindDN and -w password parameters. The same random value will be substituted into both the placeholders (for example, cn=m123 and passwd123). Random binding is specified with the -e bindeach option.


Caution	The capital letter X cannot be escaped for normal usage.

In the following example, each thread of the command will bind and unbind (-e bindeach,bindonly) using any one of the 100,000 possible bind credentials:

$ ldclt -h host -p port -e bindeach,bindonly \
-D "uid=XXXXX,dc=example,dc=com" -w testXXXXX \
-e randombinddn,randombinddnlow=0,randombinddnhigh=99999

The values substituted into the bind DN are generated independently of other random substitutions into base DNs, filters, or RDNs. See Random Filters and RDNs for more information.


Caution	When using random bind credentials, your test directory should be initialized with entries corresponding to the possible bind DNs and passwords that can be generated. Using random strings in the bind credentials may result in bind errors if a DN does not exist. Specify the -I 49 option to make the tool ignore error 49, LDAP_INVALID_CREDENTIALS. A test that uses this option will measure a mix of bind successes and failures. See Adding Entries for ldclt examples that will add all entries in a sequence of RDNs.

Alternatively, specify the -e randombindfromfile option to use bind credentials defined in a text file. The file must contain one bind DN and password pair per line, separated by any number of tabs. Leading and trailing space characters are significant in both the DN and password. As shown in the following example, the -D and -w options are not needed in this case:

$ ldclt -h host -p port -e bindeach,bindonly \
-e randombinddnfromfile=/home/user/testbind.txt

Using random bind credentials from a file allows you to write or generate DNs and passwords that test special conditions in your directory. For example, you might want to test search operations involving special ACIs (access control instructions) that apply to the DNs specified in the file.

SSL Authentication

The ldclt tool supports client authentication over SSL in order to perform tests using secure binding. The following example shows the options required:

$ ldclt -h host -p port -e bindeach,bindonly -Z certPath \
-e cltcertname=certName,keydbfile=fileName,keydbpin=password

The certPath and certName determine the certificate that will identify the client, and the fileName and password provide access to the client’s keys. Because the password to the key database is exposed on the command-line, we recommend using a special key database that is only used for testing purposes.

Searches


Caution	No randomization is possible with client authentication over SSL, therefore all threads will bind with the same certificate every time. However, you can still use the -e bindeach option to measure SSL binding and unbinding with every operation.

The numerous options for performing searches allow you to randomize what DN is being sought, where to look for it, and what to retrieve upon finding it. With many threads performing many different searches, the ldclt tool can simulate a realistic load on your directory server and measure performance under deployment-like conditions. However, ldclt does not verify or display any search results, which are discarded immediately.

Anonymous Searches

If you do not specify any bind credentials (as shown in the following command), searches may be performed anonymously:

$ ldclt -h host -p port -b "ou=people,dc=example,dc=com" \
-f uid=XXXXX -e esearch,random -r0 -R99999 \
-I 32 -e randomattrlist=cn:sn:uid:ou:l:fax:mobile

This example performs searches for a random UID in entries below the "ou=people,dc=example,dc=com" base DN. Base DNs may be randomized in searches, using the options described in Random Base DNs. This example also uses a randomized filter string to perform a different search and retrieve a different entry for every search. The syntax for this feature is described in Random Filters and RDNs. The -I 32 option will ignore code 32 (LDAP_NO_SUCH_OBJECT) errors that occur often with random searches. If the search is successful, it will return only the value of a randomly chosen attribute (-e randomattrlist=cn:sn:uid:ou:l:fax:mobile).

Wildcard Searches


Note	As with other operations, you may also perform randomized binding with every search operation.

Because the ldclt threads perform synchronous LDAP operations by default, you should avoid searches using wildcards that return large numbers of entries. Depending on the size of your test directory, a search such as -f cn=* will block each thread for long periods of time, and each thread will receive search results containing the entire directory. This search will require large amounts of memory for each thread, possibly more than a total of 1 GB in certain cases.

Random Filters and RDNs


Note	The current version of the ldclt tool is optimized for doing exact searches that match a single entry.

This section demonstrates exact searches, where the filter strings match only one entry. These filters are similar to RDNs (relative DNs) that determine a single entry under a given base DN. The ldclt tool has two interchangeable mechanisms, each of which can specify filter strings and RDNs:

The -f patternString and -e -rdn=patternString have different syntax but you may use either one for any operations of the ldclt tool. You may use the -rdn option to define a filter string for searching or the -f option to define an RDN for another operation such as addition, as long as the result of each randomized substitution is valid for the respective operation. However, the two options are mutually exclusive on the command-line.

-f patternString Parameter


Note	The -f patternString syntax is more limited and kept for backwards compatibility. The -e rdn=patternString format is more flexible and will be further expanded in the future.

The -f patternString parameter has the form attribute=value and the value may contain X characters as placeholders for random substitutions. Only the first, left-most sequence of X characters is replaced. (The capital letter X cannot be escaped for normal usage.) The following example uses the random filter string syntax to search for entries based on the mail attribute.

$ ldclt -h host -p port -b "ou=people,dc=example,dc=com" \
-f mail=XXXXX@example.com -e esearch,random -r0 -R99999 \
-e attrlist=cn:telephonenumber

Table 11-6 -f patternString Parameter Options
Option and Parameter	Description Of Operation
-e random -rlow -Rhigh	Each X placeholder will be replaced with a digit of a randomly chosen number between low and high, inclusive. Numbers are padded to the left with zeros to fill all placeholders, and you should specify as many placeholders as digits in the largest number. If there are fewer placeholders than digits, the substitution will truncate the high digits.
-e incr[,noloop][,commoncounter] -rlow -Rhigh	Each X placeholder will be replaced with a digit of the current counter value. Numbers are padded to the left with zeros to fill all placeholders, and you should specify as many placeholders as digits in the largest number. High digits will be truncated if there are not enough placeholders. The counter is incremented after every operation, and it increases from low to high. By default each thread has a separate counter and will loop through all values indefinitely. Specify the noloop option to enumerate the counter values once and stop the thread afterwards. With the commoncounter option, all threads will increment the same counter and never substitute the same value.
-e random,string[,ascii]	Each X placeholder will be replaced with an arbitrary UTF-8 character to generate a random string. Use the ascii option to generate only 7-bit characters (hexadecimal values less than or equal to 0x7f). Both UTF-8 and ASCII strings will be valid LDAP strings, including any escape characters that may be needed for special characters.

-e -rdn patternString Parameter

The -e -rdn=patternString parameter has the form attribute:value and the value may contain keyword placeholders for random substitutions. The following example uses the RDN keyword syntax to perform exactly the same type of searches as the filter string example used in -e -rdn patternString Parameter. Use single quotes (’’) for the special syntax of the keywords in the shell.

$ ldclt -h host -p port -b "ou=people,dc=example,dc=com" \
-e rdn=’mail:[RNDN(0;99999;5)]@example.com’ \
-o object=foo.txt -e attrlist=cn:telephonenumber

The keywords determine the type of substitution and are replaced according to the rules detailed in Table 11-7.

Table 11-7 Substitution Keywords for -e -rdn patternString Parameter
Parameter	Description
[RNDN(low;high;length)]	This keyword will be replaced with a randomly chosen number between low and high, inclusive. Numbers are padded to the left with zeros or truncated on the left to have the exact length of digits.
[RNDFROMFILE(filename)]	This keyword will be replaced with a line chosen randomly from the file with the given filename. Each line of the file is substituted in whole, including any leading or trailing whitespace.
[INCRN(low;high;length)] or [INCRNNOLOOP(low;high;length)]	This keyword will be replaced with the current value of a counter increasing from low to high, inclusive. Numbers are padded to the left with zeros or truncated on the left to have the exact length of digits. Each use of the keywords will represent a separate counter. In the first form, the counter will loop indefinitely, and in the second form, it will stop upon reaching the high value. Each thread will use a separate counter, or you may use the -e commoncounter option to use a single counter per keyword for all threads.
[INCRNFROMFILE(filename)] or [INCRNFROMFILENOLOOP(filename)]	This keyword will be replaced with a line chosen sequentially from the file with the given filename. Each line of the file is substituted in whole, including any leading or trailing whitespace. In the first form, the substitution will read through the file repeatedly, and in the second form, it will stop upon reaching the end of the file. Each thread will read the same file separately, or you may use the -e commoncounter option to read each line only once.
[RNDS(length)]	This keyword will be replaced with a string of the given length of randomly chosen ASCII characters. Random strings will be valid LDAP strings, including any escape characters that may be needed for special characters.

One limitation of the -e rdn=patternString option is that it must be used with a valid template file given by the -o object=filename option. This template file is needed by the randomization mechanism (Random Entry Generation) but has no effect if not required by the operation. The file foo.txt in this example contains the following minimal, valid template:

Regardless of the format, search filters in performance tests usually include random numbers so that search operations are “scattered” over the directory. As we shall see in the following sections, additions, deletions and modify operations rely more often on counters and random strings to target specific entries.

Adding Entries

The entry creation functionality of the ldclt tool can be used to test the performance of the add operation and to populate a database for other tests. The command-line options provide many opportunities to randomize and customize the entries that are added to your directory. The following command is a simple example for entry addition:

$ ldclt -h host -p port -D "cn=Directory Manager" -w password \
-f cn=MrXXXXX -b "ou=people,dc=example,dc=com" \
-e add,person,incr,noloop,commoncounter -r0 -R99999

Performing additions requires write privileges in your directory, so you should specify the bind credentials of a user that can create the entries under the given base DN. This example will create 100,000 sequentially numbered entries. All threads use a common counter, so no two will try to create the same entry. This command will create simple person entries as reproduced in Code Example 11-2.

The following sections demonstrate how to randomize the base DN and customize the generated entry.

Random Base DNs

When adding entries to a directory, you may want to place them under different suffixes to simulate real usage. To do this, the ldclt tool supports randomized base DNs in add operations; the same functionality is also available in modify and delete operations.

To use a different, numeric base DN with every operation, specify the -e randombase options and use X character placeholders in the -b baseDN parameter. Only the first, left-most sequence of X characters is replaced. (The capital letter X cannot be escaped for normal usage.) For example, each thread of the following command will create person entries in any one of the 1,000 possible department base DNs:

$ ldclt -h host -p port -D "cn=Directory Manager" -w password \
        -b "ou=DeptXXX,dc=example,dc=com" \
        -e randombase,randombaselow=0,randombasehigh=999 \
        -f cn=MrXXXXX -e add,person,incr,noloop -r0 -R99999

The values substituted into the base DN are generated independently of other random substitutions into bind DNs, filters, or RDNs. See Random Filters and RDNs for more information.

Create Missing Nodes

When creating entries with random base DNs, the tool has the ability to automatically create the corresponding nodes if they do not already exist. This allows you to populate large directories of any depth with the add operation. The randomly named nodes are also created during rename operations. See modRDN Operations for more information.

In order to create node entries automatically, the naming attributes of the base DN must be cn, o or ou. Depending on the naming attribute, the ldclt tool will create intermediate nodes of the following object classes:

For example, the add operation may try to create the entry with the randomized DN cn=Mr00007,ou=Dept123,dc=example,dc=com. Assuming the base DN does not exist in the directory yet, the operation will first create the following node:

Random Entry Generation

Just as there are two ways to specify filters and RDN strings, there are two different mechanisms to define the entries that the tool will add to the directory. The simpler method seen in previous examples creates simple entries from fixed templates. The more powerful method allows you to specify the object class template and perform many random substitutions.


Caution	Due to the implementation of the missing node mechanism, the entry that triggers the creation of a missing node is not created. In the above example, the DN cn=Mr00007,ou=Dept123,dc=example,dc=com will not be added. Only subsequent entries in this node will be added, now that the node exists.

$ ldclt -h host -p port -D "bindDN" -w password \
-f cn=patternString -b "baseDN" \
-e add,person [randomizingOptions]

The created entries will have the contents detailed in Code Example 11-3, with any randomizations substituted into the patternString and baseDN.

To generate simple inetorgperson entries, use the following command:

$ ldclt -h host -p port -D "bindDN" -w password \
-f cn=patternString -b "baseDN" \
-e add,inetorgperson,imagesdir=Path [randomizingOptions]

The created entries will have the contents detailed in Code Example 11-4, with any randomizations substituted into the patternString and baseDN.

Specifying the emailperson option will generate entries that are identical to the inetorgperson entries, except for the value of the objectclass attribute. In all cases, the patternString must use either sn (surname) or cn (common name) as the naming attribute. If sn is used, cn will contain the value toto cn.

More complex entries can be created from custom templates with the -e rdn=patternString,object=filename options. The template file provides all of the attributes for the entry with the following syntax:

The keywords are the same as defined in Random Filters and RDNs. Variables definitions should be used in the RDN patternString so that the same values can be used as the naming attributes inside the template file. For example, the following command and template file will generate 100,000 sequentially numbered entries:

$ ldclt -h host -p port -D "bindDN" -w password \
        -b "ou=people,dc=example,dc=com" -I 68 \
        -e object=inet.txt,rdn=’uid:[A=INCRNNOLOOP(0;99999;5)]’ \
        -e add,commoncounter

The associated template file, inet.txt (Code Example 11-5), uses the variable A and other keywords to create a realistic entry:

Code Example 11-5 inet.txt File


# inet.txt - template for a more complete inetOrgPerson
#
objectclass: inetOrgPerson
sn: [B=RNDFROMFILE(installDir/data/dbgen-FamilyNames)]
cn: [C=RNDFROMFILE(installDir/data/dbgen-GivenNames)] [B]
password: test[A]
description: user id [A]
mail: [C].[B]@example.com
telephonenumber: (555) [RNDN(0;999;3)]-[RNDN(0;9999;4)]
jpegPhoto:: /9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAoHBwkHBgoJCAkLCwoM
DxkQDw4ODx4WFxIZJCAmJSMgIyIoLTkwKCo2KyIjMkQyNjs9QEBAJjBGS0U+Sjk
/QD3/2wBDAQsLCw8NDx0QEB09KSMpPT09PT09PT09PT09PT09PT09PT09PT09PT
09PT09PT09PT09PT09PT09PT09PT09PT3/wAARCAAiACIDASIAAhEBAxEB/8QAG
gAAAgMBAQAAAAAAAAAAAAAAAAYDBAUBB//EACoQAAIBAwMDAgYDAAAAAAAAAAEC
AwQFEQASIQYTMRVBFBYiI1FhBxdx/8QAGQEAAwADAAAAAAAAAAAAAAAAAAQGAQM
F/8QAJhEAAQMCBAYDAAAAAAAAAAAAAQACEQMEBRIhQRMxYXGh8BQikf/aAAwDAQ
ACEQMRAD8A9Xul0W3okccffrJsiCANjdjGWY87UGRlsHGQACxVSuyx1MkNO3Ud8
eKSdlQU9JOaSHvEcLGykSsfPDOQTk7RwFloEjr71cru6K0hmaigdlG9IYm2Mn+G
VZW/JBXPgAKv8o9IXTqv0j0tIm7EkiS7327A+36v2BtOcZPIwDqRvsTfXvPitqc
NgkE7yAd++iYayG5olNi2UwHdQ3W7U0h4LmseoyPxtn3qPbkDPHnBOb9ou9Z3lo
b4tOlW7N2JqcMIqhRk4AYkrIFGShJyMlScNtRPm+6f3H8v74vTe3s7fb5z2e7u3
ec54/GPbPOm6+W03O2SRxLF8ZF96jkkGRDOvMb+D4bHscjIIIJBUtsSu7CrTFw/
M14B1M6Hqd1ksa4GE0aNVbZXxXW1UlwgV1iqoUnQOAGCsoYZxnnB0auUslmghmF
LerdBP2KuGtqQJwgcI0pM6MAfOFmTIPuCPHJnpbzDJUpR1g+ErzkCCTOJCBkmJi
AJBjnK8gEbgp41eu1ok9RW80CvJVpD2JafuELURBiwAydokUlipPB3MpIzuXNgu
Nq6hpzRTpG0rxiSa3VkYE0YyD9yJuRgkc4x4IJBB1BYxYPoV31C0ljjMjYnnPvY
jVNU3SIXbrcbNZJluFyemhqXjMSSbN0zoCCVUAFmAJBIGceTqChvYbpyrv08hkp
Cj1UcMYUtDEqDKE8ZfKsSD4ZiuSFyV7rPoyzJRLVUj2y0S5EZErLTwTeSASB9Lj
kggZ4IPsV0Oh+k5Xt0LV8dVBTx1nxhjkDRNPOmFU7eGES7FYBwGdgGIQABiywyn
eU28MkmdZHIe9fzVb38JtIOzfaeUeZ98SXSwUEtq6ctlvnZGlpaSKByhJUsqBTj
OOMjRrQ0avVzkaq19sobrAsNxo6ariVt4jqIlkUNgjOCDzgnn96NGhChoLBaLVO
01utVDSSsuwyU9OkbFcg4yAOMgcfrWho0aEI0aNGhC//Z

The entry creation mechanism automatically adds the naming attribute in the RDN to the new entry. In this example, the attribute and substituted value uid: [A] will be added to every new entry.

Generating LDIF Files


Caution	Do not specify naming attributes in the template file, otherwise they will be duplicated in the new entry and cause an LDAP error.

The add operation can be used to populate large databases for testing. The same random entry generation mechanisms can be used to create large database files in LDIF (LDAP Data Interchange Format). Instead of the -e add option, use the -e genldif=filename[,append] options to specify the output file and whether it should be appended to. No host, port, or bind credentials are needed because no LDAP operations are performed on a directory. The following command creates a file of 100,000 sequentially numbered users with the template file in Code Example 11-5.

$ ldclt -b "ou=people,dc=example,dc=com" \
-e object=inet.txt,rdn=’uid:[A=INCRNNOLOOP(0;99999;5)]’ \
-e genldif=100Kinet.ldif,commoncounter

Only the target entries are generated in LDIF, missing nodes are not created. Use the command twice with the append option and different templates to generate LDIF files with numerous intermediate nodes. To create an entire tree in LDIF, edit an empty file manually to add a root entry at the beginning, and then append as many levels of nodes or leaves to the file as you wish.

Modify Operations


Tip	For creating entries with random DNs, such as names, it is common practice to have a non-looping counter in the template file to limit the contents of the LDIF file. By placing this counter in the description attribute, for example, its high and low range will determine the number of entries created by each thread.

The ldclt tool performs simple LDAP modify operations that are limited to changing the value of a single attribute. The -e attreplace option specifies the attribute name which will be modified and a new value that may be randomized for every operation.


Note	Performing modifications requires write privileges in your directory, so you should specify bind credentials, whether randomized or not, that can modify the entries under the given base DN.

The following command uses the Directory Manager credentials to modify the description attribute of randomly selected entries. The X placeholders in the new value string will be replaced with random characters, independently of any other patternString, bind DN or base DN substitutions:

$ ldclt -h host -p port -D "cn=Directory Manager" -w password \
        -b "ou=people,dc=example,dc=com" \
        -e rdn=’uid:[RNDN(0;99999;5)]’ -I 32 \
        -e attreplace=’description: random modify XXXXX’

Use the -I 32 option to ignore error 32, LDAP_NO_SUCH_OBJECT, that will occur if the randomization defines a DN for an entry that does not exist.

The following example performs modification under several nodes of the directory tree, using options explained in Random Base DNs. In this case multiple threads may access the same entry if they randomly happen to be in the same node, but concurrent modifications should not cause an error. Again, the new attribute value will contain a random string generated for each operation.

$ ldclt -h host -p port -D "cn=Directory Manager" -w password \
  -b "ou=DeptXXX,dc=example,dc=com" \
  -e randombase,randombaselow=0,randombasehigh=999 \
  -e rdn=’cn:Mr[INCRNNOLOOP(0;99999;5)]’ -I 32 \
  -e attreplace=’sn: XXXXXX’

Delete Operations

The command for performing delete operations is similar to that for performing modifications. The bind credentials must have write privileges to remove entries, and you may use the randomization features to determine the base DN and RDN of the target entry of each operation. The following command deletes random entries from a directory. Use the -I 32 option to ignore error 32, LDAP_NO_SUCH_OBJECT, that is likely to occur because entries may be randomly selected to be deleted a second time.

$ ldclt -h host -p port -D "cn=Directory Manager" -w password \
        -b "ou=DeptXXX,dc=example,dc=com" \
        -e randombase,randombaselow=0,randombasehigh=999 \
        -e delete,rdn=’cn:Mr[RNDN(0;99999;5)]’ -I 32

The delete operation supports the more flexible -e rdn=patternString for specifying the RDN of the target entry. The following command will delete all entries in branch of your directory, providing you specify the entire range of sequential numbers:

$ ldclt -h host -p port -D "cn=Directory Manager" -w password \
-b "ou=people,dc=example,dc=com" -e delete -I 32 \
-e rdn=’uid:[INCRNNOLOOP(0;99999;5)]’,commoncounter

modRDN Operations

The rename functionality is similar to that for modifications: the bind credentials require write privileges whether randomized or not, and you can randomize both the base DN and the RDN.

The ldclt tool performs LDAP modRDN operations on target entries by generating a second randomized RDN as the new name. Because the same mechanism is used to generate the new RDN, sequential numbering of target entries is not recommended. The following command uses the -e rdn=patternString options to specify a random RDN for the target entry.

$ ldclt -h host -p port -D "cn=Directory Manager" -w password \
-b "ou=people,dc=siroe,dc=com" -I 32 -I 68 \
-e rename,rdn=’uid:[RNDN(0;99999;5)]’

If the base DN is also randomized, you may specify the -e withnewparent option to also generate a new random parent. The following command shows an example using this option:

$ ldclt -h host -p port -D "cn=Directory Manager" -w password \
        -b "ou=DeptXXX,dc=example,dc=com" -I 32 -I 68 \
        -e randombase,randombaselow=0,randombasehigh=999 \
        -e rename,withnewparent,rdn=’cn:Mr[RNDN(0;99999;5)]’

Since the new parent DN is randomly generated, it may not exist in your directory. However, the ldclt tool will create the parent node automatically when needed. (See Create Missing Nodes for more information.) Due to the implementation of the missing node mechanism, the entry that triggered the creation of a missing node is not renamed, but subsequent reparenting operations will use the new node. Also, both examples use the -I option to ignore errors that may occur when renaming random RDNs.

Troubleshooting

This section explains the most common messages and errors encountered when using the ldclt tool and gives potential solutions.

No Thread Activity

When a thread has not performed any successful operations during a certain time, the ldclt tool displays the following message:

The most common reason for this message is that the server is overloaded and the thread is waiting for an operation to finish. When the server is overloaded, all threads will have short moments of inactivity, perhaps many in total, but never very many in a row. To account for server load, you may use the -i intervals option to allow longer periods of inactivity.

Another common cause for this message is that very few operations are succeeding. For example, when deleting randomly generated RDNs, successful delete operations will become rarer unless another process is creating entries with the same RDN pattern. Use the -e counteach option to account for unsuccessful operations and avoid the message about false inactivity.

A more unlikely scenario occurs if only one thread constantly reports inactivity messages, as shown in the following output sample:

T002: No activity for 40 seconds -- 1 in row, total 1
T002: No activity for 40 seconds -- 2 in row, total 2
T002: No activity for 40 seconds -- 3 in row, total 3
...
T002: No activity for 40 seconds -- 8 in row, total 8
...

In this case, it is likely to be waiting for a response which may be blocked for an abnormal condition such as a library or server bug. Since the condition may override any time-out mechanism on the server side, the thread will remain inactive until the ldclt process is stopped.

Exit Status 3

The ldclt tool maintains counters for most common errors and will terminate with exit status 3 at the end of the interval where the error limit is exceeded for any one error. The following output shows this situation:

Max error limit reached - exiting.
Global average rate: 0.00/thr ( NaN/sec), total: 0
Global number times "no activity" reports: never
Global error 32 (No such object) occurs 1020 times
Ending at Mon Apr 9 17:32:51 2001
Exit status 3 - Max errors reached.

By default the error limit is 1000 for each type of error. To avoid exiting in this condition, specify one of the following options:

Error 32

Error code 32 (LDAP_NO_SUCH_OBJECT) occurs commonly for searches, modifications, deletions, and rename operations. In these cases, the error is expected when using randomized base DN or RDN values that may not be present.

This error may also occur during add operations due to a DN syntax error. The missing node functionality will recursively attempt to create the parent node and fail with error 32. The following output shows this situation:

T007: Cannot add (cn=,00007,ou=people,dc=example,dc=com),
error=32 (No such object)

T007: Cannot create the intermediate nodes for cn=,00007,ou=people,dc=example,dc=com

To this error, check the RDN for illegal characters in the patternString or possibly in the file containing lists of DNs to add with the RNDFROMFILE keyword.

Error 65

Error code 65 (LDAP_OBJECT_CLASS_VIOLATION) occurs when trying to perform an addition with an invalid template in the -e object option. For example:

T004: Cannot add (uid=00007,dc=example,dc=com), error=65 (Object
class violation)

The server returns this error because the entry created from the template is not allowed by the directory’s schema. The exact schema violation will be given in the server’s error logs, for example:

[14/Feb/2002:19:38:52 -0700] - Entry "uid=00007,dc=example,dc=com"
-- attribute "mailAlternateAddress" not allowed

With Directory Server versions 4.x, extra space characters in the template file are not ignored. This will cause error 65 that can be solved by removing extra spaces from the template file. The following example shows an extra space before the closing parenthesis in the error log:

[04/Apr/2001:15:40:28 +0200] - Entry (cn=Mr00007,ou=people,
dc=example,dc=com) unknown object class (person )

Error 81

Error code 81 (LDAP_SERVER_DOWN) can be misleading because it may occur when a thread cannot establish a connection. The error message is the following:

This actually means that the client cannot connect to the server for some communication reason, such as no more operating system-level connections are available on either the client or the server. It does not necessarily mean the directory server is not running.

You may use the -I 81 option to ignore this error if it occurs during a bind. Instead of exiting, the corresponding thread will attempt to rebind, after a 1 second idle period to avoid overloading the server. (See the -e dontsleeponserverdown option in Table 11-1.) Error 81 is fatal to the calling thread if it occurs on any other operation, or if it is not ignored during the bind.

Error 89

Error code 89 (LDAP_PARAM_ERROR) can occur when calling the functions of the LDAP SDK for C with bad parameters. For example:

This error usually occurs in the ldclt tool when trying to parse the result of an operation that failed due to a connection error. Because the operation failed, there is no result to parse and the error is compounded. This error often occurs after error 81 described previously and can also be ignored by using the -I 89 option.

Error 91

Error code 91 (LDAP_CONNECT_ERROR) is returned when there is no process accepting connections on the given port and the given machine. It usually means there is no directory server responding at the given -h host and -p port options. For example:

T003: Cannot ldap_simple_bind_s (cn=directory manager,password),
error=91 (Can’t connect to the LDAP server)

You may use the -I 91 option to ignore this error if it occurs during a bind. Instead of exiting, the corresponding thread will attempt to rebind, after a 1 second idle period to avoid overloading the server (see the -e dontsleeponserverdown option in Table 11-1). Error 91 is fatal to the calling thread if it occurs on any other operation, or if it is not ignored during the bind.

Undocumented Options

The ldclt tool has several -e scalab01 options that are used internally for specialized test scenarios. These options implement behavior for a custom simulation that remains undocumented. These options are unsupported and will cause unexpected behavior.

Previous Contents Index Next
Sun ONE Directory Server Resource Kit 5.2 Tools Reference