Troubleshooting Oracle Internet Directory

A.17.1.1 Installation Errors

This section provide information on Oracle database installation errors.

During installation and configuration of the Oracle Database, Oracle recommends that you select the character set AL32UTF8 to avoid possible problems with multibyte characters.

A.17.1.2 Oracle Database Server Errors

Because Oracle Internet Directory relies on Oracle Database, database errors can cause directory server problems.

This section lists some database errors you might see in the Oracle Internet Directory logs.

A.17.1.2.1 Oracle Database Server Connection is Down

Oracle Internet Directory shuts down. You see error ORA-3113 or ORA-3114 in the log file.

Problem

Oracle Internet Directory has lost its connection to Oracle Database.

Solution

Check database and listener status, either directly on the host where they are running, or through. Restart them if necessary. OIDMON automatically detects that the database is up and restarts OIDLDAPD servers.

A.17.1.2.2 Oracle Database Server Error Due to Interrupted Client Connection

You get error sgslunrRead or 30SendPort

Problem

These errors indicates that an LDAP client has disconnected abruptly.

Possible reasons include:

The client program terminated the connection without performing an unbind or abandon.

The client machine shut down.
A network component, such as a load balancer or firewall, broke the connection due to a configured timeout setting.
The network is down.

Solution

These errors are due to conditions external to the server. If necessary, inform the network administrator.

A.17.1.2.3 Oracle Database Server Error Due to Schema Modifications

You get error ORA-1562.

Problem

If you attempt to add more schema components than can fit in the rollback segment space, you encounter this error and the modifications do not commit.

Solution

To solve this, increase the size of the rollback segments in the database server.

A.17.1.2.4 Oracle Database TNS Listener Error (ORA-12520)

Problem

You cannot connect to the Oracle database, or database connectivity is going down and coming up intermittently and is returning the following error when the connectivity goes down:

Error ORA-12520: TNS:listener could not find available handler for requested type of server

This problem has these potential causes and solutions:

A.17.1.2.4.1 Troubleshooting Low Oracle Database PROCESSES Parameter Value

The PROCESSES parameter determines the maximum number of operating system processes that can be connected concurrently to the Oracle Database.

If you see this problem intermittently (that is, the database connectivity is going down and coming up intermittently) and is returning the ORA-12520 error when the connectivity goes down, follow this procedure to troubleshoot the problem:

Login to the Oracle Database as the sysdba user.

Check the value of the PROCESSES parameter. For example:

SQL> SHOW PARAMETERS PROCESSES;

NAME                                TYPE        VALUE
------------------------------------ ----------- ------------------------------
processes                           integer     150

Check the V$RESOURCE_LIMIT view to display information about global resource use for some of the system resources. For example:
```
SQL> SELECT * FROM V$RESOURCE_LIMIT WHERE RESOURCE_NAME IN ('PROCESSES','SESSIONS');
```
If the MAX_UTILIZATION value is same as the INITIAL_ALLOCATION values, then the value of PROCESSES parameter might be too low.
Also, check the listener.log file for any additional errors. This file is usually located in the $ORACLE_HOME/network/log or $ORACLE_HOME/bin directory.
Determine if the ORA-12520 error is followed by this error:
```
TNS-12516: TNS:listener could not find available handler with matching protocol stack error message
```
If TNS-12516 is present, the number of connections (threads on Windows systems or processes on UNIX and Linux systems) might have reached the PROCESSES database parameter limit.
To resolve this problem, increase the value of the PROCESSES parameter. For example, the following command sets the value to 500:
```
SQL> ALTER SYSTEM SET PROCESSES=500 SCOPE=spfile;
```
The value of the PROCESSES parameter will depend on your specific environment.
Restart the Oracle Database server. For example:
```
SQL> SHUTDOWN IMMEDIATE
SQL> STARUP
```

A.17.1.2.4.2 Diagnosing Listener Control Utility (`lsnrctl`) State

If the Listener Control Utility (lsnrctl) shows the state as "blocked" instead of "ready", check for a problem with the listener configuration:

Check the value of the LOCAL_LISTENER entry for the Oracle database. For example:

# sqlplus ods@oiddb
SQL> SHOW PARAMETER LOCAL_LISTENER
NAME                                 TYPE       VALUE
------------------------------------ ----------- ------------------------------
local_listener                       string     LISTENER_ORCL

Compare the LOCAL_LISTENER value from the previous step with the value in the tnsnames.ora file. For example, consider the following the output from the tnsnames.ora file:
```
...
LISTENER_ORCL =
(ADDRESS = (PROTOCOL = TCP)(HOST = host.example.com)(PORT = 1521))
```
If the listener entry in the tnsnames.ora file does not match the LOCAL_LISTENER parameter from the database, configure the LOCAL_LISTENER parameter to point to the listener.
Update the LOCAL_LISTENER parameter for the database. For example:
```
SQL> ALTER SYSTEM SET LOCAL_LISTENER ='(ADDRESS=(PROTOCOL=TCP)(HOST=host.example.com)(PORT=1521))' scope=spfile;
```
Or, if you have an entry in your tnsnames.ora that points to the listener, use that listener name instead. For example:
```
ALTER SYSTEM SET LOCAL_LISTENER ='listener_name' scope=spfile;
```

A.17.1.3 Directory Server Error Messages and Causes

This section contains a list of Oracle directory server error messages that you might encounter.

Each message is followed by its most probable causes. Also, see OID Error Messages section in Oracle Fusion Middleware Error Messages Reference.

A.17.1.3.1 Inappropriate Authentication Error

You see the following error message on the command line when attempting an anonymous bind to the server:

ldap_bind: Inappropriate authentication
ldap_bind: additional info: Server is Configured to Deny Anonymous Binds

Problem

Anonymous binds are disabled. In most environments, some clients require anonymous access.

Solution

Enable anonymous binds.

See Also:

Managing Anonymous Binds for more information.

A.17.1.3.2 Constraint Violation Error Due to Editing a User or Group or Creating a Realm

You get the following error in oidldap*.log:

ORA-01483: invalid length for DATE or NUMBER bind variable.

You may also see the following error on your screen:

LDAP: error code 19 - Constraint Violation

These errors might only occur intermittently.

Problem

If you loaded the OracleAS Metadata Repository into an Oracle 10g Database that uses the AL32UTF8 character set, you may encounter some errors when you try to edit a user or Group, or Create Identity Management Realms in Oracle Internet Directory. Editing a user includes editing attributes for an existing user.

Solution

As a workaround, you can wait a bit and try editing the user again.

A.17.1.3.3 Standard Error Messages Returned from Oracle Directory Server

Table A-28 lists standard error messages and their causes. Oracle Internet Directory also returns other messages listed and described in Additional Directory Server Error Messages.

Table A-28 Standard Error Messages

Error	Cause
00: LDAP_SUCCESS	The operation was successful.
01: LDAP_OPERATIONS_ERROR	General errors encountered by the server when processing the request.
02: LDAP_PROTOCOL_ERROR	The client request did not meet the LDAP protocol requirements, such as format or syntax. This can occur in the following situations: Server encounters a decoding error while parsing the incoming request. The request is an add or modify request that specifies the addition of an attribute type to an entry but no values specified. Error reading SSL credentials. An unknown type of modify operation is specified (other than LDAP_MOD_ADD, LDAP_MOD_DELETE, and LDAP_MOD_REPLACE) Unknown search scope
03: LDAP_TIMELIMIT_EXCEEDED	Search took longer than the time limit specified. If you have not specified a time limit for the search, Oracle Internet Directory uses a default time limit of one hour.
04: LDAP_SIZELIMIT_EXCEEDED	More entries match the search query than the size limit specified. If you have not specified a size limit for the search, Oracle Internet Directory uses a default size limit of 1000.
05: LDAP_COMPARE_FALSE	Presented value is not the same as the one in the entry.
06: LDAP_COMPARE_TRUE	Presented value is same as the one in the entry.
07: LDAP_STRONG_AUTH_NOT_SUPPORTED	The requested bind method is not supported by the server. For example, SASL clients requesting Kerberos authentication from Oracle Internet Directory receive this error in response.
09: LDAP_PARTIAL_RESULTS	Server returned a referral.
10: LDAP_REFERRAL	Server returned a referral.
12: LDAP_UNAVAILABLE_CRITICALEXTENSION	Specified request is not supported
16: LDAP_NO_SUCH_ATTRIBUTE	Attribute does not exist in the entry specified in the request.
17: LDAP_UNDEFINED_TYPE	Specified attribute type is undefined in the schema.
19: LDAP_CONSTRAINT_VIOLATION	The value in the request violated certain constraints.
20: LDAP_TYPE_OR_VALUE_EXISTS	Duplicate values specified for the attribute.
21: LDAP_INVALID_SYNTAX	Specified attribute syntax is invalid. In a search, the filter syntax is invalid.
32: LDAP_NO_SUCH_OBJECT	The base specified for the operation does not exist.
34: LDAP_INVALID_DN_SYNTAX	Error in the DN syntax.
49: LDAP_INVALID_CREDENTIALS	Bind failed because the credentials are not correct.
50: LDAP_INSUFFICIENT_ACCESS	The client does not have access to perform this operation.
53: LDAP_UNWILLING_TO_PERFORM	General error, or server is in read-only mode.
65: LDAP_OBJECT_CLASS_VIOLATION	A change to the entry violates the object class definition.
66: LDAP_NOT_ALLOWED_ON_NONLEAF	The entry to be deleted has children.
67: LDAP_NOT_ALLOWED_ON_RDN	Cannot perform the operation on RDN attributes—for example, you cannot delete the RDN attribute of the entry.
68: LDAP_ALREADY_EXISTS	Duplicate ADD condition.
81: LDAP_SERVER_DOWN	Cannot contact the directory server. This message is returned from the SDK.
82: LDAP_LOCAL_ERROR	The client encountered an internal error. This message is returned from the client SDK.
83: LDAP_ENCODING_ERROR	The client encountered an error in encoding the request. This message is returned from the SDK.
84: LDAP_DECODING_ERROR	The client encountered an error in decoding the request. This message is returned from the SDK.
85: LDAP_TIMEOUT	Client encountered the time out specified for the operation. This message is returned from the SDK.
86: LDAP_AUTH_UNKNOWN	Authentication method is unknown to the client SDK.
87: LDAP_FILTER_ERROR	Bad search filter
88: LDAP_USER_CANCELLED	User cancelled operation
89: LDAP_PARAM_ERROR	Bad parameter to an LDAP routine
90: LDAP_NO_MEMORY	Out of memory

A.17.1.3.4 Additional Directory Server Error Messages

Table A-29 lists additional directory server error messages and their causes. These messages do not display error codes.

The Oracle Internet Directory application replaces the parameter tag seen in some of the following messages with the appropriate run-time value.

Table A-29 Additional Error Messages

Error	Cause
%s attribute not found	The particular attribute type is not defined in the schema.
parameter not found for attribute parameter	Value not found in the attribute. (ldapmodify)
Admin domain does not contain schema information for objectclass parameter	The object class specified in the request is not present in the schema.
Attempted to add a Class with oid parameter taken by other class	Duplicate object identifier specified. (schema modification)
Attribute parameter already in use	Duplicate attribute name. (schema modification)
Attribute parameter has syntax error.	Syntax error in the attribute name definition. (schema modification)
Attribute parameter is not supported in the schema.	Attribute not defined. (all operations)
Attribute parameter is single valued.	Attribute is single-valued. (ldapadd and ldapmodify)
Attribute parameter not present in the entry.	This attribute does not exist in the entry. (ldapmodify)
Bad attribute definition.	Syntax error in attribute definition. (schema modification)
Currently Not Supported	The version of LDAP request is not supported by this server.
Entry to be deleted not found.	DN specified in the delete operation not found.
Entry to be modified not found	The entry specified in the request is not found.
Error encountered while adding parameter to the entry	Returned when modify add operation is invoked. A possible cause is that the system resource is unavailable.
Error encountered while encrypting an attribute value.	Error in encrypting user password. (all operations)
Error in DN Normalization.	DN specified is invalid. Syntax error encountered in parsing the DN. (all operations)
Error in hashing parameter attribute.	Error in creating hash entry for the attribute. (schema modification)
Error in hashing parameter objectclass.	Error in creating hash entry for the objectclass. (schema modification)
Error in Schema hash creation.	Error while creating hash table for schema. (schema modification)
Error replacing parameter.	Error in replacing this attribute. (ldapmodify)
Error while normalizing value for attribute parameter.	Error in normalizing value for the attribute. (all operations)
Failed to find parameter in mandatory or optional attribute list.	Attribute specified does not exist in either the mandatory or optional attribute list as required by the object class(es).
Function Not Implemented	The feature/request is currently not supported. (Specifying a non-indexed attribute in a search can generate this error.)
INVALID ACI is parameter	The particular ACI you specified in a request is invalid.
Mandatory attribute parameter is not defined in Admin Domain parameter.	MUST refers to attribute not defined. (schema modification)
Mandatory Attribute missing.	The mandatory attribute for the particular entry is missing, as required by the particular object class.
Matching rule, parameter, not defined.	Matching rule not defined in the server. (schema modification)
MaxConn Reached	The maximum number of concurrent connections to the LDAP server has been reached.
Modifying the Naming attribute for the entry without modifying the DN.	Cannot modify the naming attributes using ldapmodify. A naming attribute, such as `cn` is an element in the DN.
New Parent not found.	New parent specified in modifydn operation does not exist.(ldapmodifydn)
Object already exists.	Duplicate entry. (ldapadd and ldapmodifydn)
Object ID parameter already in use.	Duplicate object identifier specified. (schema modification)
Objectclass parameter already in use.	Duplicate Objectclass name. (schema modification)
Objectclass attribute missing.	The objectclass attribute is missing for this particular entry.
OID parameter has syntax error.	syntax error in the object identifier definition. (schema modification)
One of the attributes in the entry has duplicate value.	You entered two values for the same attribute in the entry you are creating.
Operation not allowed on the parameter.	Operation not allowed on this entry. (modify, add, and delete)
Operation not allowed on the DSE Entry.	Can't do this operation on DSE entry. (delete)
Optional attribute parameter is not defined in Admin Domain parameter.	MAY refers to attribute not defined. (schema modification)
Parent entry not found in the directory.	Parent entry does not exist. (ldapadd and perhaps ldapmodifydn)
Super object parameter is not defined in Admin Domain parameter.	SUP types refer to non-existing class. (schema modification)
Super type undefined.	SUP type does not exist. (schema modification)
Superuser addition not permitted.	Cannot create superuser entry. (ldapadd)
Syntax, parameter, not defined.	Syntax not defined in the server. (schema modification)
The attribute or the value specified in the RDN does not exist in the entry.	AVA specified as the RDN does not exist in the entry. (ldapadd)
Unknown search scope	The search scope specified in the LDAP request is not recognized.
Version Not Supported	The version of the LDAP request is not supported by this server.
Alias Problem	Either of the following have occurred: An alias was dereferenced, but it did not point to an entry in the DIT. The user tries to add an alias entry whose parent is an alias.
Alias Dereferencing Problem	The user cannot dereference an alias because of access control issues.
No Such Object	The server cannot find the base DN specified in the search request.
Invalid DN Syntax	When adding or modifying an alias entry, if the value specified for `aliasedObjectName` has invalid DN syntax, then the directory server returns this error message to the client.
Insufficient Access Rights	The user does not have access to the dereferenced entry.

A.17.1.4 Core Dump and Stack Trace Occurs When Oracle Internet Directory Crashes

You can control the type of information Oracle Internet Directory provides when it crashes by changing the value of the orclsdumpflag attribute in the instance-specific configuration entry.

If the server crashes, it leaves a core file under the directory

$DOMAIN_HOME/tools/OID/logs

If orclsdumpflag is set to 0, and the server crashes, in addition to the core dump, the server also attempts to leave a stack trace. The location for the stack trace is:

$DOMAIN_HOME/servers/OID/logs/componentName/oidldapd_stack00_pid.dmp

Some operating system-specific settings can affect the generation of a core dump or stack trace. Consult your operating system documentation to determine whether the following settings are required:

The coredump parameter must be set to allow core dumps.
The file size limit, as specified with the ulimit command, must be sufficient to allow core dumps.
The file permissions on the $ORACLE_HOME/bin/oidldapd binary file must allow read by group. You can ensure that group has read permission by typing:
```
chmod g+r $ORACLE_HOME/bin/oidldapd 
```
as the root user.

A.17.1.5 TCP/IP Problems

TCP/IP bugs in the operating system can interfere with Oracle Internet Directory service.

Refer to the following topics:

A.17.1.5.1 Do Not Use TCP-Based Monitoring of Server Availability on Windows 2003 Server

If you use the F5 load balancer for monitoring Oracle Internet Directory server availability, configure the load balancer to use LDAP- or HTTP-based monitoring, as described in Configuring Oracle HTTP Server for High Availability in the Oracle Fusion Middleware High Availability Guide . Using TCP-based monitoring might cause the service to become unavailable, due to an operating system bug on Microsoft Windows 2003 Server.

A.17.1.5.2 Do Not Install DaimondCS Port Explorer

Oracle Internet Directory does not work if DaimondCS Port Explorer is installed on the system.

A.17.1.6 Troubleshooting Password Policies

This section describes error messages and problems related to password policies.

This section includes the following:

A.17.1.6.1 Password Policy is Not Enforced

The password policy is not being enforced for a given user or set of users. For example, users can reset their password using a syntax that is disallowed by the defined password policy.

Problem

Just creating a password policy is not sufficient. You must also specify the subtree to be governed by the policy.

Solution

Add and populate a pwdPolicysubentry attribute with the policy's DN, at the root of that subtree.

See Also:

Creating and Applying a Password Policy for more information.

A.17.1.6.2 Password Policy Error Messages

Table A-30 contains the error messages sent to the client as a result of password policy violations. The error codes are not standard LDAP error codes. They are messages sent as a part of additional information in the LDAP result.

Table A-30 Password Policy Violation Error Messages

Error Number	Exception	Comment or Resolution
9000	`GSL_PWDEXPIRED_EXCP`	User's password has expired.
9001	`GSL_ACCOUNTLOCKED_EXCP`	User account is locked.
9002	`GSL_EXPIREWARNING_EXCP`	User password will expire in `pwdexpirewarning` seconds. Please change your password now.
9003	`GSL_PWDMINLENGTH_EXCP`	User password is not the required number of characters long.
9004	`GSL_PWDNUMERIC_EXCP`	User password does not contain required numeric characters.
9005	`GSL_PWDNULL_EXCP`	User password is a null password, which is disallowed.
9006	`GSL_PWDINHISTORY_EXCP`	User's new password is the same as an old one saved in history, which is disallowed.(The `pwdinhistory` attribute controls the number of passwords saved in history.)
9007	`GSL_PWDILLEGALVALUE_EXCP`	The user password supplied is an illegal value defined in `orclpwdillegalvalues`, and therefore cannot be used.
9008	`GSL_GRACELOGIN_EXCP`	User password has expired. User has `pwdgraceloginlimit` grace logins left or `orclpwdgracelogintimelimit` seconds in which grace logins are allowed.
9012	`GSL_PWDALPHA_EXCP`	Your Password must contain at least `orclpwdminalphachars` alphabetic characters.
9013	`GSL_PWDSPECIAL_EXCP`	Your Password must contain at least `orclpwdminspecialchars` special characters.
9014	`GSL_PWDUPPER_EXCP`	Your Password must contain at least `orclpwdminuppercase` upper case characters
9015	`GSL_PWDMAXCHAR_EXCP`	Your Password can only contain `orclpwdmaxrptchars` repeated characters
9016	`GSL_PWDLOWER_EXCP`	Your Password must contain at least `orclpwdminlowercase` lowercase characters.
9017	`GSL_EC_PWDPOLSUBENTINV`	The `pwdPolicysubentry` provided is invalid. (The DN is not that of a valid password policy present in the directory.)
9018	`GSL_EC_PWDPOLINUSE`	The `pwdPolicy` entry you are deleting is currently in use. (You must remove the references to the password policy before the policy itself can be removed.)
9019	`GSL_EC_PWDPOLOBJ`	The DN of a `pwdPolicy` entry may not be modified.
9020	`GSL_PWDMINAGE_EXCP`	Your Password has to be at least `pwdminage` seconds old before it can be changed.
9032	`GSL_EC_GRACE_CONST`	`orclgracelogintimelimit` and `pwdgraceloginlimit` are mutually exclusive. Both cannot be nonzero.
9033	`GSL_EC_NOROOTDSEPWDPOL`	The `pwdpolicysubentry` attribute in the root DSEcannot be deleted. (This is not allowed because it would leave the directory without an applicable password policy.)
9034	`GSL_EC_NOTROCPWDPOL`	Only password policies defined in the Root Oracle Context are applicable in the Root DSE. (This ensures that only a policy specified by an admin who has directory-wide privileges can be applied to the entire directory.)
9050	`GSL_ACCTDISABLED_EXCP`	User account has been disabled.

A.17.1.7 Troubleshooting Directory Performance

This section gives some quick pointers for common performance-related problems.

Refer to the following topics:

A.17.1.7.1 Poor LDAP Search Performance

LDAP search performance is poor.

Problem

Various problems.

Solution

Ensure that:

Schema associated with the ODS user is ANALYZED
For searches involving multiple filter operands, make sure that the order in which they are given goes from the most specific to the least specific. For example, &(uid=john.doe)(objectclass=person) is better than &(objectclass=person)(uid=john.doe).

Also see Diagnosing Poor Oracle Database Server Performance.

A.17.1.7.2 Improving Poor LDAP Add or Modify Performance

LDAP add or modify performance is poor.

Problem

Various problems

Solution

Ensure that:

There are enough redo log files in the database
The undo tablespace in the database is large enough
The schema associated with the ODS user is ANALYZED

When estimating the statistics, you can use the OID Database Statistics Collection tool to analyze the various database ODS schema objects.

Both the tracing functionality described in Managing Logging and the database tracing event 10046 can assist you in diagnosing performance issues.

See Also:

The oidstats.sqlcommand-line tool reference in Oracle Fusion Middleware Reference for Oracle Identity Managementfor instructions on using the OID Database Statistics Collection tool

The Oracle Internet Directory Performance Tuning in Oracle Fusion Middleware Performance and Tuning Guide for instructions on optimizing searches

Note 243006.1 on My Oracle Support (formerly MetaLink), http://metalink.oracle.com, for information on performance issues with group entries

A.17.1.7.3 Diagnosing Poor Oracle Database Server Performance

Problem

Oracle database server is consuming lot of processor resources during LDAP search operations.

Solution

Proceed as follows:

Identify the LDAP operations that are processor-intensive by running:
```
oidctl connect=connstr status -diag
```
This command displays the LDAP operation and associated SQL that is being executed.
Tune the database appropriately for this kind of query. See the Basic Tuning Considerations chapter in Oracle Fusion Middleware Performance and Tuning Guide.
If possible, change the applications's search signature. If that is not possible, tune the Oracle Internet Directory attribute orclinmemfiltprocess. See the Oracle Internet Directory chapter in Basic Tuning Considerations chapter in Oracle Fusion Middleware Performance and Tuning Guide.

A.17.1.7.4 Troubleshooting Database Performance Issues Using AWR Reports

The Automatic Workload Repository (AWR) is a built-in repository that exists in Oracle Database. At regular intervals, the database makes a snapshot of its vital statistics and workload information and stores this snapshot in the workload repository.

By default, Oracle database automatically generates snapshots of the performance data once every hour and retains the statistics in the repository for eight days.

If you suspect Oracle Internet Directory as the component causing a database performance issue, you can use AWR reports to further investigate the problem, as described in the following sections:

A.17.1.7.4.1 Generating an AWR Report

To generate a specific AWR report, run the awrrpt.sql SQL script:

Connect to the Oracle database as the sysdba user. For example:
```
sqlplus /sys/sysdba-password@oiddb as sysdba
```
Generate the AWR report by running the awrrpt.slq script at the SQL prompt. For example:
```
SQL> $ORACLE_HOME/rdbms/admin/awrrpt.sql
```
When prompted, specify the following information for the report:
- HTML or text format. The default format is HTML.
- Number of days for which you want to list snapshot IDs.
  
  This value should indicate the time period in days when the performance issue occurred. For example, if the issue occurred today, then enter 1. If the issue occurred yesterday, enter 2.
  
  The awrrpt.sql script lists snapshot IDs for the number of days selected. Each snapshot is assigned a unique snapshot ID.
- Beginning and ending snapshot ID for the workload repository report.
  
  Identify the time when the performance issue occurred. Based on this time period, define the range of snapshots for which AWR data needs to be extracted by specifying a beginning and ending snapshot ID. For example:
```
Enter value for begin_snap: 110
Enter value for end_snap: 120
```
- Report name. Accept the default report name, or enter a new name.
The database generates the AWR report in the directory where you executed the sqlplus command.
Analyze the AWR report to try to determine the issue causing the performance issue, as described in the next section.

A.17.1.7.4.2 Analyzing an AWR Report

To analyze an AWR report:

Check the "Top 5 Timed Foreground Events" table. This table can provide information about events that might be causing performance issues. Check the "Waits" and the "Avg wait (ms)" columns and note the event with the highest average wait time.
In the "SQL Statistics" section, check the "SQL ordered by Elapsed Time" table:
1. Check the values in the "Elapsed Time per Exec (s)" column.
  
  Under normal conditions, a query should execute in less than one second (shown in milliseconds in the table). If a query is taking a second or more to execute, it should be investigated in the next step.
2. For the event that displays the highest "Elapsed Time per Exec (s)", click the value under the "SQL Id" column to show the SQL query that was executed.
  
  If this SQL query has fields such as "attrvalue = 'referral'", the query can be skipped from being executed in environments where only one Oracle Internet Directory instance is running and there are no referrals to other Oracle Internet Directory processes.
If there are no referral entries in your environment, you can improve performance by preventing the previous query from being executed, as follows:
1. In "cn=dsaconfig,cn=configsets,cn=oracle internet directory", set the orclskiprefinsql attribute to 1, to skip the referral for the search.
  
  For more information, see Attributes of the DSA Configuration Entry.
2. Restart Oracle Internet Directory for the change to take effect.

A.17.1.8 Troubleshooting Port Configuration

You can find out which ports the Oracle Internet Directory dispatcher is using for SSL and non-SSL connections in the following ways:

In Oracle Enterprise Manager Fusion Middleware Control, select Port Usage from the OID menu.

From the command line, execute:

oid_instanceStatus(instanceName = 'instance-name')

From the command line, execute:
```
oidctl connect=oiddb status
```

A.17.1.9 Troubleshooting Starting Oracle Internet Directory

This section describes problems you might encounter when starting Oracle Internet Directory.

Refer to the following topics:

A.17.1.9.1 Oracle Internet Directory is Down

Problem

Oracle Enterprise Manager Fusion Middleware Control shows Oracle Internet Directory down. The command:

oid_instanceStatus(instanceName = 'instance-name')

shows that oidmon is down, as well as all the oidldapd processes.

Solution

Consult the oidmon log, $DOMAIN_HOME/servers/OID/logs/componentName/oidmon-0000.log and nodemanager log, $DOMAIN_HOME /nodemanager/nodemanager.log to determine why oidmon is not starting.

Problem

Oracle Enterprise Manager Fusion Middleware Control shows Oracle Internet Directory down. The command:

oid_instanceStatus(instanceName = 'instance-name')

shows that oidmon is up, but the oidldapd processes are down.

Solution

Check the following logs in the order shown:

The oidmon log,$DOMAIN_HOME/servers/OID/logs/componentName/oidmon-0000.log contains details as to why oidmon cannot start the oidldapd process. The most common issues are
- Unable to connect to Oracle Database: Ensure that the Oracle database and listener are up and running.
- Time difference between the two nodes is more than 250 seconds: Adjust the system time.
- Oidmon keeps trying to start oidldapd processes, but they fail to run. To debug, see Step 2.
The Oracle Internet Directory dispatcher log, $DOMAIN_HOME/servers/OID/logs/componentName/oidldapd01-0000.log contains information about why oidldapd server processes fail to start. The most common reasons are:
- Configured PORT for Oracle Internet Directory is not free: Execute
```
 netstat -an | grep oidPort
 
```
  to see if it the port is free.
- Oracle Internet Directory is configured to listen on a port number less than 1024 on a UNIX or Linux system and the executable binary file $ORACLE_HOME/bin/oidldapd is either not owned by root or does not have the setuid bit set.
- The oidldapd dispatcher keeps spawning oidldapd server processes, but they fail to run. In this case, you might see a single oidldapd dispatcher process running if you use ps on UNIX or Linux or Task Manager on Windows.
The Oracle Internet Directory server log, $DOMAIN_HOME/servers/OID/logs/componentName/oidldapd01sPID-0000.log contains information about why the server processes fail to run. Common issues include:
- Unable to create Oracle Database connection pool: Check the Oracle Database PROCESSES parameter and increase if necessary.
- Oracle Internet Directory is configured to use an SSL wallet file, and that file is inaccessible.

A.17.1.9.2 Oracle Internet Directory is Read-Only

Problem

The Oracle Internet Directory server starts in read-only mode.

Solution

This usually indicates that the Oracle Internet Directory server has been started against the wrong schema. To verify, type these two commands:

oidldapd -v

ldapsearch -p oidPort -D cn=orcladmin -q -b "" -s base "objectclass=*" Orcldirectoryversion

If these commands show different versions, the server starts in read-only mode.

A.17.1.10 Oracle Internet Directory Error Due to Interrupted Client Connection

This section provides information on fixing the errors that may occur due to interrupted client connection in Oracle Internet Directory.

You get the following error:

ServerDispatcher:  sgslufread: Hard error on read, OS error = 110

Problem

This error indicates that an LDAP client has disconnected abruptly. Possible causes include:

The client machine crashed or the client program terminated the connection without performing an unbind or abandon.
A network component, such as a load balancer or firewall, broke the connection due to a configured timeout setting such as the idle connection timeout.
The load balancer issues a TCP ping instead of an LDAP ping.

Solution

This error is due to conditions external to the server. If necessary, inform the network administrator.

A.17.1.11 Troubleshooting Starting, Stopping, and Restarting of the Directory Server

To troubleshoot starting and stopping the directory server, you must know the purpose of each tool involved, how all the tools work together, and the overall process for starting and stopping the server.

This section includes the following topics:

See Also:

The Oracle Internet Directory Performance Tuning chapter in Oracle Fusion Middleware Performance and Tuning Guide.

A.17.1.11.1 Starting, Stopping, and Restarting the Directory Server Instance Using the Tool

You start the directory server instance by typing:

$DOMAIN_HOME/bin/startComponent.sh <instance-name>

OIDCTL

When OIDCTL is executed, it connects to the database as user ODS. Depending on the options used in the command, it either inserts or updates rows into a table named ODS.ODS_PROCESS_STATUS_STATUS. If the START option is used, then a row is inserted. If either the STOP or RESTART option is used, then a row is updated.

The ODS.ODS_PROCESS_STATUS table includes the following information:

instance: The unique number of the instance, any value between 0 and 1000
pid: Process identifier, which is updated by OIDMON when the process is started
state: The type of operation requested

The possible values for state are:
- 0=stop
- 1=start
- 2=running
- 3=restart
- 4=shutdown
- 5=failedover

OIDMON

To start, stop, or restart a directory server instance, OIDMON must be running. At specified intervals, this daemon checks the value of the state column in the ODS.ODS_PROCESS_STATUS table.

If it finds a row with state=0, then it reads the pid and stops the process.
If it finds one with state=1 or state=4, then it starts a new process and updates the pid column with a new process identifier.
If it finds one with state=2, then it reads the pid and verifies that the process with that pid is running. If it is not running, then OIDMON starts a new process and updates the pid column with a new process identifier.
If it finds a row with state=3, then OIDMON reads the pid, stops the process, starts a new one, and updates the pid accordingly.
If it is unsuccessful, it pushes the request to another node.

In short, OIDCTL inserts and updates state information in the rows in the ODS.ODS_PROCESS_STATUS table. OIDMON then reads that information and performs the specified task.

About the Processes Involved in Starting, Stopping, and Restarting the Directory Server

Starting, stopping and restarting the directory server involves processes. OIDMON is one process. On UNIX, it is called oidmon. In a Microsoft Windows environment, it is called oidmon.exe.

To start an instance, OIDMON checks the unique number in the instance column mentioned in the previous section. It then starts another process, namely, the listener/dispatcher, which is different from the Oracle Net Services listener process. It stores the process identifier for that new process in the pid column.

The listener/dispatcher, in turn, starts a number of server processes as defined in the configuration set entry. Note that these server processes are controlled by the listener/dispatcher and not by OIDMON. If one of these processes fails, then it is automatically restarted by the listener/dispatcher.

Together, the listener/dispatcher and the server processes constitute a directory server instance. On UNIX, this directory server instance is called oidldapd. On Microsoft Windows, they are called oidldapd.exe.

In short, there are at least three processes: one for OIDMON and at least two for the directory server itself. When all processes are running, you should see something like the following on UNIX computers:

% ps -ef|grep oid
root 12387 12381 0 Mar 28 ? 0:05 oidldapd -i 1 -conf 0 key=811436710
root 12381 1 0 Mar 28 ? 0:10 oidmon start
root 13297 1 0 Mar 28 ? 0:14 oidldapd

Another way to obtain server information is by running:

oidctl connect=oiddb status.

A.17.1.11.2 Problems Starting, Stopping, and Restarting the Directory Server

This section describes some problems you might have when starting, stopping, or restarting the directory server.

A.17.1.11.2.1 OIDCTL or OIDMON fails

Either OIDCTL or OIDMON can fail for reasons.

Problem

Incorrect syntax

Solution

Verify that you are using the correct syntax as described in Oracle Internet Directory Administration Tools Oracle Fusion Middleware Reference for Oracle Identity Management. Note that the correct value of the connect option when using OIDCTL is the TNS alias—that is, the connect string—and not a host name or other value.

Problem

The Oracle Internet Directory-designated database is not running.

The Oracle Net Services configurations are incorrect.

Solution

Verify that the Oracle Internet Directory-designated database and the Oracle Net Services components are correctly configured and running. To do this, see if you can connect to the database by using SQL*Plus that is installed in the same ORACLE_HOME as OIDCTL. Log in as ODS/ods_password@tns_alias where tns_alias is the same as that used in the connect option with OIDCTL.

Problem

Missing oidldapd file.

Solution

See $DOMAIN_HOME/servers/OID/logs/componentName/oidmon-XXXX.log. Look for the message: No such file or directory. To correct the problem, replace the executable file.

Problem

Wrong permissions on oidldapd/oiddispd executable file.

Solution

Part 1

This solution is applicable for Oracle Internet Directory pre 11g Release 1 (11.1.1.7.0) release.

Look for the message Exec of OIDLDAPD failed with error 13. On UNIX, the $ORACLE_HOME/bin/oidldapd file must have the following permissions:

-rws--x---  1 root   dba       1691802 Jan 20 10:30 oidldapd

If the permissions are not correct, type the following, as root:

cd $ORACLE_HOME/bin
chown root:dba oidldapd
chmod 0710 oidldapd  
chmod u+s oidldapd

Part 2

This solution is applicable for Oracle Internet Directory 11g Release 1 (11.1.1.7.0) release and later.

Look for the message Exec of OIDDISPD failed with error 13. On UNIX, the $ORACLE_HOME/bin/oiddispd file must have the following permissions:

-rws--x---  1 root   dba       1691802 Jan 20 10:30 oiddispd

If the permissions are not correct, you can perform either of the following options:

A) Run the following, as root:

cd $ORACLE_HOME
oidRoot.sh

Or

B) Type the following, as root:

cd $ORACLE_HOME/bin
chown root:dba oiddispd
chmod 0710 oiddispd  
chmod u+s oiddispd

It is recommended that you verify the ownership and permission are set correctly with the following command after applying the preceding solution:

$ORACLE_HOME/bin/ls -al oiddispd

Problem

You are running as a user with insufficient privilege

Solution

To confirm that this is the problem, see $DOMAIN_HOME/servers/OID/logs/componentName/oidmon-XXXXX.log.

Look for the message: Permission denied or Open Wallet failed. This happens if you are not running either as root or as the user who is in the dba group. To correct the problem, try again as the correct user.

Problem

A port is in use.

Solution

See

$DOMAIN_HOME/servers/OID/logs/componentName/oidldapd00sPID-XXXX.log.

Look for the message: Bind failed on... This indicates that the port that oidldapd is configured to listen on is in use by some other process. To determine which process is using the port, type:

netstat -a | grep portNum

If necessary, reconfigure the other process to use a different port or configure oidladapd to listen on another port by adding a configset. Remember that, by default, oidladapd listens on two ports, an SSL and non-SSL port.

Problem

On a cluster or Oracle Application Server Cluster (Identity Management) configuration, OIDMON pushes the server to another node in a cluster when it cannot start the server on the local node.

Solution

See oidmon.log. Look for the message: gslsgfrPushServer: Could not start serveron NodeA, trying to start on nodeNodeB. To correct this problem, you must first determine why OIDMON cannot start the server on the local node.

Problem

A possible problem with Oracle Net Services or with the database itself.

Solution

See oidmon.log, oidldapdxx.log, where xx is the server instance number.

A Row is Missing from ODS.ODS_PROCESS_STATUS

Problem

In a cluster or Oracle Application Server Cluster (Identity Management) configuration, OIDMON successfully starts oidldapd on both nodes, but then initiates failover due to a time stamp difference.

Solution

See the trace files oidldapdxx.log where xx is the instance number, and oidldapdxxsyy.log where xx is the instance number and yy is the process identifier. If the trace files do not give useful information or pointers to My Oracle Support (formerly MetaLink) documents, then do the following: (1) Stop the directory server processes; (2) Remove or rename old trace files; (3) Start OIDMON and a directory server with maximum debug level, namely, 11744051. Note that, to get the trace files, you must first stop, then start, the server; you cannot simply restart it. Investigate the new trace files, and, if needed, log an iTAR with Oracle Support Services and upload the trace files to the iTAR.

See Also:

The oidctl command-line tool reference in Oracle Fusion Middleware Reference for Oracle Identity Management for more information on failover.

A.17.1.12 Troubleshooting Oracle Internet Directory Replication

This section discusses directory replication problems.

This section includes the following topics:

A.17.1.12.1 Bootstrapping Fails

Disable referential integrity during the replication bootstrapping process. If referential integrity is enabled, bootstrapping fails.

A.17.1.12.2 Log Files to Diagnose Replication Issues

Whenever you investigate a replication problem, be sure to consult the log files for information. The log files are $DOMAIN_HOME/servers/OID/logs/componentName/oidrepld-XXXX.log, oidldapd00-XXXX.log and $DOMAIN_HOME/servers/OID/logs/componentName/oidldapd00sPID-XXXX.log where PID is the server process identifier and XXXX is a number from 0000 to orclmaxlogfiles configured.

The replication server supports multiple debugging levels. To turn on replication debugging, use either ldapmodify or the Shared Properties, Replication tab, in Oracle Enterprise Manager Fusion Middleware Control to change orcldebuglevel in the replication configuration set.

Note:

Turning on debugging affects replication performance.

See Also:

Managing Replication Configuration Attributes for more information.

A.17.1.12.3 Replication Server Does Not Start

There are several problems that can prevent the replication server from starting.

Problem

Invalid oidctl syntax

Solution

Use the following syntax to start the replication server.:

oidctl server=oidrepld connect=connect string instance=instance_number \
       flags="-h host -p port"

Problem

Oracle Internet Directory is not running at the host and port you specified on the command line when you attempted to start the replication server. This caused the anonymous bind to the target Oracle Internet Directory to fail.

Solution

Make sure the target Oracle Internet Directory is up and running at the specified host and port.

Problem

The replication server is attempting to bind to the host and port specified in either the orclreplicaprimaryurl or the orclreplicasecondaryurl attribute of the Replica entry, but Oracle Internet Directory is running at a different host or port.

Solution

If you decide to run Oracle Internet Directory at a different host or port, add the new information to the orclreplicasecondaryurl attribute of the replica entry, as follows:

Prepare a modification file, mod.ldif. For example, to change to host my.us.example.com and port 4444, you would specify:

dn: orclreplicaid=replica_ID, cn=replication configuration
changetype: modify 
add: orclreplicasecondaryurl 
orclreplicasecondaryurl: ldap://my.us.example.com:4444/

Run:
```
ldapmodify -h host -p port -f mod.ldif
```

Problem

The ReplBind credential in the replication wallet $DOMAIN_HOME/config/fmwconfig/components/OID/admin/oidpwdrORACLE_SID is corrupt or invalid. That is, the password stored in the wallet is not the same as the password that is stored in the directory, or the wallet does not exist. This causes the replication bind to fail and the replication server to exit with an error.

You might see messages similar to this example in the file oidrepldXX.log:

2005/07/21:11:13:28 * gslrcfdReadReplDnPswd:Error reading repl passwd 
2005/07/21:11:13:28 * gslrcfcReadReplConfig:Error found. 
2005/07/21:11:13:28 * Failed to read replication configuration information.

Solution

Use remtool to fix the replication bind credential in the replication wallet or to synchronize between Oracle Internet Directory and the replication wallet.

remtool -pchgpwd changes the password of the replication dn of a replica. Use this option if you know the current replication DN password stored in the directory and you want to change it both in the directory and in the wallet.
remtool -presetpwd resets the password or the replication dn of a replica. Use this option if you know the current replication DN password stored in the directory and you want to change it both in the directory and in the wallet.
remtool -pchgwalpwd changes password of replication dn of a replica only in the wallet. Use this option if you know the replication DN password stored in the directory but you are not sure whether the wallet has the correct password or you want to create the wallet file.

All of these options create a wallet if one does not already exist.

See Also:

Theremtool command-line tool reference in Oracle Fusion Middleware Reference for Oracle Identity Management for more information about using remtool
The oidpasswd command-line tool reference in Oracle Fusion Middleware Reference for Oracle Identity Management for more information about using oidpasswd

Problem

The replication server is attempting to bind to an SSL port that is configured for one-way or two-way authentication.

Solution

Configure the replication server to use either the non-SSL port or an SSL port configured for no authentication. You can use a separate Oracle Internet Directory server instance just for replication.

A.17.1.12.4 Errors in Replication Bootstrap

errors can occur in replication bootstrap.

Problem

Some of the naming contexts failed to be bootstrapped.

Solution

Identify the naming contexts that failed to be bootstrapped, and use the oidcmprec tool to reconcile them.Then resume replication by setting the consumer's replica state to ONLINE mode

Problem

Various causes.

Solution

Identify the cause of the bootstrap failure and fix the cause, then restart bootstrapping by setting consumer's replica state to BOOTSTRAP mode.

Solution

To determine the exact cause of the error, examine the log file oidldapdxx.log. Look for error messages like those in the following example:

2004/09/14:12:57:23 * Starting OIDREPLD against dlsun1418:4444...
2004/09/14:12:57:25 * Starting scheduler...
2004/09/14:12:57:26 * Start to BootStrap from supplier=dlsun1418_replica to consumer=dlsun1418_replica2
2004/09/14:12:57:27 * gslrbssSyncDIT:Replicating namingcontext=cn=oraclecontext ......
2004/09/14:12:58:21 * gslrbssSyncDIT:Sync done successfully for namingctx: cn=oraclecontext, 222 entries matched
2004/09/14:12:58:21 * gslrbssSyncDIT:Replicating namingcontext=cn=joe smith ......
2004/09/14:12:58:23 * BootStrap failure when adding DN=cn=Joe Smith,
server=dlsun1418_replica2,err=Constraint violation.
2004/09/14:12:58:23 * gslrbssSyncDIT:Sync failed for namingctx: cn=joe smith, only 1 entries retrieved
2004/09/14:12:58:23 * gslrbssSyncDIT:Replicating namingcontext=cn=oracleschemaversion ......
2004/09/14:12:58:25 * gslrbssSyncDIT:Sync done successfully for namingctx: cn=oracleschemaversion, 10 entries matched
2004/09/14:12:58:51 * gslrbsbBootStrap: Failure occurred when bootstrapping 1 out of 3 namingcontext(s) from the supplier

Identify the cause of the bootstrap failure and fix it. You can identify the naming contexts that caused the problem, then use oidcmprec to compare and reconcile the naming contexts. After you resolve the problem, start bootstrapping again by starting the Oracle Internet Directory replication server.

Problem

The Oracle Internet Directory server was shut down during the bootstrapping

Solution

Make sure both the supplier Oracle Internet Directory and the consumer Oracle Internet Directory servers are up and running during replication bootstrapping.

Problem

Some of the entries being bootstrapped cannot be applied at the consumer due to a constraint violation.

Solution

Make sure the Oracle Internet Directory schema of the consumer are synchronized with those of the supplier before starting replication bootstrap. When you add an LDAP replica, remtool ensures that the Oracle Internet Directory schema on the consumer replica are synchronized with those on the supplier replica.

Problem

Improper replication filtering during bootstrapping. Replication supports excluding one or more attributes during bootstrapping. However, if a mandatory attribute of an entry is configured to be excluded, that entry cannot be applied at the consumer due to an objectclass violation.

Solution

Follow the replication naming context configuration rules in Setting Up Replication to configure replication filtering properly.

If you are debugging LDAP replication, you should become familiar with the LDAP replica states. If LDAP-based replication is configured, when the replication server starts, it reads the replica state from the local replica. The replication server behaves differently, depending upon the local replica state. LDAP replication errors appear in oidldapdxx.log

See Also:

LDAP Replica States

Problem

When you restart the replication server after the replication server failed to bootstrap a naming context having more than 5000 entries, you may see error messages similar to this in the log file oidrepld00.log:

2005/04/05:13:21:55 * gslrbssSyncDIT:Replicating namingcontext=dc=com ...... 
2005/04/05:15:36:09 * gslrbssSyncDIT:Subtree delete on dc=com failed. 
Error=DSA is unwilling to perform 
2005/04/05:15:36:09 * gslrbssSyncDIT:Sync failed for namingctx: dc=com, only 
0 entries retrieved

The replication server performs two steps during bootstrap operation. First, in the consumer, it deletes the naming contexts that it has to bootstrap. Second, it copies entries belonging to those naming contexts from supplier to consumer. Deletion by the replication server of a naming context having several thousands of entries results in a big transaction. The undo tablespace must have sufficient space to accommodate a big transaction. If the database's undo tablespace does not have sufficient space, it results in an ORA-30036 error.

Solution

Either have the database administrator add more space to the undo tablespace, or use the bulkdelete tool to delete the required naming context before you start the replication server.

A.17.1.12.5 Changes Are Not Replicated

Changes are not replicated from one node to another.

Problem

The replication server has run out of table space

Solution

Look for the following message in the server log:

OCI Error ORA-1653 : ORA-01653: unable to extend table ODS.ASR_CHG_LOG by 8192 in tablespace OLTS_DEFAULT

Extend the table space and investigate why the table space keeps growing.

Problem

The target Oracle Internet Directory server is down.

Solution

Restart the target Oracle Internet Directory server.

Problem

Various causes

Solution

Make sure the replication server is started on all nodes, in multi-master replication, and at the consumer node in single-master or fan-out replication.

Check the replication log and LDAP log for error messages and fix the cause of the error after investigation.

See Also:

The remtool command-line tool reference in Oracle Fusion Middleware Reference for Oracle Identity Management for more information about using remtool.

A.17.1.12.6 Replication Stops Working

Problem

Data is not replicated between the replicas. In some cases, a working replication setup stops working after OID Human Intervention Queue entries are applied to one of the nodes. In other cases, adding or deleting a new replica causes problems or failures.

Problem

Various causes

Solution

See the following Notes on My Oracle Support (formerly MetaLink), http://metalink.oracle.com:

Note 171693.1, "Resolving Conflicts"

Note 213910.1, "Debugging OID Replication when ASR_CHG_LOG Never Gets Populated."

You can search for Notes by entering a term such as "replication" into the search box.

A.17.1.13 Troubleshooting Change Log Garbage Collection

Both replication and Oracle Directory Integration Platform use change logs to propagate information from a supplier directory to a consumer directory. All change logs are stored in the table ods_chg_log. In addition, replication change logs are stored in asr_chg_log.

This section discusses possible problems you might encounter with change log garbage collection.

Problem

Garbage collection is not working and Oracle Internet Directory is using Oracle Database 11.2.0.1.

Solution

Apply 11.2.0.1.3 PSU to the database.

Problem

Change logs are not being purged due to a replication issue. For example, if a replication server has been down for a few days, replication change logs are not purged because they are needed for replication recovery.

Solution

Resolve the replication issue. See "Troubleshooting Oracle Internet Directory Replication"".

Problem

The attribute orclpurgetargetage is set too high and there are one or more enabled but inactive change log subscribers that do not update orclLastAppliedChangeNumber in their subscriber profiles. Change number-based purging won't purge change logs that are not yet consumed and time-based purging won't purge them because they're not old enough.

Solution

Set the attribute orclpurgetargetageto a smaller value so that change logs are purged sooner.

Solution

Disable inactive changelog subscribers so that change logs are purged by change log number-based purging. Locate such enabled but inactive subscriber profiles by examining the orclLastAppliedChangeNumber in all subscriber profiles by typing:

ldapsearch -v -p port -h host -D cn=orcladmin -q \
           -b "cn=changelog subscriber,cn=oracle internet directory" \
           -s sub "objectclass=orclchangesubscriber" \
           orcllastappliedchangenumber orclsubscriberdisable

Look for an entry that has orclSubscriberDisabled equal to zero and an orclLastAppliedChangeNumber value that never changes. If such an entry exists, and the change log garbage collector's orclpurgetargetage is zero or greater, delete the value of orclpurgetargetage. When orclpurgetargetage is not defined or less than zero, the garbage collector purges changes applied by the replication server, even if another subscriber has not updated its orclLastAppliedChangeNumber.

See Also:

"Overview of Change Log Purging".

A.17.1.14 Troubleshooting Dynamic Password Verifiers

This section lists and describes the ways to troubleshoot the error messages for dynamic password verifiers.

Table A-31 lists and describes the error messages for dynamic password verifiers.

Table A-31 Error Messages for Dynamic Password Verifiers

Error Code	Description
9022	A reversible encrypted password is missing from the user entry.
9023	The crypto type specified in the LDAP request control is not supported.
9024	The username parameter is missing from the LDAP request control.

If the directory is able to compare verifiers, and the comparison evaluates as false, the directory sends the standard error LDAP_COMPARE_FALSE to the client. Similarly, if the user being authenticated lacks a directory entry, the directory sends the standard error LDAP_NO_SUCH_OBJECT.

See Also:

Password Verifier Schema Elements in Oracle Fusion Middleware Reference for Oracle Identity Management.

A.17.1.15 Troubleshooting Oracle Internet Directory Password Wallets

The Oracle Internet Directory Server has two password wallets: oidpwdlldap1 and oidpwdrSID.

The oidpwdlldap1 file contains the DN and password of an ODS user in encrypted format. The Oracle Internet Directory server uses the credential to connect to the back end database at startup time.

A.17.1.15.1 Oracle Internet Directory Server Does Not Start

The oidctl daemon process fails to start an Oracle Internet Directory server instance.

Problem

The password stored in the oidpwdlldap1 wallet is not synchronized with the ODS password in the back end database.

Solution

Try to connect to the database again using the sqlplus command:

sqlplus ods /ods_password@connect_string

If the connection succeeds, try to synchronize the password in the wallet with the ODS password by using the oidpasswd tool to create a new wallet with the correct password. For example, ensure that DOMAIN_HOME is set, then type:

>> oidpasswd connect=connect_string create_wallet=true

If the connection attempt fails, you must login into the back end database as a database administrator and change the ODS password by using the sql command:

>> alter user ods identified by some_new_password

Then try to create a new oidpwdlldap1 to store the new password.

Solution

Try to start the Oracle Internet Directory server again.

The oidpwdrSID file contains the DN and password of a replica DN in an encrypted format. The Oracle Internet Directory replication server uses the credential to connect to the Oracle Internet Directory server at startup time.

This is an example of a replication password wallet, oidpwdrSID:

/------BEGIN REPL CREDENTIAL:cn=replication dn,orclreplicaid=qdinh-sun_
adeldap,cn=replication configuration-----
ezNkZXMtY2JjLXBrY3M1cGFkfQUnaz0TsfzcP0nM1HcHAXchf5mJw+sb4y0bLvvw3RvSg7H
S7/WsKJB02fdSGRlmfWAV+6llkRQ26g==
-----END REPL CREDENTIAL:cn=replication dn,orclreplicaid=qdinh-sun_
adeldap,cn=replication configuration-----/

A.17.1.15.2 Password Not Synchronized

Either oidctl or wlst fails to start an Oracle Internet Directory server instance and the replication server log file oidrepld00.log reports that it is not able to bind.

Problem

The replica DN password stored in the oidpwdrSID is not synchronized with the replica DN password in the Oracle Internet Directory server.

Solution

Try to connect to the Oracle Internet Directory server instance using the ldapbind command. Specify the replica DN stored in oidpwdrSID and the replica DN password. For example:

>> ldapbind -h host -p port -D "cn=replication dn,orclreplicaid=qdinh-sun_adeldap, cn=replication configuration" -q

If the connection succeeds, then you can reset the password in the oidpwdrSID wallet using remtool with the option -pchgwalpwd, which changes the password of the replication DN of a replica only in the wallet. If you do not remember the replication dn password, then you can reset it using remtool with the option -prestpwd, which resets the password of the replication dn of a replica.

After resetting the replication password wallet, restart the replication server instance again a using start(name='instance-name',type='OID') command.

A.17.1.16 Troubleshooting bulkload Errors

Oracle highly recommends that you investigate and correct all errors thrown by bulkload before proceeding with the next step. If you ignore an bulkload error, you are likely to run into serious problems later.

To get more information about the reason for error, run the command with debug enabled (debug=t). Debug information is available in $DOMAIN_HOME/tools/OID/logs/bulkload.log and in the database ods.ds_ldap_log table.

Most bulkload errors occur during data load or during index creation.

Problem

The bulkload command-line tool fails during data load.

Solution

Restore the directory to the state it was in before the data load by using one of these methods:

Solution

Use the bulkload recover option
Restore the database from a backup taken before you invoked bulkload.

Problem

The bulkload command-line tool fails during index creation.

Solution

Examine bulkload.log. Find and fix the specific issue that caused index creation failure. Run bulkload with the index option again.

Failure to correct index errors can cause duplicate entries or duplicate rows in the Oracle Internet Directory's tables.

Problem

The bulkload command-line tool fails because of a broken connection to the database. This can occur, for example, due to a host crash or in to a failover in Real Application Clusters.

Solution

Follow the following procedure:

Ensure that the database is restarted properly.
If the bulkload invocation employed only the check="TRUE" or generate="TRUE" options, but not the load="TRUE" option, go to step 3.

If it was the bulkload load="TRUE" option that failed, you must restore the database to its state before the failure. How you do that depends on whether you have a backup of the database before you issued the bulkload load="TRUE" command.
- If you have a backup, use it to restore the database to its original state before you issued the bulkload command.
- If you do not have a backup, use the bulkload recover command to return the database to its state before the bulkload load="TRUE" command.
Re-issue the bulkload command that failed.

A.17.1.17 Troubleshooting bulkdelete, bulkmodify, and ldifwrite Errors

Oracle highly recommends that you investigate and correct all errors thrown by the bulk tools before proceeding with the next step. To get more information about the reason for error, run the command with debug enabled (debug=t).

Debug information is available in the corresponding log file, bulkdelete.log, bulkmodify.log, or ldifwrite.log, under $DOMAIN_HOME/tools/OID/logs. In the database, debug information is available in the ods.ds_ldap_log table.

Problem

The bulkdelete or bulkmodify command-line tool fails because of a broken connection to the database. This can occur, for example, due to a host crash or in to a failover in Real Application Clusters.

Solution

Ensure that the database is restarted properly. Then retry the bulkdelete or bulkmodify command that failed.

A.17.1.18 Troubleshooting catalog Errors

Oracle highly recommends that you investigate and correct all errors thrown by the bulk tools before proceeding with the next step. To get more information about the reason for error, run the command with debug enabled (debug=t).

.Debug information is available in $DOMAIN_HOME/tools/OID/logs/catalog.log and in database ods.ds_ldap_log table.

Problem

The catalog command-line tool fails because of a broken connection to the database. This can occur, for example, due to a host crash or in to a failover in Real Application Clusters.

Solution

Ensure that the database is restarted properly. Retry the catalog command that failed. If the original invocation employed the add="TRUE" option, the retry might fail because the first command partially completed. If the retry fails, use catalog delete="TRUE" to delete the attribute index, then retry the command again.

Problem

The catalog command throws an error because more than 1000 attributes are present in the file.

Solution

If you need to index more than 1000 attributes, use multiple files.

A.17.1.19 Troubleshooting remtool Errors

The remtool query may hang sometimes and the subsequent efforts to bind to the server with other tool might fail.

Problem

A remtool query such as

remtool -pdispqstat -v -bind host:port

hangs. While it is hanging, attempts to bind to the server with other tools might fail.

Solution

If there is a large backlog of changelogs waiting to be purged, the remtool search query runs for a long time. Ensure that changelog purging is configured appropriately for your environment. See Overview of Change Log Purging.

You can also increase the number of worker threads so that other tools can bind while remtool is running the query. See "Attributes of the Instance-Specific Configuration Entry" and the Oracle Internet Directory chapter in Oracle Internet Directory Performance Tuning in Oracle Fusion Middleware Performance and Tuning Guide.

A.17.1.20 Troubleshooting Server Chaining Error

This section provides information on troubleshooting server chaining error.

Problem

The log contains the error message Server Chaining error followed by javax.naming.AuthenticationException.

Solution

In ODSM, go to the Advanced tab and expand Server Chaining. In each enabled entry, click Verify Login Credential, Verify User Container, and Verify Group Container.

If the verification fails, examine the values you entered for errors. If the problem persists, consult the external directory administrator to verify the accuracy of the values you entered.

A.17.1.21 View Version Information

On the Oracle Directory Services Manager home page for Oracle Internet Directory, you can view version information about Oracle Directory Services Manager, Oracle Internet Directory, and the associated Oracle Database.

For information about using Oracle Directory Services Manager, see Overview of Oracle Directory Services Manager.

A.17.1.22 Troubleshooting Oracle Enterprise Manager Fusion Middleware Control and WLST

Oracle Enterprise Manager Fusion Middleware Control and WLST do not work after the system is patched to 11g Release 1 (11.1.1.4.0).

Problem

Oracle Enterprise Manager Fusion Middleware Control and WLST do not work after the system is patched to 11g Release 1 (11.1.1.4.0).

Solution

This problem occurs if you had SSL server authentication enabled and cipher suites configured prior to patching. To fix this problem after patching, remove the orclsslciphersuite attribute from the instance-specific configuration entry by using ldapmodify. The LDIF file for deleting the orclsslciphersuite attribute in the instance-specific entry is:

dn: cn=componentname,cn=osdldapd,cn=subconfigsubentry
changetype: modify
delete: orclsslciphersuite

The command is:

ldapmodify -D cn=orcladmin -q -p portNum -h hostname -f ldifFile

Restart Oracle Internet Directory.

Problem

Oracle Internet Directory is up and running, but you cannot change Oracle Internet Directory parameters by using Oracle Enterprise Manager Fusion Middleware Control or WLST. You might see the error message: Unable to connect backend OID.

Solution

This can occur if the Oracle Internet Directory port number was changed and the server was not restarted or the Oracle Internet Directory component registration was not updated.

Solution

This occurs if you specify an SSL port configured for server authentication or mutual authentication when using the replication wizard. The replication wizard can only connect to SSL ports that are configured for no authentication. Always specify a non-SSL port or an SSL port configured for no authentication when prompted to log in or when specifying a node.

A.17.1.23 Troubleshooting Oracle Directory Services Manager

This section lists issues related to Oracle Directory Services Manager.

This section includes the following topics:

A.17.1.23.1 Cannot Invoke ODSM from Fusion Middleware Control

Problem

You attempt to invoke Oracle Directory Services Manager from Oracle Enterprise Manager Fusion Middleware Control by selecting Directory Services Manager from the Oracle Internet Directory menu in the Oracle Internet Directory target, then Data Browser, Schema, Security, or Advanced.

ODSM does not open. You might see an error message.

Solution

This is probably an installation problem. See OID with ODSM and Fusion Middleware Control in a New WebLogic Domain in Oracle Fusion Middleware Installation Guide for Oracle Identity Management

A.17.1.23.2 Invoking ODSM from Fusion Middleware Control in Multiple NIC and DHCP Enabled Environment Fails

Problem

The WebLogic Managed Server where Oracle Directory Services Manager is deployed has multiple Network Interface Cards (NIC) or is DHCP enabled. Attempts to invoke Oracle Directory Services Manager from Oracle Enterprise Manager Fusion Middleware Control fail and return 404 errors.

Solution

Use the WebLogic Server Administration Console to change the listen address of the Managed WebLogic Server so that the IP address or hostname in the URL for Oracle Directory Services Manager is accessible.

Perform the following steps:

Using a web browser, access the WebLogic Server Administration Console.
In the left pane of the WebLogic Server Administration Console, click Lock & Edit to edit the server configuration.
In the left pane of the WebLogic Server Administration Console, expand Environment and select Servers.
On the Summary of Servers page, click the link for the WebLogic Managed Server where Oracle Directory Services Manager is deployed.
On the Settings page for the WebLogic Managed Server, update the Listen Address to the host name of the server where Oracle Directory Services Manager is deployed.
Click Save to save the configuration.
Click Activate Changes to update the server configuration.

A.17.1.23.3 Resolving Failover Issues

Problem

When you perform an Oracle Directory Services Manager failover using Oracle HTTP Server, the failover is not transparent.

Solution

The problem sequence and its resolution are as follows:

Oracle Directory Services Manager is deployed in a High Availability active-active configuration using Oracle HTTP Server.
Display an Oracle Directory Services Manager page using the Oracle HTTP Server name and port number.
Make a connection to an Oracle Internet Directory server.
Work with the Oracle Internet Directory server using the current Oracle Directory Services Manager Oracle HTTP Server host and port.
Shut down one managed server at a time using the WebLogic Server Administration Console.
Go back to the Oracle Directory Services Manager page and port, and the connection which was established earlier with Oracle Internet Directory. When you do, a message is displayed advising you to re-establish a new connection to the Oracle Directory Services Manager page.
If you encounter this problem, in your web browser, exit the current Oracle Directory Services Manager page.
Launch a new web browser page and specify the same Oracle Directory Services Manager Oracle HTTP Server name and port.
Re-establish a new connection to the Oracle Internet Directory server you were working with earlier.

Note:

Oracle Directory Services Manager High Availability in The Oracle Fusion Middleware High Availability Guidefor more information about Oracle Directory Services Manager in High Availability configurations.
"Configuring Oracle HTTP Server to Support Oracle Directory Services Manager in an Oracle WebLogic Server Cluster"

Problem

ODSM temporarily loses its connection to Oracle Internet Directory and displays the message LDAP Server is down.

Solution

In a High Availability configuration where ODSM is connected to Oracle Internet Directory through a load balancer, ODSM reports that the server is down during failover from one instance of Oracle Internet Directory to another. In other configurations, this message might indicate that Oracle Internet Directory has been shut down and restarted. In either case, the connection is reestablished in less than a minute, and you are able to continue without logging in again.

Problem

ODSM temporarily loses its connection to an Oracle Internet Directory instance that is using an Oracle RAC database. ODSM might display the message Failure accessing Oracle database (oracle errcode=errcode), where errcode is one of the following values: 3113, 3114, 1092, 28, 1041, or 1012.

Solution

This error can occur during failover of the Oracle Database that the Oracle Internet Directory instance is using. The connection is reestablished in less than a minute, and you are able to continue without logging in again.

A.17.1.23.4 ODSM Displays an Error Message

Problem

ODSM displays the error message: Error :Posn: -1, Size: 0

Solution

This error can be ignored. It usually indicates that Oracle Internet Directory has detected an error in an ODSM operation. JNDI, which ODSM uses to connect to Oracle Internet Directory, sometimes returns this error code instead of the actual error code. Oracle Internet Directory server log files show a more meaningful error message.

A.17.1.23.5 Cursor Loses Focus

Problem

When you access ODSM in accessibility mode, using only the keyboard, in Internet Explorer 7, the cursor loses focus. This behavior has been observed under the following circumstances:

You access the directory in SSL-enabled mode and the server certificate appears.
You type an invalid password and the error dialog appears.

Solution

Press the Tab key nine times, then press the Enter key.

A.17.1.23.6 Second popup of ODSM displays an Unresolvable Error

Problem

You can invoke Oracle Directory Services Manager from Oracle Enterprise Manager Fusion Middleware Control by selecting Directory Services Manager from the Oracle Internet Directory menu in the Oracle Internet Directory target, then Data Browser, Schema, Security, or Advanced.

A new browser window, containing the ODSM Welcome screen, pops up. For example, if select the Schema tab, a popup window opens up and the Schema page gets loaded.

Now, if you attempt to invoke a second ODSM from Oracle Enterprise Manager Fusion Middleware Control by selecting Directory Services Manager from the Oracle Internet Directory menu in the Oracle Internet Directory target, then Data Browser, Schema, Security, or Advanced.

A second browser window pops up and ODSM displays the following error: An unresolvable error has occurred. Please contact your administrator for more information.

Solution

When the error screen comes up, click on the browser back button, and it will take you back to the ODSM page.

A.17.1.24 Troubleshooting a Locked User Account

A user account can sometimes become locked because of multiple ldapbind, ldapcompare, or ldapsearch operations performed by a user using the wrong credentials (password). Applications that depend on this user account can then fail to operate correctly until the account is unlocked.

To unlock the user account, you must first determine the unknown source (IP address) providing the wrong credentials for the LDAP operations.

To find the unknown source causing this problem, follow these steps:

Set the debug level to enable logging to capture all incoming ldapbind, ldapcompare, and ldapsearch operations.

The following examples use oid1 as the component name. The generated log files will be in the following directory:

$DOMAIN_HOME/servers/OID/logs/componentName
1. Create a file named debug.ldif with the following content:
```
dn: cn=oid1,cn=osdldapd,cn=subconfigsubentry
changetype: modify
replace: orcldebugflag
orcldebugflag: 1

dn: cn=oid1,cn=osdldapd,cn=subconfigsubentry
changetype: modify
replace: orcldebugop
orcldebugop: 193
```
  The debug.ldif file enables debugging by setting orcldebugflag to 1. The file also sets the orcldebugop flag to 193 to log ldapbind, ldapcompare, and ldapsearch operations, which are the basic operations that can cause a user account to get locked.
  
  For information about setting the orcldebugop flag, see Table 24-4.
2. Load the LDIF file using the ldapmodify command. For example:
```
ldapmodify -h oid_host -p oid_port -D cn=orcladmin -p admin_password -f debug.ldif
```
For more information about logging, see Managing Logging.
After waiting for the problem to reoccur, check for information about the ldapbind, ldapcompare, and ldapsearch operations in the following Oracle Internet Directory server logs:

$DOMAIN_HOME/servers/OID/logs/componentName/oidldap01s*.log

In this log location, oidldap01s*.log specifies all server logs for the oid1 component. For more information, see Table 24-1.

Because these logs will contain a large amount of information, find the relevant information by searching through the file for "useraccount". On some systems, you can use a grep command. For example:
```
cd $DOMAIN_HOME/servers/OID/logs/componentName
grep -A 5 -B 3 "useraccount" oidldap01s*.log
```
In this example, "useraccount" is the user account that is locked. The -A and -B options, which are supported on some systems, return five lines above and three lines below the line with the user account.

For example, if faadmin is the user account that is locked, the following grep command returns the relevant lines:
```
grep -A 5 -B 3 "faadmin" oidldap01s*.log
```

Analyze the output of the grep command. For example, consider this sample output:

idldapd01s20149-0082.log-[2012-06-26T15:54:45-07:00] [OID] [TRACE:16] []
[OIDLDAPD] [host: adcdk04] [pid: 20149] [tid: 13] [ecid: 
004kqVw6GZG2nJK_ITDCif0004up00Nd8Q,0] ServerWorker (REG):[[
oidldapd01s20149-0082.log-BEGIN
oidldapd01s20149-0082.log ConnID:40614 mesgID:306 OpID:25 OpName:bind
ConnIP:::ffff:10.240.109.21
ConnDN:cn=faadmin,cn=users,dc=us,dc=example,dc=com
oidldapd01s20149-0082.log:INFO : gslfbidbDoBind * Version=3 BIND dn="cn=FAAdmin,cn=Users,dc=us,dc=example,dc=com" method=128
oidldapd01s20149-0082.log- ConnId = 40614, op=25, IpAddr=::ffff:10.240.109.21
oidldapd01s20149-0082.log-2012-06-26T15:54:45 * Adding pwdfailuretime in gslsbmApplyModtoEntry()
oidldapd01s20149-0082.log-2012-06-26T15:54:45 * INFO:gsleswrASndResult
OPtime=4929 micro sec RESULT=49 tag=97 nentries=0
oidldapd01s20149-0082.log-END
oidldapd01s20149-0082.log-]]

This output indicates the IP address of the source that is executing the LDAP operations using the wrong credentials:

Request ID (remains constant for a particular request): ecid: 004kqVw6GZG2nJK_ITDCif0004up00Nd8Q,0
IP Address: IpAddr=::ffff:10.240.109.21
Operation Name: OpName:bind
Bind DN: BIND dn="cn=FAAdmin,cn=Users,dc=us,dc=example,dc=com"

Using information from the previous step, stop the operation causing the problem.

Note:

Multiple LDAP operations might be causing the user to be locked, so continue searching the log file for the source of any other operations that should be stopped.

After you have stopped the operations causing the problem, you can disable debugging by setting the orcldebugflag and orcldebugop values to 0:

Create a file named debug_disable.ldif with the following content:

dn: cn=oid1,cn=osdldapd,cn=subconfigsubentry
changetype: modify
replace: orcldebugflag
orcldebugflag: 0

dn: cn=oid1,cn=osdldapd,cn=subconfigsubentry
changetype: modify
replace: orcldebugop
orcldebugop: 0

Load the LDIF file using the ldapmodify command. For example:

ldapmodify -h oid_host -p oid_port -D cn=orcladmin -p admin_password -f debug_disable.ldif

A.17.1.25 Troubleshooting Policy Store Migration

This section describes the procedure to troubleshoot policy store migration related errors.

If Oracle Internet Directory is used as the Policy Store, during the policy migration from a Fusion Applications dedicated environment to a shared environment, migration of the Security Store can run very slowly because of slow OPSS queries to Oracle Internet Directory. In some cases, the migration can fail because of a timeout.

Solution

To improve OPSS query performance, set the following tuning values for the Oracle Internet Directory Policy Store:

Oracle Database Tuning Parameters
- SGA_MAX_SIZE: 4G or higher
- Oracle Database server processes: 500 or higher

Oracle Internet Directory Attributes

orclecacheenabled: 2 (Enable both Entry Cache and Result Set Cache.)

orclrscacheattr - Set multi-valued attribute as follows:

orclrscacheattr: orcljaznprincipal
orclrscacheattr: orcljaznpermissiontarget
orclrscacheattr: orcljpsresourcename
orclrscacheattr: uniquemember
orclrscacheattr: orcljpsassignee

orclecachemaxsize: 16G or higher

orclinmemfiltprocess - Set multi-valued attribute as follows:

orclinmemfiltprocess: (orcljpsresourcetypename=taskflowresourcetype)
orclinmemfiltprocess: (orcljpsresourcetypename=regionresourcetype

A.17.1.25.1 Troubleshooting Policy Store Migration With Oracle ZFS

If you are using Oracle ZFS and are experiencing performance issues during Policy Store migration, the issue might be related to the Oracle Database using Dynamic Intimate Shared Memory (DISM). The ZFS file system and DISM might be locking over memory page access.

Solution

Set the SGA_MAX_SIZE and SGA_TARGET to the same size (4G), which effectively disables DISM in the database.

A.17.1 Problems and Solutions

A.17.1.1 Installation Errors

A.17.1.2 Oracle Database Server Errors

A.17.1.2.1 Oracle Database Server Connection is Down

A.17.1.2.2 Oracle Database Server Error Due to Interrupted Client Connection

A.17.1.2.3 Oracle Database Server Error Due to Schema Modifications

A.17.1.2.4 Oracle Database TNS Listener Error (ORA-12520)

A.17.1.2.4.1 Troubleshooting Low Oracle Database PROCESSES Parameter Value

A.17.1.2.4.2 Diagnosing Listener Control Utility (lsnrctl) State

A.17.1.3 Directory Server Error Messages and Causes

A.17.1.3.1 Inappropriate Authentication Error

A.17.1.3.2 Constraint Violation Error Due to Editing a User or Group or Creating a Realm

A.17.1.3.3 Standard Error Messages Returned from Oracle Directory Server

A.17.1.3.4 Additional Directory Server Error Messages

A.17.1.4 Core Dump and Stack Trace Occurs When Oracle Internet Directory Crashes

A.17.1.5 TCP/IP Problems

A.17.1.5.1 Do Not Use TCP-Based Monitoring of Server Availability on Windows 2003 Server

A.17.1.5.2 Do Not Install DaimondCS Port Explorer

A.17.1.6 Troubleshooting Password Policies

A.17.1.6.1 Password Policy is Not Enforced

A.17.1.6.2 Password Policy Error Messages

A.17.1.7 Troubleshooting Directory Performance

A.17.1.7.1 Poor LDAP Search Performance

A.17.1.7.2 Improving Poor LDAP Add or Modify Performance

A.17.1.7.3 Diagnosing Poor Oracle Database Server Performance

A.17.1.7.4 Troubleshooting Database Performance Issues Using AWR Reports

A.17.1.7.4.1 Generating an AWR Report

A.17.1.7.4.2 Analyzing an AWR Report

A.17.1.8 Troubleshooting Port Configuration

A.17.1.9 Troubleshooting Starting Oracle Internet Directory

A.17.1.9.1 Oracle Internet Directory is Down

A.17.1.9.2 Oracle Internet Directory is Read-Only

A.17.1.10 Oracle Internet Directory Error Due to Interrupted Client Connection

A.17.1.11 Troubleshooting Starting, Stopping, and Restarting of the Directory Server

A.17.1.11.1 Starting, Stopping, and Restarting the Directory Server Instance Using the Tool

A.17.1.11.2 Problems Starting, Stopping, and Restarting the Directory Server

A.17.1.11.2.1 OIDCTL or OIDMON fails

A.17.1.12 Troubleshooting Oracle Internet Directory Replication

A.17.1.12.1 Bootstrapping Fails

A.17.1.12.2 Log Files to Diagnose Replication Issues

A.17.1.12.3 Replication Server Does Not Start

A.17.1.12.4 Errors in Replication Bootstrap

A.17.1.12.5 Changes Are Not Replicated

A.17.1.12.6 Replication Stops Working

A.17.1.13 Troubleshooting Change Log Garbage Collection

A.17.1.14 Troubleshooting Dynamic Password Verifiers

A.17.1.15 Troubleshooting Oracle Internet Directory Password Wallets

A.17.1.15.1 Oracle Internet Directory Server Does Not Start

A.17.1.15.2 Password Not Synchronized

A.17.1.16 Troubleshooting bulkload Errors

A.17.1.17 Troubleshooting bulkdelete, bulkmodify, and ldifwrite Errors

A.17.1.18 Troubleshooting catalog Errors

A.17.1.19 Troubleshooting remtool Errors

A.17.1.20 Troubleshooting Server Chaining Error

A.17.1.21 View Version Information

A.17.1.22 Troubleshooting Oracle Enterprise Manager Fusion Middleware Control and WLST

A.17.1.23 Troubleshooting Oracle Directory Services Manager

A.17.1.23.1 Cannot Invoke ODSM from Fusion Middleware Control

A.17.1.23.2 Invoking ODSM from Fusion Middleware Control in Multiple NIC and DHCP Enabled Environment Fails

A.17.1.23.3 Resolving Failover Issues

A.17.1.23.4 ODSM Displays an Error Message

A.17.1.23.5 Cursor Loses Focus

A.17.1.23.6 Second popup of ODSM displays an Unresolvable Error

A.17.1.24 Troubleshooting a Locked User Account

A.17.1.25 Troubleshooting Policy Store Migration

A.17.1.25.1 Troubleshooting Policy Store Migration With Oracle ZFS

A.17.2 Need More Help?

A.17.2.1 Oracle Internet Directory Debug Logs

A.17.1.2.4.2 Diagnosing Listener Control Utility (`lsnrctl`) State