Sun Java System Directory Server Enterprise Edition 6.0 Installation Guide

Chapter 12 Troubleshooting

This chapter provides information to help you troubleshoot problems you may encounter while using Identity Synchronization for Windows. The information is organized as follows:

Before you start the troubleshooting process, make sure you have installed the correct versions of the following:

Operating System versions and patches/service packs
Directory Server versions and patches

For information on Operating System and Directory Server correct versions and patches, see Sun Java System Directory Server Enterprise Edition 6.0 Release Notes.

Troubleshooting Checklist

Note –

Administrators: When you are debugging problems, adjust the logging level (as described in Configuring Your Log Files problems.

Some events (such as the program failing to synchronize a user change because the user was not included in the SUL) are not included in a log file until you adjust the log level to FINE or higher. The log level should be left at INFO during all idsync resync operations.

The idsync printstat command can be a useful tool as you install and configure Identity Synchronization for Windows. When you run printstat (see Using printstat ), it displays a list of the remaining steps you have to perform to complete the installation and configuration process.

To Troubleshoot Issues with Identity Synchronization for Windows 6.0

Are there any problems reported in the central error.log?

isw-hostname/logs/central/error.log

Almost all errors will be reported in the central error log file. Also, additional information about any error is usually available in the audit.log file. To ease correlation of related log entries, the audit.log file also includes all entries in the error log.

Is this problem explained as a known issue in the Release Notes document?

Was the installation performed on a clean machine? Problems might occur when this product is reinstalled if the uninstallation of the previous configuration was not complete. Please refer to Chapter 9, Removing the Software.

Was the Core properly installed? If Core installation completed successfully, then log files will exist in the isw-hostname/logs/central/ directory.

Was the Directory Server running during resource configuration?

Is the Core, including the Message Queue and the System Manager, currently running? On Windows, check for the appropriate service name. On Solaris and Linux, check for the appropriate daemon name. Use the idsync printstat command to verify that the Message Queue and System Manager are active.

Was a configuration saved successfully? If the idsync printstat command lists connectors, then a configuration was saved successfully.

Were all connectors installed? One connector must be installed for each directory source being synchronized.

Were all subcomponents installed? Directory Server and Windows NT Connectors require subcomponents to be installed after the Connector installation. The Directory Server Plug-in must be installed in each Directory Server replica.

Were post-installation procedures followed? The Directory Server must be restarted after the Directory Server Plug-in is installed. The Windows NT Primary Domain Controller must be restarted after the Windows NT subcomponents are installed.

Was synchronization started either from the Console or command line?

Are all connectors currently running?

Verify that all connectors are in the SYNCING state using the Console or idsync printstat.

Are the directory sources being synchronized currently running?

Verify using the Console that modifications and/or creates are synchronized in the expected direction(s).

If synchronizing users and groups that existed in only one directory source, were these users and groups created in the other directory source using the idsync resync command?

Note –
You must run idsync resync whenever there are existing users and groups. If you do not resynchronize existing users, resynchronization behavior remains undefined.

If synchronizing users that existed in both directory sources, were these users linked using the idsync resync command?

If user creates fail from Active Directory or Windows NT to the Sun Java System Directory Server, verify that all mandatory attributes in the Directory Server objectclass are specified as creation attributes and values for the corresponding attributes are present in the original user entry.

If synchronizing creates from Directory Server to Windows NT and the user creation succeeded, but the account is unusable, verify that the user name does not violate Windows NT requirements.

For example, if you specify a name that exceeds the maximum allowable length for Windows NT, the user will be created on NT but will remain unusable and uneditable until you rename the user (User -> Rename).

For the Windows NT SAM Change Detector subcomponent to be effective, you must turn on the NT audit log. Select Start -> Programs -> Administrative Tools -> User Manager, and then select Policies -> Audit Policies. Select Audit These Events and then both the Success and Failure boxes for User and Group Management.

Select Event Log Settings in the Event Viewer -> Event Log Wrapping, and then select Overwrite Events as Needed.

Are the users that fail to synchronize within a Synchronization User List? For example, do they match the base DN and filter of a Synchronization User List? In deployments that include Active Directory, on-demand password synchronization fails silently if the Sun Java System Directory Server entry is not in any Synchronization User List. This most often occurs because the filter on the Synchronization User List is incorrect.

Were the synchronization settings changed? If the synchronization settings changed from only synchronizing users from Active Directory to the Sun Java System Directory Server to synchronizing users from the Directory Server to Active Directory, then the Active Directory SSL CA certificate must be added to the connector’s certificate database. The idsync certinfo command reports what SSL certificates must been installed based on the current SSL settings.

Are all host names properly specified and resolvable in DNS? The Active Directory domain controller should be DNS-resolvable from the machine where the Active Directory Connector is running and the machine where the Sun Java System Directory Server Plug-in is running.

Does the IP address of the Active Directory domain controller resolve to the same name that the connector uses to connect to it?

Does the source connector detect the change to the user? Use the central audit.log to determine if the connector for the directory source where the user was added or modified detects the modification.

Does the destination connector process this modification?

Are multiple Synchronization User Lists configured? If so, are these in conflict? More specific Synchronization User Lists should be ordered before less specific ones using the Console.

If flow is set to bidirectional or from Sun to Windows and there are Active Directory data sources in your deployment, are the connectors configured to use SSL communication?

If memory problems are suspected on Solaris or Linux environments check the processes. To view which components are running as different processes, enter

/usr/ucb/ps -gauxwww | grep com.sun.directory.wps

The output gives the full details including the ID of connectors, system manager and central logger. This can be useful to see if any of the processes are consuming excessive memory.

If you are creating or editing the Sun Java System Directory source, and the Directory Server does not display in the Choose a known server drop-down list, check that the Directory Server is running. The Directory Server must be running to appear in the drop down list of available hosts.

If the server in question is down temporarily, type the host and port into the Specify a server by providing a hostname and port field.

Note –
Identity Synchronization for Windows uses a short host name by default; however, the default host name may not work with your configuration. We recommend using a fully qualified name whenever you are asked to provide a host name.

Do you receive the following error while running uninstaller program?

./runInstaller.sh
IOException while making /tmp/SolarisNativeToolkit_5.5.1_1  
executable:java.io.IOException: Not enough space java.io.IOException: Not enough space

Increase the size of the swap file mounted at /tmp.

Troubleshooting Connectors

Use the information in this section to troubleshoot problems with your connectors. The information is organized as follows:

How to Determine the ID of a Connector Managing a Directory Source?

You can use one of the following methods to determine the connector ID:

Using the Central Logs

Determine the connector IDs of the directory sources being synchronized by looking in the central audit.log. At startup, the central logger logs the IDs of each connector and the directory source that it manages. Look for the last instance of the startup banner for the most recent information.

For example, in the following log message there are two connectors:

CNN101 is a Sun Directory Connector that manages dc=example,dc=com
CNN100 is an Active Directory Connector that manages the example.com domain

[2006/03/19 00:00:00.722 -0600] INFO    16
"System Component Information:
SysMgr_100 is the system manager (CORE);
console is the Product Console User Interface;
CNN101 is the connector that manages
[dc=example,dc=com (ldap://host1.example.com:389)];
CNN100 is the connector that manages
[example.com (ldaps://host2.example.com:636)];"

Using idsync printstat

The connector IDs and status are also available from the idsync printstat command (see Using printstat).

A sample output of this command follows:

Connector ID: CNN100
Type:     Active Directory
Manages:  example.com (ldaps://host2.example.com:636)
State:    READY
Connector ID: CNN101
Type:     Sun Java System Directory
Manages: dc=example,dc=com 
(ldap://host1.example.com:389)
State:    READY
Sun Java System 
Message Queue Status:  Started
Checking the System Manager status over the Sun Java System
Message Queue.
System Manager Status:  Started SUCCESS

How to Determine a Connector’s Current State?

You can determine the current state of the connectors involved in synchronization, using the Status pane in the Console, the idsync printstat command (as shown previously), or by looking in the central audit.log.

Search for the last message in the audit.log that reports the connector state. For example, in the following log message you can see that connector CNN101 is in the READY state.

[2006/03/19 10:20:16.889 -0600]
 INFO    13  SysMgr_100 host1
  "Connector [CNN101] is now in state "READY"."

How to Determine a Connector’s Current State describes the different connector states.

Table 12–1 Connector State Meanings


State	Meaning
UNINSTALLED	The connector has not been installed.
INSTALLED	The connector has been installed, but it has not received its configuration.
READY	The connector has been installed and has received its configuration, but it has not started to synchronize.
SYNCING	The connector has been installed, has received its configuration, and has attempted to start synchronizing.

What to Do if the Connector is in the UNINSTALLED State?

Install the connector.

What to Do if the Connector Install Failed but You Cannot Reinstall?

If the connector installation failed, but the Identity Synchronization for Windows installation program thinks that the connector is installed, the installation program will not allow you to reinstall the connector.

Run idsync resetconn (as described in Using resetconn) to reset the connector’s state to UNINSTALLED, and then re-install the connector.

What to Do if the Connector is in the INSTALLED State?

If a connector remains in the installed state for a long period of time, then most likely it is not running, or it is unable to communicate with the Message Queue.

At the machine where the connector was installed, look in the connector’s logs (audit.log and error.log) for potential errors. If the connector cannot connect to the Message Queue, then that error will be reported here. If this is the case, see Troubleshooting Message Queue for possible causes.

If the most recent messages in the audit log are old, then perhaps the connector is not running. See Troubleshooting Components.

What to Do if the Connector is in the READY State?

A connector remains in the READY state until synchronization has been started and all of its subcomponents have been installed and have connected to the connector. If synchronization has not been started, then start it using the Console or command line utility.

If synchronization has been started, but a connector does not enter the SYNCING state, then there is likely a problem with subcomponent. See Troubleshooting Subcomponents

What to Do if the Connector is in the SYNCING State?

If all connectors are in the SYNCING state, but modifications are not being synchronized, then verify that the synchronization settings are correct:

Using the Console, verify that modifications and/or creates are synchronized in the expected direction (for example, from Windows to the Sun Java System Directory Server).
Using the Console, verify that the attribute being modified is a synchronized attribute (note: passwords are always synchronized). If created user entries are not being synchronized, then verify that user creation flow is enabled in the Console.
Does the source connector detect the change to the user? Use the central audit.log to determine if the connector for the directory source where the user was added or modified detects the modification. Does the destination connector process this modification?

What to Do if the Active Directory Connector Fails to Contact Active Directory Over SSL?

If the Active Directory Connector fails to contact Active Directory over SSL and the following error message displays, restart the AD domain controller.

Failed to open connection to
ldaps://server.example.com:636,
error(91): Cannot connect to the LDAP server,
reason: SSL_ForceHandshake failed: (-5938)
Encountered end of file.

What to Do if Detecting and Applying Changes in Active Directory Fails?

If a non-admin account is used for the Active Directory connector, then the default permissions for this user are not sufficient. Some operations such as a resync process from Active Directory to Directory Server succeeds, but other operations such as detecting and applying changes in Active Directory could fail abruptly. For example, if you synchronize the deletions from Active Directory to Directory Server, then even full control is insufficient. To resolve this, you must use a Domain Administrator account for the Active Directory connector.

Troubleshooting Components

Use the information in this section to troubleshoot components. The information is organized as follows:

On Solaris

The command /usr/ucb/ps -auxww | grep com.sun.directory.wps will list all of the Identity Synchronization for Windows processes running. This table shows which processes should be running.

Table 12–2 Identity Synchronization for Windows Processes


Java Process Class Name	Component	When Present
`com.sun.directory.wps.watchdog.server.WatchDog`	System Watchdog	Always
`com.sun.directory.wps.centrallogger.CentralLoggerManager`	Central Logger	Only where Core is installed
`com.sun.directory.wps.manager.SystemManager`	System Manager	Only where Core is installed
`com.sun.directory.wps.controller.AgentHarness`	Connector	One for each connector installed

If the expected number of processes are not running, then issue the following commands to restart all Identity Synchronization for Windows processes.

# /etc/init.d/isw stop
# /etc/init.d/isw start

If the WatchDog process is running, but the expected number of java.exe processes are not running, then see the “Examining WatchList.properties” section to verify that all components were installed properly.

Like other system components, the Sun Java System Directory Server Plug-in sends log records over the bus that are managed by the central logger for end-user viewing. However, the Plug-in also logs some messages that may not show up over the bus (for instance when the subcomponent cannot contact the connector). In this case the log messages only show up in the Plug-in’s logs directory on the file system, which should look something like the following:

serverroot/isw-hostname/logs/SUBCid

Because the Plug-in runs with the Directory Server process, there could potentially be a problem for the Plug-in’s ability to write into its logs directory. This happens if the directory server runs as a different user than the owner of the logs directory. In this case, it may be necessary to give the Plug-in permission explicitly by changing the directories permission or owner using native operating system commands.

On Linux

The command /usr/ucb/ps -auxww | grep com.sun.directory.wps will list all of the Identity Synchronization for Windows processes running. This table shows which processes should be running.

Table 12–3 Identity Synchronization for Windows Processes


Java Process Class Name	Component	When Present
`com.sun.directory.wps.watchdog.server.WatchDog`	System Watchdog	Always
`com.sun.directory.wps.centrallogger.CentralLoggerManager`	Central Logger	Only where Core is installed
`com.sun.directory.wps.manager.SystemManager`	System Manager	Only where Core is installed
`com.sun.directory.wps.controller.AgentHarness`	Connector	One for each connector installed

If the expected number of processes are not running, then issue the following commands to restart all Identity Synchronization for Windows processes.

# /etc/init.d/isw stop
# /etc/init.d/isw start

serverroot/isw/logs/SUBC

On Windows

Using the Service control panel, check that the “Sun Java System Identity Synchronization for Windows service is started. If it is not started, then Identity Synchronization for Windows is not running on that machine, and should be started. If the service is started, then verify using the Task Manager that pswwatchdog.exe (the Watchdog process) is running and that the expected number of java.exe processes are running:

One for the Message Queue broker only if the Core is installed
One for the System Manager only if the Core is installed
One for the Central Logger only if the Core is installed
One for each Connector installed on that machine

Note –

There might be other active java processes, such as the Directory Server Console. If pswwatchdog.exe is not running, then restart the “Sun Java System Identity Synchronization for Windows” service. If it is running but the expected number of java.exe processes are not running, then see Examining WatchList.properties to verify that all components were installed properly.

Examining WatchList.properties

On each machine where a Identity Synchronization for Windows component is installed, the isw-machine_name/resources/WatchList.properties file enumerates the components that should run on that machine. The process.name[n] properties name the components that should be running.

On machines where Core is installed, WatchList.properties will include entries for the Central Logger and System Manager:

process.name[1]=Central Logger
process.name[2]=System Manager

On machines where connectors are installed, WatchList.properties will include a separate entry for each connector. The process.name property is the connector ID:

process.name[3]=CNN100
process.name[4]=CNN101

If there is a mismatch between the entries in WatchList.properties and the actively running processes, then restart the Identity Synchronization for Windows daemon or service.

If there are fewer than expected entries in WatchList.properties (e.g. only one connector entry even though two were installed), then examine the installation logs for possible installation failures.

On Solaris systems: Installation logs are written to /opt/SUNWisw
On Linux Systems: Installation logs are written to /var/opt/sun/isw/logs
On Windows systems: Installation logs are written to the %TEMP% directory, which is a subdirectory of the Local Settings folder located under
```
C:\Documents and Settings\Administrator
```

Note –

On some Windows systems (such as Windows 2000 Advanced Server), the Local Settings folder is a hidden folder.

To View Hidden Folders and the `Temp` subdirectory

Open your Windows Explorer and select Tools -> Folder Options from the menu bar.

When the Folder Options dialog box is displayed, select the View tab and enable the Show Hidden Files option.

Troubleshooting Subcomponents

To Troubleshoot Subcomponents in Your Deployment

Have all subcomponents been installed?

Subcomponent installation must be done after the connector is installed:
- For Active Directory Connectors, no subcomponents are installed.
- For Sun Java System Directory Server Connectors, the Directory Server Plug-in must be installed on the Sun Java System Directory Server being synchronized.
- For Windows NT Connectors, the Windows Change Detector and Password Filter subcomponents must be installed on the primary domain controller for each Windows NT domain being synchronized. These two subcomponents are installed together after the Windows NT Connector has been installed.
Note –
For the Windows NT SAM Change Detector subcomponent to be effective, you must turn on the NT audit log. Select Start -> Programs -> Administrative Tools -> User Manager, and then select Policies -> Audit Policies. Select Audit These Events and then both the Success and Failure boxes for User and Group Management.

Select Event Log Settings in the Event Viewer -> Event Log Wrapping, and then select Overwrite Events as Needed.

Have the subcomponent post-installation steps been followed?

After the Directory Server Plug-in has been installed at the Sun Java System Directory Server, the server must be restarted. After the NT Change Detector and Password Filter have been installed on the primary domain controller, the server must be restarted.

Are the subcomponents running?

Is the Directory Server where the Plug-in was installed running? Is the Primary Domain Controller where the Change Detector and Password Filter were installed running?

Have the subcomponents established a network connection to the connector?

On the machine where the connector is running, verify that the connector is listening for the subcomponent’s connection by running netstat -n -a. The following examples show the results of this command for three different scenarios. (The connector was configured to listen on port 9999.)
1. The connector is listening for incoming connections, and the subcomponent has successfully connected, which is the expected result:
  # netstat -n -a | grep 9999 *.9999 *.* 0 0 65536 0 LISTEN 12.13.1.2.44397 12.13.1.2.9999 73620 0 73620 0 ESTABLISHED 12.13.1.2.9999 12.13.1.2.44397 73620 0 73620 0 ESTABLISHED
2. The connector is listening for incoming connections, but the subcomponent has not connected:
  # netstat -n -a | grep 9999 *.9999 *.* 0 0 65536 0 LISTEN
  After verifying that the subcomponent is running, examine the subcomponent’s local logs for potential problems.
3. The connector is not listening for incoming connections:
  # netstat -n -a | grep 9999 #
  Verify that the correct port number was specified. Verify that the connector is running and is in the READY state. Examine the connector’s local logs for potential problems.

Troubleshooting Message Queue

Verify that the Sun Java System Message Queue broker is running. Issuing a telnet command to the machine and port where the Message Queue broker is running will return a list of the active Message Queue services:

# telnet localhost 7676
Trying 127.0.0.1...
Connected to localhost.
Escape character is \q^]\q.
101 psw-broker 3.0.1
cluster tcp CLUSTER 32914
admin tcp ADMIN 32912
portmapper tcp PORTMAPPER 7676
ssljms tls NORMAL 32913
jms tcp NORMAL 32911
Connection closed by foreign host.

If the “ssljms tcp NORMAL” service is not listed in the output, then examine the Message Queue logs for potential problems.

If the Core was installed on Solaris, then the Message Queue broker’s log is: /var/imq/instances/psw-broker/log/log.txt
If the Core was installed on Linux, then the broker’s log is: /var/opt/sun/mq/instances/isw-broker/log/log.txt
If the Core was installed on Windows, then the broker’s log is: installation_root\isw-machine_name\imq\var\instances\isw-broker\log\log.txt

If the telnet command fails, then either the broker is not running or the wrong port was specified. Check the port number in the broker’s log. The broker’s port is specified in the following line

[13/Mar/2003:18:17:09 CST] [B1004]:
'Starting the portmapper service using tcp
[ 7676, 50 ]
with min threads 1 and max threads of 1'

If the broker is not running, then it can be started on Solaris and on Linux by running /etc/init.d/imq start and on Windows by starting the iMQ Broker Windows service.

If you are installing Message Queue on Solaris 8, and you will be running mquinstall to install all of the packages, be sure to set IMQ_JAVAHOME before running mqinstall to ensure the software picks the correct version of Java.

If you have not yet installed Core, you do not have to set IMQ_JAVAHOME because the Identity Synchronization for Windows installation program tells the Message Queue broker which JVM to use.

Troubleshooting Broker Configuration Directory Communication

The Message Queue broker authenticates clients against the Directory Server that stores the Identity Synchronization for Windows configuration. If the broker cannot connect to this Directory Server, no clients will be able to connect to the Message Queue, and the broker log will mention some javax.naming exception, such as “ javax.naming.CommunicationException” or “javax.naming.NameNotFoundException ”.

If a javax.naming exception occurs, do the following:

Verify that all imq.user_repository.ldap properties in /var/imq/instances/isw-broker/props/config.properties have the correct values. If any of these is incorrect, stop the Message Queue broker, correct and save the file, and restart the broker. The directory server host name must be resolvable from the broker’s machine.
Verify that the imq.user_repository.ldap.password property in /etc/imq/passfile is correct.
In some cases, the broker cannot search for entries if the root suffix contains spaces.

Troubleshooting Broker Memory Settings

During normal operation, the Message Queue broker consumes a modest amount of memory. However, during idsync resync operations, the broker’s memory requirements increase. If the broker reaches its memory limit, undelivered messages will accumulate, the idsync resync operation will slow down dramatically (or stop completely), and Identity Synchronization for Windows might be unresponsive afterwards.

When the broker enters a low-memory state, the following messages will appear in its log:

[03/Nov/2003:14:07:51 CST] [B1089]:
In low memory condition,
Broker is attempting to f
ree up resources [03/Nov/2003:14:07:51 CST] [B1088]:
Entering Memory State [B0024]:
RED  from previous state [B0023]:
ORANGE - current memory is 1829876K,
90% of total memory

To avoid this situation:

Increase the broker’s memory limit to 1 or 2 GB, as explained in Sun Java System Directory Server Enterprise Edition 6.0 Release Notes.
During an idsync resync operation, keep the log level set to INFO. Changing the log level to FINE or higher increases the load at the broker as more log messages are sent to the central logger.
Run idsync resync for a single Synchronization User List at a time.

To Recover from Out of Memory Problems by the Broker

Verify that the broker has a backlog of undelivered messages by examining its persistent message store in the appropriate directory.
- On Solaris: /var/imq/instances/psw-broker/filestore/message/
- On Linux: /var/opt/sun/mq/instances/isw-broker/filestore/message/
- On Windows: installation_root\isw- machine_name\imq\var\instances\isw-broker\filestore\message\

Each file in this directory contains a single undelivered message. If there are more than 10000 files in this directory, then the broker has a backlog of messages. Otherwise, there is another problem with the broker.

The message backlog is probably only log files related to an idsync resync operation and you can safely remove them.

Stop the Message Queue broker as described in Starting and Stopping Services

Remove all files in the persistent message store. The easiest way to remove these files is by recursively removing the message/ directory and then re-creating it.

Restart the Message Queue broker.

Follow the instructions in this section to ensure the broker does not run out of memory again.

Troubleshooting SSL Problems

When diagnosing problems with SSL, also see Chapter 4, Preparing for Installation and Chapter 10, Configuring Security. This section contains:

SSL Between Core Components

The Identity Synchronization for Windows installation program cannot verify that the SSL port provided during Core installation is correct. If you type the SSL port incorrectly during Core installation, then the Core components will not be able to communicate properly. You may not notice a problem until you try to save your configuration for the first time. The Console will display the following warning:

The configuration was successfully saved,
however, the System Manager could not be
notified of the new configuration.

The system manager logs will have the following entry:

[10/Nov/2003:10:24:35.137 -0600] WARNING 14
example  "Failed to connect
to the configuration directory because "Unable to connect: (-5981)
Connection refused by peer."
Will retry shortly."

In this situation, uninstall the Core and install it again with the correct SSL port number.

SSL between Connectors and Directory Server or Active Directory

If a connector is unable to connect over SSL to the Directory Server or Active Directory, then this message will appear in the central error log:

[06/Oct/2006:14:02:48.911 -0600]
WARNING 14  CNN100 host1
"failed to open connection
to ldaps://host2.example.com:636."

Open the Console and check the Specifying Advanced Security Options panel (see Creating an Active Directory Source).

Untrusted Certificates

More information will be available in the central audit log. For example, if the LDAP server’s SSL certificate is not trusted this message will be logged

[06/Oct/2006:14:02:48.951 -0600] INFO
14  CNN100 host1  "failed to open connection to 
ldaps://host2.example.com:636, error(91):
Cannot connect to the LDAP server,
reason: SSL_ForceHandshake failed:
(-8179) Peer's Certificate issuer
is not recognized."

In most situations, the CA certificate has not been added to the connector’s certificate database. This can be confirmed by running the certutil program that ships with the Directory Server.

Note –

Certificate management utilities such as certutil are provided in the SUNWtlsu package, which is not bundled with the Directory Server. (You can download this package from Sun Microsystems at no cost.)

After downloading this package, you will find certutil in:

/usr/sfw/bin/certutil

In this example, the certificate database contains no certificates:

# /usr/sunone/servers/shared/bin/certutil
 -L -d /usr/sunone/servers/
 isw-host1/etc/CNN100
Certificate Name             Trust Attributes
p    Valid peer
P    Trusted peer (implies p)
c    Valid CA
T    Trusted CA to issue client certs (implies c)
C    Trusted CA to certs(only server certs for ssl) (implies c)
u    User cert
w    Send warning

In the following example, the certificate database contains only the Active Directory CA certificate:

# /usr/sunone/servers/shared/bin/certutil -L -d
/usr/sunone/servers/ isw-host1/etc/CNN100
Certificate Name                                 Trust Attributes
example.com CA                                    C,c,
p    Valid peer
P    Trusted peer (implies p)
c    Valid CA
T    Trusted CA to issue client certs (implies c)
C    Trusted CA to certs(only server certs for ssl) (implies c)
u    User cert
w    Send warning

As shown here, the trust flags of the CA certificate must be “ C,,”. If the certificate exists and the trust flags are set properly, but the connector still cannot connect, then first verify that the connector was restarted after adding the certificate, and then use the ldapsearch command that ships with the Sun Java System Directory to help diagnose the problem. If ldapsearch does not accept the certificate, then neither will the connector. For example, ldapsearch can reject certificates if they are not trusted

# /usr/sunone/servers/shared/bin/ldapsearch 
-Z -P /usr/sunone/ servers/isw-host1/etc/CNN100
-h host2 -b "" -s base "(objectclass=*)
"ldap_search: Can't contact LDAP server
SSL error -8179 
Peer's Certificate issuer is not recognized.)

The -P option directs ldapsearch to use connector CNN100’s certificate database for SSL certificate validation. After the correct certificate is added to the connector’s certificate database, verify that ldapsearch accepts the certificate, and then restart the connector.

Mismatched Hostnames

When Identity Synchronization for Windows tries to establish SSL connections (with the trust all certificates setting disabled), the Identity Synchronization for Windows’ Connectors verify that the server’s hostname matches the hostname in the certificate that is presented by the server during the SSL negotiation phase. If the hostnames do not match, the connector will refuse to establish the connection.

The directory source hostname in the Identity Synchronization for Windows configuration must always match the hostname embedded in the certificate used by that directory source.

You can use ldapsearch to verify that the hostnames match, as follows:

/var/mps/serverroot/shared/bin/ldapsearch.exe
-Z -P /var/opt/SUNWisw/etc/CNN100 -3
-h host2.example.com -p 636 
-s base -b "" "(objectclass=*)"

If there is a mismatch between the hostname in the command line (host2.example.com ) and the hostname embedded in the certificate, then the following error message is displayed:

ldap_search: Can't contact LDAP server 
SSL error -12276
(Unable to communicate securely with peer: requested 
do main name does not match 
the server's certificate.)

If the hostnames match, the ldapsearch command is successful and displays the contents of the root DSE.

Expired Certificates

If the server’s certificate has expired, this message will be logged

[06/Oct/2006:14:06:47.130 -0600]
INFO    20  CNN100 host1 
"failed to open connection to ldaps://host2.example.com:636,
error(91): Cannot connect to the LDAP server,
reason: SSL_ForceHandshake failed: 
(-8181) Peer's Certificate has expired."

In this case, the server must be issued a new certificate.

SSL between the Directory Server Plug-in and Active Directory

By default, Directory Server does not communicate with Active Directory over SSL when performing on-demand password synchronization. If the default is overridden to protect this communication with SSL, then the Active Directory CA certificate must be added to the Directory Server certificate database of each master replica as described in Chapter 3, Understanding the Product If this certificate is not added, users will fail to bind to Directory Server with the error “DSA is unwilling to perform.”, and the Plug-in’s log (for example, isw-hostname /logs/SUBC100/pluginwps_log_0.txt) will report:

[06/Nov/2006:15:56:16.310 -0600]
INFO    td=0x0376DD74 logCode=81 
ADRepository.cpp:310
"unable to open connection to Active Directory server 
at ldaps://host2.example.com:636, reason: "

In this situation, you must add the Active Directory CA certificate to Directory Server’s certificate database and restart Directory Server.

Troubleshooting Controller Problems

Some counters will not be reset when you restore an Active Directory domain controller from back-up files.

To ensure all counters will be reset appropriately, resynchronize all users after restoring the an Active Directory domain controller.

Chapter 12 Troubleshooting

Troubleshooting Checklist

To Troubleshoot Issues with Identity Synchronization for Windows 6.0

Troubleshooting Connectors

How to Determine the ID of a Connector Managing a Directory Source?

Using the Central Logs

Using idsync printstat

How to Determine a Connector’s Current State?

What to Do if the Connector is in the UNINSTALLED State?

What to Do if the Connector Install Failed but You Cannot Reinstall?

What to Do if the Connector is in the INSTALLED State?

What to Do if the Connector is in the READY State?

What to Do if the Connector is in the SYNCING State?

What to Do if the Active Directory Connector Fails to Contact Active Directory Over SSL?

What to Do if Detecting and Applying Changes in Active Directory Fails?

Troubleshooting Components

On Solaris

On Linux

On Windows

Examining WatchList.properties

To View Hidden Folders and the Temp subdirectory

Troubleshooting Subcomponents

To Troubleshoot Subcomponents in Your Deployment

Troubleshooting Message Queue

Troubleshooting Broker Configuration Directory Communication

Troubleshooting Broker Memory Settings

To Recover from Out of Memory Problems by the Broker

Troubleshooting SSL Problems

SSL Between Core Components

SSL between Connectors and Directory Server or Active Directory

Untrusted Certificates

Mismatched Hostnames

Expired Certificates

SSL between the Directory Server Plug-in and Active Directory

Troubleshooting Controller Problems

To View Hidden Folders and the `Temp` subdirectory