Sun Java System Directory Server Enterprise Edition 6.0 Installation Guide

Troubleshooting SSL Problems

When diagnosing problems with SSL, also see Chapter 4, Preparing for Installation and Chapter 10, Configuring Security. This section contains:

SSL Between Core Components

The Identity Synchronization for Windows installation program cannot verify that the SSL port provided during Core installation is correct. If you type the SSL port incorrectly during Core installation, then the Core components will not be able to communicate properly. You may not notice a problem until you try to save your configuration for the first time. The Console will display the following warning:

The configuration was successfully saved,
however, the System Manager could not be
notified of the new configuration.

The system manager logs will have the following entry:

[10/Nov/2003:10:24:35.137 -0600] WARNING 14
example  "Failed to connect
to the configuration directory because "Unable to connect: (-5981)
Connection refused by peer."
Will retry shortly."

In this situation, uninstall the Core and install it again with the correct SSL port number.

SSL between Connectors and Directory Server or Active Directory

If a connector is unable to connect over SSL to the Directory Server or Active Directory, then this message will appear in the central error log:

[06/Oct/2006:14:02:48.911 -0600]
WARNING 14  CNN100 host1
"failed to open connection
to ldaps://host2.example.com:636."

Open the Console and check the Specifying Advanced Security Options panel (see Creating an Active Directory Source).

Untrusted Certificates

More information will be available in the central audit log. For example, if the LDAP server’s SSL certificate is not trusted this message will be logged

[06/Oct/2006:14:02:48.951 -0600] INFO
14  CNN100 host1  "failed to open connection to 
ldaps://host2.example.com:636, error(91):
Cannot connect to the LDAP server,
reason: SSL_ForceHandshake failed:
(-8179) Peer's Certificate issuer
is not recognized."

In most situations, the CA certificate has not been added to the connector’s certificate database. This can be confirmed by running the certutil program that ships with the Directory Server.

Note –

Certificate management utilities such as certutil are provided in the SUNWtlsu package, which is not bundled with the Directory Server. (You can download this package from Sun Microsystems at no cost.)

After downloading this package, you will find certutil in:

/usr/sfw/bin/certutil

In this example, the certificate database contains no certificates:

# /usr/sunone/servers/shared/bin/certutil
 -L -d /usr/sunone/servers/
 isw-host1/etc/CNN100
Certificate Name             Trust Attributes
p    Valid peer
P    Trusted peer (implies p)
c    Valid CA
T    Trusted CA to issue client certs (implies c)
C    Trusted CA to certs(only server certs for ssl) (implies c)
u    User cert
w    Send warning

In the following example, the certificate database contains only the Active Directory CA certificate:

# /usr/sunone/servers/shared/bin/certutil -L -d
/usr/sunone/servers/ isw-host1/etc/CNN100
Certificate Name                                 Trust Attributes
example.com CA                                    C,c,
p    Valid peer
P    Trusted peer (implies p)
c    Valid CA
T    Trusted CA to issue client certs (implies c)
C    Trusted CA to certs(only server certs for ssl) (implies c)
u    User cert
w    Send warning

As shown here, the trust flags of the CA certificate must be “ C,,”. If the certificate exists and the trust flags are set properly, but the connector still cannot connect, then first verify that the connector was restarted after adding the certificate, and then use the ldapsearch command that ships with the Sun Java System Directory to help diagnose the problem. If ldapsearch does not accept the certificate, then neither will the connector. For example, ldapsearch can reject certificates if they are not trusted

# /usr/sunone/servers/shared/bin/ldapsearch 
-Z -P /usr/sunone/ servers/isw-host1/etc/CNN100
-h host2 -b "" -s base "(objectclass=*)
"ldap_search: Can't contact LDAP server
SSL error -8179 
Peer's Certificate issuer is not recognized.)

The -P option directs ldapsearch to use connector CNN100’s certificate database for SSL certificate validation. After the correct certificate is added to the connector’s certificate database, verify that ldapsearch accepts the certificate, and then restart the connector.

Mismatched Hostnames

When Identity Synchronization for Windows tries to establish SSL connections (with the trust all certificates setting disabled), the Identity Synchronization for Windows’ Connectors verify that the server’s hostname matches the hostname in the certificate that is presented by the server during the SSL negotiation phase. If the hostnames do not match, the connector will refuse to establish the connection.

The directory source hostname in the Identity Synchronization for Windows configuration must always match the hostname embedded in the certificate used by that directory source.

You can use ldapsearch to verify that the hostnames match, as follows:

/var/mps/serverroot/shared/bin/ldapsearch.exe
-Z -P /var/opt/SUNWisw/etc/CNN100 -3
-h host2.example.com -p 636 
-s base -b "" "(objectclass=*)"

If there is a mismatch between the hostname in the command line (host2.example.com ) and the hostname embedded in the certificate, then the following error message is displayed:

ldap_search: Can't contact LDAP server 
SSL error -12276
(Unable to communicate securely with peer: requested 
do main name does not match 
the server's certificate.)

If the hostnames match, the ldapsearch command is successful and displays the contents of the root DSE.

Expired Certificates

If the server’s certificate has expired, this message will be logged

[06/Oct/2006:14:06:47.130 -0600]
INFO    20  CNN100 host1 
"failed to open connection to ldaps://host2.example.com:636,
error(91): Cannot connect to the LDAP server,
reason: SSL_ForceHandshake failed: 
(-8181) Peer's Certificate has expired."

In this case, the server must be issued a new certificate.

SSL between the Directory Server Plug-in and Active Directory

By default, Directory Server does not communicate with Active Directory over SSL when performing on-demand password synchronization. If the default is overridden to protect this communication with SSL, then the Active Directory CA certificate must be added to the Directory Server certificate database of each master replica as described in Chapter 3, Understanding the Product If this certificate is not added, users will fail to bind to Directory Server with the error “DSA is unwilling to perform.”, and the Plug-in’s log (for example, isw-hostname /logs/SUBC100/pluginwps_log_0.txt) will report:

[06/Nov/2006:15:56:16.310 -0600]
INFO    td=0x0376DD74 logCode=81 
ADRepository.cpp:310
"unable to open connection to Active Directory server 
at ldaps://host2.example.com:636, reason: "

In this situation, you must add the Active Directory CA certificate to Directory Server’s certificate database and restart Directory Server.