Oracle Fusion Middleware Troubleshooting Guide for Oracle Directory Server Enterprise Edition

Chapter 6 Troubleshooting Data Management Problems

This chapter provides information to help you troubleshoot Directory Server and Directory Proxy Server data management problems.

This chapter contains the following sections:

Troubleshooting LDAP Operation Failures

This section describes how to troubleshoot LDAP operation failures. It describes the possible causes of the operation failures, the information to collect to help you troubleshoot the problem, and how to analyze this information.

Possible Causes of an Operation Failure

An operation may fail for the following reasons:

Collecting and Analyzing Data About Operation Failures

To determine if ACIs are the source of your problem, gather information about all of the ACIs from the suffix level to the entry you are trying to access. Gather this data using the ldapsearch operation as follows:


# ldapsearch -b base-suffix -D "cn=Directory Manager"  -w - \
-s scope "(objectclass=*)" aci

Collect the access and errors log files that contain the operation. Be sure to enable the ACI logging level. Enable the ACI logging level for the errors log file as follows:


# dsconf set-log-prop errors level:err-acl

Enable the ACI logging level for the access log file as follows:


# dsconf set-log-prop access level:acc-internal

To view the contents of the error log, use the dsadm command as follows:


dsadm show-error-log -A duration [-L last-lines] install-path

The -A option specifies the maximum age of lines to be returned from the log. For example, to search for all entries younger than 24 hours, use -A 24h. The -L option specifies the number of lines to be returned from the log. For example, to return the last 50 lines, use -L 50. By default, 20 lines are returned.

To view the access log, use the dsadm command as follows:


dsadm show-access-log -A duration [-L last-lines] install-path

The log files themselves are located in the following directories:


instance-path/logs/errors*
instance-path/logs/access*

If you are unable to troubleshoot your problem yourself, collect the error and access log files from the time during which the database was inaccessible and send them to Oracle Support for analysis. By default, the log files are located in the instance-path/logs directory. To find the path to your error and access logs, use the following command:


# dsconf get-log-prop ERROR path

or


# dsconf get-log-prop ACCESS path

Troubleshooting SSL Problems

This section helps you troubleshoot when an SSL connection fails. It includes the following sections:

This chapter contains the following sections:

For information about troubleshooting SSL problems with Identity Synchronization for Windows, see Troubleshooting Problems With Identity Synchronization for Windows Over SSL

Overview of Important SSL Concepts

This section describes concepts to help you troubleshoot problems using SSL for Directory Server multi-master replication. Problems with SSL always appear on the supplier side. The error log will contain security related messages such as “SSL init failed.” or “Certificate not accepted.”

SSL connections always involve two participants:

The SSL client initiates requests and the SSL server always receives the requests. During this exchange, the SSL server must provide credentials. Any SSL server needs to verify the credentials sent by the SSL client. In order to make this verification, the certificate database on the peer must contain the CA certificate of the certificate sent by the other peer.

In replication, SSL must be enabled in all replicas, even master replicas that only accept non-SSL operations. For example, a master server communicates with a hub server using SSL. The hub must listen on the SSL port. The master does not need listen on the SSL port because it is an SSL client. However, it must still define an SSL port, otherwise Directory Server can not initiate SSL certificate exchange for communication with the host server.

By default, SSL is enabled on all Directory Server instances. For a detailed explanation of how SSL works, see Secure Sockets Layer (SSL) in Oracle Fusion Middleware Reference for Oracle Directory Server Enterprise Edition.

Possible Causes of SSL Problems

Failure of an SSL connection could be the result of one of the following:

Collecting and Analyzing SSL Data

This section provides information about collecting and analyzing data to help you troubleshoot SSL problem, including problems replicating over SSL.

About the ssltap Tool

The Mozilla website provides NSS Security Tools that are helpful for debugging and troubleshooting SSL problems. You can obtain the source-code of the ssltap tools from http://www.mozilla.org/projects/security/pki/nss/tools.

The ssltap tool can capture the SSL communications between two systems. You must place the ssltap program between the connection from a Directory Server and an LDAP client. The program behaves like a Directory Server when it communicates with the LDAP client and behaves like the LDAP client when communicating with the Directory Server.

Verifying the Certificates Using dsadm

The certificates database resides instance-path/alias directory. Get the contents of this directory for each server involved in the problem.

For example, to see a list of the certificates that can be used as ns-slapd certificates (certificates with a u,, trust flags) use the dsadm command as follows:


dsadm list-certs instance-path

The command lists the certificates, such as defaultCert, the date from which it is valid, the date it expires, whether it is self-signed, who issued it, and to whom it is issued.

To see information about valid and trusted CA certificates (certificates with CT,, trust flags) use the dsadm command as follows:


dsadm list-certs --ca instance-path

This command provides the certificate alias, its dates of validity and expiration, whether it is built in, who issued it, and to whom it was issued. Verify that the SSL server and client certificates are generated by a certificate authorities that appear in the output of this command.

For detailed information about a particular certificate, use the dsadm command as follows:


dsadm show-cert instance-path certificate-alias

For example, the output of this command appears as follows:


server1 [/var/dsee/instances]> dsadm show-cert ds1 defaultCert
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            00:85:8b:13:ef
        Signature Algorithm: PKCS #1 MD5 With RSA Encryption
        Issuer:
            "CN=server1,CN=Directory Server,O=example.com"
        Validity:
            Not Before: Fri Mar 23 14:10:51 2007
            Not After : Sat Jun 23 14:10:51 2007
        Subject:
            "CN=server1,CN=Directory Server,O=example.com"
        Subject Public Key Info:
            Public Key Algorithm: PKCS #1 RSA Encryption
            RSA Public Key:
                Modulus:
                    9a:c9:52:bd:ec:32:43:1a:39:96:90:02:f5:7e:18:45:
                    78:37:ca:8d:8f:c4:cc:6f:d1:7e:6c:38:d1:a1:53:41:
                    96:67:07:c7:c8:56:78:d1:f2:24:df:1f:eb:b2:07:5d:
                    6e:1f:58:fa:7a:f2:00:e4:95:d1:57:97:37:9d:22:31:
                    1c:b7:99:29:df:a3:8a:2a:87:e1:8b:54:ea:1f:7c:b7:
                    28:23:ce:be:7e:73:b3:87:f5:32:88:56:4e:58:68:f6:
                    f6:01:2c:51:ca:07:00:40:ca:b3:9e:33:40:e8:f2:18:
                    bc:16:d4:ac:ae:69:a7:c9:d7:g5:34:d4:87:11:2c:b1
                Exponent: 65537 (0x10001)
    Signature Algorithm: PKCS #1 MD5 With RSA Encryption
    Signature:
        29:76:4f:9f:ca:00:09:7b:05:ac:0f:26:6f:d1:93:aa:
        a8:c0:eb:a9:2a:39:e2:6e:08:0a:90:41:e5:7f:18:4a:
        17:05:03:04:9b:ee:0a:dc:3c:ef:ee:aa:fc:ea:85:bf:
        f9:05:32:65:35:2c:e8:1f:32:9d:d6:a7:aa:68:a4:7a:
        e8:d9:4a:a0:a6:bc:fd:36:ba:d3:80:8a:1b:d3:81:8a:
        68:1a:73:cc:36:7a:92:dc:eb:ec:af:02:6b:14:c7:77:
        e3:7d:95:19:e7:17:9d:d2:35:67:60:6b:9f:9b:d9:af:
        01:f2:55:7f:5f:ce:23:a0:49:67:01:cd:30:38:8b:d2
    Fingerprint (MD5):
        B8:34:27:AA:02:F6:07:FC:8F:D1:4A:AD:38:29:09
    Fingerprint (SHA1):
        3C:3B:BD:15:E8:1F:68:2E::E8:EJ:02:63:CD:8F:39:BE:DD:70

    Certificate Trust Flags:
        SSL Flags:
            Valid CA
            Trusted CA
            User
            Trusted Client CA
        Email Flags:
            User
        Object Signing Flags:
            User

Confirm the validity of the certificate. Also, confirm that the issuer of the certificate is a valid and trusted certificate authority.

Checking Client Authentication Settings

You can configure client authentication to be required or allowed. Verify the setting client authentication settings by using DSCC or by using the dsconf get-server-prop ssl-client-auth-mode command.


Note –

User's of migrated 5.2 instances of Directory Server can verify the client authentication settings by checking the nsSSLClientAuth property in the dse.ldif file.


ProcedureTo Verify Client Authentication Settings Using the DSCC

  1. Go to the Directory Servers tab in the DSCC, and select the server from the table.

  2. Click the Security tab and then the General tab.

  3. In the Client Authentication section, go to LDAP Settings.

    If you want only the SSL server to require the certificate, select Allow Certificate Based Client Authentication.

    If you want both the SSL server and the SSL client to require a certificate, select Require Certificate Based Client Authentication.

Checking the Libraries

Get a list of all the dynamically loaded libraries to see which NSS/SSL and NSPR libraries are being loaded. To get the list of dynamically loaded libraries on Solaris Intel or Linux, use the following command:


# cd install-path/lib; ldd ns-slapd

To get the list of dynamically loaded libraries on Solaris SPARC, Solaris AMD64 or HPUX, use the following command:


# cd install-path/lib/64;  ldd ns-slapd

The dynamically loaded libraries will be located in the following directory:


install-path/lib/private

Verify SSL Communications Using the ssltap Tool

You can use the ssltap tool to check if the hand shake is working on your system. The tool works like an SSL proxy, showing the communications between the LDAP client and the Directory Server and the packages being exchanged. For example, using this tool you might see where the server asks for a certificate but the client does not send the certificate or where the client proposes a cipher suite that the server does not support.

Since the SSL port 636 is hard-coded on the client side, the ssltap tool run on the Directory Server, where it must list on port 636 for incoming client requests. The SSL port of the Directory Server needs to be changed to a number other than 636 while running the ssltap tool.

For example, run ssltap as follows:


ssltap -vhfsxl -p 636 localhost:637 > output.html

After running some simple LDAP request on the client, such as ldaplist, the tool should have captures some SSL packets. Stop the tool by pressing CTRL-C and view the output file in a browser window. The output data is color coded so that data sent by the client is marked in blue and data sent by the server is marked in red.