Troubleshooting Memory Leaks

This section describes how to troubleshoot a memory leak.

Possible Causes of a Memory Leak

Memory leaks are caused by problems allocating memory, either in Directory Server itself or in custom plug-ins. Troubleshooting these problems can be very difficult, particularly in the case of custom plug-ins.

Collecting Data About a Memory Leak

It is important to do the following before collecting data about your memory leak:

• Disable any custom plug-ins

• Reduce the cache setting to very low values

• Enable the audit log

One you have done the above, run a test that proves your memory leak. During the life of the test run, gather output from the pmonitor utility as follows:

The pmonitor utility is a process monitor.

Collect the generic Directory Server data, as described in Collecting Generic Data. This data includes the version of Directory Server that you are running, logs from the test run, in particular the audit log, and the Directory Server configuration file.

With the data you collected, you can now contact the Sun Support Center for assistance with your problem.

Analyzing Memory Leaks Using the libumem Library

On Solaris systems, the libumem library is a memory agent library that tracks all of the address allocated into the process memory footprint. Usually it is not used in a production environment because it is much slower. However, it is helpful for analyzing the cause of a memory leak. For more information about the libumem library, see the technical article at the following location: http://access1.sun.com/techarticles/libumem.html

Restart the Directory Server using the following command:

# SUN_SUPPORT_SLAPD_NOSH=true LD_PRELOAD=libumem.so \
UMEM_DEBUG=contents,audit=40,guards UMEM_LOGGING=transaction ./start-slapd

The libumem library is now loaded before the Directory Server starts, instead of using SmartHeap.

Next, run the gcore command several times, once before the memory use started to grow and once after. The gcore command provides a list of addresses and pointers. Use these to read the libumem library.

# cd install-root/bin/slapd/server
gcore -o /tmp/directory-core process-id

Finally, use the mdb and splitrec tools to analyze the results. The splitrec tool compares the results to see the complete stack of the leak.

# cd install-root/bin/slapd/server
echo "::umausers -e" | mdb ./ns-slapd path_gcore1 > res.1
eacho "::umausers -e" | mdb ./ns-slapd path_gcore2 > res.2
splitrec -1 res.1 res.2

The splitrec tool is available through Sun Support. This tool provides a summary of the stacks that have been identified as responsible for leaking allocation stacks. Sun Support can use the contents of these stacks to identify known memory leaks in the SunSolve database. Sometimes the splitrec tool does not provide any output because by default it is configured to report leaks only for stacks that have been identified as leaking more than 100 times. Configure this limit to a lower value using the splitrec -l option.