This section describes the kinds of debug data you need to provide based on the problem you are experiencing.
This section contains the following tasks:
To Collect Required Debug Data For Any Directory Server Problem
To Collect Required Debug Data For Directory Server Installation Problems
To Collect Required Debug Data For an Unresponsive or Hung Directory Server Process
To Collect Required Debug Data For a Crashed Directory Server Process
To Collect Required Debug Data For Directory Server Replication Problems
To Collect Required Debug Data For Directory Server Schema Problems
All problems described in this technical note need basic information collected about when the problem occurred and about the system having the problem. Use this task to collect that basic information.
Note the time or times the problem occurred.
If possible, collect explorer output from SUNWexplo software on the system where the problem occurred.
Provide graphical representation of your deployment.
The graphical representation of your deployment is key to understanding the context of the problem. Show the following in the graphical representation.
All computers involved, with their IP addresses, hostnames, roles in the deployment, operating systems, and versions used.
All other relevant systems, including load balancers, firewalls, and so forth.
If possible, collect a copy of the database on spare disk space or backup media to be used later if necessary.
Database files and the transaction log are often indispensable for diagnosing deadlocks involving database locks.
Note the operating system version.
uname -a
uname -r
cat /etc/redhat-release
C:\Program Files\Common files\Microsoft Shared\MSInfo\msinfo32.exe /report C:\report.txt
Note the patch level.
showrev -p
swlist
rpm -qa
Already provided in C:\report.txt.
Collect Directory Server version information.
server-root/bin/slapd/server/ns-slapd -D instance-dir -V
The server executable is slapd.exe on Windows systems.
Collect the Directory Server configuration file, server-root/slapd-serverID/config/dse.ldif.
Collect Directory Server access, errors, and audit logs.
By default, you find these logs in the following locations:
server-root/slapd-serverID/logs/access
server-root/slapd-serverID/logs/errors
server-root/slapd-serverID/logs/audit (if enabled)
If these log files are not in the default locations, examine the Directory Server configuration file, server-root/slapd-/serverID/config/dse.ldif, to find the paths to the logs. The paths are specified as the values of attributes nsslapd-accesslog, nsslapd-errorlog, and nsslapd-auditlog.
This section describes what data to collect when you cannot complete Directory Server installation.
If the problem concerns a general installation failure for Java Enterprise System, first check the installation troubleshooting chapter in the Installation Guide for your version of Java Enterprise System.
For compressed archive installations, collect installation output showing system calls.
Reinstall while using the appropriate command for your system from the following list. Collect the output of the command that is displayed during installation.
truss -ealf -rall -wall -vall -o /tmp/install-directory-truss.out ./setup
tusc -v -fealT -rall -wall -o /tmp/install-directory-tusc.out ./setup
strace -fv -o /tmp/install-directory-strace.out ./setup
Use DebugView.
DebugView is available at http://www.sysinternals.com/Utilities/DebugView.html.
For Java Enterprise System installations, collect installation error logs.
The log file is named after the date and time that the installation failed. For example, a log file for an installation that failed on December 16 at 3:32 p.m. would have a name like Java_Enterprise_System*_install.B12161532.
On Solaris systems, installation logs are located under /var/sadm/install/logs.
On Red Hat and HP-UX systems, installation logs are located under /var/opt/sun/install/logs.
On Windows systems, installation logs are located under C:\Documents and Settings\current-user\Local Settings\Temp.
This procedure describes what data to collect when Directory Server is still running, but is no longer responding to client application requests.
Collect the data describe in this procedure while the server is hanging.
Note the time during which the hang is seen to occur.
Collect information about the port used during the hang.
netstat -an | grep ds-port
netstat -an
Collect statistics about the system running Directory Server.
ps -aux | grep server-root
vmstat 5 5
iostat -x
top
uptime
ps -aux | grep server-root
vmstat 5 5
iostat -x
top
sar
ps -aux | grep server-root
vmstat 5 5
top
uptime
sar
Get the process ID using the tlist.exe command, then get process details using the same command.
win-dbg-root\tlist.exe pid
Collect swap information.
swap -l
swapinfo
free
Already provided in C:\report.txt.
On Solaris systems, collect output from pstack and pmap five times, once every ten seconds.
pstack pid
pmap -x pid
Get output showing system calls during the hang, by letting each of the commands listed here run for about a minute, then stopping them by typing Control-C.
truss -ealf -rall -wall -vall -o /tmp/truss.out -p pid
tusc -v -fealT -rall -wall -o /tmp/truss.out -p pid
strace -fv -o /tmp/strace.out -p pid
Use Debug View.
DebugView is available at http://www.sysinternals.com/Utilities/DebugView.html.
Collect core files or crash dumps, and related command output.
When the server is hanging, attempt to get several core files that show the state of the server threads over time. You can do this by generating a core file, changing the name of the core file, waiting 30 seconds to a minute, and generating another core file. Repeat the process at least once to get a minimum of three sets of core files and related data.
cd server-root/bin/slapd/server
gcore -o /tmp/directory-core pid
pstack /tmp/directory-core
For each core file on Solaris OS, collect output from the following commands.
cd server-root/bin/slapd/server
file core-file
pstack core-file
pmap core-file
pflags core-file
For at least one of the core files on Solaris OS, collect output from the pkg_app script.
./pkg_app.ksh pid /tmp/directory-core
Here, pid is the server process ID number. See To Run the pkg_app Script for instructions on using pkg_app.
cd server-root/bin/slapd/server
If you have the patches PHKL_31876 and PHCO_32173 patches installed, generate the core file using the gcore command.
gcore -p pid
Otherwise, use the gdb command to generate the core file.
gdb
(gdb) attach pid
Attaching to process pid
No executable file name was specified
(gdb) dumpcore
Dumping core to the core file core.pid
(gdb) quit
The program is running. Quit anyway (and detach it)? (y or n) y
Detaching from program: , process pid
cd server-root/bin/slapd/server
gdb
(gdb) attach pid
Attaching to process pid
No executable file was specified
(gdb) gcore
Saved corefile core.pid
(gdb) backtrace
(gdb) quit
win-dbg-root\tlist.exe
win-dbg-root\adplus.vbs -hang -p pid -o C:\dump-dir
Collect everything in the folder under C:\dump-dir.
Collect output from idsktune.
The idsktune command provides information on system parameters, patch level, tuning recommendations, and so forth. The command is described in the product documentation.
server-root/bin/slapd/server/idsktune
This procedure describes what data to collect when a Directory Server process unexpectedly dies.
Try to restart Directory Server and record the results.
Collect statistics about the system running Directory Server.
ps -aux | grep server-root
vmstat 5 5
iostat -x
top
uptime
ps -aux | grep server-root
vmstat 5 5
iostat -x
top
sar
ps -aux | grep server-root
vmstat 5 5
top
uptime
sar
Get the process ID using the tlist.exe command, then get process details using the same command.
win-dbg-root\tlist.exe pid
Collect swap information.
swap -l
swapinfo
free
Already provided in C:\report.txt.
Collect system logs.
/var/adm/messages
/var/log/syslog
/var/adm/syslog/syslog.log
/var/log/messages
/var/log/syslog
Retrieve the Event log files.
From the Control Panel, open the Event Viewer. Select the event log. Then select Action > Save log file as to save the log.
Collect core files.
For instructions on preparing your system to produce core files or crash dumps in the event of a crash, see 1.6 Configuring the Operating System to Generate Core Files.
For each core file on Solaris OS, collect output from the following commands.
cd server-root/bin/slapd/server
file core-file
pstack core-file
pmap core-file
pflags core-file
For at least one of the core files on Solaris OS, collect output from the pkg_app script.
./pkg_app.ksh pid core-file
Here, pid is the server process ID number. See To Run the pkg_app Script for instructions on using pkg_app.
This procedure describes what data to collect when experiencing inconsistencies and other issues involving Directory Server replication.
Directory Server commands used here are described in the product documentation.
Collect the hostname, IP address, and serverID for each server instance in the replication topology.
Collect a copy of the schema folder, server-root/slapd-serverID/config/schema, and all the files in the folder.
Collect access logs configured to show replication information.
Before you collect access logs as described in To Collect Required Debug Data For Any Directory Server Problem, adjust the log level to keep replication information.
You can use the Console to adjust the access log level.
Alternatively, you can use the ldapmodify command as follows.
$ ldapmodify -h host -p port -D "cn=Directory Manager" -w password dn: cn=config changetype: modify replace: nsslapd-infolog-area # nsslapd-errorlog-level in 5.1 nsslapd-infolog-area: 8192 |
To return to the default log level, use the following command.
$ ldapmodify -h host -p port -D "cn=Directory Manager" -w password dn: cn=config changetype: modify replace: nsslapd-infolog-area # nsslapd-errorlog-level in 5.1 nsslapd-infolog-area: 0 |
You must restart Directory Server for the change to take effect.
Provide a listing of the changelog directory.
ls -la changelog-dir
dir changelog-dir
If you cannot find the change log, examine the Directory Server configuration file, server-root/slapd-serverID/config/dse.ldif, to find the path. The path is specified as the values of attribute nsslapd-changelogdir.
Collect output from the insync command.
The insync command indicates the state of synchronization between a master replica and one or more consumer replicas. The following command shows the state over a period of 30 seconds.
server-root/shared/bin/insync -s "cn=Directory Manager:password@hostname1:ldap-port" -c "cn=Directory Manager:password@hostname2:ldap-port" 30
Collect output from the repldisc command.
The repldisc command displays the replication topology, building a graph of all known replica, then showing the result as a matrix.
server-root/shared/bin/repldisc -D "cn=Directory Manager" -w password -b base-dn -s host:ldap-port
Here, base-dn is the DN of the replicated suffix.
This procedure describes what data to collect when experiencing Directory Server schema violation errors.
Collect a copy of the schema folder, server-root/slapd-serverID/config/schema, and all the files in the folder.
Indicate whether you have the same problem both in the Console and on the command line.
Provide a list of the last changes made to the schema files.
If the schema violation occurred during an add, modify, delete, or replace operation, provide an LDIF representation of the changes and a list of the commands used.