To Gather Debug Data on Web Server Crashed Process

Use this task to gather data when a Web Server process has stopped (crashed) unexpectedly. Run all the commands on the actual machine where the core file(s) were generated.

1. Gather the general system information as explained in To Gather General Debug Data for Any Web Server Problem.

2. Try to restart Web Server.

3. If the Web Server is using a Directory Server, provide the access, errors and audit logs of the Directory Server used by the Web Server

   • Access log

     UNIX (Solaris and HP-UX) and Linux

     server-root/slapd-identifier/logs/access

     Windows

     server-root\slapd-identifier\logs\access

   • Errors log

     UNIX (Solaris and HP-UX) and Linux

     server-root/slapd-identifier/logs/errors

     Windows

     server-root\slapd-identifier\logs\errors

   • Audit log

     UNIX (Solaris and HP-UX) and Linux

     server-root/slapd-identifier/logs/audit

     Windows

     server-root\slapd-identifier\logs\audit

   ---

   Note –

   The paths of these logs files are specified by the following parameters in the dse.1dif file.nsslapd-accesslog,nsslap-errorlog, and nsslapd-auditlog

   The dse.1dif file is located in the config directory.

   UNIX (Solaris and HP-UX) and Linux

   server-root/slapd-identifier/config/dse.ldif

   Windows

   server-root\slapd-identifier\config\dse.ldif

   ---

4. Get the output of the following commands.

   Solaris

   ps -aux | server-rootvmstat 5 5iostat -xtopuptime

   HP-UX

   ps -aux | server-rootvmstat 5 5iostat -xtopsar

   Linux

   ps -aux | server-rootvmstat 5 5topuptimesar

   Windows

   Obtain the WEB process PID:

   C:\windbg-root>tlist.exe

   Obtain process details of the WEB running process PID:

   C:\windbg-root>tlist.exe web-pid

5. Get the swap information.

   Solaris

   swap -l

   HP-UX

   swapinfo

   Linux

   free

   Windows

   Already provided in C:\report.txt as described in To Gather General Debug Data for Any Web Server Problem.

6. Get the system logs.

   Solaris and Linux

   /var/adm/messages/var/log/syslog

   HP-UX

   /var/adm/syslog/syslog.log

   Windows

   Event log files:Start-> Settings-> Control Panel —> Event Viewer-> Select LogThen click Action-> Save log file as and type the name for the resulting file.

7. Get core files (called “Crash Dumps” in Windows).

   Solaris

   See 1.6 Configuring Solaris to Generate Core Files if a core file was not generated.

   Linux

   Core dumps are turned off by default in the /etc/profile file. You can make user-specific changes by editing your ~/.bash_profile file. Look for the following line:

   ulimit -S -c 0 > /dev/null 2>&1

   You can either comment out the entire line to set no limit on the size of the core files or set your own maximum size.

   Windows

   Generate a crash dump during a crash of Web Server by using the following commands:

   Get the WEB process PID :

   C:\windbg-root>tlist.exe

   Generate a crash dump when the WEB process crashes, by executing the following commands:

   C:\windbg-root>adplus.vbs -crash -FullOnFirst -p web-pid -o C:\crashdump_dir

   The adplus.vbs command monitors web-pid until it crashes and generates the dmp file. Provide the complete generated folder under C:\crashdump_dir.

   ---

   Note –

   If you have not installed the Debugging Tools for Windows, you can use the drwtsn32 -i command to select Dr. Watson as the default debugger. Use the drwtsn32 command, check all options, and choose the path for crash dumps. Then provide the dump and the drwtsn32.log files.

   ---

8. (Solaris) For each core file, provide the output of the following commands.

   cd server-root/bin/https/bin
   file corefile
   pstack corefile
   pmap corefile
   pflags corefile
   

9. (Solaris) Archive the result of the script pkg_app (one core file is sufficient).

   ./pkg_app.ksh Pid-of-application corefile
   

   ---

   Note –

   The Sun Support Center must have the output from the pkg_app script to properly analyze the core file(s). For more information on how to run the pkg_app script, see To Run the pkg_app Script.

   All these commands must be executed on the actual machine where the core file(s) were generated.

   ---