1.5.4 To Collect Debug Data on a Web Proxy Server Crashed Process (Sun Gathering Debug Data for Sun Java System Web Proxy Server)

Sun Gathering Debug Data for Sun Java System Web Proxy Server

Previous: 1.5.3 To Collect Debug Data on a Hung or Unresponsive Web Proxy Server Process
Next: 1.5.5 To Collect Debug Data on Web Proxy Server Authentication Problems

1.5.4 To Collect Debug Data on a Web Proxy Server Crashed Process

Use this task to collect data when a Web Proxy Server process has stopped (crashed) unexpectedly. Run all the commands on the actual machine where the core file(s) were generated.

Collect the general system information as explained in To Collect Required Debug Data for Any Web Proxy Server Problem.

Note whether you can restart Web Proxy Server.

If the Web Server is using a Directory Server, provide the access, errors and audit logs of the Directory Server used by the Web Server
- Access log
  
  UNIX - Linux
  
  server-root/slapd-identifier/logs/access
  
  Windows
  
  server-root\slapd-identifier\logs\access
- Errors log
  
  UNIX - Linux
  
  server-root/slapd-identifier/logs/errors
  
  Windows
  
  server-root\slapd-identifier\logs\errors
- Audit log
  
  UNIX - Linux
  
  server-root/slapd-identifier/logs/audit (if enabled)
  
  Windows
  
  server-root\slapd-identifier\logs\audit (if enabled)
Note –
The paths of these logs files are specified by the following parameters in the dse.ldif file: nsslapd-accesslog, nsslap-errorlog, and nsslapd-auditlog

The dse.1dif file is located in the config directory:

UNIX and Linux: server-root/slapd-identifier/config/dse.ldif

Windows: server-root\slapd-identifier\config\dse.ldif

Check if the problem is reproducible. If yes, provide a test case for reproducing the problem.

Get the output of the following commands.

Solaris

ps -aux | grep server-rootvmstat 5 5iostat -xtopuptime

HP-UX

ps -aux | grep server-rootvmstat 5 5iostat -xtopsar

Linux

ps -aux | grep server-rootvmstat 5 5topuptimesar

Windows

Obtain the PROXY process PID: C:\windbg-root>tlist.exe

Obtain process details of the PROXY running process PID: C:\windbg-root>tlist.exe proxy-pid

Note –
Install the debugging tools to use the debug command. You can download the same at http://www.microsoft.com/whdc/devtools/debugging/default.mspx. Install the latest version of debugging tools and the OS symbols for the version of Windows that you are using.

Get the swap information.

Solaris

swap -l

HP-UX

swapinfo

Linux

free

Windows

Already provided in C:\report.txt as described in To Collect Required Debug Data for Any Web Proxy Server Problem.

Get the system logs.

Solaris and Linux

/var/adm/messages/var/log/syslog

HP-UX

/var/adm/syslog/syslog.log

Windows

Event log files:Start> Settings>Control Panel> Event Viewer> Select LogThen click Action> Save log file as

Get core files (called “Crash Dumps” by Windows).

Solaris

See Configuring Solaris to Generate Core Files if a core file was not generated.

Linux

Core dumps are turned off by default in the /etc/profile file. You can make user-specific changes by editing your ~/.bash_profile file. Look for the following line:

ulimit -S -c 0 > /dev/null 2>&1

You can either comment out the entire line to set no limit on the size of the core files or set your own maximum size.

Windows

Generate a crash dump during a crash of Web Proxy Server by using the following commands:

Get the Proxy process PID : C:\windbg-root>tlist.exeGenerate a crash dump when the Proxy process crashes: C:\windbg-root>adplus.vbs -crash -FullOnFirst -p proxy-pid -o C:\crashdump_dir

The adplus.vbs command watches proxy-pid until it crashes and will generate the dmp file. Provide the complete generated folder under C:\crashdump_dir.

Note –
If you have not installed the Debugging Tools for Windows, you can use the drwtsn32.exe -i command to select Dr. Watson as the default debugger. Use the drwtsn32.exe command, check all options, and choose the path for crash dumps. Then provide the dump and the drwtsn32.log files.

(Solaris OS only) For each core file, provide the output of the following commands. See To Run the pkg_app Script
```
file corefile
pstack corefile
pmap corefile
pflags corefile
```

(Solaris OS only) Archive the result of the script pkg_app (one core file is sufficient).
```
./pkg_app.ksh proxy-pid corefile
```
Note –
The Sun Support Center must have the output from the pkg_app script to properly analyze the core file(s).

Provide network trace files between components, such as these:
- Browser and Proxy Server
- Proxy Server and Firewall
- Proxy Server and Directory Server
- Firewall and the Web
Here are examples of commands on the proxy server side:

Solaris OS

snoop -V -vvv -d <interface> -o /tmp/proxy-snoop-web <IP_WEB_SERVER>

HP-UX

tcpdump -i <interface> -w /tmp/proxy-snoop-web host <IP_WEB_SERVER>

Note –
tcpdump for HP-UX is available at: http://hpux.connect.org.uk. You can also use the native command nettl.

Linux

tethereal -V -F snoop -i <interface> -w /tmp/proxy-snoop-web host <IP_WEB_SERVER>

Note –
You can use the graphical user interface for tethereal or use the command tcpdump. tethereal is available at: http://www.ethereal.com.

Windows

tethereal -vvv -i <interface> -w /tmp/proxy-snoop-web host <IP_WEB_SERVER>

Note –
You can use either the graphical user interface or the command for tethereal. tethereal is available at: http://www.ethereal.com.

Note –
Clearly indicate IP and hostname for each component. This will help to read the network trace files correctly .

Note –
The commands listed in this step must be executed on the machine where the core files were generated.