Using Log Files

You can monitor your server’s activity using several different methods. This chapter discusses how to monitor your server by recording and viewing log files. For information on using the built-performance monitoring services, quality of service features, or SNMP, see Monitoring Servers.

About Log Files

Server log files record your server’s activity. You can use these logs to monitor your server and to help you when troubleshooting. The error log file, located in https-server_name/logs/errors in the server root directory, lists all the errors the server has encountered. The access log, located in https-server_name/logs/access in the server root directory, records information about requests to the server and the responses from the server. You can configure the information recorded in the Sun ONE Web Server access log file. You use the log analyzer to generate server statistics. You can back up server error and access log files by archiving them.


Note	Due to limitations in the operating system, Sun ONE Web Server cannot work with log files larger than 2GB on Linux. As soon as the maximum file size is reached, logging will cease.

Logging on the UNIX and Windows Platform

This section discusses how log files are created. In addition, this section includes the following topics:

Default Error Logging

On both the UNIX and Windows platforms, logs from the administration server are collected in the administration server https-admserve/logs/ directory. Logs from the server instances are collected in the https-server_name/logs/ directory.

The default log level for the entire server can be set.You can redirect stdout and stderr to the server’s event log and direct the log output to the operating system’s system log. Additionally, you can direct stdout and stderr content to the server’s event log. Log messages by default are sent to stderr in addition to the specified server log file.

Another feature available is to log the virtual server ID with the log message. This is a useful feature when multiple virtual servers are used to log messages to the same log file. You can choose to write the log messages to system log. When you do so, logging is not performed on the error log file. Instead the syslog logging service on UNIX, or the system logging service on Windows platform is used to produce and manage logs.

You can also use the server.xml attributes to control the contents of this file. For details about the server.xml file, see the Sun ONE Web Server 6.1 Administrator’s Configuration File Reference.

Logging Using syslog

For stable operational environments where centralized logging is required syslog is appropriate. For environments where log output is frequently required for diagnostics and debugging, individual server instance or virtual server logs may be more manageable.


Note	All logged data for the server instance and administration server in one file may prove difficult to read and debug. It is recommended that you use the syslog master log file only for deployed applications that are running smoothly. Logged message are intermixed with all other logs from the Solaris daemon applications.

By using the syslog log file, in conjunction with syslogd, and the system log daemon, you can configure the syslog.conf file to:

Since logging to syslog means, logs from Sun ONE Web Server, and other daemon applications are collected in the same file, logged messages are enhanced with the following information to identify Sun ONE Web Server-specific messages from the particular server or virtual server instance:

The LOG element can be configured for both the administration server and the server instance in the server.xml file.

For more information on the syslog logging mechanism used in the UNIX operating environment, use the following man commands at a terminal prompt:

Logging Using the Windows eventlog

For more information on the event log mechanism used in the Windows operating environment, refer to the Windows help system index for the keywords Event Logging.

Log Levels

The following table defines the log levels and messages in Sun ONE Web Server, in increasing order of severity.

Table 10-1 Log Levels
Log level	Description
finest finer fine	Messages indicate extent of verbosity of debug messages. finest gives the maximum verbosity.
info	Messages are informative in nature, usually related to server configuration or server status. These messages do not indicate errors that need immediate action.
warning	Messages indicate a warning. The message would probably be accompanied by an exception.
failure	Messages indicate a failure of considerable importance that can prevent normal application execution.
config	Messages relate to a variety of static configuration information, to assist in debugging problems that may be associated with particular configurations.
security	Messages indicate a security issue.
catastrophe	Messages indicate a fatal error.

About Virtual Servers and Logging

The Sun ONE Web Server can have virtual server instances. Each virtual server within a Sun ONE Web Server instance has its own identity and may have its own log file. The use of separate log files for each virtual server can help track server activity for particular transactions and resources.

You can also direct logged messages from multiple virtual servers to one server log file. When so doing, you may wish to enable the logvsid in the LOG element of the server.xml file. This helps users to distinguish log messages originating from different virtual servers.

In this example, <LOG logvsid="true"> is responsible for including the virtual server ID in every log message. This allows you to differentiate messages coming from different virtual servers. The absence of attribute errorlog in the VS element, causes all the virtual servers to log messages to a single file.

Redirecting Application and Server Log Output

For developers, it is important that the application logs and server logs be made readily available during unit testing for Web application components and J2EE applications. On the Windows platform, developers prefer to see log messages displayed in a command window on the desktop. On the UNIX platform, many developers are comfortable with simply having the log messages stream to stderr in the terminal window in which the server instance is started, or, use the command tail -f to see the log messages written in log files.

The server.xml file contains attributes that can be set for stdout and stderr to direct logged messages to a log file or to the terminal window, and so forth. See the Sun ONE Web Server 6.1 Administrator’s Configuration File Reference for more information on the use of stdout and stderr.

Archiving Log Files

You can set up your access and error log files to be automatically archived. At a certain time, or after a specified interval, your logs will be rotated. Sun ONE Web Server saves the old log files and stamps the saved file with a name that includes the date and time they were saved.

For example, you can set up your access log files to rotate every hour, and Sun ONE Web Server saves and names the file “access.200307152400,” where name of the log file, year, month, day, and 24-hour time is concatenated together into a single character string. The exact format of the log archive file varies depending upon which type of log rotation you set up.

Sun ONE Web Server offers the two types of log rotation for archiving files: Internal-daemon log rotation and Cron-based log rotation.

Internal-daemon Log Rotation

This type of log rotation happens within the HTTP daemon, and can only be configured at startup time. Internal daemon log rotation allows the server to rotate logs internally without requiring a server restart. Logs rotated using this method are saved in the following format:

You can specify the time used as a basis to rotate log files and start a new log file. For example, if the rotation start time is 12:00 a.m., and the rotation interval is 1440 minutes (one day), a new log file will be created immediately when you save and apply changes regardless of the present time. The log file will rotate every day at 12:00 a.m., and the access log will be stamped at 12:00 a.m. and saved as access.200307152400. Likewise, if you set the interval at 240 minutes (4 hours), the 4 hour intervals begin at 12:00 a.m. such that the access log files will contain information gathered from 12:00 a.m. to 4:00 a.m., from 4:00 a.m. to 8:00 a.m., and so forth.

If log rotation is enabled, log file rotation starts at server startup. The first log file to be rotated gathers information from the current time until the next rotation time. Using the previous example, if you set your start time at 12:00 a.m. and your rotation interval at 240 minutes, and the current time is 6:00 a.m., the first log file to be rotated will contain the information gathered from 6:00 a.m. to 8:00 a.m, and the next log file will contain information from 8:00 a.m. to 12:00 p.m. (noon), and so forth.

Scheduler-based Log Rotation

This type of log rotation is based on the time stored in the scheduler.conf file in the server_root/https-admserv/config/ directory. This method allows you to archive log files immediately or have the server archive log files at a specific time on specific days. The server’s scheduler configuration options are stored in schedulerd.conf in the server_root/https-admserv/config/ directory. Logs rotated using the scheduler-based method are saved in the following format:

For example, access might become access.200307151630 when it is rotated at 4:30 p.m.

Log rotation is initialized at server startup. If rotation is turned on, Sun ONE Web Server creates a time-stamped access log file and rotation starts at server startup.

Once the rotation starts, Sun ONE Web Server creates a new time stamped log file when there is a request or error that needs to be logged to the access or error log file and it occurs after the prior-scheduled “next rotate time”.


Note	You should archive the server logs before running the log analyzer.

To archive log files and to specify whether to use the Internal daemon method or the scheduler-based method, use The Archive Log Files Page in the Server Manager.

Setting Access Log Preferences

During installation, an access log file named access is created for the server. You can customize access logging for any resource by specifying whether to log accesses, what format to use for logging, and whether the server should spend time looking up the domain names of clients when they access a resource.

When changing the format of an existing log file, you should first delete/rename the existing log file OR use a different file name.


Note	When adding the custom format string ‘%vsid%, you must use a new access log file.

Server access logs can be in Common Logfile Format, flexible log format, or your own customizable format. The Common Logfile Format is a commonly supported format that provides a fixed amount of information about the server. The flexible log format allows you to choose (from Sun ONE Web Server) what to log. A customizable format uses parameter blocks that you specify to control what gets logged. For a list of customizable format parameters, see the NSAPI Programmer’s Guide.

Once an access log for a resource has been created, you cannot change its format unless you archive it or create a new access log file for the resource.

You can specify logging preferences using the Access Log Preferences page in the Server Manager, or you can manually configure the following directives in the obj.conf file. In magnus.conf, the server calls the function flex-init to initialize the flexible logging system and the function flex-log to record request-specific data in a flexible log format. To log requests using the common log file format, the server calls init-clf to initialize the Common Log subsystem which is used in obj.conf, and common-log to record request-specific data in the common log format (used by most HTTP servers).

For more information on the NSAPI logging functions, including valid directives and parameters, see the NSAPI Programmer’s Guide.

Easy Cookie Logging

Sun ONE Web Server has an easy way to log a specific cookie using the flexlog facility. Add “Req->headers.cookie.cookie_name” to the line that initializes the flex-log subsystem in the configuration file obj.conf. This logs the value of the cookie variable cookie_name if the cookie variable is present in the request’s headers, and logs “-” if it is not present.

Setting Error Logging Options

Sun ONE web Server 6.1 allows you to configure the information to be logged in the server’s errors logs.

For the Administration Server instance

For the Server Instance

Configuring the LOG Element

The following table describes the attributes for the LOG element you can configure in the server.xml file:

Table 2 LOG attributes
Attribute	Default	Description
file	errors	Specifies the file that stores messages from the default virtual server. Messages from other configured virtual servers also go here, unless the errorlog attribute is explicitly specified in the VS element.
loglevel	info	Controls the default type of messages logged by other elements to the error log. Allowed values are as follows, from highest to lowest: finest, fine, fine, info, warning, failure, config, security, and catastrophe.
logvsid	false	(optional) If true, virtual server IDs are displayed in the virtual server logs. These are useful if multiple VS elements share the same log file. Note that in Sun ONE Web Server 6.1 the logvsid element cannot be configured in the magnus.conf file.
logstdout	true	(optional) If true, redirects stdout output to the errors log. Legal values are on, off, yes, no, 1, 0, true, false.
logstderr	true	(optional) If true, redirects stderr output to the errors log. Legal values are on, off, yes, no, 1, 0, true, false.
logtoconsole	true	(optional, UNIX only) If true, redirects log messages to the console.
createconsole	false	(optional, Windows only) If true, creates a Windows console for stderr output. Legal values are on, off, yes, no, 1, 0, true, false.
usesyslog	false	(optional) If true, uses the UNIX syslog service or Windows Event Logging to produce and manage logs. Legal values are on, off, yes, no, 1, 0, true, false.

Viewing an Access Log File

To view the Administration Server’s access log from the Administration Server, choose the Preferences tab, and then choose The View Access Log Page.

To view an access log for the server instance from the Server Manager, choose the Logs tab, and then choose The View Access Log Page.

To view an access log for an individual virtual server from the Class Manager, select a virtual server to manage from the highlighted Manage Virtual Servers page, then click the link under the heading Access Log on the Virtual Server Manager page. You can specify the number of entries to view or entries with a conditional qualifier of your choice.

The following is an example of an access log in the Common Logfile Format (you specify the format in the Log Preferences window; see Setting Access Log Preferences for more information):

Table 10-3 The fields in the last line of the sample access log file
Access Log Field	Example
Hostname or IP address of client	arrow.a.com. (In this case, the hostname is shown because the web server’s setting for DNS lookups is enabled; if DNS lookups were disabled, the client’s IP address would appear.
RFC 931 information	- (RFC 931 identity not implemented)
Username	john (username entered by the client for authentication)
Date/time of request	29/Mar/1999:4:36:53 -0800
Request	GET /help
Protocol	HTTP/1.0
Status code	401
Bytes transferred	571

The following is an example of an access log using the flexible logging format (you specify the format in the Log Preferences page; see Setting Access Log Preferences for more information):

Viewing the Error Log File

The error log file contains errors the server has encountered since the log file was created; it also contains informational messages about the server, such as when the server was started. Unsuccessful user authentication is also recorded in the error log. Use the error log to find broken URL paths or missing files.

To view the Administration Server’s error log file, from the Administration Server, choose the Preferences tab, and choose The View Error Log Page.

To view a server instance’s error log file, from the Server Manager, choose the Logs tab, and choose The View Error Log Page.

To view an error log for an individual virtual server, from the Class Manager, select a virtual server to manage from the highlighted Manage Virtual Servers page, then click the link under the heading Error Log on the Virtual Server Manager page. You can specify the number of entries to view or entries with a conditional qualifier of your choice.

The following are two examples of entries in the error log; the first example shows an informational message indicating successful start up of the server, the second example indicates that the client wiley.a.com requested the file report.html, but the file wasn’t in the primary document directory on the server.

Running the Log Analyzer

The server-root/extras/log_anly directory contains the log analysis tool that runs through the Server Manager user interface. This log analyzer analyzes files in common log format only. The HTML document in the log_anly directory that explains the tool’s parameters. The server-root/extras/flex_anlg directory contains the command-line log analyzer for the flexible log file format. However, the Server Manager defaults to using the flexible log file reporting tool, regardless of whether you’ve selected common or flexible log file format.

Use the log analyzer to generate statistics about your default server, such as a summary of activity, most commonly accessed URLs, times during the day when the server is accessed most frequently, and so on. You can run the log analyzer from Sun ONE Web Server or the command line. The log analyzer cannot generate statistics for virtual servers other than the default server. However, statistics can be viewed for each virtual server as described in Viewing an Access Log File.

You must set the library path before attempting to run the flexanlg command line utility. The settings for various platforms are as follows:


Note	Before running the log analyzer, you should archive the server logs. For more information about archiving server logs, see Archiving Log Files.

To analyze access log files from the command line, run the tool, flexanlg, which is in the directory server-install/extras/flex_anlg.

flexanlg [ -P ] [-n name] [-x] [-r] [-p order] [-i file]* [ -m metafile ]* [ o file][ c opts] [-t opts] [-l opts]

flexanlg -h.):

-P: proxy log format Default: no

-n servername: The name of the server

-x : Output in HTML Default: no

-r : Resolve IP addresses to hostnames Default: no

-p [c,t,l]: Output order (counts, time stats, lists) Default: ctl

-i filename: Input log file(s) Default: none

-o filename: Output log file Default: stdout

-m filename: Meta file(s) Default: none

-c [h,n,r,f,e,u,o,k,c,z]: Count these item(s) - Default: hnreuokc

h: total hits

n: 304 Not Modified status codes (Use Local Copy)

r: 302 Found status codes (Redirects)

f: 404 Not Found status codes (Document Not Found)

e: 500 Server Error status codes (Misconfiguration)

u: total unique URL's

o: total unique hosts

k: total kilobytes transferred

c: total kilobytes saved by caches

z: Do not count any items.

-t [sx,mx,hx, xx,z]: Find general stats - Default:s5m5h24x10

s(number): Find top (number) seconds of log

m(number): Find top (number) minutes of log

h(number): Find top (number) hours of log

u(number): Find top (number) users of log

a(number): Find top (number) user agents of log

r(number): Find top (number) referers of log

x(number): Find top (number) for miscellaneous keywords

z: Do not find any general stats.

-l [cx,hx]: Make a list of - Default: c+3h5

c(x,+x): Most commonly accessed URLs

(x: Only list x entries)

(+x: Only list if accessed more than x times)

h(x,+x): Hosts (or IP addresses) most often accessing your server