Chapter 9   Using Log Files


You can monitor your server's activity using several different methods. This chapter discusses how to monitor your server by recording and viewing log files. For information on using the built-performance monitoring services, quality of service features, or SNMP, see Chapter 10 "Monitoring Servers."

This chapter contains the following sections:

• About Log Files

• Viewing an Access Log File

• Viewing the Error Log File

• Archiving Log Files

• Setting Log Preferences

• Running the Log Analyzer

• Viewing Events (Windows NT)

About Log Files

---

Server log files record your server's activity. You can use these logs to monitor your server and to help you when troubleshooting. The error log file, located in https-server_name/logs/errors in the server root directory, lists all the errors the server has encountered. The access log, located in https-server_name/logs/access in the server root directory, records information about requests to the server and the responses from the server. You can configure the information recorded in the iPlanet Web Server access log file. You use the log analyzer to generate server statistics. You can back up server error and access log files by archiving them.

---

Note	Due to limitations in the operating system, iPlanet Web Server cannot work with log files larger than 2GB on Linux. As soon as the maximum file size is reached, logging will cease.

---

Viewing an Access Log File

---

You can view the server's active and archived access log files.

To view the Administration Server's access log from the Administration Server, choose the Preferences tab, and then choose The View Access Log Page.

To view an access log for the server instance from the Server Manager, choose the Logs tab, and then choose The View Access Log Page.

To view an access log for an individual virtual server from the Class Manager, select a virtual server to manage from the highlighted Manage Virtual Servers page, then click the link under the heading Access Log on the Virtual Server Manager page. You can specify the number of entries to view or entries with a conditional qualifier of your choice.

The following is an example of an access log in the Common Logfile Format (you specify the format in the Log Preferences window; see Setting Log Preferences for more information):

wiley.a.com - - [16/Feb/2001:21:18:26 -0800] "GET / HTTP/1.0" 200 751 wiley.a.com - - [17/Feb/2001:1:04:38 -0800] "GET /docs/grafx/icon.gif HTTP/1.0" 204 342 wiley.a.com - - [20/Feb/2001:4:36:53 -0800] "GET /help HTTP/1.0" 401 571 arrow.a.com - john [29/Mar/2001:4:36:53 -0800] "GET /help HTTP/1.0" 401 571

Table 9-1 describes the last line of this sample access log.

Table 9-1    The fields in the last line of the sample access log file 

Access Log Field	Example
Hostname or IP address of client	arrow.a.com. (In this case, the hostname is shown because the web server's setting for DNS lookups is enabled; if DNS lookups were disabled, the client's IP address would appear.
RFC 931 information	- (RFC 931 identity not implemented)
Username	john (username entered by the client for authentication)
Date/time of request	29/Mar/1999:4:36:53 -0800
Request	GET /help
Protocol	HTTP/1.0
Status code	401
Bytes transferred	571

The following is an example of an access log using the flexible logging format (you specify the format in the Log Preferences page; see Setting Log Preferences for more information):

wiley.a.com - - [25/Mar/2001:12:55:26 -0800] "GET /index.htm HTTP/1.0" "GET" "/?-" "HTTP/   1.0" 304 0 - Mozilla/2.0 (WinNT; I) wiley.a.com - - [25/Mar/2001:12:55:26 -0800] "GET / HTTP/1.0" "GET" "/?-" "HTTP/1.0" 304 0    - Mozilla/2.0 (WinNT; I) wiley.a.com - - [25/Mar/2001:12:55:26 -0800] "GET / HTTP/1.0" "GET" "/?-" "HTTP/1.0" 304 0    - Mozilla/2.0 (X11; I; IRIX 5.3 IP22)

Viewing the Error Log File

---

The error log file contains errors the server has encountered since the log file was created; it also contains informational messages about the server, such as when the server was started. Unsuccessful user authentication is also recorded in the error log. Use the error log to find broken URL paths or missing files.

To view the Administration Server's error log file, from the Administration Server, choose the Preferences tab, and choose The View Error Log Page.

To view a server instance's error log file, from the Server Manager, choose the Logs tab, and choose The View Error Log Page.

To view an error log for an individual virtual server, from the Class Manager, select a virtual server to manage from the highlighted Manage Virtual Servers page, then click the link under the heading Error Log on the Virtual Server Manager page. You can specify the number of entries to view or entries with a conditional qualifier of your choice.

The following are two examples of entries in the error log; the first example shows an informational message indicating successful start up of the server, the second example indicates that the client wiley.a.com requested the file report.html, but the file wasn't in the primary document directory on the server.

[[22/Jan/2001:14:31:41] info (39700): successful server startup [22/Jan/2001:14:31:41] info (39700): iPlanet-WebServer-Enterprise/6.0 BB1-01/22/2001 01:45 [22/Jan/2001:14:31:42] warning (13751): for host wiley.a.com trying to GET /report.html, send-file reports: can't find /usr1/irenem/ES60-0424/docs/report.html (File not found)

Archiving Log Files

---

You can set up your access and error log files to be automatically archived. At a certain time, or after a specified interval, your logs will be rotated. iPlanet Web Server saves the old log files and stamps the saved file with a name that includes the date and time they were saved. For example, you can set up your access log files to rotate every hour, and iPlanet Web Server saves and names the file "access.199907152400," where name of the log file, year, month, day, and 24-hour time is concatenated together into a single character string. The exact format of the log archive file varies depending upon which type of log rotation you set up.

iPlanet Web Server offers the two types of log rotation for archiving files: Internal-daemon log rotation and Cron-based log rotation.

Internal-daemon Log Rotation

This type of log rotation happens within the HTTP daemon, and can only be configured at startup time. Internal daemon log rotation allows the server to rotate logs internally without requiring a server restart. Logs rotated using this method are saved in the following format:

access.<4 digit year><2 digit month><2 digit day><4 digit 24-hour time>
error.<4 digit year><2 digit month><2 digit day><4 digit 24-hour time>

You can specify the time used as a basis to rotate log files and start a new log file. For example, if the rotation start time is 12:00 a.m., and the rotation interval is 1440 minutes (one day), a new log file will be created immediately upon save regardless of the present time and collect information until the rotation start time. The log file will rotate every day at 12:00 a.m., and the access log will be stamped at 12:00 a.m. and saved as access.199907152400. Likewise, if you set the interval at 240 minutes (4 hours), the 4 hour intervals begin at 12:00 a.m. such that the access log files will contain information gathered from 12:00 a.m. to 4:00 a.m., from 4:00 a.m. to 8:00 a.m., and so forth.

If log rotation is enabled, log file rotation starts at server startup. The first log file to be rotated gathers information from the current time until the next rotation time. Using the previous example, if you set your start time at 12:00 a.m. and your rotation interval at 240 minutes, and the current time is 6:00 a.m., the first log file to be rotated will contain the information gathered from 6:00 a.m. to 8:00 a.m, and the next log file will contain information from 8:00 a.m. to 12:00 p.m. (noon), and so forth.

Cron-based Log Rotation

This type of log rotation is based on the time stored in the cron.conf file in the server_root/https-admserv/config/ directory. This method allows you to archive log files immediately or have the server archive log files at a specific time on specific days. The server's cron configuration options are stored in ns-cron.conf in the server_root/https-admserv/config/ directory. Logs rotated using the cron based method are saved as the original filename followed by the date and time the file was rotated. For example, access might become access.24Apr-0430PM when it is rotates at 4:30 p.m.

Log rotation is initialized at server startup. If rotation is turned on, iPlanet Web Server creates a time-stamped access log file and rotation starts at server startup.

Once the rotation starts, iPlanet Web Server creates a new time stamped log file when there is a request or error that needs to be logged to the access or error log file and it occurs after the prior-scheduled "next rotate time".

---

Note	You should archive the server logs before running the log analyzer.

---

To archive log files and to specify whether to use the Internal daemon method or the cron based method, use The Archive Log Files Page in the Server Manager.

Setting Log Preferences

---

During installation, an access log file named access was created for the server. You can customize access logging for any resource by specifying whether to log accesses, what format to use for logging, and whether the server should spend time looking up the domain names of clients when they access a resource.

To use one log file for multiple virtual servers, that log file should have %vsid% in its format string for the access log and LogVsid should be turned on in the magnus.conf file for the error log. Follow the steps below to access the magnus.conf file and the logging preferences in the Administration Server UI to make the needed changes. The changes are effected following restart of the Administration server.

To turn on LogVsId:

1. Access the Server Manager and choose the Preferences tab.

2. Click the Magnus Editor link.

3. Select the Logging Settings from the Select a Setting drop-down list and click Manage.

4. Select On for the LogVsId value

5. Click the OK button.

To add %vsid% to the log file format string:

1. Access the Server Manager and choose the Logs tab.

2. Click the Log Preferences link.

3. Enter a new log file location and filename in the Log File: text box.

4. Click the Only Log: radio button.

5. Click the Vsid check box. Alternatively to this, you can click the Custom Format: radio button and add the string `%vsid%.

   
   

   ---

   Note	When adding the custom format string `%vsid%, you must use a new access log file.

   ---

   
   

For information on the LogVsId directive in magnus.conf, see the section "Error Logging and Statistic Collection" in the NSAPI Programmer's Guide.

When changing the format of an existing log file, you should first delete/rename the existing log file OR use a different file name.

Server access logs can be in Common Logfile Format, flexible log format, or your own customizable format. The Common Logfile Format is a commonly supported format that provides a fixed amount of information about the server. The flexible log format allows you to choose (from iPlanet Web Server) what to log. A customizable format uses parameter blocks that you specify to control what gets logged. For a list of customizable format parameters, see the NSAPI Programmer's Guide.

Once an access log for a resource has been created, you cannot change its format unless you archive it or create a new access log file for the resource.

You can specify logging preferences using the Log Preferences page in the Server Manager, or you can manually configure the following directives in the obj.conf (or magnus.conf) file. In magnus.conf, the server calls the function flex-init to initialize the flexible logging system and the function flex-log to record request-specific data in a flexible log format. To log requests using the common log file format, the server calls init-clf to initialize the Common Log subsystem which is used in obj.conf, and common-log to record request-specific data in the common log format (used by most HTTP servers).

For more information on the NSAPI logging functions, including valid directives and parameters, see the NSAPI Programmer's Guide.

Easy Cookie Logging

iPlanet Web Server has an easy way to log a specific cookie using the flexlog facility. Add "Req->headers.cookie.cookie_name" to the line that initializes the flex-log subsystem in the configuration file obj.conf. This logs the value of the cookie variable cookie_name if the cookie variable is present in the request's headers, and logs "-" if it is not present.

Running the Log Analyzer

---

The server-root/extras/log_anly directory contains the log analysis tool that runs through the Server Manager user interface. This log analyzer analyzes files in common log format only. The HTML document in the log_anly directory that explains the tool's parameters. The server-root/extras/flex_anlg directory contains the command-line log analyzer for the flexible log file format. However, the Server Manager defaults to using the flexible log file reporting tool, regardless of whether you've selected common or flexible log file format.

Use the log analyzer to generate statistics about your default server, such as a summary of activity, most commonly accessed URLs, times during the day when the server is accessed most frequently, and so on. You can run the log analyzer from iPlanet Web Server or the command line. The log analyzer cannot generate statistics for virtual servers other than the default server. However, statistics can be viewed for each virtual server as described in Viewing an Access Log File

---

Note	Before running the log analyzer, you should archive the server logs. For more information about archiving server logs, see Archiving Log Files.

---

To run the log analyzer from the Server Manager, follow these steps:

1. From the Server Manager, click the Logs tab.

2. Click Generate Report.

3. Fill in the fields.

4. Click OK.

   The report appears in a new window.

For more information, see The Generate Report Page in the online help.

To analyze access log files from the command line, run the tool, flexanlg, which is in the directory server-install/extras/flex_anlg.

To run flexanlg, type the following command and options at the command prompt:

flexanlg [ -P ] [-n name] [-x] [-r] [-p order] [-i file]* [ -m metafile ]* [ o file][ c opts] [-t opts] [-l opts]

The following describes the syntax.

flexanlg -h.): -P: proxy log format Default: no -n servername: The name of the server -x : Output in HTML Default: no -r : Resolve IP addresses to hostnames Default: no -p [c,t,l]: Output order (counts, time stats, lists) Default: ctl -i filename: Input log file(s) Default: none -o filename: Output log file Default: stdout -m filename: Meta file(s) Default: none -c [h,n,r,f,e,u,o,k,c,z]: Count these item(s) - Default: hnreuokc     h: total hits     n: 304 Not Modified status codes (Use Local Copy)     r: 302 Found status codes (Redirects)     f: 404 Not Found status codes (Document Not Found)     e: 500 Server Error status codes (Misconfiguration)     u: total unique URL's     o: total unique hosts     k: total kilobytes transferred     c: total kilobytes saved by caches     z: Do not count any items. -t [sx,mx,hx, xx,z]: Find general stats - Default:s5m5h24x10     s(number): Find top (number) seconds of log     m(number): Find top (number) minutes of log     h(number): Find top (number) hours of log     u(number): Find top (number) users of log     a(number): Find top (number) user agents of log     r(number): Find top (number) referers of log     x(number): Find top (number) for miscellaneous keywords     z: Do not find any general stats. -l [cx,hx]: Make a list of - Default: c+3h5     c(x,+x): Most commonly accessed URLs              (x: Only list x entries)              (+x: Only list if accessed more than x times)     h(x,+x): Hosts (or IP addresses) most often accessing your server              (x: Only list x entries)              (+x: Only list if accessed more than x times)     z: Do not make any lists

Viewing Events (Windows NT)

---

In addition to logging errors to the server error log (see Viewing the Error Log File), iPlanet Web Server logs severe system errors to the Event Viewer. The Event Viewer lets you monitor events on your system. Use the Event Viewer to see errors resulting from fundamental configuration problems, which can occur before the error log can be opened.

To use the Event Viewer, perform the following steps:

1. From the Start menu, select Programs and then Administrative Tools. Choose Event Viewer in the Administrative Tools program group.

2. Choose Application from the Log menu.

The Application log appears in the Event Viewer. Errors from iPlanet Web Server has a source label of https-serverid or WebServer6.0.

3. Choose Find from the View menu to search for one of these labels in the log. Choose Refresh from the View menu to see updated log entries.

   For more information about the Event Viewer, consult your system documentation.