| Previous Contents Index DocHome Next |
| iPlanet Web proxy Server 3.6 Administrator's Guide - NT Version |
Chapter 13 Monitoring the Server's Status
You can monitor your server's status in realtime by using the Simple Network Management Protocol (SNMP). SNMP is an Internet network management protocol used to monitor network devices. You can also monitor your server by recording and viewing log files.
Working with Log Files
Server log files record your server's activity. You can use these logs to monitor your server and help you when troubleshooting. The error log file, located in server root/proxy-id/logs, lists all the errors the server has encountered. The access log, also located in proxy-id/logs in the server root directory, records information about requests to the server and responses from the server. You can specify what is included in the access log file from the Server Manager. Use the log analyzer to generate server statistics. You can back up server error and access log files by archiving them.
Viewing the Error Log File
The error log file contains errors the server has encountered since the log file was created; it also contains informational messages about the server, such as when the server was started. Incorrect user authentication is also recorded in the error log.To view the error log file from the Server Manager,
In the Server Manager, choose Server Status|View Error Log.
This is an example of an error log:If you want to see more or less than 25 lines of the error log, use the Number of errors to view field to enter the number of lines you'd like to see.
If you'd like to filter the error messages for a particular word, type the word in the "Only show entries with" field.
- Make sure the case for your entry matches the case of the word for which you're searching. (For example, if you want to see only error messages that contain "warning," type warning.)
[13/Feb/1996:16:56:51] info: successful server startup
[20/Mar/1996 19:08:52] warning: for host wiley.a.com trying to GET /report.html, append-trailer reports: error opening /usr/ns-home/docs/report.html(No such file or directory)
[30/Mar/1996 15:05:43] security: for host arrow.a.com trying to GET /, basic-ncsa reports: user jane password did not match database /usr/ns-home/authdb/mktgdbIn this example, the first line is an informational message—the server started up successfully. The second log entry shows that the client wiley.a.com requested the file report.html, but the file wasn't in the primary document directory on the server. The third log entry shows that the password entered for the user jane was incorrect.
Viewing an Access Log File
You can view the server's active and archived access log files from the Server Manager.
In the Server Manager, choose Server Status|View Access Log.
This is a sample of an access log in the common logfile format:Choose the access log file you want to see.
To limit how much of the access log you'll see, type the number of lines you want to see in the Number of entries field.
- Active log files for resources and archived log files appear in the list.
If you'd like to filter the access log entries for a particular word, type the word in the Only show entries with field.
- Make sure the case for your entry matches the case of the word for which you're searching. (For example, if you want to see only access log entries that contain "POST," type POST.)
wiley.a.com - - [16/Feb/1996:21:18:26 -0800] "GET / HTTP/1.0" 200 751
wiley.a.com - - [17/Feb/1996:1:04:38 -0800] "GET /docs/grafx/icon.gif HTTP/1.0" 204 342
wiley.a.com - - [20/Feb/1996:4:36:53 -0800] "GET /help HTTP/1.0" 401 571
arrow.a.com - john [29/Mar/1996:4:36:53 -0800] "GET /help HTTP/1.0" 401 571Table 13-1 describes the last line of the sample access log.
Understanding Access Logfile Syntax
There are three predetermined logfile formats:
Common
Common format is the most basic of the log formats.
Syntax
host - usr [time] "req" s1 c1
Fields
host is the client's DNS host name. If reverse DNS lookup is not enabled on your proxy, host is the client's IP address.- is the RFC 931 style remote identity. (This parameter is not supported unless you are running your proxy as a SOCKS server.)
usr is the name of the user authenticated to the proxy.
time is the time and date of the request.
req is the first HTTP request line as it came into the proxy.
s1 is the proxy's HTTP response status code to the client.
c1 is the content-length sent to the client by the proxy.
Extended
Extended format is more detailed than common format because it includes all of the fields of the common format as well as some additional fields.
Syntax
host - usr [time] "req" s1 c1 s2 c2 b1 b2 h1 h2 h3 h4 xt
Fields
The following are the fields that the extended format includes that the common format does not include:s2 is the remote server's HTTP response status code to the proxy whenever the proxy makes a request in part of the client.
c2 is the content-length received from the remote server by the proxy.
b1 is the size of the client's HTTP request message body. (In other words, it is POST-data that will be forwarded to the remote server. This data will also be passed to the remote server if no error occurs.)
b2 is the size of the proxy's HTTP request message body. It is the amount of data in the body that was sent to the remote server. (This data is the same as b1 if no error occurs.)
h1 is the size of the client's HTTP request header to the proxy.
h2 is the size of the proxy server's response header to the client.
h3 is the size of the proxy server's request header to the remote server.
h4 is the size of the remote server's HTTP response header to the proxy.
xt is the total transfer time, in seconds.
Extended-2
Extended-2 format is the most detailed log format because it includes all of the fields of the extended format as well as some additional fields.
Syntax
host - usr [time] "req" s1 c1 s2 c2 b1 b2 h1 h2 h3 h4 xt route cs ss cs
Fields
The following are the fields that the extended-2 format includes that the other two formats do not include:route is the route used to retrieve the resource. The route field can hold one of the following:
DIRECT means that the resource was retrieved directly.
cs is the client finish status. This field specifies if the request to the client was successfully carried out to completion, interrupted by the client clicking the Stop button in Navigator, or aborted by an error condition. The cs field can hold one of the following:PROXY(host:port) means that the resource was retrieved through a proxy server at a specified host and port.
SOCKS(host:port) means that the resource was retrieved through a SOCKS server at a specified host and port.
- means that the request was never started.
ss is the remote server finish status. This field specifies if the request to the remote server was successfully carried out to completion, interrupted by the client clicking the Stop button in Navigator, or aborted by an error condition. The ss field can hold one of the following:FIN means that the request was completed successfully.
INTR means that the request was interrupted by the client or terminated by a proxy or server time out.
- means that the request was never started.
cs is the cache finish status. This field specifies whether the cache file was written, refreshed, or returned by an up-to-date check. The cs field can hold one of the following:FIN means that the request was completed successfully.
INTR means that the request was interrupted by the client or terminated by the proxy.
- means that the resource was not cacheable.
WRITTEN means that the cache file was created.
REFRESHED means that the cache file was updated or refreshed.
NO-CHECK means that the cache file was returned without an up-to-date check.
UP-TO-DATE means that the cache file was returned with an up-to-date check.
HOST-NOT-AVAILABLE means that the remote server was not available for an up-to date check, so the cache file was returned without a check.
CL-MISMATCH means that the cache file write was aborted due to a content-length mismatch.
ERROR means that the cache file write was aborted due to any error other than the above. These errors include a client interruption and a server timeout.
Understanding Status Codes
Table 13-2 lists and defines all of the status codes specified in the HTTP/1.1 RFC 2068. For more detailed descriptions of the codes, see the HTTP/1.1 specification, RFC 2068.
Note The 1xx status codes are not supported by HTTP/1.0.
Setting Access Log Preferences
During installation, an access log file named access was created for the server. You can customize access logging for your server by specifying whether or not to log accesses, whether or not the server should record domain names or IP addresses and what format the log file should be in.Server access logs can be in common logfile format, extended log format, extended-2 log format, a format that includes only specified information, or a custom format of your own design. For more information about these logfile formats, see Understanding Access Logfile Syntax.
To set access logging preferences,
From the Server Manager, choose Server Status|Log Preferences.
Use the template to which you'd like to apply custom logging.
Select whether or not to log client accesses.
- If you are configuring the logging preferences for your entire server, continue following the numbered steps below. If you are configuring the logging preferences for specific resources, continue with step number 3 and then skip to step number 8.
Type the full path for the log file.
Choose whether or not to record domain names or IP addresses in the log.
- By default, the log files are kept in the logs directory in the server root directory.
Choose which format the log file should be: common, extended, extended-2, only specified information (Only log radio button), or custom.
- If you click Only log, the following flexible log format items are available:
Client host name—The host name (or IP address if DNS is disabled) of the client requesting access.
If you choose a custom format, type it in the Custom format field.Authenticate user name—If authentication was necessary, you can have the authenticated user name listed in the access log.
System date—The date and time of the client request.
Full request—The exact request the client makes.
Status—The status code the server returned to the client.
Content length—The length, in bytes, of the document sent to the client.
HTTP header, "referer"—The referer tells you the page the client used previously to access the current page. For example, if a user is looking at the results from a text search query, the referer is the page from which the user accessed the text search engine. Referers allow the server to create a list of backtracked links.
HTTP header, "user-agent"—The user-agent information, which includes the type of browser the client is using, its version, and what operating system it's running on, comes from the user-agent field in the HTTP header information the client sends to the server.
Method—The request method used.
URI—Universal Resource Identifier is the path part of a URL. For example, for http://www.a.com:8080/special/docs, the URI is /special/docs.
Query string of the URI—Anything after the question mark in a URI. For example, for http://www.a.com:8080/special/docs?find_this, the query string of the URI is find_this.
Protocol—The transport protocol and version used.
Cache finish status—The method by which a document is placed in the cache. It can be written, refreshed, or returned by an up-to-date check.
Status code from server—The status code returned from the server.
Route to proxy—The route used to retrieve the resource. The document can be retrieved directly, through a proxy, or through a SOCKS server.
Transfer time—The length of time of the transfer, in seconds or milliseconds.
Header length from server response—The length of the header from the server response.
Request header size from proxy to server—The size of the request header from the proxy to the server.
Response header size sent to client—The size of the response header sent to the client.
Request header size received from client—The size of the request header received from the client.
Content-length from proxy to server request—The length, in bytes, of the document sent from the proxy to the server.
Content-length received from client—The length, in bytes, of the document received from the client.
Content-length from server response—The length, in bytes, of the document from the server.
Unverified user from client—The user name given to the remote server during authentication.
Note Using a custom format may significantly impact the performance of your proxy server.
Working with the Log Analyzer
Use the log analyzer to generate statistics about your server, such as a summary of activity, most commonly accessed URLs, times during the day when the server is accessed most frequently, and so on. You can run the log analyzer from the Server Manager, as described in Running the Log Analyzer from the Server Manager.
Note Before running the log analyzer, you should archive the server logs. For more information about archiving server logs, see Archiving Log Files.
If you use the extended or extended-2 logging format, the log analyzer generates several reports within the output file in addition to the information that you designate to be reported. The following sections describe these reports.
Transfer Time Distribution Report
The transfer time distribution report shows the time it takes your proxy server to transfer requests. This report displays the information categorized by service time and by percentage finished. The following is a sample transfer time distribution report.< 1 sec [64.4%] ........................................
< 2 sec [33.3%] ....................
< 3 sec [ 2.7%] .
< 4 sec [ 1.7%] .
< 5 sec [ 0.6%]
< 6 sec [ 0.4%]
< 7 sec [ 0.2%]
< 8 sec [ 0.0%]
< 9 sec [ 0.0%]< 1 sec [64.4%] ........................................
< 2 sec [97.7%] ....................................
< 3 sec [100.4%] ...............................................................
Status Code Report
The status code report shows which and how many status codes the proxy server received from the remote server and sent to the client. The status code report also provides explanations for all of these status codes. The following is a sample status code report.
Data Flow Report
The data flow report shows the data flow (the number of bytes transferred) from the client to the proxy, the proxy to the client, the proxy to the remote server, and the remote server to the proxy. For each of these scenarios, the report shows how much data was transferred in the form of headers and content. The data flow report also shows the data flow from the cache to the client. The following is a sample data flow report.
Requests and Connections Report
The requests and connections report shows the number of requests the proxy server receives from clients, the number of connections the proxy makes to a remote server (initial retrievals, up-to-date checks, and refreshes), and the number of remote connections the proxy server avoids by using cached documents. The following is a sample requests and connections report.- Total requests............. 478
- Remote connections......... 439
- Avoided remote connects.... 39 [ 8.2%]
Cache Performance Report
The cache performance report shows the performance of the clients' cache, the proxy server's cache, and the direct connections.
For the client's cache, the report shows:
client and proxy cache hits: a client cache hit in which the proxy server and the client both have a copy of the requested document and the remote server is queried for an up-to-date check with respect to the proxy's copy and the client's request is then evaluated with respect to the proxy's copy. The cache performance report shows the number of requests of this type that the proxy serviced and the average amount of time it took to service these requests.
proxy shortcut no-check: a client cache hit in which the proxy server and the client both have a copy of the requested document and the proxy server tells the client (without checking with the remote server) that the document in the client's cache is up-to-date. The cache performance report shows the number of requests of this type that the proxy serviced and the average time it took to service these requests.
client cache hits only: a client cache hit in which only the client has a cached copy of the requested document. In this type of request, the proxy server directly tunnels the client's If-modified-since GET header. The cache performance report shows the number of requests of this type that the proxy serviced and the average time it took to service these requests.
total client cache hits: the total number of client cache hits and the average amount of time it took to service these requests.
Proxy Cache
A proxy cache hit occurs when a client requests a document from a proxy server and the proxy server already has the document in its cache. For the proxy server's cache hits, the report shows:
proxy cache hits with check: a proxy cache hit in which the proxy server queries the remote server for an up-to-date check on the document. The cache performance report shows the number of requests of this type that the proxy serviced and the average time it took to service these requests.
proxy cache hits without check: a proxy cache hit in which the proxy server does not query the remote server for an up-to-date check on the document. The cache performance report shows the number of requests of this type that the proxy serviced and the average time it took to service these requests.
pure proxy cache hits: a proxy cache hit in which the client does not have a cached copy of the requested document. The cache performance report shows the number of requests of this type that the proxy serviced and the average time it took to service these requests.
Proxy Cache Hits Combined
For the proxy cache hits combined, the report shows:
total proxy cache hits: the total number of hits to the proxy server's cache and the average amount of time it took to service these requests.
Direct Transactions
Direct transactions are those that go directly from the remote server to the proxy server to the client without any cache hits. For the direct transactions, the report shows:
retrieved documents: documents retrieved directly from the remote server. The cache performance report shows the number of requests of this type that the proxy serviced, the average time it took to service these requests, and the percentage of total transactions.
The following is a sample cache performance report.other transactions: transactions that are returned with a status code other than 200 or 304. The cache performance report shows the number of requests of this type that the proxy serviced and the average time it took to service these requests.
total direct traffic: requests (both failed requests and successfully retrieved documents) that went directly from the client to the remote server. The cache performance report shows the number of requests of this type that the proxy serviced, the average time it took to service these requests, and the percentage of total transactions.
- Client & proxy cache hits... 86 reqs [18.0%] 0.21 sec/req
- Proxy shortcut no-check........ 13 reqs [ 2.7%] 0.00 sec/req
- Client cache hits only.....
- TOTAL client cache hits.......... 99 reqs [20.7%] 0.18 sec/req- Proxy cache hits w/check........ 4 reqs [ 0.8%] 0.50 sec/req
- Proxy cache hits w/o check.. 10 reqs [ 2.1%] 0.00 sec/req
- Pure proxy cache hits...... 14 reqs [ 2.9%] 0.14 sec/req- TOTAL proxy cache hits....... 113 reqs [23.6%] 0.18 sec/req
- Retrieved documents..313 reqs [65.5%] 0.90 sec/req 2 MB
- Other transactions.. 52 reqs [10.9%] 7.79 sec/req
- TOTAL direct traffic..365 reqs [76.4%] 1.88 sec/req 2 MB
Transfer Time Report
The transfer time report shows the information about the time it takes for the proxy server to process a transaction. This report shows values for the following categories:average transaction time: the average of all transfer times logged
average transfer time without caching: the average of transfer times for transactions which are not returned from the cache (200 response from remote server).
average with caching, without errors: the average of transfer times for all non-error transactions (2xx and 3xx status codes).
average transfer time improvement: the average transaction time minus the average transfer time with caching, without errors.
The following is a sample transfer time report.
- Average transaction time... 1.48 sec/req
- Ave xfer time w/o caching.. 0.90 sec/req
- Ave w/caching, w/o errors.. 0.71 sec/req
- Ave xfer time improvement.. 0.19 sec/req
Hourly Activity Report
For each analyzed hour, the hourly activity report shows:
the load average
the number of cache hits with no up-to-date check to the remote server
the number of hits to the proxy server's cache with an up-to-date check to the remote sever that proves that the document is up-to-date and the document is in the client cache
the number of hits to the proxy server's cache with an up-to-date check to the remote sever that proves that the document is up-to-date and the document is not in the client cache
the number of hits to the proxy server's cache with an up-to-date check to the remote server that caused part of the document to be updated.
the number of hits to the proxy server's cache with an up-to-date check to the remote server that returned a new copy of the requested document with a 200 status code.
the number of requests for which documents are directly retrieved from the remote server without any hits to the proxy server's cache
Running the Log Analyzer from the Server Manager
To run the log analyzer from the Server Manager,
In the Server Manager, choose Server Status|Generate Report.
Type the name of your server; this name appears in the generated report.
Choose whether or not the report will appear in HTML or plain text format.
Select the log file you want to analyze.
If you want to save the results in a file, type an output filename in the Output file field.
Select whether or not to generate totals for certain server statistics. The following totals can be generated:
- If you leave the field blank, the report results print to the screen. For large log files, you should save the results to a file because printing the output to the screen might take a long time.
Total hits—The total number of hits the server received since access logging was enabled.
Choose to generate general statistics.304 (Not Modified) status codes—The number of times a local copy of the requested document was used, rather than the server returning the page.
302 (Redirects) status codes—The number of times the server redirected to a new URL because the original URL moved.
404 (Not Found) status codes—The number of times the server couldn't find the requested document or the server didn't serve the document because the client was not an authorized user.
500 (Server Error) status codes—The number of times a server-related error occurred.
Total unique URLs—The number of unique URLs accessed since access logging was enabled.
Total unique hosts—The number of unique hosts who have accessed the server since access logging was enabled.
Total kilobytes transferred—The number of kilobytes the server transferred since access logging was enabled.
Top number of one-second periods—You can specify the number of one-second periods that had the highest number of requests.
Select whether or not to generate a list of server access statistics. You can generate a list of the following:Top number of one-minute periods—You can specify the number of one-minute periods that had the highest number of requests.
Top number of one-hour periods—You can specify the number of one-hour periods that had the highest number of requests.
Top number of users—You can specify the maximum number of users that accessed your server, provided that you included this as an item to log when you enabled access logging.
Top number of referers—You can specify the number of referers that appear in your log analysis, provided that you included this as an item to log when you enabled access logging.
Top number of user agents—You can specify the number of user agents that appear in your log analysis, provided that you included this as an item to log when you enabled access logging.
Top number of miscellaneous logged items—You can specify the number of items that appear in your log, provided you included this as an item to log when you enabled access logging. These miscellaneous items include the request method, the URI, and the URI query.
Most commonly accessed URLs—You can have the log analyzer show the most commonly accessed URLs or URLs that were accessed more than a specified number of times.
Specify the order in which you want to see the results.Hosts most often accessing your server—You can have the log analyzer show the hosts most often accessing your server or hosts that have accessed your server more than a specified number of times.
Archiving Log Files
You can archive the access and error log files and have the server create new ones.When you archive log files, the server renames the current log files and then creates new log files with the original names. You can back up or archive (or delete) the old log files, which are saved with the original filename appended with the date the file was archived. For example, access becomes access.24-Apr.
You can archive log files immediately or have the server archive them at a specific time on specific days. The information about when to archive log files is stored in the cron.conf file in the admserv directory in the server root directory; the server's cron configuration options are stored in ns-cron.conf in the admserv directory.
Note Before running the log analyzer, you should archive the server logs.
From the Server Manager, choose Server Status|Archive Log.
Click Archive if you want to archive the log files immediately. Or, if you want archiving to occur at a specific time on specific days, click the Rotate log at button, choose times from the list, and select the days for archiving to occur.
Shut down and restart the administration server.
Note If you chose to archive your server logs at specific times on specific days, step 4 is necessary in order for archiving to take place.
Monitoring the Server Using SNMP
You can monitor your server in real-time by using the Simple Network Management Protocol (SNMP). SNMP is a protocol used to exchange data about network activity. With SNMP, data travels between a managed device and a network management station (NMS) where users remotely manage the network.A managed device is anything that runs SNMP (for example, hosts, or routers). Your proxy server is a managed device. An NMS is usually a powerful workstation with one or more network management applications installed. A network management application graphically shows information about managed devices (which device is up or down, which and how many error messages were received, and so on).
Every managed device contains an SNMP agent that gathers information regarding the network activity of the device. This agent is known as the subagent. Each iPlanet server (except the administration server) has a subagent.
Another SNMP agent exchanges information between the subagent and NMS. This agent is called the master agent. A master agent runs on the same host machine as the subagents it talks to. You can have multiple subagents installed on a host machine. All of these subagents can communicate with the master agent.
Values for various variables that can be queried are kept on the managed device and reported to the NMS as necessary. Each variable is known as a managed object, which is anything the agent can access and send to the NMS. All managed objects are defined in a management information base (MIB), which is a database with a tree-like hierarchy. The top level of the hierarchy contains the most general information about the network. Each branch underneath is more specific and deals with separate network areas.
How Does SNMP Work?
SNMP exchanges network information in the form of protocol data units (PDUs). PDUs contain information about various variables stored on the managed device. These variables, also known as managed objects, have values and titles that are reported to the NMS as necessary. Communication between an NMS and managed device can take place in one of two forms:NMS-initiated communication: NMS-initiated communication is the most common type of communication between an NMS and a managed device. In this type of communication, the NMS either requests information from the managed device or changes the value of a variable stored on the managed device.
These are the steps that make up an NMS-initiated SNMP session:
The NMS searches the server's MIB to determine which managed devices and objects need to be monitored.
Managed device-initiated communication: This type of communication occurs when the managed device needs to inform the NMS of an event that has occurred. A managed device such as a terminal would initiate communication with an NMS to inform the NMS of a shut down or start up. Communication initiated by a managed device is also known as a "trap".The NMS sends a PDU to the managed device's subagent through the master agent. This PDU either requests information from the managed device or tells the subagent to change the values for variables stored on the managed device.
The subagent for the managed device receives the PDU from the master agent.
If the PDU from the NMS is a request for information about variables, the subagent gives information to the master agent and the master agent sends it back to the NMS in the form of another PDU. The NMS then displays the information textually or graphically.
- If the PDU from the NMS requests that the subagent set variable values, the subagent sets these values.
These are the steps that make up a managed device-initiated SNMP session:
An event occurs on the managed device.
For information on setting up and configuring your server to use SNMP, see Managing Netscape Servers.The subagent informs the master agent of the event.
The master agent sends a PDU to the NMS to inform the NMS of the event.
The Proxy Server MIB
Each iPlanet server has its own MIB (management information base). The proxy server's MIB is a file called ns-proxy.mib. This MIB contains the definitions for various variables pertaining to network management for the proxy server. These variables are known as managed objects. Using the proxy server MIB and network management software, such as HP OpenView, you can monitor your web server like all other devices on your network.The proxy server MIB has an object identifier of iplanet 1 (i.e. http OBJECT IDENTIFIER : := { iplanet 1 }) and is located in the server-root/plugins/snmp directory.
You can see administrative information about your web server and monitor the server in real-time using the proxy server MIB.
Enabling the Subagent
NT Only
Before you can monitor your server with SNMP, you need to enable the subagent that comes with your server. The subagent will then be able to communicate with the master agent built into the Windows NT operating system. You can enable the subagent via the Server Manager.
From the Server Manager, choose Server Status|SNMP Subagent Configuration.
Type the name of the system that has the master agent installed on it.
- The SNMP Configuration form appears.
Type the proxy server's location.
Using the Performance Monitor
iPlanet Web Proxy Server is integrated with the Performance Monitor tool of Windows NT. The Performance Monitor enables you to view the current activity of your proxy server in the form of a detailed chart.The Performance Monitor is located in the Administrative Tools program group and can be activated by selecting the item. To view the proxy server's current activity with the Performance Monitor, simply select iPlanet Web Proxy Server as the chart object, select the counters you wish to view, and add them to the chart.
Previous Contents Index DocHome Next
Copyright © 2001 Sun Microsystems, Inc. Some preexisting portions Copyright © 2001 Netscape Communications Corp. All rights reserved.
Last Updated March 28, 2001