Sun Java System Web Server 7.0 Update 4 Performance Tuning, Sizing, and Scaling Guide

Chapter 1 Performance and Monitoring Overview

Sun Java System Web Server (hereafter Web Server) is designed to meet the needs of the most demanding, high-traffic sites in the world. It can serve both static and dynamically generated content. Web Server can also run in SSL mode, enabling the secure transfer of information.

This guide helps you to define your server workload and size a system to meet your performance needs. Because your environment is unique, the impacts of the suggestions provided in this guide will depend on your specific environment. Ultimately you must rely on your own judgement and observations to select the adjustments that are best for you.

This chapter provides a general discussion of server performance considerations, and more specific information about monitoring server performance.

This chapter includes the following topics:

Performance Issues

The first step toward sizing your server is to determine your requirements. Performance means different things to users than it means to webmasters. Users want fast response times (typically less than 100 milliseconds), high availability (no “connection refused” messages), and as much interface control as possible. Webmasters and system administrators, on the other hand, want to see high connection rates, high data throughput, and uptime approaching 100%. In addition, for virtual servers the goal might be to provide a targeted level of performance at different price points. You need to define what performance means for your particular situation.

Here are some areas to consider:

The number of peak concurrent users
Security requirements

Encrypting your Web Server’s data streams with SSL makes an enormous difference to your site’s credibility for electronic commerce and other security conscious applications, but it can also seriously impact your CPU load. For more information, see SSL Performance.
The size of the document tree
Dynamic or static content

The content you serve affects your server’s performance. A Web Server delivering mostly static HTML can run much faster than a server that must execute CGIs for every query.

Configuration

Certain tuning parameters are set at the configuration level, so that every server instance that is based on the configuration has the same tuning information. In addition, some monitoring information is available at the configuration level, so you can monitor the performance of all instances based on the configuration. However, the bulk of the monitoring information is available at the individual server instance, or virtual server level. If you are using a single Web Server instance per configuration, meaning your server is not part of a server farm, the configuration-level statistics show the information for the single server instance based on that configuration.

Virtual Servers

Virtual servers add another layer to the performance improvement process. Certain settings are tunable for the configuration, while others are based on an individual virtual server.

You can also use the quality of service (QoS) features to set resource utilization constraints for an individual virtual server. For example, you can use QoS features to limit the amount of bandwidth and the number of connections allowed for a virtual server. You can set these performance limits, track them, and optionally enforce them.

For more information about using the quality of service features, see Sun Java System Web Server 7.0 Update 4 Administrator’s Guide.

Server Farms

The clustering features of Web Server allow you to easily deploy to a server farm. Because all servers in a server farm share identical configurations, tuning is not done on a server-by-server basis.

64–Bit Servers

The performance for the 64–bit Web Server is not necessarily better than the performance for the 32–bit Web Server, but the 64–bit server scales better. Because the 32–bit Web Server process is confined to 4 GB of address space, it can run out of address space when attempting to support simultaneous sessions beyond a certain limit. Even if the host machine has available memory and CPU resources, the 32–bit Web Server might not be able to take advantage of it because of the address space limit. The 64–bit Web Server can run more applications and servlets than the 32-bit server. Also, the 64–bit Web Server can cache several GBs of static content, while the 32-bit Web Server is confined to 4 GB of address space.

In general, the tuning for the 64–bit Web Server is similar to the tuning for the 32–bit Web Server. The differences are mostly tuned at the operating system level. Tuning specifics are discussed in Tuning UltraSPARC® T1–Based Systems for Performance Benchmarking.

SSL Performance

SSL always has a significant impact on throughput, so for best performance minimize your use of SSL, or consider using a multi-CPU server to handle it.

For SSL, the Web Server uses the NSS library. However, there are other options available for SSL:

If you are using the Solaris 10 operating system, kernel SSL (KSSL) is available. It does not contain all the algorithms available, as does NSS, but it often provides better performance.
A cryptographic card hardware accelerator for SSL can also improve performance.
If you are using the 64–bit Web Server on Solaris, you can use the cryptographic accelerator of the UltraSPARC T1 processor.

Monitoring Server Performance

Making the adjustments described in this guide without measuring their effects doesn’t make sense. If you don’t measure the system’s behavior before and after making a change, you won’t know whether the change was a good idea, a bad idea, or merely irrelevant. You can monitor the performance of Web Server in several different ways.

Table 1–1 Methods of Monitoring Performance


Monitoring Method	How to Enable	How to Access	Advantages and Requirements
Statistics through the Admin Console	Enabled by default	In the Admin Console, for a configuration, click the Monitor tab	Accessible when session threads are hanging. Administration Server must be running.
Statistics through individual `wadm` commands	Enabled by default	Through `wadm` commands: `get-config-statsget-virtual-server-statsget-webapp-statsget-servlet-stats`	Accessible when session threads are hanging. Administration Server must be running.
XML-formatted statistics (`stats-xml`) through a browser	Enable through Admin Console or through editing a configuration file	Through a URI	Administration Server need not be running.
XML-formatted statistics (`stats-xml`) through the command-line interface	Enabled by default	Through the `wadm` command `get-stats-xml`	Accessible when session threads are hanging. Administration Server must be running.
`perfdump` through a browser	Enable through Admin Console or through editing a configuration file	Through a URI	Administration Server need not be running.
`perfdump` through the command-line interface	Enabled by default	Through `wadm` command `get-perfdump`	Accessible when session threads are hanging. Administration Server must be running.
Java ES Monitoring	Enabled by default	Through the Java ES Monitoring Console	Only for Java ES installations. Administration Server must be running.

Monitoring the server does have some impact on computing resources. In general, using perfdump through the URI is the least costly, followed by using stats-xml through a URI. Because using the Administration Server requires computing resources, the command-line interface and the Admin Console are the most costly monitoring methods.

For more information on these monitoring methods, see the following sections:

About Statistics

You can monitor many performance statistics through the Admin Console user interface, through the command-line interface, through the stats-xml URI, and through perfdump. For all these monitoring methods, the server uses the statistics it collects. None of these monitoring methods will work if statistics are not collected.

The statistics give you information at the configuration level, the server instance level, or the virtual server level. The statistics are broken up into functional areas.

For the configuration, statistics are available in the following areas:

Requests
Errors
Response Time

For the server instance, statistics are available in the following areas:

Requests
Errors
Response Time
General
Java Virtual Machine (JVM^TM)
Connection Queue
Keep Alive
DNS
File Cache
Thread Pools
Session Replication
Session Threads, including Profiling data (available if profiling is enabled)
Java DataBase Connectivity (JDBC^TM) (available if a JDBC resource is created and the connection pool is accessed)

For the virtual server, statistics are available in the following areas:

General
Response
Web Applications
Profiling Data (available if profiling is enabled)
Servlet and Servlet Response Cache (available if the Servlet cache is enabled in sun.web.xml)

Some statistics default to zero if Quality of Service (QoS) is not enabled, for example, the count of open connections, the maximum open connections, the rate of bytes transmitted, and the maximum byte transmission rate.

Enabling Statistics

Statistics are activated by default on Web Server. However, if you have disabled them, you need to enable them again to monitor your server for performance. To enable statistics, use Admin Console or the wadm command-line utility (CLI).

Note –

Collecting statistics causes a slight hit to performance.

To Enable Statistics from the Admin Console

From the Admin Console Common Tasks page, select the configuration.

Click Edit Configuration.

Click the General tab.

Click the Monitoring Settings sub tab.

On the Monitoring Settings page, under General Settings, select the Statistics Enabled checkbox.

Configure the interval and profiling.
- The Interval is the period in seconds between statistics updates. A higher setting with less frequent updates improves performance. The minimum value is .001 seconds; the default value is 5 seconds.
- Profiling is activated by default. Deactivating it results in slightly less monitoring overhead.

Restart the server.

To Enable Statistics from the CLI

Enter the following CLI command to enable statistics collection:

./wadm set-stats-prop --user=admin_user –password-file=password-file --config=myconfig enabled=true

To disable statistics, set enabled to false.

To set the interval and enable profiling, use the set-stats-prop interval and profiling properties. For more information, see the help for set-stats-prop.

Restart the server.

Monitoring Current Activity Using the Admin Console

Frequently-used statistics are available through the Admin Console, viewed as general statistics, instance statistics, and virtual server statistics.

To Monitor Statistics from the Admin Console

In the Admin Console, from the Common Tasks page, select the Monitoring tab.

Select the configuration.

The configuration statistics are displayed.

From the drop-down list, select a View interval.

The statistics displayed in your browser are automatically updated at this interval.

Select the type of statistics to display.

The initial list of statistics types includes General Statistics, Instance Statistics, and Virtual Server Statistics.

If you choose Instance Statistics, click the name of the instance to monitor. Detailed statistics are then displayed, including information on processes and session replications.

If you choose Virtual Server Statistics, click the name of the virtual server to monitor. Statistics for the virtual server are displayed, including response statistics and web application statistics. This information is not provided through perfdump.

Monitoring Current Activity Using the CLI

You can also view statistics information using the wadm commands get-config-stats, get-virtual-server-stats, get-webapp-stats and get-servlet-stats. Note that the examples below do not contain all possible command options. For the complete syntax, see the help for the command.

To Monitor Statistics from the CLI

To deploy statistics for configuration on a single node, enter:

./wadm get-config-stats --user=admin-user --password-file=admin-password-file --config=config-name --node=node-name

Using the node option in this syntax restricts the output to a single node. To view the statistics at the configuration level, use the command without the node option.

The following shows an example of the output for a single node:

timeStarted=1168035653
secondsRunning=1404
countRequests=690546
rpsLast1MinAvg=4491.7666
rpsLast5MinAvg=1844.6061
rpsLast15MinAvg=637.37305
countErrors=0
epsLast1MinAvg=0.0
epsLast5MinAvg=0.0
epsLast15MinAvg=0.0
maxResponseTime=0.30789953
rtLast1MinAvg=5.3970284
rtLast5MinAvg=5.208407
rtLast15MinAvg=35.56042
countBytesReceived=96800935
countBytesTransmitted=689929574
countChildDied=0
countVirtualServers=2
instanceName=https-test
process.1.countThreadPools=2
process.1.jdbcPoolCount=1
process.1.countThreads=64
process.1.fractionSystemMemoryUsage=2887.0
process.1.countConnectionQueues=1
process.1.sizeResident=0
process.1.countIdleThreads=32
process.1.mode=1
process.1.sizeVirtual=0
process.1.countConfigurations=1
process.1.pid=15874
process.1.timeStarted=Jan 5, 2007 2:20:53 PM
process.1.DNSCache.countCacheHits=687804
process.1.DNSCache.countAsyncNameLookup=0
process.1.DNSCache.countAsyncLookupsInProgress=0
process.1.DNSCache.flagAsyncEnabled=false
process.1.DNSCache.countAsyncAddrLookups=0
process.1.DNSCache.flagCacheEnabled=true
process.1.DNSCache.countCacheMisses=75
process.1.JDBCPool.1.countQueued=32
process.1.JDBCPool.1.countFreeConnections=0
process.1.JDBCPool.1.peakConnections=32
process.1.JDBCPool.1.millisecondsPeakWait=72
process.1.JDBCPool.1.countWaitQueueTimeouts=288
process.1.JDBCPool.1.peakQueued=64
process.1.JDBCPool.1.maxConnections=32
process.1.JDBCPool.1.currentConnections=32
process.1.JDBCPool.1.millisecondsAverageQueued=1.0
process.1.JDBCPool.1.countTotalFailedValidationConnections=0
process.1.JDBCPool.1.countLeasedConnections=32
process.1.JDBCPool.1.countTotalLeasedConnections=414
process.1.JDBCPool.1.countConnectionIdleTimeouts=1
process.1.JDBCPool.1.name=jdbc/jdbc-simple_1
process.1.connectionQueue.1.countQueued15MinuteAverage=4.3203125
process.1.connectionQueue.1.countQueued=0
process.1.connectionQueue.1.countQueued1MinuteAverage=0.046875
process.1.connectionQueue.1.countTotalQueued=79171
process.1.connectionQueue.1.countQueued5MinuteAverage=4.03125
process.1.connectionQueue.1.countOverflows=0
process.1.connectionQueue.1.maxQueued=1288
process.1.connectionQueue.1.ticksTotalQueued=724956383
process.1.connectionQueue.1.countTotalConnections=863
process.1.connectionQueue.1.peakQueued=64
process.1.connectionQueue.1.name=cq1
process.1.fileCache.countContentMisses=7
process.1.fileCache.maxMmapCacheSize=0
process.1.fileCache.sizeHeapCache=27520
process.1.fileCache.countMisses=22
process.1.fileCache.countContentHits=620662
process.1.fileCache.maxEntries=1024
process.1.fileCache.flagEnabled=true
process.1.fileCache.secondsMaxAge=30
process.1.fileCache.sizeMmapCache=0
process.1.fileCache.countInfoHits=1862013
process.1.fileCache.maxHeapCacheSize=10747924
process.1.fileCache.countOpenEntries=0
process.1.fileCache.countHits=2482682
process.1.fileCache.maxOpenEntries=1024
process.1.fileCache.countEntries=12
process.1.fileCache.countInfoMisses=19
process.1.jvm.countGarbageCollections=96
process.1.jvm.sizeHeap=67762048
process.1.jvm.countThreads=79
process.1.jvm.countClassesUnloaded=0
process.1.jvm.vMVendor=Sun Microsystems Inc.
process.1.jvm.countTotalClassesLoaded=3170
process.1.jvm.vMName=Java HotSpot(TM) Server VM
process.1.jvm.countTotalThreadsStarted=81
process.1.jvm.countClassesLoaded=3170
process.1.jvm.peakThreads=79
process.1.jvm.millisecondsGarbageCollection=1981
process.1.jvm.vMVersion=1.5.0_09-b03
process.1.keepalive.countConnections=32
process.1.keepalive.maxConnections=200
process.1.keepalive.countFlushes=0
process.1.keepalive.countRefusals=0
process.1.keepalive.countTimeouts=6
process.1.keepalive.countHits=686943
process.1.keepalive.secondsTimeout=30
process.1.threadPool.1.countQueued=0
process.1.threadPool.1.countThreadsIdle=1
process.1.threadPool.1.threadPoolId=NativePool
process.1.threadPool.1.maxThreads=128
process.1.threadPool.1.countThreads=1
process.1.threadPool.1.maxQueued=0
process.1.threadPool.1.peakQueued=0
process.1.threadPool.1.name=NativePool
process.1.threadPool.2.countQueued=0
process.1.threadPool.2.countThreadsIdle=1
process.1.threadPool.2.threadPoolId=my-custom-pool
process.1.threadPool.2.maxThreads=128
process.1.threadPool.2.countThreads=1
process.1.threadPool.2.maxQueued=0
process.1.threadPool.2.peakQueued=0
process.1.threadPool.2.name=my-custom-pool

To obtain statistics for a virtual server, enter:

./wadm get-virtual-server-stats --user=admin-user --password-file=admin-password-file --config=config-name --vs=virtual-server-name

Because the node option is not used, this syntax provides aggregate statistics for the virtual server across all the nodes where the configuration has been deployed. Using the node option restricts the output to a single node.

To obtain statistics for a deployed web application, enter:

./wadm get-webapp-stats --user=admin-user --password-file=admin-password-file --config=config-name --node=node-name --vs=virtual-server-name --uri=URI

The syntax obtains statistics for a given web application deployed on the given virtual server of the given instance. To obtain the aggregated web application statistics for a given configuration across all the nodes where the configuration has been deployed, use the command without the node option.

The following example shows the output for the URI hello:
```
countActiveSessions=1
countExpiredSessions=0
countJsps=1
countRejectedSessions=0
countReloadedJsps=1
countSessions=1
peakActiveSessions=1
secondsSessionAliveAverage=0
secondsSessionAliveMax=0
uri=/hello
vsName=myvs.sun.com
```

Monitoring Current Activity Using stats.xml

You can also display statistics using stats-xml, which displays statistics in XML format. The output of stats-xml is in XML so that various tools can easily parse the statistics. You can view the stats-xml output through a URI, which you have to enable, or you can view the stats-xml output through the CLI, which is enabled by default.

To Enable the stats-xml URI from the Admin Console

If you enable the stats-xml URI, you can access statistics for your server in XML format through a browser. Note that when you use the stats-xml URI, you can access statistics even when the Administration Server is not running. Also, with the stats-xml URI activated, users can see the statistics information for your server, unless you take precautions to deny access.

On the Common Tasks page, select the configuration from the pull-down menu on the left.

Select the virtual server from the pull-down menu on the right, then click Edit Virtual Server.

On the Server Settings tab, click the Monitoring Settings sub tab.

Select the XML Report enabled checkbox.

Provide a URI, for example, /stats-xml.

Click Save.

Deploy the configuration.

Access the stats-xml URI, for example:

http://yourhost:port/stats-xml

The statistics are displayed in XML format.

To Enable the stats-xml URI from the CLI

Use the following command to enable stats-xml:

./wadm enable-stats-xml --user=admin-user --password-file=admin-password-file [--uri-prefix=prefix]--config=config-name --vs=virtual-server-name

Use the uri-prefix option to set the stats-xml URI.

Deploy the configuration using the wadm deploy-config command.

Access the stats-xml URI, for example:

http://yourhost:port/stats-xml

The statistics are displayed in XML format.

To Limit the stats-xml Statistics Displayed in the URI

You can modify the stats-xml URI to limit the data it provides.

Modify the stats-xml URI to limit the information by setting elements to 0 or 1. An element set to 0 is not displayed on the stats-xml output. For example:

http://yourhost:port/stats-xml?thread=0&process=0

This syntax limits the stats-xml output so that thread and process statistics are not included. By default all statistics are enabled (set to 1).

Most of the statistics are available at the server level, but some are available at the process level.

Use the following syntax elements to limit stats-xml:
- cache-bucket
- connection-queue
- connection-queue-bucket (process-level)
- cpu-info
- dns-bucket
- jdbc-resource-bucket
- keepalive-bucket
- process
- profile
- profile-bucket (process-level)
- request-bucket
- servlet-bucket
- session-replication
- thread
- thread-pool
- thread-pool-bucket (process-level)
- virtual-server
- web-app-bucket

To View stats-xml Output from the CLI

In addition to a URI, you can also access stats-xml output through the command-line interface which is enabled by default. Unlike viewing stats-xml output through the URI, the Administration Server must be running to view stats-xml output at the command-line. However, if request processing threads are hanging in your server, and you cannot use the URI, you can still access stats-xml output through the CLI.

To view the stats-xml output through the command-line interface, enter:

./wadm get-stats-xml --user=admin-user --password-file=admin-password-file --config=config-name --node=node-name

Monitoring Current Activity Using perfdump

perfdump is a Server Application Function (SAF) built into Web Server that collects various pieces of performance data from the Web Server internal statistics and displays them in ASCII text. The perfdump output does not display all the statistics available through the command-line statistics or the Admin Console, but it can still be a useful tool. For example, you can still use perfdump even if the Administration Server is not running. You can view the perfdump output through the CLI, which is enabled by default, or you can view the perfdump output through a URI, which you have to enable. If you enable the URI, you must control access to the perfdump URI, otherwise it can be visible to users.

With perfdump, the statistics are unified. Rather than monitoring a single process, statistics are multiplied by the number of processes, which gives you an accurate view of the server as a whole.

For information on tuning the information displayed by perfdump, see Using Monitoring Data to Tune Your Server.

To Enable the perfdump URI from the Admin Console

You can enable perfdump URI for a virtual server through the Admin Console.

Note –

The statistics displayed by perfdump are for the server as a whole. If you enable perfdump on one virtual server, it displays statistics for the whole server, not an individual virtual server.

From Common Tasks, select a configuration.

Select the virtual server and click Edit Virtual Server.

Click the Monitoring Settings tab.

Select the Plain Text Report Enabled checkbox.

Provide a URI for accessing the report, for example /.perf.

Click Save.

Deploy the configuration.

To access perfdump, access the URI on the virtual server.

For example: http://localhost:80/.perf

You can request the perfdump statistics and specify how frequently the browser should automatically refresh as measured in seconds. The following example sets the refresh to every 5 seconds:

http://yourhost/.perf?refresh=5

To Enable the perfdump URI from the CLI

Use the following command to enable perfdump:

./wadm enable-perfdump --user=admin-user --password-file=admin-password-file [--uri=uri]--config=config-name--vs=virtual-server-name

Use the uri option to set the pefdump URI.

Deploy the configuration using the wadm deploy-config command.

To access perfdump, access the URI on the virtual server.

For example: http://localhost:80/.perf

You can request the perfdump statistics and specify how frequently the browser should automatically refresh as measured in seconds. The following example sets the refresh to every 5 seconds:

http://yourhost/.perf?refresh=5

To View the perfdump Data from the CLI

In addition to a URI, you can also access perfdump output through the command-line interface. It is enabled by default. Unlike viewing perfdump output through the URI, the Administration Server must be running to view perfdump output at the command-line. However, if request processing threads are hanging in your server and you cannot use the URI, you can still access perfdump output through the CLI.

To view the perfdump output through the command-line interface, enter:

./wadm get-perfdump --user=admin-user --password-file=admin-password-file --config=config-name --node=node-name

The output appears in your command window.

Sample perfdump Output

The following is sample perfdump output:

webservd pid: 29133

Sun Java System Web Server 7.0 B07/13/2006 17:09 (SunOS DOMESTIC)

Server started Fri Jul 14 14:34:15 2006
Process 29133 started Fri Jul 14 14:34:17 2006

ConnectionQueue:
-----------------------------------------
Current/Peak/Limit Queue Length            2/237/1352
Total Connections Queued                   67364017
Average Queue Length (1, 5, 15 minutes)    4.52, 4.73, 4.85
Average Queueing Delay                     13.63 milliseconds

ListenSocket ls1:
------------------------
Address                   https://0.0.0.0:2014
Acceptor Threads          1
Default Virtual Server    https-test

KeepAliveInfo:
--------------------
KeepAliveCount        198/200
KeepAliveHits         0
KeepAliveFlushes      0
KeepAliveRefusals     56844280
KeepAliveTimeouts     365589
KeepAliveTimeout      10 seconds

SessionCreationInfo:
------------------------
Active Sessions           128
Keep-Alive Sessions       0
Total Sessions Created    128/128

Server cache disabled

Native pools:
----------------------------
NativePool:
Idle/Peak/Limit               1/1/128
Work Queue Length/Peak/Limit  0/0/0
TestPool:
Idle/Peak/Limit               5/5/10
Work Queue Length/Peak/Limit  0/0/15

DNSCacheInfo:
------------------
enabled             yes
CacheEntries        4/1024
HitRatio            62854802/62862912 ( 99.99%)

Async DNS disabled

Performance Counters:
------------------------------------------------
                           Average         Total      Percent

Total number of requests:               62647125
Request processing time:    0.0343  2147687.2500

default-bucket (Default bucket)
Number of Requests:                     62647125    (100.00%)
Number of Invocations:                3374170785    (100.00%)
Latency:                    0.0008    47998.2500    (  2.23%)
Function Processing Time:   0.0335  2099689.0000    ( 97.77%)
Total Response Time:        0.0343  2147687.2500    (100.00%)

Sessions:
-----------------------------------------------------------------------------------------------------------
Process  Status    Client     Age  VS          Method URI                                      Function

29133    response  192.6.7.7  115  https-test  GET   /qa_webapp/CheckNetwork.class             service-j2ee
29133    response  192.6.7.7  8    https-test  GET   /qa_webapp/CheckNetwork.class             service-j2ee
29133    response  192.6.7.7  4    https-test  GET   /qa_webapp/CheckNetwork.class             service-j2ee
29133    response  10.5.8.19  4    https-test  GET   /perf                                     service-dump
29133    response  192.6.7.7  3    https-test  GET   /qa_webapp/CheckNetwork.class             service-j2ee
29133    response  192.6.7.7  3    https-test  GET   /qa_webapp/CheckNetwork.class             service-j2ee
29133    response  192.6.7.7  2    https-test  GET   /qa_webapp/CheckNetwork.class             service-j2ee
29133    response  192.6.7.7  2    https-test  GET   /qa_webapp/CheckNetwork.class             service-j2ee
29133    response  192.6.7.7  2    https-test  GET   /qa_webapp/CheckNetwork.class             service-j2ee
29133    response  192.6.7.7  2    https-test  GET   /qa_webapp/CheckNetwork.class             service-j2ee
29133    request   192.6.7.7  0                                                                  
29133    request   192.6.7.7  0                                                                  
29133    request   192.6.7.7  0                                                                  
29133    request   192.6.7.7  0                                                                  
29133    request   192.6.7.7  0                                                                  
29133    response  192.6.7.7  0    https-test  GET   /file1.shtml                              shtml_send
29133    request   192.6.7.7  0                                                                     
29133    request   192.6.7.7  0                                                                     
29133    response  192.6.7.7  0    https-test  GET   /find-pathinfo-forward/pathinfo.pl/p/info send-cgi
29133    request   192.6.7.7  0                                                                     
29133    updating  192.6.7.7                                                                        
29133    updating  192.6.7.7                                                                         
29133    updating  192.6.7.7                                                                         
29133    updating  192.6.7.7                                                                         
.
.
.

Using Performance Buckets

Performance buckets allow you to define buckets and link them to various server functions. Every time one of these functions is invoked, the server collects statistical data and adds it to the bucket. For example, send-cgi and service-j2ee are functions used to serve the CGI and Java servlet requests respectively. You can either define two buckets to maintain separate counters for CGI and servlet requests, or create one bucket that counts requests for both types of dynamic content. The cost of collecting this information is minimal, and the impact on the server performance is usually negligible. This information can later be accessed using perfdump. The following information is stored in a bucket:

Name of the bucket. This name associates the bucket with a function.
Description. A description of the functions with which the bucket is associated.
Number of requests for this function. The total number of requests that caused this function to be called.
Number of times the function was invoked. This number might not coincide with the number of requests for the function, because some functions might be executed more than once for a single request.
Function latency or the dispatch time. The time taken by the server to invoke the function.
Function time. The time spent in the function itself.

The default-bucket is predefined by the server. It records statistics for the functions not associated with any user-defined bucket.

Configuration

You must specify all configuration information for performance buckets in the magnus.conf and obj.conf files. Only the default-bucket is automatically enabled.

First, you must enable performance statistics collection and perfdump.

The following examples show how to define new buckets in magnus.conf:

Init fn="define-perf-bucket" name="acl-bucket" description="ACL bucket"

Init fn="define-perf-bucket" name="file-bucket" description="Non-cached responses"

Init fn="define-perf-bucket" name="cgi-bucket" description="CGI Stats"

The previous examples create three buckets: acl-bucket, file-bucket, and cgi-bucket. To associate these buckets with functions, add bucket=bucket-name to the obj.conf function for which to measure performance.

Example

PathCheck fn="check-acl" acl="default" bucket="acl-bucket"
...
Service method="(GET|HEAD|POST)" type="*~magnus-internal/*" 
fn="send-file" bucket="file-bucket"
...
<Object name="cgi">
ObjectType fn="force-type" type="magnus-internal/cgi"
Service fn="send-cgi" bucket="cgi-bucket"
</Object>

For more information, see The bucket Parameter in Sun Java System Web Server 7.0 Update 4 Administrator’s Configuration File Reference.

Performance Report

The server statistics in buckets can be accessed using perfdump. The performance buckets information is located in the last section of the report returned by perfdump.

The report contains the following information:

Average, Total, and Percent columns give data for each requested statistic.
Request Processing Time is the total time required by the server to process all requests it has received so far.
Number of Requests is the total number of requests for the function.
Number of Invocations is the total number of times that the function was invoked. This number differs from the number of requests because a function can be called multiple times while processing one request. The percentage column for this row is calculated in reference to the total number of invocations for all of the buckets.
Latency is the time in seconds that Web Server uses to call the function.
Function Processing Time is the time in seconds that Web Server spends inside the function. The percentage of Function Processing Time and Total Response Time is calculated with reference to the total Request Processing Time.
Total Response Time is the sum in seconds of Function Processing Time and Latency.

The following is an example of the performance bucket information available through perfdump:

Performance Counters:
------------------------------------------------
                           Average         Total      Percent

Total number of requests:               62647125
Request processing time:    0.0343  2147687.2500

default-bucket (Default bucket)
Number of Requests:                     62647125    (100.00%)
Number of Invocations:                3374170785    (100.00%)
Latency:                    0.0008    47998.2500    (  2.23%)
Function Processing Time:   0.0335  2099689.0000    ( 97.77%)
Total Response Time:        0.0343  2147687.2500    (100.00%)

Monitoring Current Activity Using the Java ES Monitoring Console

The statistics available through the Web Server Admin Console and the command-line interface are also available through the Java ES Monitoring Console. Though the information is the same, it is presented in a different format, using the Common Monitoring Data Model (CMM). Though this guide covers monitoring using tools available in the Web Server, you can also monitor your server using the Java ES monitoring tools. For more information on using the Java ES monitoring tools, see Sun Java Enterprise System 5 Monitoring Guide. Use the same settings to tune the server, regardless of what monitoring method you use.