Making the adjustments described in this guide without measuring their effects doesn’t make sense. If you don’t measure the system’s behavior before and after making a change, you won’t know whether the change was a good idea, a bad idea, or merely irrelevant. You can monitor the performance of Web Server in several different ways.
Table 1–1 Methods of Monitoring Performance
Monitoring Method |
How to Enable |
How to Access |
Advantages and Requirements |
---|---|---|---|
Statistics through the Admin Console |
Enabled by default |
In the Admin Console, for a configuration, click the Monitor tab |
Accessible when session threads are hanging. Administration Server must be running. |
Statistics through individual wadm commands |
Enabled by default |
Through wadm commands: get-config-statsget-virtual-server-statsget-webapp-statsget-servlet-stats |
Accessible when session threads are hanging. Administration Server must be running. |
XML-formatted statistics (stats-xml) through a browser |
Enable through Admin Console or through editing a configuration file |
Through a URI |
Administration Server need not be running. |
XML-formatted statistics (stats-xml) through the command-line interface |
Enabled by default |
Through the wadm command get-stats-xml |
Accessible when session threads are hanging. Administration Server must be running. |
perfdump through a browser |
Enable through Admin Console or through editing a configuration file |
Through a URI |
Administration Server need not be running. |
perfdump through the command-line interface |
Enabled by default |
Through wadm command get-perfdump |
Accessible when session threads are hanging. Administration Server must be running. |
Java ES Monitoring |
Enabled by default |
Through the Java ES Monitoring Console |
Only for Java ES installations. Administration Server must be running. |
Monitoring the server does have some impact on computing resources. In general, using perfdump through the URI is the least costly, followed by using stats-xml through a URI. Because using the Administration Server requires computing resources, the command-line interface and the Admin Console are the most costly monitoring methods.
For more information on these monitoring methods, see the following sections:
You can monitor many performance statistics through the Admin Console user interface, through the command-line interface, through the stats-xml URI, and through perfdump. For all these monitoring methods, the server uses the statistics it collects. None of these monitoring methods will work if statistics are not collected.
The statistics give you information at the configuration level, the server instance level, or the virtual server level. The statistics are broken up into functional areas.
For the configuration, statistics are available in the following areas:
Requests
Errors
Response Time
For the server instance, statistics are available in the following areas:
Requests
Errors
Response Time
General
Java Virtual Machine (JVMTM)
Connection Queue
Keep Alive
DNS
File Cache
Thread Pools
Session Replication
Session Threads, including Profiling data (available if profiling is enabled)
Java DataBase Connectivity (JDBCTM) (available if a JDBC resource is created and the connection pool is accessed)
For the virtual server, statistics are available in the following areas:
General
Response
Web Applications
Profiling Data (available if profiling is enabled)
Servlet and Servlet Response Cache (available if the Servlet cache is enabled in sun.web.xml)
Some statistics default to zero if Quality of Service (QoS) is not enabled, for example, the count of open connections, the maximum open connections, the rate of bytes transmitted, and the maximum byte transmission rate.
Statistics are activated by default on Web Server. However, if you have disabled them, you need to enable them again to monitor your server for performance. To enable statistics, use Admin Console or the wadm command-line utility (CLI).
Collecting statistics causes a slight hit to performance.
From the Admin Console Common Tasks page, select the configuration.
Click Edit Configuration.
Click the General tab.
Click the Monitoring Settings sub tab.
On the Monitoring Settings page, under General Settings, select the Statistics Enabled checkbox.
Configure the interval and profiling.
Restart the server.
Enter the following CLI command to enable statistics collection:
./wadm set-stats-prop --user=admin_user –password-file=password-file --config=myconfig enabled=true
To disable statistics, set enabled to false.
To set the interval and enable profiling, use the set-stats-prop interval and profiling properties. For more information, see the help for set-stats-prop.
Restart the server.
Frequently-used statistics are available through the Admin Console, viewed as general statistics, instance statistics, and virtual server statistics.
In the Admin Console, from the Common Tasks page, select the Monitoring tab.
Select the configuration.
The configuration statistics are displayed.
From the drop-down list, select a View interval.
The statistics displayed in your browser are automatically updated at this interval.
Select the type of statistics to display.
The initial list of statistics types includes General Statistics, Instance Statistics, and Virtual Server Statistics.
If you choose Instance Statistics, click the name of the instance to monitor. Detailed statistics are then displayed, including information on processes and session replications.
If you choose Virtual Server Statistics, click the name of the virtual server to monitor. Statistics for the virtual server are displayed, including response statistics and web application statistics. This information is not provided through perfdump.
You can also view statistics information using the wadm commands get-config-stats, get-virtual-server-stats, get-webapp-stats and get-servlet-stats. Note that the examples below do not contain all possible command options. For the complete syntax, see the help for the command.
To deploy statistics for configuration on a single node, enter:
./wadm get-config-stats --user=admin-user --password-file=admin-password-file --config=config-name --node=node-name
Using the node option in this syntax restricts the output to a single node. To view the statistics at the configuration level, use the command without the node option.
The following shows an example of the output for a single node:
timeStarted=1168035653 secondsRunning=1404 countRequests=690546 rpsLast1MinAvg=4491.7666 rpsLast5MinAvg=1844.6061 rpsLast15MinAvg=637.37305 countErrors=0 epsLast1MinAvg=0.0 epsLast5MinAvg=0.0 epsLast15MinAvg=0.0 maxResponseTime=0.30789953 rtLast1MinAvg=5.3970284 rtLast5MinAvg=5.208407 rtLast15MinAvg=35.56042 countBytesReceived=96800935 countBytesTransmitted=689929574 countChildDied=0 countVirtualServers=2 instanceName=https-test process.1.countThreadPools=2 process.1.jdbcPoolCount=1 process.1.countThreads=64 process.1.fractionSystemMemoryUsage=2887.0 process.1.countConnectionQueues=1 process.1.sizeResident=0 process.1.countIdleThreads=32 process.1.mode=1 process.1.sizeVirtual=0 process.1.countConfigurations=1 process.1.pid=15874 process.1.timeStarted=Jan 5, 2007 2:20:53 PM process.1.DNSCache.countCacheHits=687804 process.1.DNSCache.countAsyncNameLookup=0 process.1.DNSCache.countAsyncLookupsInProgress=0 process.1.DNSCache.flagAsyncEnabled=false process.1.DNSCache.countAsyncAddrLookups=0 process.1.DNSCache.flagCacheEnabled=true process.1.DNSCache.countCacheMisses=75 process.1.JDBCPool.1.countQueued=32 process.1.JDBCPool.1.countFreeConnections=0 process.1.JDBCPool.1.peakConnections=32 process.1.JDBCPool.1.millisecondsPeakWait=72 process.1.JDBCPool.1.countWaitQueueTimeouts=288 process.1.JDBCPool.1.peakQueued=64 process.1.JDBCPool.1.maxConnections=32 process.1.JDBCPool.1.currentConnections=32 process.1.JDBCPool.1.millisecondsAverageQueued=1.0 process.1.JDBCPool.1.countTotalFailedValidationConnections=0 process.1.JDBCPool.1.countLeasedConnections=32 process.1.JDBCPool.1.countTotalLeasedConnections=414 process.1.JDBCPool.1.countConnectionIdleTimeouts=1 process.1.JDBCPool.1.name=jdbc/jdbc-simple_1 process.1.connectionQueue.1.countQueued15MinuteAverage=4.3203125 process.1.connectionQueue.1.countQueued=0 process.1.connectionQueue.1.countQueued1MinuteAverage=0.046875 process.1.connectionQueue.1.countTotalQueued=79171 process.1.connectionQueue.1.countQueued5MinuteAverage=4.03125 process.1.connectionQueue.1.countOverflows=0 process.1.connectionQueue.1.maxQueued=1288 process.1.connectionQueue.1.ticksTotalQueued=724956383 process.1.connectionQueue.1.countTotalConnections=863 process.1.connectionQueue.1.peakQueued=64 process.1.connectionQueue.1.name=cq1 process.1.fileCache.countContentMisses=7 process.1.fileCache.maxMmapCacheSize=0 process.1.fileCache.sizeHeapCache=27520 process.1.fileCache.countMisses=22 process.1.fileCache.countContentHits=620662 process.1.fileCache.maxEntries=1024 process.1.fileCache.flagEnabled=true process.1.fileCache.secondsMaxAge=30 process.1.fileCache.sizeMmapCache=0 process.1.fileCache.countInfoHits=1862013 process.1.fileCache.maxHeapCacheSize=10747924 process.1.fileCache.countOpenEntries=0 process.1.fileCache.countHits=2482682 process.1.fileCache.maxOpenEntries=1024 process.1.fileCache.countEntries=12 process.1.fileCache.countInfoMisses=19 process.1.jvm.countGarbageCollections=96 process.1.jvm.sizeHeap=67762048 process.1.jvm.countThreads=79 process.1.jvm.countClassesUnloaded=0 process.1.jvm.vMVendor=Sun Microsystems Inc. process.1.jvm.countTotalClassesLoaded=3170 process.1.jvm.vMName=Java HotSpot(TM) Server VM process.1.jvm.countTotalThreadsStarted=81 process.1.jvm.countClassesLoaded=3170 process.1.jvm.peakThreads=79 process.1.jvm.millisecondsGarbageCollection=1981 process.1.jvm.vMVersion=1.5.0_09-b03 process.1.keepalive.countConnections=32 process.1.keepalive.maxConnections=200 process.1.keepalive.countFlushes=0 process.1.keepalive.countRefusals=0 process.1.keepalive.countTimeouts=6 process.1.keepalive.countHits=686943 process.1.keepalive.secondsTimeout=30 process.1.threadPool.1.countQueued=0 process.1.threadPool.1.countThreadsIdle=1 process.1.threadPool.1.threadPoolId=NativePool process.1.threadPool.1.maxThreads=128 process.1.threadPool.1.countThreads=1 process.1.threadPool.1.maxQueued=0 process.1.threadPool.1.peakQueued=0 process.1.threadPool.1.name=NativePool process.1.threadPool.2.countQueued=0 process.1.threadPool.2.countThreadsIdle=1 process.1.threadPool.2.threadPoolId=my-custom-pool process.1.threadPool.2.maxThreads=128 process.1.threadPool.2.countThreads=1 process.1.threadPool.2.maxQueued=0 process.1.threadPool.2.peakQueued=0 process.1.threadPool.2.name=my-custom-pool
To obtain statistics for a virtual server, enter:
./wadm get-virtual-server-stats --user=admin-user --password-file=admin-password-file --config=config-name --vs=virtual-server-name
Because the node option is not used, this syntax provides aggregate statistics for the virtual server across all the nodes where the configuration has been deployed. Using the node option restricts the output to a single node.
To obtain statistics for a deployed web application, enter:
./wadm get-webapp-stats --user=admin-user --password-file=admin-password-file --config=config-name --node=node-name --vs=virtual-server-name --uri=URI
The syntax obtains statistics for a given web application deployed on the given virtual server of the given instance. To obtain the aggregated web application statistics for a given configuration across all the nodes where the configuration has been deployed, use the command without the node option.
The following example shows the output for the URI hello:
countActiveSessions=1 countExpiredSessions=0 countJsps=1 countRejectedSessions=0 countReloadedJsps=1 countSessions=1 peakActiveSessions=1 secondsSessionAliveAverage=0 secondsSessionAliveMax=0 uri=/hello vsName=myvs.sun.com
You can also display statistics using stats-xml, which displays statistics in XML format. The output of stats-xml is in XML so that various tools can easily parse the statistics. You can view the stats-xml output through a URI, which you have to enable, or you can view the stats-xml output through the CLI, which is enabled by default.
If you enable the stats-xml URI, you can access statistics for your server in XML format through a browser. Note that when you use the stats-xml URI, you can access statistics even when the Administration Server is not running. Also, with the stats-xml URI activated, users can see the statistics information for your server, unless you take precautions to deny access.
On the Common Tasks page, select the configuration from the pull-down menu on the left.
Select the virtual server from the pull-down menu on the right, then click Edit Virtual Server.
On the Server Settings tab, click the Monitoring Settings sub tab.
Select the XML Report enabled checkbox.
Provide a URI, for example, /stats-xml.
Click Save.
Deploy the configuration.
Access the stats-xml URI, for example:
http://yourhost:port/stats-xml
The statistics are displayed in XML format.
Use the following command to enable stats-xml:
./wadm enable-stats-xml --user=admin-user --password-file=admin-password-file [--uri-prefix=prefix]--config=config-name --vs=virtual-server-name
Use the uri-prefix option to set the stats-xml URI.
Deploy the configuration using the wadm deploy-config command.
Access the stats-xml URI, for example:
http://yourhost:port/stats-xml
The statistics are displayed in XML format.
You can modify the stats-xml URI to limit the data it provides.
Modify the stats-xml URI to limit the information by setting elements to 0 or 1. An element set to 0 is not displayed on the stats-xml output. For example:
http://yourhost:port/stats-xml?thread=0&process=0
This syntax limits the stats-xml output so that thread and process statistics are not included. By default all statistics are enabled (set to 1).
Most of the statistics are available at the server level, but some are available at the process level.
Use the following syntax elements to limit stats-xml:
cache-bucket
connection-queue
connection-queue-bucket (process-level)
cpu-info
dns-bucket
jdbc-resource-bucket
keepalive-bucket
process
profile
profile-bucket (process-level)
request-bucket
servlet-bucket
session-replication
thread
thread-pool
thread-pool-bucket (process-level)
virtual-server
web-app-bucket
In addition to a URI, you can also access stats-xml output through the command-line interface which is enabled by default. Unlike viewing stats-xml output through the URI, the Administration Server must be running to view stats-xml output at the command-line. However, if request processing threads are hanging in your server, and you cannot use the URI, you can still access stats-xml output through the CLI.
To view the stats-xml output through the command-line interface, enter:
./wadm get-stats-xml --user=admin-user --password-file=admin-password-file --config=config-name --node=node-name
perfdump is a Server Application Function (SAF) built into Web Server that collects various pieces of performance data from the Web Server internal statistics and displays them in ASCII text. The perfdump output does not display all the statistics available through the command-line statistics or the Admin Console, but it can still be a useful tool. For example, you can still use perfdump even if the Administration Server is not running. You can view the perfdump output through the CLI, which is enabled by default, or you can view the perfdump output through a URI, which you have to enable. If you enable the URI, you must control access to the perfdump URI, otherwise it can be visible to users.
With perfdump, the statistics are unified. Rather than monitoring a single process, statistics are multiplied by the number of processes, which gives you an accurate view of the server as a whole.
For information on tuning the information displayed by perfdump, see Using Monitoring Data to Tune Your Server.
You can enable perfdump URI for a virtual server through the Admin Console.
The statistics displayed by perfdump are for the server as a whole. If you enable perfdump on one virtual server, it displays statistics for the whole server, not an individual virtual server.
From Common Tasks, select a configuration.
Select the virtual server and click Edit Virtual Server.
Click the Monitoring Settings tab.
Select the Plain Text Report Enabled checkbox.
Provide a URI for accessing the report, for example /.perf.
Click Save.
Deploy the configuration.
To access perfdump, access the URI on the virtual server.
For example: http://localhost:80/.perf
You can request the perfdump statistics and specify how frequently the browser should automatically refresh as measured in seconds. The following example sets the refresh to every 5 seconds:
http://yourhost/.perf?refresh=5
Use the following command to enable perfdump:
./wadm enable-perfdump --user=admin-user --password-file=admin-password-file [--uri=uri]--config=config-name--vs=virtual-server-name
Use the uri option to set the pefdump URI.
Deploy the configuration using the wadm deploy-config command.
To access perfdump, access the URI on the virtual server.
For example: http://localhost:80/.perf
You can request the perfdump statistics and specify how frequently the browser should automatically refresh as measured in seconds. The following example sets the refresh to every 5 seconds:
http://yourhost/.perf?refresh=5
In addition to a URI, you can also access perfdump output through the command-line interface. It is enabled by default. Unlike viewing perfdump output through the URI, the Administration Server must be running to view perfdump output at the command-line. However, if request processing threads are hanging in your server and you cannot use the URI, you can still access perfdump output through the CLI.
To view the perfdump output through the command-line interface, enter:
./wadm get-perfdump --user=admin-user --password-file=admin-password-file --config=config-name --node=node-name
The output appears in your command window.
The following is sample perfdump output:
webservd pid: 29133 Sun Java System Web Server 7.0 B07/13/2006 17:09 (SunOS DOMESTIC) Server started Fri Jul 14 14:34:15 2006 Process 29133 started Fri Jul 14 14:34:17 2006 ConnectionQueue: ----------------------------------------- Current/Peak/Limit Queue Length 2/237/1352 Total Connections Queued 67364017 Average Queue Length (1, 5, 15 minutes) 4.52, 4.73, 4.85 Average Queueing Delay 13.63 milliseconds ListenSocket ls1: ------------------------ Address https://0.0.0.0:2014 Acceptor Threads 1 Default Virtual Server https-test KeepAliveInfo: -------------------- KeepAliveCount 198/200 KeepAliveHits 0 KeepAliveFlushes 0 KeepAliveRefusals 56844280 KeepAliveTimeouts 365589 KeepAliveTimeout 10 seconds SessionCreationInfo: ------------------------ Active Sessions 128 Keep-Alive Sessions 0 Total Sessions Created 128/128 Server cache disabled Native pools: ---------------------------- NativePool: Idle/Peak/Limit 1/1/128 Work Queue Length/Peak/Limit 0/0/0 TestPool: Idle/Peak/Limit 5/5/10 Work Queue Length/Peak/Limit 0/0/15 DNSCacheInfo: ------------------ enabled yes CacheEntries 4/1024 HitRatio 62854802/62862912 ( 99.99%) Async DNS disabled Performance Counters: ------------------------------------------------ Average Total Percent Total number of requests: 62647125 Request processing time: 0.0343 2147687.2500 default-bucket (Default bucket) Number of Requests: 62647125 (100.00%) Number of Invocations: 3374170785 (100.00%) Latency: 0.0008 47998.2500 ( 2.23%) Function Processing Time: 0.0335 2099689.0000 ( 97.77%) Total Response Time: 0.0343 2147687.2500 (100.00%) Sessions: ----------------------------------------------------------------------------------------------------------- Process Status Client Age VS Method URI Function 29133 response 192.6.7.7 115 https-test GET /qa_webapp/CheckNetwork.class service-j2ee 29133 response 192.6.7.7 8 https-test GET /qa_webapp/CheckNetwork.class service-j2ee 29133 response 192.6.7.7 4 https-test GET /qa_webapp/CheckNetwork.class service-j2ee 29133 response 10.5.8.19 4 https-test GET /perf service-dump 29133 response 192.6.7.7 3 https-test GET /qa_webapp/CheckNetwork.class service-j2ee 29133 response 192.6.7.7 3 https-test GET /qa_webapp/CheckNetwork.class service-j2ee 29133 response 192.6.7.7 2 https-test GET /qa_webapp/CheckNetwork.class service-j2ee 29133 response 192.6.7.7 2 https-test GET /qa_webapp/CheckNetwork.class service-j2ee 29133 response 192.6.7.7 2 https-test GET /qa_webapp/CheckNetwork.class service-j2ee 29133 response 192.6.7.7 2 https-test GET /qa_webapp/CheckNetwork.class service-j2ee 29133 request 192.6.7.7 0 29133 request 192.6.7.7 0 29133 request 192.6.7.7 0 29133 request 192.6.7.7 0 29133 request 192.6.7.7 0 29133 response 192.6.7.7 0 https-test GET /file1.shtml shtml_send 29133 request 192.6.7.7 0 29133 request 192.6.7.7 0 29133 response 192.6.7.7 0 https-test GET /find-pathinfo-forward/pathinfo.pl/p/info send-cgi 29133 request 192.6.7.7 0 29133 updating 192.6.7.7 29133 updating 192.6.7.7 29133 updating 192.6.7.7 29133 updating 192.6.7.7 . . .
Performance buckets allow you to define buckets and link them to various server functions. Every time one of these functions is invoked, the server collects statistical data and adds it to the bucket. For example, send-cgi and service-j2ee are functions used to serve the CGI and Java servlet requests respectively. You can either define two buckets to maintain separate counters for CGI and servlet requests, or create one bucket that counts requests for both types of dynamic content. The cost of collecting this information is minimal, and the impact on the server performance is usually negligible. This information can later be accessed using perfdump. The following information is stored in a bucket:
Name of the bucket. This name associates the bucket with a function.
Description. A description of the functions with which the bucket is associated.
Number of requests for this function. The total number of requests that caused this function to be called.
Number of times the function was invoked. This number might not coincide with the number of requests for the function, because some functions might be executed more than once for a single request.
Function latency or the dispatch time. The time taken by the server to invoke the function.
Function time. The time spent in the function itself.
The default-bucket is predefined by the server. It records statistics for the functions not associated with any user-defined bucket.
You must specify all configuration information for performance buckets in the magnus.conf and obj.conf files. Only the default-bucket is automatically enabled.
First, you must enable performance statistics collection and perfdump.
The following examples show how to define new buckets in magnus.conf:
Init fn="define-perf-bucket" name="acl-bucket" description="ACL bucket" Init fn="define-perf-bucket" name="file-bucket" description="Non-cached responses" Init fn="define-perf-bucket" name="cgi-bucket" description="CGI Stats" |
The previous examples create three buckets: acl-bucket, file-bucket, and cgi-bucket. To associate these buckets with functions, add bucket=bucket-name to the obj.conf function for which to measure performance.
Example
PathCheck fn="check-acl" acl="default" bucket="acl-bucket" ... Service method="(GET|HEAD|POST)" type="*~magnus-internal/*" fn="send-file" bucket="file-bucket" ... <Object name="cgi"> ObjectType fn="force-type" type="magnus-internal/cgi" Service fn="send-cgi" bucket="cgi-bucket" </Object>
For more information, see The bucket Parameter in Sun Java System Web Server 7.0 Update 6 Administrator’s Configuration File Reference.
The server statistics in buckets can be accessed using perfdump. The performance buckets information is located in the last section of the report returned by perfdump.
The report contains the following information:
Average, Total, and Percent columns give data for each requested statistic.
Request Processing Time is the total time required by the server to process all requests it has received so far.
Number of Requests is the total number of requests for the function.
Number of Invocations is the total number of times that the function was invoked. This number differs from the number of requests because a function can be called multiple times while processing one request. The percentage column for this row is calculated in reference to the total number of invocations for all of the buckets.
Latency is the time in seconds that Web Server uses to call the function.
Function Processing Time is the time in seconds that Web Server spends inside the function. The percentage of Function Processing Time and Total Response Time is calculated with reference to the total Request Processing Time.
Total Response Time is the sum in seconds of Function Processing Time and Latency.
The following is an example of the performance bucket information available through perfdump:
Performance Counters: ------------------------------------------------ Average Total Percent Total number of requests: 62647125 Request processing time: 0.0343 2147687.2500 default-bucket (Default bucket) Number of Requests: 62647125 (100.00%) Number of Invocations: 3374170785 (100.00%) Latency: 0.0008 47998.2500 ( 2.23%) Function Processing Time: 0.0335 2099689.0000 ( 97.77%) Total Response Time: 0.0343 2147687.2500 (100.00%)
The statistics available through the Web Server Admin Console and the command-line interface are also available through the Java ES Monitoring Console. Though the information is the same, it is presented in a different format, using the Common Monitoring Data Model (CMM). Though this guide covers monitoring using tools available in the Web Server, you can also monitor your server using the Java ES monitoring tools. For more information on using the Java ES monitoring tools, see Sun Java Enterprise System 5 Monitoring Guide. Use the same settings to tune the server, regardless of what monitoring method you use.