Sun Java System Message Queue 3 2005Q1 Administration Guide |
Chapter 10
Monitoring a Message ServerThis chapter describes the tools you can use to monitor a message server and how you can get metrics data. The chapter has the following sections:
Reference information on specific metrics is available in Chapter 18, "Metrics Reference."
Introduction to Monitoring ToolsThere are three monitoring interfaces for Message Queue information: log files, interactive commands, and a client API that can obtain metrics. Each has its advantages and disadvantages, as follows:
- Log files provide a long-term record of metrics data, but cannot easily be parsed.
- Commands enable you to quickly sample information tailored to your needs, but do not enable you to look at historical information or manipulate the data programmatically.
- The client API lets you extract information, process it, manipulate the data, present graphs or send alerts. However, to use it, you must write a custom application to capture and analyze the data.
Table 10-1 compares the different tools.
In addition to the differences shown in the table, each tool gathers a somewhat different subset of the metrics information generated by the broker. For information on which metrics data is gathered by each monitoring tool, see Chapter 18, "Metrics Reference".
Configuring and Using Broker LoggingThe Message Queue logger takes information generated by broker code, a debugger, and a metrics generator and writes that information to a number of output channels: to standard output (the console), to a log file, and, on Solaris operating systems, to the syslog daemon process.
You can specify the type of information gathered by the logger as well as the type written to each of the output channels. In particular, you can specify that you want metrics information written out to a log file.
This section describes the default logging configuration for the broker and explains how to redirect log information to alternative output channels, how to change log file rollover criteria, and how to send metrics data to a log file.
Default Logging Configuration
A broker is automatically configured to save log output to a set of rolling log files. The log files are located in a directory identified by the instance name of the associated broker (see Appendix A, "Operating System-Specific Locations of Message Queue Data"):
…/instances/instanceName/log/
The log files are simple text files. They are named as follows, from earliest to latest:
log.txt
log_1.txt
log_2.txt
…
log_9.txtBy default, log files are rolled over once a week; the system maintains nine backup files.
The broker supports three log levels: ERROR, WARNING, INFO. Table 10-2 explains each level.
Setting a logging level gathers messages for that level and all higher levels. The default log level is INFO, so ERROR, WARNING, and INFO messages are all logged by default.
Log Message Format
A logged message consists of a timestamp, message code, and the message itself. The volume of information varies with the log level you have set. The following is an example of an INFO message.
[13/Sep/2000:16:13:36 PDT] B1004 Starting the broker service using tcp [ 25374,100] with min threads 50 and max threads of 500
To change the timestamp time zone, see information about the imq.log.timezone property, which is described in Table 14-10.
Changing the Logger Configuration
Log-related properties are described in Table 14-10.
To Change the Logger Configuration for a Broker
You complete these steps by setting logger properties. You can do this in one of two ways:
Options passed on the command line override properties specified in the broker instance configuration files. Table 10-3 lists the imqbrokerd options that affect logging.
The following sections describe how you can change the default configuration in order to do the following:
Changing the Output Channel
By default, error and warning messages are displayed on the terminal as well as being logged to a log file. (On Solaris, error messages are also written to the system’s syslog daemon.)
You can change the output channel for log messages in the following ways:
- To have all log categories (for a given level) output displayed on the screen, use the -tty option to the imqbrokerd command.
- To prevent log output from being displayed on the screen, use the -silent option to the imqbrokerd command.
- Use the imq.log.file.output property to specify which categories of logging information should be written to the log file. For example,
imq.log.file.output=ERROR
- Use the imq.log.console.output property to specify which categories of logging information should be written to the console. For example,
imq.log.console.output=INFO
- On Solaris, use the imq.log.syslog.output property to specify which categories of logging information should be written to Solaris syslog. For example,
imq.log.syslog.output=NONE
Changing Log File Rollover Criteria
There are two criteria for rolling over log files: time and size. The default is to use a time criteria and roll over files every seven days.
- To change the time interval, you need to change the property imq.log.file.rolloversecs. For example, the following property definition changes the time interval to ten days:
imq.log.file.rolloversecs=864000
- To change the rollover criteria to depend on file size, you need to set the imq.log.file.rolloverbytes property. For example, the following definition directs the broker to rollover files after they reach a limit of 500,000 bytes
imq.log.file.rolloverbytes=500000
If you set both the time-related and the size-related rollover properties, the first limit reached will trigger the rollover. As noted before, the broker maintains up to nine rollover files.
You can set or change the log file rollover properties when a broker is running. To set these properties, use the imqcmd update bkr command.
Sending Metrics Data to Log Files
This section describes the procedure for using broker log files to report metrics information. For general information on configuring the logger, see Configuring and Using Broker Logging.
To Use Log Files to Report Metrics Information
- Configure the broker’s metrics generation capability:
- Confirm imq.metrics.enabled=true
Generation of metrics for logging is turned on by default.
- Set the metrics generation interval to a convenient number of seconds.
imq.metrics.interval=interval
This value can be set in the config.properties file or using the
-metrics interval command line option when starting up the broker.- Confirm that the logger gathers metrics information:
imq.log.level=INFO
This is the default value. This value can be set in the config.properties file or using the -loglevel level command line option when starting up the broker.
- Confirm that the logger is set to write metrics information to the log file:
imq.log.file.output=INFO
This is the default value. It can be set in the config.properties file.
- Start up the broker.
The following shows sample broker metrics output to the log file:
For reference information about metrics data, see Chapter 18, "Metrics Reference."
Logging Dead Messages
You can monitor physical destinations by enabling dead message logging for a broker. You can log dead messages whether or not you are using a dead message queue.
If you enable dead message logging, the broker logs the following types of events:
If a dead message queue is in use, logging also includes the following types of events:
Dead message logging is disabled by default. To enable it, set the broker attribute imq.destination.logDeadMsgs.
Interactively Displaying MetricsA Message Queue broker can report the following types of metrics:
- Java Virtual Machine (JVM) metrics. Information about the JVM heap size.
- Broker-wide metrics. Information about messages stored in a broker, message flows into and out of a broker, and memory use. Messages are tracked in terms of numbers of messages and numbers of bytes.
- Connection Service metrics. Information about connections and connection thread resources, a nd information about message flows for a particular connection service.
- Destination metrics. Information about message flows into and out of a particular physical destination, information about a physical destination’s consumers, and information about memory and disk space usage.
The imqcmd command can obtain metrics information for the broker as a whole, for individual connection services, and for individual physical destinations. To obtain metrics data, you generally use the metrics subcommand of imqcmd. Metrics data is written at an interval you specify, or the number of times you specify, to the console screen.
You can also use the query subcommand to view similar data that also includes configuration information. See imqcmd query for more information.
imqcmd metrics
The syntax and options of imqcmd metrics are shown in Table 10-4 and Table 10-5, respectively.
Table 10-4 imqcmd metrics Subcommand Syntax
Subcommand Syntax
Metrics Data Provided
metrics bkr
[-b hostName:port]
[-m metricType]
[-int interval]
[-msp numSamples]Displays broker metrics for the default broker or a broker at the specified host and port.
or
metrics svc -n serviceName
[-b hostName:port]
[-m metricType]
[-int interval]
[-msp numSamples]Displays metrics for the specified service on the default broker or on a broker at the specified host and port.
or
metrics dst -t destType
-n destName
[-b hostName:port]
[-m metricType]
[-int interval]
[-msp numSamples]Displays metrics information for the physical destination of the specified type and name.
Table 10-5 imqcmd metrics Subcommand Options
Subcommand Options
Description
-b hostName:port
Specifies the hostname and port of the broker for which metrics data is reported. The default is localhost:7676
-int interval
Specifies the interval (in seconds) at which to display the metrics. The default is 5 seconds.
-m metricType
Specifies the type of metric to display:
ttl Displays metrics on messages and packets flowing into and out of the broker, service, or destination (default metric type)
rts Displays metrics on rate of flow of messages and packets into and out of the broker, connection service, or destination (per second)
cxn Displays connections, virtual memory heap, and threads (brokers and connection services only)
con Displays consumer-related metrics (destinations only)
dsk Displays disk usage metrics (destinations only)
-msp numSamples
Specifies the number of samples displayed in the output. The default is an unlimited number (infinite).
-n destName
Specifies the name of the physical destination (if any) for which metrics data is reported. There is no default.
-n serviceName
Specifies the connection service (if any) for which metrics data is reported. There is no default.
-t destTyp
Specifies the type (queue or topic) of the physical destination (if any) for which metrics data is reported. There is no default.
Using the metrics Subcommand to Display Metrics Data
This section describes the procedure for using the metrics subcommand to report metrics information.
To Use the metrics Subcommand
- Start the broker for which metrics information is desired.
See Starting Brokers Interactively.
- Issue the appropriate imqcmd metrics subcommand and options as shown in Table 10-4 and Table 10-5.
Metrics Outputs: imqcmd metrics
This section contains examples of output for the imqcmd metrics s subcommand. The examples show broker-wide, connection service, and physical destination metrics.
Broker-wide Metrics.
To get the rate of message and packet flow into and out of the broker at 10 second intervals, use the metrics bkr subcommand:
imqcmd metrics bkr -m rts -int 10 -u admin
This command produces output similar to the following (see data descriptions in Table 18-2):
--------------------------------------------------------
Msgs/sec Msg Bytes/sec Pkts/sec Pkt Bytes/sec
In Out In Out In Out In Out
--------------------------------------------------------
0 0 27 56 0 0 38 66
10 0 7365 56 10 10 7457 1132
0 0 27 56 0 0 38 73
0 10 27 7402 10 20 1400 8459
0 0 27 56 0 0 38 73
Connection Service Metrics.
To get cumulative totals for messages and packets handled by the jms connection service, use the metrics svc subcommand:
imqcmd metrics svc -n jms -m ttl -u admin
This command produces output similar to the following (see data descriptions in Table 18-3):
-------------------------------------------------
Msgs Msg Bytes Pkts Pkt Bytes
In Out In Out In Out In Out
-------------------------------------------------
164 100 120704 73600 282 383 135967 102127
657 100 483552 73600 775 876 498815 149948
Physical Destination Metrics
To get metrics information about a physical destination, use the metrics dst subcommand:
imqcmd metrics dst -t q -n XQueue -m ttl -u admin
This command produces output similar to the following (see data descriptions in Table 18-4):
-----------------------------------------------------------------------------
Msgs Msg Bytes Msg Count Total Msg Bytes (k) Largest
In Out In Out Current Peak Avg Current Peak Avg Msg (k)
-----------------------------------------------------------------------------
200 200 147200 147200 0 200 0 0 143 71 0
300 200 220800 147200 100 200 10 71 143 64 0
300 300 220800 220800 0 200 0 0 143 59 0
To get information about a physical destination’s consumers, use the following metrics dst subcommand:
imqcmd metrics dst -t q -n SimpleQueue -m con -u admin
This command produces output similar to the following (see data descriptions in Table 18-4):
------------------------------------------------------------------
Active Consumers Backup Consumers Msg Count
Current Peak Avg Current Peak Avg Current Peak Avg
------------------------------------------------------------------
1 1 0 0 0 0 944 1000 525
imqcmd query
The syntax and options of imqcmd query are shown in Table 10-6 along with a description of the metrics data provided by the command.
Table 10-6 imqcmd query Subcommand Syntax
Subcommand Syntax
Metrics Data Provided
query bkr
[-b hostName:port]Information on the current number of messages and message bytes stored in broker memory and persistent store (see Displaying Broker Information)
or
query svc -n serviceName
[-b hostName:port]Information on the current number of allocated threads and number of connections for a specified connection service (see Displaying Connection Service Information)
or
query dst -t destType
-n destName
[-b hostName:port]Information on the current number of producers, active and backup consumers, and messages and message bytes stored in memory and persistent store for a specified destination (see Displaying Information about Physical Destinations)
Note
Because of the limited metrics data provided by imqcmd query, this tool is not represented in the tables presented in Chapter 18, "Metrics Reference".
Writing an Application to Monitor BrokersMessage Queue provides a metrics monitoring capability by which the broker can write metrics data into JMS messages, which it then sends to one of a number of metrics topic destinations, depending on the type of metrics information contained in the message.
You can access this metrics information by writing a client application that subscribes to the metrics topic destinations, consumes the messages in these destinations, and processes the metrics information contained in the messages.
There are five metrics topic destinations, whose names are shown in Table 10-7, along with the type of metrics messages delivered to each destination.
Table 10-7 Metrics Topic Destinations
Topic Name
Type of Metrics Messages
mq.metrics.broker
Broker metrics
mq.metrics.jvm
Java Virtual Machine metrics
mq.metrics.destination_list
List of destinations and their types
mq.metrics.destination.queue.
monitoredDestinationNameDestination metrics for queue of specified name
mq.metrics.destination.topic.
monitoredDestinationNameDestination metrics for topic of specified name
Setting Up Message-Based Monitoring
This section describes the procedure for using the message-based monitoring capability to gather metrics information. The procedure includes both client development and administration tasks.
To Set Up Message-based Monitoring
- Write a metrics monitoring client.
See the Message Queue Developer’s Guide for Java Clients for instructions on programming clients that subscribe to metrics topic destinations, consume metrics messages, and extract the metrics data from these messages.
- Configure the broker’s Metrics Message Producer by setting broker property values in the config.properties file:
- Enable metrics message production.
Set imq.metrics.topic.enabled=true
The default value is true.
- Set the interval (in seconds) at which metrics messages are generated.
Set imq.metrics.topic.interval=interval
The default is 60 seconds.
- Specify whether you want metrics messages to be persistent (that is, whether they will survive a broker failure).
Set imq.metrics.topic.persist
The default is false.
- Specify how long you want metrics messages to remain in their respective destinations before being deleted.
Set imq.metrics.topic.timetolive
The default value is 300 seconds
- Set any access control you desire on metrics topic destinations.
See the discussion in Security and Access Considerations, below.
- Start up your metrics monitoring client.
When consumers subscribe to a metrics topic, the metrics topic destination will automatically be created. Once a metrics topic has been created, the broker’s metrics message producer will begin sending metrics messages to the metrics topic.
Security and Access Considerations
There are two reasons to restrict access to metrics topic destinations:
Because of these considerations, it is advisable to restrict access to metrics topic destinations.
Monitoring clients are subject to the same authentication and authorization control as any other client. Only users maintained in the Message Queue user repository are allowed to connect to the broker.
You can provide additional protections by restricting access to specific metrics topic destinations through an access control properties file, as described in Authorizing Users: the Access Control Properties File.
For example, the following entries in an accesscontrol.properties file will deny access to the mq.metrics.broker metrics topic to everyone except user1 and user 2.
The following entries will only allow users user3 to monitor topic t1.
topic.mq.metrics.destination.topic.t1.consume.deny.user=*
topic.mq.metrics.destination.topic.t1.consume.allow.user=user3
Depending on the sensitivity of metrics data, you can also connect your metrics monitoring client to a broker using an encrypted connection. For information on using encrypted connections, see Working With an SSL-Based Service.
Metrics Outputs: Metrics Messages
The metrics data outputs you get using the message-based monitoring API is a function of the metrics monitoring client you write. You are limited only by the data provided by the metrics generator in the broker. For a complete list of this data, see Metrics Reference.