Sun Java System Message Queue 3.5 SP1 Java Client Developer's Guide |
Chapter 5
Message Queue Client Programming TechniquesSome features and capabilities of Message Queue go beyond the JMS specification. If you want to write client applications that leverage the power of these features (which are specific to Message Queue), use the techniques described here. The chapter provides programming guidelines and examples for developing clients that make use of the following Message Queue service features:
Custom Client AcknowledgementAs discussed in Message Consumption, Message Queue supports several JMS acknowledgement modes. These modes let message consumers in a session acknowledge the messages they have consumed. The different modes affect the performance and reliability of message delivery. For more flexibility, Message Queue lets you customize the JMS CLIENT_ACKNOWLEDGE mode.
In CLIENT_ACKNOWLEDGE mode, the client explicitly acknowledges message consumption by invoking the acknowledge() method of a message object. The standard behavior of this method is to cause the session to acknowledge all messages that have been consumed by any consumer in the session since the last time the method was invoked. (That is, the session acknowledges the current message and all previously unacknowledged messages, regardless of who consumed them.)
In addition to the standard behavior specified by JMS, Message Queue lets you use the CLIENT_ACKNOWLEDGE mode to acknowledge one individual message at a time.
Observe the following rules when implementing custom client acknowledgement:
- When you code an acknowledgement of an individual message, call the acknowledgeThisMessage() method. When you code an acknowledgement of all messages consumed so far, call the acknowledgeUpThroughThisMessage() method. Both are shown in Code Example 5-1.
- When you compile the resulting code, include both imq.jar and jms.jar in the classpath.
- Don’t call acknowledge(), acknowledgeThisMessage(), or acknowledgeUpThroughThisMessage() in any session except one that uses the CLIENT_ACKNOWLEDGE mode. Otherwise, the method call is ignored.
- Don’t try to mix custom-acknowledgement sessions and transacted sessions. A transacted session defines a specific way to have messages acknowledged.
If a broker fails, any message that was not acknowledged successfully (that is, any message whose acknowledgement ended in a JMSException) is held by the broker for delivery to subsequent clients.
Code Example 5-2 demonstrates both types of custom client acknowledgement.
Code Example 5-2 Example of Custom Client Acknowledgement Code
...
import javax.jms.*;
... [Look up a connection factory and create a connection.]
Session session = connection.createSession(false,
Session.CLIENT_ACKNOWLEDGE);
... [Create a consumer and receive messages.]
Message message1 = consumer.receive();
Message message2 = consumer.receive();
Message message3 = consumer.receive();
... [Process messages.]
... [Acknowledge one individual message.
Notice that the following acknowledges only message 2.]
((com.sun.messaging.jms.Message)message2).acknowledgeThisMessage();
... [Continue. Receive and process more messages.]
Message message4 = consumer.receive();
Message message5 = consumer.receive();
Message message6 = consumer.receive();
... [Acknowledge all messages up through message 4. Notice that this
acknowledges messages 1, 3, and 4, because message 2 was acknowledged
earlier.]
((com.sun.messaging.jms.Message)message4).
acknowledgeUpThroughThisMessage();
... [Continue. Finally, acknowledge all messages consumed in the session.
Notice that this acknowledges all remaining consumed messages, that is,
messages 5 and 6, because this is the standard behavior of the JMS API.]
message5.acknowledge();
Message-Based Monitoring APIBy using the Message Queue metrics monitoring capability, a broker can write metrics data into messages which the broker then sends to one of a number of metrics topic destinations. The destination depends on the type of metrics data in a given message. You get access to this metrics data when you write a client application that does three things:
The message-based monitoring API and other metrics monitoring tools are described in the Message Queue Administration Guide.
Table 5-1 shows the five metrics topic destinations and the type of metrics messages each destination can receive.
Table 5-1 Metrics Topic Destinations
Topic Destination Name
Type of Metrics Messages
mq.metrics.broker
Broker metrics: information on connections, message flow, and volume of messages in the broker
mq.metrics.jvm
Java Virtual Machine metrics: information on memory usage in the JVM
mq.metrics.destination_list
A list of all destinations on the broker, and their types
mq.metrics.destination.queue.
monitored_destination_nameDestination metrics for a queue of the specified name, such as number of consumers, message flow or volume, or disk usage
mq.metrics.destination.topic.
monitored_destination_nameSimilar destination metrics for a topic of the specified name
For an example of how these metrics messages can be useful, when a particular limit has been reached, you might want to program in an alert and a response action (such as sending mail to the administrator).
As explained in the Message Queue Administration Guide (look up “metrics messages” and “configuration files”), you can do the same thing manually by using the metrics command utility. However, if that manual approach isn’t appropriate for your purposes, you can write your JMS client so that it automatically consumes metrics messages and displays output in a convenient format.
Monitoring topics have destination names beginning with mq. (Always include the period.) These names are reserved for use by Message Queue.
No hierarchical naming scheme is implied in the message-based monitoring API. You can’t use a wildcard character (*) to identify multiple destination names.
When a metrics subscriber is detected, the broker automatically creates the metrics topic. A metrics monitoring topic can’t be created using an administrative command. Only the broker can publish messages to a metrics monitoring topic.
You specify how often to receive metrics information by configuring a property in the broker’s config.properties file. All the destinations receiving metrics on that broker receive them at that same specified interval. (For information on how to set that interval, refer to the Message Queue Administration Guide.)
This API is designed for monitoring the broker. It’s not designed for doing administrative tasks on the broker such as:
For information on how to use Message Queue administration tools to do those tasks, refer to the Message Queue Administration Guide.
Format of Metrics Messages
Subscribers to metrics topics receive JMS messages of type MapMessage. (See Message Body Types for details.) The header of a metrics message contains two properties: type and timestamp. The type property is useful if the same subscriber processes more than one type of metrics message—for example, messages from topics mq.metrics.broker and mq.metrics.jvm. The timestamp property is useful for calculating rates or drawing graphs.
The body of the message contains name-value pairs, and the data depends on the type of metrics message. The format of each metrics message type is explained in the following tables.
Notice these points:
- The names used for extracting data are case-sensitive. For example:
- Each metrics message type has a defined set of name-value pairs. A name that is specific to a particular message type can be used only with that type. For example, the name freeMemory can’t be used with a message received from the topic mq.metrics.broker; it can be used only with a message received from the topic mq.metrics.jvm.
Broker Metrics
The messages you receive when you subscribe to the topic mq.metrics.broker have the following message properties (Table 5-2) and metrics data in the message body (Table 5-3).
Table 5-2 Broker Metrics Message Properties
Property
Type
Value or Description
type
String
mq.metrics.broker
timestamp
long
Timestamp in milliseconds when metric sample was taken
Table 5-3 Data in the Body of a Broker Metrics Message
Metric Name
Value Type
Description
numConnections
long
Current number of connections to the broker
numMsgsIn
long
Number of JMS messages that have flowed into the broker since it was last started
numMsgsOut
long
Number of JMS messages that have flowed out of the broker since it was last started
numMsgs
long
Current number of JMS messages stored in broker memory and persistent store
msgBytesIn
long
Number of JMS message bytes that have flowed into the broker since it was last started
msgBytesOut
long
Number of JMS message bytes that have flowed out of the broker since it was last started
totalMsgBytes
long
Current number of JMS message bytes stored in broker memory and persistent store
numPktsIn
long
Number of packets that have flowed into the broker since it was last started; this includes both JMS messages and control messages
numPktsOut
long
Number of packets that have flowed out of the broker since it was last started; this includes both JMS messages and control messages
pktBytesIn
long
Number of packet bytes that have flowed into the broker since it was last started; this includes both JMS messages and control messages
pktBytesOut
long
Number of packet bytes that have flowed out of the broker since it was last started; this includes both JMS messages and control messages
numDestinations
long
Current number of destinations in the broker
JVM Metrics
The messages you receive when you subscribe to the topic mq.metrics.jvm have the following message properties (Table 5-4) and metrics data in the message body (Table 5-5):
Table 5-4 JVM Metrics Message Properties
Property
Type
Value or Description
type
String
mq.metrics.jvm
timestamp
long
Timestamp in milliseconds when the metric sample was taken
Table 5-5 Data in the Body of a JVM Metrics Message
Metric Name
Value Type
Description
freeMemory
long
Amount of free memory available for use in the JVM heap
maxMemory
long
Maximum size to which the JVM heap can grow
totalMemory
long
Total memory in the JVM heap
Destination-List Metrics
The messages you receive when you subscribe to a topic named mq.metrics.destination_list have the following properties (Table 5-6):
Table 5-6 Destination-List Message Properties
Property
Type
Value or Description
type
String
mq.metrics.destination_list
timestamp
long
Timestamp in milliseconds when the metric sample was taken
Each destination in the broker has a corresponding, unique map name (a name-value pair) in the message body. The name depends on whether the destination is a queue or a topic. The type of the name-value pair is hashtable.
Each hashtable in the message contains information about a specific destination on the broker. The sub-table within Table 5-7 describes the key-value pairs that can be used to extract this information.
By enumerating through the map names and extracting the hashtable described in Table 5-7, you can form a complete list of destination names and some of their characteristics.
The destination list does not include the following:
The message body contains name-value pairs as follows:
Table 5-7 Data in the Body of a Destination-List Metrics Message
Metric Name
Value Type
Value or Description
One of the following:
hashtable
The corresponding value for the map name is an object of type java.util.Hashtable. This hashtable contains the following key-value pairs.
Key (String)
Value Type
Value or Description
name
String
Destination name.
type
String
Destination type. The value is either queue or topic.
isTemporary
Boolean
Whether the destination is temporary (true) or not (false).
Notice that the destination name and type could be extracted directly from the metrics topic destination name, but the hashtable includes it for your convenience.
Destination Metrics
The messages you receive when you subscribe to the topic mq.metrics.destination.queue.monitored_destination_name or the topic mq.metrics.destination.topic.monitored_destination_name have the following message properties (Table 5-8) and metrics data in the message body (Table 5-9):
Table 5-8 Destination Metrics Message Properties
Property
Type
Value or Description
type
String
mq.metrics.destination.queue.monitored_destination_name
or
mq.metrics.destination.topic.monitored_destination_nametimestamp
long
Timestamp in milliseconds when the metric sample was taken
Table 5-9 Data in the Body of a Destination Metrics Message
Metric Name
Value Type
Description
numActiveConsumers
long
Current number of active consumers
avgNumActiveConsumers
long
Average number of active consumers since the broker was last started
peakNumActiveConsumers
long
Peak number of active consumers since the broker was last started
numBackupConsumers
long
Current number of backup consumers (applies only to queues)
avgNumBackupConsumers
long
Average number of backup consumers since the broker was last started (applies only to queues)
peakNumBackupConsumers
long
Peak number of backup consumers since the broker was last started (applies only to queues)
numMsgsIn
long
Number of JMS messages that have flowed into this destination since the broker was last started
numMsgsOut
long
Number of JMS messages that have flowed out of this destination since the broker was last started
numMsgs
long
Number of JMS messages currently stored in destination memory and persistent store
avgNumMsgs
long
Average number of JMS messages stored in destination memory and persistent store since the broker was last started
peakNumMsgs
long
Peak number of JMS messages stored in destination memory and persistent store since the broker was last started
msgBytesIn
long
Number of JMS message bytes that have flowed into this destination since the broker was last started
msgBytesOut
long
Number of JMS message bytes that have flowed out of this destination since the broker was last started
totalMsgBytes
long
Current number of JMS message bytes stored in destination memory and persistent store
avgTotalMsgBytes
long
Average number of JMS message bytes stored in destination memory and persistent store since the broker was last started
peakTotalMsgBytes
long
Peak number of JMS message bytes stored in destination memory and persistent store since the broker was last started
peakMsgBytes
long
Peak number of JMS message bytes in a single message since the broker was last started
diskReserved
long
Disk space (in bytes) used by all message records (active and free) in the destination file-based store
diskUsed
long
Disk space (in bytes) used by active message records in destination file-based store
diskUtilizationRatio
int
Quotient of used disk space over reserved disk space. The higher the ratio, the more the disk space is being used to hold active messages
Configuring Metrics Message Production on the Broker
When you use Message Queue, metrics message production is enabled by default for your client application. However, the Message Queue administrator must use the broker properties to set the reporting interval (to specify how often metrics updates are reported), and to specify whether metrics messages are persistent and how long they are to “live” in their destinations.
For details on configuring broker properties, refer to the Message Queue Administration Guide.
Also, without certain security features that Message Queue 3.5 SP1 provides, someone could obtain and misuse sensitive information about a broker and its resources. An administrator should take the approach described under “metrics monitoring tools” in the Message Queue Administration Guide to provide the proper access control to metrics topic destinations.
Using the Message-Based Monitoring API
You use the message-based monitoring API in the same way that you would write any JMS client, except that you subscribe to a special topic, you receive messages of a specific type and format, and you process the messages in a particular way.
A client that uses the message-based monitoring API to monitor broker metrics must perform the following basic tasks:
- Create or look up a TopicConnectionFactory object
- Create a TopicConnection to the Message Queue service
- Create a TopicSession
- Create a metrics Topic destination object
- Create a TopicSubscriber
- Register as an asynchronous listener to the topic, or invoke the synchronous receive() method to wait for incoming metrics messages
- Process metrics messages that are received
In general, you would use JNDI lookups of administered objects to make your client code provider-independent. However, the message-based monitoring API is specific to Message Queue, so there is no compelling reason to use JNDI lookups. You can simply instantiate these administered objects directly in your client code. This is especially true for a metrics destination for which an administrator would not normally create an administered object.
Notice that the code examples in this section instantiate all the relevant administered objects directly.
You can use the following code to extract the type (String) or timestamp (long) properties in the message header from the message:
MapMessage mapMsg;
/*
* mapMsg is the metrics message received
*/
String type = mapMsg.getStringProperty("type");
long timestamp = mapMsg.getLongProperty("timestamp");
You use the appropriate getter method in the class javax.jms.MapMessage to extract the name-value pairs. The getter method depends on the value type. Three examples follow:
long value1 = mapMsg.getLong("numMsgsIn");
long value2 = mapMsg.getLong("numMsgsOut");
int value3 = mapMsg.getInt("diskUtilizationRatio");
Metrics Monitoring Client Code Examples
Several complete monitoring example applications (including source code and full documentation) are provided when you install Message Queue. You’ll find the examples in your IMQ home directory under /demo/monitoring. Before you can run these clients, you must set up your environment (for example, the CLASSPATH environment variable). For details, see Chapter 2, "Quick Start Tutorial."
Next are brief descriptions of three examples—Broker Metrics, Destination List Metrics, and Destination Metrics—with annotated code examples from each.
These examples use the utility classes MetricsPrinter and MultiColumnPrinter to print formatted and aligned columns of text output. However, rather than explaining how those utility classes are used, the following code examples focus on how to subscribe to the metrics topic and how to extract information from the metrics messages received.
Notice that in the source files, the code for subscribing to metrics topics and processing messages is actually spread across various methods. However, for the sake of clarity, the examples are shown here as though they were contiguous blocks of code.
A Broker Metrics Example
The source file for this code example is BrokerMetrics.java. This metrics monitoring client subscribes to the topic mq.metrics.broker and prints broker-related metrics to the standard output.
Code Example 5-3 shows how to subscribe to mq.metrics.broker.
Code Example 5-3 Example of Subscribing to a Broker Metrics Topic
com.sun.messaging.TopicConnectionFactory metricConnectionFactory;
TopicConnection metricConnection;
TopicSession metricSession;
TopicSubscriber metricSubscriber;
Topic metricTopic;
metricConnectionFactory = new com.sun.messaging.TopicConnectionFactory();
metricConnection = metricConnectionFactory.createTopicConnection();
metricConnection.start();
metricSession = metricConnection.createTopicSession(false,
Session.AUTO_ACKNOWLEDGE);
metricTopic = metricSession.createTopic("mq.metrics.broker");
metricSubscriber = metricSession.createSubscriber(metricTopic);
metricSubscriber.setMessageListener(this);
The incoming message is processed in the onMessage() and doTotals() methods, as shown in Code Example 5-4.
Code Example 5-4 Example of Processing a Broker Metrics Message
public void onMessage(Message m) {
try {
MapMessage mapMsg = (MapMessage)m;
String type = mapMsg.getStringProperty("type");
if (type.equals("mq.metrics.broker")) {
if (showTotals) {
doTotals(mapMsg);
...
}
}
private void doTotals(MapMessage mapMsg) {
try {
String oneRow[] = new String[ 8 ];
int i = 0;
/*
* Extract broker metrics
*/
oneRow[i++] = Long.toString(mapMsg.getLong("numMsgsIn"));
oneRow[i++] = Long.toString(mapMsg.getLong("numMsgsOut"));
oneRow[i++] = Long.toString(mapMsg.getLong("msgBytesIn"));
oneRow[i++] = Long.toString(mapMsg.getLong("msgBytesOut"));
oneRow[i++] = Long.toString(mapMsg.getLong("numPktsIn"));
oneRow[i++] = Long.toString(mapMsg.getLong("numPktsOut"));
oneRow[i++] = Long.toString(mapMsg.getLong("pktBytesIn"));
oneRow[i++] = Long.toString(mapMsg.getLong("pktBytesOut"));
...
} catch (Exception e) {
System.err.println("onMessage: Exception caught: " + e);
}
}
Notice how the metrics type is extracted, using the getStringProperty() method, and is checked. If you use the onMessage() method to process metrics messages of different types, you can use the type property to distinguish between different incoming metrics messages.
Also notice how various pieces of information on the broker are extracted, using the getLong() method of mapMsg.
Run this example monitoring client with the following command:
java BrokerMetrics
The output looks like the following:
----------------------------------------------------------------
Msgs
Msg Bytes
Pkts
Pkt Bytes
In
Out
In
Out
In
Out
In
Out
----------------------------------------------------------------
0
0
0
0
6
5
888
802
0
1
0
633
7
8
1004
1669
A Destination List Metrics Example
The source file for this code example is DestListMetrics.java. This client application monitors the list of destinations on a broker by subscribing to the topic mq.metrics.destination_list. The messages that arrive contain information describing the destinations that currently exist on the broker, such as destination name, destination type, and whether the destination is temporary.
Code Example 5-5 shows how to subscribe to mq.metrics.destination_list.
Code Example 5-5 Example of Subscribing to the Destination List Metrics Topic
com.sun.messaging.TopicConnectionFactory metricConnectionFactory;
TopicConnection metricConnection;
TopicSession metricSession;
TopicSubscriber metricSubscriber;
Topic metricTopic;
String metricTopicName = null;
metricConnectionFactory = new com.sun.messaging.TopicConnectionFactory();
metricConnection = metricConnectionFactory.createTopicConnection();
metricConnection.start();
metricSession = metricConnection.createTopicSession(false,
Session.AUTO_ACKNOWLEDGE);
metricTopicName = "mq.metrics.destination_list";
metricTopic = metricSession.createTopic(metricTopicName);
metricSubscriber = metricSession.createSubscriber(metricTopic);
metricSubscriber.setMessageListener(this);
The incoming message is processed in the onMessage() method, as shown in Code Example 5-6:
Code Example 5-6 Example of Processing a Destination List Metrics Message
public void onMessage(Message m) {
try {
MapMessage mapMsg = (MapMessage)m;
String type = mapMsg.getStringProperty("type");
if (type.equals(metricTopicName)) {
String oneRow[] = new String[ 3 ];
/*
* Extract metrics
*/
for (Enumeration e = mapMsg.getMapNames();
e.hasMoreElements();) {
String metricDestName = (String)e.nextElement();
Hashtable destValues =
(Hashtable)mapMsg.getObject(metricDestName);
int i = 0;
oneRow[i++] = (destValues.get("name")).toString();
oneRow[i++] = (destValues.get("type")).toString();
oneRow[i++] = (destValues.get("isTemporary")).toString();
mp.add(oneRow);
}
mp.print();
System.out.println("");
mp.clear();
} else {
System.err.println("Msg received:
not destination list metric type");
}
} catch (Exception e) {
System.err.println("onMessage: Exception caught: " + e);
}
}
Notice how the metrics type is extracted and checked, and how the list of destinations is extracted. By iterating through the map names in mapMsg and extracting the corresponding value (a hashtable), you can construct a list of all the destinations and their related information.
As discussed in Format of Metrics Messages, these map names are metrics topic names having one of two forms:
mq.metrics.destination.queue.monitored_destination_name
mq.metrics.destination.topic.monitored_destination_name
(The map names can also be used to monitor a destination, but that is not done in this particular example.)
Notice that from each extracted hashtable, the information on each destination is extracted using the keys name, type, and isTemporary. The extraction code from the previous code example is reiterated here for your convenience.
Code Example 5-7 Example of Extracting Destination Information From a Hashtable
String metricDestName = (String)e.nextElement();
Hashtable destValues = (Hashtable)mapMsg.getObject(metricDestName);
int i = 0;
oneRow[i++] = (destValues.get("name")).toString();
oneRow[i++] = (destValues.get("type")).toString();
oneRow[i++] = (destValues.get("isTemporary")).toString();
Run this example monitoring client with the following command:
java DestListMetrics
The output looks like the following:
---------------------------------------------------
Destination Name
Type
IsTemporary
---------------------------------------------------
SimpleQueue
queue
false
fooQueue
queue
false
topic1
topic
false
A Destination Metrics Example
The source file for this code example is DestMetrics.java. This client application monitors a specific destination on a broker. It accepts the destination type and name as parameters, and it constructs a metrics topic name of the form mq.metrics.destination.queue.monitored_destination_name or mq.metrics.destination.topic.monitored_destination_name.
Code Example 5-8 shows how to subscribe to the metrics topic for monitoring a specified destination.
Code Example 5-8 Example of Subscribing to a Destination Metrics Topic
com.sun.messaging.TopicConnectionFactory metricConnectionFactory;
TopicConnection metricConnection;
TopicSession metricSession;
TopicSubscriber metricSubscriber;
Topic metricTopic;
String metricTopicName = null;
String destName = null,
destType = null;
for (int i = 0; i < args.length; ++i) {
...
} else if (args[i].equals("-n")) {
destName = args[i+1];
} else if (args[i].equals("-t")) {
destType = args[i+1];
}
}
metricConnectionFactory = new com.sun.messaging.TopicConnectionFactory();
metricConnection = metricConnectionFactory.createTopicConnection();
metricConnection.start();
metricSession = metricConnection.createTopicSession(false,
Session.AUTO_ACKNOWLEDGE);
if (destType.equals("q")) {
metricTopicName = "mq.metrics.destination.queue." + destName;
} else {
metricTopicName = "mq.metrics.destination.topic." + destName;
}
metricTopic = metricSession.createTopic(metricTopicName);
metricSubscriber = metricSession.createSubscriber(metricTopic);
metricSubscriber.setMessageListener(this);
The incoming message is processed in the onMessage() method, as shown in Code Example 5-9:
Code Example 5-9 Example of Processing a Destination Metrics Message
public void onMessage(Message m) {
try {
MapMessage mapMsg = (MapMessage)m;
String type = mapMsg.getStringProperty("type");
if (type.equals(metricTopicName)) {
String oneRow[] = new String[ 11 ];
int i = 0;
/*
* Extract destination metrics
*/
oneRow[i++] = Long.toString(mapMsg.getLong("numMsgsIn"));
oneRow[i++] = Long.toString(mapMsg.getLong("numMsgsOut"));
oneRow[i++] = Long.toString(mapMsg.getLong("msgBytesIn"));
oneRow[i++] = Long.toString(mapMsg.getLong("msgBytesOut"));
oneRow[i++] = Long.toString(mapMsg.getLong("numMsgs"));
oneRow[i++] = Long.toString(mapMsg.getLong("peakNumMsgs"));
oneRow[i++] = Long.toString(mapMsg.getLong("avgNumMsgs"));
oneRow[i++] = Long.toString(mapMsg.getLong("totalMsgBytes")/1024);
oneRow[i++] =
Long.toString(mapMsg.getLong("peakTotalMsgBytes")/1024);
oneRow[i++] =
Long.toString(mapMsg.getLong("avgTotalMsgBytes")/1024);
oneRow[i++] = Long.toString(mapMsg.getLong("peakMsgBytes")/1024);
mp.add(oneRow);
...
}
} catch (Exception e) {
System.err.println("onMessage: Exception caught: " + e);
}
}
Notice how the metrics type is extracted, using the getStringProperty() method as in the previous examples, and is checked. Also notice how various destination data are extracted, using the getLong() method of mapMsg.
Run this example monitoring client with one of the following commands:
java DestMetrics -t t -n topic_name
java DestMetrics -t q -n queue_name
Using a queue named SimpleQueue as an example, the command would be:
java DestMetrics -t q -n SimpleQueue
The output looks like the following:
-------------------------------------------------------------------------------------------
Msgs
Msg Bytes
Msg Count
Tot Msg Bytes (k)
Largest Msg
In Out
In Out
Curr Peak Avg
Curr Peak Avg
(k)
-------------------------------------------------------------------------------------------
500 0
318000 0
500 500 250
310 310 155
0
Client Connection Failover (Auto-reconnect)Message Queue supports client connection failover. A failed connection can be restored not only on the original broker, but also on a different broker; that is, it can reconnect to the message service rather than to a specific broker instance. This reconnection does not apply in situations where the client-side state could not be fully restored on the broker upon reconnect (for example, when using transacted sessions or temporary destinations, which exist only for the duration of a connection).
Enabling Auto-reconnect
To enable this auto-reconnect behavior, you configure the connection factory imqReconnectEnabled attribute to true. You also configure the connection factory administered object to specify the following:
- A list of message-service addresses (using the imqAddressList attribute). When the client runtime needs to establish or re-establish a connection to a message service, it attempts to connect to the brokers in the list until it finds (or fails to find) an available broker. If you specify only a single broker instance on the imqAddressList attribute, the configuration won’t support recovery from hardware failure.
When you specify more than one broker in the list, consider whether to use parallel brokers or a broker cluster. In a parallel configuration, there is no communication between brokers, while in a broker cluster, the brokers interact to distribute message delivery loads. (Refer to the Message Queue Administration Guide for more information on broker clusters.)
- To enable parallel-broker reconnection, set the imqReconnectListBehavior attribute to PRIORITY. Typically, you would specify no more than a pair of brokers for this type of reconnection. This way, the messages are published to one broker, and all clients fail over together from the first broker to the second.
- To enable clustered-broker reconnection, set the imqReconnectListBehavior attribute to RANDOM. This way, the client runtime randomizes connection attempts across the list, and client connections are distributed evenly across the broker cluster.
Each broker in a cluster uses its own separate persistent store (which means that any undelivered persistent messages are unavailable until a failed broker is back online). If one broker crashes, its client connections are re-established on other brokers.
- The number of iterations to be made over the list of brokers when attempting to create a connection or to reconnect, using the imqAddressListIterations attribute.
Notice that the value 5 means “Try five times” and the value -1 means “Don’t stop trying.”
- The number of attempts to be made to reconnect to a broker if the first connection fails, using the imqReconnectAttempts attribute.
- The interval, in milliseconds, between reconnect attempts, using the imqReconnectInterval attribute.
Auto-reconnect Behaviors
Notice that a broker treats an automatic reconnection as it would a new connection. When an original connection is lost, all the resources associated with that connection are released. For example, in a broker cluster, as soon as one broker fails, the other brokers assume that the client connections associated with the failed broker are gone. After auto-reconnect takes place, the client connections are re-created from scratch.
Sometimes the client-side state cannot be fully restored by auto-reconnect, and the connection exception handler is called. Perhaps a resource that the client needs cannot be re-created. In this case, your client receives a JMSException, and must reconnect and restore state.
If the client is being auto-reconnected explicitly to a broker instance that is different from the original, then persistent messages and other state information held by the failed or disconnected broker can be lost. The messages held by the original broker, once it is restored, might be delivered out of order. The reason is that the various broker instances in a cluster do not use a shared, highly available persistent store.
A transacted session is the most reliable method of ensuring that a message isn’t lost, if you are careful in coding the transaction. If auto-reconnect happens in the middle of a transaction, then the broker loses the information, the client runtime throws an exception when the transaction is committed, and the transaction is rolled back. Therefore, at that point, make sure that the client restarts the whole transaction. (This is especially important when you use a broker cluster.)
When auto-reconnect happens in a CLIENT_ACKNOWLEDGE session, the client runtime throws a JMSException and the acknowledgement of any set of messages must be rolled back. Therefore, if you get a JMSException message in such a session, call session.recover.
Auto-reconnect Limitations
Notice the following points when using the auto-reconnect feature:
- Messages might be redelivered to a consumer after auto-reconnect takes place. In an AUTO_ACKNOWLEDGE session, you will get no more than one redelivered message. In the other session types, you might get more than one.
- While the client runtime is trying to reconnect, any messages sent by the broker to non-durable topic consumers are lost.
- Any messages that are in queue destinations and that are unacknowledged when a connection fails are redelivered after auto-reconnect. However, in the case of queues delivering to multiple consumers, these messages cannot be guaranteed to be redelivered to the original consumers. That is, as soon as a connection fails, an unacknowledged queue message might be rerouted to other connected consumers.
- In the case of a broker cluster, the failure of the master broker has more implications than the failure of other brokers in the cluster. While the master broker is down, the following operations on any other broker do not succeed:
- Creating or destroying a new durable subscription.
- Creating or destroying a new physical destination using the imqcmd create dst command.
- Starting a new broker process. (However, the brokers that are already running continue to function normally even if the master broker goes down.)
You can configure the master broker to restart automatically using Message Queue broker support for rc scripts or the Windows service manager.
- Auto-reconnect doesn’t work if the client uses a ConnectionConsumer to consume messages. In that case, the client runtime throws an exception.
Auto-reconnect Configuration Examples
Next are examples that illustrate how to enable each type of auto-reconnect support.
Single-Broker Auto-reconnect
Configure your connection-factory object as follows:
Code Example 5-10 Example of Command to Configure a Single Broker
imqobjmgr add -t cf -l "cn=myConnectionFactory" \
-o “imqAddressList=mq://jpgserv/jms” \
-o “imqReconnect=true” \
-o “imqReconnectAttempts=10”
This command creates a connection-factory object with a single address in the broker address list. If connection fails, the client runtime will try to reconnect with the broker 10 times. If an attempt to reconnect fails, the client runtime will sleep for three seconds (the default value for the imqReconnectInterval attribute) before trying again. After 10 unsuccessful attempts, the application will receive a JMSException.
Note that you can ensure that the broker starts automatically with the machine at system start-up time. See the Message Queue Installation Guide for information on how to configure automatic broker start-up. For example, on the Solaris platform, you can use /etc/rc.d scripts.
Parallel Broker Auto-reconnect
Configure your connection-factory objects as follows:
Code Example 5-11 Example of Command to Configure Parallel Brokers
imqobjmgr add -t cf -l "cn=myCF" \
-o "imqAddressList=myhost1, mqtcp://myhost2:12345/jms" \
-o "imqReconnect=true" \
-o "imqReconnectRetries=5"
This command creates an connection factory object with two addresses in the broker list. The first address describes a broker instance running on the host myhost1 with a standard port number (7676). The second address describes a jms connection service running at a statically configured port number (12345).
Clustered-Broker Auto-reconnect
Configure your connection-factory objects as follows:
Code Example 5-12 Example of Command to Configure a Broker Cluster
imqobjmgr add -t cf -l "cn=myConnectionFactory" \
-o "imqAddressList=mq://myhost1/ssljms, \
mq://myhost2/ssljms, \
mq://myhost3/ssljms, \
mq://myhost4/ssljms” \
-o "imqReconnect=true" \
-o "imqReconnectRetries=5" \
-o "imqAddressListBehavior=RANDOM"
This command creates a connection factory object with four addresses in the imqAddressList. All the addresses point to jms services running on SSL transport on different hosts. Since the imqAddressListBehavior attribute is set to RANDOM, the client connections that are established using this connection factory object will be distributed randomly among the four brokers in the address list.
This is a clustered broker configuration, so you must configure one of the brokers in the cluster as the master broker. In the connection-factory address list, you can also specify a subset of all the brokers in the cluster.
Other Programming TopicsThe rest of this chapter discusses the following miscellaneous topics:
Managing Memory and Message Size
A client application running in a JVM needs enough memory to accommodate messages that flow in from the network as well as messages the client creates. If your client encounters OutOfMemoryError errors, chances are that not enough memory was provided to handle the size or the number of messages being consumed or produced.
The default JVM heap space is 64 meg, but your client might need more than that.
Consider the following guidelines:
- Evaluate the normal and peak system memory footprints when sizing heap space.
- You can start by doubling the heap size, as in the following command:
java -Xmx128m MyClass
- The ideal size for the heap space depends on both the operating system and the JDK release. Check the JDK documentation for restrictions.
- The size of the VM’s memory allocation pool must be less than or equal to the amount of virtual memory that is available on the system.
For better manageability, break large messages into smaller parts, and use sequencing to ensure that the partial messages are concatenated properly.
Other methods of dealing with memory issues are explained in Chapter 4, "Configuring the Message Queue Client Runtime." They include metering the message flow over the client-broker connection and limiting the per-consumer message flow.
Using Secure HTTP Connections (HTTPS)
If you run your client applications in an environment secured by a firewall, you might need to have client applications communicate with brokers using the HTTP or HTTPS protocol rather than direct TCP connections. Web-based connections are usually allowed through firewalls.
The client runtime uses a transport driver and an HTTP proxy to send messages to the firewall. A tunnel servlet on the web server reaches through the firewall, pulls messages from the client’s HTTP requests, and sends the messages to the broker.
Refer to the Message Queue Administration Guide for details on how to implement Message Queue support of HTTP and HTTPS in your JMS applications.
In Case of Server or Broker Failure
If the web server fails and is restarted, all connections are restored and there is no effect on clients. However, if the broker fails and is restarted, an exception is thrown and clients must re-establish their connections.
If both the web server and the broker fail, and the broker is not restarted, the web server restores client connections and continues waiting for a broker connection without notifying clients. To avoid this situation, always make sure the broker is restarted.
Repairing an HTTPS Tunnel Servlet Connection
If an HTTPS client can’t connect to the broker through the tunnel servlet, do the following:
When the service is resumed, your HTTPS client should be able to connect to the broker through the tunnel servlet.
Managing Client Threads
Managing threads in a JMS application often involves trade-offs. Weigh the following considerations when dealing with threading issues.
When the Message Queue client runtime creates a connection, two threads are created: one for consuming messages, and one to distribute and control flow for the connection. In addition, each JMS session creates a thread to deliver messages to message consumers. Thus, for example:
When you create several message consumers in the same session, messages are delivered serially by the session thread to these consumers. Sharing a session among several message consumers might starve some consumers of message flow while inundating other consumers. So, if the message rate across these consumers is high enough to cause an imbalance, you might want to separate the consumers into different sessions.
Note that the JMS specification restricts a session for use by a single thread at a time. Violating this restriction can result in a deadlocked client.
You can reduce the number of threads by using fewer connections and fewer sessions. However, doing this might slow your application’s throughput.
Finally, you might be able to use certain JVM runtime options to improve thread memory usage and performance. Refer to the JDK documentation for details.
For example, if you are running on the Solaris platform, you may be able to run with the same number (or more) threads by using the following vm options with the client:
Option
Result
Xss128K
This decreases the memory size of the heap.
xconcurrentIO
This improves thread performance in the 1.3 VM.
Try to observe the following “golden rules” of thread management in your JMS applications:
Synchronous Consumption in Distributed Applications
Because distributed applications involve greater processing time, such an application might not behave as expected if it were run locally. For example, calling the receiveNoWait method for a synchronous consumer might return null even when there is a message available to be retrieved.
If a client connects to the broker and immediately calls the receiveNoWait method, it is possible that the message queued for the consuming client is in the process of being transmitted from the broker to the client. The client runtime has no knowledge of what is on the broker, so when it sees that there is no message available on the client’s internal queue, it exits with a null.
You can avoid this problem by having your client do either of the following:
Client Application Deployment Considerations
When you are ready to move your client application into production, you should make sure the administrator knows what the application requires. You can start with the following checklist if you like, but tailor it to your environment and your administrator’s needs.
Table 5-10 Starter Checklist for the Message Queue Administrator
Configuring administered objects:
Connection factories to be created
Type:
JNDI lookup name:
Physical name (if your administrator wants it):
Other attributes:
Destination objects to be created
Type (queue or topic):
JNDI lookup name:
Physical name (if your administrator wants it):
Other attributes:Configuring a broker or broker cluster:
Name:
Number of destinations:
Maximum number of messages expected:
Maximum size of messages expected:
Maximum message bytes expected:
Access control and other security requirements:
For broker cluster:
Load-balancing requirements:
Geographic distribution:
Auto-reconnect implementation model, if any:Configuring physical destinations:
Type:
Name:
Attributes:
Maximum number of messages expected:
Maximum size of messages expected:
Maximum message bytes expected:
In regard to geographic distribution, notice that clients can be grouped in different areas to minimize traffic over long links.
For details on configuration, refer to the Message Queue Administration Guide.