Sun Java System Message Queue 3 2005Q1 Developer's Guide for C Clients |
Chapter 3
Client Design IssuesThis chapter describes a number of messaging issues that impact Message Queue C client design. It covers the following topics:
This chapter does not discuss the particulars of the C-API and how to use the data types and functions it defines to create messaging clients. For this information, see Chapter 2, "Using the C API".
Producers and ConsumersAside from the reliability your client requires, the design decisions that relate to producers and consumers include the following:
There are some interesting permutations here. There are times when you would want to use publish/subscribe even when you have only one subscriber. Performance considerations might make the point-to-point model more efficient than the publish/subscribe model, when the work of sorting messages between subscribers is too costly. Sometimes these decisions cannot be made in the abstract, but different prototypes must be developed and tested.
You might need to adjust the MQ_PING_INTERVAL_PROPERTY when you create your connection, so that your client gets an exception if the connection should fail. For more information see Connection Handling.
You might need to allow a small time interval between connecting and calling the MQReceiveMessageNoWait function in order not to miss a pending message. For more information, see usage information in the section MQReceiveMessageNoWait.
Using Selectors EfficientlyThe use of selectors can have a significant impact on the performance of your application. It's difficult to put an exact cost on the expense of using selectors since it varies with the complexity of the selector expression, but the more you can do to eliminate or simplify selectors the better.
One way to eliminate (or simplify) selectors is to use multiple destinations to sort messages. This has the additional benefit of spreading the message load over more than one producer, which can improve the scalability of your application. For those cases when it is not possible to do that, here are some techniques that you can use to improve the performance of your application when using selectors:
- Have consumers share selectors. As of version 3.5 of Message Queue, message consumers with identical selectors “share” that selector in the broker, which can significantly improve performance. So if there is a way to structure your application to have some selector sharing, consider doing so.
- Use IN instead of multiple string comparisons. For example, the following expression:
color IN ('red', 'green', 'white')
Determining Message Order and PriorityIn general, all messages sent to a destination by a single session are guaranteed to be delivered to a consumer in the order they were sent. However, if they are assigned different priorities, a messaging system will attempt to deliver higher priority messages first.
Beyond this, the ordering of messages consumed by a client can have only a rough relationship to the order in which they were produced. This is because the delivery of messages to a number of destinations and the delivery from those destinations can depend on a number of issues that affect timing, such as the order in which the messages are sent, the sessions from which they are sent, whether the messages are persistent, the lifetime of the messages, the priority of the messages, the message delivery policy of queue destinations (see the Message Queue Administration Guide), and message service availability.
Managing ThreadsThis section addresses a number of thread management issues that you should be aware of in designing and programming a Message Queue C client. It covers the following topics:’
Message Queue C Runtime Thread Model
The Message Queue C-API library creates the thread(s) needed to provide runtime support for a Message Queue C client. It uses NSPR (Netscape Portable Runtime) GLOBAL threads. NSPR GLOBAL threads are fully compatible with native threads on each supported platform. Table 3-1 shows the thread model that the NSPR GLOBAL threads map to on each platform. For more information on NSPR, please see
http://www.mozilla.org/projects/nspr/
Table 3-1 Thread Model for NSPR GLOBAL Threads
Platform
Thread Model
Solaris
pthreads
Linux
pthreads
Windows
Win32 threads (from Microsoft Visual C++ runtime library msvcrt)
Concurrent Use of Handles
Table 3-2 lists the handles (objects) used in a C client program and specifies which of these may be used concurrently and which can only be used by one logical thread at a time.
Single-Threaded Session Control
A session is a single-threaded context for producing and consuming messages. Multiple threads should not use the same session concurrently nor use the objects it creates concurrently. The only exception to this occurs during the orderly shutdown of the session or its connection when the client calls the MQCloseSession or the MQCloseConnection function. Follow these guidelines in designing your client:
- If a client wants to have one thread producing messages and other threads consuming messages, the client should use a separate session for its producing thread.
- Do not create an asynchronous message consumer while the connection is in started mode.
- A session created with MQ_SESION_ASYNC_RECEIVE mode uses a single thread to run all its consumers’ MQMessageListenerFunc callback functions. Clients that want concurrent delivery should use multiple sessions.
- Do not call the MQStopConnection, MQCloseSession, or the MQCloseConnection functions from a MQMessageListenerFunc callback function. (These calls will not return until delivery of messages has stopped.)
- Call the MQFreeConnection function after MQCloseConnection and all of the application threads associated with a connection and its sessions, producers, consumers, etc., have returned.
The Message Queue C runtime library provides one thread to a session in MQ_SESSION_ASYNC_RECEIVE mode for asynchronous message delivery to its consumers. When the connection is started, all its sessions that have created asynchronous consumers are dedicated to the thread of control that delivers messages. Client code should not use such a session from another thread of control. The only exception to this is the use of MQCloseSession and MQCloseConnection.
Connection Exceptions
When a connection exception occurs, the Message Queue C library thread that is provided to the connection calls its MQConnectionExceptionListenerFunc callback if one exists. If an MQConnectionExceptionListenerFunc callback is used for multiple connections, it can potentially be called concurrently from different connection threads.
You should not call the MQCloseConnection function in an MQConnectionExceptionListenerFunc callback. Instead the callback function should notify another thread to call MQCloseConnection and return.
Managing Physical Destination LimitsWhen creating a topic or queue destination, the administrator can specify how the broker should behave when certain memory limits are reached. Specifically, when the number of unconsumed messages reaching a physical destination exceeds the number specified with the maxNumMsgs property or when the total amount of memory allowed for unconsumed messages exceeds the number specified with the maxTotalMsgBytes property, the broker takes one of the following actions, depending on the setting of the limitBehavior property:
If the default value REJECT_NEWEST is specified for the limitBehavior property, the broker throws out the newest messages received when memory limits are exceeded. If the message discarded is a persistent message, the producing client gets an error which you should handle by resending the message later.
If any of the other values is selected for the limitBehavior property or if the message is not persistent (or persistent and MQ_ACK_ON_PRODUCE_PROPERTY is false), the application client is not notified if a message is discarded. Application clients should let the administrator know how they prefer this property to be set for best performance and reliability.
Managing the Dead Message QueueWhen a message is deemed undeliverable, it is automatically placed on a special queue called the dead message queue. A message placed on this queue retains all of its original headers (including its original destination) and information is added to the message’s properties to explain why it became a dead message.
This section describes the message properties that you can set or examine programatically to determine the following:
- Whether a dead message can be sent to the dead message queue.
- Whether the broker should log information when a message is destroyed or moved to the dead message queue.
- Whether the body of the message should also be stored when the message is placed on the dead message queue.
- Why the message was placed on the dead message queue and any ancillary information.
(Message Queue 3 2005Q1 clients can set properties related to the dead message queue on messages and send those messages to clients compiled against Message Queue 3.5x or earlier versions. However clients receiving such messages cannot examine these properties without recompiling against Message Queue 3 2005Q1 libraries.)
The dead message queue is automatically created by the system and called mq.sys.dmq. You can write a Java program that uses the message monitoring API, described in Message Queue Developer’s Guide for Java Clients, to determine whether that queue is growing, to examine messages on that queue, and so on.
You can set the properties described in Table 3-3 for any message to control how the broker should handle that message if it deems it to be undeliverable. Note that these message properties are needed only to override default destination, or default broker-based behavior.
The properties described in Table 3-4 are set by the client runtime for a message placed in the dead message queue.
Factors Affecting PerformanceApplication design decisions can have a significant effect on overall messaging performance. In general, the more reliable the delivery of messages, the more overhead and bandwidth are required to achieve it. The trade-off between reliability and performance is a significant design consideration. You can maximize performance and throughput by choosing to produce and consume non-persistent messages. On the other hand, you can maximize reliability by producing and consuming persistent messages using a transacted session. Between these extremes are a number of options, depending on the needs of your application. This section describes how these options or factors affect performance. They include the following:
Table 3-5 summarizes how application design factors affect messaging performance. The table shows two scenarios—a high reliability, low performance scenario and a high performance, low reliability scenario—and the choice of application design factors that characterizes each. Between these extremes, there are many choices and trade-offs that affect both reliability and performance.
Note
In the discussion that follows, performance data was generated on a two-CPU, 1002 Mhz, Solaris 8 system, using file-based persistence. The performance test first warmed up the Message Queue broker, allowing the Just-In-Time compiler to optimize the system and the persistent database to be primed.
Once the broker was warmed up, a single producer and a single consumer were created, and messages were produced for 30 seconds. The time required for the consumer to receive all produced messages was recorded, and a throughput rate (messages per second) was calculated. This scenario was repeated for different combinations of the application design factors shown in Table 3-5.
Delivery Mode (Persistent/Non-persistent)
Persistent messages guarantee message delivery in case of message server failure. The broker stores these message in a persistent store until all intended consumers acknowledge they have consumed the message.
Broker processing of persistent messages is slower than for non-persistent messages for the following reasons:
- A broker must reliably store a persistent message so that it will not be lost should the broker fail.
- The broker must confirm receipt of each persistent message it receives. Delivery to the broker is guaranteed once the method producing the message returns without an exception.
- Depending on the client acknowledgment mode, the broker might need to confirm a consuming client’s acknowledgement of a persistent message.
The differences in performance for persistent and non-persistent modes can be significant--about 25% faster for non-persistent messages.
Use of Transactions
A transaction guarantees that all messages produced in a transacted session and all messages consumed in a transacted session will be either processed or not processed (rolled back) as a unit. The C-API supports local transactions.
A message produced or acknowledged in a transacted session is slower than in a non-transacted session for the following reasons:
- Additional information must be stored with each produced message.
- In some situations, messages in a transaction are stored when normally they would not be. For example, a persistent message delivered to a topic destination with no subscriptions would normally be deleted, however, at the time the transaction is begun, information about subscriptions is not available.
- Information on the consumption and acknowledgement of messages within a transaction must be stored and processed when the transaction is committed.
Acknowledgement Mode
Other than using transactions, you can ensure reliable delivery by having the client acknowledge receiving a message. If a session is closed without the client acknowledging the message or if the message server fails before the acknowledgment is processed, the broker redelivers that message, setting the MQ_REDELIVERED_HEADER_PROPERTY message header.
For a non-transacted session, the client can choose one of three acknowledgement modes, each of which has its own performance characteristics:
- AUTO_ACKNOWLEDGE. The system automatically acknowledges a message once the consumer has processed it. This mode guarantees at most one redelivered message after a provider failure.
- CLIENT_ACKNOWLEDGE. The application controls the point at which messages are acknowledged. All messages that have been received in the same session up to the message where the acknowledge function is called upon are acknowledged. If the message server fails while processing a set of acknowledgments, one or more messages in that group might be redelivered.
Performance is impacted by acknowledgement mode for the following reasons:
- Extra control messages between broker and client are required in AUTO_ACKNOWLEDGE and CLIENT_ACKNOWLEDGE modes. The additional control messages add processing overhead and can interfere with JMS payload messages, causing processing delays.
- In AUTO_ACKNOWLEDGE and CLIENT_ACKNOWLEDGE modes, the client must wait until the broker confirms that it has processed the client’s acknowledgment before the client can consume more messages. (This broker confirmation guarantees that the broker will not inadvertently redeliver these messages.)
- The Message Queue persistent store must be updated with the acknowledgement information for all persistent messages received by consumers, thereby decreasing performance.
Durable vs. Non-Durable Subscriptions
Subscribers to a topic destination have either durable or non-durable subscriptions. Durable subscriptions provide increased reliability at the cost of slower throughput for the following reasons:
- The Message Queue message server must persistently store the list of messages assigned to each durable subscription so that should a message server fail, the list is available after recovery.
- Persistent messages for durable subscriptions are stored persistently, so that should a message server fail, the messages can still be delivered after recovery, when the corresponding consumer becomes active. By contrast, persistent messages for non-durable subscriptions are not stored persistently (should a message server fail, the corresponding consumer connection is lost and the message would never be delivered).
For nonpersistent messages, performance is about the same for durable and non durable subscriptions. For persistent messages, performance is about 20% lower for durable subscriptions than for nondurable subscriptions.
Use of Selectors (Message Filtering)
Application developers can have the messaging provider sort messages according to criteria specified in the message selector associated with a consumer and deliver to that consumer only those messages whose property value matches the message selector. For example, if an application creates a subscriber to the topic WidgetOrders and specifies the expression NumberOfOrders >1000 for the message selector, messages with a NumberOfOrders property value of 1001 or more are delivered to that subscriber.
Creating consumers with selectors lowers performance (as compared to using multiple destinations) because additional processing is required to handle each message. When a selector is used, it must be parsed so that it can be matched against future messages. Additionally, the message properties of each message must be retrieved and compared against the selector as each message is routed. However, using selectors provides more flexibility in a messaging application and may lower resource requirements at the expense of speed.
For more information on using selectors, see Using Selectors Efficiently
Message Size
Message size affects performance because more data must be passed from producing client to broker and from broker to consuming client, and because for persistent messages a larger message must be stored.
However, by batching smaller messages into a single message, the routing and processing of individual messages can be minimized, providing an overall performance gain. In this case, information about the state of individual messages is lost.
Figure 3-1 compares throughput in kilobytes per second for 1k, 10k, and 100k-sized messages for persistent and non-persistent messages. All messages are sent to a queue destination and use AUTO_ACKNOWLEDGE acknowledgement mode.
Figure 3-1 shows that in both cases there is less overhead in delivering larger messages compared to smaller messages. You can also see that the almost 50% performance gain of non-persistent messages over persistent messages shown for 1k and 10k-sized messages is not maintained for 100k-sized messages, probably because network bandwidth has become the bottleneck in message throughput for that case.
Figure 3-1 Performance Impact of a Message Size
Message Body Type
The C API supports two message body types:
Since performance varies with the complexity of the data, text messages are slightly more expensive to send than byte messages.