Multiple Consumer Queue Performance

The efficiency with which multiple queue consumers process messages in a queue destination depends on the following configurable queue destination attributes:

• The number of active consumers (maxNumActiveConsumers)

• The maximum number of messages that can be delivered to a consumer in a single batch (consumerFlowLimit)

To achieve optimal message throughput there must be a sufficient number of active consumers to keep up with the rate of message production for the queue, and the messages in the queue must be routed and then delivered to the active consumers in such a way as to maximize their rate of consumption. The general mechanism for balancing message delivery among multiple consumers is described in the Message Queue Technical Overview.

If messages are accumulating in the queue, it is possible that there is an insufficient number of active consumers to handle the message load. It is also possible that messages are being delivered to consumers in batch sizes that cause messages to be backing up on the consumers. For example, if the batch size (consumerFlowLimit) is too large, one consumer might receive all the messages in a queue while other active consumers receive none. If consumers are very fast, this might not be a problem. However, if consumers are relatively slow, you want messages to be distributed to them evenly, and therefore you want the batch size to be small. Although smaller batch sizes require more overhead to deliver messages to consumers, for slow consumers there is generally a net performance gain in using small batch sizes. In the extreme case, setting consumerFlowLiimit to 1 configures the queue for “round-robin” delivery, in which single messages are delivered, one to each consumer in rotation.