Managing Broker System-Wide Memory

Once clients are connected to the broker, the routing and delivery of messages can proceed. In this phase, the broker is responsible for creating and managing different types of physical destinations, ensuring a smooth flow of messages, and using resources efficiently. You can use the broker configuration properties described under Routing and Delivery Properties to manage these tasks in a way that suits your application’s needs.

The performance and stability of a broker depend on the system resources (such as memory) available and how efficiently they are utilized. You can set configuration properties to prevent the broker from becoming overwhelmed by incoming messages or running out of memory. These properties function at three different levels to keep the message service operating as resources become scarce:

• Systemwide message limits apply collectively to all physical destinations on the system. These include the maximum number of messages held by a broker (imq.system.max_count) and the maximum total number of bytes occupied by such messages (imq.system.max_size). If either of these limits is reached, the broker will reject any new messages until the pending messages fall below the limit. There is also a limit on the maximum size of an individual message (imq.message.max_size) and a time interval at which expired messages are reclaimed (imq.message.expiration.interval).

• Individual destination limits regulate the flow of messages to a specific physical destination. The configuration properties controlling these limits are described in Chapter 18, Physical Destination Property Reference. They include limits on the number and size of messages the destination will hold, the number of message producers and consumers that can be created for it, and the number of messages that can be batched together for delivery to the destination.

  The destination can be configured to respond to memory limits by slowing down the delivery of message by message producers, by rejecting new incoming messages, or by throwing out the oldest or lowest-priority existing messages. Messages deleted from the destination in this way may optionally be moved to the dead message queue rather than discarded outright; the broker property imq.destination.DMQ.truncateBody controls whether the entire message body is saved in the dead message queue, or only the header and property data.

  As a convenience during application development and testing, you can configure a message broker to create new physical destinations automatically whenever a message producer or consumer attempts to access a nonexistent destination. The broker properties summarized in Table 17–3 parallel the ones just described, but apply to such auto-created destinations instead of administratively created ones.

• System memory thresholds define levels of memory usage at which the broker takes increasingly serious action to prevent memory overload. Four such usage levels are defined:

  • Green: Plenty of memory is available.

  • Yellow: Broker memory is beginning to run low.

  • Orange: The broker is low on memory.

  • Red: The broker is out of memory.

    The memory utilization percentages defining these levels are specified by the broker properties imq.green.threshold, imq.yellow.threshold , imq.orange.threshold, and imq.red.threshold , respectively; the default values are 0% for green, 80% for yellow, 90% for orange, and 98% for red.

    As memory usage advances from one level to the next, the broker responds progressively, first by swapping messages out of active memory into persistent storage and then by throttling back producers of nonpersistent messages, eventually stopping the flow of messages into the broker. (Both of these measures degrade broker performance.) The throttling back of message production is done by limiting the size of each batch delivered to the number of messages specified by the properties imq.resourceState .count, where resourceState is green , yellow, orange, or red , respectively.

The triggering of these system memory thresholds is a sign that systemwide and destination message limits are set too high. Because the memory thresholds cannot always catch potential memory overloads in time, you should not rely on them to control memory usage, but rather reconfigure the system-wide and destination limits to optimize memory resources.