Broker Clusters: Scalability and Availability

Message Queue brokers can be connected into a broker cluster: a set of brokers that work collectively to perform message delivery between message producers and consumers. Broker clusters add scalability and availability to the Message Queue service, as described briefly in the following sections:

• Message Service Scalability

• Message Service Availability

For additional information on broker clusters, see Chapter 4, Broker Clusters

Message Service Scalability

As the number of clients or the number of connections grows, you might need to scale a message service to eliminate bottlenecks or to improve performance. In general, you can scale a message service both vertically (increasing the number of client applications that are supported by a single broker) and horizontally (distributing client applications among a number of interconnected brokers).

Vertical scaling usually requires adding more processing power for a broker and by expanding available resources. You can do this by adding more processors or memory, by switching to a shared thread model, or by running the Java VM in 64 bit mode.

Horizontal scaling is generally achieved using a broker cluster. While it is possible to scale horizontally by simply redistributing clients among additional brokers that are not in a cluster, this approach is appropriate only if your messaging operations can be divided into independent work groups. However, if producer clients must produce messages to be consumed by consumer clients connected to remote brokers, then brokers must work collectively, as part of a broker cluster, to achieve horizontal scaling.

In a broker cluster, each broker is connected to every other broker in the cluster. Brokers can reside on the same host, but more often are distributed across a network. Each broker can route messages from producers to which it is directly connected to consumers that are connected to remote brokers in the cluster.

If you are using the point-to-point domain, you can scale the consumer side by allowing multiple consumers to access a queue. This is a Message Queue feature (the JMS specification defines messaging behavior in the case of only one consumer accessing a queue). When multiple consumers access a queue, the load-balancing among them takes into account each consumer’s capacity and message processing rate.

Message Service Availability

In addition to providing for message service scalability, broker clusters also provide for message service availability. If one broker in a cluster fails, then other brokers in the cluster are available to continue to provide messaging services to client applications.

Message Queue supports two clustering models that provide different degrees of availability:

• Conventional broker clusters. A conventional broker cluster provides message service availability. When a broker or a connection fails, clients connected to the failed broker reconnect to another broker in the cluster. However, messages and state information stored in the failed broker cannot be recovered until the failed broker is brought back online. This can result in an interruption of message delivery.

• Enhanced broker clusters. An enhanced broker cluster provides data availability in addition to message service availability. When a broker or a connection fails, another broker takes over the pending work of the failed broker. The failover broker has access to the failed broker's messages and state information. Clients connected to the failed broker reconnect to the failover broker. In an enhanced cluster, as compared to a conventional cluster, a failure results in no interruption of message delivery.

You can also achieve data availability in a conventional cluster by using Sun Cluster software. Sun Cluster software replicates broker data and provides for a hot standby broker to take over the pending work of a failed broker. For details, see the documentation for Sun Cluster Data Service Agent for Message Queue.