Sun Java System Message Queue 3 2005Q4 Technical Overview |
Chapter 4
Broker ClustersMessage Queue Enterprise Edition supports the use of broker clusters: groups of brokers working together to provide message delivery services to clients. Clusters enable an administrator to scale messaging operations with the volume of message traffic by distributing client connections among multiple brokers.
This chapter discusses the architecture and internal functioning of such broker clusters. It covers the following topics:
Note that broker clusters provide service availability but not data availability. If one broker in a cluster fails, clients connected to that broker can reconnect to another broker in the cluster but may lose some data while they are reconnected to the alternate broker.
Cluster ArchitectureFigure 4-1 shows Message Queue’s architecture for broker clusters. Each broker within a cluster is directly connected to all the others. Each client (message producer or consumer) has a single home broker with which it communicates directly, sending and receiving messages as if that broker were the only one in the cluster. Behind the scenes, the home broker works in concert with the other brokers to provide delivery services for all connected clients.
In a cluster, service availability depends on brokers being able to share information about destinations and durable subscribers. If a clustered broker fails, it is possible that this state information gets out of sync. To guard against this possibility, you can designate one broker within the cluster as the master broker. The master broker maintains a configuration change record to track changes to the cluster’s persistent entities (destinations and durable subscriptions). This record is used to propagate such change information to brokers that were offline when the changes occurred.
Figure 4-1 Cluster Architecture
The following sections discuss how message delivery takes place within a cluster and how the brokers are configured and synchronized, even in the case when one or more has been offline.
Message DeliveryIn a cluster configuration, brokers share information about destinations and message consumers: each broker knows the following information”
This allows each broker to route messages from its own directly connected message producers to remote message consumers. The home broker of a producer has different responsibilities from the home broker of the consumer:
- The producer’s home broker is responsible for persisting and routing messages originating from that producer, for logging, for managing transactions, and for processing acknowledgements from consuming clients.
- The consumer’s home broker is responsible for persisting information about consumers, for forwarding the message to the consumer, and for letting the producer’s broker know whether the consumer is still available and whether the message was successfully consumed.
Clustered brokers work together to minimize message traffic within the cluster; for example, if a remote broker has two identical subscriptions for the same topic destination, the message is sent over the wire only once. You can further reduce traffic by setting a destination property specifying that delivery to local consumers has priority over delivery to remote consumers.
If secure, encrypted message delivery between client and broker is required, you can configure a cluster to provide secure delivery of messages between brokers.
Destination Attributes
Attributes set for a physical destination on a clustered broker apply to all instances of that destination in the cluster; however, some limits specified by these attributes apply to the cluster as a whole and others to individual destination instances. Table 4-1 lists the attributes you can set for a physical destination and specifies their scope.
Clustering and Destinations
Whether a destination is admin-created, auto-created, or temporary, affects how the destination is propagated in a cluster and how it is handled in the event of connection or broker failure.
Figure 4-2 shows four clustered brokers. The direct (private) connection between brokers is shown, as is the connection between clients and the brokers to which they are connected. The figure illustrates a number of possibilities, described in the following sections.
Figure 4-2 Cluster Example
Producing to a Queue Using the Reply-To Model
As shown in the previous figure:
- The administrator creates the physical destination QW. The queue is replicated throughout the cluster at creation time.
- Producer ProdQW sends a message to queue QW and uses the reply-to model, directing replies to temporary queue TempQ1W. (The temporary queue is created and replicated when an application creates a temporary destination and adds a consumer.)
- The home broker, BrokerW, persists the message sent to QW and routes the message to the first active consumer that meets the selection criteria for this message. Depending on which consumer is ready to receive the message, the message is delivered either to consumer C1QW (on BrokerX) or to consumer C2QW (on BrokerY). The consumer receiving the message, sends a reply to the destination TempQ1W.
Producing to an Auto-Created Destination
As shown in the previous figure:
- Producer ProdAutoQY sends a message to a destination AutoQY that does not exist on the broker.
- The broker persists the message and creates destination AutoQY.
Auto-created destinations are not automatically replicated across the cluster. Only when a consumer elects to receive messages from a queue AutoQY, would that consumer’s home broker create the destination AutoQY and convey the message to the consumer. At the point where one consumer creates the autocreated destination, the destination is replicated across the cluster.
Publishing to a Topic Destination
As shown in the previous figure,
- The administrator creates the physical topic destination TY. The admin-created destination TY is replicated throughout the broker cluster (before the destination is used).
- Publisher PubTY, sends a message to TY.
- The home broker, BrokerY, persists any messages published to TY and routes the messages to all topic subscribers that match the selection criteria for this message.
Handling Destinations in the Event of Connection or Broker Failure
Table 4-2 explains how different kinds of destinations are replicated and deleted in a cluster.
Cluster ConfigurationTo establish connections between the brokers in a cluster at startup time, each broker must be passed the host names and port numbers of all the others (including the master broker, if any). This information is specified by a set of cluster configuration properties, which should be the same for all brokers in the cluster. Although you can specify the configuration properties for each broker individually, this approach is error-prone and can easily lead to inconsistencies in the cluster configuration. Instead, it is recommended that you place all the configuration properties in one central cluster configuration file that is referenced by each broker at startup time. This ensures that all brokers share the same configuration information.
See the Message Queue Administration Guide for detailed information on cluster configuration properties.
Note
Although the cluster configuration file was originally intended for configuring clusters, it is also a convenient place to store other properties that are shared by all brokers in a cluster.
Cluster SynchronizationWhenever a cluster’s configuration is changed, information about the change is automatically propagated to all brokers in the cluster. A cluster configuration changes when one of the following events occurs:
- A destination on one of the cluster’s brokers is created or destroyed.
- The properties of a destination are changed.
- A message consumer is registered with its home broker.
- A message consumer is disconnected from its home broker (whether explicitly or through failure of the client, the broker, or the network).
- A message consumer establishes a durable subscription to a topic.
Information about these changes is propagated immediately to all brokers in the cluster that are online at the time of the change. However, a broker that is offline (one that has crashed, for example) will not receive notice of the change when it occurs. To accommodate offline brokers, Message Queue maintains a configuration change record for the cluster, recording all persistent entities (destinations and durable subscriptions) that have been created or destroyed. When an offline broker comes back online (or when a new broker is added to the cluster), it consults this record for information about destinations and durable subscribers, then exchanges information with other brokers about currently active message consumers.
One broker in the cluster, designated as the master broker, is responsible for maintaining the configuration change record. Because other brokers cannot complete their initialization without the master broker, it should always be the first broker started within the cluster. If the master broker goes offline, configuration information cannot be propagated throughout the cluster, because other brokers cannot access the configuration change record. Under these conditions, you will get an exception if you try to create, reconfigure, or destroy a destination or a durable subscription or attempt a related operation such as reactivating a durable subscription. (Non-administrative message delivery continues to work normally, however.) The use of a master broker and a configuration change record is optional. They are only required if you are concerned with cluster synchronization after cluster configuration changes or a broker failure.