Oracle GlassFish Message Queue 4.4.2 Technical Overview

Chapter 4 Broker Clusters

Message Queue supports the use of broker clusters: groups of brokers working together to provide message delivery services to clients. Clusters enable a Message Queue service to scale messaging operations by distributing client connections among multiple brokers. Because a cluster consists of multiple brokers, the cluster helps protect against individual broker failure. Two cluster models provide different levels of message service availability.

This chapter discusses the architecture and internal functioning of broker clusters. It covers the following topics:

Cluster Models

Message Queue supports two clustering models both of which provide a scalable message service, but with each providing a different level of message service availability:


Note –

Despite the message service availability offered by both conventional and enhanced broker clusters, they do not provide a guarantee against failure and the possibility that certain failures, for example in the middle of a transaction, could require that some operations be repeated. It is the responsibility of the messaging application (both producers and consumers) to handle and respond appropriately to failure notifications from the messaging service.


Conventional and enhanced broker clusters are built on the same underlying infrastructure and message delivery mechanisms. They differ in how brokers in the cluster are synchronized with one another and in how the cluster detects and responds to failures.

The sections that follow first describe the infrastructure and delivery mechanisms common to both clustering models, after which the unique aspects of each model is explained.

Cluster Message Delivery

A broker cluster facilitates the delivery of messages between client applications that are connected to different brokers in the cluster.

The following illustration shows salient features of a Message Queue broker cluster. Each of three brokers is connected to the other brokers in the cluster: the cluster is fully-connected. The brokers communicate with each other and pass messages by way of a special cluster connection service, shown in Figure 4–1 by the dashed lines.

Figure 4–1 Message Queue Broker Cluster

Diagram showing basic elements of a broker cluster. Figure
explained in the text.

Each broker typically has a set of messaging clients (producers and/or consumers) that are directly connected to that broker. For these client applications, the broker to which they are directly connected is called their home broker. Each client communicates directly only with its home broker, sending and receiving messages as if that broker were the only broker in the cluster.

Accordingly, a producer in the cluster produces messages to a destination in its home broker. The home broker is responsible for routing and delivering the messages to all consumers of the destination, whether these consumers are local (connected to the home broker) or remote (connected to other brokers in the cluster). The home broker works in concert with the other brokers to deliver messages to all consumers, no matter what brokers they are connected to.

Propagation of Information Across a Cluster

To facilitate delivery of messages across the cluster, information about the destinations and consumers of each broker is propagated to all brokers in the cluster. Each broker therefore stores the following information:

Changes in this information are propagated whenever one of the following events occurs:

The propagation of destination and consumer information across the cluster means that destinations and consumers are essentially global to the cluster. In the case of destinations, properties set for a physical destination (see Configuring Physical Destinations) apply to all instances of that destination in the cluster. Distributing producers across a cluster thus results in cumulative cluster-wide limits specified by destination properties such as the maximum number of messages, the maximum number of message bytes, and the maximum number of producers.

Message Delivery Mechanisms

Despite the global nature of destinations and consumers in a cluster, a home broker has special responsibilities with respect to both its producers and consumers:

The cluster connection service transports payload messages, when needed, from destinations on a home broker to destinations on remote brokers. It also transports control messages, such as client acknowledgements, from remote brokers back to a home broker. The cluster attempts to minimize message traffic across the cluster. For example, it only sends a message to a remote broker if the remote broker is home to a consumer of the message. If a remote broker has two identical consumers for the same destination (for example two topic subscribers), the message is sent over the wire only once. (You can further reduce traffic by setting a destination property specifying that delivery to local consumers has priority over delivery to remote consumers.)

If secure message delivery is required, you can configure a cluster to also provide secure, encrypted delivery of messages between brokers.

As a result of the cluster delivery mechanisms described above, each broker in a cluster stores different persistent messages and maintains different state information. If a broker fails, the mechanisms for recovering its persistent information depends on the cluster model being used, as described in subsequent sections.

Conventional Clusters

The following figure illustrates a conventional broker cluster.

Figure 4–2 Conventional Broker Cluster

Diagram showing elements of a conventional broker cluster.
Figure explained in the text.

A conventional broker cluster has the following characteristics:

Enhanced Clusters

The following figure illustrates an enhanced broker cluster. An enhanced broker cluster provides both service availability and data availability.

Figure 4–3 Enhanced Cluster

Diagram showing elements of an enhanced broker cluster.
Figure explained in the text.

An enhanced broker cluster has the following characteristics:

To configure an enhanced cluster you set cluster configuration properties for each broker in the cluster. These properties are detailed in Enhanced Broker Cluster Properties in Oracle GlassFish Message Queue 4.4.2 Administration Guide.

Cluster Models Compared

Conventional and enhanced cluster models share the same basic infrastructure. They both use the cluster communication service to enable message delivery between producers and consumers across the cluster. However, as shown in the following figure and described in previous sections, these models differ in how destination and consumer information is synchronized across the cluster, in the mechanisms for detecting failure, in how client reconnect takes place.

Figure 4–4 Cluster Infrastructure

Diagram comparing conventional and enhanced broker cluster
infrastructures. Figure explained in the text.

In addition, while both models rely on the broker's persistence interfaces (both flat-file and JDBC), in the case of enhanced clusters the shared data store must be a highly-available JDBC database (a highly-available file-based data store has not yet been implemented).

The following table summarizes the functional differences between the two cluster models. This information might help in deciding which model to use or whether to switch from one to another.

Table 4–1 Clustering Model Differences

Functionality 

Conventional 

Enhanced 

Performance 

Faster than enhanced cluster model. 

Slower than conventional cluster model. 

Service availability 

Yes, but some operations are not possible if master broker is down. 

Yes. 

Data availability 

No. State information in failed broker is not available until broker restarts. 

Yes at all times. 

Transparent recovery from failure 

No. Message delivery is interrupted. Also, client reconnects might not be possible if failure occurs during a transaction commit (rare). 

Yes. No interruption in message delivery. If failure occurs during a transaction commit, an exception might be thrown indicating that the transaction could not be committed (extremely rare). 

Configuration 

Set appropriate cluster configuration properties for each broker. 

Set appropriate cluster configuration properties for each broker. 

Additional requirements 

None. 

Highly-available database. 

Restricted to subnet 

No. 

Yes. 

Cluster Configuration

Depending on the clustering model used, you must specify appropriate broker properties to enable the Message Queue service to manage the cluster. This information is specified by a set of cluster configuration properties,. Some of these properties must have the same value for all brokers in a cluster; others must be specified for each broker individually. It is recommended that you place all configuration properties that must be the same for all brokers in one central cluster configuration file that is referenced by each broker at startup time. This ensures that all brokers share the same common cluster configuration information.

See Configuring Broker Clusters in Oracle GlassFish Message Queue 4.4.2 Administration Guidefor detailed information on cluster configuration properties.


Note –

Although the cluster configuration file was originally intended for configuring clusters, it is also a convenient place to store other (non-cluster-related) properties that are shared by all brokers in a cluster.


For complete information about administering broker clusters, see Chapter 10, Configuring and Managing Broker Clusters, in Oracle GlassFish Message Queue 4.4.2 Administration Guide. For information about the effect of reconnection on the client, see Connection Event Notification in Oracle GlassFish Message Queue 4.4.2 Developer’s Guide for Java Clients and Client Connection Failover (Auto-Reconnect) in Oracle GlassFish Message Queue 4.4.2 Developer’s Guide for Java Clients.