Chapter 2 Introduction to Message Queue

Message Queue is a reliable asynchronous messaging service that conforms to the JMS 1.1 specification. In addition, to provide for the needs of large-scale enterprise deployments, Message Queue provides a host of features that exceed JMS specification requirements.

This chapter describes the Message Queue service architecture and introduces its enterprise features and capabilities. The chapter covers the following topics:

Message Service Architecture

As shown in the figure, a Message Queue client uses the Java or C API to send or receive a message. These APIs are implemented in a Java or C-client runtime library, which does the actual work of creating connections to the broker and packaging the bits appropriately for the connection service requested. If the application uses administered objects, the client runtime locates these objects in an object store and uses them to configure the connection and to locate physical destinations. The broker routes and delivers the message. An administrator uses Message Queue administrative tools to manage the broker and to add administered objects to the object store.

Message Server

The message server is composed of one or more brokers and performs message routing and delivery. It is the heart of the Message Queue service.

The message server consists of a single broker or a set of brokers working together (as a broker cluster) to perform message routing and delivery services. The broker is a process that performs the following tasks:

For a detailed description of the message server, its internal components, and the functions they perform, see Chapter 4, "Message Server".

Message Queue Enterprise Edition supports the use of broker clusters, consisting of multiple interconnected broker instances, allowing a message server to scale with the volume of message traffic. For a description of architecture and cluster configuration issues, see Chapter 5, "Broker Clusters."

Client Runtime

The Message Queue client runtime provides client applications with an interface to the Message Queue service. The client runtime supports all operations needed for Message Queue clients to produce messages (send them to destinations) and to consume messages (retrieve them from destinations).

There are two language implementations of the Message Queue client runtime, as shown in Figure 2-1:

Figure 2-2 illustrates the central role played by the client runtime between Message Queue clients and the message server. Message production and consumption involve an interaction between clients and the client runtime, while message delivery is an interaction between the client runtime and the message server.

The following subsections briefly describe client runtime functions. Some aspects of the behavior of the client runtime can be customized by configuring the properties of the connection factory object.

Connection Handling

To configure connection handling behavior you must specify the host name and port of the broker to which the client wants to connect and the type of connection service desired. If the connection is made to a broker that is part of a cluster, you must specify a list of addresses to which to make a connection. If one broker is not online, the client runtime can connect you to another broker in the cluster.

In the Enterprise Edition, the client runtime can automatically reconnect to a broker if a connection fails. The reconnection can be to the same broker, or to a broker different from the original connection if the client is connected to a broker that is part of a cluster.

If broker instances do not use a shared, highly available persistent store (as could be achieved through integration of Message Queue with Sun Cluster), persistent messages and other state information held by the failed (or disconnected) broker can be lost if a reconnect is to a different broker instance. That is to say, reconnection provides connection failover but not data availability.

Client Identification

A client ID can be set on any connection if an application finds it useful; it must be set to identify durable subscribers.

To keep track of durable subscriptions, the broker uses a unique client identification. The client ID is used to identify a durable subscriber that is inactive at the time that messages are delivered to a topic destination. The broker retains messages addressed to such subscribers and makes them available when the subscriber becomes active.

Therefore, a client identifier must be set whenever using durable subscriptions in deployed applications. A Message Queue feature allows you to use a special variable name syntax when you specify the Client ID. This makes it possible to obtain a different client ID for each connection obtained from a connection factory object, whether that object is created by an administrator or programmatically. For more information, see the Message Queue Administration Guide.

Message Distribution to Consumers

Messages delivered by a broker over a connection are received by the client runtime and distributed to the appropriate Message Queue sessions, where they are queued up to be consumed by their respective message consumers, as shown in Figure 2-3.

Messages are fetched off each session queue one at a time and consumed either synchronously (by a client thread invoking the receive() method) or asynchronously (by the session thread invoking the onMessage() method of a message listener object). (A session is single threaded.)

The flow of messages delivered to the client runtime is metered at a per consumer level. By appropriately adjusting connection factory properties, you can balance the flow of messages so that messages delivered to one session do not adversely affect the delivery of messages to other sessions on the same connection.

Ensuring Reliable Message Delivery

The client runtime has an important role in ensuring reliable delivery of messages. It supports the client acknowledgement and transaction modes of the JMS specification and controls the various broker acknowledgement behaviors used to guarantee reliable delivery.

The JMS specification describes a number of client acknowledgement modes that provide for different levels of reliability. These acknowledgement modes, and additional modes implemented by Message Queue, are described in the context of message consumption (see Client Acknowledgements).

In the case of persistent messages and reliable delivery, the broker normally acknowledges to the client runtime when it has completed operations used to ensure once and only once consumption of messages. You can use connection factory properties to suppress such broker acknowledgements, thereby saving on network bandwidth and processing. Of course, such suppression of broker acknowledgements, eliminates guarantees of reliable delivery.

Message Flow Control

The client runtime is the gatekeeper for the flow of messages across a connection. In addition to the regular JMS payload messages that flow across a connection, Message Queue also sends a variety of control messages that are used to guarantee reliable delivery, manage the flow of messages across a connection, and perform other control functions.

Since payload messages and control messages compete for the same connection, they can collide, causing logjams to occur. The client runtime enforces various configurable flow limits and metering schemes to minimize the collision of payload and control messages, and thereby maximize message throughput.

Overriding Message Header Values

The client runtime can override JMS message header fields that specify the persistence, lifetime, and priority of messages.

Message Queue allows message header overrides at the level of a connection: overrides apply to all messages produced in the context of a given connection.

The ability of the client runtime to override message header values gives a Message Queue administrator more control over the resources of a message server. Overriding these fields, however, has the risk of interfering with application-specific requirements (for example, message persistence). So this capability should be used only in consultation with the appropriate application users or designers.

Other Functions

Administered Objects

Administered objects encapsulate provider-specific implementation and configuration information about connections and destinations. Administered objects can be created programmatically, or they can be are created and configured using administrator tools, stored in an object store, and accessed by client applications through standard JNDI lookup code.

Message Queue provides the administered object types shown in the following table.

Table 2-1 Message Queue Administered Object Types
Type	Description
Destination	Represents a physical destination in a broker. Contains the provider-specific name of the physical destination in the broker. Message consumer and/or message producer objects use a destination administered object to access the corresponding physical destination.
Connection Factory	Establishes physical connections between a client application and a Message Queue message server. Also configures the Message Queue client runtime, which controls the behavior of physical connections. When setting the attribute values of a connection factory administered object, you specify properties that apply to all connections that it establishes.
XA Connection Factory	Used to establish physical connections that support distributed transactions (see Distributed Transactions). XA connection factory objects share the same set of attributes as regular connection factory objects, but enable the additional mechanisms needed to support distributed transactions.
SOAP Endpoint	Identifies the final destination of a SOAP message: this is the URL of a servlet that can receive the SOAP message. The SOAP endpoint administered object can be configured to specify multiple URLs. Also specifies the lookup name associated with the object, and object store attributes.

Using Administered Objects via JNDI

Although the JMS specification does not require JMS clients to look up administered objects in a JNDI namespace, there are distinct advantages to doing so: it allows for a single source of control, it allows connections (client runtime behavior) to be configured and reconfigured without having to recode, and it allows clients to be portable to other JMS providers.

Administered objects make it easier to control and manage a Message Queue service:

In other words, the use of administered objects allows a Message Queue administrator to control message service configuration details, while at the same time allowing client applications to be provider-independent.

Using administered objects means that client programmers do not have to know about provider-specific syntax and object naming conventions or provider-specific configuration properties. In fact, by specifying that administered objects be read only, administrators can ensure that client applications cannot change administered object attribute values that were set when the administered object was first created.

While it is possible for client applications to instantiate both connection factory and destination administered objects on their own, this practice undermines the basic purpose of an administered object. Message Queue administrators need to control broker resources required by an application and to tune messaging performance. In addition, directly instantiating administered objects makes client applications provider-dependent.

Notwithstanding these arguments, applications often instantiate administered objects in development environments in which administrative control is not an issue.

Object Stores

Message Queue administered objects are placed in an object store (see Figure 2-1) where they can be accessed by client applications through a JNDI lookup. Message Queue supports two types of object store: a standard LDAP directory server and a file-system object store.

LDAP Server Object Store An LDAP server is the recommended object store for production messaging systems. LDAP implementations are available from a number of vendors and are designed for use in distributed systems. LDAP servers also provide security features that are useful in production environments.

File-system Object Store Message Queue supports a file-system object store, which is not recommended for production systems but has the advantage of being very easy to use in development environments. Rather than setting up an LDAP server, all you have to do is create a directory on your local file system. A file-system object store, however, cannot be used as a centralized object store for clients deployed across multiple computer nodes unless these clients have access to the directory where the object store resides.

Administration Tools

Message Queue administration tools consist of a set of command line utilities and a graphical user interface (GUI) Administration Console.

Command Line Utilities Message Queue provides a set of command line utilities to perform all Message Queue administration tasks, such as starting up and managing a broker, creating and managing physical destinations, managing administered objects, and performing other, more specialized administrative tasks. All the command line utilities share common formats, syntax conventions, and options. For more detailed information on the use of the command line utilities, see the Message Queue Administration Guide.

The Administration Console The Console provides a subset of the capabilities of the Message Queue command line utilities. You can use the Administration Console to manage a broker, create and manage physical destinations, and manage administered objects. However you cannot perform the more specialized tasks of some of the command line utilities. For example, you cannot use the Administration Console to start up a broker, create broker clusters, or manage a user repository. These tasks must be performed using the Message Queue command line utilities.

The Message Queue Administration Guide provides a brief, hands-on tutorial to familiarize you with the Administration Console and to illustrate how you use it to accomplish basic tasks.

The Administration Console and some of the command line utilities allow for remote management of brokers and physical destinations.

Product Features

The Message Queue service, and the architecture described in the previous section, fully implement the JMS 1.1 specification for reliable, asynchronous, flexible message delivery. For documentation of JMS compliance-related issues, see Appendix A, "Message Queue Implementation of Optional JMS Functionality."

However, Message Queue has capabilities and features that go far beyond the requirements of the JMS specification. These features enable Message Queue to integrate systems consisting of large numbers of distributed components exchanging many thousands of messages in round-the-clock, mission-critical operations.

Message Queue’s enterprise features, discussed below, are grouped into the following categories:

Integration Support Features

Message Queue allows you to integrate disparate applications and components across an enterprise by including support for several transport protocols, a C client interface to the Message Queue service, support for SOAP (XML) messages, and a pluggable J2EE resource adapter.

Multiple Transport Support

Message Queue supports the ability of clients to interact with the Message Queue message server over a number of different transport protocols, including TCP and HTTP, and using secure connections.

HTTP connections HTTP transport allows messages to be delivered through firewalls. Message Queue implements HTTP support using an HTTP tunnel servlet that runs in a web server environment. Messages produced by a client are delivered over HTTP through a firewall to the tunnel servlet. The tunnel servlet extracts the message from an HTTP request and delivers the message over TCP/IP to the broker. In a similar fashion, Message Queue supports secure HTTP connections using an HTTPS tunnel servlet. For more information on the architecture of HTTP connections, see HTTP/HTTPS Support. For information on setting up and configuring HTTP/HTTPS connections, see the Message Queue Administration Guide.

Secure connections Message Queue provides for secure transmission of messages based on the Secure Socket Layer (SSL) standard over TCP/IP and HTTP transports. These SSL-based connection services allow for the encryption of messages sent between clients and broker.

SSL support is based on self-signed server certificates. Message Queue provides a utility that generates a private/public key pair and embeds the public key in a self-signed certificate. This certificate is passed to any client requesting a connection to the broker, and the client uses the certificate to set up an encrypted connection. For information on creating self-signed certificates to enable SSL-based connections services, see the Message Queue Administration Guide.

C Client Interface

In addition to supporting Java language messaging clients, Message Queue also provides a C language interface to the Message Queue service. The C API enables legacy C applications and C++ applications to participate in JMS-based messaging. However, clients using Message Queue’s C API are not portable to other JMS providers.

Message Queue’s C API is supported by a C client runtime that supports most of the standard JMS functionality, with the exception of the following: the use of administered objects; map, stream, or object message body types; distributed transactions; and queue browsers. The C client runtime also does not support most of Message Queue’s enterprise features.

For more information on the features of the C API and how it implements the JMS programming model with C data types and functions, see the Message Queue Developer’s Guide for C Clients.

SOAP (XML) Messaging Support

Message Queue supports creation and delivery of messages that conform to the Simple Object Access Protocol (SOAP) specification. SOAP allows for the exchange of structured XML data, or SOAP messages, between peers in a decentralized, distributed environment. A SOAP message is an XML document that can also contain an attachment, which does not have to be in XML.

The fact that SOAP messages are encoded in XML makes SOAP messages platform independent. They can be used to access data from legacy systems and share data between enterprises. The data integration offered by XML also makes this technology a natural for Web-based computing, such as Web services. Firewalls can recognize SOAP packets and can filter messages based on information exposed in the SOAP message header.

Message Queue implements the SOAP with Attachments API for Java (SAAJ) specification. SAAJ is an application programming interface that can be implemented to support a programming model for SOAP messaging and to furnish Java objects that you can use to construct, send, receive, and examine SOAP messages. SAAJ defines two packages:

Message Queue provides utilities to transform SOAP messages into JMS messages and vice versa. These utilities allow SOAP messages to be received by a servlet, transformed into a JMS message, delivered by the Message Queue service to a JMS consumer, transformed back into a SOAP message, and delivered to a SOAP endpoint. In other words, Message Queue supports the ability to reliably and asynchronously exchange SOAP messages between SOAP endpoints or, more simply, to publish SOAP messages to Message Queue subscribers.

For additional information, see the Message Queue Developer’s Guide for Java Clients.

J2EE Resource Adapter

The Java 2 Platform, Enterprise Edition (J2EE platform) is a specification for a distributed component model in a Java programming environment. One of the requirements of the J2EE platform is that distributed components be able to interact with one another through reliable, asynchronous message exchange. In short, the J2EE platform requires JMS support.

This support is provided in the J2EE programming model using the message-driven bean (MDB), a specialized type of Enterprise Java Bean (EJB) component that can consume JMS messages. A J2EE-compliant application server must provide an MDB container that supports JMS messaging. This can be achieved by plugging-in a JMS resource adapter into the application server. Message Queue provides such a resource adapter.

By plugging the Message Queue resource adapter into an application server, J2EE components, including MDBs, deployed and running in the application server environment can exchange JMS messages among themselves and with external JMS components. This provides a powerful integration capability for distributed components.

Security Features

Protecting stored and in-transit message data is critical for most enterprise applications. Message Queue provides security at many levels, including authentication of users, controlled access to resources, and message encryption.

Authentication Message Queue supports password-based authentication of users. Connections to the message server are granted to users based on passwords stored in a flat file or LDAP user repository. Information about all connection attempts (users and host computers) is logged and can be tracked.

Authorization Access control lists (ACLs) provide configurable, fine-grained control over access to a broker’s connections and physical destinations. Both user and group access is supported. Authorization is performed on a broker-by-broker basis; each broker can have a different access control file.

Encryption SSL support allows all message traffic between a message server and its clients (whether over TCP/IP or HTTP connections) to be encrypted using a full-strength SSL implementation.

For information on populating a user repository, managing access control lists, and setting up SSL support, see the Message Queue Administration Guide.

Scalability Features

Message Queue allows you to scale your application as users, client connections, and message loads grow.

Scalable Connection Capacity

The Message Queue broker can handle thousands of concurrent connections. By default, each connection is handled by a dedicated broker thread. Because this ties up the thread even when the connection is idle, you can configure the connection service so that multiple connections can share the same thread. This shared threadpool model can dramatically expand the number of connections that a broker can support. For more information, see Thread Pool Manager.

Broker Clusters

As the number of connections and the number of messages being delivered through a broker increases, the extra load can be managed by adding additional broker instances to the Message Queue server. Broker clusters balance client connections and message delivery across a number of broker instances, making the message server highly scalable. The broker instances can be on the same host or distributed across a network. Clustering is an ideal way to improve message throughput and expand messaging bandwidth as business needs grow. Broker clusters are introduced in Chapter 5, "Broker Clusters" and discussed more fully in Message Queue Administration Guide.

Queue Delivery to Multiple Consumers

According to the JMS specification, a message in a queue destination can be delivered only to a single consumer. Message Queue allows multiple consumers to register with a queue. The broker can then distribute messages to the different registered consumers, balancing the load among them and allowing the system to scale.

The implementation of queue delivery to multiple consumers uses a configurable load-balancing approach. Using this approach, you can specify the maximum number of active consumers and the maximum number of backup consumers standing by to take the place of active consumers should any fail. In addition, the load-balancing mechanism takes into account a consumer’s current capacity and message processing rate.

Availability Features

Message Queue provides a number of features for minimizing service downtime. These range from mechanisms for preventing failure to those that allow integration with Sun Cluster to provide high availability.

Message Service Stability

One of the most effective ways of ensuring availability of a message service is to provide a service that offers high performance and minimizes failure. Message Queue provides mechanisms for averting memory overloads or performance logjams. These operate on both the message server and client runtime.

Message server resource management Because the message server is limited in memory and CPU resources, it is possible for it to become overloaded to the point where it becomes unresponsive or unstable. This commonly happens when the rate of message production far exceeds the rate of consumption. To avert such situations, a broker can be configured on the level of individual physical destinations and on a system-wide level to prevent memory overruns. For more information, see Memory Resource Management.

Client runtime message flow control In addition, Message Queue provides mechanisms for controlling the delivery of messages to the client runtime. You can use flow control mechanisms to optimize the delivery of messages to the client runtime while preventing the client from running out of memory. For more information, see Message Flow Control.

Automatic Reconnect to Message Server

Message Queue provides an automatic reconnect capability: If a connection between a message server and client fails, Message Queue maintains the client state while attempting to reestablish the connection. In most cases, message production and consumption will transparently resume once the connection is re-established. for more information, see Message Queue Administration Guide.

High Availability Through Sun Cluster

While Message Queue’s broker clustering provides a highly scalable message server, it does not currently support failover from one broker instance in a cluster to another. However, Message Queue can be integrated with Sun Cluster software to provide a high-availability message server. Using a Sun Cluster agent developed for Message Queue, Sun Cluster can ensure that no state data is lost if a broker fails, allowing a message server to be restored immediately and transparently with virtually no downtime.

Manageability Features

Message Queue provides a number of features that you can use to monitor and administer a message service and to tune message service performance.

Robust Administration Tools

Message Queue offers both command line and GUI tools for administering a Message Queue message server and for managing destinations, transactions, durable subscriptions, and security (see Administration Tools).

Message Queue also supports remote monitoring and administration of message servers as well as tools for managing JMS administered objects, user repositories, plugged-in JDBC-compliant data stores, and self-signed server certificates. For information on using these administration tools, see the Message Queue Administration Guide.

Message-Based Monitoring API

Message Queue provides a simple JMS-based monitoring API that you can use to create custom monitoring applications. These monitoring applications are consumers that retrieve metrics messages from special topic destinations. The metrics messages contain monitoring data provided by the Message Queue broker (see Metrics Message Producer (Enterprise Edition)).

For details of the metrics quantities reported in each type of metrics message, see the Message Queue Developer’s Guide for Java Clients, which explains how to develop a Message Queue client for consuming metrics messages. For information about how to configure the production of metrics messages, see the Message Queue Administration Guide.

Tunable Performance

Message Queue offers many ways to tune both the message server and the client runtime to achieve optimal performance. You can monitor key resources and adjust memory usage, threading resources, message flow, connection services, reliability parameters, and other elements that affect message throughput and system performance. For details about how to tune message service performance, see the Message Queue Administration Guide.

Flexible Server Configuration Features

Message Queue allows you to choose how persistent objects, user information, and administered objects are stored.

Configurable Persistence

In order to guarantee delivery of messages, Message Queue stores messages and other persistent objects until messages are consumed. In addition to providing a high performance file-based persistent store, Message Queue also supports configurable persistence. This allows you to store persistent messages in embedded or external JDBC-compliant databases, such as Oracle 8i. For more information, see Persistence Manager.

LDAP Server Support

Message Queue provides file-based storage for both administered objects and user information needed for authentication and authorization. However, Message Queue also supports using LDAP servers for administered object stores and user repositories. LDAP servers provide a more secure, standard way of storing and retrieving such information, and are recommended for production systems. For information on using LDAP servers for administered object stores and user repositories, see the Message Queue Administration Guide.

Product Editions

Message Queue is available in two editions: Enterprise and Platform. Both editions provide a full implementation of the JMS specification, but each corresponds to a different feature set and licensed capacity. The feature sets are compared in the following table. For a description of the features, see Product Features.

Table 2-2 Feature Comparison: Enterprise and Platform Editions
Enterprise Edition	Platform Edition
Advanced Integration Support Features
HTTP support, TCP support	TCP support
Secure connections based on Secure Socket Layer (SSL) standard	Secure connections based on Secure Socket Layer (SSL) standard
C language API, Java API	Java API (The C API can only be used with the trial license.)
SOAP (XML) messaging support	SOAP (XML) messaging support
J2EE resource adapter	J2EE resource adapter
Security Features
Authentication from either a flat file or LDAP user repository, authorization using an access control file, and SSL encryption.	Same as Enterprise Edition
Scalability Features
Scalable connection capacity	Fixed connection capacity
Message server can be implemented as a broker cluster	Single-broker message server
Queue delivery to unlimited number of message consumers (per queue)	Queue delivery to a maximum of three message consumers (per queue)
Availability Features
Message service stability through memory resource management and message flow control	Same as Enterprise Edition
Client connection failover to a different broker in a cluster or automatic reconnect to the same broker.	No client connection failover. Automatic reconnect to same broker allowed.
High availability through Sun Cluster	High availability through Sun Cluster
Manageability Features
Robust administration tools	Robust administration tools
Message-based monitoring API in addition to administration tools and logging	Administration tools and logging, but no message-based monitoring API
Tunable performance	Tunable performance
Flexible Server Configuration Features
Pluggable Persistence	Pluggable Persistence
LDAP server support	LDAP server support

The license capacities of the Platform and Enterprise Editions are described below.

Enterprise Edition

The Message Queue enterprise edition allows you to deploy and run messaging applications in an enterprise production environment. You can also use it for developing, debugging, and load-testing messaging applications and components. The Enterprise Edition has an unlimited duration license based on the number of CPUs that are used. The license places no limit on the number of brokers in a multi-broker message service.

Platform Edition

The Message Queue Platform Edition places no limit on the number of client connections supported by the message server. It comes with a basic license or a 90-day trial license:

The Platform Edition can be downloaded free from the Sun web site and is also bundled with the Sun Java System Application Server platform. Instructions for upgrading Message Queue from Platform Edition to Enterprise Edition can be found in the Message Queue Installation Guide.


Note	For all editions of Message Queue, a portion of the product—the Message Queue client runtime—can be freely redistributed for commercial use. All other files in the product cannot be redistributed. The portion that can be freely redistributed allows a licensee to develop a Message Queue client application (one which can be connected to a Message Queue service) that can be sold to a third party without incurring any licensing fees. The third party will either need to purchase Message Queue to access a Message Queue message server or make a connection to yet another party that has a Message Queue message server installed and running.

Message Queue in a Sun Product Context

Besides being the middleware directly used by applications, Message Queue is also used by other middleware as well as by other servers and applications delivered by Sun. To facilitate this, Message Queue is delivered in Solaris and Java Enterprise System as well as being delivered in the Sun Java System Application Server.

In the Application Server, Message Queue satisfies the JMS requirement that the J2EE platform furnish a JMS provider. It is used directly by the applications that are hosted by the Application Server. For more information, see Chapter 6, "Message Queue and J2EE".

Previous Contents Index Next
Sun Java System Message Queue 3 2005Q1 Technical Overview