Chapter 1 Messaging Systems: An Introduction

Sun Java™ System Message Queue is a messaging middleware product that implements and extends the Java Message Service (JMS) standard. If this statement makes perfect and deep sense to you, you should start by reading the section Message Queue: Elements and Features. Otherwise, you should begin at the beginning.

This chapter describes the messaging technology that underlies products like Message Queue and explains how Message Queue implements and extends the JMS specification, which standardizes this technology. It covers the following topics:

Message-Oriented Middleware (MOM)

Because businesses, institutions, and technologies change continually, the software systems that serve them must be able to accommodate such changes. Following a merger, the addition of a service, or the expansion of available services, a business can ill afford to recreate its information systems. It is at this most critical point that it needs to integrate new components or to scale existing ones as efficiently as possible. The easiest way to integrate heterogeneous components is not to recreate them as homogeneous elements but to provide a layer that allows them to communicate despite their differences. This layer, called middleware, allows software components (applications, enterprise java beans, servlets, and other components) that have been developed independently and that run on different networked platforms to interact with one another. It is when this interaction is possible that the network can become the computer.

As shown in Figure 1-1, conceptually, middleware resides between the application layer and the platform layer (the operating system and underlying network services).

Applications distributed on different network nodes use the application interface to communicate without having to be concerned with the details of the operating environments that host other applications nor with the services that connect them to these applications. In addition, by providing an administrative interface, this new, virtual system of interconnected applications can be made reliable and secure. Its performance can be measured and tuned, and it can be scaled without losing function.

Remote Procedure Call or RPC-based middleware, which allows procedures in one application to call procedures in remote applications as if they were local calls. The middleware implements a linking mechanism that locates remote procedures and makes these transparently available to a caller. Traditionally, this type of middleware handled procedure-based programs; it now also includes object-based components.

Object Request Broker or ORB-based middleware, which enables an application’s objects to be distributed and shared across heterogeneous networks.

Message Oriented Middleware or MOM-based middleware, which allows distributed applications to communicate and exchange data by sending and receiving messages.

All these models make it possible for one software component to affect the behavior of another component over a network. They are different in that RPC- and ORB-based middleware create systems of tightly-coupled components, whereas MOM-based systems allow for a looser coupling of components. In an RPC- or ORB-based system, when one procedure calls another, it must wait for the called procedure to return before it can do anything else. As we mentioned before, in these models, the middleware functions partly as a super-linker, locating the called procedure on a network and using network services to pass function or method parameters to the procedure and then to return results.

MOM-based systems allows communication to happen through the asynchronous exchange of messages, as shown in Figure 1-2.

Message Oriented Middleware makes use of messaging provider to mediate messaging operations. The basic elements of a MOM system are clients, messages, and the MOM provider, which includes an API and administrative tools. The MOM provider an use different architectures to route and deliver messages: it can use a centralized message server or it can distribute routing and delivery functions to each client machine. Some MOM products combine these two approaches.

Using a MOM system, a client makes an API call to send a message to a destination managed by the provider. The call invokes provider services to route and deliver the message. Once it has sent the message, the client can continue to do other work, confident that the provider retains the message until a receiving client retrieves it. The message-based model, coupled with the mediation of the provider, makes it possible to create a system of loosely-coupled components. Such a system can continue to function reliably, without downtime, even when individual components or connections fail.

One other advantage of having a messaging provider mediate messaging between clients is that by adding an administrative interface, you can monitor and tune performance. Client applications are thus effectively relieved of every problem except that of sending, receiving, and processing messages. It is up to the code that implements the MOM system and up to the administrator to resolve issues like interoperability, reliability, security, scalability, and performance.

So far we have described the advantages of connecting distributed components using message-oriented middleware. There are also disadvantages: one of them results from the loose coupling itself. With an RPC system, the calling function does not return until the called function has finished its task. In an asynchronous system, the calling client can continue to load work upon the recipient until the resources need to handle this work are depleted and the called component fails. Of course, these conditions can be minimized or avoided by monitoring performance and adjusting message flow, but this is work that is not needed with an RPC system. The important thing is to understand the advantages and liabilities of each kind of system. Each system is appropriate for different tasks. Sometimes, you will need to combine the two kinds of systems to obtain the exact behavior you need.

Figure 1-3 shows the way a MOM system can enable communication between two RPC-based systems. The left side of the figure shows an application that distributes client, server, and data store components on different networked nodes for improved performance. This is a discount airline reservation system: an end user pays a fee to use this service, which allows it to find the lowest available fare for given destinations and times. The data store holds information about registered users and about airlines that participate in this program. Based on the user’s request, logic on the server queries participating airlines for prices, sorts through the information, and presents the three lowest bids to the user. The right side of the picture shows an RPC-based system that represents the ticket/reservation system for any one of the participating airlines. The right side of the picture would be replicated for as many airlines as the discounter is connected to. For each such airline, the data store would hold information about available flights (seating, flight times, and prices). The server component would update that information in response to data input by the end user. The airline server also subscribes to the MOM service, accepting requests for information from the discount reservation system and returning seating and pricing information. If a customer decides to purchase a discounted ticket on a PanWorld flight, the server component for that system would update the information in the data store and then either generate a ticket for the requester or send a message to the discounting service to generate the ticket.

This example illustrates some of the differences between RPC and MOM systems. We have already mentioned the difference in the way in which distributed components are coupled. Another difference is that while RPC systems are often used to distribute and connect client and server components in which the client is often an end-user, with MOM systems, clients are often heterogeneous software components that can only interoperate by means of messaging.

A more serious problem with MOM systems arises from the fact that MOMs are implemented as proprietary products. What happens when your company, which depends on SuperMOM-X acquires a company that uses SuperMOM-Y? To resolve this problem, a standard messaging interface is needed. If both SuperMOM-X and SuperMOM-Y implemented this interface, then applications developed to run on one system could also run on the other. Such an interface should be simple to learn but provide enough features to support sophisticated messaging applications. The Java Message Service (JMS) specification, introduced in 1998, aimed to do just that. The next section describes the basic features of JMS and explains how the standard was developed to embrace common elements of existing proprietary MOM products as well as to allow for differences and further growth.

JMS as a MOM Standard

The Java Messaging Service specification was originally developed to allow Java applications access to existing MOM systems. Since its introduction, it has been adopted by many existing MOM vendors and it has been implemented as an asynchronous messaging system in its own right.

In creating the JMS specification, its designers wanted to capture the essential elements of existing messaging systems. These included

The concept of a messaging provider that routes and delivers messages.

Distinct messaging patterns or domains such as point-to-point messaging and publish/subscribe messaging

Facilities for synchronous and asynchronous message receipt

Support for reliable message delivery

Common message formats such as stream, text, and byte

Vendors implement the JMS specification by supplying a JMS provider consisting of libraries that implement the JMS interfaces, of functionality for routing and delivering messages, and of administrative tools that manage, monitor, and tune the messaging service. Routing and delivery functions can be performed by a centralized message server or broker, or they could be implemented through functionality that is part of each client’s runtime.

Equally, a JMS provider can play a variety of roles: it can be created as a stand-alone product or as an embedded component in a larger distributed runtime system. As a standalone product, it could be used to define the backbone of an enterprise application integration system; embedded in an application server, it could support inter-component messaging. For example, J2EE uses a JMS provider to implement message-driven beans and to allow EJB components to send and receive messages.

To have created a standard that included all features of existing systems would have resulted in system that was hard to learn and difficult to implement. Instead, JMS defined a least common denominator of messaging concepts and features. This resulted in a standard that is easy to learn and that maximizes the portability of JMS applications across JMS providers. It’s important to note that JMS is an API standard, not a protocol standard. It is easy to move a JMS client from one vendor to another. But different JMS vendors typically cannot communicate directly with one another.

The next section describes the basic objects and messaging patterns defined by the JMS specification.

JMS Messaging Objects and Patterns

In order to send or receive messages, a JMS client must first connect to a JMS provider which is often implemented as a message broker: the connection opens a channel of communication between the client and the broker. Next, the client must set up a session for creating, producing, and consuming messages. You can think of the session as a stream of messages defining a particular conversation between the client and the broker. The client itself is a message producer and/or a message consumer. The message producer sends a message to a destination that the broker manages. The message consumer accesses that destination to consume the message. The message includes a header, optional properties, and a body. The body holds the data; the header contains information the broker needs to route and manage the message; and the properties can be defined by client applications or by a provider to serve their own needs in processing messages. Connections, sessions, destinations, messages, producers, and consumers are the basic objects that make up a JMS application.

Using these basic objects, a client application can use two messaging patterns (or domains) to send and receive messages. These are shown in Figure 1-4.

Clients A and B are message producers, sending messages to clients C, D, and E by way of two different kinds of destinations.

Messaging between clients A, C, and D illustrates the point-to-point pattern. Using this pattern, a client sends a message to a queue destination from which only one receiver may get it. No other receiver accessing that destination can get that message.

Messaging between clients B, E, and F illustrates the publish/subscribe pattern. Using this broadcast pattern, a client sends a message to a topic destination from which any number of consuming subscribers can retrieve it. Each subscriber gets its own copy of the message.

Message consumers in either domain can choose to get messages synchronously or asynchronously. Synchronous consumers make an explicit call to retrieve a message; asynchronous consumers specify a callback method that is invoked to pass a pending message. Consumers can also filter out messages by specifying selection criteria for incoming messages.

Administered Objects

The JMS specification created a standard that combined many elements of existing MOM systems without attempting to exhaust all possibilities. Rather, it sought to set up an extensible scheme that could accommodate differences and future growth. JMS leaves a number of messaging elements up to the individual providers to define and implement. These include load balancing, standard error messages, administrative APIs, security, the underlying wire protocols, and message stores. The next section, Message Queue: Elements and Features, describes how Message Queue implements many of these elements and how it extends the JMS specification.

Two messaging elements that JMS does not completely define are connection factories and destinations. Although these are fundamental elements in the JMS programming model, there were so many existing and anticipated differences in the ways in which providers define and manage these objects, that it was neither possible nor desirable to create a common definition. Therefore, these two objects, rather than being created programmatically, are normally created and configured using administration tools. They are then stored in an object store, and accessed by a JMS client through standard JNDI lookups.

Connection factory administered objects are used to generate a client’s connections to the broker. They encapsulate provider-specific information that governs certain aspects of messaging behavior: connection handling, client identification, message header overrides, reliability, and flow control, etc. Every connection derived from a given connection factory exhibits the behavior configured for that factory.

Destination administered objects are used to reference physical destinations on the broker. They encapsulate provider-specific naming (address-syntax) conventions and they specify the messaging domain within which the destination is used: queue or topic.

JMS clients are not required to look up administered objects; they can create these objects programmatically (which are then stored in the broker’s memory). For quick prototyping, creating these objects programmatically might be easiest. But for deployment in a production environment, looking up administered objects in a central repository makes it much easier to control and manage messaging behavior:

By using administrative objects for connection factory objects, administrators can tune messaging performance by reconfiguring these objects. Performance can be improved without having to recode.

By using administrative objects for physical destinations, administrators can control the proliferation of these destinations on the broker by requiring clients to access these preconfigured objects.

Administered objects shield developers from provider-specific implementation details and allow the code they develop for one provider to be portable to other providers with little or no change.

The use of administered objects adds a final wrinkle to the picture of the basic JMS application, which is shown in Figure 1-5.

Figure 1-5 shows how a message producer and a message consumer use a destination administered object to access the physical destination to which it corresponds. The marked steps denote the actions that need to be taken by the administrator and by the client applications to send and receive messages using this mechanism.

The administrator creates a physical destination on the broker.

The administrator creates a destination administered object and configures it by specifying the name of the physical destination to which it corresponds and its type: queue or topic.

The message producer looks up the destination administered object using a JNDI lookup call.

The message producer sends a message to the destination.

The message consumer looks up the destination administered object where it expects to get messages.

The message consumer gets the message from the destination.

The process of using connection factory administered objects is similar. The administrator creates and configures a connection factory administered object using administration tools. The client looks up the connection factory object and uses it to create a connection.

Although the use of administered objects adds a couple of steps to the messaging process, it also adds robustness and portability to messaging applications.

Message Queue: Elements and Features

So far we have described the elements of message-oriented middleware and the use of JMS as a way of adding portability to MOM applications. It now remains to describe how Message Queue implements the JMS specification and to introduce the features and tools it uses to provide reliable, secure, and scalable messaging service.

First, like many JMS provider, Message Queue can be used as a stand-alone product or it can be used as an enabling technology, embedded in a J2EE application server to provide asynchronous messaging. Chapter 5, "Message Queue and J2EE," describes the role Message Queue plays in J2EE in greater detail. Unlike other JMS providers, Message Queue has been designated as the JMS reference implementation. This designation attests to the fact that Message Queue is a correct and complete JMS implementation. It also guarantees that the Message Queue product will remain current with any future JMS revisions and extensions.

The Message Queue Service

As a JMS provider, Message Queue offers a messaging service that implements the JMS interfaces and that provides administrative services and control. So far, in illustrating JMS providers, we have focused mainly on the role of the broker in relaying messages. But in fact, a JMS provider must includes many elements in addition to the broker to provide reliable, secure, scalable messaging. Figure 1-6 shows the elements that make up the Message Queue message service. These include a variety of connection services (supporting different protocols), administrative tools, and data stores for messaging, monitoring, and user information. The Message Queue service includes all elements marked in gray in the figure.

As you can see, a full-featured JMS provider is more complex than the basic JMS model would lead one to suspect. The following sections describe the elements of the Message Queue service shown above. These elements can be divided into three categories: the broker, client runtime support, and administration.

Connecting to the Broker

As shown in Figure 1-6 both application clients and administration clients can connect to the broker. The JMS specification does not dictate that providers implement any specific wire protocols. Message Queue services, used by application clients and administration clients to connect to the broker, are currently layered on top of TCP, TLS, HTTP, or HTTPS protocols. (Services layered on top of HTTP allow messages to pass through firewalls.)

Services that provide JMS support and allow clients to connect to the broker (jms, ssljms, http, or https) have a service type of NORMAL and are layered on top of TCP, TLS, HTTP, or HTTPS protocols.

Services that allow administrators to connect to the broker (admin, ssladmin) have a service type of ADMIN and are layered on top of TCP or TLS protocols.

By default, when you start the broker, jms and admin services are up and running. Additionally, you can configure a broker to run any or all of these connection services. Each service supports specific authentication and authorization (access control) features and each service is multi-threaded, supporting multiple connections.

Should a connection fail, the Message Queue service can automatically retry connecting the client to the same broker or to a different broker if this feature is enabled. For more information, see the description of the automatic reconnect feature in Appendix B, "Message Queue Features."

Clients can configure connection runtime support when they create the connection factory from which they obtain their connections. Options allow you to specify which brokers to connect to, how to handle reconnection, message flow control, and so on. For additional information about how connections can be configured, see Connection Factories and Connections.

The Broker

At the heart of the message service is the broker, which routes and delivers messages reliably, authenticates users, and gathers data for monitoring performance.

To route and deliver messages, the broker places incoming messages in their respective destinations and manages message flow into and out of these destinations.

To provide reliable delivery, the broker uses a persistent store to save state information and persistent messages until they are received. Should the broker or the connection fail, the saved information allows the broker to restore the broker’s state and to retry operations.

To provide security for the data being exchanged the broker uses authenticated connections. Optionally data may be encrypted by running over a secure protocol like SSL. The broker also uses and manages a repository that holds information about users and the data or operations they can access. The broker authenticates users requesting services and authorizes the operations they want to carry out by looking up information in this repository.

To monitor the system, the broker generates metrics and diagnostic information that an administrator can access to measure performance and to tune the broker. Metrics information is also available programmatically to allow applications to adjust message flow and patterns to improve performance.

The Message Queue service provides a variety of administrative tools that the administrator can use to configure broker support. For more information, see Administration.

Client Runtime Support

Client runtime support is provided in libraries that you link with when building Message Queue clients. You can think of the client runtime as the bit of the Message Queue service that becomes part of the client. For example, when client code makes an API call to send a message, code in these libraries is invoked that packages the message bits appropriately for the protocol that will be used to relay the message to the physical destination on the broker.

Java and C Client Support

A JMS provider is only required to support Java clients; however, as Figure 1-6 shows, a Message Queue client can use either the Java or a provider-specific C API to send or receive a message. These interfaces are implemented in Java or C runtime libraries, which do the actual work of creating connections to the broker and packaging the bits appropriately for the connection service requested.

The Java client runtime supplies Java clients with the objects needed to interact with the broker. These objects include connections, sessions, messages, message producers, and message consumers.

The C client runtime supplies C clients with the functions and structures needed to interact with the broker. It supports a procedural version of the JMS programming model. C clients cannot use JNDI to access administered objects, but can create connection factories and destinations programmatically.

The Message Queue service provides a C API to enable legacy C and C++ applications to participate in JMS-based messaging. There are a number of differences in the functionality provided by these two APIs; these are documented in Java and C Clients.

It is important to remember that the JMS specification is a standard for Java clients only. C support is specific to the Message Queue provider and should not be used in client applications that you plan to port to other providers.

SOAP Support for Java Clients

Message Queue Java clients are also able to send and receive SOAP messages, wrapped as JMS messages. SOAP (Simple Object Access Protocol) allows the exchange of structured data between two peers in a distributed environment. The data exchanged is specified by an XML scheme.

Sun SOAP processing is currently limited to using a point-to-point model and does not guarantee reliability. By wrapping a SOAP message in a JMS message and routing it using the broker, you can take advantage of full featured Message Queue messaging which guarantees reliable delivery and allows you to use the topic as well as the point-to-point domain. Message Queue provides utility routines that a message producer can use to wrap a SOAP message into a JMS message and that a message consumer can use to extract a SOAP message from the JMS message.

Administration

The Message Queue service offers command line tools that you can use to do the following:

Start and configure the broker.

Create and manage destinations, manage broker connections, and manage broker resources.

Add, list, update, and deleted administered objects in a JNDI object store.

Populate and manage a file-based user repository.

Create and manage a JDBC compliant database for persistent storage.

You can also use a GUI-based administration console to perform the following command-line functions:

Scaling the Message Queue Service

As the number of clients or the number of connections grows, you may need to scale the message service to eliminate bottlenecks or to improve performance. The Message Queue message service offers a number of scaling options, depending on your needs. These may be conveniently sorted into the following categories:

If you are using the point-to-point domain, you can scale the consumer side by allowing multiple consumers to access a queue. Using this approach, you can specify the maximum number of active and backup consumers. The load-balancing mechanism also takes into account a consumer’s current capacity and message processing rate. This is a Message Queue feature. (The JMS specification defines messaging behavior if only one consumer is accessing a queue; behavior for queues allowing more than one consumer is provider-specific. The Message Queue developer guides provide more information about this scaling option.)

Stateless horizontal scaling is achieved by using additional brokers and redistributing existing clients to these brokers. This approach is easy to implement, but it is appropriate only if your messaging operations can be divided into independent work groups.

Stateful horizontal scaling is achieved by connecting brokers into a cluster. In a broker cluster, each broker is connected to every other broker in the cluster as well as to its local application clients. Brokers can be on the same host or distributed across a network. Information about destinations and consumers is replicated on all the brokers in the cluster. Updates to destinations or subscribers is also propagated Each broker can therefore route messages from producers to which it is directly connected to consumers that are connected to other brokers in the cluster. In situations where backup consumers are used, if one broker or connection fails, messages sent to inaccessible consumers can be forwarded to a backup consumers on another broker.

In the event of broker or connection failure, state information about persistent entities (destinations and durable subscriptions) can get out of sync. For example, if a clustered broker goes down and a destination is created on another broker in the cluster, when the first broker restarts, it will not know about the new destination. To forestall this problem, you can designate one broker in the cluster to be the master broker. This broker is responsible for tracking all changes to destinations and durable subscriptions in a master configuration file and for updating brokers in the cluster that are temporarily offline. For additional information, see Chapter 4, "Broker Clusters."

Even when using a master broker, Message Queue only provides service availability, not data availability in the case of broker or connection failure. For example, if a clustered broker becomes unavailable, any persistent messages held by that broker become unavailable until that broker recovers. Currently, the only means to assure data availability is through the use of a SunCluster Message Queue agent. In this case, a persistent store is kept on a shared file system. If a broker fails the Message Queue agent on a second node starts a broker that takes over the shared store. Clients are reconnected to that broker, thereby getting both continuous service and access to persistent data.

Message Queue as an Enabling Technology

The Java 2 Platform, Enterprise Edition (J2EE platform) is a specification for a distributed component model in a Java programming environment. One of the requirements of the J2EE platform is that distributed components be able to interact with one another through reliable, asynchronous message exchange. This capacity is furnished by a JMS provider, which can play two roles: it can be used to provide a service and it can support message-driven beans (MDB), a specialized type of Enterprise Java Bean (EJB) component that can consume JMS message.

A J2EE-compliant application server must use a resource adapter furnished by a given JMS provider to use the functionality of that provider. Message Queue provides such a resource adapter. Using the support of a plugged in JMS provider, J2EE components, including MDBs, deployed and running in the application server environment can exchange JMS messages among themselves and with external JMS components. This provides a powerful integration capability for distributed components.

Product Editions

Message Queue is available in two editions: Enterprise and Platform. Both editions provide a full implementation of the JMS specification, but each corresponds to a different feature set and licensed capacity.

HTTP and HTTPS support

C client support

Scalable connection capacity

Broker clusters

Queue delivery to an unlimited number of message consumers per queue. (Platform Edition limits delivery to three consumers per queue.)

Client connection failover to a different broker in a cluster.

Message-based monitoring API

The Message Queue Enterprise Edition has an unlimited duration license based on the number of CPUs that are used. The license places no limit on the number of brokers in a multi-broker message service.

The Message Queue Platform Edition places no limit on the number of client connections supported by the broker. It comes with a basic license or a 90-day trial license:

A basic license has an unlimited duration, but does not include Enterprise Edition features.

A 90-day trial enterprise license allows you to use and evaluate all Enterprise Edition features but has a limited 90-day duration enforced by the software. For instructions on using this license, see the startup options discussed in the Message Queue Administration Guide.

For further information about licensed features and capacity, redistribution rights, and upgrading Platform Edition to Enterprise Edition, please see Message Queue Installation Guide.

Message Queue Feature Summary

Message Queue has capabilities and features that go far beyond the requirements of the JMS specification. These features enable Message Queue to integrate systems consisting of large numbers of distributed components exchanging many thousands of messages in round-the-clock, mission-critical operations. For a summary of these features, see Appendix B, "Message Queue Features."

Previous Contents Index Next
Sun Java System Message Queue 3 2005Q4 Technical Overview