|
| Sun ONE Message Queue, Version 3.0.1 Administrator's Guide |
This chapter provides an introduction to Sun™ ONE Message Queue (MQ) and is of interest to both administrators and programmers.
What Is Sun ONE Message Queue?
The MQ product is a standards-based solution to the problem of inter-application communication and reliable message delivery. MQ is an enterprise messaging system that implements the Java Message Service (JMS) open standard: it is a JMS provider.
The JMS specification describes a set of programming interfaces (see "JMS Programming Model")—which provide a common way for Java applications to create, send, receive, and read messages in a distributed environment.
With Sun ONE Message Queue software, processes running on different platforms and operating systems can connect to a common MQ message service (see "Message Service Architecture") to send and receive information. Application developers are free to focus on the business logic of their applications, rather than on the low-level details of how their applications communicate across a network.
MQ has features which exceed the minimum requirements of the JMS specification. Among these features are the following:
Centralized administration Provides both command-line and GUI tools for administering an MQ message service and managing application-specific aspects of messaging, such as destinations and security.
Scalable message service Allows you to service increasing numbers of JMS clients (components or applications) by balancing the load among a number of MQ message service components (brokers) working in tandem (multi-broker cluster).
Tunable performance Lets you increase performance of the MQ message service when less reliability of delivery is acceptable.
Multiple transports Supports the ability of JMS clients to communicate with each other over a number of different transports, including TCP and HTTP, and using secure (SSL) connections.
JNDI support Supports both file-based and LDAP implementations of the Java Naming and Directory Interface (JNDI) as object stores and user repositories.
SOAP messaging support Supports creation and delivery of SOAP messages—messages that conform to the Simple Object Access Protocol (SOAP) specification— via JMS messaging. SOAP allows for the exchange of structured XML data between peers in a distributed environment. See the MQ Developer's Guide for more information.
See the MQ 3.0.1 Release Notes for documentation of JMS compliance-related issues.
Product Editions
The Sun ONE Message Queue product is available in two editions: Platform and Enterprise—each corresponding to a different licensed capacity, as described below. (To upgrade MQ from one edition to another, see the instructions in the MQ Installation Guide.)
Platform Edition
This edition can be downloaded free from the Sun website and is also bundled with the latest Sun ONE Application Server platform. The Platform Edition places no limit on the number of JMS client connections supported by each MQ message service. It comes with two licenses, as described below:
- a basic license This license provides basic JMS support (it's a full JMS provider), but does not include such enterprise features as load balancing (multi-broker message service), HTTP/HTTPS connections, secure connection services, scalable connection capability, and multiple queue delivery policies. The license has an unlimited duration, and can therefore be used in less demanding production environments.
- a 90-day trial enterprise license This license includes all enterprise features (such as support for multi-broker message services, HTTP/HTTPS connections, secure connection services, scalable connection capability, and multiple queue delivery policies) not included in the basic license. However, the license has a limited 90-day duration enforced by the software, making it suitable for evaluating the enterprise features available in the Enterprise Edition of the product (see "Enterprise Edition").
Note The 90-day trial license can be enabled by starting the MQ message service—an MQ broker instance—with the -license command line option (see Table 5-2) and passing "try" as the license to use:
You must use this option each time you start the broker instance, otherwise it defaults back to the basic Platform Edition license.
Enterprise Edition
This edition is for deploying and running messaging applications in a production environment. It includes support for multi-broker message services, HTTP/HTTPS connections, secure connection services, scalable connection capability, and multiple queue delivery policies. You can also use the Enterprise Edition for developing, debugging, and load testing messaging applications and components. The Enterprise Edition has an unlimited duration license that places no limit on the number of brokers in a multi-broker message service, but specifies the number of CPU's that are supported.
Enterprise Messaging Systems
Enterprise messaging systems enable independent distributed components or applications to interact through messages. These components, whether on the same system, the same network, or loosely connected through the Internet, use messaging to pass data and to coordinate their respective functions.
Requirements of Enterprise Messaging Systems
Enterprise application systems typically consist of large numbers of distributed components exchanging many thousands of messages in round-the-clock, mission-critical operations. To support such systems, an enterprise messaging system must generally meet the following requirements:
Reliable delivery Messages from one component to another must not be lost due to network or system failure. This means the system must be able to guarantee that a message is successfully delivered.
Asynchronous delivery For large numbers of components to be able to exchange messages simultaneously, and support high density throughputs, the sending of a message cannot depend upon the readiness of the consumer to immediately receive it. If a consumer is busy or offline, the system must allow for a message to be sent and subsequently received when the consumer is ready. This is known as asynchronous message delivery, popularly known as store-and-forward messaging.
Security The messaging system must support basic security features: authentication of users, authorized access to messages and resources, and over-the-wire encryption.
Scalability The messaging system must be able to accommodate increasing loads—increasing numbers of users and increasing numbers of messages—without a substantial loss of performance or message throughput. As businesses and applications expand, this becomes a very important requirement.
Manageability The messaging system must provide tools for monitoring and managing the delivery of messages and for optimizing system resources. These tools help measure and maintain reliability, security, and performance.
Centralized vs. Peer to Peer Messaging
The requirements of an enterprise messaging system are difficult to meet with a traditional peer to peer messaging system, illustrated in Figure 1-1.
Figure 1-1 Centralized vs. Peer to Peer Messaging
In such a system every messaging component maintains a connection to every other component. These connections can allow for fast, secure, and reliable delivery, however the code for supporting reliability and security must reside in each component. As components are added to the system, the number of connections rises exponentially. This makes asynchronous message delivery and scalability difficult to achieve. Centralized management is also problematic.
The preferred approach for enterprise messaging is a centralized messaging system, also illustrated in Figure 1-1. In this approach each messaging component maintains a connection to one central message service. The message service provides for routing and delivery of messages between components, and is responsible for reliable delivery and security. Components interact with the message service through a well-defined programming interface. As components are added to the system, the number of connections rises only linearly, making it easier to scale the system by scaling the message service. In addition, the central message service provides for centralized management of the system.
Messaging System Concepts
A few basic concepts underlie enterprise messaging systems. These include the following: message, message service architecture, and message delivery models.
Message
A message consists of data in some format (message body) and meta-data that describes the characteristics or properties of the message (message header), such as its destination, lifetime, or other characteristics determined by the messaging system.
Message Service Architecture
The basic architecture of a messaging system is illustrated in Figure 1-2. It consists of message producers and message consumers that exchange messages by way of a common message service. Any number of message producers and consumers can reside in the same messaging component (or application). A message producer sends a message to a message service. The message service, in turn, using message routing and delivery components, delivers the message to one or more message consumers that have registered an interest in the message. The message routing and delivery components are responsible for guaranteeing delivery of the message to all appropriate consumers.
Figure 1-2 Message Service Architecture
Message Delivery Models
There are many relationships between producers and consumers: one to one, one to many, and many to many relationships. For example, you might have messages delivered from:
- one producer to one consumer
- one producer to many consumers
- many producers to one consumer
- many producers to many consumers.
These relationships are often reduced to two message delivery models: point-to-point and publish/subscribe messaging. The focus of the point-to-point delivery model is on messages that originate from a specific producer and are received by a specific consumer. The focus of publish/subscribe delivery model is on messages that originate from any of a number of producers and are received by any number of consumers. These message delivery models can overlap.
Historically, messaging systems supported various combinations of these two message delivery models. The Java Message Service (JMS) API was intended to create a common programming approach for Java messaging. It supports both the point-to-point and publish/subscribe message delivery models (see "Programming Domains").
The JMS Specification
JMS specifies a message structure, a programming model, and a set of rules and semantics that govern messaging operations. Because MQ provides an implementation of JMS, JMS concepts are fundamental to understanding how an MQ messaging system works. This introduction explains concepts and terminology needed to understand the remaining chapters of this book.
JMS Message Structure
According to the JMS specification, a message is composed of three parts: a header, properties, and a body.
Header The header specifies the JMS characteristics of the message: its destination, whether it is persistent or not, its time to live, and its priority. These characteristics govern how the messaging system delivers the message.
Properties Properties (which can be thought of as an extension of the header) are optional—they provide values that applications can use to filter messages according to various selection criteria. Properties are optional.
Message body The message body contains the actual data to be exchanged. JMS supports six body types.
JMS Programming Model
In the JMS programming model, JMS clients (components or applications) exchange messages by way of a JMS message service. Message producers send messages to the message service, from which message consumers receive them. These messaging operations are performed using a set of objects (furnished by a JMS provider) that implement the JMS application programming interface (API).
This section introduces the objects that implement the JMS API and that are used to set up a JMS client for delivery of messages (for more information, see the MQ Developer's Guide). Figure 1-3 shows the JMS objects used to program the delivery of messages.
Figure 1-3 JMS Programming Objects
In the JMS programming model, a JMS client uses a ConnectionFactory object to create a connection over which messages are sent to and received from the JMS message service. A Connection is a JMS client's active connection to the message service. Both allocation of communication resources and authentication of the client take place when a connection is created. It is a relatively heavy-weight object, and most clients do all their messaging with a single connection.
The connection is used to create sessions. A Session is a single-threaded context for producing and consuming messages. It is used to create the message producers and consumers that send and receive messages, and it defines a serial order for the messages it delivers. A session supports reliable delivery through a number of acknowledgement options or through transactions (which can be managed by a distributed transaction manager).
A JMS client uses a MessageProducer to send messages to a specified physical destination, represented in the API as a destination object. The message producer can specify a default delivery mode (persistent vs. non-persistent messages), priority, and time-to-live values that govern all messages sent by the producer to the physical destination.
Similarly, a JMS client uses a MessageConsumer to receive messages from a specified physical destination, represented in the API as a destination object. A message consumer can use a message selector that allows the message service to deliver only those messages to the message consumer that match the selection criteria.
A message consumer can support either synchronous or asynchronous consumption of messages (see the MQ Developer's Guide). Asynchronous consumption is achieved by registering a MessageListener with the consumer. The client consumes a message when a session thread invokes the onMessage() method of the MessageListener object.
Administered Objects
Two of the objects described in the "JMS Programming Model" depend on how a JMS provider implements a JMS message service. The connection factory object depends on the underlying protocols and mechanisms used by the provider to deliver messages, and the destination object depends on the specific naming conventions and capabilities of the physical destinations used by the provider.
Normally these provider-specific characteristics would make JMS client code dependent on a specific JMS implementation. To make JMS client code provider-independent, however, the JMS specification requires that provider-specific implementation and configuration information be encapsulated in what are called administered objects. These objects can then be accessed in a standardized, non-provider-specific way.
Administered objects are created and configured by an administrator, stored in a name service, and accessed by JMS clients through standard Java Naming and Directory Service (JNDI) lookup code. Using administered objects in this way makes JMS client code provider-independent.
JMS provides for two general types of administered objects: connection factories and destinations. Both encapsulate provider-specific information, but they have very different uses within a JMS client. A connection factory is used to create connections to a message server, while destination objects are used to identify physical destinations used by the JMS message service.
For more information on administered objects, see "MQ Administered Objects".
JMS/J2EE Programming: Message-driven Beans
In addition to the general JMS client programming model introduced in "JMS Programming Model", there is a more specialized adaptation of JMS used in the context of Java 2 Enterprise Edition (J2EE) applications. This specialized JMS client is called a message-driven bean and is one of a family of Enterprise JavaBeans (EJB) components specified in the EJB 2.0 Specification (http://java.sun.com/products/ejb/docs.html).
The need for message-driven beans arises out of the fact that other EJB components (session beans and entity beans) can only be called synchronously. These EJB components have no mechanism for receiving messages asynchronously, since they are only accessed through standard EJB interfaces.
However, asynchronous messaging is a requirement of many enterprise applications. Most such applications require that server-side components be able to communicate and respond to each other without tying up server resources. Hence, the need for an EJB component that can receive messages and consume them without being tightly coupled to the producer of the message. This capability is needed for any application in which server-side components must respond to application events. In enterprise applications, this capability must also scale under increasing load.
Message-driven Beans
A message-driven bean (MDB) is a specialized EJB component supported by a specialized EJB container (a software environment that provides distributed services for the components it supports).
Message-driven Bean The MDB is a JMS message consumer that implements the JMS MessageListener interface. The onMessage method (written by the MDB developer) is invoked when a message is received by the MDB container. The onMessage() method consumes the message, just as the onMessage() method of a standard MessageListener object would. You do not remotely invoke methods on MDB's—like you do on other EJB components—therefore there are no home or remote interfaces associated with them. The MDB can consume messages from a single destination. The messages can be produced by standalone JMS applications, JMS components, EJB components, or Web components, as shown in Figure 1-4.
Figure 1-4 Messaging with MDBs
MDB Container The MDB is supported by a specialized EJB container, responsible for creating instances of the MDB and setting them up for asynchronous consumption of messages. This involves setting up a connection with the message service (including authentication), creating a pool of sessions associated with a given destination, and managing the distribution of messages as they are received among the pool of sessions and associated MDB instances. Since the container controls the life-cycle of MDB instances, it manages the pool of MDB instances so as to accommodate incoming message loads.
Associated with an MDB is a deployment descriptor that specifies the JNDI lookup names for the administered objects used by the container in setting up message consumption: a connection factory and a destination. The deployment descriptor might also include other information that can be used by deployment tools to configure the container. Each such container supports instances of only a single MDB.
Application Server Support
In J2EE architecture (see the J2EE Platform Specification located at http://java.sun.com/j2ee/download.html#platformspec), EJB containers are hosted by application servers. An application server provides resources needed by the various containers: transaction managers, persistence managers, name services, and, in the case of messaging and MDB's, a JMS provider.
In the Sun ONE Application Server, messaging resources are provided by Sun ONE Message Queue. This means that an MQ messaging system (see Chapter 2 "The MQ Messaging System") is integrated into the Sun ONE Application Server, providing the support needed to send JMS messages to MDB's and other JMS messaging components that run in the application server environment.
JMS Messaging Issues
This section describes a number of JMS programming issues that impact the administration of an MQ message service. The discussion focuses on concepts and terminology that are needed by an MQ administrator.
JMS Provider Independence
JMS specifies the use of administered objects (see "Administered Objects") to support the development of client applications that are portable to other JMS providers. Administered objects allow JMS clients to use logical names to look up and reference provider-specific objects. In this way client code does not need to know specific naming or addressing syntax or configurable properties used by a provider. This makes the code provider-independent.
Administered objects are MQ system objects created and configured by an MQ administrator. These objects are placed in a JNDI directory service, and a JMS client accesses them using a JNDI lookup.
MQ administered objects can also be instantiated by the client, rather than looked up in a JNDI directory service. This has the drawback of requiring the application developer to use provider-specific API's. It also undermines the ability of an MQ administrator to successfully control and manage an MQ message server.
For more information on administered objects, see "MQ Administered Objects".
Programming Domains
JMS supports two distinct message delivery models: point-to-point and publish/subscribe.
point-to-point (Queue Destinations) A message is delivered from a producer to one consumer. In this delivery model, the destination is a queue. Messages are first delivered to the queue destination, then delivered from the queue, one at a time, depending on the queue's delivery policy (see "Queue Destinations"), to one of the consumers registered for the queue. Any number of producers can send messages to a queue destination, but each message is guaranteed to be delivered to—and successfully consumed by—only one consumer. If there are no consumers registered for a queue destination, the queue holds messages it receives, and delivers them when a consumer registers for the queue.
Publish/Subscribe (Topic destinations) A message is delivered from a producer to any number of consumers. In this delivery model, the destination is a topic. Messages are first delivered to the topic destination, then delivered to all active consumers that have subscribed to the topic. Any number of producers can send messages to a topic destination, and each message can be delivered to any number of subscribed consumers. Topic destinations also support the notion of durable subscriptions. A durable subscription represents a consumer that is registered with the topic destination but can be inactive at the time that messages are delivered. When the consumer subsequently becomes active, it receives the messages. If there are no consumers registered for a topic destination, the topic does not hold messages it receives, unless it has durable subscriptions for inactive consumers.
These two message delivery models are handled using different API objects—with slightly different semantics—representing different programming domains, as shown in Table 1-1.
Table 1-1 JMS Programming Objects
Base Type
(Unified Domain)Point-to-Point Domain
Publish/Subscribe Domain
Destination (Queue or Topic)1
Queue
Topic
ConnectionFactory
QueueConnectionFactory
TopicConnectionFactory
Connection
QueueConnection
TopicConnection
Session
QueueSession
TopicSession
MessageProducer
QueueSender
TopicPublisher
MessageConsumer
QueueReceiver
TopicSubscriber
1 Depending on programming approach, you might specify a particular destination type. You can program both point-to-point and publish/subscribe messaging using the unified domain objects shown in the first column of Table 1-1. This is the preferred approach. However, to conform to the earlier JMS 1.02b specification, you can use the point-to-point domain objects to program point-to-point messaging, and the publish/subscribe domain objects to program publish/subscribe messaging.
Client Identifiers
JMS providers must support the notion of a client identifier, which associates a JMS client's connection to a message service with state information maintained by the message service on behalf of the client. By definition, a client identifier is unique, and applies to only one user at a time. Client identifiers are used in combination with a durable subscription name (see "Publish/Subscribe (Topic destinations)") to make sure that each durable subscription corresponds to only one user.
The JMS specification allows client identifiers to be set by the client through an API method call, but recommends setting it administratively using a connection factory administered object (see "Administered Objects"). If hard wired into a connection factory, however, each user would then need an individual connection factory to have a unique identity.
MQ provides a way for the client identifier to be both ConnectionFactory and user specific using a special variable substitution syntax that you can configure in a ConnectionFactory object. When used this way, a single ConnectionFactory object can be used by multiple users who create durable subscriptions, without fear of naming conflicts or lack of security. A user's durable subscriptions are therefore protected from accidental erasure or unavailability due to another user having set the wrong client identifier.
For details on how to use this MQ feature, see the discussion of connection factory attributes in the MQ Developer's Guide.
In any case, in order to create a durable subscription, a client identifier must be either programmatically set by the client, using the JMS API, or administratively configured in the ConnectionFactory objects used by the client.
Reliable Messaging
JMS defines two delivery modes:
Persistent messages These messages are guaranteed to be delivered and successfully consumed once and only once. Reliability is at a premium for such messages.
Non-persistent messages These messages are guaranteed to be delivered at most once. Reliability is not a major concern for such messages.
There are two aspects of assuring reliability in the case of persistent messages. One is to assure that their delivery to and from a message service is successful. The other is to assure that the message service does not lose persistent messages before delivering them to consumers.
Acknowledgements/Transactions
Reliable messaging depends on guaranteeing the successful delivery of persistent messages to and from a destination. This can be achieved using either of two general mechanisms supported by an MQ session: acknowledgements or transactions. In the case of transactions, these can either be local or distributed, under the control of a distributed transaction manager.
Acknowledgements
A session can be configured to use acknowledgements to assure reliable delivery.
In the case of a producer, this means that the message service acknowledges delivery of a persistent message to its destination before the producer's send() method returns. In the case of a consumer, this means that the client acknowledges delivery and consumption of a persistent message from a destination before the message service deletes the message from that destination.
Local Transactions
A session can also be configured as transacted, in which case the production and/or consumption of one or more messages can be grouped into an atomic unit—a transaction. The JMS API provides methods for initiating, committing, or rolling back a transaction.
As messages are produced or consumed within a transaction, the broker tracks the various sends and receives, completing these operations only when the client issues a call to commit the transaction. If a particular send or receive operation within the transaction fails, an exception is raised. The client code can handle the exception by ignoring it, retrying the operation, or rolling back the entire transaction. When a transaction is committed, all the successful operations are completed. When a transaction is rolled back, all successful operations are cancelled.
The scope of a local transaction is always a single session. That is, one or more producer or consumer operations performed in the context of a single session can be grouped into a single local transaction.
Since transactions span only a single session, you cannot have an end-to-end transaction encompassing both the production and consumption of a message. (In other words, the delivery of a message to a destination and the subsequent delivery of the message to a client cannot be placed in a single transaction.)
Distributed Transactions
MQ also supports distributed transactions. That is, the production and consumption of messages can be part of a larger, distributed transaction that includes operations involving other resource managers, such as database systems. In distributed transactions, a distributed transaction manager tracks and manages operations performed by multiple resource managers (such as a message service and a database manager) using a two-phase commit protocol defined in the Java Transaction API (JTA), XA Resource API specification. In the Java world, interaction between resource managers and a distributed transaction manager are described in the JTA specification.
Support for distributed transactions means that messaging clients can participate in distributed transactions through the XAResource interface defined by JTA. This interface defines a number of methods for implementing two-phase commit. While the API calls are made on the client side, the MQ broker tracks the various send and receive operations within the distributed transaction, tracks the transactional state, and completes the messaging operations only in coordination with a distributed transaction manager—provided by a Java Transaction Service (JTS).
As with local transactions, the client can handle exceptions by ignoring them, retrying operations, or rolling back an entire distributed transaction.
MQ implements support for distributed transactions through an XA connection factory, which lets you create XA connections, which in turn lets you create XA sessions (see "JMS Programming Model"). In addition, support for distributed transactions requires either a third party JTS or a J2EE-compliant Application Server (that provides JTS).
Persistent Storage
The other important aspect of reliability is assuring that once persistent messages are delivered to their destinations, a message service does not lose them before they are delivered to consumers. This means that upon delivery of a persistent message to its destination, the message service must place it in a persistent data store (see "Persistence Manager"). If the message service goes down for any reason, it can recover the message and deliver it to the appropriate consumers. While this adds overhead to message delivery, it also adds reliability.
A message service must also store durable subscriptions. This is because to guarantee delivery in the case of topic destinations, it is not sufficient to recover only persistent messages. The message service must also recover information about durable subscriptions for a topic, otherwise it would not be able to deliver messages to subscribers who are inactive when a message arrives, and subsequently become active.
Messaging applications that are concerned about guaranteed message delivery must specify messages as persistent and use either queue destinations or durable subscriptions to topic destinations.
Performance Trade-offs
The more reliable the delivery of messages, the more overhead and bandwidth are required to achieve it. The trade-off between reliability and performance is a significant design consideration. You can maximize performance by choosing to produce and consume non-persistent messages. On the other hand, you can maximize reliability by producing and consuming persistent messages and using transacted sessions. Between these extremes are a number of options, depending on the needs of an application, including the use of MQ-specific connection and acknowledgement properties (see the MQ Developer's Guide).
Message Selection
JMS provides a mechanism by which a message service can perform message filtering and routing based on criteria placed in message selectors. A producing client can place application-specific properties in the message, and a consuming client can indicate its interest in messages using selection criteria based on such properties. This simplifies the work of the client and eliminates the overhead of delivering messages to clients that don't need them. However, it adds some additional overhead to the message service processing the selection criteria. Message selector syntax and semantics are outlined in the JMS specification.
Message Order and Priority
In general, all messages sent to a destination by a single session are guaranteed to be delivered to a consumer in the order they were sent. However, if they are assigned different priorities, a messaging system will attempt to deliver higher priority messages first.
Beyond this, the ordering of messages consumed by a client application can have only a rough relationship to the order in which they were produced. This is because the delivery of messages to destinations and the delivery from those destinations can depend on a number of issues that affect timing, such as the order in which the messages are sent, the sessions (connections) from which they are sent, whether the messages are persistent, the lifetime of the messages, the priority of the messages, the message delivery policy of queue destinations (see "Queue Destinations"), and message service availability.
In the case of an MQ message server using multiple interconnected brokers (see "Multi-Broker Clusters (Enterprise Edition)") the ordering of messages consumed by a client is further complicated by the fact that the order of delivery from destinations on different brokers is indeterminate. Hence, a message delivered by one broker might precede a message delivered by another broker even though the latter might have received the message first.
In any case, for a given consumer, precedence is given for higher priority messages over lower priority messages.