Sun logo      Previous      Contents      Index      Next     

Sun ONE Message Queue 3.5 C Client Developer's Guide

Chapter 1
Introduction

This chapter provides an overall introduction to Sun™ ONE Message Queue (MQ) and to JMS concepts and programming issues of interest to developers. It is written specifically for the C developer who wants to interface with an MQ Message Service in order to send messages to and receive messages from another MQ client. MQ clients can reside on the same or on different platforms. The chapter covers the following topics:


What Is Message Queue?

The MQ product is a standards-based solution to the problem of reliable, asynchronous messaging for distributed applications. MQ is an enterprise messaging system that implements the Java™ Message Service (JMS) open standard: it is a JMS provider.

With Message Queue software, processes running on different platforms and operating systems can connect to a common MQ message server (broker) to send and receive information. Application developers are free to focus on the business logic of their applications, rather than on the low-level details of how their applications communicate across a network. Developers can use two programming interfaces to establish a connection to the broker, and send or receive messages:

MQ administrators can use a variety of tools to set up destinations on the broker and to configure the broker in response to performance and reliability requirements. Administrative functions and tools are described in the MQ Administrator’s Guide.

MQ Features

MQ has features that exceed the minimum requirements of the JMS specification. Among these features are the following:

Centralized administration     Provides both command-line and GUI tools for administering an MQ message service and managing application-specific aspects of messaging, such as destinations and security.

Scalable message service     Allows you to service increasing numbers of JMS clients (components or applications) by balancing the load among a number of MQ message service components (brokers) working in tandem (multi-broker cluster).

Tunable performance     Lets you increase performance of the MQ message service when less reliability of delivery is acceptable.

Multiple transports     Supports the ability of JMS clients to communicate with each other over a number of different transports and using secure (SSL) connections.

C-API     Allows you to integrate legacy systems into an MQ messaging system and to create light-weight clients that do not require an underlying JVM.

See the MQ Release Notes for documentation of JMS compliance-related issues.

Java and C Interfaces

While this manual revisits a number of topics presented in the MQ Java Client Developer’s Guide, there are differences between the two interfaces and the JMS features they support. Some of these differences are summarized below, but this list is not exhaustive. If you plan to write an MQ C client, you should read this manual in full.

The C interface, compared to the Java interface

Like the Java interface, the C interface does support the following:

Product Editions

The Message Queue product is available in two editions: Platform and Enterprise—each corresponding to a different licensed capacity. The C-API is only supported on the Enterprise Edition. For more information about these editions and for instructions on how you upgrade MQ from one edition to another, see the the MQ Installation Guide.


MQ Messaging System Architecture

This section briefly describes the main parts of an MQ messaging system. While as a developer, you do not need to be familiar with the details of all of these parts or how they interact, a high-level understanding of the basic architecture will help you understand features of the system that impact MQ C client design and development.

The main parts of an MQ messaging system, shown in Figure 1-1, are the following:

MQ Client     An MQ client can be written in C or Java, and it can send and/or receive MQ messages.

MQ message server     The MQ message server is the heart of a messaging system. It consists of a broker that provides delivery services for the system. These services include connections to C or Java clients, message routing and delivery, persistence, security, and logging. The message server also maintains physical destinations to which clients send messages, and from which the messages are delivered to consuming clients. The MQ message server is described in detail in the MQ Administrator’s Guide.

MQ client runtime     The MQ C and Java client runtimes provide MQ C and Java clients respectively with an interface to the MQ message server. They support all operations needed for clients to send messages to destinations and to receive messages from such destinations. The MQ C client runtime is described in detail in "Message Production and Consumption".

Figure 1-1  MQ System Architecture

Diagram showing the components of MQ Messaging System. Figure is described in text.

MQ message service     The MQ message service includes one or more MQ message servers and MQ client runtime support.

MQAdministration     MQ provides a number of administrative tools for managing an MQ messaging system. These tools are used to manage the message server, security, messaging application resources, and persistent data. These tools are generally used by MQ administrators and are described in the MQ Administrator’s Guide.

MQ Messaging System     The MQ messaging system includes the MQ message service and MQ administration.


The JMS Programming Model

This section briefly describes the programming model of the JMS specification. The JMS programming model is the foundation for the design of an MQ C client. Although the C-API does not provide an exhaustive implementation of the JMS programming model, this section is provided as a review of the most important concepts and terminology (defined for that model), which also apply to MQ C client design.

In the JMS programming model, JMS clients (components or applications) interact using a JMS application programming interface (API) to send and receive messages. In this context, it is important to understand that a C client’s interface is specific to the MQ provider and cannot be used with other JMS providers. A messaging application that includes a C client cannot be handled by another JMS provider.

This section introduces the C data types and functions used by an MQ C client for delivery of messages. The main data types, which are opaque to the user and accessible only through the C functions, are shown in Figure 1-2 and described in the following sections.

Message

In the MQ product, data is exchanged using JMS messages—messages that conform to the JMS specification. According to the JMS specification, a message is composed of three parts: a header, properties, and a body.

Properties are optional—they provide values that clients can use to filter messages. A body is also optional—it contains the actual data to be exchanged.

Figure 1-2  JMS Programming Objects

A diagram showing the relation between JMS objects and the JMS message server. Long description follows figure.[D]

Header

A header is required of every message. Header fields contain values used for routing and identifying messages.

Some header field values are set automatically by MQ during the process of producing and delivering a message, some depend on settings specified when message producers send a message, and others are set on a message-by-message basis by the client using the MQSetMessageHeader function. The following table lists the header fields defined (and required) by JMS and their corresponding names, as defined by the C-API.

Table 1-1  JMS-defined Message Header 

JMS Message Header Field

C-API Message Header Property Name

JMSDestination

Defined implicitly when a producer sends a message to a destination, or when a consumer receives a message from a destination.

JMSDeliveryMode

MQ_PERSISTENT_HEADER_PROPERTY

JMSExpiration

MQ_EXPIRATION_HEADER_PROPERTY

JMSPriority

MQ_PRIORITY_HEADER_PROPERTY

JMSMessageID

MQ_MESSAGE_ID_HEADER_PROPERTY

JMSTimestamp

MQ_TIMESTAMP_HEADER_PROPERTY

JMSRedelivered

MQ_REDELIVERED_HEADER_PROPERTY

JMSCorrelationID

MQ_CORRELATION_ID_HEADER_PROPERTY

JMSReplyTo

Set by the MQSetMessageReplyTo function, and obtained by the MQGetMessageReplyTo function.

JMSType

MQ_MESSAGE_TYPE_HEADER_PROPERTY

For additional information about each property type and the agent who sets it, see Table 4-6.

Properties

When data is sent between two processes, other information besides the payload data can be sent with it. These descriptive fields, or properties, can provide additional information about the data; for example, which process created it, the time it was created, and information that uniquely identifies the structure of each piece of data. Properties (which can be thought of as an extension of the header) consist of property name and property value pairs, as specified by a C client. A C client can set message properties when initializing a handle to a properties data type and passing that handle to the MQSetMessageProperties function.

Having registered an interest in a particular destination, consuming clients can fine-tune their selection by specifying certain property values as selection criteria. For instance, a client might indicate an interest in Payroll messages (rather than Facilities) but only Payroll items concerning part-time employees located in New Jersey. Messages that do not meet the specified criteria are not delivered to the consumer.

Message Body Types

JMS specifies six classes (or types) of messages. The C-API supports only two of these types, as described in Table 1-2. If an MQ C client expects to receive messages from an MQ Java client, it will be unable to process messages whose body types are other than those described in Table 1-2.

Table 1-2  C-API Message Body Types 

Type

Description

TextMessage

A message whose body contains a Java string, for example an XML message.

BytesMessage

A message whose body contains a stream of uninterpreted bytes.

Destination

A destination refers to where a message is destined to go. A physical destination is a JMS message service entity (a location on the broker) to which producers send messages and from which consumers receive messages. The message service provides the routing and delivery for messages sent to a physical destination.

When an MQ C client creates a destination programmatically using the MQCreateDestination function, a destination name must be specified. The function initializes a handle to a destination data type that holds the identity (name) of the destination. The important thing to remember is that this function does not create the physical destination on the broker; this must be done by the administrator. The destination that is created programmatically however must have the exact same name and type as the physical destination created on the broker.

Destination names starting with “mq” are reserved and should not be used by client programs.

Connection

A connection is a JMS client’s configured connection to an MQ message service. Both allocation of communication resources and authentication of a client take place when a connection is created. Hence it is a relatively heavy-weight object, and most clients do all their messaging with a single connection. A connection is used to create sessions.

Session

A session is a single-threaded context for producing and consuming messages. While there is no restriction on the number of threads that can use a session, the session should not be used concurrently by multiple threads. It is used to create the message producers and consumers that send and receive messages, and defines a serial order for the messages it consumes and the messages it produces. A session supports reliable delivery through a number of acknowledgement options or by using transactions. A transacted session can combine a series of sequential operations into a single transaction that can span a number of producers and consumers. You need to create a session before you can create its consumers or producers.

Message Producer

A client uses a message producer to send messages to a physical destination. You can create a message producer with a specified destination or you can specify a destination when you send each message. You can also specify a delivery mode, priority, and time-to-live for a message producer that govern all messages sent by a producer, except when explicitly over-ridden.

Message Consumer

A client uses a message consumer to receive messages from a physical destination. A message consumer can have a message selector that allows the message service to deliver only those messages to the consumer that match the selection criteria. A message consumer can support either synchronous or asynchronous consumption of messages (see "Synchronous and Asynchronous Consumption").

Message Listener

To support asynchronous communication, an MQ C client must write a callback function of type MQMessageListenerFunc. You pass a pointer to this function when you create an asynchronous message consumer. A client is said to consume a message when a session thread invokes this callback function.


Client Design Issues

This section describes a number of messaging issues that impact MQ C client design.

Programming Domains

When you create a session, you can specify one of two message delivery models: point-to-point and publish/subscribe. You specify the message delivery model for a C-MQ client by specifying either MQ_QUEUE_DESTINATION or MQ_TOPIC_DESTINATION for the destinationType parameter when you call the MQCreateDestination function.

Point-to-Point (Queue Destinations)     A message is delivered from a producer to one consumer. In this delivery model, the destination type is a queue. Messages are first delivered to the queue destination, then delivered from the queue, one at a time, depending on the queue’s delivery policy, to one of the consumers registered for the queue. Any number of producers can send messages to a queue destination, but each message is guaranteed to be delivered to—and successfully consumed by—only one consumer. If there are no consumers registered for a queue destination, the queue holds messages it receives, and delivers them when a consumer registers for the queue.

Publish/Subscribe (Topic destinations)     A message is delivered from a producer to any number of consumers. In this delivery model, the destination type is a topic. Messages are first delivered to the topic destination, then delivered to all active consumers that have subscribed to the topic. Any number of producers can send messages to a topic destination, and each message can be delivered to any number of subscribed consumers. Topic destinations also support the notion of durable subscriptions. A durable subscription represents a durable consumer that is registered with the topic destination but can be inactive at the time that messages are delivered. When the consumer subsequently becomes active, it receives the messages. If there are no consumers registered for a topic destination, the topic does not hold messages it receives, unless it has durable subscriptions for inactive consumers.

Client Identifiers

Clients need to be identified to a broker both for authentication purposes and to keep track of durable subscriptions.

For authentication purposes, you need to provide a user name and password. The administrator is responsible for setting up a user repository against which the broker can validate this name and password. See the MQ Administrator’s Guide for more information.

To keep track of durable subscriptions, MQ uses a unique client identifier that associates a client’s connection with state information maintained by the message service on behalf of the client. By definition, a client identifier is unique, and applies to only one connection at a time.

Client identifiers are used in combination with a durable subscription name (see "Publish/Subscribe (Topic destinations)") to make sure that each durable subscription corresponds to only one user. If a durable subscriber is inactive at the time that messages are delivered to a topic destination, the broker retains messages for that subscriber and delivers them when the subscriber once again becomes active. The only way for the broker to identify the subscriber is through its client ID. You can specify a client ID using the clientID parameter to the MQCreateConnection function.

Reliable Messaging

Reliable messaging depends on a message’s delivery mode and the use of transactions or acknowledgements to ensure the reliability of persistent messages.

Delivery Mode

JMS defines two delivery modes: persistent and non-persistent:

A message’s delivery mode is set to be persistent by default. You can override this setting by using the MQSendMessageExt function and setting the delivery mode to MQ_NONPERSISTENT_DELIVERY.

Reliable messaging guarantees the delivery of persistent messages to and from a destination. There are two aspects of assuring reliability in the case of persistent messages. One is to assure that their delivery to and from a message service is successful. The other is to assure that the message service does not lose these messages before delivering them to consumers.

Acknowledgements and Transactions

You can ensure reliable messaging by using either of two general mechanisms supported by an MQ session: acknowledgements or transactions.

Acknowledgements

Both messages that are sent and messages that are received can be acknowledged.

In the case of message producers, if you want the broker to acknowledge its having received a non-persistent message (to its physical destination), you must set the broker’s MQ_ACK_ON_PRODUCE_PROPERTY to MQ_TRUE. If you do so, the sending function will return only after the broker has acknowledged receipt of the message. By default, the broker acknowledges receipt of persistent messages.

In the case of message consumers, you can specify one of several acknowledge modes for the consuming session when you create that session. Acknowledgements on the consuming side means that the client runtime acknowledges delivery and consumption of all message from a physical destination before the message service deletes the message from that destination. For more information about a session’s acknowledge modes, see "Acknowledge Modes" and the description of the MQ_ACK_ON_ACKNOWLEDGE_PROPERTY in Table 4-2.

Transactions

A session can also be configured as transacted, in which case work spanning a session’s producers or consumers is combined into an atomic unit—a transaction. The MQ-C API provides functions for committing, or rolling back a transaction. (See "Transacted Sessions" for more information.) The C runtime does not support distributed transactions, that is a transaction cannot include operations involving other resource managers, such as database systems.

As messages are produced or consumed within a transaction, the broker tracks the various sends and receives, completing these operations only when the client issues a call to commit the transaction. If a particular send or receive operation within the transaction fails, an exception is raised. The application can handle the exception by ignoring it, retrying the operation, or rolling back the entire transaction. When a transaction is committed, all the successful operations are completed. When a transaction is rolled back, all successful operations are cancelled.

The scope of a transaction is always a single session. That is, one or more producer or consumer operations performed in the context of a single session can be grouped into a single local transaction.

Since transactions span only a single session, you cannot have an end-to-end transaction encompassing both the production and consumption of a message. (In other words, the delivery of a message to a destination and the subsequent delivery of the message to a client cannot be placed in a single transaction.)

Persistent Storage

The other important aspect of reliability is assuring that once persistent messages are delivered to their destinations, the message service does not lose them before they are delivered to consumers. This means that upon delivery of a persistent message to its destination, the message service must place it in a persistent data store. If the message service goes down for any reason, it can recover the message and deliver it to the appropriate consumers. While this adds overhead to message delivery, it also adds reliability.

A message service must also store durable subscriptions. This is because to guarantee delivery in the case of topic destinations, it is not sufficient to recover only persistent messages. The message service must also recover information about durable subscriptions for a topic, otherwise it would not be able to deliver a message to durable consumers when they become active.

Messaging applications that are concerned about guaranteeing delivery of persistent messages must either employ queue destinations or employ durable subscriptions to topic destinations.

The way in which the message service handles persistent messages depends upon a session’s delivery mode. For more information, see "Delivery Mode".

Performance Trade-offs

The more reliable the delivery of messages, the more overhead and bandwidth are required to achieve it. The trade-off between reliability and performance is a significant design consideration. You can maximize performance and throughput by choosing to produce and consume non-persistent messages. On the other hand, you can maximize reliability by producing and consuming persistent messages using a transacted session. Between these extremes are a number of options, depending on the needs of an application, including the use of MQ-specific persistence and acknowledgement properties (see "Managing Flow Control".).

Message Production and Consumption

The MQ C client runtime provides MQ C clients with an interface to the MQ message server—it supplies these clients with all the data types and functions introduced in "The JMS Programming Model". It supports all operations needed for clients to send messages to destinations and to receive messages from such destinations.

This section provides a high level description of how the MQ C client runtime supports message production and consumption. Figure 1-3 illustrates how message production and consumption involve an interaction between clients and the MQ C client runtime, while message delivery involves an interaction between the MQ C client runtime and MQ message servers.

Figure 1-3  Messaging Operations

Diagram showing messaging operations. Content of figure is explained in text.

Once a client has created a connection to a broker, created a session as a single-threaded context for message delivery, and created a message producer or a message consumer to access particular destinations in a message server, production (sending) or consumption (receiving) of messages can proceed.

Message Production

In message production, a message is created by the client, and sent over a connection to a destination on a broker. If the message delivery mode has been set to persistent (guaranteed delivery, once and only once), the client thread blocks until the broker acknowledges that the message was delivered to its destination and stored in the broker’s persistent data store. If the message is not persistent, no broker acknowledgement is returned by the broker, and the client thread does not block.

In the case of persistent messages, to increase throughput on sends, you can set the connection to not require broker acknowledgement (see "Connection Properties"), but this eliminates the guarantee that persistent messages are reliably delivered.

Message Consumption

Message consumption is more complex than production. Messages arriving at a destination on a broker are delivered over a connection to the MQ client runtime under the following conditions:

Messages delivered over the connection are distributed to the appropriate MQ sessions where they are queued to be consumed by the appropriate message consumers, as shown in Figure 1-4.

Figure 1-4  Message Delivery to MQ Client Runtime

Diagram showing message delivery to client runtime. Figure content is described in text.

Messages are fetched off each session queue one at a time (a session is single threaded). A message can consumed synchronously or asynchronously. A message is said to be consumed either when one of the MQReceiveMessage... functions returns (synchronously) or when the callback function associated with the asynchronous consumer returns.

When a broker delivers messages to the client runtime, it marks the messages accordingly, but does not really know if they have been consumed. Therefore, the broker waits for the client to acknowledge receipt of a message before deleting the message from the broker’s destination. If a connection fails, and another connection is subsequently established, the broker will re-deliver all previously delivered but unconsumed messages, setting their message header MQ_REDLIEVERED_HEADER_PROPERTY field.

There are three acknowledgment options that you can set for a client session:

Each of the three acknowledgement options requires a different level of processing and bandwidth overhead. AUTO_ACKNOWLEDGE consumes the most overhead and guarantees reliability on a message by message basis, while DUPS_OK_ACKNOWLEDGE consumes the least overhead, but allows for duplicate delivery of messages.

In the case of the AUTO_ACKNOWLEDGE or CLIENT_ACKNOWLEDGE options, the threads performing the acknowledgement, or committing a transaction, will block, waiting for the broker to return an acknowledgement of the client acknowledgement. This broker acknowledgement guarantees that the broker has deleted the corresponding persistent message and will not send it twice—which could happen were the client or broker to fail, or the connection to fail, at the wrong time.

To increase throughput, you can configure the connection to not require broker acknowledgement of client acknowledgements, but this eliminates the guarantee that persistent messages are delivered once and only once.


Note

In the DUPS_OK_ACKNOWLEDGE mode, the session does not wait for broker acknowledgements. This option is used in MQ C clients for which duplicate messages are not a problem. Also, you can call the MQRecoverSession function to explicitly request redelivery of messages that have been received but not yet acknowledged by the client. When redelivering such messages, the broker will set the header field MQ_REDLIEVERED_HEADER_PROPERTY.


Synchronous and Asynchronous Consumption

There are two ways an MQ C client can consume messages: either synchronously or asynchronously.

In synchronous consumption, a client gets a message by calling one of the MQReceive... functions. The client thread blocks until the function returns. This means that if no message is available, the client blocks until a message does become available or until the receive function times out (if it was called with a time-out specified). In this model, a client thread can only consume messages one at a time.

In asynchronous consumption, a client creates a callback function of type MQMessageListenerFunc and passes a pointer to it as a parameter to one of the MQCreateAsync...MessageConsumer functions. A client consumes a message when the session invokes this function. In this model, the client thread does not block because the thread listening for and consuming the message belongs to the MQ client runtime.

Message Selection

JMS defines a mechanism by which a message service can perform message filtering and routing based on criteria placed in message selectors. A producing client can define application-specific properties for a message, and a consuming client can indicate its interest in messages using selection criteria based on such properties. This simplifies the work of the client and eliminates the overhead of delivering messages to clients that don’t need them. However, it adds some additional overhead to the message service processing the selection criteria. Message selector syntax and semantics are outlined in the JMS specification.

Use the MQSetMessageProperties function to set properties that can be used in message filtering.

Message Order and Priority

In general, all messages sent to a destination by a single session are guaranteed to be delivered to a consumer in the order they were sent. However, if they are assigned different priorities, the messaging system will attempt to deliver higher priority messages first.

Beyond this, the ordering of messages consumed by a client can have only a rough relationship to the order in which they were produced. This is because the delivery of messages to a number of destinations and the delivery from those destinations can depend on a number of issues that affect timing, such as the order in which the messages are sent, the sessions from which they are sent, whether the messages are persistent, the lifetime of the messages, the priority of the messages, the message delivery policy of queue destinations (see the MQ Administrator’s Guide), and message service availability.


Configuring Connections

The MQ client runtime supports all the operations described in "Message Production and Consumption". It also provides connection properties that you can set to specify a broker to connect to, configure a secure connection, optimize resources, performance, and message throughput.

Connection properties can be grouped into the following categories:

Each of these categories is discussed in the following sections with a description of the properties that you can set to configure the behavior of the broker. All broker properties are described in detail in Table 4-2.

Connection Handling

Connections to a message server are specified by a broker host name and port number.

Currently, the C-API does not support auto-reconnector or failover, which allows the client runtime to automatically reconnect to a broker if a connection fails.

Reliability

Two connection properties enable the acknowledgement of messages sent to the broker and of messages received from the broker. These are described in "Message Production and Consumption". In addition to setting these properties, you can also set MQ_ACK_TIMEOUT_PROPERTY, which determines the maximum time that the client runtime will wait for any broker acknowledgement before throwing an exception.

Flow Control

A number of connection properties determine the use and flow of MQ control messages by the client runtime. Messages sent and received by MQ clients and MQ control messages pass over the same client-broker connection. Because of this, delays may occur in the delivery of control messages, such as broker acknowledgements, if these are held up by the delivery of JMS messages. To prevent this type of congestion, MQ meters the flow of JMS messages across a connection.

The C API does not currently support consumer-level flow control.

Security

The C-API supports the SSL transport protocol, which supports SSL v2, SSL v3, and TLS standards. For more information on how to set up and create a secure connection, see "Working With Secure Connections" for more information.

Version Information

Properties that specify the version of the MQ product are set by the C client runtime and can be read using the MQGetMetaData function.


Managing Flow Control

Because of the mechanisms by which messages are delivered to and from a broker, and because of the MQ control messages used to assure reliable delivery, there are a number of factors that affect reliability and performance. These factors include: delivery mode, acknowledgement mode, message flow metering, and message flow limits.

Although these factors are quite distinct, their interactions can complicate the task of balancing reliability with performance. Specifically, because client messages and MQ control messages flow across the same connection between the client and the broker, you need to understand how to balance the requirement for reliability with the need for throughput. This section describes how you can balance these requirements to manage flow control.

Delivery Mode

The delivery mode specifies whether a message is to be delivered at most once (non-persistent) or once and only once (persistent). These different reliability requirements imply different degrees of overhead. Specifically, the management of persistent messages requires greater use of broker control messages flowing across a connection.

Acknowledgement Mode

The setting of the acknowledgement mode impacts reliability and affects the number of client and broker acknowledgement messages passing over a connection:

Message Flow Metering

The connection property MQ_CONNECTION_FLOW_COUNT_PROPERTY governs the batching of messages so that only a set number are delivered; when the batch has been delivered, delivery of JMS messages is suspended, and pending control messages are delivered. This cycle repeats, as other batches of JMS messages are delivered, followed by queued up control messages.

You should keep the value of MQ_CONNECTION_FLOW_COUNT_PROPERTY low if the client is doing operations that require many responses from the broker; for example, the client is using the CLIENT_ACKNOWLEDGE or AUTO_ACKNOWLEDGE modes, persistent messages, transactions, or if the client is adding or removing consumers. If, on the other hand, the client has only simple consumers on a connection using DUPS_OK mode, you can increase the value of MQ_CONNECTION_FLOW_COUNT_PROPERTY without compromising performance.



Previous      Contents      Index      Next     


Copyright 2003 Sun Microsystems, Inc. All rights reserved.