General Features of Advanced Queuing

The following general features are discussed:

• Point-to-Point and Publish-Subscribe Messaging

• Oracle Internet Directory

• Oracle Enterprise Manager Integration

• Message Format Transformation

• SQL Access

• Support for Statistics Views

• Structured Payloads

• Retention and Message History

• Tracking and Event Journals

• Queue-Level Access Control

• Nonpersistent Queues

• Support for Oracle9i Real Application Clusters

• XMLType Attributes in Object Types

• Internet Integration and IDAP

See Chapter 8, "A Sample Application Using AQ" for a hypothetical scenario in which the messaging system for a hypothetical online bookseller, BooksOnLine, is described. Many features discussed here are exemplified in the BooksOnLine example.

Point-to-Point and Publish-Subscribe Messaging

A combination of features allows publish-subscribe messaging between applications. These features include rule-based subscribers, message propagation, the listen feature, and notification capabilities.

Advanced Queuing sends and receives messages in the following ways:

• Point-to-Point

• Publish-Subscribe

Point-to-Point

A point-to-point message is aimed at a specific target. Senders and receivers decide on a common queue in which to exchange messages. Each message is consumed by only one receiver. Figure 1-2 shows that each application has its own message queue, known as a single consumer queue.

Figure 1-2 Point-to-Point Messaging

Text description of the illustration adq81250.gif

Publish-Subscribe

A publish-subscribe message can be consumed by multiple receivers, as shown in Figure 1-3. Publish-subscribe messaging has a wide dissemination mode--broadcast--and a more narrowly aimed mode--multicast, also called point-to-multipoint.

Broadcasting is the equivalent of a radio station not knowing exactly who the audience is for a given program. The dequeuers are subscribers to multiconsumer queues In contrast, multicast is the same as a magazine publisher who knows who the subscribers are. Multicast is also referred to as point-to-multipoint because a single publisher sends messages to multiple receivers, called recipients, who may or may not be subscribers to the queues that serve as exchange mechanisms.

Figure 1-3 Publish-Subscribe Model

Text description of the illustration adq81249.gif

Oracle Internet Directory

Oracle Internet Directory is a native LDAPv3 directory service built on the Oracle database that centralizes a wide variety of information, including email addresses, telephone numbers, passwords, security certificates, and configuration data for many types of networked devices. You can look up enterprise-wide queuing information--queues, subscriptions, and events--from one location, the Oracle Internet Directory. See the Oracle Internet Directory Administrator's Guide for more information.

Oracle Enterprise Manager Integration

You can use Enterprise Manager to create and manage queues, queue tables, and propagation schedules, to view all queue propagation schedules, queue errors, and the message queue, and other Advanced Queuing functions. See "Oracle Enterprise Manager Support" in Chapter 4.

Message Format Transformation

The message format transformation feature supports applications that use data in different formats. A transformation defines a mapping from one Oracle data type to another. The transformation is represented by a SQL function that takes the source data type as input and returns an object of the target data type.

A transformation can be specified as follows:

• During enqueue, to transform the message to the correct type before inserting it into the queue.

  You can convert a message to the payload type of the queue at enqueue time. Thus, the type of the message to be enqueued need not match the payload type of the queue.

• During dequeue, to receive the message in the desired format

  A message can be transformed to the desired format before returning it to the dequeuer.

• By a remote subscriber, who can choose to receive a message in a format different from the format of the source queue

  Before propagating the message to the remote subscriber, the message is transformed according to the transformation that the remote subscriber specified when subscribing to the queue.

As Figure 1-4 shows, queuing, routing, and transformation are essential building blocks to an integrated application architecture. The figure shows how data from the Out queue of a CRM application is routed and transformed in the integration hub and then propagated to the In queue of the Web application. The transformation engine maps the message from the format of the Out queue to the format of the In queue.

Figure 1-4 Transformations in Application Integration

Text description of the illustration qintro2.gif

See "Message Format Transformation" for more information.

SQL Access

Messages are placed in normal rows in a database table, and so can be queried using standard SQL. This means that you can use SQL to access the message properties, the message history, and the payload. With SQL access you can also do auditing and tracking. All available SQL technology, such as indexes, can be used to optimize access to messages.

Support for Statistics Views

Basic statistics about queues are available using the GV$AQ view.

Structured Payloads

You can use object types to structure and manage message payloads. RDBMSs in general have a richer typing system than messaging systems. Since Oracle is an object-relational DBMS, it supports both traditional relational types as well as user-defined types. Many powerful features are enabled as a result of having strongly typed content, such as content whose format is defined by an external type system. These include:

• Content-based routing: Advanced Queuing can examine the content and automatically route the message to another queue based on the content.

• Content-based subscription: a publish and subscribe system is built on top of a messaging system so that you can create subscriptions based on content.

• Querying: the ability to execute queries on the content of the message enables message warehousing.

To see this feature applied in the context of the BooksOnLine scenario, refer to "Structured Payloads" .

Retention and Message History

The systems administrator specifies the retention duration to retain messages after consumption. Advanced Queuing stores information about the history of each message, preserving the queue and message properties of delay, expiration, and retention for messages destined for local or remote receivers. The information contains the enqueue and dequeue times and the identification of the transaction that executed each request. This allows users to keep a history of relevant messages. The history can be used for tracking, data warehouse, and data mining operations, as well as specific auditing functions.

To see this feature applied in the context of the BooksOnLine scenario, refer to Retention and Message History.

Tracking and Event Journals

If messages are retained, they can be related to each other. For example, if a message m2 is produced as a result of the consumption of message m1, m1 is related to m2. This allows users to track sequences of related messages. These sequences represent event journals, which are often constructed by applications. Advanced Queuing is designed to let applications create event journals automatically.

When an online order is placed, multiple messages are generated by the various applications involved in processing the order. Advanced Queuing offers features to track interrelated messages independent of the applications that generated them. You can determine who enqueued and dequeued messages, who the users are, and who did what operations.

With Advanced Queuing tracking features, you can use SQL SELECT and JOIN statements to get order information from AQ$QUETABLENAME and the views ENQ_TRAN_ID, DEQ_TRAN_ID, USER_DATA (the payload), CORR_ID, and MSG_ID. These views contain the following data used for tracking:

• Transaction IDs--from ENQ_TRAN_ID and DEQ_TRAN_ID, captured during enqueuing and dequeuing.

• Correlation IDs--from CORR_ID, part of the message properties

• Message content that can be used for tracking--USER_DATA

Queue-Level Access Control

The owner of an 8.1-style queue can grant or revoke queue-level privileges on the queue. Database administrators can grant or revoke new AQ system-level privileges to any database user. Database administrators can also make any database user an AQ administrator.

To see this feature applied in the context of the BooksOnLine scenario, refer to Queue-Level Access Control.

Nonpersistent Queues

Advanced Queuing can deliver nonpersistent messages asynchronously to subscribers. These messages can be event-driven and do not persist beyond the failure of the system (or instance). Advanced Queuing supports persistent and nonpersistent messages with a common API.

To see this feature applied in the context of the BooksOnLine scenario, refer to "Nonpersistent Queues".

Support for Oracle9i Real Application Clusters

An application can specify the instance affinity for a queue table. When Advanced Queuing is used with Real Application Clusters and multiple instances, this information is used to partition the queue tables between instances for queue-monitor scheduling. The queue table is monitored by the queue monitors of the instance specified by the user. If an instance affinity is not specified, the queue tables is arbitrarily partitioned among the available instances. There can be pinging between the application accessing the queue table and the queue monitor monitoring it. Specifying the instance affinity does not prevent the application from accessing the queue table and its queues from other instances.

This feature prevents pinging between queue monitors and Advanced Queuing propagation jobs running in different instances. If compatibility is set to Oracle8i, release 8.1.5 or higher, an instance affinity (primary and secondary) can be specified for a queue table. When Advanced Queuing is used with Real Application Clusters and multiple instances, this information is used to partition the queue tables between instances for queue-monitor scheduling as well as for propagation. At any time, the queue table is affiliated to one instance. In the absence of an explicitly specified affinity, any available instance is made the owner of the queue table. If the owner of the queue table is terminated, the secondary instance or some available instance takes over the ownership for the queue table.

To see this feature applied in the context of the BooksOnLine scenario, refer to "Support for Oracle Real Application Clusters".

XMLType Attributes in Object Types

You can create queues that use Oracle object types containing attributes of the new, opaque type, XMLType. These queues can be used to transmit and store messages that are XML documents. Using XMLType, you can do the following:

• Store any type of message in a queue

• Store documents internally as CLOBs

• Store more than one type of payload in a queue

• Query XMLType columns using the operators ExistsNode() and SchemaMatch()

• Specify the operators in subscriber rules or dequeue selectors

Internet Integration and IDAP

You can perform AQ operations over the Internet by using the Internet Data Access Presentation (IDAP), which defines message structure using XML. An IDAP-structured message is transmitted over the Internet using transport protocols such as HTTP or SMTP. See "Propagation over the Internet: HTTP and SMTP" and Chapter 17, "Internet Access to Advanced Queuing" for more information.

Propagation over the Internet: HTTP and SMTP

Figure 1-5 shows the architecture for performing AQ operations over HTTP. The major components are:

• The AQ client program

• The Web server/ServletRunner hosting the AQ servlet

• The Oracle database server

The AQ client program sends XML messages (conforming to IDAP) to the AQ servlet, which understands the XML message and performs AQ operations. Any HTTP client, for example Web browsers, can be used. The Web server/ServletRunner hosting the AQ servlet interprets the incoming XML messages. Examples include Apache/Jserv or Tomcat. The AQ servlet connects to the Oracle database server and performs operations on the users' queues.

Figure 1-5 Architecture for Performing AQ Operations Using HTTP

Text description of the illustration adq81430.gif

Figure 1-6 shows additional components in the architecture for sending AQ messages over SMTP:

• Email server

• LDAP server (Oracle Internet Directory)

The email server verifies client signatures using certificates stored in LDAP and then routes the request to the AQ servlet.

Figure 1-6 Architecture for Performing AQ Operations Using SMTP

Text description of the illustration adq81431.gif

The Internet Data Access Presentation (IDAP)

The Internet Data Access Presentation (IDAP) uses the Content-Type of text/xml to specify the body of the request containing an XML-encoded method request. XML provides the presentation for IDAP request and response messages as follows:

• All protocol tags are scoped to the IDAP namespace.

• The sender includes namespaces in IDAP elements and attributes.

• The receiver processes IDAP messages that have correct namespaces; for requests with incorrect namespaces, the receiver returns an invalid request error.

• The receiver processes IDAP messages without namespaces as though they had the correct namespaces, if the context is valid.

• The IDAP namespace has the value http://ns.oracle.com/AQ/schemas/envelope

• An XML document forming the request of an IDAP invocation does not require the use of an XML DTD or a schema.

For more information about IDAP, see Chapter 17, "Internet Access to Advanced Queuing"

Nonrepudiation and the AQ$<QueueTableName> View

Advanced Queuing maintains the entire history of information about a message along with the message itself. You can look up history information by using the AQ$<QueueTableName> view. This information serves as the proof of sending and receiving of messages and can be used for nonrepudiation of the sender and nonrepudiation of the receiver. See Chapter 10, "Administrative Interface: Views" for more information about the AQ$<QueueTableName> view.

The following information is kept at enqueue for nonrepudiation of the enqueuer:

• AQ agent doing the enqueue

• Database user doing the enqueue

• Enqueue time

• Transaction ID of the transaction doing the enqueue

The following information is kept at dequeue for nonrepudiation of the dequeuer:

• AQ agent doing dequeue

• Database user doing dequeue

• Dequeue time

• Transaction ID of the transaction doing dequeue

After propagation, the Original_Msgid field in the destination queue of propagation corresponds to the message ID of the source message. This field can be used to correlate the propagated messages. This is useful for nonrepudiation of the dequeuer of propagated messages.

Stronger nonrepudiation can be achieved by enqueuing the digital signature of the sender at the time of enqueue with the message and by storing the digital signature of the dequeuer at the time of dequeue.