11
Configuring Heterogeneous Outbound Message Propagation

Overview of Outbound Message Propagation

Heterogeneous Outbound Message Propagation to MQSeries is accomplished in three steps:

1. dequeuing the messages from the Oracle AQ source queue

2. translating the message service properties

3. enqueuing the messages in the MQSeries destination queue.

After the propagation schedule has started for a source/destination tuple, the propagation process will use the Oracle AQ operational Application Programming Interface (API) to dequeue those messages that have specified a list of recipients or subscribers (if the queue has defined a list of subscribers), or the propagation process will use the Oracle AQ operational API to dequeue all messages from the source queue.

For every message dequeued, the Oracle AQ service properties that are associated with the message are translated (when applicable) to MQSeries properties.  Examples of translated Oracle AQ service properties are queue table, queue, message, and enqueue properties.

After the message service properties are translated, the message is enqueued at the MQSeries destination queue.  The Oracle9i Procedural Gateway for IBM MQSeries provides PL/SQLÔ access to MQSeries.

The outbound message propagation process will call the necessary gateway's procedure calls to perform the following tasks:

• to establish a connection with the MQSeries Queue Manager

• to open the MQSeries destination queue

• to set the necessary MQSeries properties (as defined in the second step)

• to enqueue the message

• to close the queue again

Translation of Oracle AQ/IBM MQSeries Interfaces

IBM MQSeries provides a set of services that is comparable to the set of services that is provided by Oracle AQ.  For outbound message propagation, Heterogeneous Message Propagation will provide dynamic, best-effort translations of Oracle AQ interfaces and message service properties into MQSeries properties.  The next section lists the Oracle AQ interfaces, message service properties, and options that applications can specify upon enqueuing a message.  The next section then describes how these items are translated to MQI (the native API for MQSeries applications).

Oracle AQ messages are entered into Oracle AQ queues using the enqueue() procedure (refer to the Oracle AQ documentation for more details on the dbms_aq package). 

Configuring Outbound Message Propagation

Configuration is needed on the OracleÒ server, the Oracle Procedural GatewayÒ, and the IBM MQSeries to facilitate message propagation to an IBM MQSeries queue

To propagate a message from Oracle Advanced Queue 'aq12' under Oracle server 'ora1' over to IBM MQSeries queue 'ny11' under queue manager 'mgr.ny.1', the following configuration is required:

Operational Interface for Individual Messages

Oracle AQ allows propagation to be defined for queues and individual messages.  This section describes how heterogeneous outbound message propagation will work for individual messages; the section hereafter will outline how outbound message propagation is configured for a queue.

For heterogeneous outbound message propagation of individual messages, we will re-use the Oracle AQ operational interface.  Oracle AQ application will be able to specify non-Oracle consumers for a message using the standard recipient list mechanism.  Upon enqueuing a message, the message_properties argument of the enqueue() procedure allows the Oracle AQ application to specify a list of recipients.  For heterogeneous outbound message propagation, the fields of the structure used to specify the recipient will contain the communication information to access the non-Oracle AQ destination queue.

Refer to the next figure with definitions of the Oracle AQ enqueue() procedure and Oracle AQ types involved in this model:

dbms_aq.enqueue (
      queue_name          IN       varchar2,
      enqueue_options     IN       enqueue_options_t,
      message_properties  IN       message_properties_t,
      payload             IN       raw,
      msgid               OUT      raw

TYPE dbms_aq.enqueue_options_t IS RECORD (
      visibility           binary_integer,
      relative_msgid       raw(16),
      sequence_deviation   binary_integer)

TYPE dbms_aq.message_properties_t IS RECORD (
      priority            binary_integer,
      delay               binary_integer,
      expiration          binary_integer,
      correlation         varchar2(128),
      attempts            binary_integer,
      recipient_list      aq$_recipient_list,
      exception_queue     varchar2(51),
      enqueue_time        date,
      state               binary_integer) 
TYPE dbms_aq.aq$_recipient_list_t IS TABLE OF
      sys.aq$_agent INDEX BY binary_integer

TYPE dbms_aq.sys.aq$_agent IS OBJECT (
      name                  varchar2(30),
      address               varchar2(1024),
      protocol              number)

Refer to the Oracle AQ documentation for detailed information on how to use the enqueue() procedure and its arguments.  For heterogeneous outbound message propagation, the usage is generally compatible with Oracle AQ propagation.  Some arguments have different behavior however, due to the limited capabilities of the non-Oracle message queuing system:

• The payload argument contains the application data.  The payload type should be in concert with the type of the queue specified in the queue_name argument and can either be RAW or ADT.  Heterogeneous outbound message propagation supports only RAW queues.  Therefore, only RAW payloads are supported. 

• The msgid argument is a message identifier assigned to a message by the Oracle AQ system.  The msgid argument is globally unique within the Oracle domain.  However, heterogeneous outbound message propagation cannot pass the msgid along with the message to the non-Oracle message queuing system.  Although IBM MQSeries allows applications to set msgid, Oracle AQ does not support the IBM MQSeries method of setting msgid.  For this reason, propagation of msgid values cannot be supported (because the msgid returned by the enqueue() procedure will not match the msgid of the propagated messages as assigned by the non-Oracle message queuing system).

  
  

  Note: Oracle AQ applications can use the correlation field for message identification, and correlation values will be preserved when propagated to non-Oracle message queuing systems.

  
  

• The sys.aq$_agent structure that is used to specify the recipient list of the message_properties argument plays a key role in heterogeneous outbound message propagation.  Its fields are used to specify the non-Oracle destination queue of the recipients.

Table 11-6 sys.aq$_agent.usage

The following example demonstrates how the Oracle AQ operational interface can be used for both Oracle AQ and heterogeneous outbound message propagation.  The code excerpt shows part of an Oracle AQ application that specifies both Oracle AQ and non-Oracle AQ consumers as recipients for a message entered into an Oracle AQ queue:

Example:

DECLARE

    enqueue_options    DBMS_AQ.ENQUEUE_OPTIONS_T;
    message_properties DBMS_AQ.MESSAGE_PROPERTIES_T;
    recipients         DBMS_AQ.AQ$_RECIPIENT_LIST_T;
    message_handle     RAW(16);
    message            RAW(255) := 'something';

BEGIN

    /* message receiver 1:  queue aq1 at remote Oracle Database */

    recipients(1) : = SYS.AQ$_AGENT('Joe', 'aq1@remoteAq', 0);

    /* message receive 2:   queue mq1 at MQSeries */

    recipients(2) : = SYS.AQ$_AGENT('John', 
                                    'mq1@mqseries', 
                                    DBMS_HQADM.MQSERIES_OUTBOUND);  

    message_properties.recipient_list := recipients;

    /* enqueue the message */

    DBMS_AQ.ENQUEUE(queue_name => 'msg_queue',
                    enqueue_options => queue_options,
                    message_properties => message_properties,
                    payload => message,
                    msgid => message_handle);

    COMMIT;

END;
/

The example propagates the same message to two different recipients:

As the protocol number 0 indicates, this is Oracle AQ propagation with the destination queue 'aq1' residing at a remote Oracle database referred to by database link 'remoteAq'.  The intended consumer of the message is consumer 'Joe'.

Protocol DBMS_HQADM.MQSERIES_OUTBOUND indicates outbound message propagation from Oracle AQ to MQSeries.  The destination is MQSeries queue 'mq1', which can be reached using the MQSeries gateway.  The gateway is referred to by the database link 'mqseries'.  Because MQSeries does not have the concept of message consumer,  'John' will be ignored during propagation.

Operational Interface for Queues

Oracle AQ allows propagation to be defined for queues and individual messages.  The previous section described how heterogeneous outbound message propagation works for individual messages.  This section will define how to configure heterogeneous outbound message propagation for an Oracle AQ queue.

The Oracle AQ administrator's application will be able to specify non-Oracle consumers for an Oracle AQ queue using the standard subscriber mechanism.  After having created a queue, the Oracle AQ administrator can use the add_subscriber() procedure to add a non-Oracle AQ subscriber.  Subscribers are defined using a sys.aq$_agent structure. 

The next figure shows the definitions of the Oracle AQ add_subscriber() procedure and Oracle AQ structures that are used for specifying heterogeneous outbound message propagation for Oracle AQ queues.

DBMS_AQADM.ADD_SUBSCRIBER (
     queue_name IN    varchar2,
     subscriber       sys.aq$_agent)

TYPE sys.aq$_agent IS OBJECT (
      name           varchar2(30),
      address        archar2(1024),
      protocol       number)

DECLARE
  subscriber SYS.AQ$_AGENT;
BEGIN
  /* Add an AQ subscriber */
  subscriber := SYS.AQ$_AGENT('Joe', 
                              'aq1@remoteAq', 
                              0);

  DBMS_AQADM.ADD_SUBSCRIBER(queue_name => 'source_queue',
                            subscriber => subscriber );

  /* Add an non-AQ subscriber */

  subscriber := SYS.AQ$_AGENT('John', 
                              'mq1@mqseries', 
                              DBMS_HQADM.MQSERIES_OUTBOUND);

  DBMS_AQADM.ADD_SUBSCRIBER(queue_name => 'source_queue',
                            subscriber => subscriber );

END;
/

Administrative Interfaces

For propagation in a homogeneous Oracle AQ environment, the administrator uses the Oracle AQ package DBMS_AQADM with procedures to schedule a propagation job.  Specifying an Oracle AQ propagation job requires specifying three items:

1. an Oracle AQ source queue name

2. the destination

3. the scheduling times

The Oracle AQ administration package in Oracle 8.0.4 and Oracle 8.1 cannot yet accommodate heterogeneous protocols.  For this reason, a separate scheduling package is used.  This scheduling package has procedures for the scheduling and unscheduling of heterogeneous outbound message propagation.  The procedures that are in this package resemble the Oracle AQ scheduling procedures. 

The procedures maintain schedules in dictionary tables.  These procedures are read by the propagation process to determine when a queue, and which queue, is due for another round of message propagation.  The schedule information is available through a view named DBA_OUTBOUND_SCHEDULES (refer to Appendix D, " Heterogeneous Message Propagation DBA Views").  The package that contains the scheduling procedures (both inbound and outbound) is named DBMS_HQADM and contains two procedures for outbound message propagation:

• SCHEDULE_PROPAGATION

• UNSCHEDULE_PROPAGATION

The two outbound scheduling procedures that are in the package are each composed of a name, purpose, syntax, usage, and error messages.  These two outbound scheduling procedures are described next.

dbms_hqadm.schedule_propagation(
      queue_name           in      varchar2,
      protocol             in      number,
      destination          in      varchar2,
      start_time           in      date default SYSDATE,
      duration             in      number default NULL,
      next_time            in      varchar2 default NULL,
      latency              in      number default 60 )

dbms_hqadm.unschedule_propagation(
      queue_name    in     varchar2,
      protocol      in     number,
      destination   in     varchar2 default NULL,
      force         in     boolean default FALSE)

BEGIN

  /* schedule propagation for AQ */

  DBMS_AQADM.SCHEDULE_PROPAGATION(queue_name  => 'aq1',
                                  destination => 'remoteAq');

  /* schedule outbound propagation for MQSeries */

  DBMS_HQADM.SCHEDULE_PROPAGATION(queue_name => 'mq1',
                           protocol => DBMS_HQADM.MQSERIES_OUTBOUND,
                           destination =>'QUEUE1@dblink');

END;
/

Security

Heterogeneous Message Propagation does not impose new authorization requirements on either Oracle AQ or non-Oracle operational applications.  For outbound message propagation, a sending Oracle AQ application enqueues the messages into the source queue but does not require authorization for the recipient and/or subscriber's queue at the non-Oracle message queuing system.  Non-Oracle consumers require authorization only for the non-Oracle AQ destination queue.

At the administration level, the user who schedules heterogeneous propagation for a source and/or destination tuple:

• requires authorization to access the Oracle AQ source queue.

• requires access to a database link for a gateway in order to access the destination queue at the non-Oracle message queuing system.

The user name and password specified for the database link require access privileges at the message queuing system of the destination queue.  If the database link does not specify a user name and password, then it will implicitly use the ID of the current Oracle user. 

Given these access privileges, the propagation process can dequeue the messages from the source queue and enqueue them through the gateway into the destination queue.  This method of controlling access privileges ensures that the security issue for outbound message propagation remains transparent to both sender and consumer applications.

Exception Handling

If the outbound message propagation process cannot propagate messages from the Oracle AQ source queue to their destination, then the messages will be moved to either a propagation exception queue or to an inactive transmit queue, as explained below.

For each schedule that is created by using the dbms_hqadm.schedule_propagation() procedure, an exception queue is created to stage messages that could not be successfully moved to the destination queue.  Administrators can check the exception_queue field in the table DBA_OUTBOUND_SCHEDULES (refer to Appendix D, " Heterogeneous Message Propagation DBA Views") to obtain the name of this exception queue and to monitor it.

Upon unscheduling a propagation job by using the dbms_hqadm.unschedule_propagation() procedure, the exception queue is removed again (for full details, refer to the description of this procedure at "Name:  UNSCHEDULE_PROPAGATION").

Inactive transmit queues are created by the propagation process upon encountering messages in a source queue for which no propagation has been scheduled yet.  Administrators can check the transmit_queue field in the table DBA_OUTBOUND_SCHEDULES (refer to Appendix D, " Heterogeneous Message Propagation DBA Views") to obtain the name of this queue (if any) and monitor it.

Upon unscheduling a propagation job using the dbms_hqadm.unschedule_propagation() procedure, the inactive transmit queue is removed again (for full details, also refer to the description of this procedure).

The Oracle AQ API allows the propagation process to dequeue messages by protocol but not by destination or source.  This means that messages with the same protocol but different destination or source are all dequeued at once.  To avoid this, the propagation process must perform a 2-phase-commit when distributing messages to their respective destinations.  The propagation process stages these messages into dedicated transmit queues.  Each transmit queue is reserved for a particular schedule.  If, however, no schedule was created for a message, and thus no transmit queue exists yet, then the propagation process creates an inactive transmit queue and moves the message in there.

Schedule Status

Status Information can be obtained both for an outbound message propagation schedule and for individual messages.

SELECT ERROR_TIME, ERROR, ERROR_MSG 
FROM DBA_OUTBOUND_SCHEDULES
WHERE DISABLED = 'Y';