25 JMS XML

This chapter describes how to work with Java Message Services (JMS) with a XML payload in Oracle Data Integrator.

This chapter includes the following sections:

• Section 25.1, "Introduction"

• Section 25.2, "Installation and Configuration"

• Section 25.3, "Setting up the Topology"

• Section 25.4, "Setting Up an Integration Project"

• Section 25.5, "Creating and Reverse-Engineering a JMS XML Model"

• Section 25.6, "Designing an Interface"

25.1 Introduction

Oracle Data Integrator provides a simple and transparent method to integrate JMS destinations. This chapter focuses on processing JMS messages with a XML payload. For text payload processing in batch mode, refer to Chapter 24, "JMS".

25.1.1 Concepts

The JMS XML Knowledge Modules apply to most popular JMS compliant middleware, including Oracle Service Bus, Sonic MQ, IBM Websphere MQ, and so forth. Most of these Knowledge Modules include transaction handling to ensure message delivery.

25.1.1.1 JMS Message Structure

See Section 24.1.1.1, "JMS Message Structure" for information about the JMS message structure.

25.1.1.2 Using a JMS Destination

Oracle Data Integrator is able to process XML messages that are delivered by a JMS destination. Each message is considered as a container for XML data and is handled through the JMS XML Queue or JMS XML Topic technology.

With JMS XML Queue/JMS XML Topic technologies, each messages payload contains a complete XML data structure. This structure is mapped into a relational schema (XML Schema) that appears as a model, using the Oracle Data Integrator XML Driver.

Note:

This method is extremely similar to XML files processing. In JMS XML, the message payload is the XML file. See Chapter 5, "XML Files" and Appendix B, "Oracle Data Integrator Driver for XML Reference" for more information about XML Files processing and the XML Driver.

In the topology, each JMS destination is defined as a JMS XML Topic/Queue data server with a single physical schema. A data server/physical schema pair will be declared for each topic or queue delivering message in the XML format.

The structure of the XML message mapped into a relational structure (called the XML schema) appears as a data model. Each datastore in this model represents a portion (typically, an element type) in the XML file.

Processing Messages

As each XML message corresponds to an Oracle Data Integrator model, the entire model must be used and loaded as one single unit when a JMS XML message is consumed or produced. The processing unit for an XML message is the package.

It is not possible to declare the properties or header fields of the message in the model or use them as columns in an interface. They still can be used in message selectors, or be set through KM options.

Consuming an XML message

Processing an incoming XML message is performed in packages as follows:

1. Synchronize the JMS message to the XML schema: This operation reads the message and generates the XML schema. This is usually performed by the first interface accessing the message.

2. Extract the data: A sequence of interfaces use datastores from the XML schema as sources. This data is usable until the session is terminated, another message is read by a new Synchronize action, or the Commit JMS Read is performed.

3. Commit JMS Read: This operation validates the message consumption and deletes the XML schema. It should be performed by the last interface which extracts data from the XML message.

Producing an XML message

To produce an XML message, a package must be designed to perform the following tasks:

1. Initialize the XML schema: This operation creates an empty XML schema corresponding to the XML message to generate. This operation is usually performed in the first interface loading the structure.

2. Load the data: A sequence of interfaces loads data into the XML schema.

3. Synchronize the XML schema to JMS: This operation converts the XML schema to an XML message, and sends it to the JMS destination. This operation is usually performed by the last interface loading the schema.

Filtering Messages

It is possible to filter messages from a JMS XML destination by defining a Message Selector (MESSAGE_SELECTOR KM option) to filter messages on the server. This type of filter can use only the properties or header fields of the message. The filter is processed by the server, reducing the amount of information read by Data Integrator. It is also possible to filter data in the interface using data extracted from the XML schema. These filters are processed in Data Integrator, after the message is synchronized to the XML schema.

25.1.2 Knowledge Modules

Oracle Data Integrator provides the Knowledge Modules (KM) listed in Table 25-1 for handling XML messages.

Table 25-1 JMS XML Knowledge Modules

Knowledge Module	Description
IKM SQL to JMS XML Append	Integrates data into a JMS compliant message queue or topic in XML format from any ANSI SQL-92 standard compliant staging area.
LKM JMS XML to SQL	Loads data from a JMS compliant message queue or topic in XML to any ANSI SQL-92 standard compliant database used as a staging area.

25.2 Installation and Configuration

Make sure you have read the information in this section before you start using the JMS Knowledge Modules:

• System Requirements and Certifications

• Technology Specific Requirements

• Connectivity Requirements

25.2.1 System Requirements and Certifications

Before performing any installation you should read the system requirements and certification documentation to ensure that your environment meets the minimum installation requirements for the products you are installing.

The list of supported platforms and versions is available on Oracle Technical Network (OTN):

http://www.oracle.com/technology/products/oracle-data-integrator/index.html.

25.2.2 Technology Specific Requirements

The JMS destinations are usually accessed via a JNDI service. The configuration and specific requirements for JNDI and JMS depends on the JMS Provider you are connecting to. Refer to the JMS Provider specific documentation for more details.

25.2.3 Connectivity Requirements

This section lists the requirements for connecting to a JMS XML database.

Oracle Data Integrator does not include specific drivers for JMS providers. Refer to the JMS Provider documentation for the connectivity requirement of this provider.

XML Configuration

XML content is accessed through the Oracle Data Integrator JDBC for XML driver. The driver is installed with Oracle Data Integrator.

Ask your system administrator for the location of the DTD file describing the XML content.

25.3 Setting up the Topology

Setting up the Topology consists of:

1. Creating a JMS XML Data Server

2. Creating a JMS XML Physical Schema

25.3.1 Creating a JMS XML Data Server

An JMS XML data server corresponds to one JMS provider/router that is accessible through your local network.

There are two types of JMS XML data servers: JMS Queue XML and JMS Topic XML.

• A JMS Queue XML data server is used to connect a single queue in the JMS router to integrate XML messages.

• A JMS Topic XML data server is used to connect a single Topic in the JMS router to integrate XML messages.

The Oracle Data Integrator JMS driver loads the messages that contains the XML content into a relational schema in memory. This schema represents the hierarchical structure of the XML message and enables unloading the relational structure back to the JMS messages.

25.3.1.1 Creation of the Data Server

Create a data server either for the JMS Queue XML technology or for the JMS Topic XML technology using the standard procedure, as described in "Creating a Data Server" of the Oracle Fusion Middleware Developer's Guide for Oracle Data Integrator.

The creation process for a JMS XML Queue or JMS Topic XML data server is identical to the creation process of an XML data server except that you need to define a JNDI connection with JMS XML specific information in the JNDI URL. See Section 5.3.1, "Creating an XML Data Server" for more information.

This section details only the fields required or specific for defining a JMS Queue XML or JMS Topic XML data server.

1. In the Definition tab:

   • Name: Name of the data server as it will appear in Oracle Data Integrator.

   • User/Password: Not used here. Leave these fields empty.

2. In the JNDI tab:

   • JNDI Authentication: From the list, select the authentication mode.

   • JNDI User: Enter the username to connect to the JNDI directory (not mandatory).

   • Password: This user's password (not mandatory).

   • JNDI Protocol: From the list, select the JNDI protocol (not mandatory).

   • JNDI Driver: Name of the initial context factory java class to connect to the JNDI provider, for example:

     com.sun.jndi.ldap.LdapCtxFactory

   • JNDI URL: <JMS_RESOURCE>?d=<DTD_FILE>&s=<SCHEMA>&JMS_DESTINATION=<JMS_DESTINATION_NAME>.

     The JNDI URL properties are described inTable 25-2.

   • JNDI Resource: Logical name of the JNDI resource corresponding to your JMS Queue (or Topic) connection factory.

     Note:

     Specify QueueConnectionFactory if you want to access a message queue and TopicConnectionFactory if you want to access a topic. Note that these parameters are specific to the JNDI directory.

     Table 25-2 JNDI URL Properties

     Parameter	Value	Notes
     d	<DTD File location>	DTD File location (relative or absolute) in UNC format. Use slash “/” in the path name and not backslash “\” in the file path. This parameter is mandatory.
     re	<Root element>	Name of the element to take as the root table of the schema. This value is case sensitive. This parameter can be used for reverse-engineering a specific message definition from a WSDL file, or when several possible root elements exist in a XSD file.
     ro	true | false	If true, the XML file is opened in read only mode.
     s	<schema name>	Name of the relational schema where the XML file will be loaded.This value must match the one set for the physical schema attached to this data server. This parameter is mandatory.
     cs	true | false	Load the XML file in case sensitive or insensitive mode. For case insensitive mode, all element names in the DTD file should be distinct (Ex: Abc and abc in the same file are banned). The case sensitive parameter is a permanent parameter for the schema. It CANNOT be changed after schema creation. Please note that when opening the XML file in insensitive mode, case will be preserved for the XML file.
     JMSXML_ROWSEPARATOR	5B23245D	Hexadecimal code of the string used as a line separator (line break) for different XML contents. Default value is 5B23245D which corresponds to the string [#$].
     JMS_DESTINATION	JNDI Queue name or Topic name	JNDI Name of the JMS Queue or Topic. This parameter is mandatory.
     tna	boolean (true|false	Transform Non Ascii. Set to false to keep non-ascii characters. Default is true. This parameter is not mandatory.

     
     

Example

If using an LDAP directory as the JNDI provider, you should use the following parameters:

• JNDI Driver: com.sun.jndi.ldap.LdapCtxFactory

• JNDI URL: ldap://<ldap_host>:<port>/<dn>?d=<DTD_FILE>&s=<SCHEMA>&JMS_DESTINATION=<JMS_DESTINATION_NAME>

• JNDI Resource: <Name of the connection factory>

25.3.2 Creating a JMS XML Physical Schema

Create a JMS XML physical schema using the standard procedure, as described in "Creating a Physical Schema" of the Oracle Fusion Middleware Developer's Guide for Oracle Data Integrator.

Note:

For the name of the Schema and Work Schema use the schema name defined in the s=<schema name> property of the JNDI URL of the JMS Queue XML or JMS Topic XML data server.

Note:

Only one physical schema is required per JMS XML data server.

Create for this physical schema a logical schema using the standard procedure, as described in "Creating a Logical Schema" of the Oracle Fusion Middleware Developer's Guide for Oracle Data Integrator and associate it in a given context.

25.4 Setting Up an Integration Project

Setting up a project using JMS XML follows the standard procedure. See "Creating an Integration Project" of the Oracle Fusion Middleware Developer's Guide for Oracle Data Integrator.

It is recommended to import the following knowledge modules into your project for getting started with JMS XML:

• IKM SQL to JMS XML Append

• LKM JMS XML to SQL

25.5 Creating and Reverse-Engineering a JMS XML Model

This section contains the following topics:

• Create a JMS XML Model

• Reverse-Engineering a JMS XML Model

25.5.1 Create a JMS XML Model

Create a JMS Queue XML or JMS Topic XML Model using the standard procedure, as described in "Creating a Model" of the Oracle Fusion Middleware Developer's Guide for Oracle Data Integrator.

A JMS Queue XML or JMS Topic XML Model corresponds to a set of datastores, with each datastore representing an entry level in the XML file. The models contain datastores describing the structure of the JMS messages. A model contains the message structure of one topic or one queue. This model's structure is reverse-engineered from the DTD or the XML file specified in the data server definition, using standard reverse-engineering.

25.5.2 Reverse-Engineering a JMS XML Model

JMS XML supports Standard reverse-engineering - which uses only the abilities of the XML driver.

To perform a Standard Reverse-Engineering on JMS Queue XML or JMS Topic XML use the usual procedure, as described in "Reverse-engineering a Model" of the Oracle Fusion Middleware Developer's Guide for Oracle Data Integrator.

Oracle Data Integrator will automatically add the following columns to the tables generated from the XML data:

• Primary keys (PK columns) for parent-child relationships

• Foreign keys (FK columns) for parent-child relationships

• Order identifier (ORDER columns) to enable the retrieval of the order in which the data appear in the XML file.

These extra columns enable the hierarchical XML structure's mapping in a relational structure stored in the schema. See d Appendix B, "Oracle Data Integrator Driver for XML Reference" for more information.

25.6 Designing an Interface

The KM choice for an interface or a check determines the abilities and performance of this interface or check. The recommendations in this section help in the selection of the KM for different situations concerning XML messages.

25.6.1 Loading Data from a JMS XML Source

JMS XML can be used as a source or a target in an interface. Data from an XML message Queue or Topic can be loaded to any ANSI SQL-92 standard compliant database used as a staging area. The LKM choice in the Interface Flow tab to load data between JMS XML and another type of data server is essential for successful data extraction.

Oracle Data Integrator provides the LKM JMS XML to SQL for loading data from a JMS compliant message queue or topic in XML to any ANSI SQL-92 standard compliant database used as a staging area. This LKM uses the Agent to read selected messages from the source queue/topic and write the result in the staging temporary table created dynamically.To ensure message delivery, the message consumer (or subscriber) does not commit the read until the data is actually integrated into the target by the IKM.Consider using this LKM if one of your source datastores is an XML JMS message.

In order to load XML messages from a JMS provider, the following steps must be followed:

• The first interface reading the XML message from the JMS XML source must use the LKM JMS XML to SQL with the SYNCHRO_JMS_TO_XML LKM option set to Yes. This option creates and loads the XML schema from the message retrieved from the queue or topic.

• The last interface should commit the message consumption by setting the JMS_COMMIT to Yes.

Table 25-3 lists the JMS specific options of this knowledge module.

25.6.2 Integrating Data in a JMS XML Target

Oracle Data Integrator provides the IKM SQL to JMS XML Append that implements optimized data integration strategies for JMS XML. This IKM integrates data into a JMS compliant message queue or topic in XML format from any ANSI SQL-92 standard compliant staging area.

To use this IKM, the staging area must be different from the target.

In order to integrate XML data into a JMS XML target, the following steps must be followed:

• The first interface loading the XML schema must provide a value for the ROOT_TABLE (it is the model's table that corresponds to the root element of the XML file), and also set the INITIALIZE_XML_SCHEMA option to Yes.

  Note:

  The root table of the XML schema usually corresponds to the datastore at the top of the hierarchy tree view of the JMS XML model. Therefore the ROOT_TABLE parameter should take the value of the resource name for this datastore.

• The interfaces should load the datastores in the hierarchy order, starting by the top of the hierarchy and going down. The interfaces loading subsequent levels of the XML schema hierarchy should load the foreign key column linking the current hierarchy level to a higher one.

  For example, when loading the second level of the hierarchy (the one under the root table), the foreign key column should be set to '0' (Zero), as it is the value that is set by the IKM in the root table primary key when the root table is initialized.

• The last interface should send the XML schema to the JMS provider by setting the SYNCHRO_JMS_TO_XML parameter to Yes.

Example

An XML file format generates a schema with the following hierarchy of datastores:

+ GEOGRAPHY_DIM (GEO_DIMPK, ...)
  |
  +--- COUNTRY (GEO_DIMFK, COUNTRYPK, COUNTRY_NAME, ...)
       |
       +--- REGION (COUNTRYFK, REGIONPK, REGION_NAME, ...)

In this hierarchy, GEOGRAPHY_DIM is the root table, and its GEOGRAPHY_DIMPK column is set to '0' at initialization time. The tables should be loaded in the GEOGRAPHY_DIM, COUNTRY, REGION sequence.

• When loading the second level of the XML hierarchy (COUNTRY) make sure that the FK field linking this level to the root table level is set to '0'. In the model above, when loading COUNTRY, we must load the COUNTRY.GEOGRAPHY_DIMFK set to '0'.

• You must also link the records of REGION to the COUNTRY level by loading the REGION.COUNTRYFK column with values that correspond to a parent record in COUNTRY (having REGION.COUNTRYFK = COUNTRY.COUNTRYPK).

For more information on loading data to XML schemas, see Appendix B, "Oracle Data Integrator Driver for XML Reference".

Table 25-3 lists the JMS specific KM options of this IKM. Options that are specific to XML messages are in bold.

JMS XML Knowledge Modules Options

Table 25-3 lists the KM options for the LKM and IKM for JMS XML. Options that are specific to XML messages are in bold.

Although most options are the same for the LKM and IKM, there are only few differences:

• The INITIALIZE_XML_SCHEMA and ROOT_TABLE options are only provided for the IKM.

• The DELETE_TEMPORARY_OBJECTS and JMS_COMMIT options are only provided for the LKM.

• Set JMS_COMMIT to Yes to commit the message consumption on the Router (JMS XML).

Table 25-3 JMS Specific KM Options

Option	Used to	Description
CLIENTID	Read	Subscriber identification string. Not used for publication.
DURABLE	Read	D: Session is durable. Indicates that the subscriber definition remains on the router after disconnection.
INITIALIZE_XML_SCHEMA	Write	Initializes an empty XML schema. This option must be set to YES for the first interface loading the schema.
JMSDELIVERYMODE	Write	JMS delivery mode (1: Non Persistent, 2: Persistent). A persistent message remains on the server and is recovered on server crash.
JMSEXPIRATION	Write	Expiration delay in milliseconds for the message on the server [0..4 000 000 000]. 0 signifies that the message never expires. Warning! After this delay, a message is considered as expired, and is no longer available in the topic or queue. When developing interfaces it is advised to set this parameter to zero.
JMSPRIORITY	Write	Relative Priority of the message: 0 (lowest) to 9 (highest).
JMSTYPE	Write	Optional name of the message.
MESSAGEMAXNUMBER	Read	Maximum number of messages retrieved [0 .. 4 000 000 000]. 0: All messages are retrieved.
MESSAGESELECTOR	Read	Message selector in ISO SQL syntax for filtering on the router. See Section 24.7.1, "Using JMS Properties" for more information on message selectors.
MESSAGETIMEOUT	Read	Time to wait for the first message in milliseconds [0 .. 4 000 000 000]. If MESSAGETIMEOUT is equal to 0, then there is no timeout. MESSAGETIMEOUT and MESSAGEMAXNUMBER cannot be both equal to zero. If MESSAGETIMEOUT= 0 and MESSAGEMAXNUMBER =0, then MESSAGETIMEOUT takes the value 1. Warning! An interface may retrieve no message if this timeout value is too small.
NEXTMESSAGETIMEOUT	Read	Time to wait for each subsequent message in milliseconds [0 .. 4 000 000 000]. The default value is 1000. Warning! An interface may retrieve only part of the messages available in the topic or the queue if this value is too small.
ROOT_TABLE	Write	Resource name of the datastore that is the root of the XML model hierarchy.
SENDMESSAGETYPE	Write	Type of message to send (1 -> BytesMessage, 2 ->TextMessage).
SYNCHRO_XML_TO_JMS	Write	Generates the XML message from the XML schema, and sends this message. This option must be set to YES for the last interface that writes to the schema XML.