| Oracle® Retail Integration Bus Implementation Guide Release 13.0 |
|
 Previous |
 Next |
The RIB is designed as an asynchronous publication and subscription messaging integration architecture. This allows the decoupling of applications and their systems. For example, a publishing application need not know about the subscribing applications, other than the requirement that at least one durable subscriber must exist. It decouples the systems operationally. Once a subscriber is registered, the RIB persists all published messages until all subscribers have seen them.
The publishing adapter does not know, nor care, how many subscribers are waiting for the message, what types of adapters the subscribers are, what the subscribers' current states are (running or stopped), or where the subscribers are located. Delivering the message to all subscribing adapters is the responsibility of the RIB with the help of the underlying JMS server.
Physically, the message must reside somewhere so that it is available until all subscribers have processed it. The RIB uses the JMS specification for its messaging infrastructure. The JMS accepts the message from the publisher and saves it to stable storage, a JMS topic, until it is ready to be picked up by a subscriber. In all cases, message information must be kept on the JMS until all subscribers have read and processed it.
The RIB interfaces are organized by message family. Each message family contains information specific to a related set of operations on a business entity or related business entities. The publisher is responsible for publishing messages in response to actions performed on these business entities in the same sequence as they occur.
Each message family has specific message payloads based on agreed upon business elements between the Oracle Retail applications.
The design and architecture of the RIB infrastructure is based on two key requirements driven by the Oracle Retail application business model.
The RIB must preserve and persist all business events (messages) until all applications (subscribers) have looked at the message and have successfully consumed it or decided they do not care about that event (message). In other words RIB must deliver to every subscriber all messages except those filtered per a subscribing application's requirements.
A business event (message) must be redelivered to the consumer application if the business event (message) was not consumed successfully. The redelivery process is bound by the same rules of sequencing as normal (non-redelivered) business event (message).
The business event (message) must be delivered to all the subscribing applications in the order (FIFO) the business event (messages) was published by the publishing application.
To enable this, the publishing application defines a business object ID whose existence informs RIB that this and all subsequent messages with the same business object ID have to be processed in order. Business event (message) ordering (FIFO) is assured only for messages with the same business object ID within the same message family.
The RIB messaging adapters and payloads are designed around the concept of a message family.
Each RIB message belongs to a specific message family. Each message family contains information specific to a related set of operations on a business entity or related business entities. The publisher is responsible for publishing messages in response to actions performed on these entities in the same sequence as they occur.
One example of a message family is the orders message family used to contain information about purchase order events.
A message family may contain multiple message types. Each message type encapsulates the information specific to a business entity within one or more business events. For example, the order message family is published for events such as "Create PO Header", "Create PO Detail", "Update PO Header", or "Delete PO Detail"
A single business event, such as updating a purchase order, may involve multiple business entities, such as a line item within the purchase order.
Because a single business event may involve multiple business entities, the application may publish messages for this event from multiple message families for a single business transaction. More than one message type within a message family may also be created.
There are two broadly defined types of functional interfaces in the RIB (message families); foundation data and transactional data.
After populating application tables with initial company seed data, item foundation information is needed. Foundation messages are defined as those with payload that carry basic product data.
This table is an example from the RIB Integration Guide.
After populating application tables with initial seed data and after all required item foundation data messages have been subscribed to, all applications are prepared to publish and subscribe transaction data messages. Transactional messages communicate business events involving two or more organizations within a retail supply chain, for instance, between Oracle Retail Merchandising System (RMS), Oracle Retail Store Inventory Management (SIM), and Oracle Retail Warehouse Management System (RWMS), external suppliers and financial systems.
This table is an example from the RIB Integration Guide.
Table 3-2 Transaction Data
| Functional Area | Publishing Applications | Subscribing Applications |
|---|---|---|
|
Allocations |
RMS |
RWMS |
|
Appointments |
RWMS |
RMS |
|
ASN Outbound |
RWMS, SIM |
RMS, SIM, RWMS |
|
ASN Inbound |
RWMS, External |
RMS, SIM, RWMS |
|
Inventory Adjustments |
RWMS, SIM |
RMS |
|
Inventory Request |
SIM |
RMS |
|
Receipts |
RWMS, SIM |
RMS |
|
Purchase Order |
RMS, SIM |
RWMS, SIM |
|
Stock Order Status |
RWMS, SIM |
RMS |
|
Transfers |
RMS |
RWMS, SIM |
Whenever a publishing application adapter publishes a message, it wraps the message in an envelope known as the RIB message envelope. The envelope is a standard message delivery format where the message information, the data payload, is contained within the overall delivery information. The envelope itself provides information that the RIB uses, such as RIB hospital information and routing information.
The publishing application is responsible for creating the initial message contents. The RIB publishing adapter publishes it to the JMS Server and makes it available to any JMS subscribers. The RIB knows what subscribers are to receive the message due to the RIB configuration—this configuration associates a set of subscribers to each publisher and message family combination.
For PL/SQL Applications, database tables associated with the publishing application typically stage message information. One or more RIB publishing adapters poll the application via a stored procedure call. For Java EE Applications, the application calls a RIB Enterprise Java Bean (EJB) with the payload information to be published.
The message resides on a Java Message Service (JMS) immediately after publication. The JMS topic provides stable storage for the message in case a system crash occurs before all message subscribers receive and process it.
A fundamental RIB system requirement is that a message must be delivered to and processed successfully exactly once by each subscriber. Furthermore, all work performed by the subscriber and the RIB must be atomically committed or rolled back, even if the JMS server is on a remote host. The standard way to perform this is by using an XA compliant interface and two-phase commit protocol.
After initial publication, a message may undergo a series of transformation, filtering, or routing operations. A RIB component that implements these operations is known as a Transformation and Address Filter/Router (TAFR) component. TAFR is the acronym for Transform, Address, Filter, and Route. A TAFR is completely internal to the RIB and does not reside in either the publishing or subscribing application. The RIB performs these intermediate transformation and routing operations on some messages before making them available to the subscribing application.
A single TAFR may only transform a given message, only filter the message, only route it, or combine any of the three operations.
Transform - A message may be transformed from one message type into another, for example, 'WH' (warehouse) from RMS to 'Location' for RWMS.
Filter - A message may be filtered. Filtering can occur based on message type or based on content.
Route - A TAFR may route a message. For example, whenever a stock order message is published for a warehouse with an instance of RWMS, the TAFR routes it to the particular RWMS instance from where the stock will be fulfilled and not to warehouses that do not stock the order's items.
TAFR operations are specific to the set of subscribers to a specific message family. Multiple TAFRs may process a single message for a specific subscriber and different specific TAFRs may be present for different subscribers. Different sets of TAFRs are necessary for different message families. If all subscribers to a message can process all messages within a message family without any TAFR operations, then no TAFR components are needed.
Message processing continues until a subscribing adapter successfully processes the message or determines that no subscriber needs this message.
When a subscriber gets a message to be processed, the adapter checks to see if the RIB Hospital contains any messages associated with the same entity as the current message. If so, then the adapter places the current message in the hospital as well. This is to ensure messages are always processed in the proper sequence. If proper sequencing is not maintained, then the subscribing application's data can get corrupt.
If an error occurs during message processing, the subscribing adapter notes this internally and rolls back all database work associated with the message. When the message is re-processed (since it has yet to be processed successfully), the adapter now recognizes this message is problematic and checks it into the hospital.
After a message is checked into the RIB Hospital, a retry adapter extracts the message from the hospital and re-publishes it to the JMS topic for reprocessing. The message remains in the hospital during all re-tries until the subscribing adapter successfully processes it.
The RIB is a messaging system made-up of components that are packaged and shipped as an integration solution between the Oracle Retail applications. The application boundary between RIB and Oracle Retail applications can be confusing at times, so this section defines the RIB components and their responsibility and ownership. A diagram illustrating the RIB integration message flow follows:
This section describes the components of the RIB subsystem.
A RIB adapter is a component that coordinates business event (message) generation and processing with the respective Oracle Retail application interface. Each adapter in the RIB is created to handle a specific functional interface. RIB adapters are developed using Enterprise Java Beans (EJB) components architecture, subscribing adapters use Message Driven Beans (MDBs) and publishing adapters use Stateless Session Beans (SLSBs).
The RIB provides four types of adapters that Oracle Retail applications can exploit to integrate with one another. These adapter types are: publisher, subscriber, TAFR, and hospital retry. They have been built using different technologies based on their particular needs.
Subscriber and TAFR adapters use Message Driven Bean (MDB) technology to register with JMS topics and receive messages for further processing.
Publisher and hospital retry adapters make use of the Java SE (Standard Edition) timer facility to schedule repetitive events that trigger calls to Stateless Session Beans (SLSBs) to query application tables for messages to publish to the JMS server.
As stated in the introduction, a fifth type of adapter exists for publishing messages in a pushing fashion, which the Oracle Retail applications invoke at will for publishing messages. These adapters have not been considered part of the scope of this technical document in regard to providing a mechanism for starting and stopping them.
Due to the variety of technologies used by the adapters, the goal of this technical design has been to isolate users from these differences and provide them with a common management interface that can be used to control the state of the adapters. During the last few years, the Java Management Extensions (JMX) specification has become a well known standard that defines the management layer for enterprise Java applications. JMX defines standard methodologies for declaring enterprise application components as manageable resources that can be exposed in a consistent way such that any JMX compliant management application can access and provide means for control.
JMS defines two types of messaging domains: point-to-point and publish/subscribe. RIB uses publish/subscribe types of messaging domains for all its communication. Publish/subscribe is a one-to-many type of message distribution model where one source application en-queues the message and many destination applications can de-queue the same message and process independently of the other peer applications. In publish/subscribe the destinations are known as topics, the en-queuer application is known as publisher, and the de-queuer is known as subscriber. Unlike point-to-point, in publish/subscribe the publisher and subscriber are totally ignorant of each other and do not and should not know about each others existence. The JMS Topics retain the messages only as long as it takes to distribute them to current active (running) subscribers. There is also a timing dependency between publishers and subscribers. A client that subscribes to a topic can consume only messages published after the client has created a subscription, and the subscriber must continue to be active in order for it to consume messages. The JMS specification relaxes this timing dependency to some extent by allowing clients to create durable subscriptions. By creating durable subscriptions the JMS server will continue to hold the messages for all registered subscribers for that topic until the subscriber consumes the message or deletes the subscription. There are two types of subscribers, non-durable and durable subscribers. The RIB uses only durable subscribers which allows the Oracle Retail edge applications to be in up or down state independently but still not loose any messages and catch up when the application comes back up. Every subscribing RIB adapter registers its durable subscriber with a subscription name that contains its rib-<app> application name and the adapter name in it.
RIB defines logical grouping of retail specific business objects (BO) and business functions in a concept called message family. For every message family there is a corresponding JMS topic. These JMS topics are used as communication pipelines between the source and destination Oracle Retail applications for exchanging the business objects.
The list of JMS topics used by RIB components is detailed in the RIB Integration Guide - Reports.
A key aspect of the JMS usage that the RIB relies on is the attachment of message properties to published messages and the use of selectors by message subscribers. Message properties are used to convey information about the message outside of the actual message data to establish a logical channel for messages.
JMS message selectors are used by the RIB to filter the messages that each subscriber picks up. In other words, using the message properties, selectors act as a filter to weed out messages a subscriber should not process.
The message property set and used by the RIB messages is called threadValue. The thread value is associated with a logical channel of a message stream. All messages for a specific family with a specific business object ID always contain the same threadValue property. This, combined with the standard first in, first out (FIFO) message ordering on the topic, is integral to message sequencing. Messages with different threadValue properties are not guaranteed to be processed in the same relative order as publishing.
Messages published without JMS Message Property present will not be picked up by the standard subscribing RIB adapters.
The typical lifecycle of a message through the RIB is as follows:
First, the publishing adapter creates the message. The event that triggers the message creation may be a polling operation in case of PL/SQL applications or a synchronous invoke in case of Java EE applications. The message is published to a predetermined JMS topic.
The message is now available for all registered subscribers to the JMS topic for pick up. Subscription is based on the message family.
Once a subscriber gets the message, it is free to process it according to its own rules. In the case of a transformer adapter, the adapter can open the message, modify its contents, and then publish the modified message to a new topic. The source topic and destination topic that a TAFR uses must always be distinct/different topic. There may be new subscribers to the modified message, and the scenario repeated for each of these subscribers.
When each subscriber has finished (commit) processing a message, the JMS server updates the state of the message to reflect it has been processed by this subscriber.
The JMS Server deletes the messages on the topic after delivering it to all the registered subscribers.
The figure below is a generalized view of a simple RIB message flow that involves all of the basic components. Two applications require this data and subscribe to it. One subscribing application requires a certain transformation be applied to the data, but the other subscriber can process the message without any transformations.
The RIB Hospital is a collective term for a set of Java Classes and database tables whose purpose is to provide a mechanism to handle system and business related errors while meeting the fundamental RIB requirements:
Guaranteed once-and-only-once successful delivery.
Preservation of publication sequence (even in case of failures).
When a message is processed, the adapter checks to see if the RIB Hospital contains any messages associated with the same businessObjectId as the current message. If so, then the adapter places the current message in the hospital as well. This is to ensure messages are always processed in the proper sequence. If proper sequencing is not maintained, then the subscribing application's data can get corrupted.
If an error occurs during message processing, the subscribing adapter notes this internally and rolls back all work associated with the message. When the message is re-processed (since it has yet to be processed successfully), the adapter now recognizes this message is problematic and checks it into the hospital.
For Publication, there are some RMS publishers that return an 'H' status to denote a problem creating a new message for a specific business object. This status may be due to database locks being held by on-line users of an Oracle Forms application or it could also be due to some data incompatibility found in the GETNXT() procedure. Whenever a publisher recognizes that a message for a business object cannot be published due to one of these conditions, the message must go into the RIB Hospital.
After a message is checked into the RIB Hospital, a retry adapter extracts the message from the hospital and tries to re-publishes it to the integration bus.
The RIB Hospital dependency check logic assumes that each message family has a single unique businessObjectId for all business object entities its messages are associated with. This businessObjectId must be the same for the same business entity across all message types within the message family. If any message for a specific business entity is placed into the RIB Hospital, then the RIB Hospital dependency check logic automatically inserts any subsequent messages for the same business object. This is to preserve the message sequencing and guaranteed exactly once successful message processing. Otherwise, multiple update messages for a business object may be processed in an incorrect order and create incompatibilities between applications.
If the businessObjectid is not set, then there is no dependency check. Not all message families set the businessObjectId or it is not set on all message types. See the Oracle Retail Application Documentation (for example, RMS Operations Guide Volume 2 - Message Publication and Subscription Designs).
If a message is to be inserted into the RIB Hospital because of an error during processing, it is sent to the subscribing adapter twice. This is because subscribing adapters are executed within the context of a distributed transaction, using the XA two-phase commit protocol. This transaction is controlled by the Java EE Application Server. If the RIB adapter returns success, the application server removes the message and all database work is committed. If the adapter returns failure, the message never leaves the integration bus topic and the database work is rolled back.
When the initial failure occurs while processing the message, the error is flagged within the RIB Hospital software, the adapter returns failure so that the database transaction is rolled back, and the message is kept on the integration bus topic.
|
Note: The XA interface is a standard protocol between a transaction manager and a database or resource manager. Note that both the JMS topic connection and the database connection must support the XA protocol. For more information regarding the XA standard, see the URL http://www.opengroup.org. |
The RIB Hospital tables are:
RIB_MESSAGE - contains the message payload, all single-field envelope information, and a concatenated string made from <id> tags. It also contains a unique hospital ID identifying this record within the hospital.
RIB_MESSAGE_FAILURE - contains all failure information for each time the message was processed.
RIB_MESSAGE_ROUTING_INFO - contains all of the routing element information found in the message envelope.
RIB_MESSAGE_HOSPITAL_REF - contains all of the hospital reference information found in the message envelope.
A database sequence, RIB_MESSAGE_SEQ, is used to maintain a unique message number associated with each message placed into the RIB Hospital.
These tables will have been created during the database portion of the Oracle Retail application install (for example, RWMS, SIM, RPM, AIP, or RMS).
The RIB Hospital tables are internal system tables that maintain the RIB runtime state of the system. The entries in these tables must not be manipulated by non RIB tools when the RIB is running.
After a message is inserted into the RIB Hospital, the hospital retry adapter is used to re-post the message to the JMS in order to retry its processing. The assumption is that the error is a transitory one - records locked or there is an external dependency that has not been met. The number of times a message is retried is configurable.
The hospital retry is responsible for maintaining state information for hospital records - what has happened to the record or message information. Each time the message is re-processed, a record is kept of the event along with the results. The design is to provide a means to halt processing for messages that cause errors while allowing continued processing for the good messages.
One element of this information is whether the message has been queued to the JMS topic for re-try processing. Thus, manually deleting messages from the hospital database using SQL directly may produce severe processing problems. Similarly, deleting messages directly from the JMS provider may result in a message that is never retried again, as the logic in the retry assumes the message is queued within the JMS.
There are three kinds of hospital retry adapters:
Sub Retry Adapter
Jms Retry Adapter
Pub Retry Adapter.
All subscriber side retrying of messages are handled by the Sub Retry Adapter. The Sub Retry Adapter looks at all messages with reason code "SUB", then filters and identifies the messages that are ready to be reprocessed, keeping message ordering in mind.
Oracle Retail applications are not aware of the fact that the integrations of the business data is happening through a JMS server. RIB abstracts that fact that it is using a JMS server from the retail applications. When the JMS server is down or RIB has some problem publishing to the JMS server, RIB will not rollback the transaction as long as it is a recoverable problem. In such situation all messages are inserted to the RIB Hospital with a reason code of "JMS" and publications continues on. The JMS Retry Adapter retries all messages with reason code of "JMS" at a later time.
All messages with reason code of "PUB" are retried by the Pub Retry Adapter. RMS is the only retail application that needs the Pub Retry Adapter.
When the message first comes through the subscriber, if there is no businessObjectid, then there is no dependency check performed. If the message cannot be processed, it is then inserted into the hospital with an attempt_count = 1.
A message that comes through the subscriber that has a businessObjectid has a dependency check performed. If there is no dependency and the message cannot be processed, it Is then inserted into the hospital with an attempt_count = 1.
A message that comes through the subscriber that does match the ID and family of another message in the hospital is known to be dependent, so it goes to the hospital immediately, with an attempt_count = 0.
