23 Resequencing in Oracle Mediator
This chapter includes the following sections:
23.1 Introduction to the Resequencer
The resequencer in Mediator rearranges a stream of related but out-of-sequence messages into a sequential order. When incoming messages arrive, they may be in a random order. The resequencer orders the messages based on sequential or chronological information, and then sends the messages to the target services in an orderly manner. The sequencing is performed based on the sequencing strategy selected.
23.1.1 Groups and Sequence IDs
The resequencer works with two central concepts: groups and sequence IDs. The sequence ID is an identifying part of the message, and messages are rearranged based on this identifier. The messages arriving for resequencing are split into groups and the messages within a group are sequenced according to the sequence ID. Sequencing within a group is independent of the sequencing of messages in any other group. Groups in themselves are not dependent on each other and can be processed independently of each other.
As an example, messages attached to certain groups arrive to a Mediator service component in the following order:
msg9(a), msg8(b), msg7(a), msg6(c), msg5(a), msg4(b), msg3(c), msg2(b), msg1(a)
Table 23-1 shows how the Mediator sorts the messages into groups. The order of the messages in each group depends on the type of resequencer used.
Table 23-1 Messages Sorted into Groups
Group c | Group b | Group a |
---|---|---|
msg6(c), msg3(c) |
msg8(b), msg4(b), msg2(b) |
msg9(a), msg7(a), msg5(a), msg1(a) |
All the groups are processed independently of each other and any error occurring in ones group does not affect the processing of other groups.
23.1.2 Identification of Groups and Sequence IDs
Groups and sequence IDs are identified through XPath expressions in the message payload and header. You specify XPath expressions that point to the elements in the message payload on which grouping is done and on which sequencing is done.
In the message payload shown in Figure 23-1, CustomerId
is the field on which to base instance sequencing and Type
is the field on which to base grouping.
Note:
Resequencing is not supported for synchronous operations.
23.2 Resequencing Order
Mediator can resequence the incoming messages in a user-specified order. This implementation enables you to specify three types of resequencing orders:
23.2.1 Standard Resequencer
The standard resequencer supports a standard resequencer pattern. The following sections describe the standard resequencer and how it processes messages.
23.2.1.1 Overview of the Standard Resequencer
The standard resequencer is useful for applications that use identifiers from a simple numeric identifier sequence in their messages. The standard resequencer receives a stream of messages that might not arrive in order; it then stores the out-of-sequence messages until a complete sequence based on the sequence IDs is received. The in-sequence messages are then processed asynchronously based on their sequence ID.
It is important to note that the messages to outbound services of the standard resequencer Mediator service component are guaranteed to arrive in sequence.
23.2.1.2 Information Required for Standard Resequencing
When using the standard resequencer in Mediator, you must always specify a group XPath expression and a sequence ID XPath expression. These specify where the Mediator resequencer can find the group and the sequence ID in the messages. You must also supply the sequence numbering in terms of the start sequence ID and the sequence ID incremental delta. This numbering is used to form each group. In addition to the group, sequence ID, and increment properties, you can also specify a timeout period, in seconds, to wait for the expected messages.
The Mediator standard resequencer holds back messages in the Mediator resequencer database until it can produce the right sequence for different groups. This situation means that if for a given group, the message with a particular sequence ID does not arrive within the timeout period, the subsequent messages for that group are held back forever. In such a case, you must manually unlock the group through Oracle Enterprise Manager Fusion Middleware Control and go to the next available message, skipping the pending message.
23.2.1.3 Example of the Standard Resequencer
Table 23-2 shows how groups are formed differently for two different values of the incremental delta.
Table 23-2 Groups Formed Differently for Two Different Values
Start SequenceID | Incremental Delta | Group1 | Group2 | ... | Groupn |
---|---|---|---|---|---|
1 |
1 |
1,2,3,4,5,... |
1,2,3,4,5,... |
... |
1,2,3,4,5,...n |
1 |
5 |
1,5,10,15,... |
1,5,10,15,... |
... |
1,5,10,15,... |
Note:
If the sequence numbering is different for various groups (for example, if the groups do not have the same incremental delta or start sequence ID) and the messages do not arrive in order, then you can use the best effort resequencer to rearrange the messages.
23.2.2 FIFO Resequencer
The FIFO resequencer supports a standard first in, first out (FIFO) pattern. The following sections describe the FIFO resequencer and how it processes messages.
23.2.2.1 Overview of the FIFO Resequencer
The FIFO resequencer is useful for applications that need sequencing based on the time the messages arrive to the Mediator. The FIFO resequencer receives a stream of messages that are in order and processes them in sequence for each group based on the arrival time of the messages.
It is important to note that the messages to outbound services of the Mediator acting as a FIFO resequencer are guaranteed to arrive in order based on arrival time. Therefore, the messages are delivered in the order they were stored in the resequencer data store.
23.2.2.2 Information Required for FIFO Resequencing
When using the FIFO resequencer, you must always specify a group XPath expression. However, you do not need to specify a sequence ID because the messages are processed according to the time of arrival to the Mediator service component that is configured for FIFO resequencing. The group XPath expression specifies where the FIFO resequencer should find the group information in the message to group the messages. No further configuration is needed for a FIFO pattern.
23.2.2.3 Example of the FIFO Resequencer
Table 23-3 illustrates the behavior of the FIFO resequencer where msgX(Y,Z)
indicates that the message arrives as message number X to the Mediator service component and the message contains sequenceID
Y and group Z.
Table 23-3 FIFO Resequencer Behavior
Incoming Messages | Sequenced Messages |
---|---|
msg12(4,c) msg05(9,a) msg02(7,a) msg10(3,c) msg10(3,a) msg07(5,a) msg06(1,c) msg03(2,c) |
msg12(4,c),msg10(3,c),msg06(1,c),msg03(2,c) msg05(9,a), msg02(7,a), msg10(3,a), msg07(5,a) |
As shown in Table 23-3, the messages are sequenced based on their time of arrival and the sequenceID
is not used for sequencing.
Note:
When using the FIFO resequencer, use a single-threaded inbound adapter to avoid unpredictable results. For example, when you use the file/FTP adapter, the database adapter, or the AQ adapter in front of a Mediator service component that is configured as a FIFO resequencer, configure the adapter for single-threaded processing. Otherwise, unpredictable results occur because the arrival time of each message is calculated when the message arrives to the Mediator service component instead when it arrives to the adapter service.
23.2.3 Best Effort Resequencer
The Mediator resequencer supports a best effort pattern. The following sections describe the best effort resequencer and how it processes messages.
23.2.3.1 Overview of the Best Effort Resequencer
The best effort pattern is useful for applications that produce a large number of messages in a short period and cannot provide information to the resequencer about the identifier to use for sequencing. Typically, the identifier used for sequencing in such scenarios is of a dateTime
type or numeric
type. Using the dateTime
field as the sequence ID XPath enables you to control the sequencing. The messages are expected to be sent in sequence by the applications, thus the date and time the messages are sent can be used for sequencing. The Mediator makes the best effort to ensure that the messages are delivered in sequence.
The best effort resequencer can reorder messages based on no knowledge about the increment of the sequence ID. This situation means that unlike the standard resequencer, you do not need to define the increment of the sequence ID for the best effort resequencer in advance. When the messages are processed, they are processed in sequence based on the specified sequence ID and the messages that have arrived, whether a true sequence is received. The sequence IDs are either numeric
or dateTime
. Therefore, sequencing occurs on the numeric order or the dateTime
order of the sequence IDs.
23.2.3.2 Best Effort Resequencer Message Selection Strategies
The best effort resequencer processes messages asynchronously based on one of two message selection strategies: Maximum rows selected or time window. The messages selected and processed at any one time are based either on the maximum number of rows you specify or on a window of time in which they arrive.
23.2.3.2.1 Maximum Rows Selected
When the best effort resequencer is configured to use a maximum number of rows, it performs the following steps whenever new messages are available in the resequencer database:
-
The resequencer orders the messages according to the specified sequence ID (typically a date and time stamp).
-
The resequencer locks and selects the number of messages equal to the value of the
maxRowsRetrieved
parameter from the ordered messages above. -
The resequencer processes the selected messages one after another in its own transaction in sequence.
23.2.3.2.2 Time Window
When the best effort resequencer is configured to use a time window instead of a maximum number rows, the messages to select and process at one time are based on a period you specify plus an optional buffer time. Each message belongs to a specific time window, and messages that are part of one time window are processed separately from messages belonging to a different time window.
In addition to the time window, you can specify a buffer time, which is an overlap between two sequential time windows that allows messages that arrive a little late to be associated with the first time window. By default, the buffer time is 10% of the time window you specify.
When the best effort resequencer is configured to use a time window, groups are processed in an iterative manner and messages are processed in the following steps:
-
The first message arrives and the time window begins.
-
The buffer is added to the time window, and processing begins after the buffer time.
-
The resequencer retrieves the messages that arrived within the time window, and identifies the maximum sequence ID (typically a date and time stamp) from all the messages.
-
The resequencer retrieves any messages that arrive within the buffer time and that have a sequence ID that is less than the maximum sequence ID identified above.
-
The resequencer sorts all messages retrieved in the above steps in ascending order of the sequence IDs and processes the messages.
23.2.3.3 Best Effort Resequencer Message Delivery
It is important to note that the messages to outbound services of the Mediator service component configured for best effort resequencing are not guaranteed to arrive in order of a sequence ID. At any given time, a snapshot of the available messages is taken and sequencing is performed only on those messages. Therefore, unlike a standard resequencer, it is not guaranteed that a message with a lesser sequence ID value is sent before a message that ha a greater sequence ID value but that arrived earlier. Messages with a lesser sequence ID value that arrive later might be processed in the following cycle when a snapshot of available messages is taken again and the messages are reordered.
23.2.3.4 Information Required for Best Effort Resequencing
When using the best effort resequencer, you must specify a group XPath expression, a sequence ID XPath expression, and the data type of the sequence ID (numeric
or dateTime
). These specify where the resequencer should find the group and the sequence ID in the messages and how to handle the sequence ID. In addition, you must specify either a maximum number of rows to select for each resequencing batch or a time window during which the messages included in one batch arrive.
Unlike the standard resequencer, the best effort resequencer has no knowledge about how the sequence is built. No further information is used by the best effort resequencer to perform its responsibilities.
23.2.3.5 Example of Best Effort Resequencing Based on Maximum Rows
Table 23-4 illustrates the behavior of the best effort resequencer when it is configured to use the maximum number of rows to determine which messages to process. In this example, msgX(Y,Z)
indicates that the message arrives as message number X to the Mediator service component and the message contains sequenceID
Y and group
Z.
Table 23-4 Best Effort Resequencer Behavior Based on Maximum Rows
Group C | Sequenced Messages |
---|---|
msg03(1,c) msg06(2,c) msg10(3,c) msg12(4,c) |
msg12(4,c),msg10(3,c),msg06(2,c),msg03(1,c) |
Note:
For the best effort resequencer to work correctly, the messages must arrive in sequence or nearly in sequence. Otherwise, they are not resequenced correctly. If the messages do not arrive close together, set the value of the maxRowsRetrieved
parameter to 1
so the next message in the sequence has enough time to arrive and be picked up by the next processing loop (and therefore be delivered in sequence).
23.2.3.6 Example of Best Effort Resequencing Based on a Time Window
Table 23-5 illustrates the behavior of the best effort resequencer when it is configured to process messages based on the time period in which they arrive. In this example, the time window is 10 minutes, the buffer is 10% (one minute), and msgX(Y)
indicates that the message arrives as message number X to the Mediator service component and the message contains the sequence ID Y. The first message arrives at 2:00:00, which starts the time window. The time window lasts until 2:10:00, but with the addition of the buffer time, messages that arrived until 2:11:00 are processed.
Table 23-5 Best Effort Resequencer Behavior Based on a Time Window
Group CMessage/Time | Sequenced Messages |
---|---|
msg01(04)/2:00:00 msg02(05)/2:00:20 msg03(01)/2:00:30 msg04(03)/2:00:50 msg05(07)/2:04:20 msg06(02)/2:04:45 msg07(13)/2:05:10 msg08(08)/2:05:40 msg09(06)/2:08:40 msg10(12)/2:09:20 msg11(10)/2:10:30 msg12(09)/2:10:40 msg13(14)/2:10:50 msg14(11)/2:13:00 |
msg03(01), msg06(02), msg04(03), msg01(04), msg02(05), msg09(06), msg05(07), msg08(08), msg12(09), msg11(10), msg10(12), msg07(13) |
Note:
In the above example, the resequencer identified the maximum sequence ID for the time window as 13 (from message 7). Message 13 arrived within the buffer time, but has a sequence ID of 14. It is not processed with the original group, but instead begins a new time window at its arrival time of 2:10:50. Message 14 arrived too late and is included in the second time window.
23.3 Configuring the Resequencer
You can configure the resequencer using Oracle JDeveloper. This section describes how to configure the resequencer in Oracle JDeveloper.
23.3.1 How to Specify the Resequencing Level
You can define resequencing at either the service component level or the operation level. For Mediator service components with only one operation, configuring resequencing at the operation or service component level results in the same behavior. For Mediator service components having multiple operations, specifying the resequencing at the service component level applies the same resequencing rules to all the operations, and messages arriving at any operation are resequenced. By default, the resequencing level is operations.
To specify the resequencing level:
23.3.2 How to Configure the Resequencing Strategy
This section provides instructions on how to configure the three different types of resequencing strategies.
23.3.2.1 To configure a standard resequencer:
-
Set the resequence level as described in How to Specify the Resequencing Level.
-
On the Mediator Editor under either the Mediator component or the operation you want to configure, select Standard from the Resequence Mode dropdown list.
The Resequence Options box appears and includes the options for the standard resequencer, as shown in Figure 23-4.
Figure 23-4 Oracle Mediator with Resequence Mode set to Standard
Description of "Figure 23-4 Oracle Mediator with Resequence Mode set to Standard" -
Fill in the fields listed inTable 23-6.
Note:
To specify values for the Group and ID fields, click the Invoke Expression Builder button to the right of each field. This launches the Expression Builder, which provides graphical assistance in creating field expressions and adding functions.
Table 23-6 Standard Resequencing Options
Field Name | Description | Default Value | Mandatory |
---|---|---|---|
Group |
The XPath that points to the field in the incoming message on which grouping is done. |
component_name-operation |
N |
ID |
The XPath that points to the field in the incoming message on which resequencing is done. |
N/A |
Y |
Timeout |
The time period in seconds to wait for an expected message. The resequencer locks the group as timed-out if a time out occurs. |
N |
|
Start |
The starting number of the ID sequence. |
1 |
N |
Increment |
The increment of the ID sequence. |
1 |
N |
Footnote 1
This default value means that the timeout never happens for a group by default.
23.3.2.2 To configure a FIFO resequencer:
-
Set the resequence level as described in How to Specify the Resequencing Level.
-
On the Mediator Editor under either the Oracle Mediator component or the operation you want to configure, select FIFO from the Resequence Mode dropdown list.
The Resequence Options box appears and includes the option for the standard resequencer, as shown in Figure 23-5.
Figure 23-5 Oracle Mediator with Resequence Mode set to FIFO
Description of "Figure 23-5 Oracle Mediator with Resequence Mode set to FIFO" -
In the Group field, enter the XPath expression pointing to the field in the incoming message on which grouping is performed.
23.3.2.3 To configure a best effort resequencer:
-
Set the resequence level as described in How to Specify the Resequencing Level.
-
On the Mediator Editor under either the Mediator component or the operation you want to configure, select Best Effort from the Resequence Mode dropdown list.
The Resequence Options box appears and includes the option for the standard resequencer, as shown in Figure 23-6.
Figure 23-6 Oracle Mediator with Resequence Mode set to Best Effort
Description of "Figure 23-6 Oracle Mediator with Resequence Mode set to Best Effort" -
Fill in the fields listed in Table 23-7 to configure the best effort resequencer.
Note:
You can specify either a maximum number of rows to process at one time or a time window for the messages. You cannot specify both. You must set one control to zero for the other control to be enabled.
-
If needed, you can change the percent of the time window that is added as a buffer. You configure the buffer using the Oracle Enterprise Manager Fusion Middleware Control.
For instructions, see “Configuring Resequenced Messages" in the Administering Oracle SOA Suite and Oracle Business Process Management Suite.
Table 23-7 Best Effort Resequencing Options
Field Name | Description | Default Value | Mandatory |
---|---|---|---|
Group |
The XPath that points to the field in the incoming message on which grouping is performed. |
component_name-operation |
N |
ID |
The XPath that points to the field in the incoming message that contains the ID on which resequencing is performed. |
N/A |
Y |
Datatype |
The data type of the sequence ID. The ordering process is based on the data type. Supported values are datetime and numeric. |
Numeric |
Y |
Max Rows |
Number of in-sequence messages that the resequencer should pick from the data store at a time. You must specify a time window or the maximum rows, but not both. You must set one control to zero for the other control to be enabled. |
5 |
N |
Time Window |
The length of time in minutes to wait after a message arrives to select messages from the data store for resequencing. You must specify a time window or the maximum rows, but not both. You must set one control to zero for the other control to be enabled. |
0 |
N |