23 Resequencing in Oracle Mediator

This chapter describes message resequencing concepts in Oracle Mediator, and provides instructions for configuring standard resequencing, first-in/first-out resequencing, and best effort resequencing.

This chapter includes the following sections:

23.1 Introduction to the Resequencer

The resequencer in Mediator rearranges a stream of related but out-of-sequence messages into a sequential order. When incoming messages arrive, they may be in a random order. The resequencer orders the messages based on sequential or chronological information, and then sends the messages to the target services in an orderly manner. The sequencing is performed based on the sequencing strategy selected.

23.1.1 Groups and Sequence IDs

The resequencer works with two central concepts: groups and sequence IDs. The sequence ID is an identifying part of the message, and messages are rearranged based on this identifier. The messages arriving for resequencing are split into groups and the messages within a group are sequenced according to the sequence ID. Sequencing within a group is independent of the sequencing of messages in any other group. Groups in themselves are not dependent on each other and can be processed independently of each other.

As an example, messages attached to certain groups arrive to a Mediator service component in the following order:

msg9(a), msg8(b), msg7(a), msg6(c), msg5(a), msg4(b), msg3(c), msg2(b), msg1(a)

Table 23-1 shows how the Mediator sorts the messages into groups. The order of the messages in each group depends on the type of resequencer used.

Table 23-1 Messages Sorted into Groups

Group c Group b Group a

msg6(c), msg3(c)

msg8(b), msg4(b), msg2(b)

msg9(a), msg7(a), msg5(a), msg1(a)

All the groups are processed independently of each other and any error occurring in ones group does not affect the processing of other groups.

23.1.2 Identification of Groups and Sequence IDs

Groups and sequence IDs are identified through XPath expressions in the message payload and header. You specify XPath expressions that point to the elements in the message payload on which grouping is done and on which sequencing is done.

In the message payload shown in Figure 23-1, CustomerId is the field on which to base instance sequencing and Type is the field on which to base grouping.

Figure 23-1 Message Payload

Description of Figure 23-1 follows
Description of "Figure 23-1 Message Payload"

Note:

Resequencing is not supported for synchronous operations.

23.2 Resequencing Order

Mediator can resequence the incoming messages in a user-specified order. This implementation enables you to specify three types of resequencing orders:

23.2.1 Standard Resequencer

The standard resequencer supports a standard resequencer pattern. The following sections describe the standard resequencer and how it processes messages.

23.2.1.1 Overview of the Standard Resequencer

The standard resequencer is useful for applications that use identifiers from a simple numeric identifier sequence in their messages. The standard resequencer receives a stream of messages that might not arrive in order; it then stores the out-of-sequence messages until a complete sequence based on the sequence IDs is received. The in-sequence messages are then processed asynchronously based on their sequence ID.

It is important to note that the messages to outbound services of the standard resequencer Mediator service component are guaranteed to arrive in sequence.

23.2.1.2 Information Required for Standard Resequencing

When using the standard resequencer in Mediator, you must always specify a group XPath expression and a sequence ID XPath expression. These specify where the Mediator resequencer can find the group and the sequence ID in the messages. You must also supply the sequence numbering in terms of the start sequence ID and the sequence ID incremental delta. This numbering is used to form each group. In addition to the group, sequence ID, and increment properties, you can also specify a timeout period, in seconds, to wait for the expected messages.

The Mediator standard resequencer holds back messages in the Mediator resequencer database until it can produce the right sequence for different groups. This situation means that if for a given group, the message with a particular sequence ID does not arrive within the timeout period, the subsequent messages for that group are held back forever. In such a case, you must manually unlock the group through Oracle Enterprise Manager Fusion Middleware Control and go to the next available message, skipping the pending message.

23.2.1.3 Example of the Standard Resequencer

Table 23-2 shows how groups are formed differently for two different values of the incremental delta.

Table 23-2 Groups Formed Differently for Two Different Values

Start SequenceID Incremental Delta Group1 Group2 ... Groupn

1

1

1,2,3,4,5,...

1,2,3,4,5,...

...

1,2,3,4,5,...n

1

5

1,5,10,15,...

1,5,10,15,...

...

1,5,10,15,...

Note:

If the sequence numbering is different for various groups (for example, if the groups do not have the same incremental delta or start sequence ID) and the messages do not arrive in order, then you can use the best effort resequencer to rearrange the messages.

23.2.2 FIFO Resequencer

The FIFO resequencer supports a standard first in, first out (FIFO) pattern. The following sections describe the FIFO resequencer and how it processes messages.

23.2.2.1 Overview of the FIFO Resequencer

The FIFO resequencer is useful for applications that need sequencing based on the time the messages arrive to the Mediator. The FIFO resequencer receives a stream of messages that are in order and processes them in sequence for each group based on the arrival time of the messages.

It is important to note that the messages to outbound services of the Mediator acting as a FIFO resequencer are guaranteed to arrive in order based on arrival time. Therefore, the messages are delivered in the order they were stored in the resequencer data store.

23.2.2.2 Information Required for FIFO Resequencing

When using the FIFO resequencer, you must always specify a group XPath expression. However, you do not need to specify a sequence ID because the messages are processed according to the time of arrival to the Mediator service component that is configured for FIFO resequencing. The group XPath expression specifies where the FIFO resequencer should find the group information in the message to group the messages. No further configuration is needed for a FIFO pattern.

23.2.2.3 Example of the FIFO Resequencer

Table 23-3 illustrates the behavior of the FIFO resequencer where msgX(Y,Z) indicates that the message arrives as message number X to the Mediator service component and the message contains sequenceID Y and group Z.

Table 23-3 FIFO Resequencer Behavior

Incoming Messages Sequenced Messages

msg12(4,c)

msg05(9,a)

msg02(7,a)

msg10(3,c)

msg10(3,a)

msg07(5,a)

msg06(1,c)

msg03(2,c)

msg12(4,c),msg10(3,c),msg06(1,c),msg03(2,c)

msg05(9,a), msg02(7,a), msg10(3,a), msg07(5,a)

As shown in Table 23-3, the messages are sequenced based on their time of arrival and the sequenceID is not used for sequencing.

Note:

When using the FIFO resequencer, use a single-threaded inbound adapter to avoid unpredictable results. For example, when you use the file/FTP adapter, the database adapter, or the AQ adapter in front of a Mediator service component that is configured as a FIFO resequencer, configure the adapter for single-threaded processing. Otherwise, unpredictable results occur because the arrival time of each message is calculated when the message arrives to the Mediator service component instead when it arrives to the adapter service.

23.2.3 Best Effort Resequencer

The Mediator resequencer supports a best effort pattern. The following sections describe the best effort resequencer and how it processes messages.

23.2.3.1 Overview of the Best Effort Resequencer

The best effort pattern is useful for applications that produce a large number of messages in a short period and cannot provide information to the resequencer about the identifier to use for sequencing. Typically, the identifier used for sequencing in such scenarios is of a dateTime type or numeric type. Using the dateTime field as the sequence ID XPath enables you to control the sequencing. The messages are expected to be sent in sequence by the applications, thus the date and time the messages are sent can be used for sequencing. The Mediator makes the best effort to ensure that the messages are delivered in sequence.

The best effort resequencer can reorder messages based on no knowledge about the increment of the sequence ID. This situation means that unlike the standard resequencer, you do not need to define the increment of the sequence ID for the best effort resequencer in advance. When the messages are processed, they are processed in sequence based on the specified sequence ID and the messages that have arrived, whether a true sequence is received. The sequence IDs are either numeric or dateTime. Therefore, sequencing occurs on the numeric order or the dateTime order of the sequence IDs.

23.2.3.2 Best Effort Resequencer Message Selection Strategies

The best effort resequencer processes messages asynchronously based on one of two message selection strategies: Maximum rows selected or time window. The messages selected and processed at any one time are based either on the maximum number of rows you specify or on a window of time in which they arrive.

23.2.3.2.1 Maximum Rows Selected

When the best effort resequencer is configured to use a maximum number of rows, it performs the following steps whenever new messages are available in the resequencer database:

  1. The resequencer orders the messages according to the specified sequence ID (typically a date and time stamp).

  2. The resequencer locks and selects the number of messages equal to the value of the maxRowsRetrieved parameter from the ordered messages above.

  3. The resequencer processes the selected messages one after another in its own transaction in sequence.

23.2.3.2.2 Time Window

When the best effort resequencer is configured to use a time window instead of a maximum number rows, the messages to select and process at one time are based on a period you specify plus an optional buffer time. Each message belongs to a specific time window, and messages that are part of one time window are processed separately from messages belonging to a different time window.

In addition to the time window, you can specify a buffer time, which is an overlap between two sequential time windows that allows messages that arrive a little late to be associated with the first time window. By default, the buffer time is 10% of the time window you specify.

When the best effort resequencer is configured to use a time window, groups are processed in an iterative manner and messages are processed in the following steps:

  1. The first message arrives and the time window begins.

  2. The buffer is added to the time window, and processing begins after the buffer time.

  3. The resequencer retrieves the messages that arrived within the time window, and identifies the maximum sequence ID (typically a date and time stamp) from all the messages.

  4. The resequencer retrieves any messages that arrive within the buffer time and that have a sequence ID that is less than the maximum sequence ID identified above.

  5. The resequencer sorts all messages retrieved in the above steps in ascending order of the sequence IDs and processes the messages.

23.2.3.3 Best Effort Resequencer Message Delivery

It is important to note that the messages to outbound services of the Mediator service component configured for best effort resequencing are not guaranteed to arrive in order of a sequence ID. At any given time, a snapshot of the available messages is taken and sequencing is performed only on those messages. Therefore, unlike a standard resequencer, it is not guaranteed that a message with a lesser sequence ID value is sent before a message that ha a greater sequence ID value but that arrived earlier. Messages with a lesser sequence ID value that arrive later might be processed in the following cycle when a snapshot of available messages is taken again and the messages are reordered.

23.2.3.4 Information Required for Best Effort Resequencing

When using the best effort resequencer, you must specify a group XPath expression, a sequence ID XPath expression, and the data type of the sequence ID (numeric or dateTime). These specify where the resequencer should find the group and the sequence ID in the messages and how to handle the sequence ID. In addition, you must specify either a maximum number of rows to select for each resequencing batch or a time window during which the messages included in one batch arrive.

Unlike the standard resequencer, the best effort resequencer has no knowledge about how the sequence is built. No further information is used by the best effort resequencer to perform its responsibilities.

23.2.3.5 Example of Best Effort Resequencing Based on Maximum Rows

Table 23-4 illustrates the behavior of the best effort resequencer when it is configured to use the maximum number of rows to determine which messages to process. In this example, msgX(Y,Z) indicates that the message arrives as message number X to the Mediator service component and the message contains sequenceID Y and group Z.

Table 23-4 Best Effort Resequencer Behavior Based on Maximum Rows

Group C Sequenced Messages

msg03(1,c)

msg06(2,c)

msg10(3,c)

msg12(4,c)

msg12(4,c),msg10(3,c),msg06(2,c),msg03(1,c)

Note:

For the best effort resequencer to work correctly, the messages must arrive in sequence or nearly in sequence. Otherwise, they are not resequenced correctly. If the messages do not arrive close together, set the value of the maxRowsRetrieved parameter to 1 so the next message in the sequence has enough time to arrive and be picked up by the next processing loop (and therefore be delivered in sequence).

23.2.3.6 Example of Best Effort Resequencing Based on a Time Window

Table 23-5 illustrates the behavior of the best effort resequencer when it is configured to process messages based on the time period in which they arrive. In this example, the time window is 10 minutes, the buffer is 10% (one minute), and msgX(Y) indicates that the message arrives as message number X to the Mediator service component and the message contains the sequence ID Y. The first message arrives at 2:00:00, which starts the time window. The time window lasts until 2:10:00, but with the addition of the buffer time, messages that arrived until 2:11:00 are processed.

Table 23-5 Best Effort Resequencer Behavior Based on a Time Window

Group CMessage/Time Sequenced Messages

msg01(04)/2:00:00

msg02(05)/2:00:20

msg03(01)/2:00:30

msg04(03)/2:00:50

msg05(07)/2:04:20

msg06(02)/2:04:45

msg07(13)/2:05:10

msg08(08)/2:05:40

msg09(06)/2:08:40

msg10(12)/2:09:20

msg11(10)/2:10:30

msg12(09)/2:10:40

msg13(14)/2:10:50

msg14(11)/2:13:00

msg03(01), msg06(02), msg04(03), msg01(04), msg02(05), msg09(06), msg05(07), msg08(08), msg12(09), msg11(10), msg10(12), msg07(13)

Note:

In the above example, the resequencer identified the maximum sequence ID for the time window as 13 (from message 7). Message 13 arrived within the buffer time, but has a sequence ID of 14. It is not processed with the original group, but instead begins a new time window at its arrival time of 2:10:50. Message 14 arrived too late and is included in the second time window.

23.3 Configuring the Resequencer

You can configure the resequencer using Oracle JDeveloper. This section describes how to configure the resequencer in Oracle JDeveloper.

23.3.1 How to Specify the Resequencing Level

You can define resequencing at either the service component level or the operation level. For Mediator service components with only one operation, configuring resequencing at the operation or service component level results in the same behavior. For Mediator service components having multiple operations, specifying the resequencing at the service component level applies the same resequencing rules to all the operations, and messages arriving at any operation are resequenced. By default, the resequencing level is operations.

To specify the resequencing level:

  1. On the Mediator Editor, select the resequencing level from the Resequence Level dropdown list, as shown in Figure 23-2.

    Figure 23-2 Mediator Editor with Resequence Level Field

    Description of Figure 23-2 follows
    Description of "Figure 23-2 Mediator Editor with Resequence Level Field"

    If you choose component, the Resequence field under each operation no longer appears and the Resequence Mode field appears under the Resequence Level field so you can set the resequencing mode for the service component. By default, the resequencing mode is set to off.

    When you select a resequencing mode, the Resequence Options box appears under the service component or operation, as shown in Figure 23-3. If the Resequence Mode field for an operation is set to off, the Resequence Options box disappears.

    Figure 23-3 Mediator Editor with Resequence Options Section

    Description of Figure 23-3 follows
    Description of "Figure 23-3 Mediator Editor with Resequence Options Section"

    The options in the Resequence Options section change depending on the resequencing mode you select.

23.3.2 How to Configure the Resequencing Strategy

This section provides instructions on how to configure the three different types of resequencing strategies.

23.3.2.1 To configure a standard resequencer:

  1. Set the resequence level as described in How to Specify the Resequencing Level.

  2. On the Mediator Editor under either the Mediator component or the operation you want to configure, select Standard from the Resequence Mode dropdown list.

    The Resequence Options box appears and includes the options for the standard resequencer, as shown in Figure 23-4.

    Figure 23-4 Oracle Mediator with Resequence Mode set to Standard

    Description of Figure 23-4 follows
    Description of "Figure 23-4 Oracle Mediator with Resequence Mode set to Standard"

  3. Fill in the fields listed inTable 23-6.

    Note:

    To specify values for the Group and ID fields, click the Invoke Expression Builder button to the right of each field. This launches the Expression Builder, which provides graphical assistance in creating field expressions and adding functions.

Table 23-6 Standard Resequencing Options

Field Name Description Default Value Mandatory

Group

The XPath that points to the field in the incoming message on which grouping is done.

component_name-operation

N

ID

The XPath that points to the field in the incoming message on which resequencing is done.

N/A

Y

Timeout

The time period in seconds to wait for an expected message. The resequencer locks the group as timed-out if a time out occurs.

0Foot 1

N

Start

The starting number of the ID sequence.

1

N

Increment

The increment of the ID sequence.

1

N

Footnote 1

This default value means that the timeout never happens for a group by default.

23.3.2.2 To configure a FIFO resequencer:

  1. Set the resequence level as described in How to Specify the Resequencing Level.

  2. On the Mediator Editor under either the Oracle Mediator component or the operation you want to configure, select FIFO from the Resequence Mode dropdown list.

    The Resequence Options box appears and includes the option for the standard resequencer, as shown in Figure 23-5.

    Figure 23-5 Oracle Mediator with Resequence Mode set to FIFO

    Description of Figure 23-5 follows
    Description of "Figure 23-5 Oracle Mediator with Resequence Mode set to FIFO"

  3. In the Group field, enter the XPath expression pointing to the field in the incoming message on which grouping is performed.

23.3.2.3 To configure a best effort resequencer:

  1. Set the resequence level as described in How to Specify the Resequencing Level.

  2. On the Mediator Editor under either the Mediator component or the operation you want to configure, select Best Effort from the Resequence Mode dropdown list.

    The Resequence Options box appears and includes the option for the standard resequencer, as shown in Figure 23-6.

    Figure 23-6 Oracle Mediator with Resequence Mode set to Best Effort

    Description of Figure 23-6 follows
    Description of "Figure 23-6 Oracle Mediator with Resequence Mode set to Best Effort"

  3. Fill in the fields listed in Table 23-7 to configure the best effort resequencer.

    Note

    You can specify either a maximum number of rows to process at one time or a time window for the messages. You cannot specify both. You must set one control to zero for the other control to be enabled.

  4. If needed, you can change the percent of the time window that is added as a buffer. You configure the buffer using the Oracle Enterprise Manager Fusion Middleware Control.

    For instructions, see “Configuring Resequenced Messages" in the Administering Oracle SOA Suite and Oracle Business Process Management Suite.

Table 23-7 Best Effort Resequencing Options

Field Name Description Default Value Mandatory

Group

The XPath that points to the field in the incoming message on which grouping is performed.

component_name-operation

N

ID

The XPath that points to the field in the incoming message that contains the ID on which resequencing is performed.

N/A

Y

Datatype

The data type of the sequence ID. The ordering process is based on the data type. Supported values are datetime and numeric.

Numeric

Y

Max Rows

Number of in-sequence messages that the resequencer should pick from the data store at a time.

You must specify a time window or the maximum rows, but not both. You must set one control to zero for the other control to be enabled.

5

N

Time Window

The length of time in minutes to wait after a message arrives to select messages from the data store for resequencing.

You must specify a time window or the maximum rows, but not both. You must set one control to zero for the other control to be enabled.

0

N