Understanding the Integration Data Queue Definitions

The standard installation of Transportation and Global Trade Management version 6.2 has PUBLIC Data Queue Definitions configured for each of the Integration Data Queues described in the preceding section.

This section describes these definitions so that any configuration can be done without any unforeseen impact on the integrity or performance of the Transportation and Global Trade Management Cloud servers.

The Data Queue Definition is a unique (based on a Data Queue Definition GID) configuration of the following properties.

Data Queue Definition Properties

Parameter	Description
Active	Only queues which are marked as 'Active' will have Poller threads started on the application server.
Thread Count	The number of Poller threads that will be started on each application server. All Poller threads will synchronize their access to the events on a data queue. One thread at a time will retrieve a batch of events, mark them all as 'ACTIVE' and commit the transaction before releasing the queue for another thread to access. This would normally be tuned to match the expected volume of events.
Batch Size	The maximum number of events that will be retrieved by each Poller thread. This would normally be tuned to match the expected volume of events.
Polling Frequency	The duration each Poller thread will sleep between attempts to retrieve a batch of events. Note: if the time taken to process a batch exceeds the polling frequency the Poller thread will essentially start processing the next batch more or less immediately after completing the current batch. This would normally be tuned to match the expected volume of events.
Data Queue Table	Corresponds to the physical table which will contain the events This should not require customization.
Data Queue Poller	The Poller configuration specifies the behavior and attributes of the SQL statement used to retrieve the batch of events. In addition to the Class that will run the query, the following can be configured:- Filter – these are additional predicates added to the basic statement (e.g. status = 'QUEUED' etc.). For example, if a domain specific queue is in use it may be necessary to add a predicate which selects the q_queue_name column which matches the full GID. Order – this is specified as the ORDER BY criteria. This is how priority ordering is achieved. Poller Definitions are configured via Business Process Automation Power Data. Custom Poller definitions may be required if additional criteria is used to further partition events placed on the same queue e.g. by joining other tables.
Data Queue Executor	The Executor configuration species the Class that will process the actual event and whether events in the ACTIVE or FAILED state should be reprocessed on app server startup. Executor Definitions are configured via Business Process Automation Power Data. Custom Executor definitions may be required if reprocessing of events needs to be handled differently.
Finder Set	This specifies the Class which formats the queue specific information in the Data Queue Manager Events UI. This should not require customization.
Related Queue	This is used to manage the parent/child relationship between events. This is not used by Integration Data Queues.

Configuring Inbound Data Queue Definitions

There is one inbound data queue definitions created by default in the PUBLIC domain: INTEGRATION IN. Events are held in the Q_INTEGRATION _IN Data Queue Table. By default, the queue is inactive.

If these base configurations are deemed to be sufficient for the expected volumes, they can be activated by logging in as DBA.ADMIN and setting each queue definition to 'Active'. Additionally, as mentioned previously, the INTEGRATION IN Parameter set value also needs to be set to 'TRUE'.

Configured Poller Definition

The Poller Definitions for both queues are configured to use the PREEMPTIVE POLLER plug-in, which uses a 'Top N query' format to retrieve an ordered batch of events. The format of the SQL statement that would be used to retrieve a batch of events would essentially be as follows:

SELECT rownum, {column list}
   FROM (SELECT {column list} 
            FROM Q_INTEGRATION_IN 
            WHERE q_state='QUEUED' and {filter}
            ORDER BY {order})
   WHERE rownum <= {batch size}

The arguments in braces are replaced at runtime as follows:

Poller Definition Parameters

Parameter	Description
column list	The complete column list is controlled via the Event Class which for Q_INTEGRATION_IN events would be:- q_event_seq, rownum, q_start, q_process_start, domain_name, q_preemption_priority, topic_class_name, topic_parameters, topic_description, i_transmission_no, q_queue_name.
filter	The PUBLIC Poller definitions for INTEGRATION IN has a filter clause as follows:- q_start < sys_extract_utc(current_timestamp) and q_queue_name = 'INTEGRATION IN' Therefore, if a customer-defined Data Queue Definition is to be used there will need to be a corresponding customer-defined Poller Definition in order to override the q_queue_name parameter.
order	For Preemptive Data Queues the base ORDER BY clause will be as follows:- q_preemption_priority, q_event_seq That is, ordered by priority first and then event sequence number.
batch size	This is the batch size taken from the Data Queue Definition.

Configuring Outbound Data Queue Definitions

The Outbound data queue definitions are subjected to the same configuration capabilities and constraints as the inbound data queues.