13 Oracle BPEL Process Manager Performance Tuning

Oracle Business Process Execution Language (BPEL) Process Manager provides several property settings that can be configured to optimize performance at the composite, fabric, application and server levels. This chapter describes these property settings and provides recommendations on how to use them.

This chapter contains the following sections:

13.1 About BPEL Process Manager

BPEL is the standard for assembling a set of discrete services into an end-to-end process flow, radically reducing the cost and complexity of process integration initiatives. Oracle BPEL Process Manager offers a comprehensive and easy-to-use infrastructure for creating, deploying and managing BPEL business processes.

For more information, see "Configuring BPEL Process Service Components and Engines" in Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite and "Using the BPEL Process Service Component" in the Oracle Fusion Middleware Developer's Guide for Oracle SOA Suite.

13.2 Basic Tuning Considerations

This section describes the performance tuning properties at the BPEL engine level. They can be configured using Oracle Enterprise Manager. For more information, see "Configuring BPEL Process Service Engine Properties" in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.

Note:

The configuration examples and recommended settings described in this chapter are for illustrative purposes only. Consult your own use case scenarios to determine which configuration options can provide performance improvements.

13.2.1 BPEL Threading Model

When the dispatcher must schedule a dispatch message for execution, it can enqueue the message into a thread pool. Each dispatch set can contain a thread pool (java.util.concurrent.ThreadPoolExecutor). The BPEL thread pool implementation notifies the threads when a message has been enqueued and ensures the appropriate number of threads are instantiated in the pool.

Note:

dspMinThreads, dspMaxThreads and dspInvokeAllocRatio configuration properties are deprecated in Oracle 11g. In addition, the invoke threads have their own pool in Oracle 11g so the dspInvokeAllocRatio is no longer required.

13.2.1.1 Dispatcher System Threads

The dspSystemThreads property specifies the total number of threads allocated to process system dispatcher messages. System dispatcher messages are general clean-up tasks that are typically processed quickly by the server (for example, releasing stateful message beans back to the pool). Typically, only a small number of threads are required to handle the number of system dispatch messages generated during run time.

The minimum number of threads for this thread pool is 1 and it cannot be set to 0 a or negative number.

The default value is 2. Any value less than 1 thread is changed to the default.

13.2.1.2 Dispatcher Invoke Threads

The dspInvokeThreads property specifies the total number of threads allocated to process invocation dispatcher messages. Invocation dispatcher messages are generated for each payload received and are meant to instantiate a new instance. If the majority of requests processed by the engine are instance invocations (as opposed to instance callbacks), greater performance may be achieved by increasing the number of invocation threads. Higher thread counts may cause greater CPU utilization due to higher context switching costs.

The minimum number of threads for this thread pool is 1 and it cannot be set to 0 a or negative number.

The default value is 20 threads. Any value less than 1 thread is changed to the default.

13.2.1.3 Dispatcher Engine Threads

The dspEngineThreads property specifies the total number of threads allocated to process engine dispatcher messages. Engine dispatcher messages are generated whenever an activity must be processed asynchronously. If the majority of processes deployed are durable with a large number of dehydration points (mid-process receive, onMessage, onAlarm, and wait activities), greater performance may be achieved by increasing the number of engine threads. Note that higher thread counts can cause greater CPU utilization due to higher context switching costs.

The minimum number of threads for this thread pool is 1 and it cannot be set to 0 a or negative number.

The default value is 30 threads. Any value less than 1 thread is changed to the default.

13.2.1.4 Dispatcher Maximum Request Depth

The dspMaxRequestDepth property sets the maximum number of in-memory activities to process within the same request. After processing an activity request, Oracle BPEL Process Manager attempts to process as many subsequent activities as possible without jeopardizing the validity of the request. Once the activity processing chain has reached this depth, the instance is dehydrated and the next activity is performed in a separate transaction.

If the request depth is too large, the total request time can exceed the application server transaction time out limit.This process is applicable to durable processes.

The default value is 600 activities.

Note:

Note that the minimum number of threads for each thread pool is 1. dsp*Threads can not be set to 0 or negative.

13.2.2 Tuning Audit Levels

The following properties can be set to audit levels.

13.2.2.1 Audit Level

The auditLevel property sets the audit trail logging level. This configuration property is applicable to both durable and transient processes. This property controls the amount of audit events that are logged by a process. Audit events result in more database inserts into the audit_trail table which may impact performance. Audit information is used only for viewing the state of the process from Oracle Enterprise Manager Console.

Use the off value if you do not want to store any audit information. Always choose the audit level according to your business requirements and use cases. For more information on setting the audit level, see "Introduction to the Order of Precedence for Audit Level Settings" in Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.

Value Description

Inherit

Inherits the audit level from infrastructure level.

off

No audit events (activity execution information) are persisted and no logging is performed; this can result in a slight performance boost for processing instances.

Minimal

All events are logged; however, no audit details (variable content) are logged.

Error

Logs only serious problems that require immediate attention from the administrator and are not caused by a bug in the product. Using this level can help performance.

Production

All events are logged. The audit details for assign activities are not logged; the details for all other activities are logged.

Development

All events are logged; all audit details for all activities are logged.


13.2.2.2 Audit Trail Threshold

The audittrailthreshold property sets the maximum size (in kilobytes) of an audit trail details string before it is stored separately from the audit trail. If an audit trail details string is larger than the threshold setting, it is not immediately loaded when the audit trail is initially retrieved; a link is displayed with the size of the details string. Strings larger than the threshold setting are stored in the audit_details table, instead of the audit_trail table.

The details string typically contains the contents of a BPEL variable. In cases where the variable is very large, performance can be severely impacted by logging it to the audit trail.

The default value is 50000 (50 kilobytes).

13.2.2.3 Audit Store Policy

This property specifies the strategy to persist the BPEL audit data.

Value Description

syncSingleWrite (default)

AuditTrail and dehydration are persisted to DB in one transaction.

syncMultipleWrite

AuditTrail and dehydration are persisted in the same thread but separate transactions.

async

AuditTrail and dehydration are persisted by separate threads and separate transactions.

CAUTION: If you set auditStorePolicy to async, audit messages write to the database in a separate thread and transaction, which increases transaction speed. However, if a node crash occurs, audit trace functionality may be lost.

Oracle recommends setting auditStorePolicy to async only if enhanced performance is critical.

For more information, see "Oracle BPEL Process Manager High Availability Architecture and Failover Considerations" in the Oracle Fusion Middleware High Availability Guide.


By default, audit messages are stored as part of the main BPEL transaction. A BPEL instance holds on to the audit messages until the flow reaches dehydration. In some use cases, for example when you have a large loop, and there is no dehydration point in the loop, a large number of audit logs are accumulated. This could lead to an out-of-memory issue and BPEL main transaction can experience timeout errors. You may consider using syncMultipleWrite or async to store the audit message separately from the main transaction.

When you use syncMultipleWrite and async auditStorePolicy, there are a few other properties that need to be considered. Please see the sections below.

13.2.2.4 Audit Flush ByteThreshold

This property controls how often the engine should flush the audit events, basically after adding an event to the current batch, the engine checks to see if the current batch byte size is greater than this value or not.

Consider tuning this property when async or syncMultipleWrite audit strategies are used. This size needs to be tuned based on the application.

13.2.2.5 Audit Flush EventThreshold

This property controls how often the engine should flush the audit events, basically when it reaches this limit of the number of events, the engine would trigger the store call.

Consider tuning this property when async or syncMultipleWrite audit strageties are used. This size needs to be tuned based on the application.

13.2.3 Tuning Database Persistence for BPEL

The oneWayDeliveryPolicy property controls database persistence of messages entering Oracle BPEL Server. By default, incoming requests are saved in the delivery service database table dlv_message. These requests are later acquired by Oracle BPEL Server worker threads and delivered to the targeted BPEL process. This property persists delivery messages and is applicable to durable processes.

This property can be set during creation of a BPEL process service component. For more information, see "How to Add a BPEL Process Service Component" in Oracle Fusion Middleware Developer's Guide for Oracle SOA Suite.

Note:

One-way invocation messages are stored in the delivery cache until delivered. If the rate at which one-way messages arrive is much higher than the rate at which Oracle BPEL Server delivers them, or if the server fails, messages may be lost.

The oneWayDeliveryPolicy is from the Oracle 10g configuration property deliveryPersistencePolicy. The configuration property name in 11g is bpel.config.oneWayDeliveryPolicy.

Value Description

async.persist (Default)

Delivery messages are persisted in the database. With this setting, reliability is obtained with some performance impact on the database. In some cases, overall system performance can be impacted.

async.cache

Incoming delivery messages are kept only in the in-memory cache. If performance is preferred over reliability, this setting should be considered.

CAUTION: If you set OneWayDeliveryPolicy to async.cache, invoke messages do not persist to the database; they are in memory. If the SOA node that holds these messages fails, those messages will be lost.

Use the async.cache option only when performance is critical and when you have an external means to track messages and redeliver them.

For more information, see "Oracle BPEL Process Manager High Availability Architecture and Failover Considerations" in the Oracle Fusion Middleware High Availability Guide.

sync

Directs Oracle BPEL Server to bypass the scheduling of messages in the invoke queue, and invokes the BPEL instance synchronously. In some cases this setting can improve database performance.


13.2.4 Tuning Invoke Messages

The MaximumNumberOfInvokeMessagesInCache property specifies the number of invoke messages that can be kept in the in-memory cache. Once the engine hits this limit, the message is pushed to dispacther in-memory cache. The saved messages can be recovered using a recovery job. Use value -1 to disable.

The default value is 100000 messages.

13.2.5 Tuning Processed Requests List

The StatsLastN property sets the size of the most-recently processed request list. After each request is finished, statistics for the request are kept in a request list. A value less than or equal to 0 disables statistics gathering. To optimize performance, consider disabling statistics collection if you do not need them.

This property is applicable to both durable and transient processes.

The default value is -1.

13.2.6 Tuning XML Document Persistence

The largedocumentthreshold property sets the large XML document persistence threshold. This is the maximum size (in kilobytes) of a BPEL variable before it is stored in a separate table from the rest of the instance scope data.

This property is applicable to both durable and transient processes.

Large XML documents impact the performance of the entire Oracle BPEL Server if they are constantly read in and written out whenever processing on an instance must be performed.

The default value is 10000 (100 kilobytes).

13.2.7 Validating XML

The validateXML property validates incoming and outgoing XML documents. If set to True, the Oracle BPEL Process Manager applies schema validation for incoming and outgoing XML documents. Nonschema-compliant payload data is intercepted and displayed as a fault.

This setting is independent of the SOA composite application and SOA Infrastructure payload validation level settings. If payload validation is enabled at both the service engine and SOA Infrastructure levels, data is checked twice: once when it enters the SOA Infrastructure, and again when it enters the service engine

CAUTION: Enabling XML payload validation can impact performance.

This property is applicable to both durable and transient processes.

The default value is False.

13.2.8 Tuning Wait Time

The SyncMaxWaitTime property sets the maximum time the process result receiver waits for a result before returning. Results from asynchronous BPEL processes are retrieved synchronously by a receiver that waits for a result from Oracle BPEL Server.

The default value is 45 seconds.

The SyncMaxWaitTime property can be found on the BPEL Service Engine Properties page by clicking the More BPEL Configuration Properties link at the bottom of the screen. For more information, see "Configuring BPEL Process Service Engine Properties" in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.

13.2.9 Tuning Instance Key Block Size

The InstanceKeyBlockSize property controls the instance ID range size. Oracle BPEL Server creates instance keys (a range of process instance IDs) in batches using the value specified. After creating this range of in-memory IDs, the next range is updated and saved in the ci_id_range table.

For example, if instanceKeyBlockSize is set to 100, Oracle BPEL Server creates a range of instance keys in-memory (100 keys, which are later inserted into the cube_instance table as cikey). To maintain optimal performance, ensure that the block size is larger than the number of updates to the ci_id_range table.

The default value is 10000.

13.2.10 Tuning Automatic Recovery Attempts

The MaxRecoveryAttempt parameter allows you to configure the number of automatic recovery attempts to submit in the same recoverable instance. The value you provide specifies the maximum number of times invoke and callback messages are recovered. Once the number of recovery attempts on a message exceeds the specified value, a message is marked as nonrecoverable.

When a BPEL instance makes a call to another server using invokeMessage, and that call fails due to a server down, validation error, or security exception, the invokeMessage is placed in a recovery queue and BPEL attempts to retry those messages. When there are many messages, and a majority of them are being sent to the same target, the target can become overloaded. Setting the appropriate value of MaxRecoveryAttempt will prevent excessive load on servers that are targeted from BPEL web service calls.

For more information on configuring the number of automatic recovery attempts to submit in the same recoverable instance, see "Configuring Automatic Recovery Attempts for Invoke and Callback Messages" in the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.

13.3 Advanced Tuning Considerations

The following BPEL tuning considerations may not be applicable to all BPEL deployments. Always consult your own use case scenarios to determine if these configurations should be used in your deployment.

13.3.1 Tuning BPEL Properties Set Inside a Composite

The BPEL properties set inside a composite affect the behavior of the component containing the BPEL process only. Each BPEL process can be created as a component of a composite. Component properties include inMemoryOptimization, completionPersistPolicy, and auditLevel.

For more information on configuring these properties, see the "Deployment Descriptor Properties" appendix of the Oracle Fusion Middleware Developer's Guide for Oracle SOA Suite.

13.3.2 Identifying Tables Impacted By Instance Data Growth

Instance data occupies space in Oracle BPEL Process Manager schema tables. Data growth from auditing and dehydration can have a significant impact on database performance and throughput. See Section 13.2.2, "Tuning Audit Levels" for audit configuration and Section 13.3.1, "Tuning BPEL Properties Set Inside a Composite" for dehydration configuration. The table below describes the tables that are impacted by instance data growth. A brief description is provided of each table.

Table 13-1 Oracle BPEL Process Manager Tables Impacted by Instance Data Growth

Table Name Table Description

audit_trail

Stores the audit trail for instances. The audit trail viewed in Oracle BPEL Control is created from an XML document. As an instance is processed, each activity writes events to the audit trail as XML.

audit_details

Stores audit details that can be logged through the API. Activities such as an assign activity log the variables as audit details by default.

Audit details are separated from the audit_trail table due to their large size. If the size of a detail is larger than the value specified for this property, it is placed in this table. Otherwise, it is placed in the audit_trail table.

cube_instance

Stores process instance metadata (for example, the instance creation date, current state, title, and process identifier)

cube_scope

Stores the scope data for an instance (for example, all variables declared in the BPEL flow and some internal objects that help route logic throughout the flow).

dlv_message

Stores incoming (invocation) and callback messages upon receipt. This table only stores the metadata for a message (for example, current state, process identifier, and receive date).

dlv_subscription

Stores delivery subscriptions for an instance. Whenever an instance expects a message from a partner (for example, the receive or onMessage activity) a subscription is written out for that specific receive activity.

document_ci_ref

Stores cube instance references to data stored in the xml_document table.

document_dlv_msg_ref

Stores references to dlv_message documents stored in the xml_document table.

wftask

Stores tasks created for an instance. The TaskManager process keeps its current state in this table.

work_item

Stores activities created by an instance. All activities in a BPEL flow have a work_item table. This table includes the metadata for the activity (current state, label, and expiration date (used by wait activities)).

xml_document

Stores all large objects in the system (for example, dlv_message documents). This table stores the data as binary large objects (BLOBs). Separating the document storage from the metadata enables the metadata to change frequently without being impacted by the size of the documents.

Headers_properties

Stores headers and properties information.