This chapter describes how to enable and use throttling in Oracle Service Bus. Throttling lets you control the amount of message traffic to a business service and to remote servers, and helps improve performance and stability by preventing message overload on high-traffic business services.
This chapter contains the following sections:
To control the flow of messages to a business service and prevent backlogs, you can enable and configure message throttling for the business services in your Service Bus applications.
If you want to control the flow of messages to a remote server from multiple business services, you can create a throttling group to manage those business services.
When you use throttling to control message flow, a throttling queue is created in which messages are enqueued when a business service reaches its maximum concurrency or when a throttling group reaches its maximum concurrency. Messages with a higher priority are processed first. If messages have the same priority, they are processed on a first-in first-out basis. To ensure messages with a higher priority are processed first, assign priorities to messages using the routing options. The greater the integer for priority, the higher the priority is for the message.
A throttling queue is an in-memory queue. There is at most one queue per throttled business service or per server. Messages that are placed in this queue are not recoverable when a server fails or when you restart a server. Once a message has been in the throttling queue for an interval greater than the value of message expiration configured for the business service or throttling group, it becomes an expired message and is removed from the queue. When you delete or rename a business service, all the messages in the throttling queue are discarded.
When you enable throttling for a business service or throttling group, you configure information about the message capacity and about the throttling queue used to hold messages when capacity is reached. To use a throttling queue, you must specify the queue length. If the throttling queue length is 0 (zero), messages are discarded once the defined capacity (maximum concurrency) is reached. When messages are discarded or removed from the queue due to exceeding the queue length or expiring, the throttling engine throws a TransportException back to the pipeline, and the error handler is triggered if one is configured.
The maximum concurrency restricts the number of records that can be concurrently processed by a business service or throttling group. When this threshold is reached for a business service, all the incoming messages for the business service are placed in a throttling queue until the business service can accept more messages. If the queue is full, messages in the queue with a lower priority are removed and the new incoming messages are enqueued. Any change to this setting during runtime affects both new messages and those already in the queue. When you increase the value, Service Bus allows more messages to be processed once the messages in the queue are processed. When you decrease the value, Service Bus places any new messages in a throttling queue until the number of messages being processed goes below the new threshold.
The throttling queue length limits the number of messages that can be held in the throttling queue at any given time. All incoming messages beyond the maximum concurrency limit are placed in the throttling queue. When the queue is full, the message in the queue with the lowest priority is removed if a new incoming message has a higher priority. When you decrease the value for this setting during runtime, all the messages beyond the new length are discarded.
The message expiration period (or time to live) limits the amount of time a message can stay in the throttling queue. When the time period has elapsed, the message is removed from the queue. These messages are referred to as expired messages. When you increase the value for this setting, the expiration time for the new messages and the messages that are already present in the queue is increased. When you decrease the value, all the messages that have exceeded the new value are immediately discarded. This value is set at the queue level; all messages put in the queue have the same Message Expiration value. When the time period is exceeded, messages are removed from the queue without being processed.
When you configure throttling for individual business services, you control the flow of messages for those business services without taking into account the limitations of the server hosting the referenced services. If you have multiple business services sending requests to the same remote server, it could exceed the capacity of the server. Defining a throttling group lets you restrict the flow of messages sent to a specific remote server from multiple business services.
You can create throttling groups either in Oracle JDeveloper or in the Oracle Service Bus Console. When you create a throttling group, you associate business service URLs with the group. Each URL can only be associated with one throttling group. All business services in a throttling group have equal priority for message processing. The message priority set in the routing rules determine the order of processing. If multiple messages have the same priority, they are processed with a first-in/first-out policy.
When you enable throttling for a group, you must also enable throttling for each business service in the group for which you want to restrict the message flow.
The properties you set for throttling groups are the same as those you set for individual business services. Service Bus determines the maximum concurrency property as follows:
If the maximum concurrency set for the throttling group is greater than the sum of the maximum concurrencies set for each business service in the group, Service Bus uses the concurrencies set for the business services.
If the maximum concurrency set for the throttling group is less than the sum of the maximum concurrencies set for each business service in the group, the concurrency limit set for the throttling group takes precedence.
The throttling queue length and the message expiration time set for the throttling group specify default values in case they are not defined for each business service in the group. These values are inherited from the group regardless of whether the throttling group is enabled or disabled. Service Bus determines the throttling queue length and message expiration properties as follows:
The property values specified for a business service associated with a throttling group override the values specified for the throttling group itself. However, if the values for a business service are greater than the values for its associated throttling group, the values set for the throttling group take precedence.
If the property values are not specified for a business service, the values defined for the throttling group are used.
In Service Bus, a business service can be associated with multiple endpoint URIs. For more information on endpoint URIs, see Monitoring and Managing Endpoint URIs for Business Services. When you associate a business service with multiple URIs, you configure the maximum concurrency for the business service and not the individual URI. The maximum concurrency for each URI is set internally depending on the overall maximum concurrency and the load balancing weight, based on the following equation:
URI-specific max_concurrency = [User configured max_concurrency] X [weight]
For example, consider a business service B with three endpoint URIs eu1, eu2, and eu3. The load balancing algorithm is defined as random-weighted. The weights of the URIs are 1, 2, and 3 respectively. Assuming that you have defined a maximum concurrency of 10 for the business service, the URI specific maximum concurrency is 10, 20, and 30. The effective maximum concurrency of the business service B is 60. If the last endpoint URI that has a weight of 3 is offline, the effective maximum concurrency of the business service is 30.
Note:
The weights for the URI when the load balancing is round robin or random is 1. When the load balancing is None the weight of the primary URI is 1 and the weight of the backup URI is 0. The weight of the backup URI becomes 1 when the primary URI goes offline.
Messages for which the endpoint URI is overridden in routing options are not throttled. Messages are also not throttled with session stickiness enabled.
When failover is enabled on a service, retried messages are not throttled. The message is sent to the next URI regardless of the operational settings for throttling. Messages that are expired or that are discarded, because the throttling queue is full or because the service reached its maximum concurrency, are not retried.
While endpoint throttling and dispatch policies (Work Managers) both work to limit loads, they work on different areas of processing. The Work Manager configured for a proxy service limits the number of threads running on that proxy service. A Work Manager configured on a business service limits the number of threads processing responses from the back-end system. Endpoint throttling configured on a business service limits requests to the back-end system. Using a combination of Work Managers and throttling gives you control over these three processing points.
The throttling capacity is a cluster-wide setting, with each managed server having its own server-specific capacity. Throttling capacity is equally split among all managed servers, regardless of whether they are all running.
When dividing work among managed servers in a cluster, the throttling capacity (maximum concurrency) for each server is rounded up. For example, if a cluster has three managed servers and the capacity is ten, the capacity distribution is four for each server. When throttling is configured, each managed server in a cluster has a throttling capacity of at least one.
Service Bus displays the service metrics for throttling on the Service Bus Project business service Dashboard page on Oracle Enterprise Manager.
The dashboard displays the maximum throttling time, the minimum throttling time, and the average throttling time for the current aggregation interval and for the time period since the last time statistics were reset. For more information, see Viewing All Service Health Statistics for a Service. You can also access the metrics using JMX Monitoring APIs. For more information, see JMX Monitoring API.
You can define an SLA alert rule based on the available throttling metrics. For example, you could create an alert for when the maximum throttling time exceeds a specific amount of time. For more information, see Creating Service Level Agreement Alert Rules.
You enable and disable throttling, as well as configure throttling properties, on the Properties tab of the business service's home page in Fusion Middleware Control.
Any changes you apply on this page take effect immediately.
Once you deploy or activate a business service to the runtime, you can enable or disable throttling for that service and you can configure the throttling options. When throttling is enabled, the flow of messages is restricted for the endpoints and messages are processed by priority. You can optionally assign messages a priority using routing options; otherwise, messages are dequeued on a first-in, first-out basis.
To configure throttling for a business service
If you no longer want to throttle messages for a business service, you can disable throttling for that service at any time. When you disable throttling, message in the throttling queue are processed normally, and the message flow is no longer restricted by the throttling parameters.
To disable throttling for a business service
You create and configure throttling groups in either Oracle JDeveloper or the Oracle Service Bus Console, depending on which you use to develop your projects.
To enable throttling for the business services in a throttling group, you must enable throttling for the group and for each business service in the group.
Use throttling groups to restrict the flow of messages to remote servers from a group of business services.
To create a a throttling group:
Once you create a throttling group, you can associate business services with that group and remove business services that are already associated with the group.
To associate business services with a throttling group:
Once you create a throttling group you can modify the throttling properties or disable the group if you no longer need to restrict the message flow for the group.
To edit a throttling group:
When you delete a throttling group, any messages currently being processed and any messages in the throttling queue are processed completely. The associated business services might still be configured for individual throttling in Fusion Middleware Control. If you want to disable throttling for all business services in the group, be sure to disable them individually, as described in Disabling Throttling for a Single Business Service.
To delete a throttling group: