Throttling

Overview

The Throttling filter can protect a service from message flooding. You can configure the filter to only allow a certain number of messages from a specified client in a given time frame through to the protected system. If the number of messages exceeds the specified limit, the filter fails for the excess messages.

[Important] Important

The filter still succeeds for incoming messages that meet the specified constraints. For example, if the filter is configured to allow 20 messages through per second, it fails for the 21st message, but passes for the first 20 incoming messages.

The API Gateway's behavior in the case of a breach in the configured constraints is determined by the filter that is next in the failure path for the Throttling filter in the policy. Typically, an Alert, Trace, or Log filter is configured as the successor filter in the failure path in the policy.

An example use-case of this filter would be to protect a service that can only handle a maximum of 20 messages per client per second. If the filter detects a higher number of incoming requests, it blocks the messages.

General Settings

Name:

Enter an appropriate name for the filter.

Number of Messages:

If the API Gateway receives more than this number of messages during the time interval specified in the field below, the filter fails. Otherwise, the filter passes.

Time Period:

If the API Gateway receives more than the number of messages specified in the above field in the time interval specified here, the filter fails. Otherwise, the filter passes. The time period depends on the value selected below (seconds, minutes, hours, days, or weeks). For example, if you enter 10 as the Time Period and select Minutes from the Time Period Unit drop-down list below, the time period lasts 10 minutes.

Time Period Unit:

The unit that the time period is measured in must be selected from the following options: Second, Minute, Hour, Day, or Week. The value selected together with that entered above determines the time period.

Alternatively, you can specify the time period unit using a selector to represent the value of a message attribute, which is looked up and expanded to a value at runtime. Use the following syntax to specify the selector: ${attribute.name}. For more details on selectors, see Selecting Configuration Values at Runtime. The values allowed in the message attribute are Second, Minute, Hour, Day, or Week.

[Note] Note

When one of Second, Minute, or Hour is specified, you do not need to configure the Time Period Commences on Hour/Day fields. The time period commences when a message is received and lasts for the time period (for example, 10 minutes). When this time period is up, the message count is reset, and the counter starts again when another message is received. The sections that follow explain how the time period is measured when the Time Period Unit is set to Day or Week.

Time Period Commences on Hour:

This field must be configured if you select either Day or Week as the Time Period Unit above. For example, if you select Day above and enter 00:00 in this field, this means that only the specified number of messages can be received in a one day period starting from midnight tonight until midnight the next day.

Time Period Commences on Day:

This field must only be configured if you select Week as the Time Period Unit above. For example, if you select Week and 00:00 in the fields above, and enter Sunday, this means that the time period commences next Sunday at midnight and lasts for one week exactly. The time period is reset on midnight of the next Sunday.

Cache Settings

Track per Key:

Select this setting if you wish to configure the API Gateway to keep track of request messages based on a specific key value. In other words, if more messages that match this key are received than are allowed, the filter fails.

Key Value:

You can use the Track per Key option to perform message filtering based on a particular key. This key can be used to look up entries in the cache selected below. Defaults to the following:

${http.request.clientaddr.getAddress()}

The key entered can also be a combination of a fixed string value and/or API Gateway message attribute selector. For example, you could use the following key to keep track of the number of times a particular URI is requested:

MSG_COUNT-${http.request.uri}

Select Cache to Use:

In cases where multiple API Gateways are deployed for load balancing purposes and you want to maintain a single count of all messages processed by all the API Gateway instances, you can configure a distributed cache to cache request messages.

For example, assume the intention is to prevent a burst of more than 50 messages per second from reaching the back-end Web Service. Also assume that a load balancer is deployed in front of two instances of the API Gateway and round-robins requests between these two instances. By caching request messages in a global distributed cache, which is inherently replicated across all API Gateway instances, the Throttling filter can compute the total number of messages in the distributed cache and hence, the total number of messages processed by all API Gateway instances.

The Throttling filter uses the pre-configured Local maximum messages cache by default. To configure a different cache, click the button on the right, and select from the list of currently configured caches in the tree. To add a cache, right-click the Caches tree node, and select Add Local Cache or Add Distributed Cache. Alternatively, you can configure caches under the Libraries node in the Policy Studio tree. For more details, see the topic on Global Caches.

Using Multiple Throttling Filters

If you want two or more Throttling filters to maintain separate message counts, you must use a different key into the cache for each filter, or use different caches for each filter:

  • Using a Different Key per Filter:

    With this approach you can use a unique Track per key value in each Throttling filter. The easiest way to do this is to prepend the default ${http.request.clientaddr.getAddress()} selector value with the filter name, for example:

    ${filterName} ${http.request.clientaddr.getAddress()}

    This ensures that each filter maintains its own separate message count in the selected cache.

  • Using a Unique Cache per Filter:

    Alternatively, you can use a unique cache to store the message count of each Throttling filter. With this solution, you must configure a separate cache for each Throttling filter that you have configured throughout all policies running on the API Gateway.