1. Streams Developer's Guide
  2. Scaling a Streams Application

7 Scaling a Streams Application

You can scale a single streaming service to run on multiple nodes to handle a high volume of stream events from large Oracle NoSQL Database stores. The streaming service can use multiple subscribers to stream data from the Oracle NoSQL Database store.

The stream processing application does not need to know the topology of the Oracle NoSQL Database store, but can simply add or remove more independent subscribers as needed. All that the stream processing application needs to specify is the number of subscribers and a subscriber ID.

The following illustration depicts how a stream application can be scaled to use two clients to stream data from the six shards of the Oracle NoSQL Database store:

Multiple Subscribers Diagram

The Oracle NoSQL Database store strives to achieve even distribution of streams among its subscribers. As shown in the illustration, there are six shards and two subscribers. In this example, each subscriber receives streams from three shards. The subscriber does not choose which shard it gets streams from. The system determines this automatically, and the decision is transparent to the subscribers. In cases where there are more shards than of subscribers (as in this example), some subscribers can receive streams from more than one shard.

Note:

  • The maximum number of scalable subscribers cannot exceed the number of shards. For example, if the Oracle NoSQL Database has six shards, subscribers cannot be scaled to more than six clients.

  • Oracle NoSQL Databasethat you scale stream processing applications by running them on different nodes to benefit from newly added resources.

Although scalable subscribers can be created and run inside separate JVMs on the same node, such configuration would not have any benefit over running a single subscriber without using scalable subscribers. In our example, running two scalable subscribers inside different JVMs (but within the same node), streaming over three shards each, would not benefit over running a single subscriber on the same node that is subscribed to the entire (six shards) Oracle NoSQL Database store.