20 Introduction to Coherence Topics
This chapter includes the following sections:
Topics Overview
One or more publishers publish a stream of values to a Topic. One to many subscribers consume the stream of values to a Topic. The Topic values are spread evenly across all Oracle Coherence data servers, enabling high throughput processing in a distributed and fault tolerant manner. Each direct subscriber to a Topic receives all values published to the Topic.
Figure 20-1 Topic Values
In publish and subscribe messaging, messages are delivered only to active subscribers of the Topic. Subscriber groups add queue processing capabilities to Topics. After a subscriber group is created on a Topic, all values sent to the Topic are accumulated for the subscriber group. One or more subscribers can subscribe to a subscriber group. For the stream of values accumulated by a subscriber group, each value is only delivered to one of its subscriber group members, thus enabling distributed, parallel processing of these values.
Note:
For examples of basic Topic operations, see Performing Basic Topic Publish and Subscribe Operations.Figure 20-2 Topic Values in a Subscriber Group
About Channels
A publisher is configured to control what ordering guarantees exist for the messages it publishes when they are received by subscribers. This is achieved by publishing messages to specific Channels. All messages published to a Channel will be received by subscribers in the order a publisher published them to that Channel. Messages published to a Channel by different publishers may be interleaved; there is no ordering guarantee across different publishers. Messages published to different Channels may be interleaved as they are received by subscribers, and may be received by different subscribers in a subscriber group.
The number of Channels that a Topic has allows publishers to scale better as they avoid contention that may occur with many publishers publishing to a single Channel. Message consumption can be scaled because multiple subscribers (in a group) will subscribe to different Channels, so scaling up receiving of messages, while maintaining order.
As new subscribers are created in the same subscriber group, the available Channels are allocated as fairly as possible among the subscribers. If there are more subscribers than Channels, then some subscribers will not be assigned a Channel and will not receive any messages. If a subscriber disconnects for some reason, such as failure or a heart-beat timeout, then its assigned Channels are reallocated to the remaining subscribers. It is possible to register listeners to be notified of Channel allocation changes.
The Channel count for a Topic is configurable, ideally a small prime (the default is 17). There are pros and cons with very small or very large Channel counts, depending on the application use case and what sort of scaling or ordering guarantees it requires.
About Position
NamedTopic implementation
to track the Position of an element in a channel and maintain ordering of
messages.
Positions are then used by subscribers to track the elements that they have received, and when committing a Position to determine which preceding elements are also committed, and to then recover to the correct Position in the Topic when subscribers reconnect or recover from failure.
While a Position data structure is opaque, they are serializable, meaning that they can be stored into a separate data store by application code that wants to manually track message element processing. The combination of Channel and Position should be unique for each message element published and received.
Topics Resilience and Availability
On a cluster member, topics are highly resilient and will continue to function during rolling upgrades of cluster members.
As the topics API is generally asynchronous, API methods that publish to or subscribe to topics will continue to function, but will just take longer to complete. Publishing and subscribing on a cluster member can also survive the loss of all storage enabled members as the publish and subscribe operations will pause until storage members restart.
In an Extend or gRPC client, just like with caches, a client will reconnect if the proxy
it is connected to dies, or it becomes disconnected for some other reason, such as a
network issue. This means that generally, a topics operation will work as long as the
client is able to reconnect to a proxy. Publishers and subscribers will attempt to
connect behind the scenes for asynchronous calls, so these too would generally succeed.
Where there may be issues or exceptions thrown is when a request is actually in-flight
when the proxy connection fails. In this case, just like with caches, the caller will
receive an exception, but the operation may still be executing on the cluster and may
actually complete. For a publisher for example, the client may receive an exception and
then have no knowledge whether a publish request was actually successful on the cluster.
It is up to the application code to properly handle the
CompletableFuture returned from asynchronous topic API calls and
decide what action to take on errors.