13 Developing Advanced Pub/Sub Applications

This chapter describes advanced WebLogic JMS publish and subscribe (pub/sub) concepts and functionality of Uniform Distributed Topics (UDTs) necessary to design high availability (HA) applications.

Overview of Advanced High Availability Concepts

The following sections provide information on WebLogic Server high availability features and concepts:

Note:

Oracle recommends designing applications that utilize WebLogic Server MDBs or the Oracle SOA JMS Adapter rather than explicitly handling all potential topology changes.

WebLogic Messaging High Availability Features

Oracle's WebLogic messaging offer high availability (HA) and scalability using the following features:

Distributed Destinations make a group of JMS physical destinations accessible as a single, logical destination to a client. Applications that use distributed destinations usually have higher availability and better scalability because WebLogic JMS provides load balancing and failover among member destinations of a distributed destination within a cluster. Automatic Service Migration (ASM) and Whole Server Migration (WSM) enable restarting either a set of services (including JMS servers and destinations) or an entire WebLogic Server instance in a new location. These migration features provide high availability for the individual members of a distributed destination.

The nature of these technologies means that the topology of a JMS system can be unknown to a client application as:

  • The scaling of a cluster, along with the scaling of a distributed destination may exceed the number of consumers defined by the application.

  • The topology may dynamically change in the event of a server or service failure.

Typically, topology changes are handled transparently using MDBs either locally or on a remote WebLogic Server instance. However, when using other client types, these topology changes must be explicitly handled by the application, especially if the application is remote to the servers hosting the JMS destinations.

Application Design Limitations When using Replicated Distributed Topics

Applications implementing Uniform Distributed Topics prior to WebLogic Server 10.3.4.0 were constrained by the following limitations:

  • Messages are always forwarded and duplicated across a distributed topic, which means that either parallel processing, and/or ensuring that a clustered application gets exactly one copy of each message, may requires significant additional configuration, coding, and message hops.

  • Only one consumer at a time can process the messages in a given subscription except for the limited case of Non-XA MDBs where all processing of the subscription has to occur on the same server with a thread pool. This prevents most customers from designing application architectures that intend to have "round-robin" distributed or parallel processing of a single subscription's topic messages, instead of single-threaded processing.

  • MDBs only directly support durable subscriptions on distributed topics that are located in the same cluster.

  • For applications other than MDBs, a durable subscriber created for a distributed topic can only be created on a distributed topic (DT) member, and the durable subscription will only exist on that member. If the member hosting the subscription is down, the subscription will not be available to any subscriber (and is therefore not "highly available" by definition).

  • Pinning subscribers to a distributed topic member prevents automatic adjustment to changes in topology in the same way that adjustments are made for distributed queues.

Advanced Topic Features

Starting in WebLogic Server 10.3.4.0, partitioned distributed topics, combined with the ability to share subscriptions and allow multiple connections to use the same Client ID, provide the following application design patterns that provide parallel processing and HA capabilities similar to distributed queues:

  • One-copy-per-instance: Each instance of an application gets one copy of each message that is published to the Topic.

  • One-copy-per-application: Each application as a whole (that is all instances of the application together) receives one copy of each message that is published to the Topic. That is each instance only receives a subset of the messages that are sent to the Topic.

Note:

Oracle recommends designing applications that utilize WebLogic Server MDBs. See Configuring and Deploying MDBs Using JMS Topics in Programming Message-Driven Beans for Oracle WebLogic Server for detailed information on how to design and implement applications that use message-driven beans to provide improved HA and scalability.

Advanced Messaging Features for High Availability

In order to understand how an application can achieve One-copy-per-instance and One-copy-per-application design patterns, you need to understand the following new and changed features:

Shared Subscriptions and Client ID Policy

Prior to WebLogic Server 10.3.4.0, one subscription, durable or non-durable, could only be accessed by a single subscriber instance at any given time. Each subscriber receives all messages that are sent to a topic after the subscription is established and the messages for each subscription are processed sequentially by one consumer.

In this WebLogic Server release, multiple subscribers can share one subscription (durable or non-durable). Messages are distributed among multiple consumers that share the same subscription and can be processed in parallel. Subscription sharing only occurs on the same destination instance or the same member instance of a DT. See Configure Shared Subscriptions in Configuring and Managing JMS for Oracle WebLogic Server.

In order to share a subscription, durable or non-durable subscriptions need to have the Client Id set on their connection factory or connection. Prior to WebLogic Server 10.3.4.0, a Client ID was exclusively used by one connection at any given time. In this release of WebLogic Server, this restriction is relaxed and a new Client ID Policy is used to restricted or not restricted use of a Client ID. The default policy, Restricted, allows only one Client ID to be used by one connection. The Unrestricted policy allows multiple connections to use the same client ID. For more information, see How Sharing a Durable Subscription Works.

What is the Subscription Key

A subscription key is used to uniquely identify a subscription. For non-durable subscriptions, the key is composed of the Client ID and Client ID Policy. For durable subscriptions, the key is composed of the Client ID, Client ID Policy, and Subscription Name.

Configuring a Shared Subscription

To configure a shared subscription, you need to configure the Subscription Sharing Policy attribute on the connection factory. Setting the Subscription Sharing Policy to Sharable allows subscribers created using a connection factory to share their subscriptions with other subscribers, regardless of whether those subscribers are created using the same connection factory or a different connection factory. Consumers can share a non-durable subscriptions only if they have the same Client ID and Client ID Policy. Consumers can share a durable subscription only if they have the same Client ID, Client ID Policy, and Subscription Name. See:

How Sharing a Non-Durable Subscription Works

In order to share a subscription among multiple non-durable subscribers, the subscribers have to have a Client ID, which serves to identify the subscription. All subscribers that intend to share a subscription must have the same subscription key (ClientID and ClientIDPolicy) on their connection. If Subscription Sharing Policy is set to SHARABLE, but the ClientID is not set on the Connection, the subscription is not a shared subscription.

The first subscriber that is created with a subscription key creates the subscription. All subsequently created subscribers with the same subscription key share the subscription created by the first subscriber if all subscription details (such as: the selector, noLocal option, and the physical destination) match. For example:

  • If a subscription is created with a selector and noLocal option, a subscriber creation call that uses the same subscription key but a different selector, noLocal option or a different physical destination is treated as a different subscription.

  • If a ClientID is used by an EXCLUSIVE subscriber, any current or subsequent subscribers using the same ClientID, selector, and noLocal option is treated as a different subscription.

    Note:

    It is only possible to have the same ClientID if the subscriber is created with the same connection instance or a connection using the UNRESTRICTED Client ID policy).

How a Shared Subscription Policy for a Non-durable Subscription is Determined

The Subscription Sharing Policy for a particular non-durable subscription is dynamically determined by the first active subscriber on the subscription and does not change for the life of the subscription. Any attempt to change the Shared Subscription Policy for a subscription throws an InvalidSubscriptionSharingException, which extends javax.jms.JMSException. For example:

  • If a non-durable subscription has an EXCLUSIVE subscriber on a destination, the subscription is EXCLUSIVE, and any attempt to create an additional subscriber using the subscription on the same destination always fails with an InvalidSubscriptionSharingException, regardless of whether the yet-to-be-created subscriber is EXCLUSIVE or SHARABLE.

  • If a subscription has active subscribers with a SHARABLE policy, the subscription is SHARABLE, and any attempt to create a new EXCLUSIVE subscriber on the subscription fail with an InvalidSubscriptionSharingException.

How a Non-durable Subscription is Closed

After all subscribers that share the same subscription close, the subscription is cleaned up. Specifically, when the last subscriber consumer on a shared subscription calls close(), the subscription and all the associated JMS resources cleaned up.

There is no runtime mbean that represents a non-durable subscription, regardless of whether it is a shared or exclusive subscription. It is possible to monitor individual susbcribers using the appropriate JMSConsumerRuntimeMBean.

How Sharing a Durable Subscription Works

In previous releases, the subscription key (<ClientID, SubscriptionName>) uniquely identified a subscription within a cluster where the subscription could only exist on a single destination instance or a single member of a DT within the cluster. In this WebLogic Server release, the subscription key becomes <ClientID, ClientIDPolicy, SubscriptionName>. All durable subscribers that use the same subscription key share the same subscription if they subscribe to a regular topic, or if they subscribe to the same member of a distributed topic. Multiple subscriptions that use the same subscription key can exist on multiple distributed destination member destinations.

The first subscriber that is created with a particular subscription key creates the subscription. All subsequently created subscribers with the same subscription key share the subscription created by the first subscriber if all subscription details (such as: the selector, noLocal option, and the physical destination) match and they are on the same physical destination.

If a subscription is created with a selector and the noLocal option, a subscriber created on the same physical destination using the same subscription key with a different selector and noLocal option will:

  • Replace the existing durable subscription and cleanup all pending messages that are saved for the durable subscription if there are no active subscribers using this existing subscription.

  • Throw InvalidSubscriptionSharingException if there are active subscribers using the same subscription key with a different selector or noLocal option.

How a Shared Subscription Policy for a Durable Subscription is Determined

The Subscription Sharing Policy for a particular durable subscription is dynamically determined by the first active subscriber on the subscription and does not change unless all current subscribers close and new subscribers attach with a different policy. Any attempt to change the policy of a subscription that already has active subscribers throws an InvalidSubscriptionSharingException. For example:

  • If a durable subscription has an EXCLUSIVE subscriber and the Subscription Sharing Policy is EXCLUSIVE, any attempt to create an additional subscribers on the subscription throws InvalidSubscriptionSharingException, regardless of whether the yet-to-be-created subscriber is EXCLUSIVE or SHARABLE.

  • If a durable subscription has active subscribers with a SHARABLE policy, the Subscription Sharing Policy is SHARABLE, any attempt to create a new EXCLUSIVE subscriber on the subscription throws InvalidSubscriptionSharingException. Please note that changing Subscription Sharing Policy on an existing durable subscription does not delete any messages that already exist on the subscription.

Note:

Changing the Subscription Sharing Policy on an existing durable subscription does not delete any messages that already exist on the subscription.

How to Unsubscribe a Durable Subscription

Before unsubscribing a subscription, you must consider the Client ID Policy for the subscription:

  • Applications that use a Client ID Policy with a value of RESTRICTED unsubscribe a durable subscription using the standard Session.unsubscribe(String name) API.

Note:

Prior to WebLogic Server 10.3.4.0, all Client ids are RESTRICTED by default. A given client id could only be used by one connection at any given time in a WLS JMS cluster.

  • Application that use a Client ID Policy with a value of UNRESTRICTED unsubscribe a durable subscription using the WLSession.unsubscribe(String name, Topic topic) extension by supplying the subscription name and the topic or a DT member object.

Considerations when Unsubscribing a Durable Subscriber

The following section provides information on how to unsubscribe or avoid scenarios that throw an exception:

  • If there are active consumers on the subscription, a call to unsubscribe() throws a JMSException.

  • If there are no active consumers, on a subscription, a call to unsubscribe() deletes the matching durable subscription identified by the subscription key <ClientID, ClientIDPolicy, SubscriptionName>.

  • unsubscribe() of a durable subscription is done per standalone topic or per member of a DT.

  • A subscription created using a connection with a RESTRICTED Client ID can only be cleaned up from a connection that uses the same RESTRICTED Client ID.

  • A subscription created using a connection with an UNRESTRICTED Client ID can only be cleaned up from a connection using the same UNRESTRICTED Client ID.

  • If WebLogic JMS does not find a matching subscription on the topic that was created with the same Client ID and ClientIDPolicy as the unsubscribe call, an InvalidDestinationException is thrown.

  • If an unsubscribe call with an UNRESTRICTED Client ID specifies a DT or does not specify any Topic, an InvalidDestinationException is thrown.

  • Although .Net and C API messaging applications can share subscriptions by using the Client ID Policy and Subscription Sharing Policy on a connection factory deployed on WebLogic Server 10.3.4.0 or later, an unsubscribe API extension is not yet available for subscriptions that use an Unrestricted Client ID. The workaround is to use administrative measures described in Managing Durable Subscriptions.

Managing Durable Subscriptions

When there are subscriptions distributed throughout a cluster, it is possible there are some subscriptions that should have been deleted but have not been deleted. Such subscriptions are sometimes called "abandoned" subscriptions, and can continue to accumulate messages even though there's no subscriber processing the messages. If the accumulating messages never expire, they may eventually cause the topic to begin throwing resource allocation exceptions (quota exceptions), or if quotas aren't configured, can even cause a server to run out of memory.

For example, the unsubscribe call fails when there are active subscribers on the subscription and the unsubscribe call does not reach subscriptions on inactive (shutdown) members. When this happens, the subscription is left on the member where the call failed until it is manually removed by an administrator or the call is repeated.

To help handle these situations, administrators have the following options to monitor and manage durable subscriptions:

  • There is one instance of the JMSDurableSubscriptionRuntimeMBean for each durable subscription. Administrators can monitor on a topic or UDT using the Administration Console or by using WLST command line or scripts. See Monitor JMS servers in Oracle WebLogic Server Administration Console Help.

  • To find an abandoned or orphaned durable subscription, administrator can check the LastMessagesReceivedTime on the JMSDurableSubscriberRuntimeMBean. The getLastMessagesReceivedTime() method returns the last time a message was received by a subscriber from the subscription. Based on this information, together with attributes like the MessagesPendingCount or BytesPendingCount on the same MBean, an administrator can build a clear picture of the status of a particular durable subscription and take appropriate action, such as cleanup the resources.

Naming Conventions for the JMSDurableSubscriberRuntimeMbean

If a durable subscription is created using the subscription key, <MyClientID, MySubscriptionName>, the name of the associated JMSDurableSubscriberRuntimeMBean is either:

  • MyClientID_MySubscriptionName when the Client ID Policy is RESTRICTED. Where MyClientID is the Client ID for this subscription and MySubscriptionName is the name of the subscription.

  • MyClientID_MySubscriptionName@topicName@JMSServerName when the Client ID Policy is UNRESTRICTED. Where MyClientID is the Client ID for this subscription, MySubscriptionName is the name of the subscription., topicName is the name of a standalone topic or a member of a UDT, and JMSServerName is the name of the JMSServer that the topic or member is deployed on.

Design Strategies when using Topics

The following sections provide information on Topic-based design strategies that can be used to develop high availability applications:

One-copy-per-instance Design Strategy

One-copy-per-instance is the traditional design pattern and is backward compatible with WebLogic Server releases prior to 10.3.4.0. One-copy-per-instance has the following characteristics:

  • Each instance of an application gets one copy of each message that is published to the Topic.

One-copy-per-application Design Strategy

One-copy-per-application is a design pattern available in WebLogic Server 10.3.4.0 and higher releases. One-copy-per-application has the following characteristics:

  • This pattern is usually best implemented by leveraging an MDB, which sets up policies and subscriptions across a cluster automatically. See Best Practices for Distributed Topics.

  • Each application as a whole (that is all instances of the application together) receives one copy of each message that is published to the DT. That is each instance only receives a subset of the messages that are sent to the DT.

  • An UNRESTRICTED Client ID Policy

  • An SHARABLE Subscription Sharing Policy

  • Uses the same subscription name if the subscribers are durable

  • All consumers subscribe to the same topic instance (or member of a DT)

Best Practices for Distributed Topics

Oracle recommends the following when designing new applications using distributed topics: