10 Using Distributed Destinations

Understand the concepts and functionality of distributed destinations necessary to design high availability (HA) applications.

What Is a Distributed Destination?

A distributed destination is a set of destinations (queues or topics) that are accessible as a single, logical destination to a client.

A distributed destination has the following characteristics:

It is referenced by its own JNDI name.
Each member of the set can belong to a separate JMS server distributed across multiple servers within a single cluster, or can be located on JMS servers that are all on the same single non-clustered standalone server. Members of the set cannot be distributed across multiple non-clustered standalone servers, and cannot be distributed across multiple clusters.

Why Use a Distributed Destination

Applications that use distributed destinations are more highly available than applications that use simple destinations because WebLogic JMS provides load balancing and failover for member destinations of a distributed destination within a cluster.

Once properly configured, your producers and consumers are able to send and receive messages through the distributed destination. WebLogic JMS then balances the messaging load across all available members of the distributed destination. When one member becomes unavailable due a server failure, traffic is then redirected toward other available destination members in the set. For more information about how destination members are load balanced, see "Configuring Distributed Destination Resources" in Administering JMS Resources for Oracle WebLogic Server.

Creating a Distributed Destination

System administrators create distributed destinations by using the WebLogic Server Administration Console.

See Configuring Distributed Destination Resources in Administering JMS Resources for Oracle WebLogic Server.

Types of Distributed Destinations

Learn about the two types of distributed destinations supported by WebLogic Server.

Uniform Distributed Destinations

In a uniform distributed destination (UDD), each of the member destinations has a consistent configuration of all distributed destination parameters, particularly in regards to weighting, security, persistence, paging, and quotas.

Oracle recommends using UDDs because you no longer need to create or designate destination members, but instead rely on WebLogic Server to uniformly create the necessary members on the JMS servers to which a UDD is targeted. This feature of UDDs provides dynamic updating of a UDD when a new member is added or a member is removed.

For example, if a UDD is targeted to a cluster, there is a UDD member on every JMS server in the cluster. If a new JMS server is added, then a new UDD member is dynamically added to the UDD. Likewise, if a JMS server is removed, then the corresponding UDD member is removed from the UDD. This allows UDDs to provide higher availability by eliminating bottlenecks caused by configuration errors. For more information, see Configuring Distributed Destination Resources in Administering JMS Resources for Oracle WebLogic Server.

Weighted Distributed Destinations

Note:

Weighted distributed destinations are deprecated in Weblogic Server 10.3.4.0. Oracle recommends using Uniform Distributed Destinations.

In a weighted distributed destination, the member destinations do not have a consistent configuration of all distributed destination parameters, particularly in regards to weighting, security, persistence, paging, and quotas.

Oracle recommends converting weighted distributed destinations to UDDs because of the administrative inflexibility when creating members that are intended to carry extra message load or have extra capacity (more weight). Lack of a consistent member configuration can lead to unforeseen administrative and application problems because the weighted distributed destination can not be deployed consistently across a cluster.

For more information, see Configuring Distributed Destination Resources in Administering JMS Resources for Oracle WebLogic Server.

Using Distributed Destinations

A distributed destination is a set of physical JMS destination members (queues or topics) that is accessed through a single JNDI name.

As such, a distributed destination can be looked up using JNDI. Distributed destination implements the javax.jms.Destination interface, at http://docs.oracle.com/javaee/7/api/javax/jms/Destination.html, and can be used to create producers, consumers, and browsers.

For information about obtaining a reference to a distributed destination, see How to Look Up a Destination.

Using Distributed Queues

A distributed queue is a set of physical JMS queue members. As such, a distributed queue can be used to create a QueueSender, QueueReceiver, and a QueueBrowser. The fact that a distributed queue represents multiple physical queues is mostly transparent to your application.

The queue members can be located anywhere, but must all be served by JMS servers in a single server cluster. When a message is sent to a distributed queue, it is sent to one of the physical queues in the set of members for the distributed queue. Once the message arrives at the queue member, it is available for receipt by consumers of that queue member only. '

This section provides information on using distributed queues:

Queue Forwarding

Queue members can forward messages to other queue members by configuring the Forward Delay attribute in the WebLogic Server Administration Console, which is disabled by default. This attribute defines the amount of time, in seconds, that a distributed queue member with messages, but which has no consumers, will wait before forwarding its messages to other queue members that do have consumers. By default, WebLogic Server resets the delivery count when forwarding between distributed queue members. See Reset Delivery Count On Forward.

QueueSenders

After creating a queue sender, if the queue supplied at creation time was a distributed queue, then each time a message is produced using the sender a decision is made as to which queue member will receive the message. Each message is sent to a single physical queue member.

The message is not replicated. As such, the message is only available from the queue member where it was sent. If that physical queue becomes unavailable before a given message is received, then the message is unavailable until that queue member comes back online.

It is not enough to send a message to a distributed queue and expect the message to be received by a queue receiver of that distributed queue. Because the message is sent to only one physical queue member, there must be a queue receiver receiving or listening on that queue member.

Note:

For information about the load-balancing heuristics for distributed queues with zero consumers, see Configuring Distributed Destination Resources in Administering JMS Resources for Oracle WebLogic Server.

QueueReceivers

When creating a queue receiver, if the supplied queue is a distributed queue, then a single physical queue member is chosen for the receiver at creation time. The created QueueReceiver is pinned to that queue member until the queue receiver loses its access to the queue member. At that point, the consumer will receive a JMSException, as follows:

If the queue receiver is synchronous, then the exception is returned to the user directly.
If the queue receiver is asynchronous, then the exception is delivered inside of a ConsumerClosedException that is delivered to the ExceptionListener defined for the consumer session, if any.

Upon receiving such an exception, an application can close its queue receiver and recreate it. If any other queue members are available within the distributed queue, then the creation will succeed and the new queue receiver will be pinned to one of those queue members. If no other queue member is available, then the application would not be able to recreate the queue receiver and will have to try again later.

Note:

QueueBrowsers

When creating a queue browser, if the supplied queue is a distributed queue, then a single physical queue member is chosen for the browser at creation time. The created queue browser is pinned to that queue member until the receiver loses its access to the queue member. At that point, any calls to the queue browser will receive a JMSException. Any calls to the enumeration will return a NoSuchElementException.

Note:

The queue browser can only browse the queue member that it is pinned to. Even though a distributed queue was specified at creation time, the queue browser cannot see or browse messages for the other queue members in the distributed destination.

Using Replicated Distributed Topics

A distributed topic is a set of physical JMS topic members. A distributed topic can be used to create a TopicPublisher and TopicSubscriber. The fact that a distributed topic represents multiple physical topics is mostly transparent to the application.

Note:

Durable subscribers (DurableTopicSubscriber) cannot be created for distributed topics. However, you can still create a durable subscription on a distributed topic member and the other topic members will forward the messages to the topic member that has the durable subscription.

The topic members can be located anywhere but must all be served either by a single WebLogic Server or any number of servers in a cluster. When a message is sent to a distributed topic, it is sent to all of the topic members in the distributed topic set. This enables all subscribers to the distributed topic to receive messages published for the distributed topic.

A message published directly to a topic member of a distributed destination (that is, the publisher did not specify the distributed destination) is also forwarded to all the members of that distributed topic. This includes subscribers that originally subscribed to the distributed topic and happened to be assigned to that particular topic member. In other words, publishing a message to a specific distributed topic member automatically forwards it to all the other distributed topic members, just as publishing a message to a distributed topic automatically forwards it to all of its distributed topic members. For more information about looking up specific distributed destination members, see Accessing Distributed Destination Members.

This section provides information on using distributed topics:

TopicPublishers

When creating a topic publisher, if the supplied destination is a distributed destination, then any messages sent to that distributed destination are sent to all available topic members for that distributed topic (DT), as follows:

When some of the members of a uniform distributed topic are offline, non-persistent messages published to the distributed topic are saved for those members and made available when the members come back online.

In releases prior to 9.0, if you did not configure a persistent store for a JMS server or if there was a persistent store defined and storedEnabled=false was set on the distributed topic member, non persistent messages were dropped and not made available when the distributed topic member came back online. If your application depends on dropping these messages, you can approximate similar behavior by setting the time-to-live for a server to a very low value. This will allow the messages to be disregarded before an offline distributed topic member would come back online. New applications developed on WebLogic Server releases 10.3.4.0 and higher can use partitioned distributed topics with message-driven beans (MDBs) as message consumers to provide a similar capability. See "Developing Advanced Pub/Sub Applications" in Programming JMS for Oracle WebLogic Server.
If one or more of the distributed topic members is not reachable, and the message being sent is persistent, then the message is stored and forwarded to the other topic members when they become reachable. However, the message can only be persistently stored if the topic member has a JMS store configured.

Note:

Every effort is made to first forward the message to distributed members that utilize a persistent store. However, if none of the distributed members utilize a store, then the message is still sent to one of the members according to the selected load-balancing algorithm, as described in Configuring Distributed Destination Resources in Administering JMS Resources for Oracle WebLogic Server.
If all of the distributed topic members are unreachable (regardless of whether the message is persistent or non persistent), then the publisher receives a JMSException when it tries to send a message.

TopicSubscribers

When creating a topic subscriber, if the supplied topic is a distributed topic, then the topic subscriber receives messages published to that distributed topic. If one or more of the topic members for the distributed topic are not reachable by a topic subscriber, then depending on whether the messages are persistent or non persistent the following occurs:

Any persistent messages published to one or more unreachable distributed topic members are eventually received by topic subscribers of those topic members after they become reachable. However, the messages can only be persistently stored if the topic member has a JMS store configured.
Any non persistent messages published to those unreachable distributed topic members will not be received by that topic subscriber.

Note:

If a JMS store is configured for a JMS server that is hosting a distributed topic member, then all the Distributed Topic System Subscribers associated with that member destination are treated as durable subscriptions, even when a topic member does not have a JMS store explicitly configured. The saving of all the messages sent to these distributed topic subscribers in memory can result in unexpected memory and disk consumption. Therefore, a recommended best design practice when deploying distributed destination is to consistently configure all member destinations: either with a JMS store for durable messages or without a JMS store for non durable messages. For example, if you want all of your distributed topic subscribers to be no -durable, but some member destinations implicitly have a JMS store configured because their associated JMS server uses a JMS store, then you need to explicitly set the StoreEnabled attribute to False for each member destination to override the JMS server setting.

Ultimately, a topic subscriber is pinned to a physical topic member. If that topic member becomes unavailable, then the topic subscriber will receive a JMSException, as follows:

If the topic subscriber is synchronous, then the exception is returned to the user directly.
If the topic subscriber is asynchronous, then the exception is delivered inside of a ConsumerClosedException that is delivered to the ExceptionListener defined for the consumer session, if any.

After receiving this type of an exception, an application can close its topic subscriber and recreate it. If any other topic member is available within the distributed topic, then the creation should be successful and the new topic subscriber will be pinned to one of those topic members. If no other topic member is available, then the application will not be able to recreate the topic subscriber and will have to try again later.

Deploying Message-Driven Beans on a Distributed Topic

For information about how to deploy MDBs on topics, see Configuring and Deploying MDBs Using Distributed Topics in Developing Message-Driven Beans for Oracle WebLogic Server.

Using Partitioned Distributed Topics

Starting in WebLogic Server 10.3.4.0, partitioned distributed topics, combined with the ability to share subscriptions and allow multiple connections to use the same Client ID, provide the following application design patterns that provide parallel processing and HA capabilities similar to distributed queues:

One-copy-per-instance: Each instance of an application gets one copy of each message that is published to the Topic.
One-copy-per-application: Each application as a whole (that is all instances of the application together) receives one copy of each message that is published to the distributed topic . That is each instance only receives a subset of the messages that are sent to the distributed topic .

Note:

Oracle recommends designing applications that utilize WebLogic Server MDBs. See Configuring and Deploying MDBs Using Distributed Topics in Developing Message-Driven Beans for Oracle WebLogic Server for detailed information on how to design and implement applications that use message-driven beans to provide improved HA and scalability.

For more information about using Partitioned Distributed Topics, including information about replacing an existing Replicated Distributed Topic with a Partitioned Distributed Topic, see Developing Advanced Pub/Sub Applications.

Accessing Distributed Destination Members

For information on how to access distributed destinations and their members, see How to Look Up a Destination.

Distributed Destination Failover

Note:

If the distributed queue member on which a queue producer is created fails, yet the WebLogic Server instance where the producer's JMS connection resides is still running, then the producer remains active and WebLogic JMS will fail it over to another distributed queue member, irrespective of whether the Load Balancing option is enabled. For example, a WebLogic cluster contains WLSServer1, WLSServer2, and WLSServer3 and you are connected to WLServer2. If server WLSServer 2 fails, WebLogic JMS fail the producer over to one of the remaining cluster members. For more information, see Configuring Distributed Destination Resources in Administering JMS Resources for Oracle WebLogic Server.

A simple way to failover a client connected to a failed distributed destination is to write reconnect logic in the client code to connect to the distributed destination after catching onException.

Using Message-Driven Beans with Distributed Destinations

A message-driven bean (MDB) acts as a JMS message listener, which is similar to an event listener except that it receives messages instead of events.

Common Use Cases for Distributed Destinations

Understand when to use distributed destinations for your applications.

The following sections provide common use case scenarios when using distributed destinations:

Maximizing Production

To maximize message production, Oracle recommends that each member of a distributed destination be associated with a producer and a consumer. Figure 10-1 shows how to efficiently provide maximum message production and high availability using a UDD without using load balancing:

Figure 10-1 Paired Producers and Consumers

Description of "Figure 10-1 Paired Producers and Consumers"

In this situation, UDD1 is a uniform distributed destination composed of two physical members: D1 and D2. Each physical destination has a producer/consumer pair and the effective path for a message follows the solid line from the producer through the destination member to the consumer. If you are using ordering, you should have a producer for each expected Unit-of-Order. See Using Unit-of-Order with Distributed Destinations.

Maximizing Availability

This section provides information on how to maximize message availability.

Using Queues

Ideally, its best to pair a producer with a consumer but it is not always practical. The rate that messages are consumed is the limiting factor that determines the message throughput of your application. You can increase the availability of consumers by using load balancing between member destinations. In this situation, consumers are not paired with a producer as the UDD load balances an incoming message to the next available consumer using the assigned load balancing algorithm.

Note:

Some combinations of Unit-of-Order features can result in the starvation of competing Unit-of-Order message streams, including the under utilization of resources when the number of consumers exceed the number of in-flight messages with different Unit-of-Order names. You will need to test your applications under maximum loads to optimize your system's performance and eliminate conditions that under utilize resources.

Using Topics

When using a distributed topic, every member destination will forward its messages to every other member of the distributed topic.

Figure 10-2 Using Distributed Topics

Description of "Figure 10-2 Using Distributed Topics"

In Figure 10-2, UDD1 is a uniform distributed destination composed of two physical members: D1 and D2. Each physical destination has a producer/consumer pair. Each consumer receives messages sent by Producer 1 and Producer 2.

Stuck Messages

InFigure 10-3, a producer is sending messages to one member of a UDD but there is no consumer available to get the message. This typically happens as a producer sends a message to one of the destinations (D1) and a consumer is listening for messages on another destination (D2).

Figure 10-3 Stuck Messages

Description of "Figure 10-3 Stuck Messages"

UDD1 is a uniform distributed destination composed of two physical members: D1 and D2. D1 has a producer and D2 has a consumer. Avoid this configuration by using producer/consumer pairs or by configuring forwarding on the destination.