This chapter describes the different WebLogic Store-and-Forward (SAF) service concepts and features and how they work with WebLogic Server. It is assumed the reader is familiar with other WebLogic Server administration concepts.
This chapter includes the following sections:
These sections describe the components and participants of the SAF service.
The SAF service enables WebLogic Server to deliver messages reliably between applications that are distributed across WebLogic Server instances. For example, with the SAF service, an application that runs on or connects to a local WebLogic Server instance can reliably send messages to an endpoint that resides on a remote server. If the destination is not available at the moment the messages are sent, either because of network problems or system failures, then the messages are saved on a local server instance, and are forwarded to the remote endpoint once it becomes available.
WebLogic JMS utilizes the SAF service to enable local JMS message producers to reliably send messages to remote JMS queues or topics, as described in Using SAF with WebLogic JMS.
WebLogic Web Services relies on the SAF service to support the reliability of Web Services Reliable Messaging (WSRM), as described in Using SAF With Web Services Reliable Messaging.
There are two sides involved in the process of storing and forwarding messages: a local sending side and a remote receiving endpoint. SAF agents are responsible for store-and-forwarding messages between these local sending and remote receiving endpoints. A SAF agent can be configured to have only sending capabilities, receiving capabilities, or both.
JMS SAF only requires a sending agent on the sending side for JMS messages. Whereas, WSRM SAF requires both a sending agent and a receiving agent.
Sending agent — Used for JMS messages and WSRM. If message persistence is required, a sending agent stores messages to a persistent storage, forwards messages to the receiving side, and re-transmits messages when acknowledgements do not come back in time.
Receiving agent — Used only for WSRM. Receiving agents detect and eliminate duplicate messages sent by a sending agent, and delivers messages to the final destination.
Note:In the case of JMS messages, the JMS server associated with a remote exported JMS destination on the receiving side manages duplicate elimination.
A SAF agent is a configurable object that is similar to a JMS server in that it manages message persistence, paging parameters, and thresholds and quotas. The working behavior of the SAF service is controlled by a number of configurable parameters on SAF agents:
General configuration parameters, including:
select persistent storage
setting message paging defaults
specify delivery retry settings
determine window size
specify message acknowledgement intervals (WSRM only)
For more information about delivery retry settings, logging, and paging defaults, see Designing SAF Agents.
For more information about all general configuration parameters for SAF agents, see "Store-and-Forward Agents: Configuration: General" in the Oracle WebLogic Server Administration Console Online Help.
Threshold and quota parameters for controlling the message throughput of SAF agents.
For more information about threshold and quota parameters, see Designing SAF Agents.
Monitoring capabilities for SAF agents, remote endpoints, and conversations. You also have the capability to manage incoming, forwarding, and receiving message operations on SAF agents and to remote endpoints.
For more information about monitoring options for SAF agents, see Chapter 4, "Monitoring and Managing SAF Agents."
JMS message logging capabilities for imported SAF destinations. This is an external view of the events that a JMS message traverses through once it has been stored by a SAF agent and eventually forwarded to a remote JMS destination.
For more information about JMS message logging options for SAF destinations, see SAF Message Life Cycle Logging for JMS Messages
When targeting a SAF agent to a standalone server, you can either use the server's default persistent store or select an explicitly configured store. However, the following persistent store selection rules apply in order to properly target SAF agents in a cluster.
When not using migratable targets, each server instance in the cluster must use the default persistent store, and the SAF agent(s) must also use the default store.
For JMS messages, the sending SAF agent and the imported destination must reside on the same server instance. When a SAF agent is targeted to a cluster, an agent is automatically configured on each server instance in the cluster.
When targeted to a migratable target, a SAF agent must use a custom persistent store and it must targeted to the same migratable target used by the custom store. A SAF agent, JMS server, and custom store can share a migratable target. See "Understanding Migratable Targets In a Cluster" in Administering Clusters for Oracle WebLogic Server.
Note:To preserve SAF message consistency, WebLogic Server prevents you from retargeting an existing SAF agent to a migratable target. Instead, you must delete the existing SAF agent and configure a new one with the same values and target it to a migratable target.
The JMS store-and-forward uses a single sending SAF agent to provide highly-available JMS message production. For example, a JMS message producer connected to a local server instance can reliably forward messages to a remote JMS destination, even though that remote destination may be temporarily unavailable when the message was sent. JMS SAF is transparent to JMS applications; therefore, JMS client code still uses the existing JMS APIs to access remote destinations. For more information, see Chapter 3, "Configuring SAF for JMS Messages."
To ensure that a sending SAF agent for imported JMS destinations does not introduce a single point of failure for dependent JMS applications in a cluster, WebLogic Server can be configured to migrate SAF agents to a healthy server instance in the cluster. See Migratable SAF Agents for Highly Available Messaging.
WSRM uses a pair of sending and receiving SAF agents that are configured on the local and remote server instances, so that two Web Services running on different server instances can communicate reliably in the presence of failures in software components, systems, or networks. In particular, a message sent from a source endpoint to a destination endpoint is guaranteed either to be delivered, according to one or more delivery assurances, or to raise an error.
For more information, see "Using Web Service Reliable Messaging" in Developing JAX-RPC Web Services for Oracle WebLogic Server.
SAF services do not require you to configure Cross Domain Security. However, you will need to configure Cross Domain Security for all the domains your process communicates with if Cross Domain Security is configured on one domain and the membership of the Uniform Distributed Destinations are imported through SAF changes. A best practice is to keep all the domains used by your process symmetric, with respect to Cross Domain Security configuration— that is, all domains use Cross Domain Security (or are on the appropriate exception lists) or none of the domains have Cross Domain Security configured. See "Configuring Cross-Domain Security" in Administering Security for Oracle WebLogic Server 12c (12.2.1).
The SAF Service should be used when forwarding JMS or WSRM messages between WebLogic Server 9.x or later domains.
The SAF service can deliver messages:
Between two stand-alone server instances.
Between server instances in a cluster.
Across two clusters in a domain.
Across separate domains.
When not to use the SAF service:
Forwarding messages to prior releases of WebLogic Server.
Interoperating with third-party JMS products (for example, MQSeries).
For these tasks, you should use the WebLogic Messaging Bridge. See "Understanding the Messaging Bridge" in Administering the WebLogic Messaging Bridge for Oracle WebLogic Server.
When using temporary destinations with the
JMSReplyTo field to return a response to a request.
Additionally, when using JMS SAF, an application can only receive messages directly from a remote server, and only when the remote server is available.
These are the main tasks that must be completed to implement the SAF service in a domain.
Configure a SAF agent in the sending-side cluster or server instance(s). There are a number of ways to create SAF agents:
The WebLogic Server Administration Console enables you to configure, modify, target, monitor, and delete SAF agents in your environment. For step-by-step instructions of the SAF agent configuration tasks, see "Create Store-and-Forward agents" in the Oracle WebLogic Server Administration Console Online Help.
WebLogic Java Management Extensions (JMX) enables you to access the
"SAFAgentRuntimeMBean" MBeans to create and manage SAF agents. For more information see "Overview of WebLogic Server Subsystem MBeans" in Developing Custom Management Utilities Using JMX for Oracle WebLogic Server.
The WebLogic Scripting Tool (WLST) enables you to create and manage JMS servers and JMS system resources. For more information, see "Using WLST to Manage JMS Servers and JMS System Resources" in Administering JMS Resources for Oracle WebLogic Server.
When using WSRM, even if you configure a SAF agent with sending and receiving capability on the sending side, you must also configure a SAF agent with receiving capability on the receiving-side cluster or server instance.
When configuring a SAF agent, you can accept the server's default store or configure a store if you want a dedicated store for SAF messages. When targeting a SAF agent to a cluster, you must accept the server's default store. You can also target a SAF agent to a migratable target for high availability, as described in Persistent Store Rules When Targeting SAF Agents to a Cluster.
For more information about persistent stores, see "Using the WebLogic Persistent Store" in Administering the WebLogic Persistent Store.
For WebLogic JMS, configure SAF Imported Destination, SAF Context Handling, and SAF Error Handling (optional) objects in a JMS module, as described in Chapter 3, "Configuring SAF for JMS Messages."
Optionally, configure a Path Service for JMS messages when the sending-side is a cluster and the JMS producer is associated with a message unit-of-order. A message unit-of-order enables JMS message producers to group ordered messages into a single unit. For more information about JMS message unit-of-order, see "Using Message Unit-of-Order" in Developing JMS Applications for Oracle WebLogic Server.
The Path Service is a persistent map that can be used to store the mapping of a group of messages to a messaging resource such as a SAF agent. For more information about configuring a Path service, see "Using the WebLogic Path Service" in Administering JMS Resources for Oracle WebLogic Server.
Use the following information to help you design and configure SAF agents for forwarding messages.
The reliability of SAF is time based. You can configure the duration that a message needs to be delivered by a SAF agent reliably using the Time-To-Live parameter. When the configured Time-To-Live duration expires, the sending agent removes the message from its storage and discontinues attempts to retransmit the message to the receiving side. It is up to the application to decide what to do with the messages that fail to be delivered when its Time-To-Live expires.
For information on how JMS handles failed message deliveries, see SAF Error Handling.
Messages can fail to be delivered for the following reasons:
The endpoint does not exist (not configured)
The endpoint is down
Endpoint quota failure
Required QOS is not supported
The WSRM conversation times out (the conversation is idle for too long).
If the Logging parameter is enabled for the sending agent, it logs a message in the server log for every failed message. This is an alternative for applications that do not have their own failure handling or their failure handling cannot complete. For more information on WebLogic logging, see "Understanding WebLogic Logging Services" in Configuring Log Files and Filtering Log Messages for Oracle WebLogic Server.
The sending agent needs to connect to the receiving side in order to forward messages over, yet there are times that the connection may not be available. When an attempt to send a message fails, the sending agent must retry until it succeeds. Similarly, if the desired QOS is Exactly-Once or At-Least-Once, the sending agent must keep sending a message until it receives an acknowledgement for that message.
To control the frequency of attempts and the interval between two subsequent attempts, you can configure the default values for Retry Delay Base, Retry Delay Multiplier, and Retry Delay Maximum parameters. The Retry Delay Multiplier must be greater or equal to 1, and the Retry Delay Maximum value must greater or equal to the Retry Delay Base value. By default, the Retry Delay Multiplier value is set to 1, which means there's a fixed interval, defined by the Retry Delay Base value, between two successive attempts and that Retry Delay Maximum will be ignored. If the Retry Delay Multiplier is greater than 1, an exponential back-off algorithm will be used to adjust the retry intervals.
The delays will be exponentially increased, starting from the Retry Delay Base value, and will be multiplied by the Retry Delay Multiplier value each time. The amount of delay will not be increased any more once the Retry Delay Maximum value is reached. Once there is a successful attempt, the Retry Delay Multiplier value is reset to the Retry Delay Base value.
For more information about the delivery retry parameters for SAF agents, see "Store-and-Forward Agents: Configuration: General" in the Oracle WebLogic Server Administration Console Online Help.
Persistent messages are saved in the persistent store on the sending side until they are successfully forwarded to and acknowledged by the receiving side. However, non-persistent messages pending for delivery exist in-memory on the sending side, and all of the history records also exist in-memory on the receiving side. If the remote side is not available for a long time, the pending non-persistent messages could use up the sending side server's memory and even take the server down. By configuring quotas for each SAF agent, you can protect the server from running out of memory. Once the quota is about to be exceeded, the SAF agent will reject any new requests.
You can also configure SAF agents to page out messages or history records to a paging directory before the agent reaches the quotas. Paging will be triggered by certain conditions specified as thresholds in the SAF agent's configuration. The persistent store for messages or history records is also used for paging purposes.
The SAF agent threshold and quota parameters and relationship are the same as for JMS destinations and JMS servers.
For more information about threshold and quota parameters for SAF agents, see "Store-and-Forward Agents: Configuration: Thresholds & Quota" in the Oracle WebLogic Server Administration Console Online Help.
When a WebLogic Server instance reboots, the messages that were not sent before the server instance went down are recovered from the server's persistent store. The sending agent then attempts to send those messages to the remote side if they have not expired. Similarly, on the receiving side, the history records are recovered during reboot.
To ensure that a sending SAF agent for imported JMS destinations does not introduce a single point of failure for dependent JMS applications in the cluster, WebLogic Server can be configured to automatically or manually migrate SAF agents to a healthy server instance in a cluster.
You can use cluster targeted SAF agents and automatically migrate the agents and their custom stores to healthy servers in your WebLogic cluster. For more information, see the section "Simplified JMS Cluster and High Availability Configuration" in Administering JMS Resources for Oracle WebLogic Server.
For service migration, you can also use a migratable target, which serves as a grouping of pinned JMS-related services, such as JMS servers, SAF agents, and persistent stores, and that migratable target is hosted on only one physical server in a cluster. Such hosted services can be automatically migrated from the current unhealthy hosting server to a healthy active server with the help of the Health Monitoring subsystem. JMS services hosted by a migratable target can also be manually migrated, either in response to a server failure or as part of regularly scheduled maintenance. When the migratable target is migrated, all pinned services hosted by that target are also migrated.
See "Service Migration" in Administering Clusters for Oracle WebLogic Server.