23 Managing Web Services in a Cluster

This chapter describes how to manage WebLogic web services in a cluster.

This chapter includes the following sections:

Note:

For considerations specific to using web service persistence in a cluster, see Using Web Service Persistence in a Cluster.

Overview of Web Services Cluster Routing

Clustering of stateless web services—services that do not require knowledge of state information from prior invocations—is straightforward and works with existing WebLogic HTTP routing features on a third-party HTTP load balancer.

Clustering of web services that require state information be maintained provides more challenges. Each instance of such a web service is associated with state information that must be managed and persisted. The cluster routing decision is based on whether the message is bound to a specific server in the cluster. For example, if a particular server stores state information that is needed to process the message, and that state information is available only locally on that server.

Note:

Services that use session state replication to maintain their state are a separate class of problem from those that make use of advanced web service features, such as Reliable Secure Profile. The latter require a more robust approach to persistence that may include storing state that may be available only from the local server. For more information, see A Note About Persistence.

In addition to ensuring that the web service requests are routed to the appropriate server, the following general clustering requirements must be satisfied:

  • The internal topology of a cluster must be transparent to clients. Clients interact with the cluster only through the front-end host, and do not need to be aware of any particular server in the cluster. This enables the cluster to scale over time to meet the demands placed upon it.

  • Cluster migration must be transparent to clients. Resources within the cluster (including persistent stores and other resources required by a web service or web service client) can be migrated from one server to another as the cluster evolves, responds to failures, and so on.

To meet the above requirements, the following methods are available for routing web services in a cluster:

  • In-place SOAP router—Assumes request messages arrive on the correct server and, if not, forwards the messages to the correct server ("at most one additional hop"). The routing decision is made by the web service that receives the message. This routing strategy is the simplest to implement and requires no additional configuration. Though, it is not as robust as the next option.

  • Front-end SOAP router (HTTP cluster servlet only)—Message routing is managed by the front-end host that accepts messages on behalf of the cluster and forwards them onto a selected member server of the cluster. For web services, the front-end SOAP router inspects information in the SOAP message to determine the correct server to which it should route messages.

    This routing strategy is more complicated to configure, but is the most efficient since messages are routed directly to the appropriate server (avoiding any "additional hops").

    Note:

    When using Make Connection, as described in Using Asynchronous Web Service Clients From Behind a Firewall (Make Connection), only front-end SOAP routing can guarantee proper routing of all messages related to a given Make Connection anonymous URI.

This chapter describes how to configure your environment to optimize the routing of web services within a cluster. Use of the HTTP cluster servlet for the front-end SOAP router is described. The in-place SOAP router is also enabled and is used in the event the HTTP cluster servlet is not available or has not yet been initialized.

A Note About Persistence

While it is possible to maintain state for a web service using the HttpSession as described in Programming Stateful JAX-WS Web Services Using HTTP Session, in some cases this simple persistence may not be robust enough. Advanced web services features like reliable messaging, Make Connection, secure conversation, and so on, have robust persistence requirements that cannot be met by using the HttpSession alone. Advanced web service features use a dedicated persistence implementation based on the concept of a logical store. For more information, see Managing Web Service Persistence.

At this time, these two approaches to persistence of web service state are not compatible with each other. If you choose to write a clustered stateful web service using HttpSession persistence and then use the advanced web service features from that service (either as a client or service), Oracle cannot guarantee correct operation of your service in a cluster. This is because HttpSession replication may make the HttpSession available on a different set of servers than are hosting the persistence for advanced web service features.

Cluster Routing Scenarios

The following sections illustrate several scenarios for routing web service request and response messages within a clustered environment:

Scenario 1: Routing a Web Service Response to a Single Server

In this scenario, an incoming request is load balanced to a server. Any responses to that request must be routed to that same server, which maintains state information on behalf of the original request.

Figure 23-1 Routing a Web Service Response to a Single Server

Description of Figure 23-1 follows
Description of "Figure 23-1 Routing a Web Service Response to a Single Server"

As shown in the previous figure:

  1. The front-end SOAP router routes an incoming HTTP request and sends it to Server2 using standard load balancing techniques.
  2. Server2 calls Myservice at the web service endpoint address. The ReplyTo header in the SOAP message contains a pointer back to the front-end SOAP router.
  3. MyService returns the response to the front-end SOAP router.
  4. The front-end SOAP router must determine where to route the response. Because Server2 maintains state information that is relevant to the response, the front-end SOAP router routes the response to Server2.

Scenario 2: Routing Web Service Requests to a Single Server Using Routing Information

In this scenario, an incoming request is load balanced to a server. The response contains routing information that targets the original server for any subsequent requests

Figure 23-2 Routing Web Service Requests to a Single Server

Description of Figure 23-2 follows
Description of "Figure 23-2 Routing Web Service Requests to a Single Server"

As shown in the previous figure:

  1. The front-end SOAP router routes an incoming HTTP request (Request1) and sends it to Server 2 using standard load balancing techniques. The request has no routing information.
  2. Server2 calls the Myservice at the web service endpoint address. The ReplyTo header in the SOAP message contains a pointer back to the front-end SOAP router.
  3. MyService returns the response to the caller. The response contains routing information that targets Server2 for any subsequent requests. The caller is responsible for passing the routing information contained in the response in any subsequent requests (for example, Request2).
  4. The front-end SOAP router uses the routing information passed with Request2 to route the request to Server2.

Scenario 3: Routing Web Service Requests to a Single Server Using an ID

In this scenario, an incoming SOAP request contains an identifier, but no routing information. All subsequent requests with the same identifier must go to the same server.

Figure 23-3 Routing Web Service Requests to a Single Server Using an ID

Description of Figure 23-3 follows
Description of "Figure 23-3 Routing Web Service Requests to a Single Server Using an ID"

As shown in the previous figure:

  1. A request comes from a web service client that includes an ID (Make Connection anonymous URI) that will be shared by future requests that are relevant to Request1. The form of this ID is protocol-specific.
  2. The front-end SOAP router detects an ID in Request1 and checks the affinity store to determine if the ID is associated with a particular server in the cluster. In this case, there is no association defined.
  3. The front-end SOAP router load balances the request and sends it to Server 2 for handling.
  4. The MyService web service instance on Server2 handles the request (generating a response, if required). Unlike in Scenario 2: Routing Web Service Requests to a Single Server Using Routing Information, routing information cannot be propagated in this case.
  5. Request2 arrives at the front-end SOAP router using the same ID as that used in Request1.
  6. The front-end SOAP router detects the ID and checks the affinity store to determine if the ID is associated with a particular server. This time, it determines that the ID is mapped to Server2.
  7. Based on the affinity information, the front-end SOAP router routes Request2 to Server2.

How Web Service Cluster Routing Works

The following sections describe how web service cluster routing works:

Adding Routing Information to Outgoing Requests

The web services runtime adds routing information to the SOAP header of any outgoing message to ensure proper routing of messages in the following situations:

  • The request is sent from a web service client that uses a store that is not accessible from every member server in the cluster.

  • The request requires in-memory state information used to process the response.

When processing an outgoing message, the web services runtime:

  • Creates a message ID for the outgoing request, if one has not already been assigned, and stores it in the RelatesTo/MessageID SOAP header using the following format:

    uuid:WLSformat_version:store_name:uniqueID

    Where:

    • format_version specifies the WebLogic Server format version, for example WLS1.

    • store_name specifies the name of the persistent store, which specifies the store in use by the current web service or web service client sending the message. For example, Server1Store. This value may be a system-generated name, if the default persistent store is used, or an empty string if no persistent store is configured.

    • unique_ID specifies the unique message ID. For example: 68d6fc6f85a3c1cb:-2d3b89ab8:12068ad2e60:-7feb

  • Allows other web service components to inject routing information into the message before it is sent.

Detecting Routing Information in Incoming Requests

The SOAP router (in-place or front-end) inspects incoming requests for routing information. In particular, the SOAP router looks for a RelatesTo/MessageID SOAP header to find the name of the persistent store and routes the message back to the server that hosts that persistent store.

In the event that there is an error in determining the correct server using front-end SOAP routing, then the message is sent to any server within the cluster and the in-place SOAP router is used. If in-place SOAP routing fails, then the sender of the message receives a fault on the protocol-specific back channel.

Note:

SOAP message headers that contain routing information must be presented in clear text; they cannot be encrypted.

Routing Requests Within the Cluster

To assist in making a routing determination, the SOAP router (in-place or front-end) uses a dynamic map of store-to-server name associations. This dynamic map originates on the Managed Servers within a cluster and is accessed in memory by the in-place SOAP router and via HTTP response headers by the front-end SOAP router. The HTTP response headers are included automatically by WebLogic Server in every HTTP response sent by a web service in the cluster.

Note:

For more information about the HTTP response headers, see Maintaining the Routing Map on the Front-end SOAP Router.

Initially, the dynamic map is empty. It is only initialized after receiving its first response back from a Managed Server in the cluster. Until it receives back its first response with the HTTP response headers, the front-end SOAP router simply load balances the requests, and the in-place SOAP router routes the request to the appropriate server.

In the absence of SOAP-based routing information, it defers to the base routing that includes HTTP-session based routing backed by simple load balancing (for example, round-robin).

Maintaining the Routing Map on the Front-end SOAP Router

As noted in Routing Requests Within the Cluster, to assist in making a routing determination, the SOAP router (in-place or front-end) uses a dynamic map of store-to-server name associations.

To generate this dynamic map, two new HTTP response headers are provided, as described in the following sections. These headers are included automatically by WebLogic Server in every HTTP response sent by a web service in the cluster.

Note:

When implementing a third-party front-end to include the HTTP response headers described below, clients should send an HTTP request header with the following variable set to any value: X-weblogic-wsee-request-storetoserver-list

X-weblogic-wsee-storetoserver-list HTTP Response Header

A complete list of store-to-server mappings is maintained in the X-weblogic-wsee-storetoserver-list HTTP response header. The front-end SOAP router uses this header to populate a mapping that can be referenced at runtime to route messages.

The X-weblogic-wsee-storetoserver-list HTTP response header has the following format:

storename1:host_server_spec | storename2:host_server_spec | storename3:host_server_spec

In the above:

  • storename specifies the name of the persistent store.

  • host_server_spec is specifies using the following format: servername:host:port:sslport. If not known, the sslport is set to -1.

X-weblogic-wsee-storetoserver-hash HTTP Response Header

A hash mapping of the store-to-server list is provided in X-weblogic-wsee-storetoserver-hash HTTP response header. This header enables you to determine whether the new mapping list needs to be refreshed.

The X-weblogic-wsee-storetoserver-hash HTTP response header contains a String value representing the hash value of the list contained in the X-weblogic-wsee-storetoserver-list HTTP response header. By keeping track of the last entry in the list, it can be determined whether the list needs to be refreshed.

Configuring Web Services in a Cluster

The following table summarizes the steps to configure web services in a cluster.

Table 23-1 Steps to Manage Web Services in a Cluster

# Step Description

1

Set up the WebLogic cluster.

See Setting Up the WebLogic Cluster.

2

Configure the clustered domain resources required for advanced web service features.

You can configure automatically the clustered domain resources required using the cluster extension template script. Alternatively, you can configure the resources using the Oracle WebLogic Server Administration Console or WLST. See Configuring the Domain Resources Required for Web Service Advanced Features in a Clustered Environment.

3

Extend the front-end SOAP router to support web services.

Note: This step is required only if you are using the front-end SOAP router.

The web services routing servlet extends the functionality of the WebLogic HTTP cluster servlet to support routing of web services in a cluster. See Extending the Front-end SOAP Router to Support Web Services.

4

Enable routing of web services atomic transaction messages.

See Enabling Routing of Web Services Atomic Transaction Messages.

5

Enable routing of web services Make Connection messages.

See Enabling Routing of Web Services Make Connection Messages.

6

Configure the identity of the front-end SOAP router.

Each WebLogic Server instance in the cluster must be configured with the address and port of the front-end SOAP router. See Configuring the Identity of the Front-end SOAP Router.

Setting Up the WebLogic Cluster

Set up the WebLogic cluster, as described in Setting up WebLogic Clusters in Administering Clusters for Oracle WebLogic Server. Please note:

Configuring the Domain Resources Required for Web Service Advanced Features in a Clustered Environment

When creating or extending a domain using Configuration Wizard, you can apply the WebLogic Advanced Web Services for JAX-WS Extension template (wls_webservice_jaxws.jar) to configure automatically the resources required to support the advanced web service features in a clustered environment. Although use of this extension template is not required, it makes the configuration of the required resources much easier. Alternatively, you can configure the resources required for these advanced features using the Oracle WebLogic Server Administration Console or WLST.

In addition, the template installs scripts into the domain directory that can be used to manage the resource required for advanced web services in-sync as the domain evolves (for example, servers are added or removed, and so on).

For more information about how to configure the domain and run the scripts to manage resources, see Configuring Your Domain For Advanced Web Services Features.

Extending the Front-end SOAP Router to Support Web Services

Note:

If you are not using the front-end SOAP router, then this step is not required.

You extend the front-end SOAP router to support web services by specifying the RoutingHandlerClassName parameter shown in the following example (in bold), as part of the WebLogic HTTP cluster servlet definition.

<!DOCTYPE web-app PUBLIC
"-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
"http://java.sun.com/dtd/web-app_2_3.dtd">
<web-app>
  <servlet>
    <servlet-name>HttpClusterServlet</servlet-name>
    <servlet-class>weblogic.servlet.proxy.HttpClusterServlet</servlet-class>
    <init-param>
       <param-name>WebLogicCluster</param-name>
       <param-value>Server1:7001|Server2:7001</param-value>
    </init-param>
   <init-param>
      <param-name>RoutingHandlerClassName</param-name>
      <param-value>
         weblogic.wsee.jaxws.cluster.proxy.SOAPRoutingHandler
      </param-value>
    </init-param>
  </servlet>
  <servlet-mapping>
    <servlet-name>HttpClusterServlet</servlet-name>
    <url-pattern>/</url-pattern>
  </servlet-mapping>
. . .
</web-app>

Enabling Routing of Web Services Atomic Transaction Messages

High availability and routing of web services atomic transaction messages is automatically enabled in web service clustered environments. However, if the WebLogic HTTP cluster servlet is being used as the front-end server, you need to set the following system property to false on the server hosting the WebLogic HTTP cluster servlet:

weblogic.wsee.wstx.wsat.deployed=false

In addition, when using a WebLogic Server plugin, you should configure the WLIOTimeoutSecs parameter value appropriately. This parameter defines the amount of time the plug-in waits for a response to a request from WebLogic Server. If the value is less than the time the servlets take to process, then you may see unexpected results. For more information about the WLIOTimeoutSecs parameter, see General Parameters for Web Server Plug-ins in Using Oracle WebLogic Server Proxy Plug-Ins 12.2.1.2.

Enabling Routing of Web Services Make Connection Messages

To support Web Service Make Connection, as described in Using Asynchronous Web Service Clients From Behind a Firewall (Make Connection), you must configure a default logical store on the WebLogic Server that is hosting the WebLogic HTTP cluster servlet. For information about configuring the default logical store, see Configuring the Logical Store.

Configuring the Identity of the Front-end SOAP Router

Each WebLogic Server instance in the cluster must be configured with the address and port of the front-end SOAP router.

You can configure the identity of the front-end SOAP router using one of the following methods, listed in order of precedence:

Configuring the Identity of the Front-end SOAP Router Using Network Channels

Network channels enable you to provide a consistent way to access the front-end address of a cluster. For more information about network channels, see Understanding Network Channels in Administering Server Environments for Oracle WebLogic Server.

To configure the identity of the front-end SOAP router using network channels, for each server instance:

  1. Create a network channel for the protocol you use to invoke the web service. You must name the network channel weblogic-wsee-proxy-channel-XXX, where XXX refers to the protocol. For example, to create a network channel for HTTPS, call it weblogic-wsee-proxy-channel-https.

    See Configure custom network channels in Oracle WebLogic Server Administration Console Online Help for general information about creating a network channel.

  2. Configure the network channel, updating the External Listen Address and External Listen Port fields with the address and port of the proxy server, respectively.

Monitoring Cluster Routing Performance

You can monitor the following cluster routing statistics to evaluate the application performance:

  • Total number of requests and responses.

  • Total number of requests and responses that were routed specifically to the server.

  • Routing failure information, including totals and last occurrence.

You can use the WebLogic Server Administration Console or WLST to monitor cluster routing performance. For information about using WebLogic Server Administration Console to monitor cluster routing performance, see Monitor SOAP web services and Monitor SOAP web service clients in Oracle WebLogic Server Administration Console Online Help.For information about using WLST to monitor cluster routing performance, see Configuring Existing Domains in Understanding the WebLogic Scripting Tool.