Skip Headers
Oracle® Fusion Middleware High Availability Guide
11g Release 1 (11.1.1)

Part Number E10106-11
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Index
Index
Go to Master Index
Master Index
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
View PDF

5 Configuring High Availability for Oracle Fusion Middleware SOA Suite

Oracle Fusion Middleware SOA Suite provides a complete set of service infrastructure components for designing, deploying, and managing composite applications. Oracle Fusion Middleware SOA Suite enables services to be created, managed, and orchestrated into composite applications and business processes. Composites enable you to easily assemble multiple technology components into one SOA composite application. Oracle Fusion Middleware SOA Suite plugs into heterogeneous IT infrastructures and enables enterprises to incrementally adopt SOA.

This chapter provides a description of Oracle SOA Suite components from a high availability perspective. This chapter also includes sections that outline the single-instance concepts for SOA components that are important for designing a high availability deployment.

This chapter includes the following topics:

5.1 Introduction to Oracle Fusion Middleware SOA Suite

As illustrated in Figure 5-1, Oracle SOA Suite provides a comprehensive suite of products for developing, securing, and monitoring service-oriented architecture (SOA). Oracle SOA Suite 11g provides a unified runtime based on the SCA standard. The runtime consists of Service Engines (BPEL Process Manager, Human Workflow, Mediator, Rules) and Binding Components (JCA Adapters, B2B) that are managed and inter-connected by a common Service Infrastructure. Service infrastructure also provides common services such as lifecycle management and deployment.

Figure 5-1 Oracle Fusion Middleware SOA Suite and Components

Description of Figure 5-1 follows
Description of "Figure 5-1 Oracle Fusion Middleware SOA Suite and Components"

A SOA composite application is the basic unit of deployment to the SOA runtime. Service components (BPEL process, business rule, human task, and Mediator routing rule) are the building blocks of a SOA composite. Components are targeted to service engines during deployment while services and references are enabled using the binding components. At runtime, messages are received by the binding component and are then routed to the appropriate service engine(s) by the Service Infrastructure.

Figure 5-2 Oracle SOA Infrastructure Stack Diagram

Oracle SOA Infrastructure Stack Diagram
Description of "Figure 5-2 Oracle SOA Infrastructure Stack Diagram"

The SOA runtime executes within the context of an application server such as the Oracle WebLogic Application Server. It leverages the underlying application server capabilities for load balancing and high availability.

This guide provides high availability information for the following SOA Suite components:

5.2 Oracle SOA Service Infrastructure High Availability

The Oracle SOA Service Infrastructure is a Java EE application that provides the foundation services for running Oracle Fusion Middleware SOA Suite. This Java EE application is a runtime engine that is automatically deployed when Oracle Fusion Middleware SOA Suite is installed. You deploy composites (the basic artifacts in a Software Component Architecture) to the Oracle SOA Infrastructure and it provides the required services for the composites to run. Oracle SOA Infrastructure provides deployment, wiring, and thread management services for the composites. These services sustain the composite's lifecycle and runtime operations.

The information in this section guides you through the issues and considerations necessary for designing a SOA Service Infrastructure high availability cluster. Later sections of this chapter describe the same issues and considerations for the following SOA Service Infrastructure-related components.

Backup and Recovery Considerations

For general information on backing up SOA Service Infrastructure files, see the "Introducing Backup and Recovery" and "Backing Up Your Environment" chapters of the Oracle Fusion Middleware Administrator's Guide.

5.2.1 Oracle SOA Service Infrastructure Single-Instance Characteristics

The Oracle SOA Service Infrastructure is a Java EE-compliant application running on Oracle WebLogic Server. It provides the required services for running composites. A composite is basic unit of deployment for Service Component Architectures (SCA). The SCA Assembly Model is made up of a series of artifacts, which are defined by elements contained in XML files.

Composites are software packages made up of components, wires, services and references. For example, an Oracle BPEL process is a component, an inbound adapter is a service, an outbound adapter is a reference. Wires provide connections between service engines.

Individual components are targeted to specific service engines, such as Oracle BPEL PM, or Oracle Mediator. Oracle SOA Service Infrastructure connects to a SOA database to maintain composite state, and to the SOA Oracle Metadata Services (MDS) Repository to maintain composite metadata, such deployments, and version tracking. These two databases may be the same physical database, but the schemas used for each purpose are different. SOA infra provides a servlet for remote deployment of composites. The metadata and artifacts for remote deployments are also stored in the MDS repository. For more information on the MDS repository, see Section 4.1.2.1, "Configuring Multi Data Sources for MDS Repositories."

The Oracle SOA Service Infrastructure application is also responsible for targeting the individual components to their specific engine and for instantiating these composites when requests reach the SOA system. After targeting and instantiation, the Oracle SOA Service Infrastructure controls thread and resource assignment. This happens in the JVM, where the composite runs.

As illustrated in Figure 5-3, the Oracle SOA Service Infrastructure integrates SOA composite applications with UDDI registries. UDDI registries provide a standards-based foundation for locating published services, invoking services, and managing service metadata. The Oracle SOA Service Infrastructure is also the central hub used by the service engines to deliver messages through Oracle User Messaging Service infrastructure to communication channels, such as email and voice.

SOA Service Infrastructure provides the required services that sustain the different pieces in a Service Component Architecture, and allows the communication between them. For more details of the different components in an SCA system, refer to the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite.

Figure 5-3 Basic Single-Node SOA Service Infrastructure Architecture

Single-Node Oracle SOA Service Infrastructure Architecture
Description of "Figure 5-3 Basic Single-Node SOA Service Infrastructure Architecture"

5.2.1.1 Oracle SOA Service Infrastructure Application Characteristics

Oracle SOA Service Infrastructure Services are contained in the soa-infra-wls.ear file. None of the services provided by the Oracle SOA Service Infrastructure system are singletons, therefore, the Oracle SOA Service Infrastructure can run in full active-active mode. The SOA Service Infrastructure Java EE application contains a Web module that provides browsing of the deployed composites as well as links to the test pages for these composites. This Web module uses /soa-infra as the associated URL context. This Web module is stateless and does not have any specific session replication requirements.

Other modules in the Oracle SOA Service Infrastructure application provide task control for process instantiation and process tracking, as well as client services for accessing User Messaging System (UMS).

A task service controls instantiating and tracking processes asynchronously. In addition, there are multiple EJBs used by the Oracle SOA Service Infrastructure system. However, all of the EJBs are stateless, and there are no requirements for stateful session bean replication in an Oracle SOA cluster. The processing of transactions by these EJBs relies on Oracle WebLogic Server transaction control service. Configure the appropriate transaction stores as recommended in the basic Oracle WebLogic Server guidelines to guarantee recovery across failures in Oracle WebLogic Server container.

5.2.1.2 Oracle SOA Service Infrastructure Startup and Shutdown Lifecyle

An Oracle SOA composite consists of the following:

  • Components such as a BPEL process, Human Workflow task, a Mediator routing rule or Business Rules.

  • Services and References for connecting Oracle SOA composite applications to external services, applications, and technologies.

These components are assembled together into an Oracle SOA composite application. This application is a single unit of deployment that simplifies the management and lifecycle of Oracle SOA applications.

When the Oracle SOA Service Infrastructure application starts, it initializes the different service engines and loads the present composites from the MDS repository. It targets the individual components to their specific engines. Once the composite is loaded, the system is available to receive requests. At runtime, the Oracle SOA Service Infrastructure manages all communication across service components. These calls between service engines are in-process calls. The following diagram reflects the sequence for the Oracle SOA Service Infrastructure startup and processing of work:

Figure 5-4 Oracle SOA Service Infrastructure Application Startup and Shutdown Lifecyle

Startup and Shutdown Lifecyle
Description of "Figure 5-4 Oracle SOA Service Infrastructure Application Startup and Shutdown Lifecyle"

5.2.1.3 Oracle SOA Service Infrastructure External Dependencies

As described in the previous sections, the Oracle SOA Service Infrastructure system depends on the following components:

  • Instance manager service depends on the runtime SOA database schema (soa-infra).

  • Composite metadata is stored in the MDS database schema which acts as a repository.

  • In a clustered environment, the deployment coordinator service depends on the underlying Coherence cluster for signal propagation.

All three of these components must be available for the Oracle SOA Service Infrastructure to start and run properly.

5.2.1.4 Oracle SOA Service Infrastructure Startup and Shut Down of Processes

The Oracle SOA Service Infrastructure application is started by default whenever any Oracle WebLogic Managed Server to which the Oracle SOA Service Infrastructure has been deployed is started. Normally, you should not need to stop the Oracle SOA Service Infrastructure or any of its components by themselves. Some operations may require Oracle WebLogic Managed Server where the SOA Service Infrastructure runs to be rebooted. Only some patching scenarios could require stopping the application.

You can use Oracle WebLogic Server Administration Console to verify status and to start and stop Oracle WebLogic Server. You can also use the WebLogic Server WLST command line to control the application. Oracle Enterprise Manager Fusion Middleware Control also allows multiple operations and configuration of the Oracle SOA Service Infrastructure application as well as monitoring its status. See the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite for details on monitoring and controlling the Oracle SOA Service Infrastructure application.

Figure 5-5 Monitoring and Controlling the Oracle SOA Service Infrastructure Application

Controlling Oracle SOA Service Infrastructure
Description of "Figure 5-5 Monitoring and Controlling the Oracle SOA Service Infrastructure Application "

5.2.1.5 Oracle SOA Service Infrastructure Configuration Artifacts

Starting with Oracle Fusion Middleware 11g Release 1 (11.1.1.2), the configuration parameters for Oracle Service Infrastructure are stored in the SOA MDS database. The main controls of the Oracle SOA Service Infrastructure can be configured using Oracle Enterprise Manager Fusion Middleware Control.

  • The data source JNDI name for process dehydration.

    Note:

    Once the JNDI names are read from this file, the databases used by the system are determined by the data sources that matched those JNDI names in WebLogic Server JDBS resources configuration.
  • The server and callback URL

    A CallBack URL is the address that asynchronous services specify to be notified of a response to the service they invoked.

    Note:

    If a request to an external or internal asynchronous service originates from SOA, the callback URL is determined using the following in decreasing order of preference:
    • Use callbackServerURL specified as a binding property for the specific reference. (You can set this when modeling the composite or at runtime using the MBeans). This allows different service calls to have different callback URLs. At runtime this property is set using the System MBean Browser, through the corresponding binding Mbean. To add a specific URL add a callbackServerURL property to its Properties attribute, then invoke the save operation.

    • Use the callback URL as specified in the SOA Database. In this case, only one address can be specified and an address that works well for all possible services must be used.

    • Use the callback URL as the frontend host specified in WebLogic Server for the SOA_Cluster. In this case as well, only one address can be specified, and the recommendation is the same as for the SOA Database configuration option.

    • Use the local host name as provided by WebLogic Server MBean APIs. This is not recommended in high availability environments.

  • The audit level of information to be collected by the message tracking infrastructure.

Other configuration options at the container level, such as data sources, JTA configuration, and persistent stores, are maintained as part of the WebLogic Server Domain configuration, and are synchronized across a cluster of Oracle WebLogic Servers by the Oracle WebLogic Server core infrastructure.

5.2.1.6 Oracle SOA Service Infrastructure Log File Locations

The operations performed by the Oracle SOA Service Infrastructure and its components are logged by Oracle WebLogic Managed Server where the Oracle SOA Service Infrastructure is running. You can find these logs at the following location:

DOMAIN_HOME/servers/WLS_ServerName/logs/WLS_ServerName.log

The log files for the different Oracle WebLogic Server managed servers are also available from Oracle WebLogic Server Administration Console. To verify the logs, access Oracle WebLogic Server Administration Console using the following URL: admin_server_host:port/console. Click Diagnostics-Log Files.

It is also important to verify the output of the Oracle WebLogic Managed Server where the Oracle SOA Service Infrastructure is running. This information is stored at the following location:

DOMAIN_HOME/servers/WLS_ServerName/logs/WLS_ServerName.out

Additionally, a diagnostic log is produced in the log directory for the managed server. This log's granularity and logging properties can be changed through the domain_dir/config/fmwconfig/servers/WLS_ServerName/logging.xml file. The properties in this file can also be modified from Oracle Enterprise Manager Fusion Middleware Control by selecting Farm, SOA, SOA Server. Right-click, and select Logs, and then Log Configuration.

Oracle Enterprise Manager Fusion Middleware Control allows performing selective searches in all the logs in the SOA domain. To do a selective search, access Oracle Fusion Middleware Control and click on Farm-Logs and enter the search criteria that pertain to soa-infra or deployed composites. See the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite for more details on the logs and information reported for a SOA System in Oracle Enterprise Manager Fusion Middleware Control.

5.2.2 Oracle SOA Service Infrastructure High Availability Architecture and Failover Considerations

Figure 5-6 illustrates a two-node Oracle SOA Service Infrastructure cluster running on two Oracle WebLogic Servers. The Oracle WebLogic Servers are frontended by two Oracle HTTP Server instances on web tier hosts, which receive requests from a load balancer in front of them.

Figure 5-6 Oracle SOA Service Infrastructure High Availability Architecture

Description of Figure 5-6 follows
Description of "Figure 5-6 Oracle SOA Service Infrastructure High Availability Architecture"

Figure 5-6 illustrates the following main characteristics of this high availability configuration:

The SOA Service Infrastructure runs in Oracle WebLogic Server managed servers that are part of an Oracle WebLogic Server cluster. Oracle WebLogic Server cluster synchronizes the configuration for common artifacts of WebLogic Server cluster used by the Oracle SOA Service Infrastructure, for example, JTS configuration, data sources, and persistent store definitions.

The Oracle SOA Service Infrastructure uses the front end host and port information configured for the Oracle WebLogic Server cluster as the server and callback URL. This information is defined using Oracle WebLogic Server Administration Console. To define this information, select Clusters, SOA_Cluster_Name, HTTP/HTTPS frontend host and then Port. If there is no address specified for the Oracle WebLogic Server cluster where the Oracle SOA Service Infrastructure is running, the system uses the physical host name as the server and callback URL.

For SOA high availability installations frontended by Oracle HTTP Server, monitoring should be done on the Listen ports of Oracle HTTP Server. This is the case when a deployment is using all the components deployed to the SOA Managed Server. A simple HTTP monitor that pings the HTTP/HTTPS port and expects a pre-determined response in turn should suffice. If only a specific SOA component is being used (such as B2B), then a monitor that does a deeper level check all the way to the Managed server can be considered to validate the health of the component in use. Please check with your load balancer vendor on setting up the HTTP monitors with your load balancer.

For more information about the server and callback URLs see the Oracle Fusion Middleware Administrator's Guide. Changing the HTTP front end address for a cluster requires a restart of the managed servers that are part of the cluster.

The deployment coordinator is configured and used for deployment and updates of composites. The deployment coordinator sends notifications to the members of the deployment coordinator cluster to retrieve new artifacts from the MDS repository, when they are updated by the group leader. A leader node performs singleton operations for the cluster, such as updating the MDS after deployments, or changes are made to the composites.

The Oracle SOA Service Infrastructure system uses the Oracle WebLogic Server cluster name as its key to confirm a deployment coordinator group. If all nodes in a WebLogic Server cluster can communicate (over multicast or unicast) the deployment coordinator cluster is the same as the WebLogic Server cluster in which the SOA Service Infrastructure runs.

The Administration Server runs in Active-Passive mode. Whenever a failure occurs in SOAHOST1, the Administration Server can be restarted in SOAHOST2. Therefore it uses a virtual IP or virtual hostname as listen address.

For information about configuring virtual IPs for the Administration Server and configuring the Administration Server for high availability, see Chapter 12, "Transforming the Administration Server for Cold Failover Cluster."

Oracle WebLogic Server Whole Server Migration

Although SOA Service Infrastructure can run in full active-active mode, the architecture uses the Oracle WebLogic Server Migration feature to protect some SOA components against failures. This implies that each of the WebLogic Managed Servers in which the SOA Service Infrastructure runs is listening on a Virtual IP that is migrated to another box upon failover. The Administration Server and Enterprise Manager run in AdminServer, not the managed server.

As shown in Figure 5-6, WLS_SOA1 listens on VIP1, and WLS_SOA2 listens on VIP2. Each managed server uses the other node as a failover resource, therefore, the system is configured in a cross manner. In other words, WLS_SOA1 fails over to SOAHOST2, and WLS_SOA2 fails over to SOAHOST1. The appropriate capacity planning must be done to anticipate the scenario where the two SOA managed servers are running on the same node. For more information on Server Migration features, see Chapter 3, "High Availability for WebLogic Server" in this guide.

To resume transactions after a server migration, configure the transaction and JMS stores in a shared storage. In case of failure in one of the server infrastructure instances, other instances can resume transactions and JMS operation by reading the persistent stores from that shared storage.

The Metadata store is configured in an Oracle RAC database to protect from database failures. Similarly, the SOA process state information is also stored in an Oracle RAC database. In this example, both Oracle RAC databases are the same.

About Oracle SOA Service Infrastructure Components

These high availability characteristics apply to most of Oracle SOA components contained in the composite applications deployed across the cluster. For specific two-node high availability characteristics of the individual components, see the specific component sections that follow in this chapter.

5.2.2.1 Oracle SOA Service Infrastructure Protection from Failures and Expected Behavior

This section describes how an Oracle SOA high availability cluster deployment protects components from failure. This section also describes expected behavior in the event of component failure.

The SOA Service Infrastructure system is protected from all process failures by the WebLogic Server infrastructure.

5.2.2.1.1 WebLogic Server Crash

If a WLS_SOAx server crashes, Node Manager attempts to restart it locally. If repeated restarts fail, the WebLogic Server infrastructure attempts to perform a server migration of the WLS_SOAx server to the other node in the cluster. While the failover takes place, the other SOA Service Infrastructure instance becomes the leader for deployments and composite updates and provides the basic services required by the service engines in the system.

Ongoing requests from the HTTP Server time out and new requests are directed to the other WLS_SOAx server. Once the server's restart completes on the other node, Oracle HTTP Server resumes routing any incoming requests to the server. The migrated server reads MDS repository for any updates that might have taken place during restart, and joins the deployment coordinator cluster to listen for new updates. The migrated server also resumes any pending transactions from the transaction logs in shared storage.

In the server migration scenario, the service engines, such as Oracle BPEL PM and Oracle Mediator, are failed over together with the SOA Service Infrastructure. They do not re-issue any requests to the other SOA Service Infrastructure instances by themselves. They resume operations together with the SOA Service Infrastructure once failover is complete.

The Oracle SOA Service Infrastructure application may be down due to failure in accessing resources, errors caused by the deployment coordinator infrastructure, or other issues unrelated to whether the managed server is running. Therefore, Oracle recommends administrators monitoring the soa-infra application for errors caused by the application in the managed server logs. For information about log file locations, see Section 5.2.1.6, "Oracle SOA Service Infrastructure Log File Locations".

5.2.2.1.2 Node Failure

In the event of a node failure, server migration is triggered after the available server verifies the time stamp in the database leasing system. If the failed server was the deployment coordinator cluster master, the available server becomes the new master and the SOA Service Infrastructure remains available for deployment and for composite lifecycle. After the time stamp for leasing is verified, the Node Manager in the node that still remains available attempts to migrate the VIP used by the failed managed server, and restarts it locally. This effectively results in the SOA Service Infrastructure application having two instances running in the same node. For more information on the failover process, see Section 3.9, "Whole Server Migration".

Service engines are deployed to the container as a part of the Service Infrastructure application. These service engines contain all of the ear files and library calls.

5.2.2.1.3 Database Failure

The SOA Service Infrastructure system is protected against failures in the database by using multi data sources. These multi data sources are typically configured during the initial set up of the system (Oracle Fusion Middleware Configuration Wizard allows you to define these multi-pools directly at installation time) and guarantee that when an Oracle RAC database instance fails, the connections are reestablished with available database instances. The multi data source allows you to configure connections to multiple instances in an Oracle RAC database.

For information about multi data source configuration with Oracle RAC and the MDS repository, see Section 4.1.2, "Using Multi Data Sources with Oracle RAC."

5.2.2.2 Oracle SOA Service Infrastructure Cluster-Wide Deployment

As explained in previous sections, composite deployments are stored centrally by the SOA Service Infrastructure in the MDS repository. Each time the SOA Service Infrastructure is started, it synchronizes itself with the MDS repository and SOA store to get the deployment and process state. The deployment coordinator infrastructure orchestrates the notifications for composites deployments and updates. When a new deployment or update takes place, deployment coordinator notifies all members in the cluster. When all members in the cluster confirm that the deployment has succeeded, the master sends a notification to start the composite. If a deployment fails on any one of the nodes, it is rolled back to the rest of the cluster. An error message in the deployment coordinator master (WebLogic Server managed server), indicates the node on which the deployment failed. Figure 5-7 explains this process:

Figure 5-7 Cluster-Wide Deployment of Oracle SOA Composites

Cluster-Wide Deployment of Oracle SOA Composites
Description of "Figure 5-7 Cluster-Wide Deployment of Oracle SOA Composites"

5.2.2.3 Online Redeployment of Oracle SOA Service Infrastructure Composites in a Cluster

When the Oracle SOA Service Infrastructure performs an update or redeployment of a composite, it can overwrite an existing version (x) or create a new version (x+1). All composites are uniquely identified based on the composite name and revision. By default, clients accessing a composite use the version that is identified in the MDS repository as the default version (also visible from Oracle Enterprise Manager Fusion Middleware Control). It is possible to manage the lifecycle of each version separately and perform online redeployments of composites, even with one single instance of SOA Service Infrastructure. The steps for performing this operation would be as follows:

  1. Deploy version x.

  2. Mark version x as default.

  3. Deploy a new version of the composite (x+1) and mark version x as the default (this is done to keep people from accessing the new version in default access).

  4. Test the new version by accessing from a test client by specifying the version of the composite (x+1)

  5. Once verifications are complete, mark x+1 as the default version

  6. New clients are routed to version x+1, while old clients complete their work in version x.

It is possible to leave the previous version deployed, or undeploy it. If you undeploy a composite which has in-flight instances, they become stale and do not complete. The un-deployment of a composite removes the composite from the MDS repository.

5.2.2.4 Oracle SOA Service Infrastructure Cluster-Wide Configuration Changes

The standard Java EE artifacts that the Oracle SOA Service Infrastructure uses are configured as part of Oracle WebLogic Domain in which SOA is installed. Oracle WebLogic Clusters provide automatic configuration synchronization for artifacts such as data sources, persistent stores, and JMS modules, across the WebLogic Server domain. At the same time, the WebLogic Server cluster controls synchronization of the deployments and libraries used by the SOA Service Infrastructure.

As explained in Section 5.2.1, "Oracle SOA Service Infrastructure Single-Instance Characteristics," SOA Service Infrastructure-specific configuration is stored in the SOA database. Changes are applied once (per SOA server), but affect all SOA servers in the same SOA domain. For example, in the high availability topology described in this chapter, if you change the callback URL or audit level for server WLS_SOA1, the change also applies to WLS_SOA2.

Note:

Changes to properties of a Composite from the Oracle Enterprise Manager Fusion Middleware Control using the MBean browser are also reflected in all the nodes of a SOA cluster. For example, if you change the Mbean by selecting oracle.soa.config node, server:WLS_SOA1(clustered node), SCAComposite, My Composite, this change propagates to the other servers in the cluster.

5.3 Oracle BPEL Process Manager and High Availability Concepts

The information in this section guides you through the issues and considerations necessary for configuring Oracle BPEL PM for high availability.

5.3.1 Oracle BPEL Process Manager Single-Instance Characteristics

Service engines are containers that host the business logic of service components in a SOA composite application. Each service component, such as Oracle BPEL PM, Oracle Human Workflow, Decision Service, or Oracle Mediator, is executed in its own service engine (Decision Service executes in the business rules service engine). A service engine plugs into the Oracle SOA Service Infrastructure. Oracle BPEL Process engine is the service engine running in the Oracle SOA Service Infrastructure that allows the execution of BPEL Processes.

A BPEL process provides the standard for assembling a set of discrete services into an end-to-end process flow, and developing synchronous and asynchronous services into end-to-end BPEL process flows. It provides process orchestration and storage of long running, asynchronous processes.

As illustrated in Figure 5-8, the BPEL engine is a stateless part of the Oracle SOA Service Infrastructure. It is started with the Oracle SOA Service Infrastructure application and contains different modules that take care of different aspects required to execute BPEL processes. If a deployed composite contains a BPEL process, SOA Service Infrastructure invokes BPEL Process Manager to get the component from the MetaDataStore (MDS).

BPEL Process Manager uses a Dispatcher Module that maintains an in-memory logical queue containing units of work to process the incoming messages from binding components (JMS, database, Web server).

The BPEL Process Manager engine saves processes' execution state in the SOA database through a persistence module based on Oracle TopLink. The Audit framework continuously audits the work being processed by storing process execution information in the SOA database.

Figure 5-8 Oracle BPEL PM Single-Instance Architecture

BPEL PM Single-Instance Architecture
Description of "Figure 5-8 Oracle BPEL PM Single-Instance Architecture"

5.3.1.1 BPEL Process Manager Component Characteristics

The BPEL Service Engine runs inside the SOA Service Infrastructure Java EE application (soa-infra.ear). The BPEL Service Engine does not have any singleton services. All state associated with a BPEL process is stored in a database (dehydration store) and there is no in-memory state replication required.

The processing of work by SOA Service Infrastructure Java EE EJBs is transactional and relies on Oracle WebLogic Service transaction control service. You configure the appropriate transaction stores as recommended in the basic WebLogic Server guidelines to guarantee recovery across failures in the WebLogic Server container. Additionally the BPEL engine system does not contain any Web modules, therefore, session replication is not required in the servlet layer when running BPEL in active-active.

In this release of Oracle Fusion Middleware, BPEL Service Container does not rely on JMS for asynchronous message dispatching. Therefore, there is no dependency on a distributed JMS infra-structure.

External Dependencies

The BPEL engine system depends on the following components:

  • SOA Service Infrastructure database for BPEL process state persistence

  • MDS repository for BPEL process metadata store

Both components must be available for the BPEL engine system to start and run properly.

5.3.1.2 Oracle BPEL Process Manager Startup and Shutdown Lifecycle

As illustrated in Figure 5-9, when the Oracle SOA Service Infrastructure application starts, it initializes the BPEL engine and loads the composites from the MDS repository. If the composite contains any BPEL processes, it targets those individual components to the BPEL engine. Once the process is loaded, the system is available to receive requests. At runtime, the BPEL engine waits for requests from different channels, such as JMS, the database, and HTTP.

Figure 5-9 Startup and Shutdown Lifecycle of Oracle BPEL PM

Startup and Shutdown Lifecycle
Description of "Figure 5-9 Startup and Shutdown Lifecycle of Oracle BPEL PM "

A detailed startup and shutdown lifecycle is as follows:

  1. Start SOA Server.

  2. Start BPEL Engine.

  3. Composites are loaded from MDS repository by SOA Service Infrastructure.

  4. BPEL components are dispatched to the BPEL engine to be loaded.

  5. Composite-binding components are activated.

  6. The BPEL engine services requests.

  7. The shutdown signal is received by SOA Service Infrastructure.

  8. SOA Service Infrastructure starts undeploying loaded composites.

  9. Composite-binding components are disabled.

  10. BPEL components are dispatched to the BPEL engine to be unloaded.

  11. The BPEL engine shuts down.

5.3.1.3 Oracle BPEL Process Manager Request Flow and Recovery

Recoverable activities are activities that have failed and can be recovered. For example, if you are using the file adapter to initiate an asynchronous BPEL process and your system crashes while the instance is processing, you can manually perform recovery when the server restarts to ensure that all message records are recovered.

There are 2 types of BPEL Processes based on the invocation interface:

  • One-Way: Most commonly asynchronous fire-and-forget pattern.

  • Two-Way: Synchronous request-response pattern.

The recovery semantics after a server failure are based on whether the process is invoked synchronously and asynchronously. The following describes the behavior based on invocation and process type:

  • Synchronous Invocation (Request): For synchronous requests, an error is thrown back to the client in the case of a server failure. It is the client's responsibility to handle the error message and take appropriate action such as retrying the request. This holds true for both Transient and Durable processes.

    Note:

    In the case of durable processes, the message is persisted to the dehydration store at certain points. It is possible to recover the message from the dehydration store and replay the process using Enterprise Manager, however, the client cannot be notified of the response. Therefore, this is not a recommended option. It is preferable to handle all recovery from the client.
  • Asynchronous Invocation (Post): There are two types of asynchronous invocations - one that starts a new process and one that is a callback to an existing process. In the case of a callback, the engine (after recognizing that it is a continue operation) attempts to resolve the subscribing process, first by conversation ID, and then by a correlation set if defined. Messages associated with asynchronous requests are persisted to the dehydration store as part of the client call. If there is a failure prior to the persistence, the client receives an error message and has to handle the error in the same way as an error for a synchronous invocation. If the persistence is successful, the client call returns and further processing is done outside the context of the client call. In the case of a server failure, one-way processes that were invoked asynchronously can be restarted using Enterprise Manager.

Figure 5-10 Enterprise Manager BPEL Engine Recovery

Enterprise Manager BPEL Engine Recovery
Description of "Figure 5-10 Enterprise Manager BPEL Engine Recovery"

See the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and the Oracle Fusion Middleware Developer's Guide for Oracle SOA Suite for details on the synchronous and asynchronous models supported by BPEL engine.

5.3.1.4 Oracle BPEL Process Manager Configuration Artifacts

Starting with Oracle Fusion Middleware 11g Release 1 (11.1.1.2), the configuration parameters for Oracle BPEL PM are stored in the SOA database. These parameters are configurable from Oracle Enterprise Manager Fusion Middleware Control.

To configure these parameters, go to the Oracle Enterprise Manager Fusion Middleware Control, and in the navigation tree, select SOA, soa-infra, (server_name). Right-click SOA Administration, and select BPEL Properties, or use the MBean Browser to navigate to the appropriate property.

These properties are specific to each WebLogic domain directory that contains WebLogic Servers running Oracle Fusion Middleware SOA. Other configuration options at the container level, such as data sources, JTA configuration, and persistent stores are maintained as part of the WebLogic Server Domain configuration, and are synchronized across a cluster of WebLogic Servers by the WebLogic Server core infrastructure. See the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite for more details on the configuration for the BPEL engine.

Figure 5-11 shows the Oracle WebLogic Server Administration Console where you can configure Oracle BPEL PM properties

Figure 5-11 Oracle BPEL PM Configuration Properties

Oracle BPEL PM Configuration Properties
Description of "Figure 5-11 Oracle BPEL PM Configuration Properties"

If you are using Oracle SOA Suite in a clustered environment, any configuration property changes you make in Oracle Enterprise Manager on one node must be made on all nodes. Configuration properties are set in Oracle Enterprise Manager through the following options of the SOA Infrastructure menu:

Administration > System MBean Browser

SOA Administration > any property selections.

5.3.2 Oracle BPEL Process Manager High Availability Architecture and Failover Considerations

Figure 5-6 describes an Oracle SOA Service Infrastructure two-node cluster running on two WebLogic Servers. Oracle BPEL PM is deployed as part of the Oracle SOA Service infrastructure.

5.3.2.1 Oracle BPEL Process Manager Protection from Failures and Expected Behavior

For information about Oracle SOA Service Infrastructure protection from failures and expected behavior, see Section 5.2.2.1, "Oracle SOA Service Infrastructure Protection from Failures and Expected Behavior".

The BPEL engine system is protected from all process failures by the WebLogic Server infrastructure. The following process failure considerations apply to Oracle BPEL:

  • If the managed servers crash, Node Manager attempts to restart them locally. If whole server migration is configured and repeated restarts fail, the WebLogic Server infrastructure attempts to perform server migration of the managed server to the other node in the cluster, if it is configured. Once the server on the other node is restarted, Oracle HTTP Server resumes routing any incoming requests to it. The migrated server reads the SOA database, resumes any pending processing, and resume transactions from the transaction logs in shared storage.

  • The BPEL PM Service Engine or the entire SOA Service Infrastructure may be unavailable because of errors related to JDBC data sources, or Coherence configuration, even after a successful managed server startup. Therefore, it is necessary to monitor SOA Service Infrastructure logs for errors. For the location of server logs, see Section 5.2.1.6, "Oracle SOA Service Infrastructure Log File Locations".

In addition, for the handling of failures in the BPEL Engine itself with the WebLogic Server infrastructure, you can define and perform fault recovery actions on BPEL process faults identified as recoverable in Oracle Enterprise Manager Fusion Middleware Control. The recovery actions you perform on faults are based on actions you defined in your fault recovery policy files for BPEL process service components.

Three types of faults can be displayed in Oracle Enterprise Manager Fusion Middleware Control:

  • Business: Application-specific faults that are generated when there is a problem with the information being processed (for example, a social security number is not found in the database).

  • System: Network and other types of errors, such as a database server being completely unavailable, or a Web service being unreachable.

  • Oracle Web Services Manager (OWSM): Errors on policies attached to SOA composite applications, service components, or binding components.

Fault recovery policies are defined at design time. The fault policies files included with the composites define the actions to take should a failure occur. For more information on how to define and use Fault recovery, see the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite.

Missing Instances

The inbound payload for one-way invocation requests is stored in the dehydration store and committed with the current global transaction context. If the caller has already started a global transaction, the invocation message is saved when the caller commits the transaction; if no incoming transaction is present, a new transaction is started to commit the invocation message. If a BPEL instance cannot be found in Enterprise Manager, the invocation message may be present in the manual recovery console (the recovery console lists all of the incoming messages that have not been delivered yet). If the invocation message is not found in the recovery console, check the status of the transaction from the caller side.

Transactional Issues with Endpoints

BPEL asynchronously saves the audit trail for instances that cannot be dehydrated within the global transaction in which the request was started. If any transactional services that are referenced from a BPEL component happen to roll back the transaction, the BPEL instance audit trail should still appear in Enterprise Manager. If there is a problem with the dehydration store, the entire global transaction is rolled back and no audit trail is saved. In this case, a message or activity can be recovered to pick up processing from the last known dehydrated point.

If the dehydration point is an Oracle RAC database with multiple nodes and the Java EE server on which the BPEL engine is hosted loses its JDBC connection to the Oracle RAC node, the BPEL engine attempts to retry the transaction (thereby rolling back whatever changes were made in the current transaction). If a new connection cannot be established to a new Oracle RAC node, the BPEL message or activity can be recovered using the recovery console.

Logging

BPEL engine loggers are set to INFO level by default. When trying to track down a message or instance, you may find it helpful to set loggers to TRACE level to enable more output to the WebLogic Server logs. For dehydration log messages, enable oracle.soa.bpel.engine.data. For thread or dispatcher log messages, enable oracle.soa.bpel.engine.dispatcher. To capture all engine log messages, enable oracle.soa.bpel.engine.

5.3.2.1.1 Recovering Failed BPEL and Mediator Instances

In the case of a server failure, in-flight one-way processes that were invoked asynchronously may require manual recovery. Such processes are marked as Recoverable and can be restarted using Enterprise Manager as shown in Figure 5-12.

Figure 5-12 BPEL PM Instance Recovery using Oracle Enterprise Manager

BPEL PM Instance Recovery using Oracle Enterprise Manager
Description of "Figure 5-12 BPEL PM Instance Recovery using Oracle Enterprise Manager"

5.3.2.2 Oracle BPEL Process Manager Cluster-Wide Configuration Changes

The standard Java EE artifacts that BPEL engine uses are configured as part of Oracle WebLogic Domain in which SOA is installed. Oracle WebLogic Clusters provide automatic configuration synchronization for artifacts, such as data sources, persistent stores, and JMS modules across the WebLogic Server domain.

As explained in the Section 5.3.1, "Oracle BPEL Process Manager Single-Instance Characteristics," BPEL engine-specific configuration is stored in the SOA database. Changes are applied once (per SOA server) but affect all BPEL instances in the same SOA domain. For example, If WLS_SOA1 and WLS_SOA2 are part of the same SOA cluster and the number of Dispatcher threads for BPEL in WLS_SOA1 is changed to 20, WLS_SOA2's dispatcher threads are set to 20 as well.

5.4 Oracle BPM Suite and High Availability Concepts

Oracle Fusion Middleware BPM Suite provides a complete set of components for designing, deploying, and managing Business Process Management (BPM) projects.

5.4.1 Oracle BPM Suite Single Instance Concepts

As shown in Figure 5-13, Oracle BPM Suite provides a comprehensive suite of products for developing, managing and monitoring BPM Projects. Based on Oracle SOA Suite 11g, the complete set of technologies and standards like SCA can be leveraged by the BPM Suite 11g. The BPM runtime consists of service engines (BPMN Service Engine, BPEL Process Manager, Human Workflow, Rules, Mediator) and binding components (JCA Adapters, B2B) that are managed and interconnected by a common Service Infrastructure. The Service Infrastructure also provides common services for lifecycle management and deployment of BPM projects.

Figure 5-13 BPM Suite Single-Instance Architecture

Description of Figure 5-13 follows
Description of "Figure 5-13 BPM Suite Single-Instance Architecture"

A BPM project is the basic unit of deployment to the BPM runtime. A BPM project is comprised of service components (such as BPMN Process, Business Rules, Human Workflow) and references organized as a SOA Composite, organizational data (such as Roles and Business Calendars), business indicator metadata and dashboard data.

Components are targeted to service engines during deployment while services and references are enabled using the binding components. The metadata for organizational data, business indicators and dashboards is persisted and evaluated by appropriate components at runtime. At runtime, messages are received by the binding component or the BPM Workspace and are then routed to the appropriate service engine(s) by the Service Infrastructure.

Figure 5-14 shows the Oracle BPM Suite Infrastructure stack diagram:

Figure 5-14 BPM Suite Infrastructure Stack Diagram

Description of Figure 5-14 follows
Description of "Figure 5-14 BPM Suite Infrastructure Stack Diagram"

The BPM runtime executes within the context of an application server such as the Oracle WebLogic Server. It leverages the underlying application server capabilities for load balancing and high availability.

5.4.1.1 Oracle BPM Suite Component Characteristics

Figure 5-15 shows the components in BPM Suite, their relationship, and the technologies they use to interface with each other.

Figure 5-15 BPM Suite Components and Interfaces

Description of Figure 5-15 follows
Description of "Figure 5-15 BPM Suite Components and Interfaces"

The BPM Suite components are:

  • BPMN Service Engine:

    BPMN is an abbreviation for Business Process Modeling Notation. The BPMN Service Engine is an extension of the existing BPEL Service Engine and as such it leverages the core infrastructure of the BPEL. The BPMN Service Engine leverages JPA/EclipseLink to store/recover the state of a process instance in the SOA Infrastructure dehydration store maintained by a database and to persist audit records that are created in the course of running a process. MDS APIs are used to retrieve metadata Information about the BPMN Process Model and other BPM project artifacts like the Business Catalog.

  • BPM Composer:

    The BPM Composer is a standard J2EE Web Application. It is accessed by the end-user from a browser using HTTP protocol. The BPM Composer leverages the BPM infrastructure libraries. It uses the Project Metadata Layer to create and retrieve artifacts of a BPM project (like a business rule or BPMN process) from MDS.

  • BPM Workspace:

    The BPM Workspace is a standard J2EE web application. It is accessed by the end-user from a browser using HTTP protocol. The BPM Workspace uses APIs of the Oracle Business Process Interface (OBPI) to retrieve information about BPMN process instances and user tasks for display in the Worklist. Furthermore, it leverages the Cube Persistence APIs for standard and custom dashboards.

  • BPM Infrastructure Libraries:

    The most important BPM Infrastructure Libraries are:

    • OBPI:

      The Oracle Business Process Interface provides functionality to access information about process instances, user tasks, and so on. It is deployed as a library in WebLogic Server and exposes an EJB interface as well as Spring Beans for dependency injection. The OBPI leverages existing SOA Services such as Human Workflow Service, Identity Service, and Facade API. OBPI is the primary interface for clients that want to access the BPM Infrastructure.

    • PML:

      The Project Metadata Layer is an internal API for managing BPM projects. It leverages MDS for storage, retrieval and labeling of BPM projects and is used by BPM Composer and the BPMN Service Engine

    • Analytics:

      The Analytics library provides a set of functionality to manage Process Cubes. The library exposes a EJB3 (stateless) interface and uses JPA/EclipseLink for persistence. This library is used by BPM Workspace.

    Note:

    Since Oracle BPMN Service Engine is the core piece of BPM Suite, special attention and description of it is provided in the following sections.

5.4.1.2 Oracle BPM Suite Component Interaction

This section uses Table 5-1 to show how the BPM Suite components interact and the tasks that can be performed by each component during the execution of a BPMN process.

After the heading row in Table 5-1, each row in the table is for a particular BPM Suite component. After the first column in Table 5-1, each column heading represents a point in time during the execution of the BPMN process. T1 is the first point in time, T2 is the next point in time, and so on. T8 is the last point in time for the process execution. The tasks that can be performed by each component are shown in the row for that component at the point in time that the task is performed during the BPMN process execution.

Table 5-1 Tasks Performed by BPM Suite Components During BPMN Process Execution

Component T1 T2 T3 T4 T5 T6 T7 T8

BPM Workspace

 

Instantiate process instance using Initiate Task

 

Approve user task

 

Monitor task performance and workload

 

Monitor process performance

BPM Composer

Deploy BPM project

             

BPMN Service Engine

   

Execute process instance

 

Continue execution of process instance

 

Complete process instance

 

BPM Studio

Deploy BPM project

             

Web Service (WS) Client

 

Instantiate process instance invoking composite service

         

Return result


The following list provides additional details about the eight points in time (T1 through T8) during the BPMN process execution shown in Table 5-1:

  • T1:

    The example in Table 5-1 assumes that a BPM project is available either in BPM Studio or BPM Composer and is ready for deployment. As part of deploying a BPM project from BPM Studio or BPM Composer, an (SCA) archive is created and transferred to the standard SOA Composite Deployer Servlet running in the WebLogic Server. The composite deployer servlet stores the content of the BPM Project in MDS and notifies the appropriate service engines that a new BPM Project has been deployed. After that, the BPMN Service Engine is ready to receive requests for processing.

  • T2:

    There are two ways to initiate a BPMN process:

    • Using an Initiate Task:

      If the user has modeled a Initiate Task in the BPMN process diagram to start the process instance, then after deployment the task would become visible in BPM Workspace and the process instance can be started from here.

    • Invocation of the Composite Service using any Web Service (WS client) Using an Initiate Task:

      In the case where the BPMN Process exposes a service interface, that service is exposed as a composite service and as such any WS client can be used to instantiate the process instance.

  • T3:

    The BPMN Service Engine starts executing the process and continues executing it until the first User Task is executed. For a User Task, the BPMN Service Engine creates a Human Task using OBPI. As a result, a new task is created in the SOA Infrastructure dehydration store.

  • T4:

    BPM Workspace uses OBPI to query new tasks for display in the Worklist. The user logged into BPM Workspace is supposed to work on the task. Eventually the task is approved or rejected and work on it has completed. BPM Workspace uses OBPI to complete the task.

  • T5:

    When the task is completed, that also completes the User Task Activity in the BPMN Process, and the process is able to continue.

  • T6:

    At any time during the execution of the process instance, a user might use BPM Workspace to monitor the task performance and workload. For this, BPM Workspace leverages the Cube Persistence API to query task performance data from the Process Cubes stored in the database.

  • T7:

    Eventually the BPMN process is completed.

  • T8:

    Typically, the last step in a BPMN process is to return a result (via a Message End Event) to the client that invoked the process. When the process has completed, a user may use BPM Workspace to monitor the process performance.

5.4.1.3 Oracle BPM Suite Startup and Shutdown Lifecycle

For the BPMN Service Engine, the startup and shutdown lifecycle is the same as for the BPEL Service Engine, since the BPMN Service Engine is based on the BPEL Service Engine. For more information, see Section 5.3.1.2, "Oracle BPEL Process Manager Startup and Shutdown Lifecycle." and Section 5.4.2.1.3, "Oracle BPMN Service Engine Startup and Shutdown Lifecycle."

For BPM Composer and BPM Workspace, the lifecycle is the same as for any web application residing in Oracle WebLogic Server.

The lifecycle of the BPM Infrastructure Libraries is the lifecycle for the system that uses and makes invocations to the libraries.

5.4.1.4 Oracle BPM Suite Configuration Artifacts

To configure the BPMN Service Engine in Enterprise Manager, from the SOA Infrastructure list in soa-infra choose SOA Administration > BPMN Properties, as shown in Figure 5-16.

Figure 5-16 Displaying the BPMN Service Engine Properties Page

Description of Figure 5-16 follows
Description of "Figure 5-16 Displaying the BPMN Service Engine Properties Page"

This displays the BPMN Service Engine Properties page shown in Figure 5-17. Additional parameters are available from the System MBean browser. Click More BPMN Configuration Properties... on the BPMN Service Engine Properties page to launch the BPMN MBean browser.

Figure 5-17 Launching the BPMN MBean Browser

Description of Figure 5-17 follows
Description of "Figure 5-17 Launching the BPMN MBean Browser"

For detailed information on configuring the Oracle BPMN service engine using Enterprise Manager, see Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite.

For detailed instructions on configuring high availability for the Oracle BPMN service engine, refer to Section 5.13, "Configuring High Availability for Oracle SOA Service Infrastructure and Component Service Engines."

5.4.2 Oracle BPMN Service Engine High Availability

This section provides single instance and high availability information for Oracle BPMN service engine.

5.4.2.1 Oracle BPMN Service Engine Single Instance Characteristics

Service engines are containers that host the business logic of service components in a BPM project. Each service component, such as Oracle BPMN Service Engine, Oracle BPEL Process Manager, Oracle Human Workflow, Oracle Business Rules, or Oracle Mediator is executed in its own service engine. A service engine plugs into the Oracle SOA Service Infrastructure.

Oracle BPMN service engine is the service engine running in SOA Service Infrastructure that allows the execution of BPMN processes.

A BPMN process provides the standard for assembling a business process using standardized activities, gateways and events in a well-defined process flow. The BPMN service engine provides functionality for the execution of potentially long-running BPMN process models. It leverages the core-infrastructure features of the Oracle BPEL Process Manager such as dehydration, dispatching, and service orchestration.

5.4.2.1.1 Oracle BPMN Service Engine Single Instance Architecture

As shown in Figure 5-18, The BPMN service engine is a stateless part of the Oracle SOA Service Infrastructure that builds on top of the Oracle BPEL Process Manager Service Engine. For detailed information about Oracle BPEL Process Manager and high availability see Section 5.3, "Oracle BPEL Process Manager and High Availability Concepts."

Figure 5-18 Oracle BPMN Service Engine Single-Instance Architecture

Description of Figure 5-18 follows
Description of "Figure 5-18 Oracle BPMN Service Engine Single-Instance Architecture"

The BPMN service engine leverages the BPEL PM dispatcher module to dispatch incoming messages from binding components (JMS, database, Web Services) for processing.

The state of process execution is saved in the SOA database through a persistence module based on Java Persistence Architecture (JPA). The auditing infrastructure of the BPMN service engine continuously audits the work being processed by the engine and stores audit records in the BPM database. Those audit records are used as a source for measurements and integration with Oracle Business Activity Monitoring (BAM).

The BPMN service engine runs inside the SOA Service Infrastructure Java EE application (soa-infra.ear). Since it is built on top of the BPEL Process Manager, it has identical characteristics. See Section 5.3.1.1, "BPEL Process Manager Component Characteristics" for more information.

5.4.2.1.2 Oracle BPMN Service Engine External Dependencies

The BPMN service engine depends on the following components:

  • SOA Service Infrastructure database for BPMN process state persistence

  • BPM database for persistence of analytics data

    By default, the BPM database is collocated with the SOA Service Infrastructure database and requires no additional setup.

  • MDS repository for BPMN process metadata store

The components in Figure 5-18 must be available for the BPMN service engine to start and run properly.

Depending on the BPM project, the BPMN service engine might depend on the following additional components:

  • BAM adapter

  • User directory. For example, Oracle Internet Directory or another LDAP server that is configured to work with BPM.

5.4.2.1.3 Oracle BPMN Service Engine Startup and Shutdown Lifecycle

As shown in Figure 5-19, when the Oracle SOA Service Infrastructure application starts, it initializes the BPMN service engine and loads the composites from the MDS repository. If the composite contains any BPMN processes, it targets those individual components to the BPMN service engine. Once the process is loaded and its BPM specific metadata persisted in the database, the system is available to receive requests.

Figure 5-19 Startup and Shutdown Lifecycle of Oracle BPMN Service Engine

Description of Figure 5-19 follows
Description of "Figure 5-19 Startup and Shutdown Lifecycle of Oracle BPMN Service Engine"

The detailed startup and shutdown lifecycle is:

  1. Start BPM Server.

  2. Start BPMN service engine.

  3. BPM project composites are loaded from the MDS repository by the SOA Service Infrastructure.

  4. BPMN components are dispatched to the BPMN service engine to be loaded.

  5. The BPM project metadata, such as organization data and audit/measurement metadata, is persisted in the infrastructure database.

  6. Composite binding components are activated.

  7. The BPMN engine services requests.

  8. The shutdown signal is received by the SOA Service Infrastructure.

  9. The SOA Service Infrastructure starts unloading composites.

  10. Composite binding components are disabled.

  11. BPMN components are dispatched to the BPMN engine to be unloaded.

  12. The BPMN service engine shuts down.

5.4.2.1.4 Oracle BPMN Service Engine Log Files

BPMN service engine loggers are set to INFO by default. When trying to track down a message or instance, you may want to set the loggers to TRACE level to enable more output to the WebLogic Server log files.

The BPMN service engine logs trace information to the same files as the SOA Service Infrastructure. For the location of the server logs, see Section 5.2.1.6, "Oracle SOA Service Infrastructure Log File Locations."

5.4.2.2 Oracle BPMN Service Engine High Availability Considerations

This section guides you through the issues and considerations necessary for configuring Oracle BPMN Process Manager for high availability.

5.4.2.2.1 Oracle BPMN Service Engine High Availability Architecture and Failover Considerations

The BPMN service engine leverages the functionality of BPEL PM for process instance recovery. See Section 5.3.2.1, "Oracle BPEL Process Manager Protection from Failures and Expected Behavior" for more details on instance recovery.

In Oracle Enterprise Manager Fusion Middleware Control, select BPMN Engine (Service Engine) for the administration of the BPMN service engine, as shown in Figure 5-20.

Figure 5-20 BPMN Service Engine Home Page in Enterprise Manager

Description of Figure 5-20 follows
Description of "Figure 5-20 BPMN Service Engine Home Page in Enterprise Manager"

The home page of the BPMN service engine is comprised of several sub pages. The Dashboard page is the default page. For the recovery of BPMN service instances, select the Recovery tab, which opens the BPMN Service Engine Recovery page shown in Figure 5-21.

Figure 5-21 BPMN Service Recovery Page in Enterprise Manager

Description of Figure 5-21 follows
Description of "Figure 5-21 BPMN Service Recovery Page in Enterprise Manager"

5.4.2.2.2 Configuring Oracle BPMN Service Engine for High Availability

For detailed instructions on configuring high availability for Oracle BPMN Service Engine, refer to Section 5.13, "Configuring High Availability for Oracle SOA Service Infrastructure and Component Service Engines."

5.4.2.2.3 Cluster-Wide Configuration Changes for Oracle BPMN Service Engine

The standard Java EE artifacts that Oracle BPMN components use are configured as part of Oracle WebLogic's domain in which SOA is installed. Oracle WebLogic Clusters provide automatic configuration synchronization for artifacts, such as data sources, persistent stores, and JMS modules across the WebLogic Server domain. At the same time, the WebLogic Server cluster is in charge of synchronizing the deployments and libraries used by the Oracle BPMN components. As explained in Section 5.4.1.4, "Oracle BPM Suite Configuration Artifacts," the Oracle BPMN components' configuration is stored in the SOA MDS database. Changes are applied once (per SOA server), but affect all Oracle BPMN instances in the same SOA domain.

5.4.3 Oracle Business Process Web Applications High Availability

This section provides single instance and high availability information for Oracle Business Process web applications.

5.4.3.1 Oracle Business Process Web Applications Single Instance Characteristics

This section provides single instance information for Oracle Business Process web applications.

5.4.3.1.1 Oracle Business Process Web Applications Single Instance Architecture

Figure 5-22 shows Oracle BPM Suite web applications and their interactions with other BPMN components.

Figure 5-22 Oracle BPM Suite Web Applications

Description of Figure 5-22 follows
Description of "Figure 5-22 Oracle BPM Suite Web Applications"

The web applications are stateless. The state is maintained in the Dehydration Service and the data is exposed through the Operational APIs or in MDS.

5.4.3.1.2 Oracle Business Process Web Applications External Dependencies

Both BPM Workspace and BPM Composer are deployed separately from the BPMN Service Engine, but depend on it completely. If the BPMN Service Engine is stopped (soa-infra system), neither BPM Workspace nor BPM Composer will be able to access information and metadata about deployed projects. Given the dependency of soa-infra on the SOA MDS database, by extension the Business Process web applications depend on the MDS database to be available.

5.4.3.1.3 Oracle Business Process Web Applications Startup and Shutdown Lifecycle

BPM Workspace and BPM Composer are standard J2EE applications. They are started when the WebLogic Server where BPM has been deployed is started. They can be controlled from the WebLogic Server Administration Console and can be stopped with a forced shutdown or a graceful shutdown.

5.4.3.1.4 Oracle Business Process Web Applications Log Files

The Workspace and Composer web applications write to the SOA WebLogic Server log file and SOA WebLogic Server output. For the location of the server logs, refer to Section 5.2.1.6, "Oracle SOA Service Infrastructure Log File Locations."

Oracle Enterprise Manager Fusion Middleware Control can be used for diagnosing log file messages.

5.4.3.2 Oracle Business Process Web Applications High Availability Considerations

This section describes high availability considerations for Oracle Business Process web applications.

5.4.3.2.1 Oracle Business Process Web Applications High Availability Architecture and Failover Considerations

Both BPM Workspace and BPM Composer are stateless web applications. When the WebLogic Servers they reside on are deployed behind a load balancer or an HTTP Server, the front end devices route requests indistinctly to either node on which the applications are running. When a node failure occurs, requests are redirected to the other available WebLogic Server and work on the user interface can continue without interruptions.

5.4.3.2.2 Configuring Oracle Business Process Web Applications for High Availability

For detailed instructions on configuring high availability for Oracle Business Process web applications, refer to Section 5.13, "Configuring High Availability for Oracle SOA Service Infrastructure and Component Service Engines." These web applications are clustered in a stateless cluster as part of the configuration for Oracle SOA Service infrastructure.

5.4.3.2.3 Cluster-Wide Configuration Changes for Oracle Business Process Web Applications

There are no specific configuration files for the web applications that reside locally on the file system for the servers. Some property files, like the Workspace property file, workspace.properties, are part of the OracleBPMWorkspace.ear file but do not include any instance-specific settings. They are deployed and available wherever Oracle BPM Workspace is deployed.

5.4.4 Oracle Business Process Analytics High Availability

This section provides single instance and high availability information for Oracle Business Process Analytics.

5.4.4.1 Oracle Business Process Analytics Single Instance Characteristics

The following section provides information about Oracle Business Process Analytics single instance characteristics.

5.4.4.1.1 Oracle Business Process Analytics Single Instance Architecture

The Oracle BPM Suite has built-in analytical capabilities for support of business-friendly process dashboards and real-time monitoring of business processes using Oracle Business Activity Monitoring (BAM).

Figure 5-23 shows the Oracle BPM Suite Analytical Infrastructure.

Figure 5-23 Oracle BPM Suite Analytical Infrastructure

Description of Figure 5-23 follows
Description of "Figure 5-23 Oracle BPM Suite Analytical Infrastructure"

From a technical perspective, the Oracle BPM Suite Analytical Infrastructure is used for the support of process analytics as follows:

  • Audit Persistence

    The BPMN service engine is continuously generating audit events comprised of activity runtime data. The data of those audit events is persisted in audit tables of the service engine dehydration store. The audit data is the source of all analytical data.

  • JMS Topic

    To de-couple process execution from the preparation and publishing of analytical data, a JMS topic configured as part of the SOA Service Infrastructure is used.

  • Cube Action MDB

    A message driven bean (MDB) is used to trigger aggregation and persistence of the analytical data to the BPM cube schema stored in the SOA Service Infrastructure database.

  • BAM Action MDB

    A message driven bean (MDB) is used to publish analytical data towards the BAM adapter installed as part of the SOA Service Infrastructure.

  • Process Persistence

    For the persistence of audit events and analytical data to the SOA Infrastructure database, the Oracle BPM Suite leverages the Java Persistence API (JPA) Infrastructure.

    The persistence unit is configured to use the JTA data source jdbc/SOADataSource and the provider org.eclipse.persistence.jpa.PersistenceProvider.

5.4.4.1.2 Oracle Business Process Analytics External Dependencies

The Oracle Business Process Analytics libraries and components run as part of the BPMN Service Engine infrastructure. They rely heavily on the BPMN Service Engine artifacts (queues, stores, and JDBC resources) to function properly. As is the case for the service infrastructure, the SOA database must be available for the Analytics framework to work. Additionally, when feeding data to BAM, the required BAM system must be up and running to process the analytical information.

5.4.4.1.3 Oracle Business Process Analytics Startup and Shutdown Lifecycle

Since Oracle Business Process Analytics is part of the BPMN Service Engine, its lifecycle is the same as the BPMN Service Engine's lifecycle.

5.4.4.1.4 Oracle Business Process Analytics Log Files

The following loggers are available for tracing of the Oracle Business Process Analytics components:

  • oracle.bpm.analytics.measurement

  • oracle.bpm.analytics.cube

  • oracle.bpm.analytics.bam

To enable logging for one of the analytical components, set the log level to TRACE.

5.4.4.2 Oracle Business Process Analytics High Availability Considerations

This section provides information about high availability considerations for Oracle Business Process Analytics.

5.4.4.2.1 Oracle Business Process Analytics High Availability Architecture and Failover Considerations

Because Oracle Business Process Analytics is a subsystem in charge of feeding information to other systems (mainly read-oriented information), no special considerations for failover are required. A number of JMS queues used by Oracle Business Process Analytics are configured (as part of the SOA system high availability setup) as uniform distributed destinations for optimum availability and load balancing. All BPM Cube data persisted uses the SOA JDBC services, which use multi datasources configured for Oracle RAC databases.

5.4.4.2.2 Configuring Oracle Business Process Analytics for High Availability

For detailed instructions on configuring high availability for Oracle Business Process Analytics, refer to Section 5.13, "Configuring High Availability for Oracle SOA Service Infrastructure and Component Service Engines."

5.4.4.2.3 Cluster-Wide Configuration Changes for Oracle Business Process Analytics

Oracle Business Process Analytics instances do not store any local instance-specific properties. Cluster configuration changes are applied to each Oracle Business Process Analytics instance.

5.5 Oracle Mediator and High Availability Concepts

The information in this section guides you through the issues and considerations necessary for configuring Oracle Mediator for high availability.

5.5.1 Oracle Mediator Single-Instance Characteristics

Oracle Mediator is a service engine within the Oracle SOA Service Infrastructure. Oracle Mediator provides the framework to mediate between various providers and consumers of services and events. The Oracle Mediator service engine runs in-place with the SOA Service Infrastructure Java EE application. The runtime state for execution started by asynchronous interactions or involving parallel routing rules is maintained in the SOA runtime database. For details about administering Oracle Mediator, see the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite.

  • Synchronous and Asynchronous Interactions

    Oracle Mediator provides support for synchronous and asynchronous request-response interactions. In a synchronous interaction, the client making the request remains blocked, awaiting the response. In an asynchronous interaction, the client invokes the service but does not wait for the response. You can specify a time out period for an asynchronous interaction.

  • Sequential and Parallel Message Routing

    Oracle Mediator can route messages to their destinations either sequentially, or in parallel.

    • In a sequential routing, processing of data takes place in one single transaction.

    • In a parallel routing scenario, one transaction is used for en-queueing information, and another one for de-queuing it.

5.5.1.1 Oracle Mediator Component Characteristics

If a composite contains an Oracle Mediator component, SOA Service Infrastructure targets the component to the Oracle Mediator engine for deployment. None of the services provided by the Oracle Mediator engine system are singletons, therefore, Oracle Mediator engines can run in full active-active mode. The processing of messages by the worker threads in Oracle Mediator is transactional and relies on Oracle WebLogic Server transaction control service. Configure the appropriate transaction stores as recommended by WebLogic Server guidelines to guarantee recovery across failures in the WebLogic Server container. Additionally, Oracle Mediator's engine does not contain any stateful Web modules or stateful session beans, therefore you are not required to configure any sort of session replication when running Oracle Mediator in active-active mode. The state of work and work-to-be processed is maintained by Oracle Mediator in the database. Therefore, it is critical that Oracle Mediator's database be highly available. This requires configuring multi data sources for the SOA data source as described in Section 5.13.4, "Running Oracle Fusion Middleware Configuration Wizard on SOAHOST1 to Create the SOA Domain". For information about multi data source configuration with Oracle RAC and the MDS repository, see Section 4.1.2, "Using Multi Data Sources with Oracle RAC."

External Dependencies

Oracle Mediator depends on the following components:

  • SOA database for Mediator message and message state persistence

  • MDS repository for composite metadata store

Both components must be available for Oracle Mediator to start or run properly.

5.5.1.2 Oracle Mediator Startup and Shutdown Lifecycle

When the Oracle SOA Service Infrastructure application starts, it initializes the Oracle Mediator engine and loads the composites from the MDS repository. If the composite contains any Oracle Mediator components, it targets them to the Oracle Mediator engine. At runtime, Oracle Mediator routing rules can be invoked through an inbound binding component or by another service engine. Graceful shutdown of the Oracle Mediator engine is initiated by the SOA Service Infrastructure and involves sending signals to in-flight instances and unloading of loaded components.

Figure 5-24 illustrates Oracle Mediator Startup lifecyle.

Figure 5-24 Oracle Mediator Startup Lifecyle

Oracle Mediator Startup Lifecyle
Description of "Figure 5-24 Oracle Mediator Startup Lifecyle"

5.5.1.3 Oracle Mediator Request Flow

The recovery semantics of Oracle Mediator after a server failure are based on client interaction type and routing rule type.

Oracle Mediator provides support for synchronous and asynchronous request-response interactions. The following describes the behavior based on interaction type:

  • Synchronous Interaction: For synchronous interactions, an error is thrown back to the client in the case of a server failure. It is the client's responsibility to handle the error message and take appropriate action such as retrying the request.

  • Asynchronous Interaction: There are two types of asynchronous invocations - one that starts a new routing rule execution and one that is a callback to an existing rule execution. In the case of a callback, the engine attempts to resolve the subscribing instance through a correlation id. If there is a failure in handling the callback, the client receives an error message and has to handle the error appropriately. The client invocation must be transactional in order to guarantee reliable handling of a callback in the case of a server failure.

Oracle Mediator can route messages either sequentially or in parallel. Only messages processed in parallel need manual recovery in the case of a server failure. The following describes the behavior based on routing rule type:

  • Sequential Routing Rule: For sequential routing rule, complete rule is executed in a single transaction. In the case of server failure, Oracle Mediator relies on the underlying transaction manager for recovery.

  • Parallel Routing Rule: For parallel routing rule, Oracle Mediator uses a store-and-forward paradigm that involves two separate transactions - one transaction for persisting the message and a second one for executing the routing rule. If there is a failure prior to the persistence, the client receives an error message and has to handle the error in the same way as in a sequential routing rule. If the persistence is successful, the client call returns and further processing is done outside the context of the client call. In the case of a server failure, Oracle Mediator will initiate recovery upon server restart after a configurable time interval specified by ContainerIdLeaseRefresh

5.5.1.4 Oracle Mediator Configuration Artifacts

Starting with Oracle Fusion Middleware 11g Release 1 (11.1.1.2), the configuration parameters for Oracle Mediator are stored in the SOA MDS database. You can configure these parameters using Oracle Enterprise Manager Fusion Middleware Control.

To configure these parameters, go to the Oracle Enterprise Manager Fusion Middleware Control, and in the navigation tree, select SOA, soa-infra, (server_name). Right-click SOA Administration, and select Mediator Properties, or use the MBean Browser to navigate to the appropriate property.

5.5.2 Oracle Mediator High Availability Architecture and Failover Considerations

Oracle Mediator is an embedded service engine that leverages the same high availability architecture as the Oracle SOA Service Infrastructure. Figure 5-6 describes a two-node Oracle SOA Service Infrastructure cluster running on two WebLogic Servers. Oracle Mediator is deployed as part of the Oracle SOA Service infrastructure composite application.

In a clustered configuration, Oracle Mediator can run in an active-active mode as there are no singleton services and all state is stored in the SOA runtime database.

Note:

In a single-node environment, Oracle Mediator performance is slow if there a high latency between the middle tier and database (for example, if the middle tier and database are in different subnets).

5.5.2.1 Oracle Mediator Protection from Failures and Expected Behavior

For information about Oracle SOA Service Infrastructure protection from failures and expected behavior, see Section 5.2.2.1, "Oracle SOA Service Infrastructure Protection from Failures and Expected Behavior."

During the execution of a Parallel Routing Rule, Oracle Mediator obtains a logical lock on a batch of messages in the SOA runtime database. This lock contains a reference to the container ID that uniquely identifies the Oracle Mediator engine. The container ID is assigned at startup time. The user can configure the batch size using DeferredMaxRowsRetrieved. A smaller batch size ensures that there are a lower number of locked rows requiring recovery (as explained above) in the case of a server failure.

In case of unplanned outages, you must wait as much time as specified in the ContainerIdLeaseRefresh interval after restarting the server. This enables the server to complete the instances still in the running state.

In a multi-node cluster environment, if Oracle Mediator is used for an asynchronous message exchange pattern, there could be a possibility that the callback will not be handled by Oracle Mediator if it arrives before the request has completed. This could happen in a scenario where the request initiated by Oracle Mediator to the target service takes longer to complete, and callback from the target service arrives before the request completes.

Process Failure

Oracle Mediator is protected from all process failures by the WebLogic Server infrastructure. The following process failure considerations apply to Oracle Mediator:

  • If the managed servers crash, Node Manager attempts to restart them locally. If whole server migration is configured and repeated restarts fail, the WebLogic Server infrastructure attempts to perform server migration of the managed server to the other node in the cluster, if it is configured. Once the server on the other node is restarted, Oracle HTTP Server resumes routing any incoming requests to it. The migrated server reads the SOA database, resumes any pending processing, and resumes transactions from the transaction logs in shared storage.

  • The SOA Service Infrastructure application where the Oracle Mediator engine runs may be down due to failure in accessing resources, errors caused by the coherence infrastructure, or other issues unrelated to the status of the managed server where it is located. Therefore, monitor the Oracle SOA Service Infrastructure application to watch for errors caused by the application in the managed server logs. For the location of server logs, see Section 5.2.1.6, "Oracle SOA Service Infrastructure Log File Locations").

  • During recovery after a server crash, Oracle Mediator attempts to redeliver messages that were partially processed. If the automatic retry fails, Oracle Mediator enqueues the message to the error hospital for manual recovery.

Node Failure

In case of a node failure, server migration is triggered after the available server verifies the time stamp in the database leasing system. If the crashed server was the coherence cluster master, the available server become the new master, and the Oracle Mediator engine remains available for processing messages from binding components. After the time stamp for leasing is verified, the Node Manager in the remaining node attempts to migrate the VIP used by the failed managed server, and restarts it locally. This effectively results in the Oracle Mediator engine having two instances running in the same node.

Cluster-Wide Configuration Changes

The standard Java EE artifacts that Oracle Mediator engine uses are configured as part of Oracle WebLogic's domain in which SOA is installed. Oracle WebLogic Clusters provide automatic configuration synchronization for artifacts, such as data sources, persistent stores, and JMS modules across the WebLogic Server domain. At the same time, the WebLogic Server cluster is in charge of synchronizing the deployments and libraries used by the Oracle Mediator engine.

As explained in Section 5.5.1, "Oracle Mediator Single-Instance Characteristics," Oracle Mediator engine-specific configuration is stored in the SOA MDS database. Changes are applied once (per SOA server), but affect all Oracle Mediator instances in the same SOA domain. For example, if WLS_SOA1 and WLS_SOA2 are part of the same SOA cluster and the heartbeat period for Oracle Mediator in WLS_SOA1 is changed to 20, WLS_SOA2's heartbeat period is also set to 20.

5.5.2.1.1 Recovering Failed Mediator Instances

During recovery from a server failure, under certain circumstances, messages with in-flight routing rules may be placed in the error hospital for manual recovery. These messages can be recovered using Enterprise Manager as shown in Figure 5-25.

Figure 5-25 Mediator Instance Recovery using Oracle Enterprise Manager

BPEL PM Instance Recovery using Oracle Enterprise Manager
Description of "Figure 5-25 Mediator Instance Recovery using Oracle Enterprise Manager"

5.5.2.1.2 Oracle Mediator's Resequencing in a Cluster

Oracle Mediator's resequencer allows rearranging a stream of related (but not necessarily ordered) messages back into order. It sequences the incoming messages that arrive in a random order, and then sends them to the target services in an orderly manner.

The resequencer works with two central concepts: groups and sequence IDs. The sequence ID is an identifier part of the message, based on which messages are rearranged. Messages arriving for resequencing are split into groups, and the messages within a group are sequenced according to the sequence ID. The sequencing is based on the sequencing strategy selected (the Standard, FIFO and Best Effort strategies are available in the Oracle JDeveloper Composite Designer).

Message processing is based on assigning a group id to batches of messages in Oracle Mediator's queue system. The resequencer dequeue system uses an instance identifier (ID) to lock a group, then processes the group messages. The heartbeat infrastructure creates and maintains the instance ID in Oracle Mediator. There is one instance ID for every Oracle Mediator instance. The heartbeat infrastructure inserts the instance ID and current time into the table MEDIATOR_CONTAINERID_LEASE. The heartbeat infrastructure has one heartbeat thread. This thread periodically updates the time associated with the instance ID to announce its presence to other Oracle Mediator instances. The Oracle Mediator configuration parameter ContainerIdLeaseRefresh, specified in minutes (default: 1), is used for this purpose. The heartbeat thread also looks for instance IDs that have not been updated within a configurable time period. The Oracle Mediator configuration parameter ContainerIdLeaseTimeout, specified in minutes (default: 5), is used for this purpose. The thread releases the locks held by these instance IDs. After the groups are unlocked, the dequeue system of other Oracle Mediator instances can pick up the groups for processing.

The possible states for a group stored in the MEDIATOR_GROUP_STATUS table are:

  • Group Ready for processing - 0

  • Group Locked - 1

  • Group Processing Completed - 2

  • Error in processing group - 3

You can configure both the ContainerIdLeaseTimeout and ContainerIdLeaseRefresh parameters from the Oracle Enterprise Manager Fusion Middleware Control configuration screens for Oracle Mediator. These properties determine the behavior of a cluster of Oracle Mediator Instances configured for re-sequencing. Each group is processed by one single server until its lease expires (the instance ID is not updated). If a managed server goes down while it was processing a group, the other instances in the cluster will proceed to its ordering once the lease has expired (that is, once the unique ID assigned to the group has not been updated during a lease timeout period).

5.5.2.2 Troubleshooting Oracle Mediator High Availability

To debug Oracle Mediator failures, check the database tables to determine which container failed. To identify requests that were in progress when Oracle Mediator failed, find the rows that are still locked, and unlock them. You may also view the payload, as it is stored as a blob.

The poll interval for Oracle Mediator Instance Manager should be the same across all the nodes. The time stamp used is the database time stamp.

The possible states of messages in the database are:

  • Ready

  • Locked

  • Completed

  • Error

5.6 Oracle Human Workflow and High Availability Concepts

The information in this section guides you through the issues and considerations necessary for configuring Oracle Human Workflow for high availability.

5.6.1 Oracle Human Workflow Single-Instance Characteristics

Oracle Human Workflow is a service engine running in the Oracle SOA Service Infrastructure that allows the execution of interactive human driven processes. A human workflow provides the human interaction support such as approve, reject, and reassign actions within a process or outside of any process. The Human Workflow service consists of a number of services that handle various aspects of human interaction with a business process.

All human task metadata is stored and managed in the Metadata Service (MDS) repository. The Human Workflow engine consists of a Service Engine running within the SOA Service Infrastructure and additional Java EE applications for DefaultToDoTaskFlow and Worklist applications. A human workflow leverages an internal JMS queue for notifications related to a human task.

For details about administering Oracle Human Workflow, see the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite.

5.6.1.1 Oracle Human Workflow Startup and Shutdown Lifecycle

The Human Workflow startup consists of two phases: loading of the Java EE applications, and initialization of the service engine. The Java EE applications are loaded as part of the application server startup. The service engine initialization and shutdown is controlled by SOA Service Infrastructure. Post initialization, composites that contain Human Workflow components are targeted to the Human Workflow service engine by SOA Service Infrastructure.

5.6.1.2 Oracle Human Workflow Request Processing

A human workflow can be initiated by an invocation from another SOA engine (such as Oracle BPEL PM). The message is routed to the engine by SOA Service Infrastructure and is persisted by the Workflow engine within its runtime schema. The message becomes available for human actions through the browser based UI after the client transaction that is associated with the invocation is committed. Each update on the message or the runtime state through a user action is a separate transaction.

As soon as Workflow commits its transaction, the control passes back to BPEL which almost instantaneously commits its transaction. Between this window, if the Oracle RAC instance goes down, on failover, the message is retried and can cause duplicate tasks.

5.6.1.3 Oracle Human Workflow Configuration Artifacts

Starting with Oracle Fusion Middleware 11g Release 1 (11.1.1.2), the configuration parameters for Oracle Human Workflow are stored in the SOA MDS database. You can configure these parameters using Oracle Enterprise Manager Fusion Middleware Control.

To configure these parameters, go to the Oracle Enterprise Manager Fusion Middleware Control, and in the navigation tree, select SOA, soa-infra, (server_name). Right-click SOA Administration, Workflow Notification/Task Service Properties, or use the MBean Browser to navigate to the appropriate property.

5.6.1.3.1 Managing the URI of the Human Task Service Component Task Details Application

You can add or remove the URI of the task details application used in Oracle Human Workflow.

To manage the URI of the human task service component task details application:

  1. Access this page through one of the following options:

    From the SOA Infrastructure Menu... From the SOA Folder in the Navigator...
    1. Select Home.
    2. Select the Deployed Composites tab.

    3. In the Composite section, select a specific SOA composite application.

    1. Under soa-infra, select a specific SOA composite application.

  2. Select the human task service component in the Component Metrics table.

  3. Click Administration.

    The Administration page shows the URI for the task details application.

    Description of hwf_comp_admin.gif follows
    Description of the illustration hwf_comp_admin.gif

  4. Click the Add icon to specify the following details for the URI:

    • Application name

    • Host name

    • HTTP port

    • HTTPS port (optional)

    • URI

  5. Click Apply.

5.6.2 Oracle Human Workflow High Availability Architecture and Failover Considerations

Figure 5-6 describes a two-node Oracle SOA Service Infrastructure cluster running on two WebLogic Servers. Oracle Human Workflow is deployed as part of the Oracle SOA Service infrastructure composite application.

5.6.2.1 Oracle Human Workflow Protection from Failures and Expected Behavior

Oracle Human Workflow's engine uses transactional EJBs for persistence and JMS queues for user notification. All state is stored in the database and the JMS queue, and there is no in-memory session state to be replicated for recovery. Therefore, Oracle Human Workflow's service engine and the associated Java EE applications are run in an active-active topology on a WebLogic cluster. In the case of a server or hardware crash, whole server migration must be configured for recovering pending transactions and JMS messages stored on the local queue. Notifications are not sent out until the server is restarted.

5.6.2.2 Manual Recovery Required for Human Workflow Task in Rejected MSG Table

The FabricInstanceManager.persistCompositeInstanceBean API (or other InstanceManager APIs) does not have built-in retry logic. When there are transient failures caused by Oracle RAC failover, the WS invocation is rejected directly. This manual recovery of these rejected messages is done using Oracle Enterprise Manager Fusion Middleware Control.

To manually recover the rejected messages, in the Oracle Enterprise Manager Fusion Middleware Control, select SOA, soa-infra(soa_server1), <composite_name>, Faults and Rejected Messages, select the message, and choose Recover With Options. This causes the workflow request to be retried.

5.6.3 Troubleshooting Oracle Human Workflow High Availability

Human Workflow works like a standard Java EE application with a browser based client. Once a message is persisted to its runtime schema, it can be processed using the workflow UIs. If a server crashes in the middle of the transaction, you must recover the server for successful recovery or rollback. The following logs are relevant for debugging errors associated with Oracle Human Workflow:

  • oracle.soa.services.common

  • oracle.soa.services.notification

  • oracle.soa.services.workflow

  • oracle.soa.services.workflow.common

  • oracle.soa.services.workflow.evidence

  • oracle.soa.services.workflow.metadata

  • oracle.soa.services.workflow.persistency

  • oracle.soa.services.workflow.query

  • oracle.soa.services.workflow.runtimeconfig

  • oracle.soa.services.workflow.soa

  • oracle.soa.services.workflow.task

  • oracle.soa.services.workflow.task.dispatch

  • oracle.soa.services.workflow.task.routing

5.7 Oracle B2B and High Availability Concepts

The information in this section guides you through the issues and considerations necessary for configuring Oracle B2B for high availability.

5.7.1 Oracle B2B Single-Instance Characteristics

Oracle B2B connects SOA composite applications to external services, applications, and technologies. Oracle B2B offers a multi-protocol gateway that supports industry-recognized B2B standards. Oracle B2B extends Oracle SOA Suite with business protocol standards, such as electronic data interchange (EDI), ebXML, HL7, and RosettaNet. For details about Oracle B2B's functionality, see the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite.

Oracle B2B is a binding component within SOA Service Infrastructure. B2B metadata consists of trading partner details along with their supported documents and delivery channels which is stored within the MDS repository. The SOA composite has a reference to this metadata.

Oracle B2B receives messages from different channels. A listener thread logs the messages in a table in the SOA database and sends the corresponding events to a JMS queue. Event handlers listen for events to process the requests in the JMS queues.

Oracle B2B supports the following transport protocols for communication with business entities:

  • HTTP(S)

  • Oracle Advanced Queue

  • Email (SMTP 1.0, IMAP 1.0, POP3)

  • File

  • FTP and SFTP (SSH FTP)

  • TCP/IP

  • JMS

For HTTP protocol, Oracle B2B uses the system's HTTP listener. For FTP and Email, Oracle B2B uses the external FTP and email server configured in the system. Even in high availability mode, only one B2B instance polls for incoming message for FTP, File, and email protocols.

Note:

MLLP & TCP/IP protocols are not supported in a clustered environment.

5.7.1.1 Oracle B2B Component Characteristics

Oracle B2B runs in-place with SOA Service Infrastructure Java EE application. The Oracle B2B user interface is a Web application that is deployed as a standalone war file on the same managed server as the SOA Service Infrastructure. Oracle B2B UI application is stateful and stores information in the HTTP Session.

Oracle B2B stores state information within JMS queues and the SOA run-time database. Whole server migration is required for automatic JMS message and transaction recovery after a server failure.

Oracle B2B uses JMS intensively. Oracle B2B uses the Oracle WebLogic Server migration feature for protecting Oracle B2B JMS resources against failures. Oracle B2B's engine uses EJBs and Servlets, which are stateless. All state information is persisted in JMS queues and the database.

High availability of B2B depends on the high availability of the run-time database. Hence, the SOA runtime data sources should be configured as multi data sources for high availability. For information about multi data source configuration with Oracle RAC and the MDS repository, see Section 4.1.2, "Using Multi Data Sources with Oracle RAC."

External Dependencies

Oracle B2B depends on the following components:

  • Oracle SOA database for Oracle B2B message and message state persistence

  • MDS repository for Oracle B2B metadata store

  • FTP and email servers if the corresponding adapters are used

The SOA database and the MDS repository must be available for Oracle B2B to start and run properly.

5.7.1.2 Oracle B2B Startup and Shutdown Lifecycle

When the Oracle SOA Service Infrastructure application starts, it initializes Oracle B2B's engine. Oracle B2B metadata deployment should occur before composites are deployed to the Oracle SOA Service Infrastructure. Oracle B2B end-points are defined as channels which are started as part of the engine initialization. Based on the protocol, each configured channel has external dependencies, such as:

  • If FTP is used, the appropriate FTP server must be started.

  • If email is used, the appropriate email server must be started.

  • If HTTP/HTTPS is used, the HTTP server front ending the B2B system must be started

5.7.1.3 Oracle B2B Request Flow

In an Oracle B2B system all processing is done asynchronously. When a request arrives, the corresponding message is inserted in the SOA database. A notification is inserted in a JMS queue as a placeholder for the pending work. The event handler threads listen for events in these queues and process the work sequentially.

Figure 5-26 Oracle B2B Request Flow

Oracle B2B Request Flow
Description of "Figure 5-26 Oracle B2B Request Flow"

5.7.1.4 Oracle B2B Configuration Artifacts

You can enable and disable metrics for Oracle B2B's Server using Oracle Enterprise Manager Fusion Middleware Control. This is the only property directly exposed by Oracle Enterprise Manager Fusion Middleware Control. The configuration for Oracle B2B's Server is maintained in the SOA database and most of it is exposed by the Oracle B2B user interface application. For details about Oracle B2B configuration, see the Oracle Fusion Middleware User's Guide for Oracle B2B.

5.7.2 Oracle B2B High Availability Architecture and Failover Considerations

Figure 5-6 describes a two-node Oracle SOA Service Infrastructure cluster running on two WebLogic Servers. Oracle B2B is deployed as part of the Oracle SOA Service infrastructure composite application. The following high availability characteristics are specific to an Oracle B2B high availability deployment:

  • The Oracle B2B user interface application runs inside each one of Oracle WebLogic Server servers, and as part of the same cluster as the Oracle SOA Service Infrastructure application.

  • Oracle B2B's server maintains the partners, documents, and channels definitions in the SOA database using an Oracle RAC database.

5.7.2.1 Oracle B2B Protection from Failures and Expected Behavior

This section describes how an Oracle B2B high availability cluster deployment protects components from failure. This section also describes expected behavior in the event of component failure.

Oracle B2B UI Failure

The Oracle B2B user interface application maintains some navigation information in memory. When a failure occurs in one of the managed servers running the Oracle B2B user interface, users' requests are redirected to another active WebLogic Server running the application. Ongoing requests from Oracle HTTP Server time out according the time out setting in Oracle HTTP Server's configuration. Failover requires those users accessing the failed instance to log in again, since the application is not enabled for serializing the session information.

The Oracle B2B user interface in-memory data is non-transactional, and some steps may need to be revisited in the case of failover.

For general information about Oracle SOA Service Infrastructure process failure, see Section 5.2.2.1, "Oracle SOA Service Infrastructure Protection from Failures and Expected Behavior"

Node Failure

If a node failure occurs, or if the local Oracle WebLogic Server Node Manager reaches the maximum restart tries on the failed managed server, whole server migration is triggered after the available server verifies the time stamp in the database leasing system. The other Oracle B2B engine remains available for processing new messages from the different channels. At the same time, the remaining Oracle B2B server should resume the pending operations for singleton channels, such as FTP, email, or file, after the default timeout period is reached in the time stamp the failed node sets in the database (two minutes by default). For Oracle B2B's User Interface application, ongoing requests from Oracle HTTP Server time out according to Oracle HTTP Server configuration.

Database Failure

For information about Oracle B2B database failure, see Section 5.2.2.1, "Oracle SOA Service Infrastructure Protection from Failures and Expected Behavior"

5.7.2.2 Oracle B2B Cluster-Wide Configuration Changes

The standard Java EE artifacts that the Oracle B2B engine uses are configured as part of the Oracle WebLogic domain in which the Oracle SOA Service Infrastructure is installed. Oracle WebLogic Clusters provide automatic configuration synchronization for artifacts, such as data sources, persistent stores, and JMS modules across the WebLogic Server domain. At the same time, the WebLogic Server cluster controls synchronization of deployments and libraries used by Oracle B2B's engine.

As explained in Section 5.7.1, "Oracle B2B Single-Instance Characteristics," all Oracle B2B server-specific configuration is maintained in the database, and configuration changes are applied to all the SOA Servers running in a WebLogic Server domain. Therefore, configuration properties, such as Payload Size, and Outbound Dispatcher Counts are applied cluster-wide, meaning they are used by all instances in the Oracle SOA cluster.

To set up File, FTP, or Email transports in a high availability environment, specify a unique name for each instance by using b2b.HAInstanceName unique_instance_name. If you use #ServerName# for the value, Oracle B2B retrieves the WebLogic Server name as the HAInstanceName.

"Properties to Set in Fusion Middleware Control" in the Oracle Fusion Middleware User's Guide for Oracle B2B also describes setting up File, FTP, or Email transports in a high availability environment.

5.7.2.3 Oracle B2B Deployments in a Cluster

Using the command line utility for deploying, purging, or importing metadata in B2B may cause inconsistencies and errors in the B2B system. For B2B, deploy agreements and purge or import metadata ONLY from the GUI available in the B2B console, instead of using the command line utility.

5.7.2.4 Troubleshooting Oracle B2B Active-Active Configuration

This section provides troubleshooting tips and possible resolutions for Oracle B2B active-active configurations.

5.7.2.4.1 Purge, Import, or Deployment of B2B Metadata

When performing purge, import, or deployment of B2B metadata in a cluster, error timing and load balancing may cause exceptions that are unlikely to happen if a retry of the operation is performed.

There is no clean up nor other additional steps required. If errors, such as [java] MDS-02202: Content of the metadata object appear for deployment or "postTransfer: MDS-00521: error while reading document... appear for purge or import, retry the operation.

5.7.2.4.2 Error While Retrieving Oracle B2B Document Definitions

Problem: Error happens when trying to retrieve a document definition XSD from Oracle B2B. B2B resides in a cluster and is accessed through a load balancer. B2B console report the following:

An error occured while loading the document definitions.
 java.lang.IllegalArgumentException: Cluster address must be set when clustering
 is enabled.

Solution: This occurs if you do not set the front end HTTP host and port for the Oracle WebLogic cluster where Oracle B2B resides. To eliminate this error, set the front end address for the SOA Cluster:

  1. In the WebLogic Server Administration Console, in the Change Center section, click Lock & Edit.

  2. In the left pane, choose the Environment in the Domain Structure window and then choose Clusters. The Summary of Clusters page appears.

  3. Select the WLS_SOA cluster.

  4. Select HTTP.

  5. Set the values for the following:

    • Frontend Host: soa.mycompany.com

    • Frontend HTTPS Port: 443

    • Frontend HTTP Port: 80

  6. Click Save.

  7. To activate the changes, click Activate Changes in the Change Center section of the Administration Console.

  8. Restart the servers to make the Frontend Host directive in the cluster effective.

5.8 Oracle Web Services Manager and High Availability Concepts

The information in this section guides you through the issues and considerations necessary for configuring Oracle WSM for high availability.

5.8.1 Oracle WSM Single-Instance Characteristics

Oracle Web Services Manager (Oracle WSM) provides a policy framework to manage and secure Web services consistently across your organization. It provides capabilities to build, enforce, execute and monitor Web service policies including security, WSRM, MTOM and addressing policies. It typically gets deployed along with SOA Service Infrastructure.

Oracle Web Services Manager is made up of the following components:

  • Policy Manager reads and writes security and management policies including predefined and custom policies from the MDS repository. It is typically deployed on the Oracle SOA Service Infrastructure managed servers. However, you can deploy Policy Manager on separate managed servers.

    Policy Manager is a stateless Java EE application. It exposes its capabilities through stateless session beans. Policy Manager does not cache any data, (all accesses go to the MDS repository).

  • Agent is responsible for policy enforcement, execution and gathering of runtime statistics. The Oracle WSM Agent is available on all Oracle Fusion Middleware managed servers. It is configured on the same server as the application which it protects.

    The Oracle WSM Agent is made up of a set of jar files, which are a part of underlying Web service stack. It does not have any session state. The Agent maintains an in-memory policy cache, which is populated at the Agent startup time. It does not use any JTA or JMS.

    The Oracle WSM Agent is made up of the following two pieces:

    • Policy Access Point (PAP) communicates with Policy Manager. The Agent communicates with the Policy Manager through EJB invocations over T3 (or T3s if SSL is enabled) protocol.

    • Policy Interceptor is generated when a Web service is deployed and activated, or when a policy is attached to a Web service using Enterprise Manager. If new WebServices are protected using Oracle WSM, an additional instance of the interceptor is generated for each new Web service. Interceptor is responsible for policy enforcement.

  • Metadata Store Policies are stored in the MDS repository. It is typically backed by an Oracle database. For high availability purposes, Oracle recommends using an Oracle RAC database as the back end for MDS repository.

  • Enterprise Manager is used to configure Oracle WSM. It also displays different WebServices metrics gathered by Oracle WSM.

For more details of Oracle WSM architecture, please refer to the Oracle Fusion Middleware Security and Administrator's Guide for Web Services.

Figure 5-27 Oracle WSM Single-Instance Architecture

Oracle WSM Single-Instance Architecture
Description of "Figure 5-27 Oracle WSM Single-Instance Architecture"

5.8.1.1 Oracle WSM Component Characteristics

The Oracle WSM Agent is a set of .jar files available on every Oracle Fusion Middleware managed server in a Web services stack.

Policy Manager is contained in the wsm-pm.ear file. None of the services provided by Oracle WSM are singletons, therefore, it can run in full active-active mode. Oracle WSM services can be validated by http://SOAHOSTx:port/wsm-pm/validator. This validator displays Oracle WSM policies.

The Oracle WSM Agent and Oracle Enterprise Manager interact with Policy Manager using the EJB interfaces. The EJBs used in Oracle WSM are stateless and can be deployed in a clustered environment. Therefore, there is no requirement to enable state replication in the cluster.

The Oracle WSM Agent and Policy Manager need not be co-located. However, the Agent expects Policy Manager to be deployed on at least one node of the domain. The Oracle WSM Agent has capabilities to auto-discover Policy Managers deployed in the domain.

Neither the Oracle WSM Agent nor Policy Manager use JTA or JMS messaging. The MDS-based policy store also does not support JTA.

External Dependencies

Oracle WSM Policy Manager depends on the following components:

  • MDS repository for storing the policies

  • Oracle WSM Agent depends only on Oracle WSM Policy Manager.

Both components must be available for Oracle WSM to start and run properly.

5.8.1.2 Oracle WSM Startup and Shutdown Lifecycle

The following key Oracle WSM components are involved in the startup lifecycle for Oracle WSM:

  • Oracle WSM Policy Manager

  • Oracle WSM Agent

Policy Manager is a stateless application which does not perform any caching. There is no special application level startup sequence performed when the managed server where Policy Manager is deployed starts up. Policy Manager communicates with the MDS repository to retrieve policies. The MDS repository can be stored in an Oracle RAC database to provide MDS high availability.

When a managed server on which an Agent is configured comes up, the Agent connects to Policy Manager to get latest revision of policies. If it succeeds, the changes to the policies are downloaded and cached. Once the Agent is up and running, it periodically attempts a cache refresh at a configurable interval. The default time is every one minute.

Oracle WSM Agent communicates with the Policy Manager through EJB invocations over T3 (or T3s if SSL is enabled) protocol. If Policy Manager is deployed on different nodes and some of them have SSL enabled and others don't, Agent communicates only with the nodes with SSL connections.

If the Policy Manager to which the Oracle WSM Agent is connected becomes unavailable, the underlying infrastructure automatically connects to another Policy Manager instance running elsewhere in the cluster. This is achieved through Oracle WebLogic EJB clustering.

For high availability scenarios, if an Oracle WSM application is targeted to multiple nodes, it should be targeted to a cluster rather than to individual managed servers.

If a managed server has Web services deployed which are protected by Oracle WSM, and the Oracle WSM Agent is not able to communicate with any of the Policy Managers at startup time, Web service invocation fails.

5.8.1.3 Oracle WSM Request Flow

When a protected Web service is accessed by a client application, the Oracle WSM Agent queries the policy cache and enforces the applicable policies. Based on the policies, the request is authenticated, encrypted, decrypted, authorized or logged. It does not connect to Policy Manager for any of these operations. Runtime availability of Policy Manager does not affect the functioning of the Oracle WSM Agent, unless there is a configuration change, such as new Web services, which are protected by Oracle WSM, being deployed, or new policies attached to existing Web services. If there is such a configuration change, then the Oracle WSM Agent must connect to Policy Manager to get the applicable policies. If it cannot connect after initial startup, it continues to operate based on the cached policies.

5.8.1.4 Oracle WSM Configuration Artifacts

Oracle WSM Agent configuration resides in policy-accessor-config.xml. This file is located in the DOMAIN_HOME/config/fmwconfig directory. With this configuration file you can specify:

  • Policy Manager URL (if configured)

  • Cache Refresh Interval

  • Clock skew, to allow for difference in system clock of the client and servers

These options are also available from Oracle Enterprise Manager Fusion Middleware Control and are specific to each Oracle WSM Agent installation.

Other configuration options at the container level, such as data sources for MDS repository location, and application targeting, are maintained as part of Oracle WebLogic Server Domain configuration, and are synchronized across a cluster of Oracle WebLogic Servers by Oracle WebLogic Server core infrastructure.

5.8.2 Oracle WSM High Availability Architecture and Failover Considerations

Figure 5-6 describes an Oracle SOA Service Infrastructure two-node cluster running on two WebLogic Servers. Oracle WSM is deployed as part of the Oracle SOA Service infrastructure composite application. The following high availability characteristics are specific to an Oracle WSM high availability deployment:

  • Oracle WSM Policy Manager, as well as Oracle WSM Agents are deployed on WLS_SOA_INFRA. In addition, Oracle WSM Agents are available on any custom WLS cluster, in case there is a need to protect any custom WebServices deployed on them.

  • Oracle WSM Policy Manager and Oracle WSM Agents run in WebLogic Server Infrastructure managed servers that are part of a WebLogic Server cluster. The WebLogic Server cluster synchronizes configuration for common artifacts of WebLogic Server used by Oracle SOA Service Infrastructure (JDBC).

  • Oracle WSM Policy Manager EJBs leverage clustering and high availability capabilities of the WebLogic Server cluster.

  • All Oracle WSM Policy Manager instances in the cluster point to the same MDS repository.

  • The MDS repository where Oracle WSM policies are stored is configured in an Oracle RAC database to protect from database failure.

5.8.2.1 Oracle WSM Protection from Failures and Expected Behavior

Since the Oracle WSM Agent is deployed on the same managed server as the application is deployed, it will be available again as soon as the application becomes available due to server restart/migration. The following two sections describe the failover for Policy Manager.

Process Failure

Oracle WSM components are protected from process failures by the WebLogic Server infrastructure:

  • If the managed servers crash, Node Manager attempts to restart them locally. If whole server migration is configured and repeated restarts fail, the WebLogic Server infrastructure attempts to perform server migration of the managed server to another node in the cluster. Once the server on the other node is restarted, Oracle HTTP Server resumes routing any incoming requests to it. At startup time, Oracle WSM Agent and Policy Manager go through the startup lifecycle as described in Section 5.8.1.2, "Oracle WSM Startup and Shutdown Lifecycle".

  • If the managed server running Policy Manager is restarted or migrated, the failover is transparent to the Agents connected to it. Policy Manager leverages underlying EJB clustering and the failover infrastructure of Oracle WebLogic Server.

Node Failure

If node failure occurs and whole server migration is not configured, the Oracle WSM Agent fails over transparently to the other Policy Manager instance.

If whole server migration is configured, server migration is triggered after the available server verifies the time stamp in the database leasing system. After the time stamp for leasing is verified, the Node Manager in the node that still remains available attempts to migrate the VIP used by the failed managed server, and restarts it locally. This effectively results in the Oracle WSM Policy Manager application having two instances running in the same node. Refer to the Section 3.9, "Whole Server Migration" for more details on the failover process. In this situation, the Oracle WSM Agents load balance against both Policy Manager instances running on the same node.

5.8.2.2 Oracle WSM Cluster-Wide Configuration Changes

The standard Java EE artifacts that Policy Manager uses are configured as part of the Oracle WebLogic domain in which the Oracle SOA Service Infrastructure is installed. Oracle WebLogic Clusters provide automatic configuration synchronization for artifacts, such as data sources, across the WebLogic Server domain. At the same time, the WebLogic Server cluster controls synchronizing the deployments and libraries used by different Oracle WSM components.

As explained in the single-instance section, Oracle WSM instance specific aspects are configured individually per WebLogic Server domain directory (typically one per node) through the policy-accessor-config.xml file. This file is included in the .jar file that is created when using the Oracle WebLogic Server Pack utility, therefore it is propagated to other nodes when you run pack/unpack to set up high availability for the Oracle SOA Service Infrastructure. However, ongoing changes to policy-accessor-config.xml are not replicated automatically to the other nodes. This implies that for updates, you must modify each node individually, for example, if you updated the Policy Manager URL or Cache Refresh Interval. These modifications must be manually performed in all the domain directories across a high availability topology. You can edit the policy-accessor -config.xml file directly, or use Oracle Enterprise Manager Fusion Middleware Control to make the configuration change on each managed server.

Oracle WSM Agents are capable of automatically discovering the deployed Oracle WSM Policy Manager deployments. If you want to override the behavior and some point to a specific Policy Manager instance, you can do so by editing the policy-accessor-config.xml file.

5.8.2.3 Configuring the Java Object Cache for Oracle WSM

The Java Object Cache (JOC) should be configured among all the servers running Oracle WSM. This local cache is provided to increase the performance of Oracle WSM.

For instructions on configuring the Java Object Cache among all the servers running Oracle WSM, see Section F.1, "Configuring the Java Object Cache."

5.8.2.4 Configuring Distributed Notifications for the MDS Repository

In high availability environments, Oracle recommends that you configure distributed notifications for the MDS repository.

For instructions on configuring distributed notifications for the MDS repository, see Appendix G, "Configuring Distributed Notifications for MDS."

5.9 Oracle User Messaging Service and High Availability Concepts

The information in this section guides you through the issues and considerations necessary for configuring Oracle User Messaging Service for high availability.

5.9.1 Oracle User Messaging Service Single-Instance Characteristics

Oracle User Messaging Service (Oracle UMS) enables two way communication between users and deployed applications. It has support for a variety of channels, such as email, IM, SMS, and text-to-voice messages. Oracle UMS is integrated with Oracle Fusion Middleware components, such as Oracle BPEL PM, Oracle Human Workflow, Oracle BAM and Oracle WebCenter. It is typically deployed along with the Oracle SOA Service Infrastructure.

Oracle UMS is made up of the following components:

  • UMS Server is a Java EE application that runs on Oracle WebLogic Server. The UMS Server orchestrates message flows between applications and users. The server routes outbound messages from a client application to the appropriate driver, and routes inbound messages to the correct client application. The server also maintains a repository of previously sent messages in a persistent store, and correlates delivery status information with previously sent messages.

  • UMS Drivers connect UMS to the messaging gateways, adapting content to the various protocols supported by UMS. Drivers can be deployed or undeployed independently of one another depending on the messaging channels available in a given installation.

    UMS Drivers adapt sent and received content to and from external protocols, such as email, XMPP, and SMPP. UMS Drivers are also Java EE applications deployed to an Oracle WebLogic Server.

  • UMS Client applications implement the business logic of sending and receiving messages. Examples of a UMS client application include an Oracle SOA application that sends messages as one step of an Oracle BPEL PM workflow, or an Oracle WebCenter Spaces application that can send messages from a Web interface.

    UMS client applications have either a UMS-specific EJB module embedded in them, or interact as standard Web service clients.

Figure 5-28 illustrates the services and dependencies of an Oracle UMS single-instance architecture.

Figure 5-28 Oracle User Messaging Service Single-Instance Architecture

Description of Figure 5-28 follows
Description of "Figure 5-28 Oracle User Messaging Service Single-Instance Architecture"

5.9.1.1 Oracle User Messaging Service Component Characteristics

This section describes the characteristics of Oracle UMS components.

UMS Server is made up of the following:

  • Message driven beans (MDBs) that dequeue messages from JMS queues

  • Stateless session beans to implement messaging business logic

  • JAX-WS servlets to implement the messaging Webservices

  • A simple Oracle ADF Faces user interface component for managing end user messaging preferences.

UMS drivers typically contain the following:

  • JCA resource adapter (embedded within the EAR) in order to interface with external protocol gateways

  • Two MDBs, one for inbound, and one for outbound, that interface between the resource adapter and the JMS queues

  • Some UMS drivers that implement an HTTP-based protocol, such as VoiceXML, have a servlet module as well.

UMS client applications that use the EJB interface have a stateless session bean that provides the API used by client business logic, and an MDB to asynchronously receive inbound messages.

UMS depends heavily on JMS. JMS queues are also used to buffer content between clients and servers, and between servers and drivers.

Each typical messaging operation (sending an outbound message or receiving an inbound message) involves two JMS queues, one between the client and server, and one between the server and driver.

UMS messaging state is stored in the database and in persistent JMS queues.

External Dependencies

UMS depends on an external database repository to maintain message and configuration state. It shares this state between clustered instances. The UMS Server accesses the database using a Java EE JDBC data source provisioned at installation time. This data source is a non-XA data source. Therefore, UMS does not depend on the JTA capabilities of the underlying infrastructure.

UMS uses JMS to deliver messages among messaging applications. By default it is configured to use a file-based persistent JMS store, therefore it depends on the storage device where those files are located.

5.9.1.2 Oracle User Messaging Service Startup and Shutdown Lifecycle

As Java EE applications, all UMS components are started by Oracle WebLogic Server container.

UMS Server Startup

At server startup time, UMS Server initializes a TopLink session that creates a connection to the database repository. UMS Server then begins listening on Web service endpoints. EJBs also become available for invocation of functionality such as sending messages or retrieving delivery status.

UMS Driver Startup

When a UMS driver is deployed to a cluster, it sends a registration message to the local UMS server. The UMS server records the registration information in the database, making it available to all UMS servers in the cluster. Once this occurs, any UMS server can route messages to any UMS driver in the cluster. This happens when new drivers are deployed, or when existing drivers are restarted following a configuration change.

At the time of server startup, UMS drivers typically establish a connection to the external gateway for which they are configured. Some of these are persistent socket-level connections, such as SMPP and XMPP. Some connections are established and torn down for each request, such as SMTP and IMAP connections. Drivers then make a remote EJB call to the UMS server to register themselves.

UMS Client Application Startup

When the managed server where the clients are deployed starts up, EJB clients typically make a remote EJB call to register themselves with the UMS server. Web service clients do not have explicit registration mechanisms.

5.9.1.3 Oracle User Messaging Service Request Flow

UMS clients make a series of short-lived requests to the UMS server. Web service clients make requests using SOAP or HTTP. EJB clients make remote EJB calls to the server for initialization and configuration management, in addition to interacting using JMS.

After initial startup and registration, subsequent UMS request flows can be categorized as follows:

  • Outbound messaging: For an outbound message, a client application wishes to send a message to a user. The client application either makes a Web service request to the UMS Server, or enqueues a message in the UMS Server's sending JMS queue. The UMS Server receives the request, records the messaging state, and selects an appropriate driver. The UMS Server enqueues the message in a different outbound JMS queue. The corresponding driver, which is listening to the outbound queue, dequeues the message, converts it to the appropriate format for the given protocol, and delivers it to an external gateway (such as an SMTP server or IM gateway.)

  • Inbound messaging: For an inbound message, the end user wishes to send a message back to the client application. The user sends a message from a device, such as an IM client or phone. The message passes from the external gateway to the driver. The driver adapts the message to the UMS format, and enqueues it in a JMS queue. The server dequeues the message from the JMS queue, records the message state, and enqueues the message in the appropriate application's inbound JMS queue. The client application then dequeues the message and processes it as needed.

5.9.1.4 Oracle User Messaging Service Configuration Artifacts

The UMS Server and drivers each have an XML configuration file that is the artifact of a configuration Mbean, implemented using Oracle Fusion Middleware JMX Framework. You can perform configuration changes using Oracle Enterprise Manager Fusion Middleware Control.

Figure 5-29 Configuring Oracle User Messaging Service using Oracle Enterprise Manager Fusion Middleware Control

Configuring Oracle User Messaging Service
Description of "Figure 5-29 Configuring Oracle User Messaging Service using Oracle Enterprise Manager Fusion Middleware Control"

All UMS components use standard Java EE deployment descriptors and Oracle WebLogic proprietary descriptors for configuring Java EE components. These descriptors can be configured using standard Oracle WebLogic tools, such as Oracle WebLogic Server Administration Console or WLST. For more information, see UMS administration topics in Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite.

If you are using Oracle SOA Suite in a clustered environment, any configuration property changes you make in Oracle Enterprise Manager on one node must be made on all nodes. Configuration properties are set in Oracle Enterprise Manager through the following options of the SOA Infrastructure menu:

Administration > System MBean Browser

SOA Administration > any property selections.

5.9.2 Oracle User Messaging Service High Availability Architecture and Failover Considerations

See Section 5.2.2, "Oracle SOA Service Infrastructure High Availability Architecture and Failover Considerations" for a description of the Oracle SOA Service Infrastructure two-node high availability characteristics. Figure 5-6 illustrates an Oracle SOA Service Infrastructure cluster running on two WebLogic Servers. Oracle User Messaging Service is deployed as part of the Oracle SOA Service infrastructure composite application. The following high availability characteristics are specific to an Oracle User Messaging Service high availability deployment:

  • All UMS components are deployed to Oracle WebLogic Service Infrastructure managed servers that are part of an Oracle WebLogic Server cluster. An Oracle WebLogic Server cluster synchronizes configuration for common artifacts of Oracle WebLogic Server used by UMS, such as JDBC data sources.

  • All Oracle UMS components are stateless.

  • UMS Server and Client stateless EJBs leverage clustering and high availability capabilities of Oracle WebLogic Server cluster

  • UMS Server relies on a shared database repository for persistent storage.

  • UMS relies on JMS distributed destinations for load balancing and availability across cluster nodes. It also relies on the JMS connection factory's capability to failover to a different JMS server in the event of a failure.

  • The user messaging preferences user interface does not require session stickiness. It remains available through the use of a basic load balancing. There are no sticky session routing requirements, as all session state is persisted in the database and shared across the clustered instances.

  • UMS does not participate in any global transactions.

  • UMS uses Oracle WebLogic Server multi data source to connect to the back-end Oracle RAC database.

5.9.2.1 Oracle User Messaging Service Protection from Failures and Expected Behavior

Oracle UMS is typically deployed on the same managed server as the Oracle SOA Service Infrastructure. The Oracle SOA Service Infrastructure is protected from process and node failures using Oracle WebLogic Server whole server migration. Whole server migration also provides failover capabilities for JMS usage.

For information about Oracle UMS protection from failures and expected behavior, see Section 5.2.2.1, "Oracle SOA Service Infrastructure Protection from Failures and Expected Behavior".

Process Failure

This section describes specific considerations for process failure in an Oracle User Messaging Service high availability configuration:

  • Node Manager attempts to restart the managed servers locally if a crash occurs.

  • If whole server migration is configured for the Oracle WebLogic Server managed server to which Oracle UMS components are deployed, and the restart count threshold is exceeded, Oracle WebLogic Server infrastructure attempts to perform a server migration of the managed server to another node in the cluster. After the server migration completes successfully, at the startup time, UMS Server and Drivers go through the startup cycle as previously described, including driver registration. Driver registration is an independent operation and does not have any affect on other available instances. See Chapter 3, "High Availability for WebLogic Server" for more information about server migration.

  • At restart (on the same or different node), the UMS JMS server in the managed server starts producing and consuming messages from its JMS store.

  • If a managed server running UMS Server and Drivers is restarted or migrated, the failover is transparent to the connected UMS Clients. The failover is transparent because UMS components are stateless. Once the server's restart is finished, the Web server starts routing requests to it for Web service clients. Similarly, EJB clients become aware of the server availability and start routing requests to it. This is made possible by Oracle WebLogic clustering infrastructure.

Node Failure

For information about Oracle UMS node failure, see Section 5.2.2.1, "Oracle SOA Service Infrastructure Protection from Failures and Expected Behavior"

Database Failure

For information about Oracle UMS database failure, see Section 5.2.2.1, "Oracle SOA Service Infrastructure Protection from Failures and Expected Behavior"

Protection From External Messaging Gateway Failures

Before attempting a message delivery, UMS first persists the message to the database. If an external Messaging Gateway becomes unavailable, the corresponding UMS driver periodically attempts to reconnect to the gateway and deliver any undelivered messages persisted in the database. Alternatively, if the messages are not delivered, administrators can manually resend the messages using the UMS server's Message Status Page in Oracle Fusion Middleware Enterprise Manager.

5.9.2.2 Oracle User Messaging Service Cluster-Wide Configuration Changes

UMS configuration is file-based, standard Java EE deployment descriptors and JMX configuration Mbeans, using a standard JMX framework. Changes are propagated using standard Oracle WebLogic Server Mbean server mechanisms. There are no cluster-wide configuration capabilities. As a result, configuration changes must be repeated on every member of a cluster

UMS uses standard Java EE artifacts, configured as part of Oracle WebLogic's domain in which Oracle UMS is installed. Oracle WebLogic Server clusters provide automatic configuration synchronization for artifacts, such as data sources, across Oracle WebLogic Server domain. At the same time, Oracle WebLogic Server cluster controls synchronization of deployments and libraries used by different Oracle UMS components.

5.10 Oracle JCA Adapters and High Availability Concepts

The information in this section guides you through the issues and considerations necessary for configuring Oracle JCA Adapters for high availability.

5.10.1 Oracle JCA Adapters Single-Instance Characteristics

Oracle JCA Adapters are JCA binding components that allow the Service Infrastructure to communicate to endpoints using different protocols. Oracle JCA Adapters are deployed as a JCA resource (RAR) and are not part of the Oracle SOA Service Infrastructure.

The run-time component of Oracle JCA Adapters is the J2CA 1.5 resource adapter for the specific back-end application. Oracle JCA Adapters are deployed in J2CA container of Oracle WebLogic Server. Oracle Fusion Middleware integrates with these J2CA 1.5 adapters through the JCA Binding Component, which converts Web service messages into J2CA interactions and back. Adapters can, therefore, fully leverage the scalability and high availability of the underlying Oracle WebLogic Server platform.

Figure 5-30 illustrates the services and dependencies of Oracle JCA Adapters single-instance architecture.

Figure 5-30 Oracle JCA Adapters Single-Instance Architecture

Description of Figure 5-30 follows
Description of "Figure 5-30 Oracle JCA Adapters Single-Instance Architecture"

5.10.1.1 Oracle JCA Adapters Component Lifecycle

The lifecycle of an adapter is controlled by the Oracle SOA Service Infrastructure. There are three high-level steps in the initialization of an Oracle JCA Adapter:

  • Deployment: Oracle JCA Adapters are deployed during installation as J2CA 1.5 resource adapters (RAR files) within the same Oracle WebLogic Server container as Oracle Fusion Middleware. The physical deployment of adapters involves using the RAR file and registering the adapters as connectors with the underlying Oracle WebLogic Server, or the middle tier platform. The RAR file contains the ra.xml file, which contains declarative information about the contract between Oracle WebLogic Server and the resource adapter. In addition, the RAR file contains weblogic-ra.xml which is the deployment descriptor file containing deployment-specific information about the resource adapter.

  • Loading: Loading refers to the process of creating in-memory resources and creating connections to the configured end-point. Although Oracle JCA Adapters are physically deployed as J2CA 1.5 resource adapters, their logical deployment involves creating the Connection Factory entries for the J2CA 1.5 resource adapter by editing the weblogic-ra.xml file. This file is Oracle WebLogic Server-specific deployment descriptor for a resource adapter. It contains configurations for deploying resource adapters to Oracle WebLogic Server, which includes the back-end application connection information as specified in the deployment descriptor of the resource adapter, Java Naming and Directory Interface (JNDI) name to be used, connection pooling parameters, resource principal mapping mechanism, and configurations.

  • Activation: Activation refers to initiation of a JCA binding component (Service and Reference) within a Composite. Listeners are started for the endpoint referenced by the Adapter configuration within the Composite.

Property Changes During Oracle JCA Adapters Runtime

Oracle JCA Adapters expose the underlying back-end operation-specific properties as header properties and allow the manipulation of these elements within a business process.The underlying properties are as follows:

  • interactionspec or activationspec Properties - These properties require the adapter endpoint to be recycled (re-activated).

  • Endpoint Properties - Changes to these properties are notified to the adapter without requiring the endpoint to be restarted.

5.10.1.2 Oracle JCA Adapters Reliability and Transactional Behavior

Oracle JCA Adapters support global transactions based on the JCA 1.5 XA contracts that leverage the underlying application server transaction manager. Adapters supporting XA transactions include Oracle Adapters for Oracle Applications, database, Advanced Queuing, JMS and MQSeries. Non-transactional adapters include Oracle File Adapter and Oracle FTP Adapter.

Inbound Transactions

For a synchronous process, the global transaction initiated by the adapter spans message delivery and composite execution.

For an asynchronous service entry point, a transactional adapter initiates a global JTA transaction before sending an inbound message to a composite. When control returns to the adapter, it commits the JTA transaction, executing the following set of actions as an atomic unit of work:

  1. Commit the removal of the message from the inbound adapter endpoint, for example, table and queue.

  2. Commit the execution of the composite instance.

If anything fails during this process, both of these actions are rolled back based on XA guarantees.

Outbound Transactions

For transactional adapters, outbound JCA interactions (the invoke activities) are scoped with the global JTA transaction of the Composite instance. This means that all composite activities, including Oracle JCA adapter invocations, are part of a global transaction, and as such all activities are either committed or rolled back if an error occurs. Therefore, one can guarantee exactly-once message delivery when both inbound and outbound adapters are transactional and the connection factories have been configured to support XA global transactions.

Nontransactional

The Oracle File Adapter picks up a file from an inbound directory, processes the file, and sends the processed file to an output directory. However, during this process if a failover occurs in the Oracle RAC backend or in an SOA managed server, then the file is processed twice because of the nontransactional nature of Oracle File Adapter. As a result, there can be duplicate files in the output directory.

5.10.1.3 Oracle JCA Adapters - Rejected Message Handling

The messages that error out before being posted to the Oracle SOA Service Infrastructure mesh are referred to as rejected messages. For example, Oracle File Adapter picks a file having data in CSV format and tries to translate it to the XML format using NXSD. Now, if there is any errors in the translation, the message is rejected and is not posted to the target composite.

All rejected messages are stored in the database with payload. There are two destinations for rejected messages:

  • Adapter Rejection Table - Messages are stored in the Adapter Rejection table when there is a binding error related to a corrupt message.

    In case the database hosting the Rejected Message Table is offline, rejected messages are temporarily stored in the local file system (as a backup) and eventually loaded back into the database when it is detected as being online again.

  • Composite Instance Failure Table - This table is part of instance tracking, populated by each component detecting a failure.

In the case of errors being thrown by the SOA Service Infrastructure layer, the Adapter framework behavior depends on whether or not the error is marked as retriable. If the error is not retriable, the message is treated as a rejected message and persisted to the Rejected Message Table.

Configuring Rejection Handlers

Rejection handlers are defined and configured by using fault policies as described in the following steps:

  1. Define a fault policy for the rejected messages in the fault-policies.xml file, which is stored along with composite.xml in the Oracle JDeveloper project directory.

  2. Associate the fault policy with a service endpoint of the composite in the fault-bindings.xml file.

  3. Copy the fault-policies.xml file and the fault-bindings.xml file to Oracle SOA Composite project directory.

  4. Deploy the Oracle SOA Composite project.

Retry frequency and retry interval are both configurable. Service engines can mark an invocation as retriable. In a clustered environment, if rollback occurs, it may be picked up by another instance. If another node picks up the message, it generates a unique ID. The number of retries are configured within the composite, under the binding.jca element. For more information, see the Oracle Fusion Middleware User's Guide for Technology Adapters.

System-Defined Rejected Message Handlers

The following system-defined error handlers can be configured in fault policies:

  • Web Service Handler - A predefined WSDL interface must be implemented by the target service

  • Custom Java Handler - A predefined Java interface must be implemented by the target class.

  • JMS Queue - The rejected message is enqueued to a queue as a JMS message with the appropriate context and payload.

  • File - The rejected message is stored in the file system with the proper context and payload.

Payload Persistence

For resubmitting rejected messages, payload persistence is imperative. Payloads are stored in the database and be viewed through Oracle Enterprise Manager Fusion Middleware Control. Error handlers invoked during automatic error handling also receive the payload during their invocation. Error payload persistence in the database is enabled by default.

5.10.2 Oracle JCA Adapters High Availability Architecture and Failover Considerations

This section describes Oracle JCA Adapters high availability considerations for configuring Oracle JCA Adapters to run on an Oracle SOA cluster.

5.10.2.1 Oracle JCA Adapters High Availability Error Handling

This section describes Oracle JCA High Availability error handling.

Connection Errors

Oracle JCA Adapters and Oracle Adapters for Oracle Applications handle connection errors for the following interactions:

  • Outbound Interaction: Oracle JCA Adapters and Oracle Adapter for Oracle Applications raise the oracle.tip.adapter.api.exception.PCRetriableResourceException exception for transient connection errors, which are recoverable connection errors. For example, a database listener may not have started, resulting in connection errors. You can define the maximum number of attempts to reconnect made using the fault-policy.xml file. The parameters for attempts to reconnect can be specified using this file. After the configured number of retries is reached, the fabricInvocationException exception is thrown.

    Fault handling and retry properties are defined as part of the binding level configuration. When the binding level retry has expired, the fault is handled based on policies specified within the fault-policy.xml. For more information on binding properties, see the Oracle Fusion Middleware User's Guide for Technology Adapters.

  • Inbound Interaction: Oracle JCA Adapters, Oracle Adapter for Oracle Applications, and legacy adapters support a poll model for connecting to the back-end application for receiving events. In case of unrecoverable connection failures, the adapters recycle old connections, and send out alerts or notifications to the Oracle SOA Service Infrastructure. The inbound interaction connection errors are written to a log, and can be viewed through Oracle WebLogic Server Administration Console.

5.10.2.2 Oracle File and FTP Adapters High Availability

The Oracle File and FTP Adapters can be configured for high availability using an active-active topology. A cluster-based high availability configuration is supported for both inbound and outbound operations. File and FTP Adapters do not support XA and hence can only guarantee at-least-once delivery of messages. Thus, it is possible to have duplicate messages after recovery from a server crash.

Prerequisites for High Availability

The following list describes prerequisites for Oracle File and FTP Adapters high availability:

  • In a clustered configuration, inbound adapters across managed servers must point to the same physical directory, a directory on a shared drive.

  • Connection-factories must specify the same shared folder as the control directory, and their names must match. For example, if the deployment descriptor for one connection-factory has /shared/control_dir as the value for controlDir, the other deployment descriptor must also have the same value.

The Oracle File and FTP Adapters must ensure that only one node processes a particular file in a distributed topology. You can use the database table as a coordinator to ensure that Oracle File and FTP Adapters are highly available for inbound operations. See the Oracle Fusion Middleware User's Guide for Technology Adapters for details on configuring a database table as a coordinator.

The Oracle File and FTP Adapters must ensure that if multiple references write to the same directory, then these do not overwrite each other. The following locking capabilities can be used to make Oracle File and FTP Adapters highly available for outbound operations:

  • Database mutex

  • User-defined mutex

See the Oracle Fusion Middleware User's Guide for Technology Adapters for details on configuring a database mutex.

Oracle File and FTP Adapters High Availability Configuration

A cluster-based high availability configuration is supported for both inbound and outbound operations. However, consider the following:

  • Inbound Operations - Oracle File and FTP Adapters must ensure that only one node processes a particular file in a distributed topology.

  • Outbound Operations - Oracle File and FTP Adapters must ensure that if multiple references write to the same directory, these do not overwrite each other.

Database-based mutexes are used as coordinators to ensure these behaviors in a clustered topology.

Configuring a Database Mutex

Configure a database table as a coordinator by modifying Oracle File and FTP Adapter deployment descriptor for the connection-instance corresponding to eis/HAFileAdapter from Oracle WebLogic Server Administration Console:

  1. Click FileAdapter under Summary of Deployments on Oracle WebLogic Server Administration Console.

  2. Click the Outbound Connection Pools tab, and expand javax.resource.cci.ConnectionFactory to see the configured connection factories.

  3. Click eis/HAFileAdapter. The Outbound Connection Properties for the connection factory corresponding to high availability is displayed.

  4. Update the connection factory properties.

Update the Adapter configuration to use the connection factory as shown in the following example:

<adapter-config name="FlatStructureOut" adapter="File Adapter" xmlns="http://platform.integration.oracle/blocks/adapter/fw/metadata">
              <connection-factory location="eis/HAFileAdapter" adapterRef=""/>
              <endpoint-interaction portType="Write_ptt" operation="Write">
            <interaction-spec className="oracle.tip.adapter.file.outbound.FileInteractionSpec">
                  <property../>
                  <property../>
                </interaction-spec>
              </endpoint-interaction>
            </adapter-config>

Note:

The location attribute is set to eis/HAFileAdapter for the connection factory.

The new parameters in connection factory for Oracle File and FTP Adapters are as follows:

  • controlDir - Set this parameter to the directory structure where you want the control files to be stored. Set it to a shared location if multiple Oracle WebLogic Server instances are running in a cluster.

  • inboundDataSource - Set this parameter to jdbc/SOADataSource. This is the data source where the schemas corresponding to high availability are pre-created. The pre-created schemas are located under MW_HOME/AS11gR1SOA/rcu/integration/soainfra/sql/adapter/createschema_adapter_oracle.sql. To create the schemas elsewhere, use this script. Set the inboundDataSource property accordingly if you choose a different schema.

  • outboundDataSource - Set this parameter to jdbc/SOADataSource. This is the data source where the schemas corresponding to high availability are pre-created. The pre-created schemas can be found under MW_HOME/AS11gR1SOA/rcu/integration/soainfra/sql/adapter/createschema_adapter_oracle.sql. To create the schemas elsewhere, use this script. Set the outboundDataSource property if you do create the schemas elsewhere.

  • outboundLockTypeForWrite - Set this parameter to oracle if you are using Oracle database. By default Oracle File and FTP Adapters use an in-memory mutex to lock outbound write operations. Choose one of the following values for synchronizing write operations:

    • memory - Oracle File and FTP Adapters use an in-memory mutex to synchronize access to the file system.

    • oracle - The adapter uses Oracle database sequence.

    • db - The adapter uses a pre-created database table (FILEADAPTER_MUTEX) as the locking mechanism. Use this option only if you are using a schema other than the Oracle database schema.

    • user-defined - The adapter uses a user-defined mutex. In order to configure the user-defined mutex, implement the mutex interface: oracle.tip.adapter.file.Mutex and then configure a new binding-property with the name oracle.tip.adapter.file.mutex and value as the fully qualified class name for the mutex for the outbound reference.

Note:

For large payloads, increase transaction timeout for the SOA DataSource by adding the following:

<xa-set-transaction-timeout>true</xa-set-transaction-timeout>

<xa-transaction-timeout>1000</xa-transaction-timeout>

Increase global transaction timeouts if a database is used as the coordinator.

5.10.2.3 Oracle Database Adapters High Availability

The Oracle Database Adapter supports high availability in an active-active setup. In an active-active setup, distributed polling techniques can be used for inbound Database Adapters to ensure that the same data is not retrieved more than once. For more information on best practices for distributed polling, see the "Oracle JCA Adapter for Database" chapter in Oracle Fusion Middleware User's Guide for Technology Adapters. Similar to other adapters, an Oracle Database Adapter can also be configured for singleton behavior within an active-passive setup. This allows a high performance multi-threaded inbound Oracle Database Adapter instance running in an active-passive setup, to follow a fan out pattern and invoke multiple composite instances across a cluster. The Oracle Database Adapter also supports the high availability feature when there is a database failure or restart. The DB adapter picks up again without any message loss.

For information about how an inbound rejected message is handled by using fault policy, see the Oracle Fusion Middleware User's Guide for Technology Adapters.

Surviving Database Restart

The Oracle Database Adapter can survive a database down scenario and pick up again without any message loss. For avoiding data loss, the data source needs to be XA enabled and configured for Oracle RAC (multi data source). For information on configuring a datasource for high availability, see Appendix B, "Recommended Multi Data Sources."

5.10.2.4 Oracle JMS Adapters High Availability

The Oracle JMS Adapter supports multiple providers including WLS JMS, MQ Series, and Oracle AQ. For each provider, the high availability aspects may vary and depend on the underlying messaging infrastructure capabilities. For example, Oracle AQ requires configuring the appropriate data source for XA and Oracle RAC. For information on configuring a data source for high availability, see Appendix B, "Recommended Multi Data Sources."

Transaction control is very relevant for consistency and reliability purposes when using the Oracle JMS Adapter. The Oracle JMS Adapter supports global transactions based on the JCA 1.5 XA contracts that leverage the underlying application server transaction manager. In context of the JMS Adapter, a transaction enables an application to coordinate a group of messages for production and consumption, treating messages sent or received as a single unit. When an application commits a transaction, all messages it received within the transaction are removed by the JMS provider. The messages it sent within the transaction are delivered as one unit to all JMS consumers. If the application rolls back the transaction, then the messages it received within the transaction are returned to the messaging system and the messages it sent are discarded.

When using the WLS JMS provider, the Adapter supports XA transactions by using an XA-enabled connection factory (weblogic.jms.XAConnectionFactory, which is the default). For this provider, the high availability behavior is totally tied to the configuration of the queues and topics that are used by the adapter and to the JMS Infrastructure provided by Oracle Weblogic Server. There are two main aspects that affect the behavior of the JMS Adapter when using Oracle WLS JMS as the provider:

  • Message redelivery

  • Use of WebLogic Server distributed destinations

These two main aspects are described in the next two subsections.

5.10.2.4.1 Message Redelivery

Message redelivery plays an important role in a highly available configuration. In WLS JMS, you can delay the redelivery of messages when a temporary, external condition prevents an application from properly handling a message. This allows an application to temporarily inhibit the receipt of "poison" messages that it cannot currently handle. When a message is rolled back or recovered, the redelivery delay is the amount of time a message is put aside before an attempt is made to redeliver the message. If JMS immediately redelivers the message, the error condition may not be resolved and the application may still not be able to handle the message. However, if an application is configured for a redelivery delay, then when it rolls back or recovers a message, the message is set aside until the redelivery delay has passed, at which point the messages are made available for redelivery. All messages consumed and subsequently rolled back or recovered by a session receive the redelivery delay for that session at the time of rollback or recovery. Messages consumed by multiple sessions as part of a single user transaction will receive different redelivery delays as a function of the session that consumed the individual messages. Messages that are left unacknowledged or uncommitted by a client, either intentionally or as a result of a failure, are not assigned a redelivery delay.

A session in JMS context inherits the redelivery delay from its connection factory when the session is created. The RedeliveryDelay attribute of a connection factory is configured using the Administration Console (Configuration->Default Delivery screen). WebLogic JMS defines two default connection factories, which you can look up using the following JNDI names:

  • weblogic.jms.ConnectionFactory

  • weblogic.jms.XAConnectionFactory

The JMS Adapter uses weblogic.jms.XAConnectionFactory by default when configured for the WLS JMS Provider, therefore using the same redelivery settings as the XAConnectionFactory. For details on configuring connection factories for different adapters, refer to "Oracle JMS Adapters Use Cases" in Oracle Fusion Middleware User's Guide for Technology Adapters.

5.10.2.4.2 Use of WebLogic Server Distributed Destinations

Previous releases of Oracle JMS Adapter could not guarantee exactly-once message delivery when using some types of Oracle WebLogic Uniform Distributed Destinations. Oracle Fusion Middleware 11gR1 11.1.1.4 includes several enhancements in the WLS JMS layer that have been consumed by the JMS Adapter to better support Uniform Distributed Destinations. Shared ClientID, Partitioned Distributed Topics, and Shared Subscriptions are the main enhancements. These enhancements are intended to facilitate scalability, high availability, and also to guarantee consistent message delivery in different scenarios:

  • Shared ClientID: The shared ClientID feature allows durable subscribers to consume from distributed destinations using the same ClientID. The JMS standard requires that topic applications that consume from durable subscriptions must specify a ClientID and that the same ClientID should not be used by more than one running connection at any given time. This feature allows "overriding" this limitation.

  • Partitioned Distributed Topics: A partitioned distributed topic is a blend of uniform distributed queue and uniform distributed topic behavior. It enables messages to be load balanced among distributed topic members as in a distributed queue, and then allows for multiple receivers of the message (subscribers) on the single distributed topic member. One message published to a partitioned distributed topic will only land on one of its members, instead of all its members as in the previous uniform distributed topic implementation.

  • Shared Subscription: With Oracle 11g R1 11.1.1.4, multiple subscribers can optionally share the same subscription, durable or non-durable, by setting the connection's SubscriptionSharingPolicy to SHARABLE. When this feature is used in combination with a partitioned topic and an UNRESTRICTED connection client-id policy, it will be guaranteed that, among all subscribers that share the same distributed subscription, only one subscriber will receive a particular message sent to the topic. All subscribers that share the same distributed subscription will together process each of the messages that are published to the topic.

Additionally the WLS JMS infrastructure provides notification helpers in 11gR1 11.1.1.4 that allow for notifications of member runtime availabilities of specific distributed destinations.From the JMS Adapter perspective, these JMS features have the following impact when using distributed queues:

Inbound - Distributed Queues

The JMS Adapter automatically registers a notification listener with WLS JMS on endpoint activation. If an "unavailable" notification is received, the corresponding poller threads are stopped and the necessary cleanup is done. Similarly, if an "available" notification is received, the poller thread is started and the adapter begins consumption from the new member.

Note:

Oracle BAM's Enterprise Message Source (EMS) cannot be configured with distributed queues fed by producers load balancing to the UDD. UDD's behavior pins each consumer to one single physical destination, and this could cause missed messages on the EMS side. As an alternative, configure EMS with the standard destination cluster wide JNDI name, but force producers to pin to one single physical JMS queue (by using that particular server host rather than using a cluster-wide JNDI name).

Outbound - Distributed Queues

There is no change in the JMS Adapter behavior when producing a message to a distributed queue. Available/Unavailable/Failure notification has no impact on the working of outbound adapter reference since messages are produced by creating a MessageProducer for the distributed destination and not for a specific member.

These JMS features have the following impact when using distributed topics:

Inbound - Distributed Topics

The JMS Adapter automatically registers a notification listener with WLS JMS on endpoint activation. If an "unavailable" notification is received, the corresponding poller threads are stopped and the necessary cleanup is done. Similarly, if an "available" notification is received, the poller thread is started and the adapter begins consumption from the new member. The durable subscription is maintained in a similar fashion as in a non-distributed topic scenario. The adapter will unregister the notification listener on endpoint deactivation. Any message arriving at a distributed topic will be processed based on the various settings used and the type of distributed destination in use, as detailed below:

  • One copy of message per application: Each message is processed exactly once (no duplicate processing). The use of partitioned distributed topics and configuring the JMS adapter to use unrestricted clientid and shared subscription policy (these are the default settings) is recommended. In this scenario, the client id and subscription name are the same for everyone, and each adapter instance creates subscriptions on every member. The name is unique and immutable across server restarts. The specific clientid configuration in the connection instance (weblogic-ra.xml for a local cluster) for the adapter is required:

    <property>
    <name>FactoryProperties</name>
          <value>ClientID=SOAClient1;</value>
    </property>
    

    When using replicated distributed topics, it is also possible to force one copy of message per application. For this, it will be required to configure JMS adapter to use unrestricted clientid and shared subscription policy (the default settings), but in addition, it will require specifying the following message selector "(NOT JMS_WL_DDForwarded)" when defining an activation spec. Note that this type of configuration is highly inefficient and not recommended, because it distributes messages to all consumers but then does the rejection on the consumer end.

  • One message per adapter endpoint instance in a cluster: In this scenario either the clientid or subscription name are unique for each adapter instance. Also, the unique part of the name should be immutable across server restarts. This type of configuration is possible using both Replicated and Partitioned Topics.

    For Partitioned Distributed Topics, configure the JMS adapter to use unrestricted clientid and a shared subscription policy (that is, set "SubscriptionSharingPolicy" property to SHARABLE), which are the default settings. Also, to ensure subscription name uniqueness, set the "TopicMessageDistributionAll" property to true (default value is false). This property can be defined by setting the "FactoryProperties" property of the connection instance in weblogic-ra.xml. An example usage (snippet of connection instance from weblogic-ra.xml is:

    <property>
    <name>FactoryProperties</name>
          <value>ClientID=SOACLient2; TopicMessageDistributionAll=true</value>
    </property>
    

    For Replicated Distributed Topics, configure the JMS adapter to use unrestricted clientid and shared subscription policy as well (that is, set "SubscriptionSharingPolicy" property to SHARABLE), which are the default settings. As with the Partitioned Distributed Topic case, to achieve subscription name uniqueness, the adapter requires that the property "TopicMessageDistributionAll" be set to true (the default value is false). The property can be defined by setting the "FactoryProperties" property of the connection instance in weblogic-ra.xml. An example usage (snippet of connection instance from weblogic-ra.xml for a local cluster) is shown above.

    For Replicated Distributed Topics to guarantee one copy per adapter endpoint, the message selector "NOT JMS_WL_DDForwarded" needs to be used when defining an activation spec, as shown in the following .jca file example:

    <activation-spec className="oracle.tip.adapter.jms.inbound.JmsConsumeActivationSpec">
    <property name="DestinationName" value="jms/DemoInTopic"/>
    <property name="UseMessageListener" value="false"/>
    <property name="DurableSubscriber" value="dsub1"/>
    <property name="MessageSelector" value="NOT JMS_WL_DDForwarded"/>
    <property name="PayloadType" value="TextMessage"/>
    </activation-spec>
    

    Note, though, that this type of configuration is highly inefficient and not recommended, because it distributes messages to all consumers, but then does the rejection on the consumer end. To achieve better performance it is recommended to use Partitioned Distributed Topics.

Outbound - Distributed Topics

There is no change to adapter behavior when producing message to a distributed topic. Available/Unavailable/Failure notification shall have no impact on the working of outbound adapter reference. The message is produced by creating a MessageProducer for the Distributed Destination and not for a specific member.

Note:

Oracle BAM does not support durable distributed topics, but it does support non-durable (transient) distributed topics.

5.10.2.5 Oracle JCA Adapters Log File Locations

You can view the logs for Oracle JCA Adapters as follows:

Oracle JCA Adapters and Oracle Adapter for Oracle Applications: For both outbound and inbound interactions, the log files are redirected to the soa-diagnostic.log file.The log files for Oracle Fusion Middleware SOA Suite that is deployed to the server-soa managed server are located in:

MW_HOME/user_projects/domains/domain_name/servers/server-soa/logs/soa-diagnostic.log

Packaged-application adapters: These adapters do not implement the LogManager interface because it is not part of the J2CA 1.5 standard. Therefore, for OPMN-managed components the log outputs are redirected to

ORACLE_INSTANCE\diagnostics\logs\component_type\component_name

For outbound interactions, the logs are directed to the same location. On the other hand, for inbound interactions, logs are redirected to soa-diagnostic.log.

Legacy adapters: In addition to the J2CA resource adapter, legacy adapters consists of Oracle Connect, which consists of native adapters for communicating with the mainframe application and data stores. Oracle Connect logs can be viewed using Oracle Studio, which is the mainframe adapter design-time tool and Oracle Connect management tool. Oracle Connect generates various types of logs, such as the daemon log, workspace log, and server process log.

5.11 Oracle Business Activity Monitoring and High Availability Concepts

The information in this section guides you through the issues and considerations necessary for configuring Oracle BAM for high availability.

5.11.1 Oracle Business Activity Monitoring Single-Instance Characteristics

Oracle BAM provides the tools for monitoring business services and processes in the enterprise. It allows correlating of market indicators to the actual business process and to changing business processes quickly or taking corrective actions if the business environment changes. Oracle Business Activity Monitoring (Oracle BAM) provides the necessary tools and runtime services for creating dashboards that display real-time data inflow and define rules to send alerts under specified conditions.

Figure 5-31 illustrates the main services and dependencies that characterize an Oracle BAM instance.

Figure 5-31 Oracle Business Activity Monitoring Single-Instance Architecture

BAM Single-Instance Architecture
Description of "Figure 5-31 Oracle Business Activity Monitoring Single-Instance Architecture"

5.11.1.1 Oracle Business Activity Monitoring Component Characteristics

Oracle BAM is made up of the following components:

  • Oracle BAM Server is a set of runtime components that handle incoming data from different data sources. Oracle BAM components are also used to evaluate conditions for sending alerts to users and triggering the required actions configured for these alerts. The following are the main components in an Oracle BAM Server:

    • Active Data Cache is designed and optimized to handle large amounts of data in a real-time solution. To make data readily accessible and deliverable, it maintains real-time views of the data. The data feed to the Active Data Cache is a combination of business data sources, from data warehouse information to transactional feeds and other enterprise sources. The various data streaming technologies integrated with Oracle BAM send this information to the Active Data Cache in a continuous stream as data changes occur.

      The Active Data Cache hosts and runs the data objects, the view sets and the active view sets. It receives transactions (insert, update, and delete) to its data objects, and these data objects notify other data objects which are linked to them through lookups. Active view sets which are monitoring these data objects are notified of the changes and produce active data.

    • Event Engine: Event Engine monitors complex data conditions and implements specified rules. Rules can include a series of conditions and actions attached to an event. The Event Engine continuously monitors the information in the Active Data Cache for certain conditions and executes the related actions defined in associated rules.

      The Event Engine is responsible for tracking events based on date, time or data changes. The design of the Event Engine uses a satellite concept, in which there are four different systems (satellites), with which event clauses can be registered and in which they can be tracked.

    • Report Cache off loads the burden of maintaining the view set snapshot in memory from the Active Data Cache. The Report Cache opens view sets and active view sets in the Active Data Cache for the Report Server in Oracle BAM Web Applications set of components. It then caches the snapshot (in chunks) and the active data before sending it to the Report Server. This allows for random access into the snapshot and recovery from losing internet connectivity. The Report Cache also allows the Report Server to be stateless, and with the Active Data Cache it supports view set sharing.

  • Real Time Data Streaming Clients: In an Oracle BAM system clients can feed data into Oracle BAM server by using different types of protocols and APIs. The following are the available mechanisms to send data to the Oracle BAM server:

    • Oracle BAM Adapter is a JCA-compliant Adapter, which Java EE applications can use to send data to the Oracle BAM Server.

    • Enterprise message sources are used by applications to provide direct Java Message Service (JMS) connectivity to the Oracle BAM server by mapping messages directly to Oracle BAM data objects. The Oracle BAM server can read data directly from any JMS based message queue or topic. This option offers guaranteed messaging.

      In an Oracle BAM high availability environment, involving Enterprise Message Sources, the Enterprise Message Source property Start when BAM Server starts should be set to Yes.

      Durable Subscriber Name must be set in the Enterprise Message Source to ensure that no messages are lost during the failover.

    • Oracle Data Integrator is the ETL tool that is used with Oracle BAM to perform rigorous data transformations. The Oracle BAM Server has been implemented as an ODI Technology (for example, DB2, SQL Server are ODI Technologies) and Oracle BAM has ODI Knowledge Modules which let ODI perform all of the operations on the Oracle BAM Server to facilitate reading and writing data in various ways, including Change Data Capture.

    • Web Services API interacts directly with Oracle BAM data objects from a remote client.

  • Oracle BAM Web Applications are the Web user interfaces (Java EE Web applications) for viewing Oracle BAM data. Oracle BAM Web Applications also allow for building data models and creating dashboards and alerts. Oracle BAM Web Applications are the browser based interfaces provided for building reports and administrating the users that have access to the BAM User Interface system. As part of the Web applications, a Report Server applies the report definitions to the data sets retrieved from the Active Data Cache for presentation in a browser.

  • ICommand is a command line interface for administrating the data in the Active Data Cache. It provides a set of commands to export, import, rename, clear, and delete items from Active Data Cache.

The BAM application runs separately from the rest of Oracle Fusion Middleware SOA Suite components. Oracle BAM Web Applications are deployed together with the BAM Server. Oracle BAM Server uses EJBs and DAOs for implementing its services and persisting information to the database. All of the EJBs are stateless. However, the Oracle BAM Server is a singleton: only one active Oracle BAM server is used for maintaining fed data and for pushing it into the Web Application Reports. Oracle WebLogic Server Migration feature is used to protect Oracle BAM server from failures. Oracle BAM server uses JMS intensively. However, all Oracle BAM JMS queues are internal and do not require to be configured in a high availability configuration. When failover occurs, the queues are recreated on the new active Oracle BAM instance.

Oracle BAM Web Application Modules can run in full active-active mode connecting to the available Oracle BAM server. The information is retrieved using RMI protocol to access the EJBs that provide the information to remote consumers. Oracle BAM Web Applications are stateless, except for the Reports Server, which maintains a session ID for reports or users accessing the systems. This requires enabling session replication for the WebLogic Managed Server where Oracle BAM Web Applications run.

The processing of work by the EJBs and threads in Oracle BAM is non transactional and does not require Oracle WebLogic Server special transaction logs configuration. At the same time, Oracle BAM uses the database intensively, therefore it is important that Oracle BAM's database access is prepared for failure in the database. This requires configuring multi data sources for Oracle BAM data source as described in Section 5.15, "Configuring High Availability for Oracle BAM". For information about multi data source configuration with Oracle RAC and the MDS repository, see Section 4.1.2, "Using Multi Data Sources with Oracle RAC."

External Dependencies

Oracle BAM engine system depends only on the Oracle BAM database (which contains Oracle BAM schemas) for Oracle BAM data and report metadata. The database must be available for Oracle BAM system to start and run properly.

5.11.1.2 Oracle Business Activity Monitoring Startup/Shutdown Lifecycle

As shown in Figure 5-32, when the WebLogic Server is started, the Oracle BAM Server application is initialized. During startup, the Active Data Cache loads all the data from the repository. The Reports Cache is initialized with the data required for the available systems. The Event Engine starts analyzing the data and preparing notifications. Oracle BAM Web Applications are configured at boot time with an Oracle BAM Server's address. When Oracle BAM Web Applications are started, they connect to the Active Data Cache in Oracle BAM server and retrieve the corresponding viewsets through the Reports Server. Once initialization is done, the system is ready to receive data from clients and raise events and update reports dynamically.

Figure 5-32 Oracle Business Activity Monitoring Startup/Shutdown Lifecycle

BAM Startup/Shutdown Lifecycle
Description of "Figure 5-32 Oracle Business Activity Monitoring Startup/Shutdown Lifecycle"

5.11.1.3 Oracle Business Activity Monitoring Startup and Shutdown of Processes

Oracle BAM Server is an application, oracle-bam, that operates independently from the rest of the SOA Service Infrastructure. It runs in the managed server where Oracle BAM is installed. It gets started by default with Oracle WebLogic Managed Server to which Oracle BAM is deployed. Use Oracle WebLogic Server Administration Console to verify Oracle BAM Server's status and to start and stop it. You can also use WebLogic Server WLST command line to control the application.

Figure 5-33 Startup and Shutdown of Oracle Business Activity Monitoring using WebLogic Server

Startup and Shutdown of Oracle Business Activity Monitoring
Description of "Figure 5-33 Startup and Shutdown of Oracle Business Activity Monitoring using WebLogic Server"

Oracle Enterprise Manager Fusion Middleware Control allows multiple operations and configuration of the BAM Server as well as monitoring its status. For details about monitoring and controlling the Oracle BAM engine, Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite.

Figure 5-34 Startup and Shutdown of Oracle Business Activity Monitoring using Fusion Middleware Control

Startup and Shutdown of Oracle Business Activity Monitoring
Description of "Figure 5-34 Startup and Shutdown of Oracle Business Activity Monitoring using Fusion Middleware Control "

The BAM Web Applications run collocated by default in the same managed server as BAM Server. for example, the BAM server and BAM Web Applications are part of the same deployment in WebLogic Server Administration Console. They can't be stopped and managed separately from BAM Server by default. It is possible though, as explained in later sections, to manipulate WebLogic Server application targeting to separate both components. The Oracle WebLogic Administration Console allows starting and stopping Oracle BAM. This can also be done using WLST commands.

Oracle Enterprise Manager Fusion Middleware Control Console allows multiple operations and configuration of the BAM Web Applications.

Figure 5-35 Oracle Enterprise Manager Fusion Middleware Control

Oracle Enterprise Manager Fusion Middleware Control
Description of "Figure 5-35 Oracle Enterprise Manager Fusion Middleware Control"

Note:

Although Enterprise Manager offers separate start and stop for Oracle BAM Server and Oracle BAM Web Applications, this is in reality a stop operations that affects both components, meaning if Oracle BAM Server is stopped from Enterprise Manager, the corresponding Oracle BAM Web Applications are also stopped, and the reverse is also true. This also applies for start operations.

5.11.1.4 Oracle Business Activity Monitoring Configuration Artifacts

Oracle Enterprise Manager Fusion Middleware Control exposes some of the configuration options for Oracle BAM Server.

Figure 5-36 Configuring Oracle Business Activity Monitoring

Configuring Oracle Business Activity Monitoring
Description of "Figure 5-36 Configuring Oracle Business Activity Monitoring "

The properties exposed by Oracle Enterprise Manager are a mix of the information contained in two different files: BAMServerConfig.xml and BAMCommonConfig.xml. These files are located under the DOMAIN_HOME/servers/BAM_Server_Name/tmp/_WL_user/oracle-bam_11.1.1/yhryfp/APP-INF/classes/config directory (notice that yhryfp is the random directory generated at installation time for deploying BAM applications). This is the random directory generated at installation time for deploying Oracle BAM applications. The properties in these files are modifiable by using the Mbeans exposed in Oracle Enterprise Manager Fusion Middleware Control. For details on the configuration options for Oracle BAM Servers, refer to the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite and the Oracle Fusion Middleware User's Guide for Oracle Business Activity Monitoring.

Similarly, some properties are exposed for Oracle BAM Web Applications by Oracle Enterprise Manager Fusion Middleware Control as shown in Figure 5-37.

Figure 5-37 Oracle Business Activity Monitoring Configuration Properties

Oracle Business Activity Monitoring Configuration Properties
Description of "Figure 5-37 Oracle Business Activity Monitoring Configuration Properties"

Configuration options at the container level, such as data sources and persistent stores, are maintained as part of Oracle WebLogic Server Domain configuration and are synchronized across a cluster of Oracle WebLogic Servers by Oracle WebLogic Server core infrastructure. See the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite for details on configuring Oracle BAM.

5.11.2 Oracle Business Activity Monitoring High Availability Architecture and Failover Considerations

Figure 5-38 describes a two-node Oracle BAM cluster running on two Oracle WebLogic Servers. The Oracle WebLogic Servers are frontended by Oracle HTTP Servers which load balance incoming requests to them. The following are the main characteristics of this configuration:

  • Oracle BAM Web Applications run on two clustered WebLogic Server managed servers. The WebLogic Server Cluster synchronizes configuration for common artifacts of WebLogic Server used by Oracle BAM Web Applications, such as data sources, persistent store, and definitions. Oracle BAM Server is targeted by any of the servers where BAM Web applications are running. Only one WebLogic server runs Oracle BAM Server.

  • The architecture uses Oracle WebLogic Server Migration feature to protect Oracle WebLogic Server that runs both Oracle BAM Web Applications and Oracle BAM Server (in BAMHOST1) against failures. This provides protection for Oracle BAM Server which is a singleton. The WebLogic Managed Server in which Oracle BAM Server runs is listening on a Virtual IP that gets migrated to another node when the failover occurs. This is the address that Oracle BAM Web Applications in BAMHOST2 use to connect to an Oracle BAM Server. Plan appropriately to account for the scenario where Oracle BAM Server and two instances of Oracle BAM Web Applications are running on BAMHOST2. For more information on Server Migration features, see Chapter 3, "High Availability for WebLogic Server."

  • Oracle BAM's database is configured with Oracle Real Application Clusters (Oracle RAC) to protect from database failures. Oracle BAM Server performs the appropriate reconnection and operations retries if database instance failure occurs.

Figure 5-38 Oracle BAM High Availability Architecture

Description of Figure 5-38 follows
Description of "Figure 5-38 Oracle BAM High Availability Architecture"

5.11.2.1 Oracle Business Activity Monitoring Protection from Failures and Expected Behavior

Oracle BAM Server and Oracle BAM Web applications are protected from all process failures by the WebLogic Server infrastructure. This section also describes expected behavior in the event of component failure.

Process Failure

Oracle BAM Server and BAM Web Applications are protected from all process failures by the WLS infrastructure. This section describes process failure considerations for Oracle BAM.

  • If the WLS_BAMx server crashes, Node Manager attempts to restart it locally. It attempts to restart the servers according to the configured restart count threshold.

    • For failures related to the WebLogic Server on which the BAM Server is running, if repeated restarts fail, the WebLogic Server infrastructure attempts to perform a server migration to the other node in the cluster. Ongoing requests from the clients time out during failover. Once the server's restart completes on the other node, the clients should be restarted to continue routing to it. For opened reports running in the Web browser, if a request is delivered though the BAM WebApps running in the same managed server as BAM Server, a Reconnecting message appears in the BAM WebApps report. If the request is being delivered through the BAM WebApps running in the other node, no message appears. In this situation, and during failover, you may be viewing stale data (while a failover takes place or while connection from webapps to servers is reestablished). The BAMWebApps Servlet remains available, and reconnects as soon as BAM Server comes up. For new reports or requests, BAM Web Applications inBAMHOST2 reconnect once the VIP is migrated and the managed server is restarted. BAM Web Applications listening on VIP1 become functional once the server migration completes. At this point, the HTTP Server restarts routing HTTP requests to the Managed Server.

    • Failures affecting the WebLogic Server where only the BAM Web Applications are running, do not affect the BAM system. Other BAM Web Applications (running in BAMHOST1) remain available and maintain session information by using the WebLogic Server Session replication cluster. Failover should be transparent.

  • The oracle-bam application where BAM Server and BAM Web Applications run may be down due to failure in accessing resources, errors in reading the database, or other issues unrelated to the status of the managed server where it is located. Therefore, you should monitor the Oracle SOA Service Infrastructure application and watch for errors caused by the application in the managed server logs, for log file locations, see Section 5.2.1.6, "Oracle SOA Service Infrastructure Log File Locations".

  • Failover may not succeed for an open report if when the report is opened, only one BAM managed server in the cluster was up, and after starting an additional managed server in the cluster, no other operations were performed. One trigger for session state replication is opening a report. Session state replication is not triggered by active data updates.

In an Oracle BAM high availability environment, when the Oracle BAM Active Viewer is running on an Oracle BAM server that is shut down and then restarted, a "viewset not found" error will be written to the server's log. This error does not affect the functionality of the Oracle BAM Active Viewer and can be ignored.

Node Failure

For node failures in SOAHOST2, the behavior in case of a node failure is equivalent to the process failure scenario: Oracle BAM Web Applications in the other node remain available and can serve requests. Sessions are preserved by the session replication framework provided by WebLogic Server and failover to the other node should be transparent.

For node failures in SOAHOST1, server migration is triggered after the available server verifies the time stamp in the database leasing system. While failover occurs, clients are unable to feed data into the system and retry appropriately. Oracle BAM Web Applications connecting to the server attempt to reconnect until the VIP is migrated and the server is restarted.

Database Failure

For information about Oracle BAM database failure, see Section 5.2.2.1, "Oracle SOA Service Infrastructure Protection from Failures and Expected Behavior"

5.11.2.2 Oracle Business Activity Monitoring Cluster-Wide Configuration Changes

The standard Java EE artifacts that Oracle BAM Server and Oracle BAM Web Application use are configured as part of Oracle WebLogic Domain in which Oracle BAM is installed. Oracle WebLogic Clusters provide automatic configuration synchronization for artifacts such as data sources, persistent stores, and JMS modules, across the WebLogic Server domain. At the same time, the WebLogic Server cluster is in charge of synchronizing the deployments, and libraries used by Oracle BAM Web Applications and Oracle BAM Server.

As explained in the single instance section Oracle BAM Server's and Oracle BAM Web Applications configuration are maintained in the DOMAIN_HOME/servers/BAM_Server_Name/tmp/_WL_user/oracle-bam_11.1.1/yhryfp/APP-INF/classes/config directory (notice that yhryfp is the random directory generated at installation time for deploying BAM applications). The properties in these files can be modified by using the Mbeans exposed in Oracle Enterprise Manager Fusion Middleware Control. The properties exposed through MBeans are specific to each server. The properties exposed through Enterprise Manager-specific screens are cluster-wide and are only modified on one server. All properties, whether applied in Enterprise Manager or in an MBean browser require a restart of Oracle WebLogic Servers where Oracle BAM runs. For details on the configuration options for BAM Server and BAM Web Applications see the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite and the Oracle Fusion Middleware User's Guide for Oracle Business Activity Monitoring.

One of the configuration options related to high availability environments is the Application URL which is used to determine the front end host used by the system in a cluster configuration. This option is used to produce the copy shortcut URL for reports and alerts. The other relevant parameter for Oracle BAM high availability configuration is the Server Name in OracleBAM Web configuration screen. This parameter is used by Oracle Web Application to determine Oracle BAM server to which it connects for accessing the Active Data Cache.

For SOA high availability installations frontended by Oracle HTTP Server, monitoring should be done on the Oracle HTTP Server ports of the real backend servers. This is the case when a deployment is using all the components deployed to the SOA Managed Server. A simple HTTP monitor that pings the HTTP/HTTPS port and expects a pre-determined response in turn should suffice. If only a specific SOA component is being used (such as B2B), then a monitor that does a deeper level check all the way to the Managed server can be considered to validate the health of the component in use. Please check with your load balancer vendor on setting up the HTTP monitors with your load balancer.

5.11.2.3 Considerations for BAM Client Retries

Oracle BAM clients communicating with the BAM Server will retry "retryable" methods if the failure of the method execution was due to a "retryable" exception. A typical "retryable" exception is one that indicates a failure to communicate with the BAM Server (like a network connection exception). Some of the BAM Server's method are automatically retried by the BAM client when a failure occurs. Additionally the list of exceptions to be retried can be extended. This can be done by adding a comma-separated list of exception class names to be treated as retryable inside the following configuration fields in the clients' configuration files:

<BamEjbRetryableExceptions></BamEjbRetryableExceptions>

Other configuration parameters for retryable exceptions are:

  • <BamEjbRetryCount>180</BamEjbRetryCount>: This parameter configures the number of retries to be attempted if getSession or retryable method invocations fail due to a retryable exception. The default is 180.

  • <BamEjbRetryInterval>10000</BamEjbRetryInterval>: This parameter configures the number of milliseconds to pause between retry attempts. The default is 10000 milliseconds or 10 seconds.

These parameters can be added in the configuration files for the following clients accessing the BAM Server:

  • BAM Server, BAM WEB, BAM Data Control: BamCommonConfig.xml configuration file

  • ICommand: BAMICommandConfig.xml configuration file

  • ODI: BAMODIConfig.xml configuration file

  • BAM Adapter: BAMAdapterConfig.xml configuration file

5.12 Oracle Service Bus and High Availability Concepts

Oracle Service Bus is a proven Enterprise Service Bus (ESB) built from the ground up for SOA lifecycle management that provides foundation capabilities for service discovery and intermediation, rapid service provisioning and deployment, and governance.

Figure 5-39 shows the high level functional features of Oracle Service Bus.

Figure 5-39 Oracle Service Bus Functional Features

Description of Figure 5-39 follows
Description of "Figure 5-39 Oracle Service Bus Functional Features"

For more detailed information about the functional capabilities of Oracle Service Bus, see Oracle Fusion Middleware Concepts and Architecture for Oracle Service Bus.

5.12.1 Oracle Service Bus Single Node Characteristics

Oracle Service Bus is a stateless Java EE application that runs on Oracle WebLogic Server.

Oracle Service Bus uses the following Java EE technologies:

  • Servlets: Oracle Service Bus Web Console, Service Bus resource servlet, WSIL servlet, and HTTP Proxy Services.

  • EJBs: Are used for:

    • The Java EJB protocol proxy service. This service, also known as JEJB, is used to expose proxy services as EJBs.

    • Service Bus transport. Also known as SB, this protocol is used for communication between two Oracle Service Bus services.

    • The test console for testing web services.

  • JMS: Is used for inbound message processing for the FTP, SFTP, Email, and File transports. It is also used for the Oracle Service Bus reporting framework and for Web Services Reliable Messaging (WS-RM).

  • JDBC Data Source: Oracle Service Bus defines a data source for the database tables associated with the JMS Reporting provider.

  • JCA Adapters: Oracle Service Bus supports the DB, Oracle Applications, AQ, File, BAM, FTP, SAP, Siebel, PeopleSoft, JD Edwards adapters using JCA transport. The adapter implementation for these adapters is same as the one used in SOA Suite.

Oracle Service Bus also uses the following non-Java EE components:

  • Configuration Framework: The configuration framework component provides CRUD (Create, Read, Update, Delete) capabilities for Oracle Service Bus resources along with resource and file integrity protection, session management, caching and indexing, referential integrity, and configuration propagation. The Oracle WebLogic Server Administration Server has the master configuration in files and these are propagated to the managed servers using Weblogic Deployment Service.

  • Monitoring and Alert Framework: The monitoring framework defines statistics based on counters (message, error counts), intervals (min/max/avg. response or execution time) and status (end point up/down). The monitoring framework has the following sub-components:

    • Collector: collects statistics at one minute intervals called checkpoints, stores collected statistics to Weblogic Persistent Store, and sends the collected statistics to the Aggregator. Collector runs on each managed server in the cluster.

    • Aggregator: aggregates statistics received from each of the managed servers in the cluster. Aggregator is a singleton service that runs on only one managed server in the cluster.

    • Retriever: retrieves the statistics that are stored in the memory. This is present only in the managed server, which contains the Aggregator.

    • Alert Manager: The Alert Manager fires alerts based on the aggregated statistics. This is present only in the managed server that contains the Aggregator. The Aggregator pushes aggregated statistics to the Alert Manager when there is an aggregation interval (defined for a service or an alert rule) for which rules need evaluation. The Alert Manager is triggered in a separate thread than the Aggregator.

  • Transport SDK: Oracle Service Bus processes messages independently of how they flow into or out of the system. The Transport SDK provides a layer of abstraction between Oracle Service Bus and components that deal with the flow of data in and out of Oracle Service Bus. This layer of abstraction allows users to develop new transport providers to handle unique transport protocols. A transport provider implements the interfaces of the Transport SDK and provides a bridge between Oracle Service Bus and mechanisms by which messages are sent or received. A transport provider manages the life cycle and runtime behavior of transport endpoints. An endpoint is a resource where messages originate or are targeted.

  • Message Flow Engine: The Message Flow Engine provides execution of the message flow logic using actions configured by the message flow designer.

  • Split-Join Engine: The Split-Join Engine provides execution of logic specified in Split-Join, which is used for performing high-performance, in-memory processing of messages in parallel.

Figure 5-40 is a high-level architecture diagram that illustrates Oracle Service Bus and its functional subsystems.

Figure 5-40 Oracle Service Bus Single Instance Architecture

Description of Figure 5-40 follows
Description of "Figure 5-40 Oracle Service Bus Single Instance Architecture"

When Oracle Service Bus is deployed in a high availability configuration, the Aggregator and Alert Manager components are singletons that should be deployed only on one host in the cluster.

Customers can write custom client applications using the Oracle Service Bus extensibility framework, such as the Transport SDK and reporting framework.

For more information about the Oracle Service Bus architecture, see the "Oracle Service Bus Architecture" chapter in Oracle Fusion Middleware Concepts and Architecture for Oracle Service Bus.

5.12.1.1 Oracle Service Bus Session State

Oracle Service Bus is optimized for short-lived message processing/mediation use cases. Thus, all message context state is kept in memory. For synchronous transports such as HTTP, EJB, and SB, the client must retry if the Oracle Service Bus server fails in the middle of processing the synchronous request. Oracle Service Bus supports request/response pattern for asynchronous transports such as JMS and WS (WS-RM). In these cases correlation information, which maps the request to its corresponding response destination, is kept in memory. If the Oracle Service Bus server fails before the response is sent, the correlation information is lost. Therefore, Oracle Service Bus will fail to send the response after server restart. For more details about JMS transports and WS transports, see the "JMS Transport" chapter and the "WS Transport" chapter in the Oracle Fusion Middleware Developer's Guide for Oracle Service Bus.

5.12.1.2 Oracle Service Bus External Dependencies

Oracle Service Bus has the following external dependencies:

  • The JMS Reporting Provider uses a database to store the report data.

  • UDDI (Oracle Service Registry) is used to store Oracle Service Bus proxy services and to browse services to create Oracle Service Bus business services.

  • Oracle Enterprise Repository is used to browse and consume services. The result is generation of Oracle Service Bus business services. Oracle Service Bus proxy services are harvested into Oracle Enterprise Repository.

  • Oracle Service Bus implements an Oracle Web Services Manager (OWSM) agent that enforces OWSM security policies attached to Oracle Service Bus proxy and business services.

  • Oracle Service Bus depends on JCA Adapter implementations (JCA Adapters for HTTP, FTP, and so on) and the JCA framework from SOA Suite.

  • Oracle Service Bus depends on the Coherence infrastructure for Service Result Caching (caching of results from services invoked by Oracle Service Bus).

Oracle Service Bus message flows are short lived in general. Message flow state is always in memory. Oracle Service Bus provides various out of box transports. For example, the HTTP transport supports HTTP 1.1 persistent connections.

Oracle Service Bus supports the following transports: HTTP, JMS, Email, File, FTP, SFTP, JCA - DB, AQ, Oracle Applications, Peoplesoft, Siebel, SAP, JD Edwards, File, FTP, JEJB, EJB (outbound only), Local (within same JVM), MQ, SB (RMI based, between two OSB servers/domains), Tuxedo, Web Services Reliable Messaging (WS-RM), SOA-Direct (RMI based), BPEL 10g (RMI based).

Oracle Service Bus uses JTA for some transport types, including JMS, File, FTP, Email, SFTP, WS-RM, JCA - DB, AQ, SB, JEJB, EJB, SOA-DIRECT, and BPEL 10g.

5.12.1.3 Oracle Service Bus Configuration Artifacts

The main configuration files for Oracle Service Bus are:

  • DOMAIN_HOME/config/config.xml: has all applications and libraries, JMS system resources, JDBC system resources, work manager, startup/shutdown classes, SAF agent, security configuration.

  • DOMAIN_HOME/config/osb/coherence/osb-coherence-override.xml: This Oracle Coherence override file specifies Oracle Coherence unicast/multicast listener information. This file is propagated by the Administration Server from the Administration Server domain directory to the domain directory for other managed servers.

  • DOMAIN_HOME/config/osb/coherence/osb-coherence-cache-config.xml: This Coherence cache configuration file defines the cache used by the Oracle Service Bus service result caching feature.

  • DOMAIN_HOME/config/osb/transports/sftp/known_hosts: This file is used by the SFTP transport. It contains information about remote SFTP servers. See Oracle Fusion Middleware Developer's Guide for Oracle Service Bus for information about creating the known_hosts file. Note that virtual host names can be used in the known_hosts file.

5.12.1.4 Oracle Service Bus Deployment Artifacts

You can use Eclipse IDE or the Oracle Service Bus web console to export your Oracle Service Bus artifacts (resources), such as proxy service, business service, wsdl, xsd, and xslt resources, to a jar file with a name that you choose. The jar file contains all the Oracle Service Bus resources that were exported. You have two ways to import the configuration jar file (and the Oracle Service Bus resources inside it) into an Oracle Service Bus server:

  • WLST/ANT scripts using public Oracle Service Bus APIs

  • Oracle Service Bus web console: Using the Import from Configuration Jar option

Oracle Service Bus resources are managed by the configuration framework component. The configuration framework stores all its data in DOMAIN_HOME/osb/config (CONFIG_HOME) on each managed server (and admin server). Each resource is stored in its own file in CONFIG_HOME/core. The Oracle Service Bus configuration framework automatically assigns unique file names to the file used for each resource to ensure that no name collisions occur among these files. The CONFIG_HOME/core directory is the master view of all the resources and configuration data. The runtime works off of this data. It changes only when changes made in a session are activated. Each managed server has a copy of the resources in CONFIG_HOME/core. Depending on the location of the managed server, this directory could be mounted on a separate system.

5.12.1.5 Oracle Service Bus Startup and Shutdown

You can use Node Manager or the startWebLogic/stopWebLogic scripts to start and stop the Oracle Server Bus server. Oracle Enterprise Manager Grid Control supports management and monitoring of Oracle Service Bus services.

Oracle Service Bus is initialized through application listeners. The ALSB Framework Starter Application does the main framework initialization through an application listener. This application is deployed to the Administration Server and the cluster. The prestart for this application listener initializes Logging, ALSBConfigService, Security service, cluster timer service, and ALSBStatisticsManager. XBus Kernel is the next important application deployed that performs the rest of the initialization and start procedures for the Oracle Service Bus components. This application is targeted to the Administration Server and cluster, and it initializes upgraders, resources, split-join, Coherence, and so on.

Death detection capabilities are provided by Oracle WebLogic Server.

5.12.1.6 Oracle Service Bus Log File Locations

Oracle Service Bus operations are logged to the Oracle WebLogic Managed Server where the application is running:

DOMAIN_HOME/servers/WLS_ServerName/logs/WLS_ServerName.log

Oracle Service Bus also supports Debug logging, which can be enabled or disabled by updating DOMAIN_HOME/alsbdebug.xml and DOMAIN_HOME/configfwkdebug.xml.

5.12.2 Oracle Service Bus High Availability Architecture and Failover Considerations

Figure 5-41 shows a two-node Oracle Service Bus cluster running on two Oracle WebLogic Servers. The Oracle WebLogic Servers are frontended by two Oracle HTTP Servers on web tier hosts, which receive requests from a load balancer in front of them.

Figure 5-41 Oracle Service Bus High Availability Architecture

Description of Figure 5-41 follows
Description of "Figure 5-41 Oracle Service Bus High Availability Architecture"

The main characteristics of the high availability configuration in Figure 5-41 are:

  • The Oracle WebLogic Server server migration feature is used to protect some of the components against failure. This implies that each of the WebLogic managed servers in which Oracle Service Bus runs is listening on a virtual IP that is migrated to another box upon failover. Enterprise Manager is deployed to the Administration Server in Figure 5-41.

    As shown in Figure 5-41, the WLS_OSB1 managed server listens on VIP1, and the WLS_OSB2 managed server listens on VIP2. Each managed server uses the other node as a failover resource, therefore, the system is configured in a cross manner. In other words, WLS_OSB1 fails over to OSBHOST2, and WLS_OSB2 fails over to OSBHOST1. The appropriate capacity planning must be done to anticipate the scenario where the two Oracle Service Bus managed servers are running on the same node. For more information on server migration features, see Section 3.9, "Whole Server Migration."

    To resume transactions after a server migration, configure the transaction and JMS stores in a shared storage. In case of failure in one of the server infrastructure instances, other instances can resume transactions and JMS operation by reading the persistent stores from that shared storage.

  • The Administration Server runs in Active-Passive mode. Whenever a failure occurs in OSBHOST1, the Administration Server can be restarted in OSBHOST2. Therefore it uses a virtual IP or virtual hostname as listen address. For information about configuring virtual IPs for the Administration Server and configuring the Administration Server for high availability, see Chapter 12, "Active-Passive Topologies for Oracle Fusion Middleware High Availability."

  • An Oracle RAC database is recommended to protect from database failures. In this Oracle Service Bus high availability configuration, an Oracle RAC database is used for the JMS Reporting Provider and Oracle Web Services Manager policies.

Note:

To keep the Oracle Service Bus high availability configuration figure above readable, the Oracle SOA Service Infrastructure high availability configuration is not shown. See Figure 5-6 to view the Oracle SOA Service Infrastructure high availability figure.

The ALSB Cluster Singleton Marker Application determines the managed server in the cluster that runs the singleton monitoring Aggregator and Alert Manager. Similarly the Message Reporting Purger MDB, which helps purging reporting data from the database (for the JMS Reporting Provider), is deployed on only one managed server in the cluster. The remaining Oracle Service Bus components are all either targeted to the cluster, or to the cluster and Administration Server. Only the ServiceBus_Console, which provides Web console support for Oracle Service Bus, and the UDDI Manager, which registers resource types for UDDI import with the configuration framework are targeted to the Administration Server only. These components are not defined as Weblogic Singleton services. Their high availability is assured by Weblogic whole server migration.

All Oracle Service Bus services and resources are deployed uniformly to the cluster (to all managed servers in the cluster). The only exceptions to this are:

  • Inbound (Proxy Service) endpoints for the poller transports: Email, File, FTP, and SFTP. The poller polls for messages on only one managed server in the cluster. The user configuration determines the managed server that polls.

  • Inbound (Proxy Service) endpoints for JMS transport where the JMS transport is listening to a Weblogic JMS Topic destination (using compatibility Topic Message Distribution Mode) and is configured to process the message from the Topic only once (in the cluster) on the managed server configured by the user.

The basic highly available topology is a Weblogic Server homogenous cluster (except for the Oracle Service Bus singletons services mentioned above) with one Administration Server and two managed servers running on different systems.

Only one cluster of Oracle Service Bus is supported in a Weblogic domain. Local data should be stored on shared storage, such as a SAN storage system, multiported disk, or NAS storage.

The local data (accessed as local files, private to a managed server or Administration Server) includes:

  • System files like Weblogic configuration files and server logs. Optionally, WLS JMS data can be stored in an Oracle RAC database. JMS is used both internally in Service Bus and as transports to external and proxy services.

  • Oracle Service Bus data like configuration files and logs

  • User-defined Oracle Service Bus configuration data

  • User files, such as data files, that are read or written by a proxy service with a File/FTP transport

  • The JMS persistent store, which is used to store alert logs and aggregated performance metrics

The Oracle RAC database is used for high availability of the reporting provider and also for the leasing datasource used for server migration.

Optionally, a web server farm may be used as a front end to an Oracle Service Bus cluster. A hardware load balancer can be used to load balance the web servers or application servers directly. If web servers are used, WebLogic plugins should be configured to load balance HTTP traffic among the Oracle Service Bus managed servers.

Email, FTP, or NFS servers can exist in the network, and third party JMS servers or server clusters can also exist in the network.

5.12.2.1 Oracle Service Bus Protection from Failures and Expected Behavior

This section describes how an Oracle Service Bus high availability cluster deployment protects components from failure. This section also describes expected behavior in the event of component failure.

Oracle Service Bus is protected from all process failures by the WebLogic Server infrastructure.

5.12.2.1.1 WebLogic Server Failure

Oracle Service Bus does not maintain any state, nor does it support the concept of user sessions. Therefore, Oracle Service Bus does not implement session state replication and failover. For synchronous inbound transports such as HTTP, SB, and EJB, if the managed server processing the request goes down in the middle of request processing, the client receives a connection exception and needs to retry.

Node Manager should be configured to perform automatic WebLogic server migration when a managed server fails. For details on WebLogic server migration, see Section 3.9, "Whole Server Migration."

Use WebLogic server migration for JMS failover.

There is no automatic failover for Oracle Service Bus singleton components such as the Aggregator, Alert Manager, and Reporting Message Purger. Use WebLogic server migration to perform the failover of singleton components.

5.12.2.1.2 Node Failure

The external load balancer can be used to directly load balance HTTP requests to Oracle Service Bus HTTP Proxy Services on the managed servers in the cluster or to a Web Server/Oracle HTTP Server (OHS) cluster front-ending the Oracle Service Bus cluster. There are no sticky session routing requirements.

Node Manager can be configured to failover failed Oracle Service Bus nodes.

If an Oracle Service Bus managed server fails in the middle of processing HTTP requests, clients need to submit those requests again. For JEJB and SB transports, a client gets a connection error when the server goes down. For all poller transports, the integrity of message processing is handled by transactional semantics as the message is dequeued using XA connection factory. Poller transports provide "at least once" semantics. However, after whole server migration (of the failed node), the poller needs to have access to the resources it is polling. JTA transactions will also preserve transactional integrity when using JMS transport. JTA TLogs need to be recovered to recover in-flight JTA transactions.

Oracle Service Bus guarantee "Exactly Once" message delivery semantics. This behavior is controlled by the QoS property in the $outbound context variable configured in the proxy service. For more detail on the different QoS settings, see Oracle Fusion Middleware Administrator's Guide for Oracle Service Bus.

When a managed server starts up, any in-flight global transactions are recovered. For example, recovery of global transactions may be required when the proxy service is bridging between two JMS providers using XA connection factories, and QoS of exactly once. Another example is the JMS reporting provider. With the default JMS Reporting Provider, the report message is first written to the JMS queue. The MDB dequeues the report message from the JMS queue and then writes it to the database. The (database) data source is configured with transaction semantics of LLR (Last Logging Resource optimization). In this case, the database must be running during recovery, otherwise the server will not start. The Tlog is used even when LLR is in effect. However, this happens only on a per transaction basis. Therefore the Tlog must still be made highly available even when all transactions are LLR transactions. Transaction Manager persists checkpoint TLog records that are unrelated to specific transactions, but that are still required to provide full transactional safety.

There are two cases where failure of the managed server causes certain state to be lost, which prevents correct processing of messages:

  • JMS request/response business service: Maintains in-memory table mapping correlation information. This is lost when the managed server goes down. Thus, the response from the JMS service cannot be sent back to the original client of the proxy service, which routed to the JMS request/response business service.

  • WS (Reliable Messaging) business service: Similar to the JMS request/response business service, the WS business service keeps an in-memory table mapping correlation information. This is lost when the managed server goes down, and the response cannot be handled.

5.12.2.1.3 Database Failure

Beyond the implications for database leasing (used for server migration) database failures are relevant for Oracle Service Bus only in the context of the JMS reporting provider functionality. When there is report action in the proxy service, the report data is enqueued into the reporting JMS queue. The reporting MDB dequeues the message from the JMS queue and inserts the data into the database using the data source configured with the Logging Last Resource (LLR) global transaction protocol. The reporting JMS queue is configured with a redelivery limit of 2 and an error queue. When a database failure occurs, report messages are moved to the error queue after the redelivery limit is reached. When the database is running again, you can move these report messages back to the JMS reporting queue so that they are inserted into the database. If the database fails in the middle of LLR transaction processing, transaction recovery is performed as described in the chapter on Logging Last Resource Transaction Optimization in Oracle Fusion Middleware Programming JTA for Oracle WebLogic Server.

5.12.2.2 Oracle Service Bus Cluster-Wide Deployment

Oracle Service Bus resources are updated/deployed onto the WebLogic Administration Server. The Administration Server uses the WebLogic deployment service to propagate the Oracle Service Bus resources to all the managed servers in the cluster.

5.12.2.3 Online Redeployment of Oracle Service Bus in a Cluster

Oracle Service Bus resources do not have the notion of versions. When a resource is updated, it is as if a new version is deployed and the older version is overwritten. The configuration framework makes changes to resources in a session, which is a copy of the core (that all managed servers are currently using) when the session is created. When the session is committed, the core is updated with the changes. The changes are then propagated in a transactional manner to the managed servers in the cluster. In-flight requests (requests already executing the Proxy Service Message flow) continue to use the old copy of the resources when the update is in progress. Requests after the changes are added to the core use the updated resources.

5.12.2.4 Oracle Service Bus Cluster-Wide Configuration Changes

Oracle Service Bus resources are updated at the cluster level. The WebLogic Administration Server propagates these changes to all the managed servers in the cluster. You can make configuration changes using the Oracle Service Bus web console, the WebLogic Server Administration Console, and public MBeans.

The following files are excluded from the automatic propagation to the managed servers in the cluster:

  • Files in the DOMAIN_HOME/config/osb/coherence/ directory

  • DOMAIN_HOME/config/osb/transports/sftp/known_hosts

Each managed server must have access to the DOMAIN_HOME to access files in these directories.

Note:

For detailed instructions on installing and configuring Oracle Service Bus in a high availability configuration, see Section 5.14, "Configuring High Availability for Oracle Service Bus, with SOA Service Infrastructure and Component Service Engines."

5.13 Configuring High Availability for Oracle SOA Service Infrastructure and Component Service Engines

The procedures described in this section include setting up the Service engines contained in the Oracle SOA Service Infrastructure system, such as Oracle BPEL/Oracle PM, Oracle Mediator, Oracle Human Workflow and Oracle Decision Services, as well as Oracle B2B and Oracle User Messaging Service.

This section includes these topics:

Note:

Oracle strongly recommends reading the release notes for any additional installation and deployment considerations prior to starting the setup process.

Note:

The architectures and deployment procedures defined in this guide enable simple clustered deployments. The procedures described in these chapters can be used as a building block to enable this and other similar high availability topologies for these Fusion Middleware components. It is also expected that production deployments will use other required procedures, such as associating security policies with a centralized LDAP server. For complete details of secured, multi-tiered architectures, and deployment procedures, please refer to the Enterprise Deployment Guide for the component you are configuring.

Figure 5-42 represents the example architecture that the configuration steps in this section create.

Figure 5-42 Oracle SOA Service Infrastructure High Availability Architecture

Description of Figure 5-42 follows
Description of "Figure 5-42 Oracle SOA Service Infrastructure High Availability Architecture"

Figure 5-42 describes a two-node SOA cluster running on two Oracle WebLogic Servers. The Oracle WebLogic Servers are frontended by Oracle HTTP Servers, which load balance incoming requests. A load balancer front ends the system and distributes incoming requests from clients to the two Oracle HTTP Servers. A separate Oracle WebLogic Server (not shown in the figure) is typically used for custom logic and application deployment. This configuration uses an Oracle RAC database for storing metadata and SOA schemas, and shared storage for transaction and JMS stores. Virtual IP addresses (VIPs) provide manual failover for the Administration Server and for Oracle SOA Servers (for Server Migration). For more details about the components contained in this architecture, see the individual component sections in this chapter.

For information about configuring virtual IPs for the Administration Server and configuring the Administration Server for high availability, see Section 12.2.2.3, "Transforming the Administration Server for Cold Failover Cluster."

5.13.1 Preparing the Environment: Prerequisite Steps Before Setting up a SOA High Availability Configuration

The following sections provide prerequisite steps before setting up an Oracle SOA Service Infrastructure high availability configuration.

For information about platform-specific commands, see the Oracle Fusion Middleware Installation Guide for Oracle SOA Suite and Oracle Business Process Management Suite.

5.13.1.1 Database Prerequisites

Oracle SOA Suite requires the presence of a supported database and schemas.

To check if your database is certified or to see all certified databases, refer to the "Certified Databases" section in the Certification Document:

http://www.oracle.com/technology/software/products/ias/files/fusion_certification.html

To determine the database version, execute this query:

SQL>select version from sys.product_component_version where product like 'Oracle%';

5.13.1.2 VIP and IP Prerequisites

As shown in Table 5-2, you configure the Administration Server and the SOA managed servers to listen on different virtual IPs. This requires the provisioning of the corresponding VIP in the node and related host names in the DNS system in your network. Make sure that the different VIPS are available and are reachable from SOAHOST1, SOAHOST2, and the client systems before running the installation.

Table 5-2 Virtual Hosts

Virtual IP VIP Maps to... Description

VIP0

SOAHOST1VHN0

SOAHOST1VHN0 is the virtual host name that is the listen address for the Administration Server and fails over with manual failover of the Administration Server. It is enabled on the node where the Administration Server process is running (SOAHOST1 by default).

VIP1

SOAHOST1VHN1

SOAHOST1VHN1 is the virtual host name that maps to the listen address for WLS_SOA1 and fails over with server migration of this managed server. It is enabled on the node where WLS_SOA1 process is running (SOAHOST1 by default).

VIP2

SOAHOST2VHN1

SOAHOST2VHN1 is the virtual host name that maps to the listen address for WLS_SOA2 and fails over with server migration of this managed server. It is enabled on the node where WLS_SOA2 process is running (SOAHOST2 by default).


5.13.1.3 Shared Storage Prerequisites

For proper recovery in case of failure, store both JMS and transaction logs in a location that is accessible to all the nodes that can resume the operations after a failure in a managed server. This requires a shared storage location that can be referenced by multiple nodes. Table 5-3 lists the contents of shared storage.

Table 5-3 Contents of Shared Storage

Server Type of Data Vol in Shared Storage Directory Files

WLS_SOA1

Tx Logs

VOL1

ORACLE_BASE/admin/domain_name/soa_cluster_name/tlogs

Common location (stores decided by WebLogic Server)

WLS_SOA2

Tx Logs

VOL1

ORACLE_BASE/admin/domain_name/soa_cluster_name/tlogs

Common location (stores decided by WebLogic Server)

WLS_SOA1

JMS Stores

VOL1

ORACLE_BASE/admin/domain_name/soa_cluster_name/jms

Common location but Individual store per server (for example: SOAJMSStore1, UMSJMSStore1)

WLS_SOA2

JMS Stores

VOL1

ORACLE_BASE/admin/domain_name/soa_cluster_name/jms

Common location but Individual store per server (for example: SOAJMSStore2, UMSJMSStore2)


The shared storage can be a NAS or SAN device. Specifically for NFS mounted systems, different issues related to file locking and abrupt node failures have been detected. Check the Oracle Fusion Middleware Release Notes and with your storage vendor for the main recommended parameters to be used as mount options. The following is an example command based on a NAS device. Your options may be different from the ones specified in this section:

SOAHOST1> mount nasfiler:/vol/volX/FMWshared
MW_HOME -t nfs -o
rw,bg,hard,nointr,tcp,vers=3,timeo=300,rsize=32768,wsize=32768

5.13.1.4 Installing and Configuring an LDAP Provider

For production environments, it is a mandatory requirement for Oracle SOA Suite high availability topologies to have an external LDAP policy store. For more information on the supported policy stores as well as instructions on configuring LDAP, see the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite.

5.13.1.5 Synchronizing System Clocks

Synchronizing system clocks on each of the cluster nodes for high availability SOA deployments is required.

5.13.1.6 Terminology for Directories and Directory Environment Variables

The following list describes the directories and variables used in this chapter:

  • ORACLE_BASE: This environment variable and related directory path refers to the base directory under which Oracle products are installed.

  • MW_HOME: This environment variable and related directory path refers to the location where Fusion Middleware (FMW) resides.

  • WL_HOME: This environment variable and related directory path contains installed files necessary to host a WebLogic Server.

  • ORACLE_HOME: This environment variable and related directory path refers to the location where Oracle FMW SOA Suite is installed.

  • ORACLE_COMMON_HOME: This environment variable and related directory path refers to the Oracle home that contains the binary and library files required for the Oracle Enterprise Manager Fusion Middleware Control and Java Required Files (JRF).

    There can be only one Oracle Common home within each Middleware home.

  • DOMAIN Directory: This directory path refers to the location where the Oracle WebLogic Domain information (configuration artifacts) is stored.

  • ORACLE_INSTANCE: An Oracle instance contains one or more system components, such as Oracle Web Cache, Oracle HTTP Server, or Oracle Internet Directory. An Oracle instance directory contains updateable files, such as configuration files, log files, and temporary files.

The values used and recommended for consistency for this directories are:

ORACLE_BASE:

/u01/app/oracle

MW HOME (Application Tier):

ORACLE_BASE/product/fmw

WL_HOME:

MW_HOME/wlserver_10.3

ORACLE_HOME:

MW_HOME/soa

ORACLE_COMMON_HOME:

MW_HOME/oracle_common

Location for JMS file based stores and Tlogs:

ORACLE_BASE/admin/domain_name/soa_cluster_name/jms
ORACLE_BASE/admin/domain_name/soa_cluster_name/tlogs

Mount point is:

ORACLE_BASE/admin/domain_name/soa_cluster_name/

5.13.1.7 Installing and Configuring the Database Repository

This section describes how to install and configure the database repository.

Oracle Clusterware

  • For 10g Release 2 (10.2), see Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide.

  • For 11g Release 1 (11.1) and later, see Oracle Clusterware Installation Guide.

Automatic Storage Management

  • For 10g Release 2 (10.2), see Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide.

  • For 11g Release 1 (11.1) and later, see Oracle Real Application Clusters Installation Guide.

  • When you run the installer, select the Configure Automatic Storage Management option in the Select Configuration page to create a separate Automatic Storage Management home.

Oracle Real Application Clusters

You must install the 11g (11.1.1) Oracle Fusion Middleware Repository into a Real Application Clusters database before you install the Oracle Fusion Middleware components. Oracle Fusion Middleware provides a tool, Oracle Fusion Middleware Repository Creation Utility (RCU), to create the component schemas in an existing database. You install RCU in its own, separate Middleware home.

Use the latest version of RCU to install the 11g (11.1.1) Oracle Fusion Middleware Repository into a Real Application Clusters database.

See Oracle Fusion Middleware Repository Creation Utility User's Guide for more information about obtaining and running the latest version of RCU.

Database Initialization Parameters

Ensure that the following initialization parameter is set to the required value. It is checked by Repository Creation Assistant.

Table 5-4 Required Initialization Parameters

Parameter Required Value Parameter Class

PROCESSES

300 or greater

Static


To check the value of the initialization parameter using SQL*Plus, you can use the SHOW PARAMETER command.

As the SYS user, issue the SHOW PARAMETER command as follows:

SQL> SHOW PARAMETER processes

Set the initialization parameter using the following command:

SQL> ALTER SYSTEM SET processes=300 SCOPE=SPFILE

Restart the database.

Note:

The method that you use to change a parameter's value depends on whether the parameter is static or dynamic, and on whether your database uses a parameter file or a server parameter file. See the Oracle Database Administrator's Guide for details on parameter files, server parameter files, and how to change parameter values.

5.13.1.8 Using Oracle Fusion Middleware Repository Creation Utility to Load the Fusion Middleware Schemas in the Database

Install the 11g (11.1.1) Oracle Fusion Middleware Metadata Store and Oracle SOA schemas into a Real Application Clusters database before you install Oracle Fusion Middleware SOA components. Oracle Fusion Middleware provides a tool, Oracle Fusion Middleware Repository Creation Utility (RCU), to create the component schemas in an existing database. See Oracle Fusion Middleware Repository Creation Utility User's Guide for more information about installing RCU.

Note:

For domains including Oracle BPM Suite, it is required that Oracle Fusion Middleware 11.1.1.4 (Patch Set 3) RCU be used.
5.13.1.8.1 Running RCU

Run RCU to install the required metadata for Oracle Fusion Middleware 11g.

  1. For Linux, go to RCU_Home/bin and use the following command:

    ./rcu
    
  2. In the Welcome screen, click Next.

  3. In the Create Repository screen, select Create to load component schemas into a database, click Next.

  4. In the Database Connection Details screen, enter connection information for your database:

    • Database Type: Select Oracle Database.

    • Host Name: Enter the name of the node that is running the database. For an Oracle RAC database, specify the VIP name, or one of the node names as the host name: SOADBHOST1VIRTUAL.

    • Port: The port number for the database: 1521

    • Service Name: Enter the service name of the database: soaha.mycompany.com

    • Username: SYS

    • Password: Enter the password for the SYS user.

    • Role: SYSDBA

  5. Click Next.

  6. If you receive the following warning message:

    The database you are connecting is with non-UTF8 charset, if you are going to use this DB for multilingual support, you may have data loss. If you are not using for multilingual support you can continue, otherwise we strongly recommend using UTF-8 database.

  7. Click Ignore or Stop.

  8. In the Select Components screen, select Create a New Prefix, and enter a prefix to use for the database schemas, for example, SOAHA

    Write down the schema names so they are available in later procedures.

    • Select the following schemas:

      • Under AS Common Schemas, select Metadata Services.

      • Under SOA Service Infrastructure, select SOA Service Infrastructure and User Messaging.

  9. Click Next.

  10. In the Schema Passwords screen, enter passwords for the main and additional (auxiliary) schema users, and click Next.

  11. In the Map Tablespaces screen, choose the tablespaces for the selected components, and click Next.

  12. In the Summary screen, click Create.

  13. In the Completion Summary screen, click Close.

See the Oracle Fusion Middleware Repository Creation Utility User's Guide for more information about using RCU.

5.13.1.8.2 Configuring SOA Schemas for Transactional Recovery Privileges

You need the appropriate database privileges to allow the Oracle WebLogic Server transaction manager to query for transaction state information and issue the appropriate commands, such as commit and rollback, during recovery of in-flight transactions after a WebLogic Server container crash.

To configure the SOA schemas for transactional recovery privileges:

  1. Log on to SQL*Plus as a user with sysdba privileges. For example:

    sqlplus "/ as sysdba"
    
  2. Grant select on sys.dba_pending_transactions to soaha_soainfra.

  3. Grant force any transaction to soaha_soainfra.

    Note:

    These privileges should be granted to the owner of the soainfra schema, as determined by the RCU operations.

5.13.1.9 Configuring Virtual Server Names and Ports for the Load Balancer

This section describes the load balancer prerequisites for deploying an Oracle SOA Suite high availability environment.

Load Balancers

Oracle SOA Suite uses a hardware load balancer when deployed in a high availability configuration with two Oracle HTTP Servers as web tier components. The hardware load balancer should have the following features:

  • Ability to load-balance traffic to a pool of real servers through a virtual host name: Clients access services using the virtual host name (instead of using actual host names). The load balancer can then load balance requests to the servers in the pool.

  • Port translation configuration: The load balancer should have the ability to perform port translation, where it allows incoming requests received on one port to be routed to a server process running on a different port. For example, a request received on port 80 can be routed to port 7777.

  • Monitoring of ports (HTTP, HTTPS)

  • Virtual servers and port configuration: Ability to configure virtual server names and ports on your external load balancer, and the virtual server names and ports must meet the following requirements:

    • The load balancer should allow configuration of multiple virtual servers. For each virtual server, the load balancer should allow configuration of traffic management on more than one port. For example, the load balancer typically needs to be configured with a virtual server and ports for HTTP and HTTPS traffic.

    • The virtual server names must be associated with IP addresses and be part of your DNS. Clients must be able to access the load balancer through the virtual server names.

  • Ability to detect node failures and immediately stop routing traffic to the failed node.

  • Resource monitoring/port monitoring/process failure detection: The load balancer must be able to detect service and node failures (through notification or some other means) and to stop directing non-Oracle Net traffic to the failed node. If your load balancer has the ability to automatically detect failures, you should use it.

  • Fault-tolerant mode: It is highly recommended that you configure the load balancer to be in fault-tolerant mode.

  • Other: It is highly recommended that you configure the load balancer virtual server to return immediately to the calling client when the back-end services to which it forwards traffic are unavailable. This is preferred over the client disconnecting on its own after a timeout based on the TCP/IP settings on the client system.

  • Sticky routing capability: Ability to maintain sticky connections to components based on cookies or URL.

  • SSL acceleration: SSL acceleration is a method of offloading the processor-intensive public key encryption algorithms involved in SSL transactions to a hardware accelerator.

    This feature is recommended but not required.

Table 5-5 shows some example virtual server names to use for the external load balancer in the Oracle SOA Suite high availability environment.

Table 5-5 Virtual Server Names for the External Load Balancer

Component Virtual Server Name

Oracle SOA Suite

soa.mycompany.com

WebLogic Server Administration Console

admin.mycompany.com

Oracle Enterprise Manager Fusion Middleware Control

admin.mycompany.com


Virtual Server Names

This section describes the virtual server names that should be set up for the high availability deployments described in this chapter.

soa.mycompany.com

soa.mycompany.com is a virtual server name that acts as the access point for all HTTP traffic to the runtime SOA components, such as soa-infra and workflow. Traffic to the both SSL and non-SSL ports is configured and typically non-SSL is redirected to SSL. Clients access this service using the address soa.mycompany.com:443. This virtual server is defined on the load balancer.

admin.mycompany.com

This virtual server acts as the access point for all internal HTTP traffic that gets directed to the administration services. The incoming traffic from clients is non-SSL enabled. Thus, the clients access this service using the address admin.mycompany.com:80 and in turn forward these to ports 7777 on WEBHOST1 and WEBHOST2. The services accessed on this virtual host include the WebLogic Administration Server Console and Oracle Enterprise Manager.

In addition, ensure that the virtual server names are associated with IP addresses and are part of your Domain Name System (DNS). The computers on which Oracle Fusion Middleware is running must be able to resolve these virtual server names.

5.13.1.10 Installing Oracle HTTP Server on WEBHOST1 and WEBHOST2

To install Oracle HTTP Server on WEBHOST1:

  1. Verify that the servers meet the following requirements:

    • The system, patch, kernel, and other requirements meet the requirements specified in the Oracle Fusion Middleware Installation Guide for Oracle SOA Suite.

    • Port 7777 is not used by any service on the nodes. You can verify this by running the following command:

      Unix:

      netstat -an | grep LISTEN | grep "7777"
      

      Windows:

      netstat -an | findstr "LISTEN" | findstr ":7777"
      

    If the ports are in use, make them available.

  2. On UNIX platforms, if the /etc/oraInst.loc or /var/opt/oracle/oraInst.loc file exists, check that its contents are correct. Specifically, check that the inventory directory is correct, and that you have write permissions for that directory.

    If the /etc/oraInst.loc file does not exist, skip this step.

  3. Start Oracle Universal Installer for Oracle Fusion Middleware 11g Web Tier Utilities CD installation as follows:

    For UNIX, run this command: runInstaller.

    For Windows, double-click setup.exe.

  4. In the Welcome screen, click Next.

  5. In the Select Installation Type screen, select Install and Configure, and click Next.

  6. In the Prerequisite Checks screen, ensure that all the prerequisites are met, and click Next.

  7. In the Specify Installation Location screen, set the location to:

    /u01/app/oracle/product/11.1.1/ohs_1

  8. Click Next.

  9. In the Configure Components screen:

    • Select Oracle HTTP Server.

    • Do not select Associate Selected Components with WebLogic Domain.

    • Click Next.

  10. In the Specify Component Details screen, enter the following values:

    • Instance Home Location: Instance Home

    • Instance Home Location: /u01/app/oracle/product/11.1.1/ohs_1/instances/ohs_instance1

    • Instance Name: ohs_instance1

    • OHS Component Name: ohs1

  11. Click Next.

  12. In the Specify Web Tier Port Details screen:

    • Select Specify Custom Ports. If you specify a custom port, select Specify Ports using Configuration File, and use the Browse function to select the file.

    • Enter the Oracle HTTP Server port, for example, 7777.

  13. Click Next.

  14. In the Configuration Summary screen, ensure that the selections are correct, and click Install.

  15. In the Installation Progress screen:

    For UNIX systems, a dialog box appears prompting you to run the oracleRoot.sh script. Open a command window and run the script, following the prompts.

    Click Next.

  16. In the Configuration screen, several configuration assistants are launched in succession. When the configuration assistants are finished, the Configuration Completed screen appears.

  17. In the Configuration Completed screen, click Finish to exit.

Repeat the steps for WEBHOST2 and configure your LBR with a pool containing both the WEBHOST1 and WEBHOST2 addresses.

5.13.1.10.1 Validating Oracle HTTP Server

To verify that Oracle HTTP Server is set up correctly, access the root URL context of the server by using the following URL in a browser:

HTTP://WEBHOST1:7777/

If Oracle HTTP Server is set up correctly, the Oracle FMW Welcome screen appears in the browser.

5.13.2 Installing Oracle Fusion Middleware Home

This section describes the procedure for installing Oracle Fusion Middleware on all nodes in the application tier that run Oracle WebLogic Server and Oracle Fusion Middleware SOA Suite. Repeat the procedures (described below for SOAHOST1) for installing WebLogic Server and Oracle SOA in SOAHOST2. The directory paths for binary files and domains used when installing new nodes must be exactly the same as those used for first node. If these paths and domains are not exactly the same as those used for the first node, failover will not work properly.

Install the following Oracle Fusion Middleware components:

5.13.2.1 Installing Oracle WebLogic Server

See "Understanding Your Installation Starting Point" in Oracle Fusion Middleware Installation Planning Guide for the version of Oracle WebLogic Server to use with the latest version of Oracle Fusion Middleware.

To install Oracle WebLogic Server on all nodes in the application tier:

  1. Start Oracle WebLogic Server Installer.

  2. In the Welcome screen, click Next.

  3. In the Choose Middleware Home Directory screen:

    • Select Create a New Middleware Home.

    • For the Middleware Home Directory field, enter MW_HOME.

    • Click Next.

  4. In the Register for Security Updates screen, enter your contact information for security update notifications, and click Next.

  5. In the Choose Install Type screen, select Custom, and click Next.

  6. In the Choose Products and Components screen, click Next.

  7. In the JDK Selection screen, select only Oracle JRockit 160_20_D1.0.1-2124 SDK, and click Next.

  8. In the Choose Product Installation Directories screen, accept the following directory:

    WL_HOME

    Click Next.

  9. In the Installation Summary screen, click Next.

  10. In the Installation Complete screen, deselect Run QuickStart, and click Done.

5.13.2.2 Installing Oracle Fusion Middleware for Oracle SOA

On Linux platforms, if the /etc/oraInst.loc file exists, check that its contents are correct. Specifically, check that the inventory directory is correct and that you have write permissions for that directory.

If the /etc/oraInst.loc file does not exist, you can skip this step.

  1. Start Oracle Fusion Middleware 11g Oracle SOA Suite Installer:

    On UNIX:

    SOAHOST1> runInstaller
    

    On Windows:

    SOAHOST1> setup.exe
    

    When Oracle Fusion Middleware 11g Oracle SOA Suite Installer prompts you for a JRE/JDK location enter the Oracle SDK location created in the Oracle WebLogic Server installation, for example, MW_HOME/jrockit_160_20_D1.0.1-2124.

  2. In the Specify Inventory Directory screen, do the following:

    1. Enter HOME/oraInventory, where HOME is the home directory of the user performing the installation (this is the recommended location).

    2. Enter the OS group for the user performing the installation.

    3. Click Next.

    4. Follow the instructions on the screen to execute /createCentralInventory.sh as root.

    5. Click OK.

  3. In the Welcome screen, click Next.

  4. In the Prerequisite Checks screen, verify that the checks complete successfully, and click Next.

  5. In the Specify Installation Location screen:

    • For Middleware Home, enter MW_HOME.

    • For Oracle Home Directory, enter soa.

    Click Next.

  6. In the Installation Summary screen, click Install.

  7. In the Installation Complete screen, click Finish.

Note:

Before you run the Configuration Wizard by following the instructions in Section 5.13.4, "Running Oracle Fusion Middleware Configuration Wizard on SOAHOST1 to Create the SOA Domain," make sure that you have applied the latest Oracle Fusion Middleware patch set and other known patches to your Middleware Home, so that you have the latest version of Oracle Fusion Middleware.

See "Understanding Your Installation Starting Point" in Oracle Fusion Middleware Installation Planning Guide for the steps you must perform to get the latest version of Oracle Fusion Middleware.

5.13.3 Enabling VIP1 in SOAHOST1 and VIP2 in SOAHOST2

The SOA domain uses virtual hostnames as the listen addresses for the SOA managed servers. You must enable VIP mapping for each of these hostnames on the two SOA Machines, (VIP1 on SOAHOST1 and VIP2 on SOAHOST2), and must correctly resolve the virtual hostnames in the network system used by the topology (either by DNS Server or by hosts resolution).

For information about configuring virtual IPs, see Section 12.2.2.3, "Transforming the Administration Server for Cold Failover Cluster," where an example is provided for the Administration Server. Refer to Section 5.13.21, "Configuring Server Migration for the WLS_SOA Servers" for more details on configuring server migration for the SOA servers.

5.13.4 Running Oracle Fusion Middleware Configuration Wizard on SOAHOST1 to Create the SOA Domain

Run Oracle Fusion Middleware Configuration Wizard from the SOA home directory to create a domain containing the Administration Server and Oracle SOA components. Ensure that the database where you installed the repository is running. For Oracle RAC databases, all the instances must be running.

Note:

Multiple SOA clusters are not allowed in the same WebLogic Server domain.

Note:

Oracle BPM requires the WL_HOME and ORACLE_HOME homes to be patched to the Oracle Fusion Middleware 11.1.1.4 (PS3) patch set level before the Oracle Fusion Middleware Configuration Wizard steps are performed to extend a domain.
  1. Change the directory to the location of the Oracle Fusion Middleware Configuration Wizard, located in the Middleware home, SOA directory:

    SOAHOST1> cd ORACLE_COMMON_HOME/common/bin
    
  2. Start Oracle Fusion Middleware Configuration Wizard:

    For Linux:

    SOAHOST1> ./config.sh
    

    For Windows:

    SOAHOST1> config.cmd
    
  3. In the Welcome screen, select Create a New WebLogic Domain, and click Next.

  4. In the Select Domain Source screen:

    • Select Generate a domain configured automatically to support the following products.

    • Select the following products:

      • Basic Weblogic Server Domain - 10.3.4.0 [wlserver_10.3] (Selected by default and grayed out)

      • Oracle BPM Suite - 11.1.1.0 [soa] (ONLY FOR BPM Systems)

      • Oracle SOA Suite - 11.1.1.0 [soa] (Selected by default when selecting BPM Suite)

      • Oracle Enterprise Manager - 11.1.1.0 [oracle_common] (Selected by default)

      • Oracle WSM Policy Manager - 11.1.1.0 [oracle_common] (Selected by default when selecting SOA/BPM Suite)

      • Oracle JRF WebServices - 11.1.1.0 [oracle_common] (Selected by default when selecting SOA/BPM Suite)

    If you accidentally deselect some of the targets, make sure that the following selections are made in this screen:

    • Oracle BPM Suite (ONLY FOR BPM Systems)

    • Oracle SOA Suite

    • Oracle Enterprise Manager

    • Oracle WSM Policy Manager

    • Oracle JRF

    Click Next.

  5. In the Specify Domain Name and Location screen, make the following entries:

    • Domain Name: soadomain

    • Domain Location: accept the default

    • Application Location: accept the default

    Click Next.

  6. In the Configure Administrator Username and Password screen, enter the username and password to be used for the domain's administrator, and click Next.

  7. In the Configure Server Start Mode and JDK screen, make the following selections:

    • WebLogic Domain Startup Mode: select Production Mode

    • JDK Selection: select Oracle JRockit 160_20_D1.0.1-2124 SDK.

    Click Next.

  8. In the Configure JDBC Component Schema screen:

    1. Select all the component schemas that appear in the table at the bottom: SOA Infrastructure, User Messaging Service, OWSM MDS Schema and SOA MDS Schema.

    2. Select Configure selected component schemas as RAC multi data source schemas in the next panel.

    3. Ensure that the following data source appears on the screen. The user names shown in Table 5-6 assume that soaha was used as the prefix for schema creation from RCU.

      For information about multi data source configuration with Oracle RAC and the MDS repository, see Section 4.1.2, "Using Multi Data Sources with Oracle RAC."

      Table 5-6 Configuring Values for Data Sources

      Multi Data Source Schema Schema Owner

      SOA Infrastructure

      SOAHA_SOAINFRA

      User Messaging Service

      SOAHA_ORASDPM

      OWSM MDS Schema

      SOAHA_MDS

      SOA MDS Schema

      SOAHA_MDS


    4. Click Next.

  9. In the Configure RAC Multi Data Source Component Schema screen, enter values for the following fields, specifying the connect information for the Oracle RAC database that was seeded with RCU:

    Figure 5-43 Configure RAC Multi Data Source Component Schema Screen

    Configure RAC Multi Data Sources
    Description of "Figure 5-43 Configure RAC Multi Data Source Component Schema Screen"

    1. Driver: Select Oracle driver (Thin) for RAC Service-Instance connections, Versions:10, 11.

    2. Service Name: Enter the service name of the database, for example, soaha.mycompany.com.

    3. Username prefix: Enter the prefix for the schemas. The user names shown in Table 5-6 assume that soaha was used as prefix for schema creation from RCU.

    4. Password and Confirm Password: Enter the password for access to the schemas.

    5. Click Add, and enter the details for the first Oracle RAC instance.

    6. Update each multi data source schema by selecting one data source at a time, adding the appropriate details.

      Make sure that the information is entered for all the multi data source schemas: SOA Infrastructure, User Messaging Service, OWSM MDS Schema, and SOA MDS Schema.

    7. Click Next.

  10. In the Test JDBC Data Sources screen, the connections are tested automatically. The Status column displays the results. Ensure that all connections were successful. If not, click Previous to return to the previous screen and correct your entries.

    Click Next when all the connections are successful.

  11. In the Select Optional Configuration screen, select the following:

    • Administration Server

    • JMS Distributed Destinations

    • Managed Servers, Clusters and Machines

  12. In the Customize Server and Cluster Configuration screen, select Yes, and click Next.

  13. In the Configure the Administration Server screen, enter the following values:

    • Name: AdminServer

    • Listen Address: Enter the hostname for the VIP0 virtual IP

    • Listen Port: 7001

    • SSL listen port: N/A

    • SSL enabled: leave unchecked

    Click Next.

  14. In the Select JMS Distributed Destination Type screen, select UDD for UMSJMSSystemResource, SOAJMSModule, and BPMJMSModule.

  15. In the Configure Managed Servers screen, add the following managed servers:

    Table 5-7 Configuring Managed Servers

    Name Listen Address Listen Port SSL Listen Port SSL Enabled

    WLS_SOA1

    SOAHOST1VHN1

    8001

    n/a

    No

    WLS_SOA2

    SOAHOST2VHN1

    8001

    n/a

    No


    Do not delete any server that appears. You can modify the servers. If you delete a server and add a new one, targeting fails.

    Note:

    Although the standard recommendation is to run custom applications and other systems in a separate WebLogic Managed Server, the creation of the custom WLS managed servers described in Figure 5-42 is not addressed here.

    Click Next.

  16. In the Configure Clusters screen, add the following cluster:

    • Name: SOA_Cluster

    • Cluster Messaging Mode: unicast

    • Multicast Address: N/A

    • Multicast Port: N/A

    • Cluster Address: Leave empty

    Click Next.

  17. In the Assign Servers to Clusters screen, assign the following servers to SOA_Cluster:

    • WLS_SOA1

    • WLS_SOA2

    Click Next.

  18. In the Configure Machines screen:

    • Delete the LocalMachine that appears by default.

    • Click the Unix Machine tab, and add the following machines:

    Table 5-8 Configuring Machines

    Name Node Manager Listen Address

    SOAHOST1

    Hostname of SOAHOST1

    SOAHOST2

    Hostname of SOAHOST2


    Leave all other fields as their default values, and click Next.

  19. In the Assign Servers to Machines screen, assign servers to machines as follows:

    Figure 5-44 Assign Servers to Machines Screen

    Assign Servers to Machine screen
    Description of "Figure 5-44 Assign Servers to Machines Screen"

    • SOAHOST1: AdminServer, WLS_SOA1

    • SOAHOST2: WLS_SOA2

    Click Next.

  20. In the Review WebLogic Domain screen, click Next.

  21. In the Configuration Summary screen, click Create.

  22. In the Creating Domain screen, click Done.

Note:

The multicast and unicast addresses are different from the ones used by the WebLogic Server cluster for cluster communication. SOA guarantees that composites are deployed to members of a single WebLogic Server cluster even though the communication protocol for the two entities (the WebLogic Server cluster and the groups to which composites are deployed) are different.

5.13.5 Creating boot.properties for the Administration Server on SOAHOST1

This is an optional step for enabling the Administration Server to start up without prompting you for the administrator username and password. Create a boot.properties file for the Administration Server on SOAHOST1.

For the Administration Server:

  1. Create the following directories:

    SOAHOST1> mkdir -p ORACLE_BASE/product/fmw/user_
    projects/domains/soadomain/servers/AdminServer/security 
    
  2. Use a text editor to create a file called boot.properties in the security directory created in the previous step, and enter the following lines in the file:

    username=adminuser
    password=password
    

    Note:

    When you start up the Administration Server, the username and password entries in the file are encrypted.

    For security reasons, minimize the time the entries in the file are left unencrypted. After you edit the file, start up the server as soon as possible in order for the entries to be encrypted.

5.13.6 Starting and Validating the Administration Server in SOAHOST1

This section describes procedures for starting and validating the Administrator Server in SOAHOST1

5.13.6.1 Starting the Administration Server on SOAHOST1

To start the Administration Server on SOAHOST1, run the following commands:

SOAHOST1> cd MW_HOME/user_projects/domains/soadomain/bin

SOAHOST1> ./startWebLogic.sh

5.13.6.2 Validating the Administration Server

To verify that the Administration Server is properly configured:

  1. In a browser, go to http://vip0:7001/console.

  2. Log in as the administrator.

  3. Verify that the WLS_SOA1 and WLS_SOA2 managed servers are listed.

  4. Verify that the SOA_Cluster cluster is listed.

  5. Verify that you can access Enterprise Manager at http://vip0:7001/em.

5.13.7 Disabling Host Name Verification for the Administration Server and the WLS_SOAn Managed Servers

This step is required if you have not set up the appropriate certificates for hostname verification between the Administration Server and Node Manager. If SSL is not set up, you receive an error message unless you disable host name verification.

You can re-enable host name verification when you have set up SSL communication between the Administration Server and the Node Manager.

To disable host name verification:

  1. In Oracle WebLogic Server Administration Console, select Servers, and then AdminServer.

  2. Select SSL, and then Advanced.

  3. In the Change Center, click Lock & Edit.

  4. Set Hostname Verification to None.

  5. In Oracle WebLogic Server Administration Console, select AdminServer, SSL, and then Advanced.

  6. Set Hostname Verification to None.

  7. Save and activate the changes.

  8. Repeat these steps for the WLS_SOA1 and WLS_SOA2 servers.

  9. Restart the Administration Server.

5.13.8 Configuring Oracle Coherence for Deploying Composites

Although deploying composites uses multicast communication by default, Oracle recommends using unicast communication for SOA high availability. Use unicast if you disable multicast communication for security reasons.

Note:

An incorrect configuration of the Oracle Coherence framework that is used for deployment may prevent the SOA system from starting. The deployment framework must be properly customized for the network environment on which the SOA system runs. Oracle recommends the following configuration described in this section.

Enabling Communication within Clusters Using Unicast Communication

Multicast communication enables Oracle Fusion Middleware SOA to discover all of the members of a cluster to which to it deploys composites dynamically. However, unicast communication does not enable nodes to discover other cluster members in this way. Consequently, you must specify the nodes that belong to the cluster. You do not need to specify all of the nodes of a cluster, however. You need only specify enough nodes so that a new node added to the cluster can discover one of the existing nodes. As a result, when a new node has joined the cluster, it is able to discover all of the other nodes in the cluster. Additionally, in a configuration where multiple IPs are available in the same node, you must configure Oracle Coherence to use a specific hostname to create Oracle Coherence cluster.

Tip:

To guarantee high availability during deployments of SOA composites, specify enough nodes so that at least one of them is running at any given time.

Specify the nodes using the tangosol.coherence.wka<n> system property, where <n> is a number between 1 and 9. You can specify up to 9 nodes. Start the numbering at 1. This numbering must be sequential and must not contain gaps. In addition, specify the hostname used by Oracle Coherence to create a cluster through the tangosol.coherence.localhost system property. This hostname should be the virtual hostname used by the SOA servers that maps the corresponding listener addresses (VIP1 and VIP2). Set this property by adding the -Dtangosol.coherence.localhost parameters to the Arguments field of Oracle WebLogic Server Administration Console's Server Start tab (Figure 5-45).

Figure 5-45 Setting the Hostname Using the Start Server Tab of Oracle WebLogic Server Administration Console

Setting the Hostname
Description of "Figure 5-45 Setting the Hostname Using the Start Server Tab of Oracle WebLogic Server Administration Console"

Specifying the hostname

To add the hostname used by Oracle Coherence:

  1. Log into Oracle WebLogic Server Administration Console.

  2. In the Domain Structure window, expand the Environment node.

  3. Click Servers. The Summary of Servers page appears.

  4. Click the name of the server (represented as a hyperlink) in Name column of the table. The settings page for the selected server appears.

  5. Click the Server Start tab (illustrated in Figure 5-45).

  6. Enter the following for WLS_SOA1 and WLS_SOA2 into the Arguments field.

    For WLS_SOA1, enter the following (on a single line, without a carriage return):

    -Dtangosol.coherence.wka1=soahost1vhn1 -Dtangosol.coherence.wka2=soahost2vhn1 -Dtangosol.coherence.localhost=soahost1vhn1
    

    For WLS_SOA2, enter the following (on a single line, without a carriage return):

    -Dtangosol.coherence.wka1=soahost1vhn1 -Dtangosol.coherence.wka2=soahost2vhn1 -Dtangosol.coherence.localhost=soahost2vhn1
    

    Note:

    The Coherence cluster used for deployment uses port 8088 by default. This port can be changed by specifying a different port (for example, 8089) with the -Dtangosol.coherence.wkan.port and -Dtangosol.coherence.localport startup parameters. For example:

    WLS_SOA1 (enter the following into the Arguments field on a single line, without a carriage return):

    -Dtangosol.coherence.wka1=soahost1vhn1
    -Dtangosol.coherence.wka2=soahost2vhn1
    -Dtangosol.coherence.localhost=soahost1vhn1
    -Dtangosol.coherence.localport=8089
    -Dtangosol.coherence.wka1.port=8089
    -Dtangosol.coherence.wka2.port=8089
    

    WLS_SOA2 (enter the following into the Arguments field on a single line, without a carriage return):

    -Dtangosol.coherence.wka1=soahost1vhn1
    -Dtangosol.coherence.wka2=soahost2vhn1
    -Dtangosol.coherence.localhost=soahost2vhn1
    -Dtangosol.coherence.localport=8089
    -Dtangosol.coherence.wka1.port=8089
    -Dtangosol.coherence.wka2.port=8089
    
  7. Click Save and activate the changes.

  8. This change requires the SOA servers to be restarted.

Note:

The multicast and unicast addresses are different from the ones used by the WebLogic Server cluster for cluster communication. SOA guarantees that composites are deployed to members of a single WebLogic Server cluster even though the communication protocol for the two entities (the WebLogic Server cluster and the groups to which composites are deployed) are different.

Do not copy the text from above to your Administration Console's arguments text field, which can result in HTML tags being inserted in the Java arguments. The text should not include any text or characters other than the ones shown above.

5.13.9 Setting Connection Destination Identifiers for B2B Queues

Oracle B2B uses specific JMS Destination Member calls, and requires setting the Create Destination Identifier (CDI) for these calls to succeed. To set up the CDI:

  1. Log into the Oracle WebLogic Server Administration Console.

  2. In the Domain Structure window, expand the Services node, and then the Messaging node.

  3. Click JMS Modules, and then SOAJMSModule.

  4. In the Change Center, click Lock & Edit.

  5. Click the dist_B2BEventQueue_auto, Configuration, and the General tab, and then click Advanced.

  6. In the Create Destination Identifier field, add the following jndi name for the queue:

    jms/b2b/B2BEventQueue
    
  7. Repeat these steps, creating the following Create Destination Identifiers for the queues listed below:

    • B2B_OUT_QUEUE: jms/b2b/B2B_OUT_QUEUE

    • B2B_IN_QUEUE: jms/b2b/B2B_IN_QUEUE

    • B2BBroadcastTopic: jms/b2b/B2BBroadcastTopic

    • XmlSchemaChangeNotificationTopic: jms/fabric/XmlSchemaChangeNotificationTopic

  8. Click Save and Activate Changes.

5.13.10 Starting the System in SOAHOST1

This section describes how to start Node Manager on SOAHOST1 and how to start and validate the WLS_SOA1 managed server on SOAHOST1.

5.13.10.1 Starting Node Manager on SOAHOST1

Perform these steps to start Node Manager on SOAHOST1:

  1. Run the setNMProps.sh script, which is located in the ORACLE_COMMON_HOME/common/bin directory, to set the StartScriptEnabled property to true before starting Node Manager

    SOAHOST1> cd ORACLE_COMMON_HOME/common/bin
    SOAHOST1> ./setNMProps.sh
    

    Note:

    You must use the StartScriptEnabled property to avoid class loading failures and other problems.
  2. Start Node Manager:

    SOAHOST1> cd WL_HOME/server/bin
    SOAHOST1> ./startNodeManager.sh
    

5.13.10.2 Starting and Validating the WLS_SOA1 Managed Server

To start up the WLS_SOA1 managed server and check that it is configured correctly:

  1. Start the WLS_SOA1 managed server using Oracle WebLogic Server Administration Console.

  2. When WLS_SOA1 is started, the following URLs become available:

    • http://SOAHOST1VHN1:8001/b2b

      • Verify login to B2B console

    • http://SOAHOST1VHN1:8001/integration/worklistapp

      • Verify login to worklist console

    • http://SOAHOST1VHN1:8001/wsm-pm

      • Verify the policy validator link.

    • http://SOAHOST1VHN1:8001/soa/composer

    • http://SOAHOST1VHN1:8001/soa-infra

    • http://SOAHOST1VHN1:8001/bpm/composer (ONLY FOR BPM Systems)

    • http://SOAHOST1VHN1:8001/bpm/workspace (ONLY FOR BPM Systems)

5.13.11 Propagating the Domain Configuration to SOAHOST2 with pack/unpack Utilities

Follow these steps to propagate the domain configuration to SOAHOST2 using pack/unpack utilities:

  1. Run the following pack command on SOAHOST1 to create a template pack:

    SOAHOST1> cd ORACLE_COMMON_HOME/common/bin
    SOAHOST1> ./pack.sh -managed=true
    -domain=ORACLE_BASE/product/fmw/user_projects/domains/soadomain/
    -template=soadomaintemplate.jar 
    -template_name=soa_domain_template
    
  2. Run the following command on SOAHOST1 to copy the template file created in the previous step to SOAHOST2:

    SOAHOST1> scp soadomaintemplate.jar
     oracle@node2:ORACLE_COMMON_HOME/common/bin
    
  3. Run the unpack command on SOAHOST2 to unpack the propagated template:

    SOAHOST2> cd ORACLE_COMMON_HOME/common/bin
    SOAHOST2> ./unpack.sh
     -domain=ORACLE_BASE/product/fmw/user_projects/domains/soadomain/
     -template=soadomaintemplate.jar
    

    Note:

    When you run the Fusion Middleware Configuration Wizard after a pack/unpack procedure, products that already exist in the original domain and appear in the Select Extension Source screen, are not selected or greyed out.

    Templates created with the pack command are considered user-created templates. They are treated differently from default templates. User-created templates are self-contained and do not contain any information related to the creation of the original domain.

5.13.12 Extracting XEngine Files in the Second Node

To enable B2B's XEngine in the second node, it is required to extract the content of the ZEngine tar manually:

SOAHOST2>cd ORACLE_HOME/soa/thirdparty/edifecs
SOAHOST2>tar xzvf XEngine.tar.gz

5.13.13 Starting the System in SOAHOST2

This section describes procedures for starting the system in SOAHOST2.

5.13.13.1 Starting Node Manager on SOAHOST2

To start the Node Manager on SOAHOST2, repeat the steps from Section 5.13.10.1, "Starting Node Manager on SOAHOST1" on SOAHOST2.

5.13.13.2 Starting and Validating the WLS_SOA2 Managed Server

To start up the WLS_SOA2 managed server and verify that it is configured correctly:

  1. Start the WLS_SOA2 managed server using Oracle WebLogic Server Administration Console

  2. When WLS_SOA2 is started, the following URLs become available:

    • http://SOAHOST2VHN1:8001/b2b

      Verify login to B2B console

    • http://SOAHOST2VHN1:8001/integration/worklistapp

      Verify login to worklist console

    • http://SOAHOST2VHN1:8001/wsm-pm

      Verify the policy validator link

    • http://SOAHOST2VHN1:8001/soa/composer

    • http://SOAHOST2VHN1:8001/soa-infra

    • http://SOAHOST2VHN1:8001/bpm/composer (ONLY FOR BPM Systems)

    • http://SOAHOST2VHN1:8001/bpm/workspace (ONLY FOR BPM Systems)

5.13.14 Configuring Oracle HTTP Servers for the Administration Server and the WLS_SOAn Managed Servers

To enable Oracle HTTP Server to route to the SOA Cluster, which contains the WLS_SOAn managed servers, you must set the WebLogicCluster parameter to the list of nodes in the cluster.

  1. On WEBHOST1 and WEBHOST2, add the following lines to the ORACLE_BASE/admin/<instance_name>/config/OHS/<component_name>/mod_wl_ohs.conf file:

    # WSM-PM
    <Location /wsm-pm>
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001, SOAHOST2VHN1:8001
    </Location>
    
    # SOA soa-infra app
    <Location /soa-infra>
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
    
    # SOA inspection.wsil
    <Location /inspection.wsil>
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
    
    # Worklist
    <Location /integration/>
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
    
    # B2B
    <Location /b2b>
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST11VHN1:8001,SOAHOST2VHN1:8001
    </Location>
    
    <Location /b2bconsole>
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
    
    # UMS
    <Location /sdpmessaging/ >
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
     
    # UMS WS
    <Location /ucs/messaging/webservice >
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
    
    # Default to-do taskflow
    <Location /DefaultToDoTaskFlow/>
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
    
    # Workflow
    <Location /workflow>
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
    
    #Required if attachments are added for workflow tasks
     <Location /ADFAttachmentHelper> 
        SetHandler weblogic-handler 
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001 
     </Location>
    
    # SOA composer application 
     <Location /soa/composer> 
         SetHandler weblogic-handler 
         WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001 
     </Location>
    
    # BPM composer (ONLY FOR BPM Systems)
    <Location /bpm/composer >
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
    
    # BPM workspace (ONLY FOR BPM Systems)
    <Location /bpm/workspace >
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
    
  2. Make sure the httpd.conf file located in the same directory as the mod_wl_ohs file contains the following lines:

    NameVirtualHost *:7777
    <VirtualHost *:7777>
    ServerName https://soa.mycompany.com:443
    ServerAdmin you@your.address
    RewriteEngine On
    RewriteOptions inherit
    </VirtualHost>
     
    NameVirtualHost *:7777
    <VirtualHost *:7777>
        ServerName admin.mycompany.com:80
        ServerAdmin you@your.address
        RewriteEngine On
        RewriteOptions inherit
    </VirtualHost>
    

    Note:

    Values such as soa.mycompany.com:443, 7777, admin.mycompany:80, and you@youraddress that are noted in this document serve as examples only. Enter values based on the actual environment.
  3. Perform the same steps for the Oracle HTTP Server on WEBHOST2.

  4. Restart Oracle HTTP Server on both WEBHOST1 and WEBHOST2:

    WEBHOST1> ORACLE_BASE/admin/<instance_name>/bin/opmnctl restartproc ias-component=ohs1
    
    WEBHOST2> ORACLE_BASE/admin/<instance_name>/bin/opmnctl restartproc ias-component=ohs2
    

5.13.15 Validating Access Through Oracle HTTP Server

Verify that the SOA Servers status is reported as "Running" in the Administration Console. If the server is shown as "Starting" or "Resuming," wait for the server status to change to "Started." If another status is reported (such as "Admin" or "Failed"), check the server output log files for errors. Verify that you can access these URLS:

  • http://WEBHOST1:7777/wsm-pm

  • http://WEBHOST2:7777/wsm-pm

  • http://WEBHOST1:7777/soa-infra

  • http://WEBHOST2:7777/soa-infra

  • http://WEBHOST1:7777/soa/composer

  • http://WEBHOST2:7777/soa/composer

  • http://WEBHOST1:7777/integration/worklistapp

  • http://WEBHOST2:7777/integration/worklistapp

  • http://WEBHOST1:7777/sdpmessaging/userprefs-ui

  • http://WEBHOST2:7777/sdpmessaging/userprefs-ui

  • http://WEBHOST1:7777/b2bconsole

  • http://WEBHOST2:7777/b2bconsole

  • http://WEBHOST1:7777/bpm/composer (ONLY FOR BPM Systems)

  • http://WEBHOST2:7777/bpm/composer (ONLY FOR BPM Systems)

  • http://WEBHOST1:7777/bpm/workspace (ONLY FOR BPM Systems)

  • http://WEBHOST2:7777/bpm/workspace (ONLY FOR BPM Systems)

Verify these URLs also using your load balancing router address:

  • http://soa.mycompany.com:80/wsm-pm

  • http://soa.mycompany.com:80/soa-infra

  • http://soa.mycompany.com:80/soa/composer

  • http://soa.mycompany.com:80/integration/worklistapp

  • http://soa.mycompany.com:80/sdpmessaging/userprefs-ui

  • http://soa.mycompany.com:80/b2bconsole

  • http://soa.mycompany.com:80/bpm/composer (ONLY FOR BPM Systems)

  • http://soa.mycompany.com:80/bpm/workspace (ONLY FOR BPM Systems)

Follow these instructions to ensure that routing and failover from the HTTP Server to the SOA_CLuster is working correctly:

  1. While WLS_SOA2 is running, stop WLS_SOA1 from Oracle WebLogic Server Administration Console.

  2. Access the following URLs and verify the appropriate functionality:

    • WEBHOST1:7777/wsm-pm

    • WEBHOST1:7777/soa-infra

    • WEBHOST1:7777/soa/composer

    • WEBHOST1:7777/integration/worklistapp

    • WEBHOST1:7777/sdpmessaging/userprefs-ui

    • WEBHOST1:7777/b2bconsole

    • WEBHOST1:7777/bpm/composer (ONLY FOR BPM Systems)

    • WEBHOST1:7777/bpm/workspace (ONLY FOR BPM Systems)

  3. Start WLS_SOA1 from Oracle WebLogic Server Administration Console.

  4. Stop WLS_SOA2.

  5. Access the URLs in Step 2 above again and verify the appropriate functionality:

5.13.16 Configuring JMS Persistence Store as Shared Across the Servers

Configure the location for all persistence stores to a directory visible from both nodes. Change all persistent stores to use this shared base directory.

From the Fusion Middleware Administration Console, select Services, Persistence Store, Persistence_Store_Name, and then Directory.

To enable resume of pending JMS messages, you must specify a location on a persistent storage solution (NAS, SAN) that is available to other servers in the cluster. Therefore, the directory that you enter must be accessible by both WLS_SOA1 and WLS_SOA2. This directory must exist before the server is restarted.

5.13.17 Configuring a Default Persistent Store for Transaction Recovery

Each server has a transaction log, which stores information about committed transactions coordinated by the server that may not have been completed. WebLogic Server uses the transaction log when recovering from system crashes or network failures. To take advantage of the migration capability of the Transaction Recovery Service for servers in a cluster, you must store the transaction log in a location that is available to a server and its backup servers, preferably on a dual-ported SCSI disk, or a Storage Area Network (SAN). To configure the default persistent store:

  1. In the Domain Structure tree, expand Environment and select Servers.

  2. Select the server you want to modify.

  3. Select the Configuration, Services tab.

  4. Under Default Store, in the Directory field, enter the path to the folder where you want the default persistent store to store its data files.

    To enable migration of the Transaction Recovery Service, you must specify a location on a persistent storage solution that is available to other servers in the cluster. Therefore, the directory that you enter must be accessible by both WLS_SOA1 and WLS_SOA2. This directory must exist before the server is restarted.

5.13.18 Setting the Front End HTTP Host and Port

You need to set the front end HTTP host and port for Oracle WebLogic Server cluster:

  1. In the WebLogic Server Administration Console, in the Change Center section, click Lock & Edit.

  2. In the left pane, select Environment > Clusters.

  3. Select the SOA_Cluster.

  4. Select HTTP.

  5. Set the values for the following:

    • Frontend Host: soa.mycompany.com

      Frontend HTTP Port: 80

    • Frontend HTTPS Port: leave it blank

    Note:

    Make sure this address is correct and available (the load balancing router is up). An incorrect value, for example, http:// in the address, or trailing / in the hostname, may prevent the SOA system from being accessible even when using the virtual IPs to access it.
  6. Click Save.

  7. To activate the changes, click Activate Changes in the Change Center section of the Administration Console.

  8. If you have started the server before, notice this change requires a restart of the managed servers in the cluster.

    Note:

    The SOA system calculates the callback URL as follows:
    • If a request to SOA originates from an external or internal service is originating a request to SOA, then SOA uses the callback URL specified by the client.

    • If a request to an external or internal service originates from SOA, the callbackURL cannot be populated in the SOA request dynamically because SOA is the originator. Instead, callbackServerURL is used if it is specified as a binding property for the specific reference. (You can set this when modeling the composite or at runtime using the MBeans accessed through Oracle Enterprise Manager console.) This allows different service calls to have different callback URLs. That is, a callback URL from an external service is different than one to an internal service.

      However, if the callbackServer URL is not specified as a binding property for that reference, then the system uses the callback URL as specified in soa-infra-config. If the callback URL is not specified in soa-infra-config, then the system uses the callback URL as the front end host specified in WLS. If the front end host is not specified in WLS, the system uses the callback URL as the local host name as provided by WLS MBean APIs.

For SOA high availability installations frontended by Oracle HTTP Server, monitoring should be done on the Oracle HTTP Server ports of the real backend servers. This is the case when a deployment is using all the components deployed to the SOA Managed Server. A simple HTTP monitor that pings the HTTP/HTTPS port and expects a pre-determined response in turn should suffice. If only a specific SOA component is being used (such as B2B), then a monitor that does a deeper level check all the way to the managed server can be considered to validate the health of the component in use. Please check with your load balancer vendor on setting up the HTTP monitors with your load balancer.

If you do not set the front end HTTP host and port, you get the following message when trying to retrieve a document definition XSD from Oracle B2B:

An error occured while loading the document definitions.
java.lang.IllegalArgumentException: Cluster address must be set when
clustering is enabled.

5.13.19 Setting the WLS Cluster Address for Direct Binding/RMI Invocations to Composites

You must set the WLS Cluster address for the SOA_Cluster if you are going to use direct binding to composites. To do this follow these steps:

  1. In the WebLogic Server Administration Console, in the Change Center section, click Lock & Edit.

  2. In the left pane, choose Environment in the Domain Structure window and then choose Clusters. The Summary of Clusters page appears.

  3. Select the SOA_Cluster cluster.

  4. In the Configuration > General tab, enter the following in the Cluster Address field: SOAHOST1VHN1:8001,SOAHOST2VHN1:8001

  5. Click Save.

  6. To activate the changes, click Activate Changes in the Change Center section of the Administration Console.

  7. Restart the servers to make the Frontend Host directive in the cluster effective.

Note:

For asynchronous request/response interactions over direct binding, the SOA composites need to provide the JNDI provider URL for the invoked service to look up the beans for callback.

If soa-infra config properties are not specified but the WebLogic Server cluster address is specified, the cluster address will be used to form the JNDI provider URL. This cluster address can be a single DNS name which maps to the clustered servers' IP-addresses or a comma separated list of server ip:port. Alternatively, if the soa-infra config property JndiProviderURL/SecureJndiProviderURL is explicitly set by users, it can be used also for the same purpose.

5.13.20 Deploying Applications

You can deploy SOA composite applications from Oracle Enterprise Manager Fusion Middleware Control Console with the Deploy SOA Composite wizard. Use the Deploy SOA Composite wizard to deploy any of the following:

  • A new SOA composite application for the first time

  • A new revision (for example, 2.0) alongside an older revision (for example, 1.0) without impacting the latter. The revision deployed last becomes the new default revision of that composite (unless you specify otherwise at a later step during deployment).

  • A bundle (ZIP file) containing multiple SOA composite application revisions (for example, revisions 2.0, 3.0, and 4.0) of a SOA composite application that already has a different revision that is currently deployed (for example, 1.0). This option enables you to deploy revisions 1.0, 2.0, 3.0, and 4.0 at the same time. The bundle can also contain revisions of different composites. There is no restriction that all revisions must be of the same composite application.

Deployment extracts and activates the composite application in SOA Service Infrastructure. Once an application is deployed, you can perform administration tasks, such as creating instances, configuring properties, monitoring performance, managing instance life cycles, and managing policies and faults.

Note:

If you want to redeploy an existing revision of an application, do not use this wizard. Instead, use the Redeploy SOA Composite wizard.

To deploy applications:

  1. Access the Deploy SOA Composite wizard through one of the following options:

    From SOA Infrastructure Menu... From the SOA Folder in the Navigator... From the SOA Infrastructure Home Page... From the SOA Composite Menu...
    1. Select SOA Deployment > Deploy.
    1. Right-click soa-infra.
    2. Select SOA Deployment > Deploy.

    1. Click the Deployed Composites tab.
    2. Above the Composite table, click Deploy.

    1. Select SOA Deployment > Deploy Another Composite.

    The Select Archive page appears.

    Description of sca_deploy.gif follows
    Description of the illustration sca_deploy.gif

  2. In the Archive or Exploded Directory section, specify the archive of the SOA composite application to deploy. The archive contains the project files of the composite to be deployed (for example, HelloWorld_rev1.0.jar for a single archive or OrderBooking_rev1.0.zip for multiple archives).

  3. In the Configuration Plan section, optionally specify the configuration plan to include with the archive. The configuration plan enables you to define the URL and property values to use in different environments. During process deployment, the configuration plan is used to search the SOA project for values that must be replaced to adapt the project to the next target environment.

  4. Click Next.

    The Select Target page appears.

  5. Select the WebLogic Server or cluster to which to deploy the SOA composite application archive. You can deploy to multiple servers and clusters.

  6. Click Next.

    The Confirmation page appears.

  7. Review your selections.

  8. Select whether or not to deploy the SOA composite application as the default revision. The default revision is instantiated when a new request comes in.

  9. Click Deploy.

    Processing messages are displayed.

  10. When deployment has completed, click Close.

See Also:

Oracle Fusion Middleware Developer's Guide for Oracle SOA Suite for instructions on creating configuration plans and deploying applications from Oracle JDeveloper

5.13.21 Configuring Server Migration for the WLS_SOA Servers

The high availability architecture for a SOA system uses server migration to protect some singleton services against failures. For more information on whole server migration, see Oracle Fusion Middleware Using Clusters for Oracle WebLogic Server.

The WLS_SOA1 managed server is configured to be restarted on SOAHOST2 in case of failure, and the WLS_SOA2 managed server is configured to be restarted on SOAHOST1 in case of failure. For this configuration the WLS_SOA1 and WLS_SOA2 servers listen on specific floating IPs that are failed over by WLS Server Migration. To configure server migration for the WLS_SOAn managed servers, follow these steps:

Step 1   Set Up User and Tablespace for the Server Migration Leasing Table
  1. Create a tablespace called leasing.

    Example: Log on to SQL*Plus as the sysdba user and run the following command:

    SQL> create tablespace leasing
            logging datafile 'DB_HOMEs/oradata/orcl/leasing.dbf'
            size 32m autoextend on next 32m maxsize 2048m extent management local;
    
  2. Create a user named leasing and assign to it the leasing tablespace.

    SQL> create user leasing identified by welcome1;
    
    SQL> grant create table to leasing;
    
    SQL> grant create session to leasing;
    
    SQL> alter user leasing default tablespace leasing;
    
    SQL> alter user leasing quota unlimited on LEASING;
    
  3. Create the leasing table using the leasing.ddl script.

    1. Copy the leasing.ddl file, located in the WL_HOME/server/db/oracle/817 or WL_HOME/server/db/oracle/920 directories, to your database node.

    2. Connect to the database as the leasing user.

    3. Run the leasing.ddl script in SQL*Plus.

      SQL> @copy_location/leasing.ddl;
      
Step 2   Create a Multi Data Source from Oracle WebLogic Server Administration Console

The second step is to create a multi data source for the leasing table from the Oracle WebLogic Server Administration Console:

You create a data source to each of the Oracle RAC database instances during the process of setting up the multi data source, both for these data sources and the global leasing multi data source. When you create a data source:

  • Make sure that this is a non-xa data source.

  • The names of the multi data sources are in the following format: <MultiDS>-rac0, <MultiDS>-rac1

  • Use Oracle's Driver (Thin) Version 9.0.1, 9.2.0, 10, 11.

  • Data sources do not require support for global transactions. Therefore, do not use any type of distributed transaction emulation/participation algorithm for the data source (do not choose the Supports Global Transactions option, or the Logging Last Resource, Emulate Two-Phase Commit, or One-Phase Commit options of the Supports Global Transactions option), and specify a service name for your database.

  • Target these data sources to the SOA cluster.

  • Make sure the datasources' connection pool initial capacity is set to 0. To do this, select Services, JDBC, and then Datasources. In the Datasources screen, click the Datasource Name, then click the Connection Pool tab, and enter 0 in the Initial capacity field.

To create a multi data source:

  1. From the Domain Structure window in the Oracle WebLogic Server Administration Console, expand the Services node, then click Data Sources. The Summary of JDBC Data Sources page appears.

  2. In the Change Center, click Lock & Edit.

  3. Click New, then click Multi Data Sources. The Create a New JDBC Multi Data Source page appears.

  4. Enter leasing as the Name.

  5. Enter jdbc/leasing as the JNDI name.

  6. Select Failover as algorithm (default).

  7. Click Next.

  8. Select non-XA driver (the default).

  9. Click Next.

  10. Click Create New Data Source.

  11. Enter leasing-rac0 as name. Enter jdbc/leasing-rac0 as JNDI name. Enter oracle as the database type. For the driver type, enter Oracle Driver (Thin) for RAC server-Instance connection Version 10,11.

  12. Click Next.

  13. Deselect Supports Global Transactions.

  14. Click Next.

  15. Enter the service name, database name (this is actually the RAC Node instance name, for example: racdb1,racdb2), host port, and password for your leasing schema.

  16. Click Next.

  17. Click Test Configuration and verify the connection works.

  18. Target the data source to the SOA cluster.

  19. Select the data source and add it to the right screen.

  20. Click Create a New Data Source and repeat the steps for the second instance of your Oracle RAC database.

  21. Add the second data source to your multi data source.

  22. Save and Activate the changes.

Step 3   Edit the Node Manager's Properties file

For information on Node Manager and whole server migration, see Section 3.9, "Whole Server Migration."

The nodemanager.properties file is located in the WL_HOME/common/nodemanager directory. For server migration to work properly, you need to add the properties listed in this section:

  • Interface=eth0

    Note:

    Do not specify the sub interface, such as eth0:1 or eth0:2. This interface is to be used without the :0, or :1. The Node Manager's scripts traverse the different :X enabled IPs to determine which to add or remove. For example, the valid values in Linux environments are eth0, eth1, or, eth2, eth3, ethn, depending on the number of interfaces configured.

    This property specifies the interface name for the floating IP (eth0, for example, on Linux).

    Note:

    For Windows, the Interface should be set to the Network Interface Name. For example: Interface="Local Area Connection".

    Be sure that the interface provided is the public interface for this node. On multi home nodes, this interface should be the one on which the floating IP can be enabled.

  • NetMask=255.255.255.0

    This property specifies the net mask for the interface for the floating IP. The netmask provided (255.255.255.0) is just an example. The actual value depends on your network.

  • UseMACBroadcast=true

    This property specifies whether or not to use a node's MAC address when sending ARP packets, that is, whether or not to use the -b flag in the arping command.

After starting Node Manager, verify in Node Manager's output (shell where Node Manager is started) that these properties are being used, or problems may arise during migration. You should see something like this in the Node Manager's output:

StateCheckInterval=500

Interface=eth0 (Linux) or Interface="Local Area Connection" (Windows)

NetMask=255.255.255.0

UseMACBroadcast=true

Note:

The steps in this section are not required if the server properties (start properties) have been properly set and the Node Manager can start the servers remotely.
  1. Set the following property in the nodemanager.properties file.

    • StartScriptEnabled

      Set this property to true.

  2. Start the Node Manager on Node 1 and Node 2 by running the startNodeManager.sh script located in the WL_HOME/server/bin directory.

  3. Validate the changes to the nodemanager.properties file by checking the nodemanager.log file.

Step 4   Set Environment and Superuser Privileges for the wlsifconfig.sh script
  1. Grant sudo privilege to the WebLogic user ('oracle') with No Password restriction, and grant execute privilege on /sbin/ifconfig and /sbin/arping binaries.

    Note:

    On Windows, the script is named wlsifconfig.cmd and it can be run by users with the Administrator privilege.

    Make sure the script is executable by the WebLogic user ('oracle'). The following is an example of an entry inside /etc/sudoers granting sudo execution privilege for oracle and also over the ifconfig and arping:

    oracle ALL=NOPASSWD: /sbin/ifconfig,/sbin/arping
    
Step 5   Configure Server Migration Targets

Configuring Cluster Migration sets the DataSourceForAutomaticMigration property to true. Follow the steps in this section to configure cluster migration in a migration in a cluster:

  1. Log into Oracle WebLogic Server Administration Console

  2. In the left pane, expand Environment and select Clusters.

  3. Select the cluster for which you want to configure migration (SOA_Cluster).

  4. Click Migration.

  5. In the Change Center, click Lock & Edit.

  6. In the Available field, select the machine to which to allow migration and click the right arrow. In this case, select SOAHOST1 and SOAHOST2.

  7. Select the data source to be used for automatic migration. In this case select the leasing data source.

  8. Click Save.

  9. Set the Candidate Machines for Server Migration. This needs to be done for all the managed servers.

    To do this, follow these steps:

    1. In the left pane of Oracle WebLogic Server Administration Console, expand Environment and select Servers.

    2. Select the server for which you want to configure migration.

    3. Click the Migration tab.

    4. In the Available field, located in the Migration Configuration section, select the machines to which to allow migration and click the right arrow. For WLS_SOA1, select SOAHOST2. For WLS_SOA2, select SOAHOST1.

    5. Select the checkbox for Automatic Server Migration Enabled. This enables the Node Manager to start a failed server on the target node automatically.

    6. Click Save and activate the changes.

    7. These changes require restart of the servers and Node Managers to be effective.

Step 6   Test Server Migration

To verify that Server Migration is working properly, follow these steps:

From SOAHOST1:

  1. Force stop the WLS_SOA1 managed server.

    To do this, run this command:

    SOAHOST1> kill -9 <pid>
    

    pid specifies the process ID of the managed server. You can identify the pid in the node by running this command:

    SOAHOST1> ps -ef | grep WLS_SOA1
    

    Note:

    For Windows, the Managed Server can be terminated by using the taskkill command. For example:
    taskkill /f /pid <pid>
    

    Where <pid> is the process Id of the Managed Server.

    To determine the process Id of the Managed Server run the following command and identify the pid of the WLS_SOA1 Managed Server.

    MW_HOME\jrockit_160_20_D1.0.1-2124\bin\jps -l -v
    
  2. Watch the Node Manager console: you should see a message indicating that WLS_SOA1's floating IP has been disabled.

  3. Wait for the Node Manager to try a second restart of WLS_SOA1. Node Manager waits for a fence period of 30 seconds before trying this restart.

  4. Once Node Manager restarts the server, stop it again. Now Node Manager should log a message indicating that the server will not be restarted again locally.

From SOAHOST2:

  1. Watch the local Node Manager console. After 30 seconds since the last try to restart WLS_SOA1 on Node 1, Node Manager on Node 2 should prompt that the floating IP for WLS_SOA1 is being brought up and that the server is being restarted in this node.

  2. Access the soa-Service Infrastructure console in the same IP.

Migration can also be verified in Oracle WebLogic Server Administration Console:

  1. Log into Oracle WebLogic Server Administration Console.

  2. Click on Domain on the left pane.

  3. Click on the Monitoring tab and then on the Migration sub-tab.

    The Migration Status table provides information on the status of the migration.

    Figure 5-46 Migration Status Screen in Oracle WebLogic Server Administration Console

    Migration Status screen in the Administration Console
    Description of "Figure 5-46 Migration Status Screen in Oracle WebLogic Server Administration Console"

Note:

On Windows, when you manually shut down multiple servers at the same time, on the same machine, and then, on another machine, attempt to start one of the servers that was shut down, the IP bind may not work. This happens because the original machine still has claim to the IP address, even though netsh has reported that the IP address has been removed.

To resolve this, you must check the network configuration either by using the ipconfig utility or Windows Network Configuration. Either of these may show that one of the virtual/floating IP addresses is still configured even though the servers have been shut down. You can then use Windows Network Configuration to remove the IP address using the following procedure:

  1. From Windows Control Panel, select Network Connections.

  2. Select the appropriate network interface, right-click, and select Properties.

  3. Select Internet Protocol (TCP/IP) and click the Properties button.

  4. Select Advanced.

  5. Select the appropriate IP address and click the Remove button.

Note:

After a server is migrated, to fail it back to its original node/machine, stop the managed server from the Oracle WebLogic Administration Console and then start it again. The appropriate Node Manager will start the managed server on the machine to which it was originally assigned.

5.13.22 Scaling the Topology

You can scale out or scale up this high availability topology. When you "scale up" the topology, you add new managed servers to nodes that are already running one or more managed servers. When you "scale out" the topology, you add new managed servers to new nodes.

5.13.22.1 Scaling Up the Topology (Adding Managed Servers to Existing Nodes)

In this case, you already have a node that runs a managed server configured with SOA components. The node contains a Middleware home, an Oracle HOME (SOA) and a domain directory for existing managed servers.

You can use the existing installations (the Middleware home, and domain directories) for creating new WLS_SOA servers. There is no need to install SOA binaries in a new location, or to run pack and unpack.

Follow these steps for scaling up the topology:

  1. Using the Administration Console, clone WLS_SOA1 into a new managed server. The source managed server to clone should be one that already exists on the node where you want to run the new managed server.

    To clone a managed server:

    1. Select Environment -> Servers from the Administration Console.

    2. Select the managed server that you want to clone (for example, WLS_SOA1).

    3. Select Clone.

    Name the new managed server WLS_SOAn, where n is a number to identify the new managed server.

    The rest of the steps assume that you are adding a new server to SOAHOST1, which is already running WLS_SOA1.

  2. For the listen address, assign the virtual host name to use for this new managed server. If you are planning to use server migration as recommended for this server, this virtual host name will enable it to move to another node. The virtual host name should be different from the one used by the managed server that is already running.

    Perform these steps to set the managed server listen address:

    1. Log into the Oracle WebLogic Server Administration Console.

    2. In the Change Center, click Lock & Edit.

    3. Expand the Environment node in the Domain Structure window.

    4. Click Servers. The Summary of Servers page is displayed.

    5. Select the managed server whose listen address you want to update in the Names column of the table. The Setting page for that managed server is displayed.

    6. Set the Listen Address to SOAHOST1VHN1.

    7. Click Save.

    8. Save and activate the changes.

    The changes will not take effect until the managed server is restarted.

  3. Update the cluster address to include the new server:

    1. Select Environment -> Cluster in the Domain Structure window in the Administration Console.

    2. Click on the SOA_Cluster server. The Settings Screen for the SOA_Cluster appears.

    3. In the Change Center, click Lock & Edit.

    4. Add the new server's address and port to the Cluster Address field. For example:

      SOAHOST1VHN1:8001,SOAHOST2VHN1:8001,SOAHOST1VHN2:8001
      
  4. Create JMS Servers for SOA, BPM (if applicable) and UMS on the new managed server.

    1. Use the Oracle WebLogic Server Administration Console to create a new persistent store for the new SOAJMSServer and name it, for example, SOAJMSFileStore_N. Specify the path for the store. This should be a directory on shared storage, as recommended in Section 5.13.1.3, "Shared Storage Prerequisites":

      Note:

      This directory must exist before the managed server is started or the start operation fails.
      ORACLE_BASE/admin/DOMAIN_NAME/cluster_name/jms/SOAJMSFileStore_N
      
    2. Create a new JMS Server for SOA, for example, SOAJMSServer_N. Use the SOAJMSFileStore_N for this JMSServer. Target the SOAJMSServer_N Server to the recently created Managed Server (WLS_SOAn).

    3. Create a new persistence store for the new UMSJMSServer, for example, UMSJMSFileStore_N Specify the path for the store. This should be a directory on shared storage as recommended in Section 5.13.1.3, "Shared Storage Prerequisites":

      ORACLE_BASE/admin/domain_name/cluster_name/jms/UMSJMSFileStore_N.
      

      Note:

      This directory must exist before the managed server is started or the start operation fails.

      You can also assign SOAJMSFileStore_N as store for the new UMS JMS Servers. For the purpose of clarity and isolation, individual persistent stores are used in the following steps.

    4. Create a new JMS Server for UMS, for example, UMSJMSServer_N. Use the UMSJMSFileStore_N for this JMSServer. Target the UMSJMSServer_N Server to the recently created Managed Server (WLS_SOAn).

    5. (ONLY FOR BPM Systems) Create a new persistence store for the new BPMJMSServer, for example, BPMJMSFileStore_N Specify the path for the store. This should be a directory on shared storage as recommended in Section 5.13.1.3, "Shared Storage Prerequisites":

      ORACLE_BASE/admin/domain_name/cluster_name/jms/BPMJMSFileStore_N.
      

      Note:

      This directory must exist before the managed server is started or the start operation fails.

      You can also assign SOAJMSFileStore_N as store for the new BPM JMS Servers. For the purpose of clarity and isolation, individual persistent stores are used in the following steps.

    6. (ONLY FOR BPM Systems) Create a new JMS Server for BPM, for example, BPMJMSServer_N. Use the BPMJMSFileStore_N for this JMSServer. Target the BPMJMSServer_N Server to the recently created Managed Server (WLS_SOAn).

    7. Update the SubDeployment targets for the SOA JMS Module to include the recently created SOA JMS Server. To do this, expand the Services node and then expand the Messaging node. Choose JMS Modules from the Domain Structure window of the Oracle WebLogic Server Administration Console. The JMS Modules page appears. Click SOAJMSModule (represented as a hyperlink in the Names column of the table). The Settings page for SOAJMSModule appears. Click the SubDeployments tab. The subdeployment module for SOAJMS appears.

      Note:

      This subdeployment module name is a random name in the form of SOAJMSServerXXXXXX resulting from the Config Wizard JMS configuration for the first two servers (WLS_SOA1 and WLS_SOA2).

      Click the SOAJMSServerXXXXXX subdeployment. Add the new JMS Server for SOA called SOAJMSServer_N to this subdeployment. Click Save.

    8. Update the SubDeployment targets for the UMSJMSSystemResource to include the recently created UMS JMS Server. To do this, expand the Services node and then expand the Messaging node. Choose JMS Modules from the Domain Structure window of the Oracle WebLogic Server Administration Console. The JMS Modules page appears. Click UMSJMSSystemResource (represented as a hyperlink in the Names column of the table). The Settings page for UMSJMSSystemResource appears. Click the SubDeployments tab. The subdeployment module for UMSJMS appears.

      Note:

      This subdeployment module name is a random name in the form of UCMJMSServerXXXXXX resulting from the Config Wizard JMS configuration for the first two servers (WLS_SOA1 and WLS_SOA2).

      Click the UMSJMSServerXXXXXX subdeployment. Add the new JMS Server for UMS called UMSJMSServer_N to this subdeployment. Click Save.

    9. (ONLY FOR BPM Systems) Update the SubDeployment targets for BPMJMSModule to include the recently created BPM JMS Server. To do this, expand the Services node and then expand the Messaging node. Choose JMS Modules from the Domain Structure window of the Oracle WebLogic Server Administration Console. The JMS Modules page appears. Click BPMJMSSystemResource (represented as a hyperlink in the Names column of the table). The Settings page for BPMJMSSystemResource appears. Click the SubDeployments tab. The subdeployment module for BPMJMS appears.

      Note:

      This subdeployment module name is a random name in the form of BPMJMSServerXXXXXX resulting from the Config Wizard JMS configuration for the first two servers (WLS_SOA1 and WLS_SOA2).

      Click the BPMJMSServerXXXXXX subdeployment. Add the new JMS Server for BPM called BPMJMSServer_N to this subdeployment. Click Save.

  5. Configure Oracle Coherence for deploying composites, as described in Section 5.13.8, "Configuring Oracle Coherence for Deploying Composites."

    Note:

    Only the localhost field needs to be changed for the server. Replace the localhost with the listen address of the new server added, for example:
    Dtangosol.coherence.localhost=SOAHOST1VHNn
    
  6. Configure TX persistent store for the new server. This should be a location visible from other nodes as indicated in the recommendations about shared storage.

    From the Administration Console, select Server_name > Services tab. Under Default Store, in Directory, enter the path to the folder where you want the default persistent store to store its data files.

  7. Disable host name verification for the new managed server. Before starting and verifying the WLS_SOAn managed server, you must disable host name verification. You can re-enable it after you have configured server certificates for the communication between the Oracle WebLogic Administration Server and the Node Manager in SOAHOSTn. If the source server from which the new one has been cloned had already disabled hostname verification, these steps are not required (the hostname verification settings is propagated to the cloned server).

    To disable host name verification:

    1. In the Oracle Enterprise Manager Console, select Oracle WebLogic Server Administration Console.

    2. Expand the Environment node in the Domain Structure window.

    3. Click Servers. The Summary of Servers page appears.

    4. Select WLS_SOAn in the Names column of the table. The Settings page for the server appears.

    5. Click the SSL tab.

    6. Click Advanced.

    7. Set Hostname Verification to None.

    8. Click Save.

  8. Start and test the new managed server from the Administration Console.

    1. Shut down the existing managed servers in the cluster.

    2. Ensure that the newly created managed server, WLS_SOAn, is up.

    3. Access the application on the newly created managed server (http://vip:port/soa-infra). The application should be functional.

  9. Configure Server Migration for the new managed server.

    Note:

    Since this is a scale-up operation, the node should already contain a Node Manager and environment configured for server migration. The floating IP for the new SOA managed server should also be already present.

    Configure server migration following these steps:

    1. Log into the Administration Console.

    2. In the left pane, expand Environment and select Servers.

    3. Select the name of the new managed server for which you want to configure migration.

    4. Click the Migration tab.

    5. In the Available field, in the "Migration Configuration" section, select the machines to which to allow migration and click the right arrow. Select the same migration targets as for the servers that already exist on the node.

      For example, for new managed servers on SOAHOST1, which is already running WLS_SOA1, select SOAHOST2. For new managed servers on SOAHOST2, which is already running WLS_SOA2, select SOAHOST1.

      Make sure the appropriate resources are available to run the managed servers concurrently during migration.

    6. Select the "Automatic Server Migration Enabled" option. This enables the Node Manager to start a failed server on the target node automatically.

    7. Click Save.

    8. Restart the Administration Server, managed servers, and Node Manager.

  10. Test server migration for this new server. Follow these steps from the node where you added the new server:

    1. Stop the WLS_SOAn managed server.

      To do this, run "kill -9 <pid>" on the PID of the managed server. You can identify the PID of the node using "ps -ef | grep WLS_SOAn".

      Note:

      For Windows, the Managed Server can be terminated by using the taskkill command. For example:

      taskkill /f /pid <pid>

      Where <pid> is the process Id of the Managed Server.

      To determine the process Id of the Managed Server run the following command and identify the pid of the WLS_SOAn Managed Server.

      MW_HOME\jrockit_160_20_D1.0.1-2124\bin\jps -l -v

    2. Watch the Node Manager Console: you should see a message indicating that WLS_SOAn's floating IP has been disabled.

    3. Wait for the Node Manager to try a second restart of WLS_SOAn. Node Manager waits for a fence period of 30 seconds before trying this restart.

    4. Once Node Manager restarts the server, stop it again. Now Node Manager should log a message indicating that the server will not be restarted again locally.

    Note:

    After a server is migrated, to fail it back to its original node/machine, stop the managed server from the Oracle WebLogic Administration Console and then start it again. The appropriate Node Manager will start the managed server on the machine to which it was originally assigned.

5.13.22.2 Scaling Out the Topology (Adding Managed Servers to New Nodes)

When you scale out the topology, you add new managed servers configured with SOA to new nodes.

Before performing the steps in this section, check that you meet these requirements:

  • There must be existing nodes running managed servers configured with SOA within the topology.

  • The new node can access the existing home directories for WebLogic Server and SOA. (Use the existing installations in shared storage for creating a new WLS_SOA or WLS_WSM managed server. You do not need to install WebLogic Server or SOA binaries in a new location but you do need to run pack and unpack to bootstrap the domain configuration in the new node.)

    Note:

    If there is no existing installation in shared storage, installing WebLogic Server and SOA in the new nodes is required as described in Section 5.13, "Configuring High Availability for Oracle SOA Service Infrastructure and Component Service Engines."

    Note:

    When an ORACLE_HOME or WL_HOME is shared by multiple servers in different nodes, Oracle recommends keeping the Oracle Inventory and Middleware home list in those nodes updated for consistency in the installations and application of patches. To update the oraInventory in a node and "attach" an installation in a shared storage to it, use ORACLE_HOME/oui/bin/attachHome.sh. To update the Middleware home list to add or remove a WL_HOME, edit the user_home/bea/beahomelist file. See the following steps.

Follow these steps for scaling out the topology:

  1. On the new node, mount the existing Middleware home, which should include the SOA installation and the domain directory, and ensure that the new node has access to this directory, just like the rest of the nodes in the domain.

  2. To attach ORACLE_HOME in shared storage to the local Oracle Inventory, execute the following command:

    SOAHOSTn>cd ORACLE_BASE/product/fmw/soa/
    SOAHOSTn>./attachHome.sh -jreLoc ORACLE_BASE/fmw/jrockit_160_20_D1.0.1-2124
    

    To update the Middleware home list, create (or edit, if another WebLogic installation exists in the node) the MW_HOME/bea/beahomelist file and add ORACLE_BASE/product/fmw to it.

  3. Log in to the Oracle WebLogic Administration Console.

  4. Create a new machine for the new node that will be used, and add the machine to the domain.

  5. Update the machine's Node Manager's address to map the IP of the node that is being used for scale out.

  6. Use the Oracle WebLogic Server Administration Console to clone WLS_SOA1 into a new managed server. Name it WLS_SOAn, where n is a number and assign it to the new machine.

    Note:

    These steps assume that you are adding a new server to node n, where no managed server was running previously.
  7. For the listen address, assign the virtual host name to use for this new managed server. If you are planning to use server migration as recommended for this server, this virtual host name will enable it to move to another node. The virtual host name should be different from the one used by the managed server that is already running.

    Perform these steps to set the managed server listen address:

    1. Log into the Oracle WebLogic Server Administration Console.

    2. In the Change Center, click Lock & Edit.

    3. Expand the Environment node in the Domain Structure window.

    4. Click Servers. The Summary of Servers page is displayed.

    5. Select the managed server whose listen address you want to update in the Names column of the table. The Setting page for that managed server is displayed.

    6. Set the Listen Address to SOAHOSTnVHN1.

    7. Click Save.

    8. Save and activate the changes.

    The changes will not take effect until the managed server is restarted.

  8. Update the cluster address to include the new server:

    1. Select Environment -> Cluster from the Administration Console.

    2. Click on the SOA_Cluster server. The Settings Screen for the SOA_Cluster appears.

    3. In the Change Center, click Lock & Edit.

    4. Add the new server's address and port to the Cluster Address field. For example:

      SOAHOST1VHN1:8001,SOAHOST2VHN1:8001,SOAHOSTNVHN1:8001
      
  9. Create JMS servers for SOA, BPM (if applicable), and UMS on the new managed server.

    1. Use the Oracle WebLogic Server Administration Console to create a new persistent store for the new SOAJMSServer and name it, for example, SOAJMSFileStore_N. Specify the path for the store. This should be a directory on shared storage as recommended in Section 5.13.1.3, "Shared Storage Prerequisites."

      ORACLE_BASE/admin/domain_name/cluster_name/jms/SOAJMSFileStore _N
      

      Note:

      This directory must exist before the managed server is started or the start operation fails.
    2. Create a new JMS Server for SOA, for example, SOAJMSServer_N. Use the SOAJMSFileStore_N for this JMSServer. Target the SOAJMSServer_N Server to the recently created managed server (WLS_SOAn).

    3. Create a new persistence store for the new UMSJMSServer, and name it, for example, UMSJMSFileStore_N. Specify the path for the store. This should be a directory on shared storage as recommended in Section 5.13.1.3, "Shared Storage Prerequisites."

      ORACLE_BASE/admin/domain_name/cluster_name/jms/UMSJMSFileStore _N
      

      Note:

      This directory must exist before the managed server is started or the start operation fails.

      Note:

      It is also possible to assign SOAJMSFileStore_N as the store for the new UMS JMS Servers. For the purpose of clarity and isolation, individual persistent stores are used in the following steps.
    4. Create a new JMS Server for UMS: for example, UMSJMSServer_N. Use the UMSJMSFileStore_N for this JMS Server. Target the UMSJMSServer_N Server to the recently created managed server (WLS_SOAn).

    5. (ONLY FOR BPM Systems) Create a new persistence store for the new BPMJMSServer, and name it, for example, BPMJMSFileStore_N. Specify the path for the store. This should be a directory on shared storage as recommended in Section 5.13.1.3, "Shared Storage Prerequisites."

      ORACLE_BASE/admin/domain_name/cluster_name/jms/BPMJMSFileStore _N
      

      Note:

      This directory must exist before the managed server is started or the start operation fails.

      Note:

      It is also possible to assign SOAJMSFileStore_N as the store for the new BPM JMS Servers. For the purpose of clarity and isolation, individual persistent stores are used in the following steps.
    6. (ONLY FOR BPM Systems) Create a new JMS Server for BPM: for example, BPMJMSServer_N. Use the BPMJMSFileStore_N for this JMS Server. Target the BPMJMSServer_N Server to the recently created managed server (WLS_SOAn).

    7. Update the SubDeployment targets for the SOA JMS Module to include the recently created SOA JMS Server. To do this, expand the Services node and then expand the Messaging node. Choose JMS Modules from the Domain Structure window of the Oracle WebLogic Server Administration Console. The JMS Modules page appears. Click SOAJMSModule (represented as a hyperlink in the Names column of the table). The Settings page for SOAJMSModule appears. Open the SubDeployments tab. The subdeployment module for SOAJMS appears.

      Note:

      This subdeployment module name is a random name in the form of SOAJMSServerXXXXXX resulting from the Config Wizard JMS configuration for the first two servers (WLS_SOA1 and WLS_SOA2).

      Click the SOAJMSServerXXXXXX subdeployment. Add the new JMS Server for SOA called SOAJMSServer_N to this subdeployment. Click Save.

    8. Update the SubDeployment targets for UMSJMSSystemResource to include the recently created UMS JMS Server. To do this, expand the Services node and then expand the Messaging node. Choose JMS Modules from the Domain Structure window of the Oracle WebLogic Server Administration Console. The JMS Modules page appears. Click UMSJMSSystemResource (represented as a hyperlink in the Names column of the table). The Settings page for UMSJMSSystemResource appears. Open the SubDeployments tab. The subdeployment module for UMSJMS appears

      Note:

      This subdeployment module is a random name in the form of UMSJMSServerXXXXXX resulting from the Config Wizard JMS configuration for the first two servers (WLS_SOA1 and WLS_SOA2).

      Click the UMSJMSServerXXXXXX subdeployment. Add the new JMS Server for UMS called UMSJMSServer_N to this subdeployment. Click Save.

    9. (ONLY FOR BPM Systems) Update the SubDeployment Targets for BPMJMSModule to include the recently created BPM JMS Server. To do this, expand the Services node and then expand the Messaging node. Choose JMS Modules from the Domain Structure window of the Oracle WebLogic Server Administration Console. The JMS Modules page appears. Click BPMJMSSystemResource (represented as a hyperlink in the Names column of the table). The Settings page for BPMJMSSystemResource appears. Click the SubDeployments tab. The subdeployment module for BPMJMS appears.

      Note:

      This subdeployment module is a random name in the form of BPMJMSServerXXXXXX resulting from the Config Wizard JMS configuration for the first two servers (WLS_SOA1 and WLS_SOA2).

      Click the BPMJMSXXXXXX subdeployment. Add the new JMS Server for BPM called BPMJMSServer_N to this subdeployment. Click Save.

  10. Run the pack command on SOAHOST1 to create a template pack as follows:

    SOAHOST1> cd ORACLE_COMMON_HOME/common/bin
     
    SOAHOST1> ./pack.sh -managed=true -domain=MW_HOME/user_projects/domains/soadomain/
    -template=soadomaintemplateScale.jar -template_name=soa_domain_templateScale
    

    Run the following command on SOAHOST1 to copy the template file created to SOAHOSTN:

    SOAHOST1> scp soadomaintemplateScale.jar oracle@SOAHOSTN:/ ORACLE_BASE/product/fmw/soa/common/bin
    

    Run the unpack command on SOAHOSTN to unpack the template in the managed server domain directory as follows:

    SOAHOSTN> cd ORACLE_BASE/product/fmw/soa/common/bin
     
    SOAHOSTN> ./unpack.sh -domain=ORACLE_BASE/product/fmw/user_projects/domains/soadomain/ -template=soadomaintemplateScale.jar
    
  11. Configure Oracle Coherence for deploying composites, as described in Section 5.13.8, "Configuring Oracle Coherence for Deploying Composites."

    Note:

    Only the localhost field needs to be changed for the server. Replace the localhost with the listen address of the new server added, for example:
    Dtangosol.coherence.localhost=SOAHOSTnVHN1
    
  12. Configure TX persistent store for the new server. This should be a location visible from other nodes as indicated in the recommendations about shared storage.

    From the Administration Console, select Server_name > Services tab. Under Default Store, in Directory, enter the path to the folder where you want the default persistent store to store its data files.

  13. Disable host name verification for the new managed server. Before starting and verifying the WLS_SOAN managed server, you must disable host name verification. You can re-enable it after you have configured server certificates for the communication between the Oracle WebLogic Administration Server and the Node Manager in SOAHOSTn. If the source server from which the new one has been cloned had already disabled hostname verification, these steps are not required (the hostname verification setting is propagated to the cloned server).

    To disable host name verification:

    1. In the Oracle Enterprise Manager Console, select Oracle WebLogic Server Administration Console.

    2. Expand the Environment node in the Domain Structure window.

    3. Click Servers. The Summary of Servers page appears.

    4. Select WLS_SOAn in the Names column of the table.

      The Settings page for server appears.

    5. Click the SSL tab.

    6. Click Advanced.

    7. Set Hostname Verification to None.

    8. Click Save.

  14. Start the Node Manager on the new node. To start the Node Manager, use the installation in shared storage from the existing nodes, and start Node Manager by passing the host name of the new node as a parameter as follows:

    SOAHOSTN> WL_HOME/server/bin/startNodeManager <new_node_ip>
    
  15. Start and test the new managed server from the Oracle WebLogic Server Administration Console:

    1. Shut down all the existing managed servers in the cluster.

    2. Ensure that the newly created managed server, WLS_SOAn, is running.

    3. Access the application on the newly created managed server (http://vip:port/soa-infra). The application should be functional.

  16. Configure server migration for the new managed server.

    Note:

    Since this new node is using an existing shared storage installation, the node is already using a Node Manager and an environment configured for server migration that includes netmask, interface, wlsifconfig script superuser privileges. The floating IP for the new SOA Managed Server is already present in the new node.

    Configure server migration following these steps:

    1. Log into the Administration Console.

    2. In the left pane, expand Environment and select Servers.

    3. Select the server (represented as hyperlink) for which you want to configure migration from the Names column of the table. The Setting page for that server appears.

    4. Click the Migration tab.

    5. In the Available field, in the Migration Configuration section, select the machines to which to allow migration and click the right arrow.

      Note:

      Specify the least-loaded machine as the migration target for the new server. The required capacity planning must be completed so that this node has enough available resources to sustain an additional managed server.
    6. Select the Automatic Server Migration Enabled option. This enables the Node Manager to start a failed server on the target node automatically.

    7. Click Save.

    8. Restart the Administration Server, managed servers, and Node Manager.

    9. Test server migration for this new server. Follow these steps from the node where you added the new server:

      1. Abruptly stop the WLS_SOAn managed server.

      2. To do this, run "kill -9 <pid>" on the PID of the managed server. You can identify the PID of the node using "ps -ef | grep WLS_SOAn".

      Note:

      For Windows, the Managed Server can be terminated by using the taskkill command. For example:

      taskkill /f /pid <pid>

      Where <pid> is the process Id of the Managed Server.

      To determine the process Id of the Managed Server run the following command and identify the pid of the WLS_SOAn Managed Server.

      MW_HOME\jrockit_160_20_D1.0.1-2124\bin\jps -l -v

      3. Watch the Node Manager Console: you should see a message indicating that WLS_SOA1's floating IP has been disabled.

      4. Wait for the Node Manager to try a second restart of WLS_SOAn. Node Manager waits for a fence period of 30 seconds before trying this restart.

      5. Once Node Manager restarts the server, stop it again. Now Node Manager should log a message indicating that the server will not be restarted again locally.

    Note:

    After a server is migrated, to fail it back to its original node/machine, stop the managed server from the Oracle WebLogic Administration Console and then start it again. The appropriate Node Manager will start the managed server on the machine to which it was originally assigned.

5.14 Configuring High Availability for Oracle Service Bus, with SOA Service Infrastructure and Component Service Engines

The procedures described in this section include setting up Oracle Service Bus, and also other Service engines contained in the Oracle SOA Service Infrastructure system (such as Oracle BPEL/Oracle PM, Oracle Mediator, Oracle Human Workflow and Oracle Decision Services, as well as Oracle B2B and Oracle User Messaging Service).

This section includes these topics:

Note:

Oracle strongly recommends reading the release notes for any additional installation and deployment considerations prior to starting the setup process.

Note:

The architectures and deployment procedures defined in this guide enable simple clustered deployments. The procedures described in these chapters can be used as a building block to enable this and other similar high availability topologies for these Fusion Middleware components. It is also expected that production deployments will use other required procedures, such as associating security policies with a centralized LDAP server. For complete details of secured, multi-tiered architectures, and deployment procedures, please refer to the Enterprise Deployment Guide for the component you are configuring.

Figure 5-47 represents the example architecture that the configuration steps in this section create.

Figure 5-47 Oracle SOA Service Infrastructure, Component Services Engines, and Oracle Service Bus High Availability Architecture

Description of Figure 5-47 follows
Description of "Figure 5-47 Oracle SOA Service Infrastructure, Component Services Engines, and Oracle Service Bus High Availability Architecture"

Figure 5-47 shows a two-node Oracle Service Bus cluster running on two Oracle WebLogic Servers. The Oracle WebLogic Servers are frontended by Oracle HTTP Servers, which load balance incoming requests. A load balancer front ends the system and distributes incoming requests from clients to the two Oracle HTTP Servers. This configuration uses an Oracle RAC database to store reporting data from the JMS reporting provider, and two phase commit logs using LLR with the JTA transaction manager (only used by the reporting data provider). It may also contain heartbeat data from servers in the cluster (used in the WebLogic Server whole server migration feature). The persistent store for transaction and JMS stores is on shared storage. Virtual IP addresses (VIPs) provide manual failover for the Administration Server and for Oracle Service Bus managed servers (for server migration).

Note:

To keep the Oracle Service Bus high availability configuration figure above readable, the Oracle SOA Service Infrastructure high availability configuration is not shown. See Figure 5-6 to view the Oracle SOA Service Infrastructure high availability figure.

For information about configuring virtual IPs for the Administration Server and configuring the Administration Server for high availability, see Section 12.2.2.3, "Transforming the Administration Server for Cold Failover Cluster."

5.14.1 Preparing the Environment: Prerequisite Steps Before Setting up a SOA High Availability Configuration

The following sections provide prerequisite steps before setting up an Oracle SOA Service Infrastructure high availability configuration.

For information about platform-specific commands, see the Oracle Fusion Middleware Installation Guide for Oracle SOA Suite and Oracle Business Process Management Suite.

5.14.1.1 Database Prerequisites

Oracle SOA Suite requires the presence of a supported database and schemas. Oracle Service Bus depends upon the database only for the JMS Reporting Provider and the OWSM Policy Manager.

To check if your database is certified or to see all certified databases, refer to the "Certified Databases" section in the Certification Document:

http://www.oracle.com/technology/software/products/ias/files/fusion_certification.html

To determine the database version, execute this query:

SQL>select version from sys.product_component_version where product like 'Oracle%';

5.14.1.2 VIP and IP Prerequisites

As shown in Table 5-9, you configure the Administration Server and the SOA and OSB managed servers to listen on different virtual IPs. This requires the provisioning of the corresponding VIP in the node and related host names in the DNS system in your network. Make sure that the different VIPS are available and are reachable from SOAHOST1, SOAHOST2, and the client machines before running the installation.

Table 5-9 Virtual Hosts

Virtual IP VIP Maps to... Description

VIP0

SOAHOST1VHN0

SOAHOST1VHN0 is the virtual host name that is the listen address for the Administration Server and fails over with manual failover of the Administration Server. It is enabled on the node where the Administration Server process is running (SOAHOST1 by default).

VIP1

SOAHOST1VHN1

SOAHOST1VHN1 is the virtual host name that maps to the listen address for WLS_SOA1 and fails over with server migration of this managed server. It is enabled on the node where the WLS_SOA1 process is running (SOAHOST1 by default).

VIP2

SOAHOST2VHN1

SOAHOST2VHN1 is the virtual host name that maps to the listen address for WLS_SOA2 and fails over with server migration of this managed server. It is enabled on the node where the WLS_SOA2 process is running (SOAHOST2 by default).

VIP3

OSBHOST1VHN1

OSBHOST1VHN1 is the virtual host name that maps to the listen address for WLS_OSB1 and fails over with server migration of this managed server. It is enabled on the node where the WLS_OSB1 process is running (SOAHOST1 by default).

VIP4

OSBHOST2VHN1

OSBHOST2VHN1 is the virtual host name that maps to the listen address for WLS_OSB2 and fails over with server migration of this managed server. It is enabled on the node where the WLS_OSB2 process is running (SOAHOST2 by default).


5.14.1.3 Shared Storage Prerequisites

For proper recovery in case of failure, store both JMS and transaction logs in a location that is accessible to all the nodes that can resume the operations after a failure in a managed server. This requires a shared storage location that can be referenced by multiple nodes. Table 5-10 lists the contents of shared storage.

Table 5-10 Contents of Shared Storage

Server Type of Data Vol in Shared Storage Directory Files

WLS_SOA1

Tx Logs

VOL1

ORACLE_BASE/admin/domain_name/soa_cluster_name/tlogs

Common location (stores decided by WebLogic Server)

WLS_SOA2

Tx Logs

VOL1

ORACLE_BASE/admin/domain_name/soa_cluster_name/tlogs

Common location (stores decided by WebLogic Server)

WLS_OSB1

Tx Logs

VOL1

ORACLE_BASE/admin/domain_name/osb_cluster_name/tlogs

Common location (stores decided by WebLogic Server)

WLS_OSB2

Tx Logs

VOL1

ORACLE_BASE/admin/domain_name/osb_cluster_name/tlogs

Common location (stores decided by WebLogic Server)

WLS_SOA1

JMS Stores

VOL1

ORACLE_BASE/admin/domain_name/soa_cluster_name/jms

Common location but Individual store per server (for example: SOAJMSStore1, UMSJMSStore1)

WLS_SOA2

JMS Stores

VOL1

ORACLE_BASE/admin/domain_name/soa_cluster_name/jms

Common location but Individual store per server (for example: SOAJMSStore2, UMSJMSStore2)

WLS_OSB1

JMS Stores

VOL1

ORACLE_BASE/admin/domain_name/osb_cluster_name/jms

Common location but Individual store per server (for example: WseeFileStore_auto_1)

WLS_OSB2

JMS Stores

VOL1

ORACLE_BASE/admin/domain_name/osb_cluster_name/jms

Common location but Individual store per server (for example: WseeFileStore_auto_2)


The shared storage can be a NAS or SAN device. Specifically for NFS mounted systems, different issues related to file locking and abrupt node failures have been detected. Check the Oracle Fusion Middleware Release Notes and with your storage vendor for the main recommended parameters to be used as mount options. The following is an example command based on a NAS device. Your options may be different from the ones specified in this section:

SOAHOST1> mount nasfiler:/vol/volX/FMWshared
MW_HOME -t nfs -o
rw,bg,hard,nointr,tcp,vers=3,timeo=300,rsize=32768,wsize=32768

5.14.1.4 Installing and Configuring an LDAP Provider

For production environments, it is a mandatory requirement for Oracle SOA Suite high availability topologies to have an external LDAP policy store. For more information on the supported policy stores as well as instructions on configuring LDAP, see the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite.

5.14.1.5 Synchronizing System Clocks

Synchronizing system clocks on each of the cluster nodes for high availability SOA deployments is required.

5.14.1.6 Terminology for Directories and Directory Environment Variables

The following list describes the directories and variables used in this chapter:

  • ORACLE_BASE: This environment variable and related directory path refers to the base directory under which Oracle products are installed.

  • MW_HOME: This environment variable and related directory path refers to the location where Fusion Middleware (FMW) resides.

  • WL_HOME: This environment variable and related directory path contains installed files necessary to host a WebLogic Server.

  • ORACLE_HOME: This environment variable and related directory path refers to the location where Oracle FMW SOA Suite or Oracle Service Bus is installed.

  • ORACLE_COMMON_HOME: This environment variable and related directory path refers to the Oracle home that contains the binary and library files required for the Oracle Enterprise Manager Fusion Middleware Control and Java Required Files (JRF).

    There can be only one Oracle Common home within each Middleware home.

  • DOMAIN Directory: This directory path refers to the location where the Oracle WebLogic Domain information (configuration artifacts) is stored.

  • ORACLE_INSTANCE: An Oracle instance contains one or more system components, such as Oracle Web Cache, Oracle HTTP Server, or Oracle Internet Directory. An Oracle instance directory contains updateable files, such as configuration files, log files, and temporary files.

The values used and recommended for consistency for this directories are:

ORACLE_BASE:

/u01/app/oracle

MW HOME (Application Tier):

ORACLE_BASE/product/fmw

WL_HOME:

MW_HOME/wlserver_10.3

ORACLE_HOME:

MW_HOME/soa

ORACLE_COMMON_HOME:

MW_HOME/oracle_common

Location for JMS file based stores and Tlogs:

ORACLE_BASE/admin/domain_name/soa_cluster_name/jms
ORACLE_BASE/admin/domain_name/soa_cluster_name/tlogs

Mount point is:

ORACLE_BASE/admin/domain_name/soa_cluster_name/

Shared Storage location:

Shared storage location: ORACLE_BASE/admin/domain_name/aserver

5.14.1.7 Installing and Configuring the Database Repository

This section describes how to install and configure the database repository.

Oracle Clusterware

  • For 10g Release 2 (10.2), see Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide.

  • For 11g Release 1 (11.1) and later, see Oracle Clusterware Installation Guide.

Automatic Storage Management

  • For 10g Release 2 (10.2), see Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide.

  • For 11g Release 1 (11.1) and later, see Oracle Real Application Clusters Installation Guide.

  • When you run the installer, select the Configure Automatic Storage Management option in the Select Configuration page to create a separate Automatic Storage Management home.

Oracle Real Application Clusters

You must install the 11g (11.1.1) Oracle Fusion Middleware Repository into a Real Application Clusters database before you install the Oracle Fusion Middleware components. Oracle Fusion Middleware provides a tool, Oracle Fusion Middleware Repository Creation Utility (RCU), to create the component schemas in an existing database. You install RCU in its own, separate Middleware home.

Use the latest version of RCU to install the 11g (11.1.1) Oracle Fusion Middleware Repository into a Real Application Clusters database.

See Oracle Fusion Middleware Repository Creation Utility User's Guide for more information about obtaining and running the latest version of RCU.

Database Initialization Parameters

Ensure that the following initialization parameter is set to the required value. It is checked by Repository Creation Assistant.

Table 5-11 Required Initialization Parameters

Parameter Required Value Parameter Class

PROCESSES

300 or greater

Static


To check the value of the initialization parameter using SQL*Plus, you can use the SHOW PARAMETER command.

As the SYS user, issue the SHOW PARAMETER command as follows:

SQL> SHOW PARAMETER processes

Set the initialization parameter using the following command:

SQL> ALTER SYSTEM SET processes=300 SCOPE=SPFILE

Restart the database.

Note:

The method that you use to change a parameter's value depends on whether the parameter is static or dynamic, and on whether your database uses a parameter file or a server parameter file. See the Oracle Database Administrator's Guide for details on parameter files, server parameter files, and how to change parameter values.

5.14.1.8 Using Oracle Fusion Middleware Repository Creation Utility to Load the Fusion Middleware Schemas in the Database

Install the 11g (11.1.1) Oracle Fusion Middleware Metadata Store and Oracle SOA schemas into a Real Application Clusters database before you install Oracle Fusion Middleware SOA components. Oracle Fusion Middleware provides a tool, Oracle Fusion Middleware Repository Creation Utility (RCU), to create the component schemas in an existing database. See Oracle Fusion Middleware Repository Creation Utility User's Guide for more information about installing RCU.

Note:

For domains including Oracle BPM Suite, it is required that Oracle Fusion Middleware 11.1.1.4 (Patch Set 3) RCU be used.
5.14.1.8.1 Running RCU

Run RCU to install the required metadata for Oracle Fusion Middleware 11g.

  1. For Linux, go to RCU_Home/bin and use the following command:

    ./rcu
    
  2. In the Welcome screen, click Next.

  3. In the Create Repository screen, select Create to load component schemas into a database, click Next.

  4. In the Database Connection Details screen, enter connection information for your database:

    • Database Type: Select Oracle Database.

    • Host Name: Enter the name of the node that is running the database. For an Oracle RAC database, specify the VIP name, or one of the node names as the host name: SOADBHOST1VIRTUAL.

    • Port: The port number for the database: 1521

    • Service Name: Enter the service name of the database: soaha.mycompany.com

    • Username: SYS

    • Password: Enter the password for the SYS user.

    • Role: SYSDBA

  5. Click Next.

  6. If you receive the following warning message:

    The database you are connecting is with non-UTF8 charset, if you are going to use this DB for multilingual support, you may have data loss. If you are not using for multilingual support you can continue, otherwise we strongly recommend using UTF-8 database.

  7. Click Ignore or Stop.

  8. In the Select Components screen, select Create a New Prefix, and enter a prefix to use for the database schemas, for example, SOAHA

    Write down the schema names so they are available in later procedures.

    • Select the following schemas:

      • Under AS Common Schemas, select Metadata Services.

      • Under SOA Service Infrastructure, select SOA Service Infrastructure and User Messaging (the OSB Reporting Provider schema is part of SOA Service Infrastructure).

  9. Click Next.

  10. In the Schema Passwords screen, enter passwords for the main and additional (auxiliary) schema users, and click Next.

  11. In the Map Tablespaces screen, choose the tablespaces for the selected components, and click Next.

  12. In the Summary screen, click Create.

  13. In the Completion Summary screen, click Close.

See the Oracle Fusion Middleware Repository Creation Utility User's Guide for more information about using RCU.

5.14.1.8.2 Configuring SOA Schemas for Transactional Recovery Privileges

You need the appropriate database privileges to allow the Oracle WebLogic Server transaction manager to query for transaction state information and issue the appropriate commands, such as commit and rollback, during recovery of in-flight transactions after a WebLogic Server container crash.

To configure the SOA schemas for transactional recovery privileges:

  1. Log on to SQL*Plus as a user with sysdba privileges. For example:

    sqlplus "/ as sysdba"
    
  2. Grant select on sys.dba_pending_transactions to soaha_soainfra.

  3. Grant force any transaction to soaha_soainfra.

    Note:

    These privileges should be granted to the owner of the soainfra schema, as determined by the RCU operations.

5.14.1.9 Configuring Virtual Server Names and Ports for the Load Balancer

This section describes the load balancer prerequisites for deploying an Oracle SOA Suite high availability environment.

Load Balancers

Oracle SOA Suite uses a hardware load balancer when deployed in a high availability configuration with two Oracle HTTP Servers as web tier components. The hardware load balancer should have the following features:

  • Ability to load-balance traffic to a pool of real servers through a virtual host name: Clients access services using the virtual host name (instead of using actual host names). The load balancer can then load balance requests to the servers in the pool.

  • Port translation configuration: The load balancer should have the ability to perform port translation, where it allows incoming requests received on one port to be routed to a server process running on a different port. For example, a request received on port 80 can be routed to port 7777.

  • Monitoring of ports (HTTP, HTTPS)

  • Virtual servers and port configuration: Ability to configure virtual server names and ports on your external load balancer, and the virtual server names and ports must meet the following requirements:

    • The load balancer should allow configuration of multiple virtual servers. For each virtual server, the load balancer should allow configuration of traffic management on more than one port. For example, the load balancer typically needs to be configured with a virtual server and ports for HTTP and HTTPS traffic.

    • The virtual server names must be associated with IP addresses and be part of your DNS. Clients must be able to access the load balancer through the virtual server names.

  • Ability to detect node failures and immediately stop routing traffic to the failed node.

  • Resource monitoring/port monitoring/process failure detection: The load balancer must be able to detect service and node failures (through notification or some other means) and to stop directing non-Oracle Net traffic to the failed node. If your load balancer has the ability to automatically detect failures, you should use it.

  • Fault-tolerant mode: It is highly recommended that you configure the load balancer to be in fault-tolerant mode.

  • Other: It is highly recommended that you configure the load balancer virtual server to return immediately to the calling client when the back-end services to which it forwards traffic are unavailable. This is preferred over the client disconnecting on its own after a timeout based on the TCP/IP settings on the client machine.

  • Sticky routing capability: Ability to maintain sticky connections to components based on cookies or URL.

  • SSL acceleration: SSL acceleration is a method of offloading the processor-intensive public key encryption algorithms involved in SSL transactions to a hardware accelerator.

    This feature is recommended but not required.

Table 5-12 shows some example virtual server names to use for the external load balancer in the Oracle SOA Suite high availability environment.

Table 5-12 Virtual Server Names for the External Load Balancer

Component Virtual Server Name

Oracle SOA Suite

soa.mycompany.com

WebLogic Server Administration Console

admin.mycompany.com

Oracle Enterprise Manager Fusion Middleware Control

admin.mycompany.com

Oracle Service Bus Web Console

admin.mycompany.com


Virtual Server Names

This section describes the virtual server names that should be set up for the high availability deployments described in this chapter.

soa.mycompany.com

soa.mycompany.com is a virtual server name that acts as the access point for all HTTP traffic to the runtime SOA components, such as soa-infra and workflow. Traffic to the both SSL and non-SSL ports is configured and typically non-SSL is redirected to SSL. Clients access this service using the address soa.mycompany.com:443. This virtual server is defined on the load balancer.

admin.mycompany.com

This virtual server acts as the access point for all internal HTTP traffic that gets directed to the administration services. The incoming traffic from clients is non-SSL enabled. Thus, the clients access this service using the address admin.mycompany.com:80 and in turn forward these to ports 7777 on WEBHOST1 and WEBHOST2. The services accessed on this virtual host include the WebLogic Administration Server Console, Oracle Enterprise Manager, and the Oracle Service Bus Web Console.

In addition, ensure that the virtual server names are associated with IP addresses and are part of your Domain Name System (DNS). The computers on which Oracle Fusion Middleware is running must be able to resolve these virtual server names.

5.14.1.10 Installing Oracle HTTP Server on WEBHOST1 and WEBHOST2

To install Oracle HTTP Server on WEBHOST1:

  1. Verify that the servers meet the following requirements:

    • The system, patch, kernel, and other requirements meet the requirements specified in the Oracle Fusion Middleware Installation Guide for Oracle SOA Suite.

    • Port 7777 is not used by any service on the nodes. You can verify this by running the following command:

      Unix:

      netstat -an | grep LISTEN | grep "7777"
      

      Windows:

      netstat -an | findstr "LISTEN" | findstr ":7777"
      

    If the ports are in use, make them available.

  2. On UNIX platforms, if the /etc/oraInst.loc or /var/opt/oracle/oraInst.loc file exists, check that its contents are correct. Specifically, check that the inventory directory is correct, and that you have write permissions for that directory.

    If the /etc/oraInst.loc file does not exist, skip this step.

  3. Start Oracle Universal Installer for Oracle Fusion Middleware 11g Web Tier Utilities CD installation as follows:

    For UNIX, run this command: runInstaller.

    For Windows, double-click setup.exe.

  4. In the Welcome screen, click Next.

  5. In the Select Installation Type screen, select Install and Configure, and click Next.

  6. In the Prerequisite Checks screen, ensure that all the prerequisites are met, and click Next.

  7. In the Specify Installation Location screen, set the location to:

    /u01/app/oracle/product/11.1.1/ohs_1

  8. Click Next.

  9. In the Configure Components screen:

    • Select Oracle HTTP Server.

    • Do not select Associate Selected Components with WebLogic Domain.

    • Click Next.

  10. In the Specify Component Details screen, enter the following values:

    • Instance Home Location: Instance Home

    • Instance Home Location: /u01/app/oracle/product/11.1.1/ohs_1/instances/ohs_instance1

    • Instance Name: ohs_instance1

    • OHS Component Name: ohs1

  11. Click Next.

  12. In the Specify Web Tier Port Details screen:

    • Select Specify Custom Ports. If you specify a custom port, select Specify Ports using Configuration File, and use the Browse function to select the file.

    • Enter the Oracle HTTP Server port, for example, 7777.

  13. Click Next.

  14. In the Configuration Summary screen, ensure that the selections are correct, and click Install.

  15. In the Installation Progress screen:

    For UNIX systems, a dialog box appears prompting you to run the oracleRoot.sh script. Open a command window and run the script, following the prompts.

    Click Next.

  16. In the Configuration screen, several configuration assistants are launched in succession. When the configuration assistants are finished, the Configuration Completed screen appears.

  17. In the Configuration Completed screen, click Finish to exit.

Repeat the steps for WEBHOST2 and configure your LBR with a pool containing both the WEBHOST1 and WEBHOST2 addresses.

5.14.1.10.1 Validating Oracle HTTP Server

To verify that Oracle HTTP Server is set up correctly, access the root URL context of the server by using the following URL in a browser:

HTTP://WEBHOST1:7777/

If Oracle HTTP Server is set up correctly, the Oracle FMW Welcome screen appears in the browser.

5.14.2 Installing Oracle Fusion Middleware Home

This section describes the procedure for installing Oracle Fusion Middleware on all nodes in the application tier that run Oracle WebLogic Server and Oracle Fusion Middleware SOA Suite. Repeat the procedures (described below for SOAHOST1) for installing WebLogic Server and Oracle SOA in SOAHOST2. The directory paths for binary files and domains used when installing new nodes must be exactly the same as those used for first node. If these paths and domains are not exactly the same as those used for the first node, failover will not work properly.

Install the following Oracle Fusion Middleware components:

5.14.2.1 Installing Oracle WebLogic Server

See "Understanding Your Installation Starting Point" in Oracle Fusion Middleware Installation Planning Guide for the version of Oracle WebLogic Server to use with the latest version of Oracle Fusion Middleware.

To install Oracle WebLogic Server on all nodes in the application tier:

  1. Start Oracle WebLogic Server Installer.

  2. In the Welcome screen, click Next.

  3. In the Choose Middleware Home Directory screen:

    • Select Create a New Middleware Home.

    • For the Middleware Home Directory field, enter MW_HOME.

    • Click Next.

  4. In the Register for Security Updates screen, enter your contact information for security update notifications, and click Next.

  5. In the Choose Install Type screen, select Custom, and click Next.

  6. In the Choose Products and Components screen, click Next.

  7. In the JDK Selection screen, select only Oracle JRockit 160_20_D1.0.1-2124 SDK, and click Next.

  8. In the Choose Product Installation Directories screen, accept the following directory:

    WL_HOME

    Click Next.

  9. In the Installation Summary screen, click Next.

  10. In the Installation Complete screen, deselect Run QuickStart, and click Done.

5.14.2.2 Installing Oracle Fusion Middleware for Oracle SOA

On Linux platforms, if the /etc/oraInst.loc file exists, check that its contents are correct. Specifically, check that the inventory directory is correct and that you have write permissions for that directory.

If the /etc/oraInst.loc file does not exist, you can skip this step.

  1. Start Oracle Fusion Middleware 11g Oracle SOA Suite Installer:

    On UNIX:

    SOAHOST1> runInstaller
    

    On Windows:

    SOAHOST1> setup.exe
    

    When Oracle Fusion Middleware 11g Oracle SOA Suite Installer prompts you for a JRE/JDK location enter the Oracle SDK location created in the Oracle WebLogic Server installation, for example, MW_HOME/jrockit_160_20_D1.0.1-2124.

  2. In the Specify Inventory Directory screen, do the following:

    1. Enter HOME/oraInventory, where HOME is the home directory of the user performing the installation (this is the recommended location).

    2. Enter the OS group for the user performing the installation.

    3. Click Next.

    4. Follow the instructions on the screen to execute /createCentralInventory.sh as root.

    5. Click OK.

  3. In the Welcome screen, click Next.

  4. In the Prerequisite Checks screen, verify that the checks complete successfully, and click Next.

  5. In the Specify Installation Location screen:

    • For Middleware Home, enter MW_HOME.

    • For Oracle Home Directory, enter soa.

    Click Next.

  6. In the Installation Summary screen, click Install.

  7. In the Installation Complete screen, click Finish.

5.14.2.3 Installing Oracle Service Bus

To install Oracle Service Bus, follow these steps:

  1. Start the Oracle Fusion Middleware 11g Oracle Service Bus Installer:

    On UNIX:

    SOAHOST1> runinstaller
    

    On Windows:

    SOAHOST1> setup.exe
    

    When Oracle Fusion Middleware 11g Oracle Service Bus Installer prompts you for a JRE/JDK location enter the Oracle SDK location created in the Oracle WebLogic Server installation, for example, MW_HOME/jrockit_160_20_D1.0.1-2124.

  2. In the Welcome screen, click Next.

  3. In the Install Software Updates screen, enter your preferences for updates.

  4. In the Specify Installation Location screen:

    • For Middleware Home, enter MW_HOME.

    • For Oracle Home Directory, enter osb.

    Click Next.

  5. In the Installation Type screen, select Custom and click Next.

  6. In the Components to Install screen, deselect Oracle Service Bus IDE and Oracle Service Bus Examples.

  7. In the Prerequisite Checks screen, verify that the checks complete successfully, and click Next.

  8. In the Product Home Location screen, accept the default and click Next.

  9. In the Installation Summary screen, click Install.

  10. In the Installation Complete screen, click Finish.

Note:

Before you run the Configuration Wizard by following the instructions in Section 5.14.4, "Running Oracle Fusion Middleware Configuration Wizard on SOAHOST1 to Create the SOA and OSB Domain," make sure that you have applied the latest Oracle Fusion Middleware patch set and other known patches to your Middleware Home, so that you have the latest version of Oracle Fusion Middleware.

See "Understanding Your Installation Starting Point" in Oracle Fusion Middleware Installation Planning Guide for the steps you must perform to get the latest version of Oracle Fusion Middleware.

5.14.3 Enabling VIP1 and VIP3 in SOAHOST1 and VIP2 and VIP4 in SOAHOST2

The SOA domain uses virtual hostnames as the listen addresses for the SOA and OSB managed servers. You must enable VIP mapping for each of these hostnames on the two SOA Machines, (VIP1 and VIP3 on SOAHOST1 and VIP2 and VIP4 on SOAHOST2), and must correctly resolve the virtual hostnames in the network system used by the topology (either by DNS Server or by hosts resolution).

For information about configuring virtual IPs, see Section 12.2.2.3, "Transforming the Administration Server for Cold Failover Cluster," where an example is provided for the Administration Server. Refer to Section 5.14.20, "Configuring Server Migration for the WLS_SOA Servers" for more details on configuring server migration for the SOA servers.

5.14.4 Running Oracle Fusion Middleware Configuration Wizard on SOAHOST1 to Create the SOA and OSB Domain

Run Oracle Fusion Middleware Configuration Wizard from the MW_HOME/Oracle_common/common/bin directory to create a domain containing the Administration Server and Oracle SOA and OSB components. Ensure that the database where you installed the repository is running. For Oracle RAC databases, all the instances must be running.

Note:

Multiple SOA clusters are not allowed in the same WebLogic Server domain.

Multiple OSB clusters are not allowed in the same WebLogic Server domain.

One single cluster holding both SOA and OSB is not allowed in a single WebLogic Server domain.

  1. Change the directory to the location of the Oracle Fusion Middleware Configuration Wizard, located in the Middleware home, oracle_common/common/bin directory:

    SOAHOST1> cd ORACLE_HOME/oracle_common/common/bin
    
  2. Start Oracle Fusion Middleware Configuration Wizard:

    For Linux:

    SOAHOST1> ./config.sh
    

    For Windows:

    SOAHOST1> config.cmd
    
  3. In the Welcome screen, select Create a New WebLogic Domain, and click Next.

  4. In the Select Domain Source screen:

    • Select Generate a domain configured automatically to support the following products.

    • Select the following products:

      • Basic Weblogic Server Domain - 10.3.4.0 [wlserver_10.3] (Selected by default and grayed out)

      • Oracle SOA Suite - 11.1.1.0 [soa] (Selected by default)

      • Oracle Service Bus OWSM Extension - 11.1.1.4 [osb]

      • Oracle Enterprise Manager - 11.1.1.0 [oracle_common] (Selected by default)

      • Oracle Service Bus - 11.1.1.4 [osb]

      • WebLogic Advanced WebService for JAX-RPC Extension - 10.3.4 [wlserver_10.3]

      • Oracle WSM Policy Manager - 11.1.1.0 [oracle_common] (Selected by default when selecting SOA Suite)

      • Oracle JRF WebServices - 11.1.1.0 [oracle_common] (Selected by default when selecting SOA Suite)

    If you accidentally deselect some of the targets, make sure that the selections above are made in this screen.

    Click Next.

  5. In the Specify Domain Name and Location screen, make the following entries:

    • Domain Name: soadomain

    • Domain Location: accept the default

    Click Next.

  6. In the Configure Administrator Username and Password screen, enter the username and password to be used for the domain's administrator, and click Next.

  7. In the Configure Server Start Mode and JDK screen, make the following selections:

    • WebLogic Domain Startup Mode: select Production Mode

    • JDK Selection: select Oracle JRockit 160_20_D1.0.1-2124 SDK.

    Click Next.

  8. In the Configure JDBC Component Schema screen:

    1. Select all the component schemas that appear in the table at the bottom: SOA Infrastructure, User Messaging Service, OWSM MDS Schema, SOA MDS Schema, and OSB JMS Reporting Provider.

    2. Select Configure selected component schemas as RAC multi data source schemas in the next panel.

    3. Ensure that the following data source appears on the screen. The user names shown in Table 5-13 assume that soaha was used as the prefix for schema creation from RCU.

      For information about multi data source configuration with Oracle RAC and the MDS repository, see Section 4.1.2, "Using Multi Data Sources with Oracle RAC."

      Table 5-13 Configuring Values for Data Sources

      Multi Data Source Schema Schema Owner

      SOA Infrastructure

      SOAHA_SOAINFRA

      User Messaging Service

      SOAHA_ORASDPM

      OWSM MDS Schema

      SOAHA_MDS

      SOA MDS Schema

      SOAHA_MDS

      OSB JMS Reporting Provider

      SOAHA_SOAINFRA


    4. Click Next.

  9. In the Configure RAC Multi Data Source Component Schema screen, enter values for the following fields, specifying the connect information for the Oracle RAC database that was seeded with RCU:

    Figure 5-48 Configure RAC Multi Data Source Component Schema Screen

    Configure RAC Multi Data Source Component Schema
    Description of "Figure 5-48 Configure RAC Multi Data Source Component Schema Screen"

    1. Driver: Select Oracle driver (Thin) for RAC Service-Instance connections, Versions:10, 11.

    2. Service Name: Enter the service name of the database, for example, soaha.mycompany.com.

    3. Username prefix: Enter the prefix for the schemas. The user names shown in Table 5-13 assume that soaha was used as prefix for schema creation from RCU.

    4. Password and Confirm Password: Enter the password for access to the schemas.

    5. Click Add, and enter the details for the first Oracle RAC instance.

    6. Update each multi data source schema by selecting one data source at a time, adding the appropriate details.

      Make sure that the information is entered for all the multi data source schemas: SOA Infrastructure, User Messaging Service, OWSM MDS Schema, SOA MDS Schema., and OSB JMS Reporting Provider.

    7. Click Next.

  10. In the Test JDBC Data Sources screen, the connections are tested automatically. The Status column displays the results. Ensure that all connections were successful. If not, click Previous to return to the previous screen and correct your entries.

    Click Next when all the connections are successful.

  11. In the Select Optional Configuration screen, select the following:

    • Administration Server

    • JMS Distributed Destinations

    • Managed Servers, Clusters and Machines

    • JMS File Store

  12. In the Customize Server and Cluster Configuration screen, select Yes, and click Next.

  13. In the Configure the Administration Server screen, enter the following values:

    • Name: AdminServer

    • Listen Address: Enter the hostname for the VIP0 virtual IP

    • Listen Port: 7001

    • SSL listen port: N/A

    • SSL enabled: leave unchecked

    Click Next.

  14. In the Select JMS Distributed Destination Type screen, select UDD for UMSJMSSystemResource, SOAJMSModule, jmsResources, and WseejmsModule.

  15. In the Configure Managed Servers screen, add the following managed servers:

    Table 5-14 Configuring Managed Servers

    Name Listen Address Listen Port SSL Listen Port SSL Enabled

    WLS_SOA1

    SOAHOST1VHN1

    8001

    n/a

    No

    WLS_SOA2

    SOAHOST2VHN1

    8001

    n/a

    No

    WLS_OSB1

    OSBHOST1VHN1

    8011

    n/a

    No

    WLS_OSB2

    OSBHOST2VHN1

    8011

    n/a

    No


    Do not delete any server that appears. You can modify the servers. If you delete a server and add a new one, targeting fails.

    Note:

    Although the standard recommendation is to run custom applications and other systems in a separate WebLogic Managed Server, the creation of the custom WLS managed servers described in Figure 5-47 is not addressed here.

    Click Next.

  16. In the Configure Clusters screen, add the following clusters:

    Table 5-15 Adding the SOA and OSB Clusters

    Name Cluster Messaging Mode Multicast Address Multicast Port Cluster Address

    SOA_Cluster

    unicast

    N/A

    N/A

    SOAHOST1VHN1:8001, SOAHOST2VHN1:8001

    OSB_Cluster

    unicast

    N/A

    N/A

    OSBHOST1VHN1:8011, OSBHOST2VHN1:8011


    Click Next.

  17. In the Assign Servers to Clusters screen, assign the following servers to SOA_Cluster:

    • WLS_SOA1

    • WLS_SOA2

    Assign the following servers to the OSB_Cluster:

    • WLS_OSB1

    • WLS_OSB2

    Click Next.

  18. In the Configure Machines screen:

    • Delete the LocalMachine that appears by default.

    • Click the Unix Machine tab, and add the following machines:

    Table 5-16 Configuring Machines

    Name Node Manager Listen Address

    SOAHOST1

    Hostname of SOAHOST1

    SOAHOST2

    Hostname of SOAHOST2

    OSBHOST1

    Hostname of OSBHOST1

    OSBHOST2

    Hostname of OSBHOST2


    Leave all other fields as their default values, and click Next.

  19. In the Assign Servers to Machines screen, assign servers to machines as follows:

    Figure 5-49 Assign Servers to Machines Screen

    Assign Servers to Machine screen
    Description of "Figure 5-49 Assign Servers to Machines Screen"

    • SOAHOST1: AdminServer, WLS_SOA1

    • SOAHOST2: WLS_SOA2

    • OSBHOST1: WLS_OSB1

    • OSBHOST2: WLS_OSB2

    Click Next.

  20. In the Configure JMS Files Stores screen, enter a directory location for each store that is accessible from all nodes that the servers can be failed over to. As recommended in Section 5.14.1.3, "Shared Storage Prerequisites," use a directory like the following on a shared device:

    Table 5-17 Configuring JMS File Stores

    Name Directory Synchronous Write Policy*

    FileStore_auto_1

    /u01/app/oracle/admin/soadomain/soacluster/FileStore_auto_1

    Direct-Write

    FileStore_auto_2

    /u01/app/oracle/admin/soadomain/soacluster/FileStore_auto_2

    Direct-Write

    SOAJMSFileStore_auto_1

    /u01/app/oracle/admin/soadomain/soacluster/SOAJMSFileStore_auto_1

    Direct-Write

    SOAJMSFileStore_auto_2

    /u01/app/oracle/admin/soadomain/soacluster/SOAJMSFileStore_auto_2

    Direct-Write

    UMSJMSFileStore_auto_1

    /u01/app/oracle/admin/soadomain/soacluster/UMSJMSFileStore_auto_1

    Direct-Write

    UMSJMSFileStore_auto_2

    /u01/app/oracle/admin/soadomain/soacluster/UMSJMSFileStore_auto_2

    Direct-Write

    WseeFileStore_auto_1

    /u01/app/oracle/admin/soadomain/soacluster/WseeFileStore_auto_1

    Direct-Write

    WseeFileStore_auto_2

    /u01/app/oracle/admin/soadomain/soacluster/WseeFileStore_auto_2

    Direct-Write


    * The default synchronous write policy (when none is specified) is Direct-Write.

    Click Next.

  21. In the Configuration Summary screen, click Create.

  22. In the Creating Domain screen, click Done.

5.14.5 Creating boot.properties for the Administration Server on SOAHOST1

This is an optional step for enabling the Administration Server to start up without prompting you for the administrator username and password. Create a boot.properties file for the Administration Server on SOAHOST1.

For the Administration Server:

  1. Create the following directories:

    SOAHOST1> mkdir -p ORACLE_BASE/product/fmw/user_
    projects/domains/soadomain/servers/AdminServer/security 
    
  2. Use a text editor to create a file called boot.properties in the security directory created in the previous step, and enter the following lines in the file:

    username=adminuser
    password=password
    

    Note:

    When you start up the Administration Server, the username and password entries in the file are encrypted.

    For security reasons, minimize the time the entries in the file are left unencrypted. After you edit the file, start up the server as soon as possible in order for the entries to be encrypted.

5.14.6 Starting and Validating the Administration Server in SOAHOST1

This section describes procedures for starting and validating the Administrator Server in SOAHOST1

5.14.6.1 Starting the Administration Server on SOAHOST1

To start the Administration Server on SOAHOST1, run the following commands:

SOAHOST1> cd MW_HOME/user_projects/domains/soadomain/bin

SOAHOST1> ./startWebLogic.sh

5.14.6.2 Validating the Administration Server

To verify that the Administration Server is properly configured:

  1. In a browser, go to http://vip0:7001/console.

  2. Log in as the administrator.

  3. Verify that the WLS_SOA1 and WLS_SOA2 managed servers are listed. Also verify that the WLS_OSB1 and WLS_OSB2 managed servers are listed.

  4. Verify that the SOA_Cluster cluster is listed. Also verify that the OSB_Cluster cluster is listed.

  5. Verify that you can access Enterprise Manager at http://vip0:7001/em.

  6. Verify that you can access the Oracle Service Bus Console at http://vip0:7001/sbconsole.

5.14.7 Disabling Host Name Verification for the Administration Server and the WLS_SOAn/WLS_OSBn Managed Servers

This step is required if you have not set up the appropriate certificates for hostname verification between the Administration Server and Node Manager. If SSL is not set up, you receive an error message unless you disable host name verification.

You can re-enable host name verification when you have set up SSL communication between the Administration Server and the Node Manager.

To disable host name verification:

  1. In Oracle WebLogic Server Administration Console, select Servers, and then AdminServer.

  2. Select SSL, and then Advanced.

  3. In the Change Center, click Lock & Edit.

  4. Set Hostname Verification to None.

  5. Save and activate the changes.

  6. Repeat these steps for the WLS_SOA1, WLS_SOA2, WLS_OSB1, and WLS_OSB2 servers.

  7. Restart the Administration Server.

5.14.8 Configuring Oracle Coherence for Deploying Composites

Although deploying composites uses multicast communication by default, Oracle recommends using unicast communication for SOA high availability. Use unicast if you disable multicast communication for security reasons.

Note:

An incorrect configuration of the Oracle Coherence framework that is used for deployment may prevent the SOA system from starting. The deployment framework must be properly customized for the network environment on which the SOA system runs. Oracle recommends the following configuration described in this section.

Enabling Communication within Clusters Using Unicast Communication

Multicast communication enables Oracle Fusion Middleware SOA to discover all of the members of a cluster to which to it deploys composites dynamically. However, unicast communication does not enable nodes to discover other cluster members in this way. Consequently, you must specify the nodes that belong to the cluster. You do not need to specify all of the nodes of a cluster, however. You need only specify enough nodes so that a new node added to the cluster can discover one of the existing nodes. As a result, when a new node has joined the cluster, it is able to discover all of the other nodes in the cluster. Additionally, in a configuration where multiple IPs are available in the same node, you must configure Oracle Coherence to use a specific hostname to create Oracle Coherence cluster.

Tip:

To guarantee high availability during deployments of SOA composites, specify enough nodes so that at least one of them is running at any given time.

Specify the nodes using the tangosol.coherence.wka<n> system property, where <n> is a number between 1 and 9. You can specify up to 9 nodes. Start the numbering at 1. This numbering must be sequential and must not contain gaps. In addition, specify the hostname used by Oracle Coherence to create a cluster through the tangosol.coherence.localhost system property. This hostname should be the virtual hostname used by the SOA servers that maps the corresponding listener addresses (VIP1 and VIP2). Set this property by adding the -Dtangosol.coherence.localhost parameters to the Arguments field of Oracle WebLogic Server Administration Console's Server Start tab (Figure 5-50).

Figure 5-50 Setting the Hostname Using the Start Server Tab of Oracle WebLogic Server Administration Console

Setting the Hostname
Description of "Figure 5-50 Setting the Hostname Using the Start Server Tab of Oracle WebLogic Server Administration Console"

Specifying the hostname

To add the hostname used by Oracle Coherence:

  1. Log into Oracle WebLogic Server Administration Console.

  2. In the Domain Structure window, expand the Environment node.

  3. Click Servers. The Summary of Servers page appears.

  4. Click the name of the server (represented as a hyperlink) in Name column of the table. The settings page for the selected server appears.

  5. Click the Server Start tab (illustrated in Figure 5-50).

  6. Enter the following for WLS_SOA1 and WLS_SOA2 into the Arguments field.

    For WLS_SOA1, enter the following (on a single line, without a carriage return):

    -Dtangosol.coherence.wka1=soahost1vhn1 -Dtangosol.coherence.wka2=soahost2vhn1 -Dtangosol.coherence.localhost=soahost1vhn1
    

    For WLS_SOA2, enter the following (on a single line, without a carriage return):

    -Dtangosol.coherence.wka1=soahost1vhn1 -Dtangosol.coherence.wka2=soahost2vhn1 -Dtangosol.coherence.localhost=soahost2vhn1
    

    Note:

    The Coherence cluster used for deployment uses port 8088 by default. This port can be changed by specifying a different port (for example, 8089) with the -Dtangosol.coherence.wkan.port and -Dtangosol.coherence.localport startup parameters. For example:

    WLS_SOA1 (enter the following into the Arguments field on a single line, without a carriage return):

    -Dtangosol.coherence.wka1=soahost1vhn1
    -Dtangosol.coherence.wka2=soahost2vhn1
    -Dtangosol.coherence.localhost=soahost1vhn1
    -Dtangosol.coherence.localport=8089
    -Dtangosol.coherence.wka1.port=8089
    -Dtangosol.coherence.wka2.port=8089
    

    WLS_SOA2 (enter the following into the Arguments field on a single line, without a carriage return):

    -Dtangosol.coherence.wka1=soahost1vhn1
    -Dtangosol.coherence.wka2=soahost2vhn1
    -Dtangosol.coherence.localhost=soahost2vhn1
    -Dtangosol.coherence.localport=8089
    -Dtangosol.coherence.wka1.port=8089
    -Dtangosol.coherence.wka2.port=8089
    
  7. Click Save and activate the changes.

  8. This change requires the SOA servers to be restarted.

Note:

The multicast and unicast addresses are different from the ones used by the WebLogic Server cluster for cluster communication. SOA guarantees that composites are deployed to members of a single WebLogic Server cluster even though the communication protocol for the two entities (the WebLogic Server cluster and the groups to which composites are deployed) are different.

Do not copy the text from above to your Administration Console's arguments text field, which can result in HTML tags being inserted in the Java arguments. The text should not include any text or characters other than the ones shown above.

5.14.9 Configure Oracle Coherence for the Oracle Service Bus Result Cache

The default configuration for the Coherence infrastructure that is used by the Oracle Service Bus result cache uses multicast. Oracle recommends using unicast communication for the Oracle Service Bus result cache. To enable unicast for the Oracle Service Bus result cache Coherence infrastructure, follow these steps:

  1. Log into Oracle WebLogic Server Administration Console. In the Change Center, click Lock & Edit.

  2. In the Domain Structure window, expand the Environment node.

  3. Click Servers. The Summary of Servers page appears.

  4. Click the name of the server (represented as a hyperlink) in the Name column of the table. The settings page for the selected server appears.

  5. Click the Server Start tab.

  6. Enter the following for WLS_OSB1 (on a single line, without a carriage returns):

    -DOSB.coherence.localhost=osbhost1vhn1 -DOSB.coherence.localport=7890 
    -DOSB.coherence.wka1= osbhost1vhn1 -DOSB.coherence.wka1.port=7890 
    -DOSB.coherence.wka2= osbhost2vhn1 -DOSB.coherence.wka1.port=7890
    

    For WLS_OSB2, enter the following (on a single line, without a carriage return):

    -DOSB.coherence.localhost=osbhost2vhn1 -DOSB.coherence.localport=7890 
    -DOSB.coherence.wka1= osbhost1vhn1 -DOSB.coherence.wka1.port=7890 
    -DOSB.coherence.wka2= osbhost2vhn1 -DOSB.coherence.wka1.port=7890
    
  7. Save and activate the changes (this change requires the Oracle Service Bus servers to be restarted to be effective).

5.14.10 Setting Connection Destination Identifiers for B2B Queues

Oracle B2B uses specific JMS Destination Member calls, and requires setting the Create Destination Identifier (CDI) for these calls to succeed. To set up the CDI:

  1. Log into the Oracle WebLogic Server Administration Console.

  2. In the Domain Structure window, expand the Services node, and then the Messaging node.

  3. Click JMS Modules, and then SOAJMSModule.

  4. In the Change Center, click Lock & Edit.

  5. Click the dist_B2BEventQueue_auto, Configuration, and the General tab, and then click Advanced.

  6. In the Create Destination Identifier field, add the following jndi name for the queue:

    jms/b2b/B2BEventQueue
    
  7. Repeat these steps, creating the following Create Destination Identifiers for the queues listed below:

    • B2B_OUT_QUEUE: jms/b2b/B2B_OUT_QUEUE

    • B2B_IN_QUEUE: jms/b2b/B2B_IN_QUEUE

    • B2BBroadcastTopic: jms/b2b/B2BBroadcastTopic

    • XmlSchemaChangeNotificationTopic: jms/fabric/XmlSchemaChangeNotificationTopic

  8. Click Save and Activate Changes.

5.14.11 Starting the System in SOAHOST1

This section describes how to start Node Manager on SOAHOST1 and how to start and validate the WLS_SOA1 managed server on SOAHOST1.

5.14.11.1 Starting Node Manager on SOAHOST1

Perform these steps to start Node Manager on SOAHOST1:

  1. Run the setNMProps.sh script, which is located in the ORACLE_COMMON_HOME/common/bin directory, to set the StartScriptEnabled property to true before starting Node Manager

    SOAHOST1> cd ORACLE_COMMON_HOME/common/bin
    SOAHOST1> ./setNMProps.sh
    

    Note:

    You must use the StartScriptEnabled property to avoid class loading failures and other problems.
  2. Start Node Manager:

    SOAHOST1> cd WL_HOME/server/bin
    SOAHOST1> ./startNodeManager.sh
    

5.14.11.2 Starting and Validating the WLS_SOA1 Managed Server

To start up the WLS_SOA1 managed server and check that it is configured correctly:

  1. Start the WLS_SOA1 managed server using Oracle WebLogic Server Administration Console.

  2. When WLS_SOA1 is started, the following URLs become available:

    • http://SOAHOST1VHN1:8001/b2b

      • Verify login to B2B console

    • http://SOAHOST1VHN1:8001/integration/worklistapp

      • Verify login to worklist console

    • http://SOAHOST1VHN1:8001/wsm-pm

      • Verify the policy validator link.

    • http://SOAHOST1VHN1:8001/soa/composer

    • http://SOAHOST1VHN1:8001/soa-infra

5.14.12 Propagating the Domain Configuration to SOAHOST2, OSBHOST1, and OSBHOST2 with pack/unpack Utilities

Follow these steps to propagate the domain configuration to SOAHOST2 using pack/unpack utilities:

  1. Run the following pack command on SOAHOST1 to create a template pack:

    SOAHOST1> cd ORACLE_COMMON_HOME/common/bin
    SOAHOST1> ./pack.sh -managed=true
    -domain=ORACLE_BASE/product/fmw/user_projects/domains/soadomain/
    -template=soadomaintemplate.jar 
    -template_name=soa_domain_template
    
  2. Run the following command on SOAHOST1 to copy the template file created in the previous step to SOAHOST2:

    SOAHOST1> scp soadomaintemplate.jar
     oracle@node2:ORACLE_COMMON_HOME/common/bin
    
  3. Run the unpack command on SOAHOST2 to unpack the propagated template:

    SOAHOST2> cd ORACLE_COMMON_HOME/common/bin
    SOAHOST2> ./unpack.sh
     -domain=ORACLE_BASE/product/fmw/user_projects/domains/soadomain/
     -template=soadomaintemplate.jar
    

    Note:

    When you run the Fusion Middleware Configuration Wizard after a pack/unpack procedure, products that already exist in the original domain and appear in the Select Extension Source screen, are not selected or greyed out.

    Templates created with the pack command are considered user-created templates. They are treated differently from default templates. User-created templates are self-contained and do not contain any information related to the creation of the original domain.

  4. Repeat the steps above for OSBHOST1 and OSBHOST2 (copy the template and unpack it in each node).

5.14.13 Extracting XEngine Files in the Second Node

To enable B2B's XEngine in the second node, it is required to extract the content of the ZEngine tar manually:

SOAHOST2>cd ORACLE_HOME/soa/thirdparty/edifecs
SOAHOST2>tar xzvf XEngine.tar.gz

5.14.14 Starting the System in SOAHOST2, OSBHOST1, and OSBHOST2

This section describes procedures for starting the system in SOAHOST2.

5.14.14.1 Starting Node Manager on SOAHOST2, OSBHOST1, and OSBHOST2

To start the Node Manager on SOAHOST2, OSBHOST1, and OSBHOST2, repeat the steps from Section 5.14.11.1, "Starting Node Manager on SOAHOST1" on those nodes.

5.14.14.2 Starting and Validating the WLS_SOA2, WLS_OSB1, and WLS_OSB2 Managed Server

To start up the WLS_SOA2 managed server and verify that it is configured correctly:

  1. Start the WLS_SOA2 managed server using Oracle WebLogic Server Administration Console. Watch for errors in the server's output file while it starts.

  2. When WLS_SOA2 is started, the following URLs become available:

    • http://SOAHOST2VHN1:8001/b2b

      Verify login to B2B console

    • http://SOAHOST2VHN1:8001/integration/worklistapp

      Verify login to worklist console

    • http://SOAHOST2VHN1:8001/wsm-pm

      Verify the policy validator link

    • http://SOAHOST2VHN1:8001/soa/composer

    • http://SOAHOST2VHN1:8001/soa-infra

To start up the WLS_OSB1 managed server and verify that it is configured correctly follow these steps:

  1. Start the WLS_OSB1managed server using Oracle WebLogic Server Administration Console. Watch for errors in the server´s output file while it starts.

  2. When WLS_OSB1 is started, the following URL becomes available:

    http://OSBHOST1VHN1:8011/sbinspection.wsil/
    

    An HTTP reply like the following should be seen:

    <?xml version="1.0" encoding="UTF-8" ?> 
    <ins:inspection xmlns:ins="http://schemas.xmlsoap.org/ws/2001/10/inspection/">
    <ins:link referencedNamespace="http://schemas.xmlsoap.org/ws/2001/10/inspection/" location="http://OSBHOSTVHN1:8011/sbinspection.wsil?refpath=default">
    <ins:abstract>default</ins:abstract> 
    <ins:abstract>LinkType: Project</ins:abstract> 
    </ins:link>
    </ins:inspection>
    

Repeat these steps for WLS_OSB2.

5.14.15 Configuring Oracle HTTP Servers for the Administration Server, WLS_SOAn, and WLS_OSBn Managed Servers

To enable Oracle HTTP Server to route to the SOA Cluster, which contains the WLS_SOAn managed servers, you must set the WebLogicCluster parameter to the list of nodes in the cluster.

  1. On WEBHOST1 and WEBHOST2, add the following lines to the ORACLE_BASE/admin/<instance_name>/config/OHS/<component_name>/mod_wl_ohs.conf file:

    # WSM-PM
    <Location /wsm-pm>
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001, SOAHOST2VHN1:8001
    </Location>
    
    # SOA soa-infra app
    <Location /soa-infra>
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
    
    # SOA inspection.wsil
    <Location /inspection.wsil>
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
    
    # Worklist
    <Location /integration/>
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
    
    # B2B
    <Location /b2b>
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
    
    <Location /b2bconsole>
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
    
    # UMS
    <Location /sdpmessaging/ >
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
     
    # UMS WS
    <Location /ucs/messaging/webservice >
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
    
    # Default to-do taskflow
    <Location /DefaultToDoTaskFlow/>
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
    
    # Workflow
    <Location /workflow>
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
    
    #Required if attachments are added for workflow tasks
     <Location /ADFAttachmentHelper> 
        SetHandler weblogic-handler 
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001 
     </Location>
    
    # SOA composer application 
     <Location /soa/composer> 
         SetHandler weblogic-handler 
         WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001 
     </Location>
    
    # OSB wsil 
     <Location /sbinspection.wsil*> 
         SetHandler weblogic-handler 
         WebLogicCluster OSBHOST1VHN1:8011,OSBHOST2VHN2:8011 
     </Location>
     
    # OSB resource 
     <Location /sbresource*> 
         SetHandler weblogic-handler 
         WebLogicCluster OSBHOST1VHN1:8011,OSBHOST2VHN2:8011
     </Location>
    
  2. Make sure the httpd.conf file located in the same directory as the mod_wl_ohs file contains the following lines:

    NameVirtualHost *:7777
    <VirtualHost *:7777>
    ServerName https://soa.mycompany.com:443
    ServerAdmin you@your.address
    RewriteEngine On
    RewriteOptions inherit
    </VirtualHost>
     
    NameVirtualHost *:7777
    <VirtualHost *:7777>
        ServerName admin.mycompany.com:80
        ServerAdmin you@your.address
        RewriteEngine On
        RewriteOptions inherit
    </VirtualHost>
    

    Note:

    Values such as soa.mycompany.com:443, 7777, admin.mycompany:80, and you@youraddress that are noted in this document serve as examples only. Enter values based on the actual environment.

    Note:

    Each HTTP proxy service in Oracle Service Bus exposes its own context root, which is by default the full path of the proxy service. For example, 'ordersubmit.proxy' in a project called 'OrderProcessing' and in folder 'submission' exposes /OrderProcessing/submission/ordersubmit as context path. These url contexts need to be added in Oracle HTTP Server for each service.
  3. Perform the same steps for the Oracle HTTP Server on WEBHOST2.

  4. Restart Oracle HTTP Server on both WEBHOST1 and WEBHOST2:

    WEBHOST1> ORACLE_BASE/admin/<instance_name>/bin/opmnctl restartproc ias-component=ohs1
    
    WEBHOST2> ORACLE_BASE/admin/<instance_name>/bin/opmnctl restartproc ias-component=ohs2
    

5.14.16 Validating Access Through Oracle HTTP Server

Verify that the SOA Servers status is reported as "Running" in the Administration Console. If the server is shown as "Starting" or "Resuming," wait for the server status to change to "Started." If another status is reported (such as "Admin" or "Failed"), check the server output log files for errors. Verify that you can access these URLS:

  • http://WEBHOST1:7777/wsm-pm

  • http://WEBHOST2:7777/wsm-pm

  • http://WEBHOST1:7777/soa-infra

  • http://WEBHOST2:7777/soa-infra

  • http://WEBHOST1:7777/soa/composer

  • http://WEBHOST2:7777/soa/composer

  • http://WEBHOST1:7777/integration/worklistapp

  • http://WEBHOST2:7777/integration/worklistapp

  • http://WEBHOST1:7777/sdpmessaging/userprefs-ui

  • http://WEBHOST2:7777/sdpmessaging/userprefs-ui

  • http://WEBHOST1:7777/b2bconsole

  • http://WEBHOST2:7777/b2bconsole

  • http://WEBHOST1:7777/sbinspection.wsil

  • http://WEBHOST2:7777/sbinspection.wsil

Verify these URLs also using your load balancing router address:

  • http://soa.mycompany.com:80/wsm-pm

  • http://soa.mycompany.com:80/soa-infra

  • http://soa.mycompany.com:80/soa/composer

  • http://soa.mycompany.com:80/integration/worklistapp

  • http://soa.mycompany.com:80/sdpmessaging/userprefs-ui

  • http://soa.mycompany.com:80/b2bconsole

  • http://soa.mycompany.com:80/sbinspection.wsil

Follow these instructions to ensure that routing and failover from the HTTP Server to the SOA_CLuster is working correctly:

  1. While WLS_SOA2 is running, stop WLS_SOA1 from Oracle WebLogic Server Administration Console.

  2. Access the following URLs and verify the appropriate functionality:

    • WEBHOST1:7777/wsm-pm

    • WEBHOST1:7777/soa-infra

    • WEBHOST1:7777/soa/composer

    • WEBHOST1:7777/integration/worklistapp

    • WEBHOST1:7777/sdpmessaging/userprefs-ui

    • WEBHOST1:7777/b2bconsole

  3. Start WLS_SOA1 from Oracle WebLogic Server Administration Console.

  4. Stop WLS_SOA2.

  5. Access the URLs in Step 2 above again and verify the appropriate functionality:

Follow these instructions to ensure that routing and failover from the HTTP Server to the OSB_Cluster is working correctly:

  1. While WLS_OSB2 is running, stop WLS_OSB1 from Oracle WebLogic Server Administration Console.

  2. Access the following URL and verify the appropriate functionality:

    • WEBHOST1:7777/sbinspection.wsil

  3. Start WLS_OSB1 from Oracle WebLogic Server Administration Console.

  4. Stop WLS_OSB2.

  5. Access the URL in Step 2 above again and verify the appropriate functionality:

5.14.17 Setting the Front End HTTP Host and Port

You need to set the front end HTTP host and port for Oracle WebLogic Server cluster:

  1. In the WebLogic Server Administration Console, in the Change Center section, click Lock & Edit.

  2. In the left pane, select Environment > Clusters.

  3. Select the SOA_Cluster.

  4. Select HTTP.

  5. Set the values for the following:

    • Frontend Host: soa.mycompany.com

      Frontend HTTP Port: 80

    • Frontend HTTPS Port: leave it blank

    Note:

    Make sure this address is correct and available (the load balancing router is up). An incorrect value, for example, http:// in the address, or trailing / in the hostname, may prevent the SOA system from being accessible even when using the virtual IPs to access it.
  6. Click Save.

  7. To activate the changes, click Activate Changes in the Change Center section of the Administration Console.

  8. Repeat steps 1 through 7 for the OSB_Cluster.

  9. If you have started the server before, notice this change requires a restart of the managed servers in the cluster.

    Note:

    The SOA system calculates the callback URL as follows:
    • If a request to SOA originates from an external or internal service is originating a request to SOA, then SOA uses the callback URL specified by the client.

    • If a request to an external or internal service originates from SOA, the callbackURL cannot be populated in the SOA request dynamically because SOA is the originator. Instead, callbackServerURL is used if it is specified as a binding property for the specific reference. (You can set this when modeling the composite or at runtime using the MBeans accessed through Oracle Enterprise Manager console.) This allows different service calls to have different callback URLs. That is, a callback URL from an external service is different than one to an internal service.

      However, if the callbackServer URL is not specified as a binding property for that reference, then the system uses the callback URL as specified in soa-infra-config. If the callback URL is not specified in soa-infra-config, then the system uses the callback URL as the front end host specified in WLS. If the front end host is not specified in WLS, the system uses the callback URL as the local host name as provided by WLS MBean APIs.

For SOA high availability installations frontended by Oracle HTTP Server, monitoring should be done on the Oracle HTTP Server ports of the real backend servers. This is the case when a deployment is using all the components deployed to the SOA Managed Server. A simple HTTP monitor that pings the HTTP/HTTPS port and expects a pre-determined response in turn should suffice. If only a specific SOA component is being used (such as B2B), then a monitor that does a deeper level check all the way to the managed server can be considered to validate the health of the component in use. Please check with your load balancer vendor on setting up the HTTP monitors with your load balancer.

If you do not set the front end HTTP host and port, you get the following message when trying to retrieve a document definition XSD from Oracle B2B:

An error occured while loading the document definitions.
java.lang.IllegalArgumentException: Cluster address must be set when
clustering is enabled.

5.14.18 Configuring a Default Persistent Store for Transaction Recovery

Each server has a transaction log, which stores information about committed transactions coordinated by the server that may not have been completed. WebLogic Server uses the transaction log when recovering from system crashes or network failures. To take advantage of the migration capability of the Transaction Recovery Service for servers in a cluster, you must store the transaction log in a location that is available to a server and its backup servers (the nodes the WebLogic Server can failover to), preferably on a dual-ported SCSI disk, or a Storage Area Network (SAN). To configure the default persistent store:

  1. In the Domain Structure tree, expand Environment and select Servers.

  2. Select the server you want to modify.

  3. Select the Configuration, Services tab.

  4. Under Default Store, in the Directory field, enter the path to the folder where you want the default persistent store to store its data files.

    To enable migration of the Transaction Recovery Service, you must specify a location on a persistent storage solution that is available to other servers in the cluster. Therefore, the directory that you enter must be accessible by both WLS_SOA1 and WLS_SOA2 for the SOA_Cluster and WLS_OSB1 and WLS_OSB2 for the OSB_Cluster. This directory must exist before the server is restarted.

5.14.19 Deploying Applications

You can deploy SOA composite applications from Oracle Enterprise Manager Fusion Middleware Control Console with the Deploy SOA Composite wizard. Use the Deploy SOA Composite wizard to deploy any of the following:

  • A new SOA composite application for the first time

  • A new revision (for example, 2.0) alongside an older revision (for example, 1.0) without impacting the latter. The revision deployed last becomes the new default revision of that composite (unless you specify otherwise at a later step during deployment).

  • A bundle (ZIP file) containing multiple SOA composite application revisions (for example, revisions 2.0, 3.0, and 4.0) of a SOA composite application that already has a different revision that is currently deployed (for example, 1.0). This option enables you to deploy revisions 1.0, 2.0, 3.0, and 4.0 at the same time. The bundle can also contain revisions of different composites. There is no restriction that all revisions must be of the same composite application.

Deployment extracts and activates the composite application in SOA Service Infrastructure. Once an application is deployed, you can perform administration tasks, such as creating instances, configuring properties, monitoring performance, managing instance life cycles, and managing policies and faults.

Note:

If you want to redeploy an existing revision of an application, do not use this wizard. Instead, use the Redeploy SOA Composite wizard.

To deploy applications:

  1. Access the Deploy SOA Composite wizard through one of the following options:

    From SOA Infrastructure Menu... From the SOA Folder in the Navigator... From the SOA Infrastructure Home Page... From the SOA Composite Menu...
    1. Select SOA Deployment > Deploy.
    1. Right-click soa-infra.
    2. Select SOA Deployment > Deploy.

    1. Click the Deployed Composites tab.
    2. Above the Composite table, click Deploy.

    1. Select SOA Deployment > Deploy Another Composite.

    The Select Archive page appears.

    Description of sca_deploy.gif follows
    Description of the illustration sca_deploy.gif

  2. In the Archive or Exploded Directory section, specify the archive of the SOA composite application to deploy. The archive contains the project files of the composite to be deployed (for example, HelloWorld_rev1.0.jar for a single archive or OrderBooking_rev1.0.zip for multiple archives).

  3. In the Configuration Plan section, optionally specify the configuration plan to include with the archive. The configuration plan enables you to define the URL and property values to use in different environments. During process deployment, the configuration plan is used to search the SOA project for values that must be replaced to adapt the project to the next target environment.

  4. Click Next.

    The Select Target page appears.

  5. Select the WebLogic Server or cluster to which to deploy the SOA composite application archive. You can deploy to multiple servers and clusters.

  6. Click Next.

    The Confirmation page appears.

  7. Review your selections.

  8. Select whether or not to deploy the SOA composite application as the default revision. The default revision is instantiated when a new request comes in.

  9. Click Deploy.

    Processing messages are displayed.

  10. When deployment has completed, click Close.

See Also:

Oracle Fusion Middleware Developer's Guide for Oracle SOA Suite for instructions on creating configuration plans and deploying applications from Oracle JDeveloper

5.14.20 Configuring Server Migration for the WLS_SOA Servers

The high availability architecture for a SOA system uses server migration to protect some singleton services against failures. For more information on whole server migration, see Oracle Fusion Middleware Using Clusters for Oracle WebLogic Server.

The WLS_SOA1 managed server is configured to be restarted on SOAHOST2 in case of failure, and the WLS_SOA2 managed server is configured to be restarted on SOAHOST1 in case of failure. For this configuration the WLS_SOA1 and WLS_SOA2 servers listen on specific floating IPs that are failed over by WLS Server Migration. To configure server migration for the WLS_SOAn managed servers, follow these steps:

Step 1   Set Up User and Tablespace for the Server Migration Leasing Table
  1. Create a tablespace called leasing.

    Example: Log on to SQL*Plus as the sysdba user and run the following command:

    SQL> create tablespace leasing
            logging datafile 'DB_HOMEs/oradata/orcl/leasing.dbf'
            size 32m autoextend on next 32m maxsize 2048m extent management local;
    
  2. Create a user named leasing and assign to it the leasing tablespace.

    SQL> create user leasing identified by welcome1;
    
    SQL> grant create table to leasing;
    
    SQL> grant create session to leasing;
    
    SQL> alter user leasing default tablespace leasing;
    
    SQL> alter user leasing quota unlimited on LEASING;
    
  3. Create the leasing table using the leasing.ddl script.

    1. Copy the leasing.ddl file, located in the WL_HOME/server/db/oracle/817 or WL_HOME/server/db/oracle/920 directories, to your database node.

    2. Connect to the database as the leasing user.

    3. Run the leasing.ddl script in SQL*Plus.

      SQL> @copy_location/leasing.ddl;
      
Step 2   Create a Multi Data Source from Oracle WebLogic Server Administration Console

The second step is to create a multi data source for the leasing table from the Oracle WebLogic Server Administration Console:

You create a data source to each of the Oracle RAC database instances during the process of setting up the multi data source, both for these data sources and the global leasing multi data source. When you create a data source:

  • Make sure that this is a non-xa data source.

  • The names of the multi data sources are in the following format: <MultiDS>-rac0, <MultiDS>-rac1

  • Use Oracle's Driver (Thin) Version 9.0.1, 9.2.0, 10, 11.

  • Data sources do not require support for global transactions. Therefore, do not use any type of distributed transaction emulation/participation algorithm for the data source (do not choose the Supports Global Transactions option, or the Logging Last Resource, Emulate Two-Phase Commit, or One-Phase Commit options of the Supports Global Transactions option), and specify a service name for your database.

  • Target these data sources to the SOA cluster.

  • Make sure the datasources' connection pool initial capacity is set to 0. To do this, select Services, JDBC, and then Datasources. In the Datasources screen, click the Datasource Name, then click the Connection Pool tab, and enter 0 in the Initial capacity field.

To create a multi data source:

  1. From the Domain Structure window in the Oracle WebLogic Server Administration Console, expand the Services node, then click Data Sources. The Summary of JDBC Data Sources page appears.

  2. In the Change Center, click Lock & Edit.

  3. Click New, then click Multi Data Sources. The Create a New JDBC Multi Data Source page appears.

  4. Enter leasing as the Name.

  5. Enter jdbc/leasing as the JNDI name.

  6. Select Failover as algorithm (default).

  7. Click Next.

  8. Select non-XA driver (the default).

  9. Click Next.

  10. Click Create New Data Source.

  11. Enter leasing-rac0 as name. Enter jdbc/leasing-rac0 as JNDI name. Enter oracle as the database type. For the driver type, enter Oracle Driver (Thin) for RAC server-Instance connection Version 10,11.

  12. Click Next.

  13. Deselect Supports Global Transactions.

  14. Click Next.

  15. Enter the service name, database name (this is actually the RAC Node instance name, for example: racdb1,racdb2), host port, and password for your leasing schema.

  16. Click Next.

  17. Click Test Configuration and verify the connection works.

  18. Target the data source to the SOA cluster.

  19. Select the data source and add it to the right screen.

  20. Click Create a New Data Source and repeat the steps for the second instance of your Oracle RAC database.

  21. Add the second data source to your multi data source.

  22. Save and Activate the changes.

Step 3   Edit the Node Manager's Properties file

For information on Node Manager and whole server migration, see Section 3.9, "Whole Server Migration."

The nodemanager.properties file is located in the WL_HOME/common/nodemanager directory. For server migration to work properly, you need to add the properties listed in this section:

  • Interface=eth0

    Note:

    Do not specify the sub interface, such as eth0:1 or eth0:2. This interface is to be used without the :0, or :1. The Node Manager's scripts traverse the different :X enabled IPs to determine which to add or remove. For example, the valid values in Linux environments are eth0, eth1, or, eth2, eth3, ethn, depending on the number of interfaces configured.

    This property specifies the interface name for the floating IP (eth0, for example, on Linux).

    Note:

    For Windows, the Interface should be set to the Network Interface Name. For example: Interface="Local Area Connection".

    Be sure that the interface provided is the public interface for this node. On multi home nodes, this interface should be the one on which the floating IP can be enabled.

  • NetMask=255.255.255.0

    This property specifies the net mask for the interface for the floating IP. The netmask provided (255.255.255.0) is just an example. The actual value depends on your network.

  • UseMACBroadcast=true

    This property specifies whether or not to use a node's MAC address when sending ARP packets, that is, whether or not to use the -b flag in the arping command.

After starting Node Manager, verify in Node Manager's output (shell where Node Manager is started) that these properties are being used, or problems may arise during migration. You should see something like this in the Node Manager's output:

StateCheckInterval=500

Interface=eth0 (Linux) or Interface="Local Area Connection" (Windows)

NetMask=255.255.255.0

UseMACBroadcast=true

Note:

The steps in this section are not required if the server properties (start properties) have been properly set and the Node Manager can start the servers remotely.
  1. Set the following property in the nodemanager.properties file.

    • StartScriptEnabled

      Set this property to true.

  2. Start the Node Manager on Node 1 and Node 2 by running the startNodeManager.sh script located in the WL_HOME/server/bin directory.

  3. Validate the changes to the nodemanager.properties file by checking the nodemanager.log file.

Step 4   Set Environment and Superuser Privileges for the wlsifconfig.sh script
  1. Grant sudo privilege to the WebLogic user ('oracle') with No Password restriction, and grant execute privilege on /sbin/ifconfig and /sbin/arping binaries.

    Note:

    On Windows, the script is named wlsifconfig.cmd and it can be run by users with the administrator privilege.

    Make sure the script is executable by the WebLogic user ('oracle'). The following is an example of an entry inside /etc/sudoers granting sudo execution privilege for oracle and also over the ifconfig and arping:

    oracle ALL=NOPASSWD: /sbin/ifconfig,/sbin/arping
    
Step 5   Configure Server Migration Targets

Configuring Cluster Migration sets the DataSourceForAutomaticMigration property to true. Follow the steps in this section to configure cluster migration in a migration in a cluster:

  1. Log into Oracle WebLogic Server Administration Console

  2. In the left pane, expand Environment and select Clusters.

  3. Select the cluster for which you want to configure migration (SOA_Cluster).

  4. Click Migration.

  5. In the Change Center, click Lock & Edit.

  6. In the Available field, select the machine to which to allow migration and click the right arrow. In this case, select SOAHOST1 and SOAHOST2.

  7. Select the data source to be used for automatic migration. In this case select the leasing data source.

  8. Click Save.

  9. Set the Candidate Machines for Server Migration. This needs to be done for all the managed servers.

    To do this, follow these steps:

    1. In the left pane of Oracle WebLogic Server Administration Console, expand Environment and select Servers.

    2. Select the server for which you want to configure migration.

    3. Click the Migration tab.

    4. In the Available field, located in the Migration Configuration section, select the machines to which to allow migration and click the right arrow. For WLS_SOA1, select SOAHOST2. For WLS_SOA2, select SOAHOST1.

    5. Select the checkbox for Automatic Server Migration Enabled. This enables the Node Manager to start a failed server on the target node automatically.

    6. Click Save and activate the changes.

    7. These changes require restart of the servers and Node Managers to be effective.

Step 6   Test Server Migration

To verify that Server Migration is working properly, follow these steps:

From SOAHOST1:

  1. Force stop the WLS_SOA1 managed server.

    To do this, run this command:

    SOAHOST1> kill -9 <pid>
    

    pid specifies the process ID of the managed server. You can identify the pid in the node by running this command:

    SOAHOST1> ps -ef | grep WLS_SOA1
    

    Note:

    For Windows, the Managed Server can be terminated by using the taskkill command. For example:
    taskkill /f /pid <pid>
    

    Where <pid> is the process Id of the Managed Server.

    To determine the process Id of Managed server run the following command and identify the pid of the WLS_SOA1 Managed Server.

    MW_HOME\jrockit_160_20_D1.0.1-2124\bin\jps -l -v
    
  2. Watch the Node Manager console: you should see a message indicating that WLS_SOA1's floating IP has been disabled.

  3. Wait for the Node Manager to try a second restart of WLS_SOA1. Node Manager waits for a fence period of 30 seconds before trying this restart.

  4. Once Node Manager restarts the server, stop it again. Now Node Manager should log a message indicating that the server will not be restarted again locally.

From SOAHOST2:

  1. Watch the local Node Manager console. After 30 seconds since the last try to restart WLS_SOA1 on Node 1, Node Manager on Node 2 should prompt that the floating IP for WLS_SOA1 is being brought up and that the server is being restarted in this node.

  2. Access the soa-Service Infrastructure console in the same IP.

Migration can also be verified in Oracle WebLogic Server Administration Console:

  1. Log into Oracle WebLogic Server Administration Console.

  2. Click on Domain on the left pane.

  3. Click on the Monitoring tab and then on the Migration sub-tab.

    The Migration Status table provides information on the status of the migration.

    Figure 5-51 Migration Status Screen in Oracle WebLogic Server Administration Console

    Migration Status screen in the Administration Console
    Description of "Figure 5-51 Migration Status Screen in Oracle WebLogic Server Administration Console"

Note:

On Windows, when you manually shut down multiple servers at the same time, on the same machine, and then, on another machine, attempt to start one of the servers that was shut down, the IP bind may not work. This happens because the original machine still has claim to the IP address, even though netsh has reported that the IP address has been removed.

To resolve this, you must check the network configuration either by using the ipconfig utility or Windows Network Configuration. Either of these may show that one of the virtual/floating IP addresses is still configured even though the servers have been shut down. You can then use Windows Network Configuration to remove the IP address using the following procedure:

  1. From Windows Control Panel, select Network Connections.

  2. Select the appropriate network interface, right-click, and select Properties.

  3. Select Internet Protocol (TCP/IP) and click the Properties button.

  4. Select Advanced.

  5. Select the appropriate IP address and click the Remove button.

Note:

After a server is migrated, to fail it back to its original node/machine, stop the managed server from the Oracle WebLogic Administration Console and then start it again. The appropriate Node Manager will start the managed server on the machine to which it was originally assigned.

5.14.21 Configuring Server Migration for the WLS_OSB Servers

The high availability architecture for an OSB system uses server migration to protect some singleton services against failures. For more information on whole server migration, see Oracle Fusion Middleware Using Clusters for Oracle WebLogic Server.

The WLS_OSB1 managed server is configured to be restarted on OSBHOST2 in case of failure, and the WLS_OSB2 managed server is configured to be restarted on OSBHOST1 in case of failure. For this configuration the WLS_OSB1 and WLS_OSB2 servers listen on specific floating IPs that are failed over by WLS Server Migration. To configure server migration for the WLS_OSBn managed servers, follow these steps:

Note:

If a tablespace, schema multi data source, and data sources have already been set up for SOA (as described in Section 5.14.20, "Configuring Server Migration for the WLS_SOA Servers" above, then the OSB Cluster can reuse the existing leasing schema and data source and Step 1 and Step 2 below are not required. Instead, simply target the data sources and multi data sources created for SOA to the OSB Cluster.
Step 1   Set Up User and Tablespace for the Server Migration Leasing Table
  1. Create a tablespace called leasing.

    Example: Log on to SQL*Plus as the sysdba user and run the following command:

    SQL> create tablespace leasing
            logging datafile 'DB_HOMEs/oradata/orcl/leasing.dbf'
            size 32m autoextend on next 32m maxsize 2048m extent management local;
    
  2. Create a user named leasing and assign to it the leasing tablespace.

    SQL> create user leasing identified by welcome1;
    
    SQL> grant create table to leasing;
    
    SQL> grant create session to leasing;
    
    SQL> alter user leasing default tablespace leasing;
    
    SQL> alter user leasing quota unlimited on LEASING;
    
  3. Create the leasing table using the leasing.ddl script.

    1. Copy the leasing.ddl file, located in the WL_HOME/server/db/oracle/817 or WL_HOME/server/db/oracle/920 directories, to your database node.

    2. Connect to the database as the leasing user.

    3. Run the leasing.ddl script in SQL*Plus.

      SQL> @copy_location/leasing.ddl;
      
Step 2   Create a Multi Data Source from Oracle WebLogic Server Administration Console

The second step is to create a multi data source for the leasing table from the Oracle WebLogic Server Administration Console:

You create a data source to each of the Oracle RAC database instances during the process of setting up the multi data source, both for these data sources and the global leasing multi data source. When you create a data source:

  • Make sure that this is a non-xa data source.

  • The names of the multi data sources are in the following format: <MultiDS>-rac0, <MultiDS>-rac1

  • Use Oracle's Driver (Thin) Version 9.0.1, 9.2.0, 10, 11.

  • Data sources do not require support for global transactions. Therefore, do not use any type of distributed transaction emulation/participation algorithm for the data source (do not choose the Supports Global Transactions option, or the Logging Last Resource, Emulate Two-Phase Commit, or One-Phase Commit options of the Supports Global Transactions option), and specify a service name for your database.

  • Target these data sources to the OSB cluster.

  • Make sure the datasources' connection pool initial capacity is set to 0. To do this, select Services, JDBC, and then Datasources. In the Datasources screen, click the Datasource Name, then click the Connection Pool tab, and enter 0 in the Initial capacity field.

To create a multi data source:

  1. From the Domain Structure window in the Oracle WebLogic Server Administration Console, expand the Services node, then click Data Sources. The Summary of JDBC Data Sources page appears.

  2. In the Change Center, click Lock & Edit.

  3. Click New, then click Multi Data Sources. The Create a New JDBC Multi Data Source page appears.

  4. Enter leasing as the Name.

  5. Enter jdbc/leasing as the JNDI name.

  6. Select Failover as algorithm (default).

  7. Click Next.

  8. Select non-XA driver (the default).

  9. Click Next.

  10. Click Create New Data Source.

  11. Enter leasing-rac0 as name. Enter jdbc/leasing-rac0 as JNDI name. Enter oracle as the database type. For the driver type, enter Oracle Driver (Thin) for RAC server-Instance connection Version 10,11.

  12. Click Next.

  13. Deselect Supports Global Transactions.

  14. Click Next.

  15. Enter the service name, database name (this is actually the RAC Node instance name, for example: racdb1,racdb2), host port, and password for your leasing schema

  16. Click Next.

  17. Click Test Configuration and verify the connection works.

  18. Target the data source to the OSB cluster.

  19. Select the data source and add it to the right screen.

  20. Click Create a New Data Source and repeat the steps for the second instance of your Oracle RAC database.

  21. Add the second data source to your multi data source.

  22. Save and Activate the changes.

Step 3   Edit the Node Manager's Properties file

For information on Node Manager and whole server migration, see Section 3.9, "Whole Server Migration."

The nodemanager.properties file is located in the WL_HOME/common/nodemanager directory. For server migration to work properly, you need to add the properties listed in this section:

  • Interface=eth0

    Note:

    Do not specify the sub interface, such as eth0:1 or eth0:2. This interface is to be used without the :0, or :1. The Node Manager's scripts traverse the different :X enabled IPs to determine which to add or remove. For example, the valid values in Linux environments are eth0, eth1, or, eth2, eth3, ethn, depending on the number of interfaces configured.

    This property specifies the interface name for the floating IP (eth0, for example, on Linux).

    Note:

    For Windows, the Interface should be set to the Network Interface Name. For example: Interface="Local Area Connection".

    Be sure that the interface provided is the public interface for this node. On multi home nodes, this interface should be the one on which the floating IP can be enabled.

  • NetMask=255.255.255.0

    This property specifies the net mask for the interface for the floating IP. The netmask provided (255.255.255.0) is just an example. The actual value depends on your network.

  • UseMACBroadcast=true

    This property specifies whether or not to use a node's MAC address when sending ARP packets, that is, whether or not to use the -b flag in the arping command.

After starting Node Manager, verify in Node Manager's output (shell where Node Manager is started) that these properties are being used, or problems may arise during migration. You should see something like this in the Node Manager's output:

StateCheckInterval=500

Interface=eth0 (Linux) or Interface="Local Area Connection" (Windows)

NetMask=255.255.255.0

UseMACBroadcast=true

Note:

The steps in this section are not required if the server properties (start properties) have been properly set and the Node Manager can start the servers remotely.
  1. Set the following property in the nodemanager.properties file.

    • StartScriptEnabled

      Set this property to true.

  2. Start the Node Manager on OSBHOST1 and OSBHOST2 by running the startNodeManager.sh script located in the WL_HOME/server/bin directory.

  3. Validate the changes to the nodemanager.properties file by checking the nodemanager.log file.

Step 4   Set Environment and Superuser Privileges for the wlsifconfig.sh script
  1. Grant sudo privilege to the WebLogic user ('oracle') with No Password restriction, and grant execute privilege on /sbin/ifconfig and /sbin/arping binaries.

    Note:

    On Windows, the script is named wlsifconfig.cmd and it can be run by users with the administrator privilege.

    Make sure the script is executable by the WebLogic user ('oracle'). The following is an example of an entry inside /etc/sudoers granting sudo execution privilege for oracle and also over the ifconfig and arping:

    oracle ALL=NOPASSWD: /sbin/ifconfig,/sbin/arping
    
Step 5   Configure Server Migration Targets

Configuring Cluster Migration sets the DataSourceForAutomaticMigration property to true. Follow the steps in this section to configure cluster migration in a migration in a cluster:

  1. Log into Oracle WebLogic Server Administration Console

  2. In the left pane, expand Environment and select Clusters.

  3. Select the cluster for which you want to configure migration (OSB_Cluster).

  4. Click Migration.

  5. In the Change Center, click Lock & Edit.

  6. In the Available field, select the machine to which to allow migration and click the right arrow. In this case, select OSBHOST1 and OSBHOST2.

  7. Select the data source to be used for automatic migration. In this case select the leasing data source.

  8. Click Save.

  9. Set the Candidate Machines for Server Migration. This needs to be done for all the managed servers.

    To do this, follow these steps:

    1. In the left pane of Oracle WebLogic Server Administration Console, expand Environment and select Servers.

    2. Select the server for which you want to configure migration.

    3. Click the Migration tab.

    4. In the Available field, located in the Migration Configuration section, select the machines to which to allow migration and click the right arrow. For WLS_OSB1, select OSBHOST2. For WLS_OSB2, select OSBHOST1.

    5. Select the checkbox for Automatic Server Migration Enabled. This enables the Node Manager to start a failed server on the target node automatically.

    6. Click Save and activate the changes.

    7. These changes require restart of the servers and Node Managers to be effective.

Step 6   Test Server Migration

To verify that Server Migration is working properly, follow these steps:

From OSBHOST1:

  1. Force stop the WLS_OSB1 managed server.

    To do this, run this command:

    OSBHOST1> kill -9 <pid>
    

    pid specifies the process ID of the managed server. You can identify the pid in the node by running this command:

    OSBHOST1> ps -ef | grep WLS_OSB1
    

    Note:

    For Windows, the Managed Server can be terminated by using the taskkill command. For example:
    taskkill /f /pid <pid>
    

    Where <pid> is the process Id of the Managed Server.

    To determine the process Id of the Managed Server run the following command and identify the pid of the WLS_OSB1 Managed Server.

    MW_HOME\jrockit_160_20_D1.0.1-2124\bin\jps -l -v
    
  2. Watch the Node Manager console: you should see a message indicating that WLS_OSB1's floating IP has been disabled.

  3. Wait for the Node Manager to try a second restart of WLS_OSB1. Node Manager waits for a fence period of 30 seconds before trying this restart.

  4. Once Node Manager restarts the server, stop it again. Now Node Manager should log a message indicating that the server will not be restarted again locally.

From OSBHOST2:

  1. Watch the local Node Manager console. After 30 seconds since the last try to restart WLS_OSB1 on Node 1, Node Manager on Node 2 should prompt that the floating IP for WLS_OSB1 is being brought up and that the server is being restarted in this node.

  2. Access the sbinspection.wsil urlconsole in the same IP.

Migration can also be verified in Oracle WebLogic Server Administration Console:

  1. Log into Oracle WebLogic Server Administration Console.

  2. Click on Domain on the left pane.

  3. Click on the Monitoring tab and then on the Migration sub-tab.

    The Migration Status table provides information on the status of the migration.

    Figure 5-52 Migration Status Screen in Oracle WebLogic Server Administration Console

    Migration Status screen in the Administration Console
    Description of "Figure 5-52 Migration Status Screen in Oracle WebLogic Server Administration Console"

Note:

On Windows, when you manually shut down multiple servers at the same time, on the same machine, and then, on another machine, attempt to start one of the servers that was shut down, the IP bind may not work. This happens because the original machine still has claim to the IP address, even though netsh has reported that the IP address has been removed.

To resolve this, you must check the network configuration either by using the ipconfig utility or Windows Network Configuration. Either of these may show that one of the virtual/floating IP addresses is still configured even though the servers have been shut down. You can then use Windows Network Configuration to remove the IP address using the following procedure:

  1. From Windows Control Panel, select Network Connections.

  2. Select the appropriate network interface, right-click, and select Properties.

  3. Select Internet Protocol (TCP/IP) and click the Properties button.

  4. Select Advanced.

  5. Select the appropriate IP address and click the Remove button.

Note:

After a server is migrated, to fail it back to its original node/machine, stop the managed server from the Oracle WebLogic Administration Console and then start it again. The appropriate Node Manager will start the managed server on the machine to which it was originally assigned.

5.14.22 Scaling the Topology

You can scale out or scale up this high availability topology. When you "scale up" the topology, you add new managed servers to nodes that are already running one or more managed servers. When you "scale out" the topology, you add new managed servers to new nodes.

5.14.22.1 Scaling Up the Topology (Adding Managed Servers to Existing Nodes)

In this case, you already have a node that runs a managed server configured with SOA and Oracle Service Bus components. The node contains a Middleware home, an Oracle HOME (SOA and Oracle Service Bus) and a domain directory for existing managed servers.

You can use the existing installations (the Middleware home, and domain directories) for creating new WLS_OSB servers. There is no need to install SOA or Oracle Service Bus binaries in a new location, or to run pack and unpack.

For scaling up the SOA Servers, follow the steps in Section 5.13.22.1, "Scaling Up the Topology (Adding Managed Servers to Existing Nodes)."

Follow these steps for scaling up the topology for the OSB server:

  1. Using the Administration Console, clone WLS_OSBn into a new managed server. The source managed server to clone should be one that already exists on the node where you want to run the new managed server.

    To clone a managed server:

    1. Select Environment -> Servers from the Administration Console.

    2. Select the managed server that you want to clone (for example, WLS_OSB1).

    3. Select Clone.

    Name the new managed server WLS_OSBn, where n is a number to identify the new managed server.

    The rest of the steps assume that you are adding a new server to OSBHOST1, which is already running WLS_OSB1.

  2. For the listen address, assign the virtual host name to use for this new managed server. If you are planning to use server migration as recommended for this server, this virtual host name will enable it to move to another node. The virtual host name should be different from those in use by other managed servers (may be in the same or different domain) that are running in the nodes used by the OSB/SOA domain.

    Perform these steps to set the managed server listen address:

    1. Log into the Oracle WebLogic Server Administration Console.

    2. In the Change Center, click Lock & Edit.

    3. Expand the Environment node in the Domain Structure window.

    4. Click Servers. The Summary of Servers page is displayed.

    5. Select the managed server whose listen address you want to update in the Names column of the table. The Setting page for that managed server is displayed.

    6. Set the Listen Address to OSBHOST1VHN1.

    7. Click Save.

    8. Save and activate the changes.

    The changes will not take effect until the managed server is restarted.

  3. Update the cluster address to include the new server:

    1. Select Environment -> Cluster from the Administration Console.

    2. Click on the OSB_Cluster server. The Settings Screen for the OSB_Cluster appears.

    3. In the Change Center, click Lock & Edit.

    4. Add the new server's address and port to the Cluster Address field. For example:

      OSBHOST1VHN1:8011,OSBHOST2VHN1:8011,OSBHOST1VHN2:8011
      
  4. If your Oracle Service Bus configuration includes one or more business services that use JMS request/response functionality, then you must also perform the following procedure using the Oracle Service Bus Console after adding the new managed server to the cluster:

    1. In the Change Center, click Create to create a session.

    2. Using the Project Explorer, locate and select a business service that uses JMS request/response. Business services of this type display Messaging Service as their Service Type.

    3. At the bottom of the View Details page, click Edit.

    4. If there is a cluster address in the endpoint URI, add the new server to the cluster address.

    5. On the Edit a Business Service - Summary page, click Save.

    6. Repeat the previous steps for each remaining business service that uses JMS request/response.

    7. In the Change Center, click Activate.

    8. Restart the managed server.

    9. Restart the Administration Server.

    The business services are now configured for operation in the extended domain.

    Note:

    For business services that use a JMS MesageID correlation scheme, you must edit the connection factory settings to add an entry to the table mapping managed servers to queues. For information on how to configure queues and topic destinations, see "JMS Server Targeting" in Oracle Fusion Middleware Configuring and Managing JMS for Oracle WebLogic Server.
  5. If your Oracle Service Bus configuration includes one or more proxy services that use JMS endpoints with cluster addresses, then you must also perform the following procedure using the Oracle Service Bus Console after adding the new managed server to the cluster:

    1. In the Change Center, click Create to create a session.

    2. Using the Project Explorer, locate and select a proxy service that uses JMS endpoints with cluster addresses.

    3. At the bottom of the View Details page, click Edit.

    4. If there is a cluster address in the endpoint URI, add the new server to the cluster address.

    5. On the Edit a Proxy Service - Summary page, click Save.

    6. Repeat the previous steps for each remaining proxy service that uses JMS endpoints with cluster addresses.

    7. In the Change Center, click Activate.

    8. Restart the managed server.

    The proxy services are now configured for operation in the extended domain.

  6. Update the Oracle Service Bus result cache Coherence configuration for the new server:

    1. Log into Oracle WebLogic Server Administration Console. In the Change Center, click Lock & Edit.

    2. In the Domain Structure window, expand the Environment node.

    3. Click Servers. The Summary of Servers page appears.

    4. Click the name of the server (represented as a hyperlink) in the Name column of the table. The settings page for the selected server appears.

    5. Click the Server Start tab.

    6. Enter the following for WLS_OSBn (on a single line, without a carriage returns):

      -DOSB.coherence.localhost=osbhost1vhnn -DOSB.coherence.localport=7890 
      -DOSB.coherence.wka1= osbhost1vhn1 -DOSB.coherence.wka1.port=7890 
      -DOSB.coherence.wka2= osbhost2vhn1 -DOSB.coherence.wka1.port=7890
      

      Note:

      The above configuration assumes that servers WLS_OSB1 and WLS_OSB2 are running when WLS_OSBn is started. This will enable WLS_OSBn to join the coherence cluster started by either WLS_OSB1 or WLS_OSB2 using the WKA addresses specified. Also, make sure WLS_OSB1 and WLS_OSB2 are started before WLS_OSBn is started when all three servers are restarted. This will ensure WLS_OSBn join the cluster started by one of WLS_OSB1 or WLS_OSB2. If you want to have a configuration where the order in which the servers are started does not matter, you must add the host and port for WLS_OSBn as WKA for WLS_OSB1 and WLS_OSB2, and also add WLS_OSBn as WKA for WLS_OSBn.
    7. Save and activate the changes (this change requires the Oracle Service Bus servers to be restarted to be effective).

  7. Create JMS Servers and persistent stores for OSB reporting/internal destinations on the new managed server.

    1. Use the Oracle WebLogic Server Administration Console to create a new persistent store for the new WseeJMSServer and name it, for example, OSB_rep_JMSFileStore_N. Specify the path for the store. This should be a directory on shared storage, as recommended in Section 5.14.1.3, "Shared Storage Prerequisites":

      Note:

      This directory must exist before the managed server is started or the start operation fails.
      ORACLE_BASE/admin/DOMAIN_NAME/cluster_name/jms/OSB_rep_JMSFileStore_N
      
    2. Create a new JMS Server for OSB, for example, OSB_rep_JMSServer_N. Use the OSB_rep_JMSFileStore_N for this JMSServer. Target the OSB_rep_JMSServer_N Server to the recently created Managed Server (WLS_OSBn).

    3. Update the SubDeployment targets for the jmsresources OSB JMS Module to include the recently created OSB JMS Server. To do this, expand the Services node and then expand the Messaging node. Choose JMS Modules from the Domain Structure window of the Oracle WebLogic Server Administration Console. The JMS Modules page appears. Click jmsResources (represented as a hyperlink in the Names column of the table). The Settings page for jmsResources appears. Click the SubDeployments tab. The subdeployment module for jmsresources appears.

      Note:

      This subdeployment module name for destinations is a random name in the form of wlsbJMSServerXXXXXX resulting from the Configuration Wizard JMS configuration for the first two servers (WLS_OSB1 and WLS_OSB2).

      Click on the wlsbJMSServerXXXXXX subdeployment and update the targets to include the new OSB_rep_JMSServer_n server.

  8. Create JMS Servers, persistent stores and destinations for OSB JAX-RPC on the new managed server.

    Note:

    WebLogic Advanced Web Services for JAX-RPC Extension uses regular (non-distributed) destinations to ensure that a locally processed request on a service gets enqueued only to a local member.
    1. Use the Oracle WebLogic Server Administration Console to create a new persistent store for the new WseeJMSServer and name it, for example, Wsee_rpc_JMSFileStore_N. Specify the path for the store. This should be a directory on shared storage, as recommended inSection 5.14.1.3, "Shared Storage Prerequisites":

      Note:

      This directory must exist before the managed server is started or the start operation fails.
      ORACLE_BASE/admin/DOMAIN_NAME/cluster_name/jms/Wsee_rpc_JMSFileStore_N
      
    2. Create a new JMS Server for OSB JAX-RPC, for example, OSB_rpc_JMSServer_N. Use the Wsee_rpc_JMSFileStore_N for this JMSServer. Target the OSB_rpc_JMSServer_N Server to the recently created Managed Server (WLS_OSBn).

    3. Update the WseeJMSModule OSB JMS Module with destinations and the recently created OSB JMS Server. To do this, expand the Services node and then expand the Messaging node. Choose JMS Modules from the Domain Structure window of the Oracle WebLogic Server Administration Console. The JMS Modules page appears. Click WseeJmsModule (represented as a hyperlink in the Names column of the table). The Settings page for WseeJmsModule appears. Follow steps d through v to complete this step.

    4. In the Change Center, click Lock and Edit. Click New.

    5. Select Queue.

    6. Click Next.

    7. Enter DefaultCallbackQueue-WseeJmsServer_auto_n as name for the queue.

    8. Enter weblogic.wsee.DefaultCallbackQueue-WseeJmsServer_auto_n as the JNDI name.

    9. Click Next.

    10. Click Create a New Subdeployment.

    11. Accept the default name.

    12. Click OK.

    13. Select OSB_rpc_JMSServer_n as the target.

    14. Click Finish.

    15. Activate the changes.

    16. Update the local JNDI name for the destination by following steps q though u.

    17. In the Change Center, click Lock & Edit.

    18. In the Settings for the WseeJmsModule page, click on the DefaultCallbackQueue-WseeJmsServer_auto_n destination.

    19. In the general Configuration tab, click on Advanced.

    20. Update the local JNDI name to weblogic.wsee.DefaultCallbackQueue.

    21. Activate the changes.

    22. Repeat steps d through n for the DefaultQueue-WseeJmsServer_auto_n queue, using weblogic.wsee.DefaultQueue-WseeJmsServer_auto_n as the JNDi name and weblogic.wsee.DefaultQueue as the local JNDI name.

  9. Create a new SAF agent and target it to the newly added managed server, as follows:

    In the Oracle WebLogic Server Administration Console, expand Services > Messaging > Store-and-Forward Agents, and add a new SAF agent ReliableWseeSAFAgent_auto_N.

    Select persistent store Wsee_rpc_JMSFileStore_N (persistent store created for OSB JAX-RPC). Target the SAF Agent to the new managed server and activate changes.

  10. Configure TX persistent store for the new server. This should be a location visible from other nodes as indicated in the recommendations about shared storage.

    From the Administration Console, select Server_name > Services tab. Under Default Store, in Directory, enter the path to the folder where you want the default persistent store to store its data files.

  11. Disable host name verification for the new managed server. Before starting and verifying the WLS_OSBn managed server, you must disable host name verification. You can re-enable it after you have configured server certificates for the communication between the Oracle WebLogic Administration Server and the Node Manager in OSBHOSTn. If the source server from which the new one has been cloned had already disabled hostname verification, these steps are not required (the hostname verification settings is propagated to the cloned server).

    To disable host name verification:

    1. In the Oracle Enterprise Manager Console, select Oracle WebLogic Server Administration Console.

    2. Expand the Environment node in the Domain Structure window.

    3. Click Servers. The Summary of Servers page appears.

    4. Select WLS_OSBn in the Names column of the table. The Settings page for the server appears.

    5. Click the SSL tab.

    6. Click Advanced.

    7. Set Hostname Verification to None.

    8. Click Save.

  12. If it is not already started, start the Node Manager on the node. To start the Node Manager, use the installation in shared storage from the existing nodes as follows:

    OSBHOSTN> WL_HOME/server/bin/startNodeManager
    
  13. Start and test the new managed server from the Administration Console.

    1. Shut down the existing managed servers in the cluster.

    2. Ensure that the newly created managed server, WLS_OSBn, is up.

    3. Access the application on the newly created managed server (http://vip:port/sbinspection.wsil).

  14. Configure Server Migration for the new managed server.

    Note:

    Since this is a scale-up operation, the node should already contain a Node Manager and environment configured for server migration. The floating IP for the new OSB managed server should also be already present.

    Configure server migration following these steps:

    1. Log into the Administration Console.

    2. In the left pane, expand Environment and select Servers.

    3. Select the name of the new managed server for which you want to configure migration.

    4. Click the Migration tab.

    5. In the Available field, in the "Migration Configuration" section, select the machines to which to allow migration and click the right arrow. Select the same migration targets as for the servers that already exist on the node.

      For example, for new managed servers on OSBHOST1, which is already running WLS_OSB1, select OSBHOST2. For new managed servers on OSBHOST2, which is already running WLS_OSB2, select OSBHOST1.

      Make sure the appropriate resources are available to run the managed servers concurrently during migration.

    6. Select the "Automatic Server Migration Enabled" option. This enables the Node Manager to start a failed server on the target node automatically.

    7. Click Save.

    8. Restart the Administration Server, managed servers, and Node Manager.

  15. Test server migration for this new server. Follow these steps from the node where you added the new server:

    1. Stop the WLS_OSBn managed server.

      To do this, run "kill -9 <pid>" on the PID of the managed server. You can identify the PID of the node using "ps -ef | grep WLS_OSBn".

      Note:

      For Windows, the Managed Server can be terminated by using the taskkill command. For example:

      taskkill /f /pid <pid>

      Where <pid> is the process Id of the Managed Server.

      To determine the process Id of the Managed Server run the following command and identify the pid of the WLS_OSBn Managed Server.

      MW_HOME\jrockit_160_20_D1.0.1-2124\bin\jps -l -v

    2. Watch the Node Manager Console: you should see a message indicating that WLS_OSBn's floating IP has been disabled.

    3. Wait for the Node Manager to try a second restart of WLS_OSBn. Node Manager waits for a fence period of 30 seconds before trying this restart.

    4. Once Node Manager restarts the server, stop it again. Now Node Manager should log a message indicating that the server will not be restarted again locally.

    Note:

    After a server is migrated, to fail it back to its original node/machine, stop the managed server from the Oracle WebLogic Administration Console and then start it again. The appropriate Node Manager will start the managed server on the machine to which it was originally assigned.

5.14.22.2 Scaling Out the Topology (Adding Managed Servers to New Nodes)

When you scale out the topology, you add new managed servers configured with OSB to new nodes.

For scaling out the SOA servers, see the steps in Section 5.13.22.2, "Scaling Out the Topology (Adding Managed Servers to New Nodes)."

Follow the steps in this section for scaling out the topology for the OSB server.

Before performing the steps in this section, check that you meet these requirements:

  • There must be existing nodes running managed servers configured with OSB within the topology.

  • The new node optionally can access the existing home directories for WebLogic Server and OSB installation. (Use the existing installations in shared storage for creating a new WLS_OSB managed server. You do not need to install WebLogic Server or OSB binaries in every new location in this case, but you do need to run pack and unpack to bootstrap the domain configuration in the new node, unless you are scaling the OSB server to machines containing other servers of the same domain (the SOA servers).)

    Note:

    If there is no existing installation in shared storage, installing WebLogic Server, SOA, and OSB in the new nodes is required as described in Section 5.14, "Configuring High Availability for Oracle Service Bus, with SOA Service Infrastructure and Component Service Engines."

    Note:

    When an ORACLE_HOME or WL_HOME is shared by multiple servers in different nodes, Oracle recommends keeping the Oracle Inventory and Middleware home list in those nodes updated for consistency in the installations and application of patches. To update the oraInventory in a node and "attach" an installation in a shared storage to it, use ORACLE_HOME/oui/bin/attachHome.sh. To update the Middleware home list to add or remove a WL_HOME, edit the user_home/bea/beahomelist file. See the following steps.

Follow these steps for scaling out the topology:

  1. On the new node, mount the existing Middleware home, which should include the OSB (and SOA, if homes are shared) installation and the domain directory, and ensure that the new node has access to this directory, just like the rest of the nodes in the domain.

  2. To attach ORACLE_HOME in shared storage to the local Oracle Inventory, execute the following command:

    OSBHOSTn>cd ORACLE_BASE/product/fmw/soa/
    OSBHOSTn>./attachHome.sh -jreLoc ORACLE_BASE/fmw/jrockit_160_20_D1.0.1-2124
    

    To update the Middleware home list, create (or edit, if another WebLogic installation exists in the node) the MW_HOME/bea/beahomelist file and add ORACLE_BASE/product/fmw to it.

  3. Log in to the Oracle WebLogic Administration Console.

  4. Create a new machine for the new node that will be used, and add the machine to the domain.

  5. Update the machine's Node Manager's address to map the IP of the node that is being used for scale out.

  6. Use the Oracle WebLogic Server Administration Console to clone WLS_OSB1 into a new managed server. Name it WLS_OSBn, where n is a number and assign it to the new machine.

    Note:

    These steps assume that you are adding a new server to node n, where no managed server was running previously.
  7. For the listen address, assign the virtual host name to use for this new managed server. If you are planning to use server migration as recommended for this server, this virtual host name will enable it to move to another node. The virtual host name should be different from those in use by other managed servers (may be in the same or different domain) that are running in the nodes used by the OSB/SOA domain.

    Perform these steps to set the managed server listen address:

    1. Log into the Oracle WebLogic Server Administration Console.

    2. In the Change Center, click Lock & Edit.

    3. Expand the Environment node in the Domain Structure window.

    4. Click Servers. The Summary of Servers page is displayed.

    5. Select the managed server whose listen address you want to update in the Names column of the table. The Setting page for that managed server is displayed.

    6. Set the Listen Address to OSBHOSTnVHN1.

    7. Click Save.

    8. Save and activate the changes.

    The changes will not take effect until the managed server is restarted.

  8. Update the cluster address to include the new server:

    1. Select Environment -> Cluster from the Administration Console.

    2. Click on the OSB_Cluster server. The Settings Screen for the OSB_Cluster appears.

    3. In the Change Center, click Lock & Edit.

    4. Add the new server's address and port to the Cluster Address field. For example:

      OSBHOST1VHN1:8011,OSBHOST2VHN1:8011,OSBHOSTNVHN1:8011
      
  9. Create JMS servers and persistent stores for OSB reporting/internal destinations on the new managed server.

    1. Use the Oracle WebLogic Server Administration Console to create a new persistent store for the new WseeJMSServer and name it, for example, OSB_rep_JMSFileStore_N. Specify the path for the store. This should be a directory on shared storage as recommended in Section 5.14.1.3, "Shared Storage Prerequisites."

      Note:

      This directory must exist before the managed server is started or the start operation fails.
      ORACLE_BASE/admin/domain_name/cluster_name/jms/OSB_rep_JMSFileStore _N
      
    2. Create a new JMS Server for OSB, for example, OSB_rep_JMSServer_N. Use the OSB_rep_JMSFileStore_N for this JMSServer. Target the OSB_rep_JMSServer_N Server to the recently created managed server (WLS_OSBn).

    3. Update the SubDeployment targets for the jmsresources OSB JMS Module to include the recently created OSB JMS Server. To do this, expand the Services node and then expand the Messaging node. Choose JMS Modules from the Domain Structure window of the Oracle WebLogic Server Administration Console. The JMS Modules page appears. Click jmsresources (represented as a hyperlink in the Names column of the table). The Settings page for jmsResources appears. Open the SubDeployments tab. The subdeployment module for jmsresources appears.

      Note:

      This subdeployment module name is a random name in the form of wlsbJMSServerXXXXXX resulting from the Configuration Wizard JMS configuration for the first two servers (WLS_OSB1 and WLS_OSB2).

      Click the wlsbJMSServerXXXXXX subdeployment and update the targets to include the new OSB_rep_JMSServer_N server.

  10. Create JMS Servers, persistent stores and destinations for OSB JAX-RPC on the new managed server.

    Note:

    WebLogic Advanced Web Services for JAX-RPC Extension uses regular (non-distributed) destinations to ensure that a locally processed request on a service gets enqueued only to a local member.
    1. Use the Oracle WebLogic Server Administration Console to create a new persistent store for the new WseeJMSServer and name it, for example, Wsee_rpc_JMSFileStore_N. Specify the path for the store. This should be a directory on shared storage, as recommended inSection 5.14.1.3, "Shared Storage Prerequisites":

      Note:

      This directory must exist before the managed server is started or the start operation fails.
      ORACLE_BASE/admin/DOMAIN_NAME/cluster_name/jms/Wsee_rpc_JMSFileStore_N
      
    2. Create a new JMS Server for OSB JAX-RPC, for example, OSB_rpc_JMSServer_N. Use the Wsee_rpc_JMSFileStore_N for this JMSServer. Target the OSB_rpc_JMSServer_N Server to the recently created Managed Server (WLS_OSBn).

    3. Update the WseeJMSModule OSB JMS Module with destinations and the recently created OSB JMS Server. To do this, expand the Services node and then expand the Messaging node. Choose JMS Modules from the Domain Structure window of the Oracle WebLogic Server Administration Console. The JMS Modules page appears. Click WseeJmsModule (represented as a hyperlink in the Names column of the table). The Settings page for WseeJmsModule appears. Follow steps d through v to complete this step.

    4. In the Change Center, click Lock and Edit. Click New.

    5. Select Queue.

    6. Click Next.

    7. Enter DefaultCallbackQueue-WseeJmsServer_auto_n as name for the queue.

    8. Enter weblogic.wsee.DefaultCallbackQueue-WseeJmsServer_auto_n as the JNDI name.

    9. Click Next.

    10. Click Create a New Subdeployment.

    11. Accept the default name.

    12. Click OK.

    13. Select OSB_rpc_JMSServer_n as the target.

    14. Click Finish.

    15. Activate the changes.

    16. Update the local JNDI name for the destination by following steps q though u.

    17. In the Change Center, click Lock & Edit.

    18. In the Settings for the WseeJmsModule page, click on the DefaultCallbackQueue-WseeJmsServer_auto_n destination.

    19. In the general Configuration tab, click on Advanced.

    20. Update the local JNDI name to weblogic.wsee.DefaultCallbackQueue.

    21. Activate the changes.

    22. Repeat steps d through n for the DefaultQueue-WseeJmsServer_auto_n queue, using weblogic.wsee.DefaultQueue-WseeJmsServer_auto_n as the JNDi name and weblogic.wsee.DefaultQueue as the local JNDI name.

  11. Create a new SAF agent and target it to the newly added managed server, as follows:

    In the Oracle WebLogic Server Administration Console, expand Services > Messaging > Store-and-Forward Agents, and add a new SAF agent ReliableWseeSAFAgent_auto_N.

    Select persistent store Wsee_rpc_JMSFileStore_N (persistent store created for OSB JAX-RPC). Target the SAF Agent to the new managed server and activate changes.

  12. If your Oracle Service Bus configuration includes one or more business services that use JMS request/response functionality, then you must also perform the following procedure using the Oracle Service Bus Console after adding the new managed server to the cluster:

    1. In the Change Center, click Create to create a session.

    2. Using the Project Explorer, locate and select a business service that uses JMS request/response. Business services of this type display Messaging Service as their Service Type.

    3. At the bottom of the View Details page, click Edit.

    4. If there is a cluster address in the endpoint URI, add the new server to the cluster address.

    5. On the Edit a Business Service - Summary page, click Save.

    6. Repeat the previous steps for each remaining business service that uses JMS request/response.

    7. In the Change Center, click Activate.

    8. Restart the managed server.

    9. Restart the Administration Server.

    The business services are now configured for operation in the extended domain.

    Note:

    For business services that use a JMS MesageID correlation scheme, you must edit the connection factory settings to add an entry to the table mapping managed servers to queues. For information on how to configure queues and topic destinations, see "JMS Server Targeting" in Oracle Fusion Middleware Configuring and Managing JMS for Oracle WebLogic Server.
  13. If your Oracle Service Bus configuration includes one or more proxy services that use JMS endpoints with cluster addresses, then you must also perform the following procedure using the Oracle Service Bus Console after adding the new managed server to the cluster:

    1. In the Change Center, click Create to create a session.

    2. Using the Project Explorer, locate and select a proxy service that uses JMS endpoints with cluster addresses.

    3. At the bottom of the View Details page, click Edit.

    4. If there is a cluster address in the endpoint URI, add the new server to the cluster address.

    5. On the Edit a Proxy Service - Summary page, click Save.

    6. Repeat the previous steps for each remaining proxy service that uses JMS endpoints with cluster addresses.

    7. In the Change Center, click Activate.

    8. Restart the managed server.

    The proxy services are now configured for operation in the extended domain.

  14. Update the Oracle Service Bus result cache Coherence configuration for the new server:

    1. Log into Oracle WebLogic Server Administration Console. In the Change Center, click Lock & Edit.

    2. In the Domain Structure window, expand the Environment node.

    3. Click Servers. The Summary of Servers page appears.

    4. Click the name of the server (represented as a hyperlink) in the Name column of the table. The settings page for the selected server appears.

    5. Click the Server Start tab.

    6. Enter the following for WLS_OSBn (on a single line, without a carriage returns):

      -DOSB.coherence.localhost=osbhostnvhn1 -DOSB.coherence.localport=7890 
      -DOSB.coherence.wka1= osbhost1vhn1 -DOSB.coherence.wka1.port=7890 
      -DOSB.coherence.wka2= osbhost2vhn1 -DOSB.coherence.wka1.port=7890
      

      Note:

      The above configuration assumes that servers WLS_OSB1 and WLS_OSB2 are running when WLS_OSBn is started. This will enable WLS_OSBn to join the coherence cluster started by either WLS_OSB1 or WLS_OSB2 using the WKA addresses specified. Also, make sure WLS_OSB1 and WLS_OSB2 are started before WLS_OSBn is started when all three servers are restarted. This will ensure WLS_OSBn join the cluster started by one of WLS_OSB1 or WLS_OSB2. If you want to have a configuration where the order in which the servers are started does not matter, you must add the host and port for WLS_OSBn as WKA for WLS_OSB1 and WLS_OSB2, and also add WLS_OSBn as WKA for WLS_OSBn.
    7. Save and activate the changes (this change requires the Oracle Service Bus servers to be restarted to be effective).

  15. Run the pack command on SOAHOST1 to create a template pack as follows:

    SOAHOST1> cd ORACLE_COMMON_HOME/common/bin
     
    SOAHOST1> ./pack.sh -managed=true -domain=MW_HOME/user_projects/domains/soadomain/
    -template=soadomaintemplateScale.jar -template_name=soa_domain_templateScale
    

    Run the following command on SOAHOST1 to copy the template file created to OSBHOSTn:

    SOAHOST1> scp soadomaintemplateScale.jar oracle@OSBHOSTN:/ ORACLE_BASE/product/fmw/soa/common/bin
    

    Run the unpack command on OSBHOSTN to unpack the template in the managed server domain directory as follows:

    OSBHOSTN> cd ORACLE_BASE/product/fmw/soa/common/bin
     
    OSBHOSTN> ./unpack.sh -domain=ORACLE_BASE/product/fmw/user_projects/domains/soadomain/ -template=soadomaintemplateScale.jar
    
  16. Configure TX persistent store for the new server. This should be a location visible from other nodes as indicated in the recommendations about shared storage.

    From the Administration Console, select Server_name > Services tab. Under Default Store, in Directory, enter the path to the folder where you want the default persistent store to store its data files.

  17. Disable host name verification for the new managed server. Before starting and verifying the WLS_OSBn managed server, you must disable host name verification. You can re-enable it after you have configured server certificates for the communication between the Oracle WebLogic Administration Server and the Node Manager in OSBHOSTn. If the source server from which the new one has been cloned had already disabled hostname verification, these steps are not required (the hostname verification setting is propagated to the cloned server).

    To disable host name verification:

    1. In the Oracle Enterprise Manager Console, select Oracle WebLogic Server Administration Console.

    2. Expand the Environment node in the Domain Structure window.

    3. Click Servers. The Summary of Servers page appears.

    4. Select WLS_OSBn in the Names column of the table.

      The Settings page for server appears.

    5. Click the SSL tab.

    6. Click Advanced.

    7. Set Hostname Verification to None.

    8. Click Save.

  18. Start the Node Manager on the new node. To start the Node Manager, use the installation in shared storage from the existing nodes, and start Node Manager by passing the host name of the new node as a parameter as follows:

    OSBHOSTN> WL_HOME/server/bin/startNodeManager <new_node_ip>
    
  19. Start and test the new managed server from the Oracle WebLogic Server Administration Console:

    1. Shut down all the existing managed servers in the cluster.

    2. Ensure that the newly created managed server, WLS_OSBn, is running.Access the application on the newly created managed server (http://vip:port/sbinspection.wsil). The application should be functional.

  20. Configure server migration for the new managed server.

    Note:

    Since this new node is using an existing shared storage installation, the node is already using a Node Manager and an environment configured for server migration that includes netmask, interface, wlsifconfig script superuser privileges. The floating IP for the new OSB Managed Server is already present in the new node.

    Configure server migration following these steps:

    1. Log into the Administration Console.

    2. In the left pane, expand Environment and select Servers.

    3. Select the server (represented as hyperlink) for which you want to configure migration from the Names column of the table. The Setting page for that server appears.

    4. Click the Migration tab.

    5. In the Available field, in the Migration Configuration section, select the machines to which to allow migration and click the right arrow.

      For example, for new managed servers on OSBHOST1, which is already running WLS_OSB1, select OSBHOST2. For new managed servers on OSBHOST2, which is already running WLS_OSB2, select OSBHOST1.

      Note:

      It is recommended to specify the least-loaded machine as the migration target for the new server. The required capacity planning must be completed so that this node has enough available resources to sustain an additional managed server.
    6. Select the Automatic Server Migration Enabled option. This enables the Node Manager to start a failed server on the target node automatically.

    7. Click Save.

    8. Restart the Administration Server, managed servers, and Node Manager.

  21. Test server migration for this new server. Follow these steps from the node where you added the new server:

    1. Abruptly stop the WLS_OSBn managed server.

      To do this, run "kill -9 <pid>" on the PID of the managed server. You can identify the PID of the node using "ps -ef | grep WLS_OSBn".

      Note:

      For Windows, the Managed Server can be terminated by using the taskkill command. For example:

      taskkill /f /pid <pid>

      Where <pid> is the process Id of the Managed Server.

      To determine the process Id of the Managed Server run the following command and identify the pid of the WLS_OSBn Managed Server.

      MW_HOME\jrockit_160_20_D1.0.1-2124\bin\jps -l -v

    2. Watch the Node Manager Console: you should see a message indicating that WLS_OSBn's floating IP has been disabled.

    3. Wait for the Node Manager to try a second restart of WLS_OSBn. Node Manager waits for a fence period of 30 seconds before trying this restart.

    4. Once Node Manager restarts the server, stop it again. Now Node Manager should log a message indicating that the server will not be restarted again locally.

      Note:

      After a server is migrated, to fail it back to its original node/machine, stop the managed server from the Oracle WebLogic Administration Console and then start it again. The appropriate Node Manager will start the managed server on the machine to which it was originally assigned.

5.14.23 Troubleshooting SOA High Availability

In a high availability environment, SOA Composer may show a Java exception. There are two possible exceptions:

java.io.NotSerializableException:

oracle.bpel.ruledictionarydc.model.impl.DFCollHandler

and

java.io.NotSerializableException:

oracle.integration.console.metadata.view.AuthenticationHandler

These exceptions do not cause any functional loss.

To prevent Java exceptions as you use SOA Composer to modify Rules, Domain Value Maps, and other items, Oracle recommends that you save your work regularly.

5.15 Configuring High Availability for Oracle BAM

This section describes how to set up a 2 node Highly Available configuration for BAM. This includes the BAM Server and the BAM Web Applications.

This section includes these topics:

The architecture targeted for the configuration steps is as follows:

Note:

Oracle strongly recommends reading the release notes for any additional installation and deployment considerations prior to starting the setup process.

Note:

The architectures and deployment procedures defined in this guide enable simple clustered deployments. The procedures described in these chapters can be used as a building block to enable this and other similar high availability topologies for these Fusion Middleware components. It is also expected that production deployments will use other required procedures, such as associating security policies with a centralized LDAP server. For complete details of secured, multi-tiered architectures, and deployment procedures, please refer to the Enterprise Deployment Guide for the component you are configuring.

Figure 5-53 represents the example architecture that configuration steps in this section create.

Figure 5-53 Oracle BAM High Availability Architecture

Description of Figure 5-53 follows
Description of "Figure 5-53 Oracle BAM High Availability Architecture "

Figure 5-53 illustrates a two-node cluster running Oracle BAM Web Applications. One of them is also running Oracle BAM Server. The Oracle WebLogic Servers are frontended by Oracle HTTP Servers, which loads balance incoming requests to BAM Web Applications. A load balancer front ends the system and distributes requests from clients to the two HTTP Servers. An Oracle RAC database is used for storing metadata and BAM schemas.

5.15.1 Preparing the Environment: Prerequisite Steps Before Setting up a High Availability Configuration for Oracle BAM

The following sections provide prerequisite steps before setting up an Oracle BAM high availability configuration:

5.15.1.1 Database Prerequisites

Oracle BAM requires the presence of a supported database.

To check if your database is certified or to see all certified databases, refer to the "Certified Databases" section in the Certification Document:

http://www.oracle.com/technology/software/products/ias/files/fusion_certification.html

To determine the database version, execute this query:

SQL>select version from sys.product_component_version where product like 'Oracle%';

5.15.1.2 VIP and IPs Prerequisites

As shown in Table 5-18, you configure the Administration and the BAM Server in BAMHOST1 to listen on two different virtual IPs, as shown in the architecture diagram. This requires the provisioning of the corresponding VIP in the box and related host names in the DNS system in your network. Make sure that the different VIPS are available and are reachable from BAMHOST1 andBAMHOST2 before running the installation.

Table 5-18 Virtual Hosts

Virtual IP VIP Maps to... Description

VIP0

BAMHOST1VHN0

BAMHOST1VHN0 is the virtual host name that is the listen address for the Administration Server and fails over with manual failover of the Administration Server. It is enabled on the node where the Administration Server process is running (BAMHOST1 by default).

VIP1

BAMHOST1VHN1

BAMHOST1VHN1 is the virtual host name that maps to the listen address for WLS_BAM1 and fails over with server migration of this managed server. It is enabled on the node where WLS_BAM1 process is running (BAMHOST1 by default).


5.15.1.3 Installing and Configuring the Database Repository

This section describes how to install and configure the database repository.

Oracle Clusterware

  • For 10g Release 2 (10.2), see the Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide.

  • For 11g Release 1 (11.1) and later, see Oracle Clusterware Installation Guide.

Automatic Storage Management

  • For 10g Release 2 (10.2), see Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide.

  • For 11g Release 1 (11.1) and later, see Oracle Clusterware Installation Guide.

  • When you run the installer, select the Configure Automatic Storage Management option in the Select Configuration page to create a separate Automatic Storage Management home.

Oracle Real Application Clusters

Database Initialization Parameters

Ensure that the following initialization parameter is set to the required value. It is checked by Repository Creation Assistant.

Table 5-19 Required Initialization Parameters

Parameter Required Value Parameter Class

PROCESSES

300 or greater

Static


To check the value of the initialization parameter using SQL*Plus, you can use the SHOW PARAMETER command.

As the SYS user, issue the SHOW PARAMETER command as follows:

SQL> SHOW PARAMETER processes

Set the initialization parameter using the following command:

SQL> ALTER SYSTEM SET processes=300 SCOPE=SPFILE

Restart the database.

Note:

The method that you use to change a parameter's value depends on whether the parameter is static or dynamic, and on whether your database uses a parameter file or a server parameter file. See the Oracle Database Administrator's Guide for details on parameter files, server parameter files, and how to change parameter values.

5.15.1.4 Using Oracle Fusion Middleware Repository Creation Utility to Load Oracle Fusion Middleware Schemas

Install the 11g (11.1.1) Oracle Fusion Middleware Metadata Store and BAM schemas into a Real Application Cluster database before you install Oracle Fusion Middleware BAM components. Oracle Fusion Middleware provides a tool, Oracle Fusion Middleware Repository Creation Utility (RCU), to create the component schemas in an existing database. See Oracle Fusion Middleware Installation Concepts and Repository Creation Utility User's Guide for more information about installing RCU.

5.15.1.4.1 Running RCU

Run RCU to install the required metadata for Oracle Fusion Middleware 11g.

  1. Start up RCU using the following command:

    RCU_HOME/bin/rcu &
    
  2. In the Welcome screen, click Next.

  3. In the Create Repository screen, select Create to load component schemas into a database, click Next.

  4. In the Database Connection Details screen, enter connection information for your database:

    • Database Type: Select Oracle Database.

    • Host Name: Enter the name of the node that is running the database. For an Oracle RAC database, specify the VIP name, or one of the node names as the host name: BAMDBHOST1VIRTUAL.

    • Port: The port number for the database: 1521

    • Service Name: Enter the service name of the database: bamha.mycompany.com

    • Username: SYS

    • Password: Enter the password for the SYS user.

    • Role: SYSDBA

  5. Click Next.

  6. If you receive the following warning message:

    The database you are connecting is with non-UTF8 charset, if you are going to use this DB for multilingual support, you may have data loss. If you are not using for multilingual support you can continue, otherwise we strongly recommend using UTF-8 database.

  7. Click Ignore or Stop.

  8. In the Select Components screen, select Create a New Prefix, and enter a prefix to use for the database schemas, for example, BAMHA

    Write down the schema names so they are available in later procedures.

    • Select the following schemas:

      • AS Common Schemas:

        Metadata Services

      • SOA Infrastructure:

        Business Activity Monitoring

        User Messaging

  9. Click Next.

  10. In the Schema Passwords screen, enter passwords for the main and additional (auxiliary) schema users, and click Next.

  11. In the Map Tablespaces screen, choose the tablespaces for the selected components, and click Next.

  12. In the Summary screen, click Create.

  13. In the Completion Summary screen, click Close.

See Oracle Fusion Middleware Installation Concepts and Repository Creation Utility User's Guide for more information about using RCU.

5.15.1.5 Configuring Virtual Server Names and Ports for the Load Balancer

This section describes the Load Balancer prerequisites for deploying an Oracle BAM high availability environment.

Load Balancers

Oracle BAM uses a hardware load balancer when deployed in a high availability configuration with two Oracle HTT Servers as web tier components. The hardware load balancer should have the following features:

  • Ability to load-balance traffic to a pool of real servers through a virtual host name: Clients access services using the virtual host name (instead of using actual host names). The load balancer can then load balance requests to the servers in the pool.

  • Port translation configuration: The load balancer should have the ability to perform port translation, where it allows incoming requests received on one port to be routed to a server process running on a different port. For example, a request received on port 80 can be routed to port 7777.

  • Monitoring of ports (HTTP/HTTPS)

  • Virtual servers and port configuration: Ability to configure virtual server names and ports on your external load balancer, and the virtual server names and ports must meet the following requirements:

    • The load balancer should allow configuration of multiple virtual servers. For each virtual server, the load balancer should allow configuration of traffic management on more than one port. For example, the load balancer typically needs to be configured with a virtual server and ports for HTTP and HTTPS traffic.

    • The virtual server names must be associated with IP addresses and be part of your DNS. Clients must be able to access the load balancer through the virtual server names.

  • Ability to detect node failures and immediately stop routing traffic to the failed node.

  • Resource monitoring/port monitoring/ process failure detection: The load balancer must be able to detect service and node failures (through notification or some other means) and to stop directing non-Oracle Net traffic to the failed node. If your load balancer has the ability to automatically detect failures, you should use it.

  • Fault-tolerant mode: It is highly recommended that you configure the load balancer to be in fault-tolerant mode.

  • Other: It is highly recommended that you configure the load balancer virtual server to return immediately to the calling client when the back-end services to which it forwards traffic are unavailable. This is preferred over the client disconnecting on its own after a timeout based on the TCP/IP settings on the client machine.

  • Sticky routing capability: Ability to maintain sticky connections to components based on cookies or URL.

  • SSL acceleration: SSL acceleration is a method of offloading the processor-intensive public key encryption algorithms involved in SSL transactions to a hardware accelerator.

    This feature is recommended but not required.

Table 5-20 shows some example virtual server names to use for the external load balancer in the Oracle BAM high availability environment:

Table 5-20 Virtual Server Names and Ports for the BAM External Load Balancer

Component Virtual Server Name

Oracle BAM

bam.mycompany.com

WebLogic Server Administration Console

admin.mycompany.com

Oracle Enterprise Manager Fusion Middleware Control

admin.mycompany.com


Virtual Server Names

This section describes the virtual server names that should be set up for the high availability deployments described in this section:

bam.mycompany.com

bam.mycompany.com is a virtual server name that acts as the access point for all HTTP traffic to the runtime BAM Web components. Traffic to the both SSL and non-SSL ports is configured and typically non-SSL is redirected to SSL. Clients access this service using the address bam.mycompany.com:443. This virtual server is defined on the load balancer.

admin.mycompany.com

This virtual server acts as the access point for all internal HTTP traffic that gets directed to the administration services. The incoming traffic from clients is non-SSL enabled. Thus, the clients access this service using the address admin.mycompany.com:80 and in turn forward these to ports 7777 on WEBHOST1 and WEBHOST2. The services accessed on this virtual host include the WebLogic Administration Server Console and Oracle Enterprise Manager.

In addition, ensure that the virtual server names are associated with IP addresses and are part of your Domain Name System (DNS). The computers on which Oracle Fusion Middleware is running must be able to resolve these virtual server names.

5.15.2 Installing Oracle HTTP Server on WEBHOST1

To install Oracle HTTP Server on WEBHOST1:

  1. Verify that the servers meet the following requirements:

    • The system, patch, kernel, and other requirements meet the requirements specified in the Oracle Fusion Middleware Installation Guide for Oracle SOA Suite.

    • Port 7777 is not used by any service on the nodes. You can verify this by running the following command:

      Unix:

      netstat -an | grep LISTEN | grep ":7777"
      

      Windows:

      netstat -an | findstr "LISTEN" | findstr ":7777"
      

    If the ports are in use, make them available.

  2. On UNIX platforms, if the /etc/oraInst.loc or /var/opt/oracle/oraInst.loc file exists, check that its contents are correct. Specifically, check that the inventory directory is correct, and that you have write permissions for that directory.

    If the /etc/oraInst.loc file does not exist, skip this step.

  3. Start Oracle Universal Installer for Oracle Fusion Middleware 11g Web Tier Utilities CD installation as follows:

    For UNIX, run this command: runInstaller.

    For Windows, double-click setup.exe.

  4. In the Welcome screen, click Next.

  5. In the Select Installation Type screen, select Install and Configure, and click Next.

  6. In the Prerequisite Checks screen, ensure that all the prerequisites are met, and click Next.

  7. In the Specify Installation Location screen, set the location to:

    /u01/app/oracle/product/11.1.1/ohs_1

  8. Click Next.

  9. In the Configure Components screen:

    • Select Oracle HTTP Server.

    • Do not select Associate Selected Components with WebLogic Domain.

    • Click Next.

  10. In the Specify Component Details screen, enter the following values:

    • Instance Home Location: /u01/app/oracle/product/11.1.1/ohs_1/instances/ohs_instance1

    • Instance Name: ohs_instance1

    • OHS Component Name: ohs1

  11. Click Next.

  12. In the Configure Ports screen, do the following:

    • Select Specify Ports using Configuration File and copy the staticports.ini template file from your installation disk (the file is located in the /Disk1/stage/Response directory) to your user's home. Then use the Browse button to select this file.

    • Click View/Edit File to open the staticports.ini file in an editor.

    • Change the Oracle HTTP Server port in that file to 7777.

    • Save the file.

  13. Click Next.

  14. In the Configuration Summary screen, ensure that the selections are correct, and click Install.

  15. In the Installation Progress screen:

    For UNIX systems, a dialog box appears prompting you to run the oracleRoot.sh script. Open a command window and run the script, following the prompts.

    Click Next.

  16. In the Configuration screen, several configuration assistants are launched in succession. When the configuration assistants are finished, the Configuration Completed screen appears.

  17. In the Configuration Completed screen, click Finish to exit.

  18. Repeat the steps for WEBHOST2 and configure your load balancing router with a pool containing both the WEBHOST1 and WEBHOST2 addresses.

5.15.2.1 Validating Oracle HTTP Server

To verify that Oracle HTTP Server is set up correctly, access the root URL context of the server by using the following URL a browser:

HTTP://WEBHOST1:7777/

If Oracle HTTP Server is set up correctly, the Hello World page appears in the browser.

5.15.3 Installing Oracle Fusion Middleware Home

This section describes how to install Oracle Fusion Middleware on all nodes in the application tier that run Oracle WebLogic Server and Oracle Fusion Middleware BAM.

Install the following Oracle Fusion Middleware components:

5.15.3.1 Installing Oracle WebLogic Server

See "Understanding Your Installation Starting Point" in Oracle Fusion Middleware Installation Planning Guide for the version of Oracle WebLogic Server to use with the latest version of Oracle Fusion Middleware.

To install Oracle WebLogic Server on all nodes in the application tier:

  1. Start Oracle WebLogic Server Installer.

  2. In the Welcome screen, click Next.

  3. In the Choose Middleware Home Directory screen:

    • Select Create a New Middleware Home

    • For the Middleware Home Directory field, enter MW_HOME

    • Click Next.

  4. In the Register for Security Updates screen, enter your contact information for security update notifications, and click Next.

  5. In the Choose Install Type screen, select Custom, and click Next.

  6. In the Choose Products and Components screen, click Next.

  7. In the JDK Selection screen, select only Oracle JRockit 160_20_D1.0.1-2124 SDK, and click Next.

  8. In the Choose Product Installation Directories screen, accept the following directory:

    WL_HOME

    Click Next.

  9. In the Installation Summary screen, click Next.

  10. In the Installation Complete screen, deselect Run QuickStart, and click Done.

5.15.3.2 Installing Oracle BAM Using the Oracle Fusion Middleware SOA Suite Installer

Oracle BAM is installed as part of the Oracle Fusion Middleware SOA Suite. The 11g Oracle Fusion Middleware SOA Suite Installer is used to install the required Oracle BAM binaries on all the nodes in the application tier:

  1. On Linux platforms, if the /etc/oraInst.loc file exists, check that its contents are correct. Specifically, check that the inventory directory is correct and that you have write permissions for that directory.

    If the /etc/oraInst.loc file does not exist, you can skip this step.

  2. Start the Oracle FMW 11g SOA Suite Installer:

    On UNIX:

    BAMHOST1> runInstaller
    

    On Windows:

    BAMHOST1> setup.exe
    

    When the Oracle FMW 11g SOA Suite Installer prompts you for a JRE/JDK location enter Oracle SDK location created in Oracle WebLogic Server installation, for example, MW_HOME/jrockit_160_20_D1.0.1-2124.

  3. In the Welcome screen, click Next.

  4. In the Prerequisite Checks screen, verify that the checks complete successfully, and click Next.

  5. In the Specify Installation Location screen:

    • For Middleware Home, enter MW_HOME

    • For Oracle Home Directory, enter soa

    Click Next.

  6. In the Installation Summary screen, click Install.

  7. In the Installation Complete screen, click Finish.

5.15.4 Enabling VIP0 and VIP1 on BAMHOST1

For information about configuring virtual IPs for the Administration Server and configuring the Administration Server for high availability, see Section 12.2.2.3, "Transforming the Administration Server for Cold Failover Cluster."

5.15.5 Running Oracle Fusion Middleware Configuration Wizard on BAMHOST1 to Create the WebLogic Server Oracle BAM Domain

Run Oracle Fusion Middleware Configuration Wizard from the SOA home directory to create a domain containing the Administration Server and Oracle BAM components. Ensure that the database where you installed the repository is running. For Oracle RAC databases, all the instances must be running.

  1. Change the directory to the location of Oracle Fusion Middleware Configuration Wizard, located in the SOA/BAM home directory:

    BAMHOST1> cd ORACLE_HOME/common/bin
    
  2. Start Oracle Fusion Middleware Configuration Wizard

    For UNIX:

    BAMHOST1> ./config.sh
    

    For Windows:

    BAMHOST1> config.cmd
    
  3. In the Welcome screen, select Create a New WebLogic Domain, and click Next.

  4. In the Select Domain Source screen, select Generate a domain configured automatically to support the following products, and select the following products:

    • Oracle Business Activity Monitoring

    • Oracle WSM-PM (selected by default)

    • Enterprise Manager

    • JRF (selected by default)

    Click Next.

  5. In the Specify Domain Name and Location screen, make the following entries:

    • Domain Name: bamdomain

    • Domain Location: accept the default

    • Application Location: accept the default

    Click Next.

  6. In the Configure Administrator Username and Password screen, enter the username and password to be used for the domain's administrator, and click Next.

  7. In the Configure Server Start Mode and JDK screen, make the following selections:

    • WebLogic Domain Startup Mode: select Production Mode

    • JDK Selection: select Oracle SDK1.6.0_05.

    Click Next.

  8. In the Configure JDBC Component Schema screen:

    1. Ensure that the following data source appears on the screen. The user names shown in Table 5-21, assume that bamha was used as the prefix for schema creation from RCU.

      Table 5-21 Configuring Values for Data Sources

      Data Source User Name

      BAMDataSource

      bamha_orabam

      OraSDPMDataSource

      bamha_orasdpm

      mds-owsm

      bamha_mds


    2. Select Configure selected component schemas as RAC multi data source schemas in the next panel.

    3. Click Next.

  9. In the Configure RAC Multi Data Sources screen enter the following:

    1. With all schemas selected enter values for the following fields, specifying the connect information for the Oracle RAC database that was seeded with RCU.

      Driver: Oracle driver (Thin) for RAC Service-Instance connections, Versions:10, 11.

      Service Name: Enter the service name of the database, for example, bamha.mycompany.com.

      Username: Enter the complete user name (including the prefix) for the schemas.

      Password: Enter the password to use to access the schemas.

    2. Enter the host name, instance name, and port.

    3. Click Add.

    4. Repeat for each Oracle RAC instance.

    5. Select the BAM Schema Only and enter the User Name and Password (bamha_orabam/passwd).

    6. Select the User Messaging Service Schema Only and enter the User Name and Password (bamha_orasdpm/passwd)

    7. Select the OWSM MDS Schema Only and enter the User Name and Password (bamha_mds/passwd)

    8. Click Next.

  10. In the Test JDBC Data Sources screen, the connections are tested automatically. The Status column displays the results. Ensure that all connections were successful. If not, click Previous to return to the previous screen and correct your entries.

    Click Next when all the connections are successful.

  11. In the Select Optional Configuration screen, select Administration Server, Managed Servers, Cluster and Machines and Deployment and services, and click Next.

  12. In the Configure the Administration Server screen, enter the following values:

    • Name: AdminServer

    • Listen Address: VIP0

    • Listen Port: 7001

    • SSL listen port: N/A

    • SSL enabled: leave unchecked

    Click Next.

  13. In the Configure Managed Servers screen, add the following managed servers:

    Table 5-22 Configuring Managed Servers

    Name Listen Address Listen Port SSL Listen Port SSL Enabled

    WLS_BAM1

    BAMHOST1VHN1

    8001

    n/a

    No

    WLS_BAM2

    BAMHOST2 (BAM Server WLS_BAM2 does not use server migration)

    8001

    n/a

    No


    Note:

    Although the standard recommendation is to run custom applications and other systems in a separate WebLogic Managed Server, the creation of custom WLS managed servers described in Figure 5-53 is not addressed here.

    Click Next.

  14. In the Configure Clusters screen, add the following cluster:

    • Name: BAM_Cluster

    • Cluster Messaging Mode: unicast

    • Multicast Address: N/A

    • Multicast Port: N/A

    • Cluster Address: Leave empty

    Click Next.

  15. In the Assign Servers to Clusters screen, assign the following servers to BAM_Cluster:

    • WLS_BAM1

    • WLS_BAM2

    Click Next.

  16. In the Configure Machines screen:

    • Delete the LocalMachine that appears by default.

    • Click the Unix Machine tab, and add the following machines:

    Table 5-23 Configuring Machines

    Name Node Manager Listen Address

    BAMHOST1

    BAMHOST1

    BAMHOST2

    BAMHOST2


    Leave all other fields to their default values, and click Next.

  17. In the Assign Servers to Machines screen, assign servers to machines as follows:

    • BAMHOST1: AdminServer, WLS_BAM1

    • BAMHOST2: WLS_BAM2

    Click Next.

  18. In the Target Deployments to Cluster or Services screen, ensure the following targets:

    • User Messaging Deployments should be targeted only to BAM_Cluster. (The usermessaging-xmpp, usermessaging-smpp, and usermessaging-voicexml applications are optional.)

    • wsm-pm should be targeted only to BAM_Cluster.

    • The DMS Application should be targeted to BAM_Cluster and Administration Server.

    • The oracle.rules.*, oracle.sdp.* and oracle.bam.* deployments should be targeted only to BAM_Cluster.

    • The oracle.wsm.seedpolicies library should be targeted only to the BAM_Cluster.

  19. In the Target Services to Cluster or Servers screen, ensure the following targets:

    • WSM Startup Class should be targeted only to BAM_Cluster.

    • mds-wsm, mds-wsm-rac0, and mds-wsm-rac1 should be targeted to both BAM_Cluster and AdminServer.

    • OrasDPMDatasource, OrasDPMDatasource-rac0, and OrasDPMDatasource-rac1 should be targeted to the BAM_Cluster.

    • OWSM Startup Class is only targeted to BAM_Cluster *DMS Startup is targeted both to BAM_Cluster and AdminServer*mds-wsm, mds-wsm-rac0, and mds-wsm-rac1 should be targeted to both BAM_Cluster and AdminServer

    • mds-soa, mds-soa-rac0, and mds-soa-rac1 should be targeted to both BAM_Cluster and AdminServer.

  20. In the Configuration Summary screen, click Create.

  21. In the Creating Domain screen, click Done.

5.15.6 Creating boot.properties for the Administration Server and for WLS_BAM1 on BAMHOST1

This is an optional step for enabling the Administration Server to start up without prompting you for the administrator username and password. Create a boot.properties file for the Administration Server on BAMHOST1.

For the Administration Server:

  1. Create the following directories:

    mkdir ORACLE_BASE/product/fmw/user_projects/domains/bamdomain/servers/
    
    mkdir ORACLE_BASE/product/fmw/user_projects/domains/bamdomain/servers/AdminServer
    
    mkdir ORACLE_BASE/product/fmw/user_projects/domains/bamdomain/servers/AdminServer/security
    
  2. Use a text editor to create a file called boot.properties in the directory created in the previous step, and enter the following lines in the file:

    username=adminuser
    password=password
    

    Note:

    When you start up the Administration Server, the username and password entries in the file are encrypted.

    For security reasons, minimize the time the entries in the file are left unencrypted. After you edit the file, start up the server as soon as possible in order for the entries to be encrypted.

5.15.7 Starting the Administration Server on BAMHOST1

To start the Administration Server on BAMHOST1 run the following commands:

For Linux:

BAMHOST1> cd MW_HOME/user_projects/domains/bamdomain/bin

BAMHOST1> ./startWebLogic.sh

For Windows:

startWebLogic.cmd

To verify that the Administration Server is properly configured:

  1. In a browser, go to http://VIP0:7001/console.

  2. Log in as the administrator.

  3. Verify that the WLS_BAM1 and WLS_BAM2 managed servers are listed.

  4. Verify that the BAM_Cluster cluster is listed.

  5. Verify that you can access Enterprise Manager at http://VIP0:7001/em.

5.15.8 Disabling Host Name Verification for the Servers

This step is required if you have not set up the appropriate certificates to authenticate the different nodes with the Administration Server. If you have not configured the server certificates, you receive errors when managing the different WebLogic Servers. To avoid these errors, disable host name verification while setting up and validating the topology, and enable them again once the high availability topology configuration is complete.

To disable host name verification:

  1. In Oracle WebLogic Server Administration Console, select Administration Server, SSL, and then Advanced.

  2. Set Hostname Verification to None.

  3. In Oracle WebLogic Server Administration Console, select WLS_BAM1, SSL, and then Advanced.

  4. Set Hostname Verification to None.

  5. In Oracle WebLogic Server Administration Console, select WLS_BAM2, SSL, and then Advanced.

  6. Set Hostname Verification to None.

  7. Restart the Administration Server for the changes to take affect.

  8. Save and activate the changes.

5.15.9 Configuring a JMS Persistence Store for BAM UMS

Configure the location for all of the persistence stores as a directory that is visible from both BAMHOST1 and BAMHOST2. This is required in order to enable the resume of transactions when a server is migrated to a different node. By using a shared location for the persistent stores from both nodes, it is guaranteed that no messages are lost should a failover take place.

  1. Log into the Oracle WebLogic Server Administration Console.

  2. In the Domain Structure window, expand the Services node and then click the Persistence Stores node.

    The Summary of Persistence Stores page appears.

  3. Select the a persistent store (represented as a hyperlink) from the Name column of the table.

    The Settings page for the persistence store appears.

  4. In the Configuration tab, enter the location on a persistent storage solution (such as NAS or SAN) that is available to other servers in the cluster in the Directory field. Specifying this location enables pending JMS messages to be sent.

  5. Click Save.

  6. Repeat these steps for all persistent stores.

  7. Save and Activate the changes.

5.15.10 Configuring a Default Persistence Store for Transaction Recovery

Each server has a transaction log that stores information about committed transactions that are coordinated by the server that may not have been completed. The WebLogic Server uses this transaction log for recovery from system crashes or network failures. To leverage the migration capability of the Transaction Recovery Service for the servers within a cluster, store the transaction log in a location accessible to a server and its backup servers.

Note:

Preferably, this location should be a dual-ported SCSI disk or on a Storage Area Network (SAN).

To set the location for the default persistence store:

  1. Log into the Oracle WebLogic Server Administration Console.

  2. In the Domain Structure window, expand the Environment node and then click the Servers node.

    The Summary of Servers page appears.

  3. Click the name of the server (represented as a hyperlink) in Name column of the table.

    The settings page for the selected server appears and defaults to the Configuration tab.

  4. Click the Services tab.

  5. In the Default Store section of the page, enter the path to the folder where the default persistent stores will store its data files.

  6. Save and Activate the changes.

    Note:

    To enable migration of the Transaction Recovery Service, specify a location on a persistent storage solution that is available to other servers in the cluster. This directory must also exist before you restart the server.

5.15.11 Untargeting the BAM Server System from BAMHOST2

Because the BAM server component in BAM is a singleton, you must untarget it from one of the WLS_BAM servers before you configure it for server migration. Otherwise the system would use two active BAM Servers which could cause different data inconsistencies. This way, BAM Web applications run in both BAMHOST1 and BAMHOST2, but BAM Server is initially active only in BAMHOST1.

In this step, you remove the BAM server runtime from WLS_BAM2. To untarget the BAM server artifacts from WLS_BAM2:

  1. Log into the Oracle WebLogic Administration Console at http://BAMHOST1VHN0:7001/console.

  2. In the Domain Structure window, choose Environment and then Servers.

    The Summary of Servers page appears.

  3. Select WLS_BAM2 in Name column of the table.

    The Settings page for WLS_BAM2 appears.

  4. Click the Deployments tab.

  5. Select the oracle-bam application from the Name column of the table.

    The Settings page for the oracle-bam application appears.

  6. Click the Targets tab.

  7. In the Change Center, click Lock & Edit.

  8. Change the targets for the modules as described in Table 5-24.

    Save and Activate the changes

    Note:

    You must target all of these components as described in Table 5-24, as incorrect targeting can prevent the BAM system from starting.

    Table 5-24 oracle-bam Component Target Types

    Component Type Target

    oracle-bam(11.1.1)

    Enterprise Application

    BAM_Cluster

    /oracle/bam

    WEBAPP

    WLS_BAM1

    oracle-bam-adc-ejb.jar

    EJB

    WLS_BAM1

    oracle-bam-ems-ejb.jar

    EJB

    WLS_BAM1

    oracle-bam-eventengine-ejb.jar

    EJB

    WLS_BAM1

    oracle-bam-reportcache-ejb.jar

    EJB

    WLS_BAM1

    oracle-bam-statuslistener-ejb.jar

    EJB

    WLS_BAM1

    OracleBAM

    WEBAPP

    BAM_Cluster

    OracleBAMWS

    WEBAPP

    BAM_Cluster

    sdpmessagingclient-ejb.jar

    EJB

    WLS_BAM1


5.15.12 Propagating the Domain Configuration from BAMHOST1 with pack/unpack Utilities

Follow these steps to propagate the domain configuration to BAMHOST1 using Pack/Unpack utilities:

  1. Run the following pack command on BAMHOST1 to create a template pack:

    BAMHOST1> cd ORACLE_COMMON_HOME/common/bin
    BAMHOST1> ./pack.sh -managed=true
    domain=ORACLE_BASE/product/fmw/user_projects/domains/bamdomain/
    -template=bamdomaintemplate.jar 
    -template_name=bam_domain_template
    
  2. Run the unpack command on BAMHOST2 to unpack the propagated template:

    BAMHOST2> cd ORACLE_COMMON_HOME/common/bin
    BAMHOST2> ./unpack.sh
     -domain=MW_HOME/user_projects/domains/bamdomain/
     -template=bamdomaintemplate.jar
    

5.15.13 Starting Node Manager on BAMHOST1 and BAMHOST2

To start Node Manager on BAMHOST1 and BAMHOST2:

  1. Run the setNMProps.sh script, located in the ORACLE_COMMON_HOME/common/bin directory.

    This script sets the StartScriptEnabled property to true before starting Node Manager, and allows staring servers from the Administration Console (the environment variables required for the BAM Servers are set in the default start script in the domain directory):

    BAMHOST1> cd ORACLE_COMMON_HOME/common/bin
    BAMHOST1> ./setNMProps.sh
    

    Note:

    You must use the StartScriptEnabled property to avoid class loading failures and other problems.
  2. Start Node Manager using the following command:

    BAMHOST1> cd WL_HOME/server/bin
    BAMHOST1> ./startNodeManager.sh
    
  3. Repeat step1 and 2 for Node Manager in BAMHOST2.

5.15.14 Starting the Oracle BAM System

To start the WLS_BAM1 managed server on BAMHOST1:

  1. Start the WLS_BAM1 managed servers:

    1. Log into the Oracle WebLogic Server Administration Console using the following URL:

      http://bamhost1vhn0:7001/console

    2. In the Domain Structure window, expand the Environment node and then select Servers.

      The Summary of Servers page appears.

    3. Click the Control tab.

    4. Select WLS_BAM1 from the Servers column of the table.

    5. Click Start.

  2. Access http://bamhost1vhn1:9001/OracleBAM to verify status of WLS_BAM1.

  3. Start the WLS_BAM2 managed servers:

    1. Log into the Oracle WebLogic Server Administration Console at http://bamhost1vhn0:7001/console.

    2. In the Domain Structure window, expand the Environment node and then select Servers.

      The Summary of Servers page appears.

    3. Click the Control tab.

    4. Select WLS_BAM2 from the Servers column of the table.

    5. Click Start.

  4. Access HTTP://BAMHOST2:9001/OracleBAM to verify status of WLS_BAM2.

If the managed servers fails to start with the following message:

Listener refused the connection with the following error:
ORA-12519, TNS:no appropriate service handler found
The Connection descriptor used by the client was <db_connect_string>

Verify that the PROCESSES initialization parameter for the database is set to a high enough value.

5.15.15 Configuring Oracle RAC Failover for the WLS_BAM Servers

Oracle BAM allows customizing the behavior of a BAM server when Oracle RAC is used as the repository for the BAM schemas and a failure occurs in the database. The properties that allow this customization can be adjusted depending on the application and based on the desired expected behavior for each BAM system. The properties are configured in the Fusion Middleware Control Console System MBean Browser, or in the corresponding Oracle BAM-specific XML configuration file.

If you want to completely disable Oracle BAM's failover to the database in an Oracle RAC configuration set UseDBFailover to false in the Fusion Middleware Control Console System MBean Browser for your BAM server. The default value of this property is true, therefore, failover is provided. You can also increase or decrease the number of retries in the access to the database when there is a database instance failure (for BAM to retry the in flight transactions). To adjust the number of retries change the MaxDBNodeFailoverRetries in the Fusion Middleware Control Console System MBean Browser. The default value for MaxDBNodeFailoverRetries is 5 times. See the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite for more details on the properties that can be configured for Oracle BAM.

5.15.16 Configuring the BAM Web Applications to Use the BAM Server in BAMHOST1

To configure the OracleBamWeb (WLS_BAM1) and OracleBamWeb (WLS_BAM2) applications to use the BAM server in BAMHOST1:

  1. Access Oracle Enterprise Manager Fusion Middleware Control at the following URL

    http://bamhost1vhn0:7001/em
    
  2. Expand BAM in the navigation tree.

  3. Right-click OracleBamWeb(WLS_BAM1).

    • Select BAM Web Properties from the context menu.

      The BAM Web Properties page appears.

    • Define the following properties:

      • Enter WEBHOST:7777 for the application URL.

      • Enter BAMHOST1VHN1 for the server name.

    • Modify the listening port for the BAM server using the Mbean browser. To do this, follow these steps in the order shown:

      • Log onto the Oracle Enterprise Manager Fusion Middleware Control.

      • Expand the domain name in the left navigation tree.

      • Expand the BAM item in the left navigation tree.

      • On the BAM drop down menu at the top right, select Mbean Browser.

      • Navigate to the oracle.bam.web -> Server -> Application -> Config -> BAMWebConfig on the right.

      • In the ServerPort field, replace the "DEFAULT" value with 9001.

  4. Select OracleBamWeb(WLS_BAM2) from the navigation tree and repeat these steps.

5.15.17 Configuring Oracle HTTP Servers for the Administration Server and the WLS_BAMn Managed Servers

To enable Oracle HTTP Server to route to the BAM_Cluster, which contains the WLS_BAMn managed servers, you must set the WebLogicCluster parameter to the list of nodes in the cluster:

  1. On WEBHOST1 and WEBHOST2, add the following lines to the ORACLE_BASE/admin/<instance_name>/config/OHS/<component_name>/mod_wl_ohs.conf file:

    # WSM-PM
    <Location /wsm-pm>
        SetHandler weblogic-handler
        WebLogicCluster bamhost1vhn1:9001,bamhost2:9001
    </Location>
    
    # UMS
    <Location /sdpmessaging/ >
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
     
    # UMS WS
    <Location /ucs/messaging/webservice >
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN1:8001,SOAHOST2VHN1:8001
    </Location>
    
    # BAM Web app
    <Location /OracleBAM>
        SetHandler weblogic-handler
        WebLogicCluster bamhost1vhn1:9001,bamhost2:9001
    </Location>
    
    # BAM Web Services
    <Location /OracleBAMWS>
        SetHandler weblogic-handler
        WebLogicCluster bamhost1vhn1:9001,bamhost2:9001
    </Location>
    

    Make sure the httpd.conf file located in the same directory as the mod_wl_ohs.conf file contains the following lines pointing to your load balancing router addresses:

    NameVirtualHost *:7777
    <VirtualHost *:7777>
        ServerName https://bam.mycompany.com:443
        ServerAdmin you@your.address
        RewriteEngine On
        RewriteOptions inherit
    </VirtualHost>
    
    NameVirtualHost *:7777
    <VirtualHost *:7777>
        ServerName admin.mycompany.com:80
        ServerAdmin you@your.address
        RewriteEngine On
        RewriteOptions inherit
    </VirtualHost>
    

    Note:

    Values such as bam.mycompany.com:443, 7777, admin.mycompany:80, and you@youraddress that are noted in this document serve as examples only. Enter values based on the actual environment.
  2. Restart Oracle HTTP Server on WEBHOST1 and WEBHOST2.

    WEBHOST1> ORACLE_BASE/admin/<instance_name>/bin/opmnctl restartproc ias-component=ohs1
    
    WEBHOST2> ORACLE_BASE/admin/<instance_name>/bin/opmnctl restartproc ias-component=ohs2
    

5.15.18 Validating Access through Oracle HTTP Server

Verify that the BAM servers' status is reported as "Running" in the Admin Console. If the server is shown as "Starting" or "Resuming," wait for the server status to change to "Started." If another status is reported (such as "Admin" or "Failed"), check the server output log files for errors. Verify that you can access these URLs, where 'WEBHOSTN' specifies the name of each Oracle HTTP Server host (for example, WEBHOST1, WEBHOST2):

  • http://WEBHOST1:7777/wsm-pm

  • http://WEBHOST2:7777/wsm-pm

  • http://WEBHOST1:7777/sdpmessaging/userprefs-ui

  • http://WEBHOST2:7777/sdpmessaging/userprefs-ui

  • http://WEBHOST1:7777/OracleBAM

  • http://WEBHOST2:7777/OracleBAM

Verify these URLs also using the load balancing router address:

  • http://bam.mycompany.com:80/wsm-pm

  • http://bam.mycompany.com:80/sdpmessaging/userprefs-ui

  • http://bam.mycompany.com:80/OracleBAM

Verify the following URLs to ensure that routing and failover from the HTTP Server to the BAM cluster is working correctly:

  1. While WLS_BAM2 is running, stop WLS_BAM1 from Oracle WebLogic Server Administration Console.

  2. Access the following URLs and verify the appropriate functionality:

    • http://WEBHOST1:7777/wsm-pm

    • http://WEBHOST1:7777/sdpmessaging/userprefs-ui

    • http://WEBHOST1:7777/OracleBAM

  3. Start WLS_BAM1 from Oracle WebLogic Server Administration Console.

  4. Stop WLS_BAM2.

  5. Access the URLs from step 2 above again and verify the appropriate functionality.

5.15.19 Configuring Server Migration for the WLS_BAM Servers

The high availability architecture for Oracle BAM uses server migration to protect singleton services against failures. The WLS_BAM1 managed server is configured so that it can be restarted on BAMHOST2 in case of failure. For this configuration, WLS_BAM1 listens on a specific floating IP address that is failed over by Oracle WebLogic Server Migration. To configure server migration for the WLS_BAM1 managed servers, follow these steps:

Note:

If server migration was configured previously for SOA, the BAM system can use the same data sources and database schemas. In that case, steps 1 through 5 may not be required, but you must also target the corresponding server-migration/leasing datasources to the BAM Cluster.

5.15.19.1 Setting Up the User and Tablespace for the Server Migration Leasing Table

To create the user and tablespace:

  1. Create a tablespace called leasing. For example, log on to SQL*Plus as the sysdba user and run the following command:

    Example: Log on to SQL*Plus as the sysdba user and run the following command:

    SQL> create tablespace leasing
            logging datafile 'DB_HOME/oradata/orcl/leasing.dbf'
            size 32m autoextend on next 32m maxsize 2048m extent management local;
    
  2. Create a user named leasing and assign to it the leasing tablespace.

    SQL> create user leasing identified by welcome1;
    
    SQL> grant create table to leasing;
    
    SQL> grant create session to leasing;
    
    SQL> alter user leasing default tablespace leasing;
    
    SQL> alter user leasing quota unlimited on LEASING;
    
  3. Create the leasing table using the leasing.ddl script.

    1. Copy the leasing.ddl file located in the WL_HOME/server/db/oracle directory to your database node.

    2. Connect to the database as the leasing user.

    3. Run the leasing.ddl script in SQL*Plus.

      SQL> @copy_location/leasing.ddl;
      

5.15.19.2 Creating a Multi Data Source from the WebLogic Server Administration Console

Create a multi data source for the leasing table from the Oracle WebLogic Server Administration Console:

You create a data source to each of the Oracle RAC database instances during the process of setting up the multi data source, both for these data sources and the global leasing multi data source. When you create a data source:

  • Make sure that this is a non-xa data source

  • The names of the multi data sources are in the format of <MultiDS>-rac0, <MultiDS>-rac1, and so on

  • Use Oracle's Driver (Thin) Version 9.0.1, 9.2.0, 10, 11

  • Data sources do not require support for global transactions. Therefore, do not use any type of distributed transaction emulation/participation algorithm for the data source (do not choose the Supports Global Transactions option, or the Logging Last Resource, Emulate Two-Phase Commit, or One-Phase Commit options of the Supports Global Transactions option), and specify a service name for your database.

  • Target these data sources to the BAM cluster

  • Make sure the datasources' connection pool initial capacity is set to 0. To do this, select Services, JDBC, and then Datasources. In the Datasources screen, click the Datasource Name, then click the Connection Pool tab, and enter 0 in the Initial capacity field.

Creating a Multi Data Source

To create a multi data source, complete these steps:

  1. From the Domain Structure window in the Oracle WebLogic Server Administration Console, expand the Services node, then click Data Sources. The Summary of JDBC Data Sources page appears.

  2. In the Change Center, click Lock & Edit.

  3. Click New, then click Multi Data Sources. The Create a New JDBC Multi Data Source page appears.

  4. Enter leasing as the Name

  5. Enter jdbc/leasing as the JNDI name.

  6. Select Failover as algorithm (default).

  7. Click Next.

  8. Select BAM_Cluster as the target.

  9. Click Next.

  10. Select non-XA driver (the default).

  11. Click Next.

  12. Click Create New Data Source.

  13. Enter leasing-rac0 as name. Enter jdbc/leasing-rac0 as JNDI name. Enter oracle as the database type. For the driver type, enter Oracle Driver (Thin) for RAC server-Instance connection Version 10,11.

    Note:

    When creating the multi data sources for the leasing table, enter names in the format of <MultiDS>-rac0, <MultiDS>-rac1, and so on.
  14. Click Next.

  15. Deselect Supports Global Transactions.

  16. Click Next.

  17. Enter the service name, database name (this is actually the RAC Node instance name, for example: racdb1,racdb2), host port, and password for your leasing schema.

  18. Click Next.

  19. Click Test Configuration and verify the connection works.

  20. Target the data source to the BAM cluster.

  21. Select the data source and add it to the right screen.

  22. Click Create a New Data Source second instance of your Oracle RAC database, target it to BAM_Cluster, repeating the steps for the second instance of your Oracle RAC database.

  23. Add the second data source to your multi data source.

  24. Click Save.

  25. Click Activate Changes.

5.15.19.3 Edit the Node Manager's Properties File

The nodemanager.properties file is located in the WL_HOME/server/bin directory.

Interface=eth0
NetMask=255.255.255.0
UseMACBroadcast=true
  • Interface=eth0

    This property specifies the interface name for the floating IP (eth0, for example, on Linux).

    Note:

    Do not specify the sub interface, such as eth0:1 or eth0:2. This interface is to be used without the :0, or :1. The Node Manager's scripts traverse the different :X enabled IPs to determine which to add or remove. For example, the valid values in Linux environments are eth0, eth1, or, eth2, eth3, ethn, depending on the number of interfaces configured.

    Note:

    For Windows, the Interface should be set to the Network Interface Name. For example: Interface="Local Area Connection".
  • NetMask

    This property specifies the net mask for the interface for the floating IP.

  • UseMACBroadcast

    This property specifies whether or not to use a node's MAC address when sending ARP packets, that is, whether or not to use the -b flag in the arping command.

Perform this configuration in the two nodes BAMHOST1 and BAMHOST2. Verify in the output of Node Manager (the shell where the Node Manager is started) that these properties are in use. Otherwise, problems may occur during migration. The output should be similar to the following:

...
StateCheckInterval=500
Interface=eth0 (Linux) or Interface="Local Area Connection" (Windows)
NetMask=255.255.255.0
...

5.15.19.4 Set Environment and Superuser Privileges for the wlsifconfig.sh Script

To set the environment and superuser privileges for the wlsifconfig.sh script:

Note:

On Windows, the script is named wlsifconfig.cmd and it can be run by users with the administrator privilege.
  1. Ensure that your PATH environment variable includes the files listed in Table 5-25.

    Table 5-25 Files Required to be in the PATH Environment Variable

    File Located in this directory

    wlsifconfig.sh

    MW_HOME/user_projects/domains/bamdomain/bin/server_migration

    wlscontrol.sh

    WL_HOME/common/bin

    nodemanager.domains

    WL_HOME//common


  2. Grant sudo configuration for the wlsifconfig.sh script.

    • Configure sudo to work without a password prompt.

    • For security reasons, sudo should be restricted to the subset of commands required to run the wlsifconfig.sh script. For example, to set the environment and superuser privileges for the wlsifconfig.sh script, complete these steps:

      • Grant sudo privilege to the WebLogic user ('oracle') with no password restriction, and grant execute privilege on the /sbin/ifconfig and /sbin/arping binaries.

      • Make sure the script is executable by the WebLogic user ('oracle'). The following is an example of an entry inside /etc/sudoers granting sudo execution privilege for oracle and also over ifconfig and arping:

        oracle ALL=NOPASSWD: /sbin/ifconfig,/sbin/arping
        

        Note:

        Ask the system administrator for the sudo and system rights as appropriate to this step.

5.15.19.5 Configure Server Migration Targets

Configuring Cluster Migration sets the DataSourceForAutomaticMigration property to true. Follow these steps to configure cluster migration in a cluster:

  1. Log into the Oracle WebLogic Server Administration Console.

  2. In the Domain Structure window, expand Environment and select Clusters.

    The Summary of Clusters page appears.

  3. Click the cluster for which you want to configure migration (BAM_Cluster) in the Name column of the table.

  4. Click the Migration tab.

  5. In the Change Center, click Lock & Edit.

  6. In the Available field, select the machine to which to allow migration and click the right arrow. In this case, select BAMHOST1 and BAMHOST2.

  7. Select the data source to be used for automatic migration. In this case select the leasing data source.

  8. Click Save.

  9. Set the Candidate Machines for Server Migration.

    You must perform this task for WLS_BAM1:

    1. In Domain Structure window of the Oracle WebLogic Server Administration Console, expand Environment and select Servers.

    2. Select the server for which you want to configure migration.

    3. Click the Migration tab.

    4. In the Available field, located in the Migration Configuration section, select the machines to which to allow migration and click the right arrow. Select BAMHOST2 for WLS_BAM1.

    5. Select Automatic Server Migration Enabled.

      This enables the Node Manager to start a failed server on the target node automatically.

    6. Click Save and Activate the changes.

    7. Restart the Administration Server and the WLS_BAM1 server.

5.15.19.6 Test Server Migration

To verify that Server Migration is working properly, follow these steps:

From BAMHOST1:

  1. Force stop the WLS_BAM1 managed server using the following command:

    BAMHOST1> kill -9 <pid>
    

    pid specifies the process ID of the managed server. You can identify the pid in the node by running the following command:

    BAMHOST1> ps -ef | grep WLS_BAM1
    

    Note:

    For Windows, the Managed Server can be terminated by using the taskkill command. For example:

    taskkill /f /pid <pid>

    Where <pid> is the process Id of the Managed Server.

    To determine the process Id of the Managed Server run the following command and identify the pid of the WLS_BAM1 Managed Server.

    MW_HOME\jrockit_160_20_D1.0.1-2124\bin\jps -l -v

  2. In the Node Manager console, a message appears indicating that WLS_BAM1's floating IP has been disabled.

  3. Wait for the Node Manager to try a second restart of WLS_BAM1.

    Node Manager waits for a fence period of thirty seconds before trying this restart.

  4. Once Node Manager restarts the server, stop it again.

    Node Manager should now log a message indicating that the server will not be restarted again locally.

From BAMHOST2:

  1. Watch the local Node Manager console.

    After thirty seconds since the last try to restart WLS_BAM1 on Node 1, Node Manager on BAMHOST2 should prompt that the floating IP for WLS_BAM1 is being brought up and that the server is being restarted in this node.

  2. Access the Oracle BAM console using BAMHOST1VHN1/OracleBAM and WEBHOST1:7777/OracleBAM.

Verification From the Administration Console

You can verify Migration in the Administration Console:

  1. Log into the Administration Console.

  2. Click on Domain on the left console.

  3. Click the Monitoring tab and then on the Migration tab.

The Migration Status table provides information on the status of the migration.

Note:

After a server is migrated, to fail it back to its original node/machine, stop the managed server from the Oracle WebLogic Administration Console and then start it again. The appropriate Node Manager will start the managed server on the machine to which it was originally assigned.

5.15.20 Configuring Clients Connecting to the BAM System

When accessing a BAM server using the BAM Adapter (RMI), the virtual hostname of the BAM server (BAMHOST1VNH1) should be used for the connection. SOAP requests will come through HTTP, therefore it is required to use the load balancing router addresses when using the BAM Adapter in this situation.