5 Configuring High Availability for Oracle Fusion Middleware SOA Suite

Oracle Fusion Middleware SOA Suite provides a complete set of service infrastructure components for designing, deploying, and managing composite applications. Oracle Fusion Middleware SOA Suite enables services to be created, managed, and orchestrated into composite applications and business processes. Composites enable you to easily assemble multiple technology components into one SOA composite application. Oracle Fusion Middleware SOA Suite plugs into heterogeneous IT infrastructures and enables enterprises to incrementally adopt SOA.

This chapter provides a description of Oracle SOA Suite components from a high availability perspective. The sections in this chapter outline the single-instance concepts that are important for designing a high availability deployment.

This chapter includes the following topics:

5.1 Introduction to Oracle Fusion Middleware SOA Suite

As illustrated in Figure 5-1, Oracle SOA Suite provides a comprehensive suite of products for developing, securing, and monitoring service-oriented architecture (SOA). Oracle SOA Suite 11g provides a unified runtime based on the SCA standard. The runtime consists of Service Engines (BPEL Process Manager, Human Workflow, Mediator, Rules) and Binding Components (JCA Adapters, B2B) that are managed and inter-connected by a common Service Infrastructure. Service infrastructure also provides common services such as lifecycle management and deployment.

Figure 5-1 Oracle Fusion Middleware SOA Suite and Components

Single-Node Oracle SOA Service Infrastructure Architecture
Description of "Figure 5-1 Oracle Fusion Middleware SOA Suite and Components"

A SOA composite application is the basic unit of deployment to the SOA runtime. Service components (BPEL process, business rule, human task, and mediator routing rule) are the building blocks of a SOA composite. Components are targeted to Service Engines during deployment while Services and References are enabled using the Binding Components. At runtime, messages are received by the Binding Component and are then routed to the appropriate Service Engine(s) by the Service Infrastructure.

Figure 5-2 Oracle SOA Infrastructure Stack Diagram

Oracle SOA Infrastructure Stack Diagram
Description of "Figure 5-2 Oracle SOA Infrastructure Stack Diagram"

The SOA runtime executes within the context of an application server such as the Oracle WebLogic Application Server. It leverages the underlying application server capabilities for load balancing and high availability.

This guide provides high availability information for the following SOA Suite components:

  • Oracle Service Infrastructure

  • Oracle BPEL Process Manager

  • Oracle Mediator

  • Oracle Human Workflow

  • Oracle JCA Adapters

  • Oracle B2B

  • Oracle Web Service Manager

  • Oracle User Messaging Service

  • Oracle Business Activity Monitoring

5.2 Oracle SOA Service Infrastructure High Availability

Oracle SOA Service Infrastructure is a Java EE application that provides the foundation services for running Oracle Fusion Middleware SOA Suite. This Java EE application is a runtime engine that is automatically deployed when Oracle Fusion Middleware SOA Suite is installed. You deploy composites (the basic artifacts in a Software Component Architecture) to the Oracle SOA Infrastructure and it provides the required services for the composites to run. Oracle SOA Infrastructure provides deployment, wiring, and thread management services for the composites. These services sustain the composite's lifecycle and runtime operations.

The information in this section guides you through the issues and considerations necessary for designing a SOA Service Infrastructure high availability cluster. Later sections of this chapter describe the same issues and considerations for the following SOA Service Infrastructure-related components.

  • Oracle BPEL Process Manager (Oracle BPEL PM)

  • Oracle Mediator

  • Oracle JCA Adapters

  • Oracle Human Workflow

  • Oracle B2B

  • Oracle Web Services Management and Security (Oracle WMS)

  • Oracle User Messaging Service

  • Oracle Business Activity Monitoring

Backup and Recovery Considerations

For general information on backing up SOA Service Infrastructure files, see the "Introducing Backup and Recovery" and "Backing Up Your Environment" chapters of the Oracle Fusion Middleware Administrator's Guide.

5.2.1 Oracle SOA Service Infrastructure Single-Instance Characteristics

The SOA Service Infrastructure is a Java EE-compliant application running on Oracle WebLogic Server. It provides the required services for running composites. A composite is basic unit of deployment for Service Component Architectures (SCA). The SCA Assembly Model is made up of a series of artifacts, which are defined by elements contained in XML files.

Composites are software packages made up of components, wires, services and references. For example, an Oracle BPEL process is a component, an inbound adapter is a service, an outbound adapter is a reference. Wires provide connections between service engines.

Individual components are targeted to specific service engines, such as Oracle BPEL PM, or Oracle Mediator. Oracle SOA Service Infrastructure connects to a SOA database to maintain composite state, and to the SOA Oracle Metadata Services (MDS) Repository to maintain composite metadata, such deployments, and version tracking. These two databases may be the same physical database, but the schemas used for each purpose are different. SOA infra provides a servlet for remote deployment of composites. The metadata and artifacts for remote deployments are also stored in the MDS repository. For more information on the MDS repository, see Section 4.1.2.1, "Configuring Multi Data Sources for MDS Repositories."

Oracle SOA Service Infrastructure application is also responsible for targeting the individual components to their specific engine and for instantiating these composites when requests reach the SOA system. After targeting and instantiation, Oracle SOA Service Infrastructure controls thread and resource assignment. This happens in the JVM, where the composite runs.

As illustrated in Figure 5-3, Oracle SOA Service Infrastructure integrates SOA composite applications with UDDI registries. UDDI registries provide a standards-based foundation for locating published services, invoking services, and managing service metadata. Oracle SOA Service Infrastructure is also the central hub used by the service engines to deliver messages through Oracle User Messaging Service infrastructure to communication channels, such as email and voice.

SOA Service Infrastructure provides the required services that sustain the different pieces in a Service Component Architecture, and allows the communication between them. For more details of the different components in an SCA system, refer to the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite.

Figure 5-3 Basic Single-Node SOA Service Infrastructure Architecture

Single-Node Oracle SOA Service Infrastructure Architecture
Description of "Figure 5-3 Basic Single-Node SOA Service Infrastructure Architecture"

5.2.1.1 Oracle SOA Service Infrastructure Application Characteristics

Oracle SOA Service Infrastructure Services are contained in the soa-infra-wls.ear file. None of the services provided by Oracle SOA Service Infrastructure system are singletons, therefore, Oracle SOA Service Infrastructure can run in full active-active mode. The SOA Service Infrastructure Java EE application contains a Web module that provides browsing of the deployed composites as well as links to the test pages for these composites. This Web module uses /soa-infra as the associated URL context. This Web module is stateless and does not have any specific session replication requirements.

Other modules in Oracle SOA Service Infrastructure application provide task control for process instantiation and process tracking, as well as client services for accessing User Messaging System (UMS).

A task service controls instantiating and tracking processes asynchronously. In addition, there are multiple EJBs used by Oracle SOA Service Infrastructure system. However, all of the EJBs are stateless, and there are no requirements for stateful session bean replication in an Oracle SOA cluster. The processing of transactions by these EJBs relies on Oracle WebLogic Server transaction control service. Configure the appropriate transaction stores as recommended in the basic Oracle WebLogic Server guidelines to guarantee recovery across failures in Oracle WebLogic Server container.

5.2.1.2 Oracle SOA Service Infrastructure Startup and Shutdown Lifecyle

An Oracle SOA composite consists of the following:

  • Components such as a BPEL process, Human Workflow task, a Mediator routing rule or Business Rules.

  • Services and References for connecting Oracle SOA composite applications to external services, applications, and technologies.

These components are assembled together into an Oracle SOA composite application. This application is a single unit of deployment that simplifies the management and lifecycle of Oracle SOA applications.

When Oracle SOA Service Infrastructure application starts, it initializes the different service engines and loads the present composites from the MDS repository. It targets the individual components to their specific engines. Once the composite is loaded, the system is available to receive requests. At runtime, Oracle SOA Service Infrastructure manages all communication across service components. These calls between service engines are in-process calls. The following diagram reflects the sequence for Oracle SOA Service Infrastructure start, and processing of work:

Figure 5-4 Oracle SOA Service Infrastructure Application Startup and Shutdown Lifecyle

Startup and Shutdown Lifecyle
Description of "Figure 5-4 Oracle SOA Service Infrastructure Application Startup and Shutdown Lifecyle"

5.2.1.3 Oracle SOA Service Infrastructure External Dependencies

As described in the previous sections, Oracle SOA Service Infrastructure systems depends on the following components:

  • Instance manager service depends on the runtime SOA database schema (soa-infra).

  • Composite meta-data is stored in the MDS database schema which acts as a repository.

  • In a clustered environment, the deployment coordinator service depends on the underlying Coherence cluster for signal propagation.

All three of these components must be available for Oracle SOA Service Infrastructure to start and run properly.

5.2.1.4 Oracle SOA Service Infrastructure Startup and Shut Down of Processes

Oracle SOA Service Infrastructure application is started by default when ever any Oracle WebLogic Managed Server to which Oracle SOA Service Infrastructure has been deployed is started. Normally, you should not need to stop Oracle SOA Service Infrastructure or any of its components by themselves. Some operations may required Oracle WebLogic Managed Server where the SOA Service Infrastructure runs to be rebooted. Only some patching scenarios could require stopping the application.

You can use Oracle WebLogic Server Administration Console to verify status and to start and stop Oracle WebLogic Server. You can also use WebLogic Server WLST command line to control the application. Oracle Enterprise Manager Fusion Middleware Control also allows multiple operations and configuration of Oracle SOA Service Infrastructure application as well as monitoring its status. See the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite for details on monitoring and controlling Oracle SOA Service Infrastructure application.

Figure 5-5 Monitoring and Controlling Oracle SOA Service Infrastructure Application

Controlling Oracle SOA Service Infrastructure
Description of "Figure 5-5 Monitoring and Controlling Oracle SOA Service Infrastructure Application "

5.2.1.5 Oracle SOA Service Infrastructure Configuration Artifacts

Starting with Oracle Fusion Middleware 11g Release 1 (11.1.1.2), the configuration parameters for Oracle Service Infrastructure are stored in the SOA MDS database. The main controls of Oracle SOA Service Infrastructure can be configured using Oracle Enterprise Manager Fusion Middleware Control.

  • The data source JNDI name for process dehydration.

    Note:

    Once the JNDI names are read from this file, the databases used by the system are determined by the data sources that matched those JNDI names in WebLogic Server JDBS resources configuration.
  • The server and callback URL

    A CallBack URL is the address that asynchronous services specify to be notified of a response to the service they invoked.

    Note:

    If a request to an external or internal asynchronous service originates from SOA, the callback URL is determined using the following in decreasing order of preference:
    • Use callbackServerURL specified as a binding property for the specific reference. (You can set this when modeling the composite or at runtime using the MBeans). This allows different service calls to have different callback URLs. At runtime this property is set using the System MBean Browser, through the corresponding binding Mbean. To add a specific URL add a callbackServerURL property to its Properties attribute, then invoke the save operation.

    • Use the callback URL as specified in the SOA Database. In this case, only one address can be specified and an address that works well for all possible services must be used.

    • Use the callback URL as the frontend host specified in WebLogic Server for the SOA_Cluster. In this case as well, only one address can be specified, and the recommendation is same as for SOA Database configuration option.

    • Use the local host name as provided by WebLogic Server MBean APIs. This is not recommended in high availability environments.

  • The audit level of information to be collected by the message tracking infrastructure.

Other configuration options at the container level, such as data sources, JTA configuration, and persistent stores, are maintained as part of the WebLogic Server Domain configuration, and are synchronized across a cluster of Oracle WebLogic Servers by Oracle WebLogic Server core infrastructure.

5.2.1.6 Oracle SOA Service Infrastructure Log File Locations

The operations performed by Oracle SOA Service Infrastructure and it's components are logged by Oracle WebLogic Managed Server where the SOA Service Infrastructure is running. You can find these logs at the following location:

DOMAIN_HOME/servers/WLS_ServerName/logs/WLS_ServerName.log

The log files for the different Oracle WebLogic Sever managed server are also available from Oracle WebLogic Server Administration Console. To verify the logs, access Oracle WebLogic Server Administration Console using the following URL: admin_server_host:port/console. Click Diagnostics-Log Files.

It is also important to verify the output of Oracle WebLogic Managed Server where Oracle SOA Service Infrastructure is running. This information is stored at the following location:

DOMAIN_HOME/servers/WLS_ServerName/logs/WLS_ServerName.out

Additionally, a diagnostic log is produced in the log directory for the managed server. This log's granularity and logging properties can be changed through the domain_dir/config/fmwconfig/servers/WLS_ServerName/logging.xml file. The properties in this file can also be modified from Oracle Enterprise Manager Fusion Middleware Control by selecting Farm, SOA, SOA Server. Right-click, and select Logs, and then Log Configuration.

Oracle Enterprise Manager Fusion Middleware Control allows performing selective searches in all the logs in the SOA domain. To do a selective search, access Oracle Fusion Middleware Control and click on Farm-Logs and enter the search criteria that pertain to soa-infra or deployed composites. See the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite for more details on the logs and information reported for a SOA System in Oracle Enterprise Manager Fusion Middleware Control.

5.2.2 Oracle SOA Service Infrastructure High Availability Architecture and Failover Considerations

Figure 5-6 illustrates a two-node Oracle SOA Service Infrastructure cluster running on two Oracle WebLogic Servers. Oracle WebLogic Servers are front ended by an Oracle HTTP Server, which load balances incoming requests to the servers.

Note:

The diagram describes high availability with a focus only on Oracle SOA without addressing other entities, such as Oracle HTTP Server and load balancers.

Figure 5-6 Oracle SOA Service Infrastructure High Availability Architecture

Service Infrastructure High Availability
Description of "Figure 5-6 Oracle SOA Service Infrastructure High Availability Architecture"

Figure 5-6 illustrates the following main characteristics of this high availability configuration:

The SOA Service Infrastructure runs in a Oracle WebLogic Server managed servers that are part of an Oracle WebLogic Server cluster. Oracle WebLogic Server cluster synchronizes the configuration for common artifacts of WebLogic Server cluster used by Oracle SOA Service Infrastructure, for example, JTS configuration, data sources, and persistent store definitions.

Oracle SOA Service Infrastructure uses the front end host and port information configured for Oracle WebLogic Server cluster as the server and callback URL. This information is defined using Oracle WebLogic Server Administration Console. To define this information, select Clusters, SOA_Cluster_Name, HTTP/HTTPS frontend host and then Port. If there is no address specified for Oracle WebLogic Server cluster where Oracle SOA Service Infrastructure is running, the system uses the physical host name as the server and callback URL.

For SOA high availability installations frontended by Oracle HTTP Server, monitoring should be done on the Listen ports of Oracle HTTP Server. This is the case when a deployment is using all the components deployed to the SOA Managed Server. A simple HTTP monitor that pings the HTTP/HTTPS port and expects a pre-determined response in turn should suffice. If only a specific SOA component is being used (such as B2B), then a monitor that does a deeper level check all the way to the Managed server can be considered to validate the health of the component in use. Please check with your load balancer vendor on setting up the HTTP monitors with your load balancer.

For more information about the server and callback URLs see the Oracle Fusion Middleware Administrator's Guide. Changing the HTTP Front End address for a cluster requires a restart of the managed servers that are part of the cluster.

The deployment coordinator is configured and used for deployment and updates of composites. The deployment coordinator sends notifications to the members of the deployment coordinator cluster to retrieve new artifacts from the MDS repository, when they are updated by the group leader. A leader node performs singleton operations for the cluster, such as updating the MDS after deployments, or changes are made to the composites.

Oracle SOA Service Infrastructure system uses Oracle WebLogic Server cluster name as its key to confirm a deployment coordinator group. If all nodes in a WebLogic Server cluster can communicate (over multicast or unicast) the deployment coordinator cluster is the same as the WebLogic Server cluster in which the SOA Service Infrastructure runs.

The Administration Server runs in Active-Passive mode. Whenever a failure occurs in APPHOST1, the Administration Server can be restarted in APPHOST2. Therefore it uses a virtual IP or virtual hostname as listen address.

For information about configuring virtual IPs for the Administration Server and configuring the Administration Server for high availability, see Chapter 10, "Transforming the Administration Server for Cold Failover Clusters."

Oracle WebLogic Server Whole Server Migration

Although SOA Service Infrastructure can run in full active-active mode, the architecture uses Oracle WebLogic Server Migration feature to protect some SOA components against failures. This implies that each of the WebLogic Managed Server in which the SOA Service Infrastructure runs is listening on a Virtual IP that is migrated to another box upon failover. The Administration Server and Enterprise Manager run in AdminServer, not the managed server.

As shown in Figure 5-6, WLS_SOA1 listens on VIP1, and WLS_SOA2 listens on VIP2. Each managed server uses the other node as a failover resource, therefore, the system is configured in a cross manner. In other words, WLS_SOA1 fails over to APPHOST2, and WLS_SOA2 fails over to APPHOST1. The appropriate capacity planning must be done to anticipate the scenario where the two SOA managed servers are running on the same node. For more information on Server Migration features, see Chapter 3, High Availability for WebLogic Server in this guide.

To resume transactions after a server migration, configure the transaction and JMS stores in a shared storage. In case of failure in one of the server infrastructure instances, other instances can resume transactions and JMS operation by reading the persistent stores from that shared storage.

The Metadata store is configured in an Oracle RAC database to protect from database failures. Similarly, the SOA process state information is also stored in an Oracle RAC database. In this example, both Oracle RAC databases are the same.

About Oracle SOA Service Infrastructure Components

These high availability characteristics apply to most of Oracle SOA components contained in the composite applications deployed across the cluster. For specific two-node high availability characteristics of the individual components, see the specific component sections the follow in this chapter.

5.2.2.1 Oracle SOA Service Infrastructure Protection from Failures and Expected Behavior

This section describes how an Oracle SOA high availability cluster deployment protects components from failure. This section also describes expected behavior in the event of component failure.

The SOA Service Infrastructure system is protected from all process failures by the WebLogic Server infrastructure.

5.2.2.1.1 WebLogic Server Crash

If a WLS_SOAx server crashes, Node Manager attempts to restart it locally. If repeated restarts fail, the WebLogic Server infrastructure attempts to perform a server migration of the WLS_SOAx server to the other node in the cluster. While the failover takes place, the other SOA Service Infrastructure instance becomes the leader for deployments and composite updates and provides the basic services required by the service engines in the system.

Ongoing requests from the HTTP Server time out and new requests are directed to the other WLS_SOAx server. Once the server's restart completes on the other node, Oracle HTTP Server resumes routing any incoming requests to the server. The migrated server reads MDS repository for any updates that might have taken place during restart, and joins the deployment coordinator cluster to listen for new updates. The migrated server also resumes any pending transactions from the transaction logs in shared storage.

In the server migration scenario, the service engines, such as Oracle BPEL PM and Oracle Mediator, are failed over together with the SOA Service Infrastructure. They do not re-issue any requests to the other SOA Service Infrastructure instances by themselves. They resume operations together with the SOA Service Infrastructure once failover is complete.

Oracle SOA Service Infrastructure application may be down due to failure in accessing resources, errors caused by the deployment coordinator infrastructure, or other issues unrelated to whether the managed server is running. Therefore, Oracle recommends administrators monitoring the soa-infra application for errors caused by the application in the managed server logs. For information about log file locations, see Section 5.2.1.6, "Oracle SOA Service Infrastructure Log File Locations".

5.2.2.1.2 Node Failure

In the event of a node failure, server migration is triggered after the available server verifies the time stamp in the database leasing system. If the failed server was the deployment coordinator cluster master, the available server becomes the new master and the SOA Service Infrastructure remains available for deployment and for composite lifecycle. After the time stamp for leasing is verified, the Node Manager in the node that still remains available attempts to migrate the VIP used by the failed managed server, and restarts it locally. This effectively results in the SOA Service Infrastructure application having two instances running in the same node. For more information on the failover process, see Section 3.9, "Whole Server Migration".

Service engines are deployed to the container as a part of the Service Infrastructure application. These service engines contain all of the ear files and library calls.

5.2.2.1.3 Database Failure

The SOA Service Infrastructure system is protected against failures in the database by using multi data sources. These Multi Data sources are typically configured during the initial set up of the system (Oracle Fusion Middleware Configuration Wizard allows you to define these multi-pools directly at installation time) and guarantee that when an Oracle RAC database instance fails, the connections are reestablished with available database instances. The Multi Data source allows you to configure connections to multiple instances in an Oracle RAC database.

For information about multi data source configuration with Oracle RAC and the MDS repository, see Section 4.1.2, "Using Multi Data Sources with Oracle RAC."

5.2.2.2 Oracle SOA Service Infrastructure Cluster-Wide Deployment

As explained in previous sections, composite deployments are stored centrally by the SOA Service Infrastructure in the MDS repository. Each time the SOA Service Infrastructure is started, it synchronizes itself with the MDS repository and SOA store to get the deployment and process state. The deployment coordinator infrastructure orchestrates the notifications for composites deployments and updates. When a new deployment or update takes place, deployment coordinator notifies all members in the cluster. When all members in the cluster confirm that the deployment has succeeded, the master sends a notification to start the composite. If a deployment fails on any one of the nodes, it is rolled back to the rest of the cluster. An error message in the deployment coordinator master (WebLogic Server managed server), indicates the node on which the deployment failed. Figure 5-7 explains this process:

Figure 5-7 Cluster-Wide Deployment of Oracle SOA Composites

Cluster-Wide Deployment of Oracle SOA Composites
Description of "Figure 5-7 Cluster-Wide Deployment of Oracle SOA Composites"

5.2.2.3 Online Redeployment of Oracle SOA Service Infrastructure Composites in Cluster

When SOA Service Infrastructure performs an update or redeployment of a composite, it can overwrite an existing version or create a new version (v+1). All composites are uniquely identified based on the composite name and revision. Clients accessing a composite use by default, the version that is identified in the MDS repository (also visible from Oracle Enterprise Manager Fusion Middleware Control), as the default version. It is possible to manage the lifecycle of each version separately and perform online redeployments of composites, even with one single instance of SOA Service Infrastructure. The steps for performing this operation would be as follows:

  1. Deploy version x.

  2. Mark version x as default.

  3. Deploy a new version of the composite (x+1) and mark it again version x as default (this is done to elude people accessing the new version in default access).

  4. Test the new version by accessing from test client specifying the version of the composite (x+1)

  5. Once verifications are complete, mark x+1 as the default version

  6. New clients are routed to version x+1, old clients complete their work in version x

It is possible to leave the previous version deployed, or undeploy it. If you undeploy a composite which has in flight instances, they become stale and do not complete. The un-deployment of a composite removes the composite from the MDS repository.

5.2.2.4 Oracle SOA Service Infrastructure Cluster-Wide Configuration Changes

The standard Java EE artifacts that SOA Service Infrastructure uses are configured as part of Oracle WebLogic Domain in which SOA is installed. Oracle WebLogic Clusters provide automatic configuration synchronization for artifacts such as data-sources, persistent stores, and JMS modules, across the WebLogic Server domain. At the same time, the WebLogic Server cluster controls synchronization of the deployments and libraries used by the SOA Service Infrastructure.

As explained in Section 5.2.1, "Oracle SOA Service Infrastructure Single-Instance Characteristics," SOA Service Infrastructure-specific configuration is stored in the SOA database. Changes are applied once (per SOA server), but affect all SOA servers in the same SOA domain. For example, in the high availability topology described in this chapter, if you change the callback URL or audit level for server WLS_SOA1, the change also applies to WLS_SOA2.

5.3 Oracle BPEL Process Manager and High Availability Concepts

The information in this section guides you through the issues and considerations necessary for configuring Oracle BPEL PM for high availability.

5.3.1 Oracle BPEL Process Manager Single-Instance Characteristics

Service engines are containers that host the business logic of service components in a SOA composite application. Each service component, such as Oracle BPEL PM, Oracle Human Workflow, Decision Service, or Oracle Mediator, is executed in its own service engine (Decision Service execute in the business rules service engine). A service engine plugs into Oracle SOA Service Infrastructure. Oracle BPEL Process engine is the service engine running in SOA Service Infrastructure that allows the execution of BPEL Processes.

A BPEL process provides the standard for assembling a set of discrete services into an end-to-end process flow, and developing synchronous and asynchronous services into end-to-end BPEL process flows. It provides process orchestration and storage of long running, asynchronous processes.

As illustrated in Figure 5-8, the BPEL engine is a stateless part of Oracle SOA Service Infrastructure. It is started with the SOA Service Infrastructure application and contains different modules that take care of different aspects required to execute BPEL processes. If a deployed composite contains a BPEL process, SOA Service Infrastructure invokes BPEL Process Manager to get the component from the MetaDataStore (MDS).

BPEL Process Manager uses a Dispatcher Module that maintains an in-memory logical queue containing units of work to process the incoming messages from binding components (JMS, database, Web server).

The BPEL Process Manager engine saves processes' execution state in the SOA database though a persistence module based on Oracle Toplink. The Audit framework Continuously audits the work being processed by storing process execution information in the SOA database.

Figure 5-8 Oracle BPEL PM Single-Instance Architecture

BPEL PM Single-Instance Architecture
Description of "Figure 5-8 Oracle BPEL PM Single-Instance Architecture"

5.3.1.1 BPEL Process Manager Component Characteristics

The BPEL Service Engine runs inside the SOA Service Infrastructure Java EE application (soa-infra.ear). The BPEL Service Engine does not have any singleton services. All state associated with a BPEL process is stored in a database (dehydration store) and there is no in-memory state replication required.

The processing of work by SOA Service Infrastructure Java EE EJBs is transactional and relies on Oracle WebLogic Service transaction control service. You configure the appropriate transaction stores as recommended in the basic WebLogic Server guidelines to guarantee recovery across failures in the WebLogic Server container. Additionally the BPEL engine system does not contain any Web modules, therefore, session replication is not required in the servlet layer when running BPEL in active-active.

In this release of Oracle Fusion Middleware, BPEL Service Container does not rely on JMS for asynchronous message dispatching. Therefore, there is no dependency on a distributed JMS infra-structure.

External Dependencies

The BPEL engine system depends on the following components:

  • SOA Service Infrastructure database for BPEL process state persistence

  • MDS repository for BPEL process metadata store

Both components must be available for the BPEL engine system to start and run properly.

5.3.1.2 Oracle BPEL Process Manager Startup and Shutdown Lifecycle

As illustrated in Figure 5-9, when Oracle SOA Service Infrastructure application starts, it initializes the BPEL engine and loads the composites from the MDS repository. If the composite contains any BPEL processes, it targets those individual components to the BPEL engine. Once the process is loaded, the system is available to receive requests. At runtime, the BPEL engine waits for requests from different channels, such as JMS, the database, and HTTP.

Figure 5-9 Startup and Shutdown Lifecycle of Oracle BPEL PM

Startup and Shutdown Lifecycle
Description of "Figure 5-9 Startup and Shutdown Lifecycle of Oracle BPEL PM "

A detailed startup and shutdown lifecycle is as follows:

  1. Start SOA Server.

  2. Start BPEL Engine.

  3. Composites are loaded from MDS repository by SOA Service Infrastructure.

  4. BPEL components are dispatched to the BPEL engine to be loaded.

  5. Composite-binding components are activated.

  6. The BPEL engine services requests.

  7. The shutdown signal is received by SOA Service Infrastructure.

  8. SOA Service Infrastructure starts undeploying loaded composites.

  9. Composite-binding components are disabled.

  10. BPEL components are dispatched to the BPEL engine to be unloaded.

  11. The BPEL engine shuts down.

5.3.1.3 Oracle BPEL Process Manager Request Flow and Recovery

Recoverable activities are activities that have failed and can be recovered. For example, if you are using the file adapter to initiate an asynchronous BPEL process and your system crashes while the instance is processing, you can manually perform recovery when the server restarts, to ensure that all message records are recovered.

There are 2 types of BPEL Processes based on the invocation interface:

  • One-Way: Most commonly asynchronous fire-and-forget pattern.

  • Two-Way: Synchronous request-response pattern.

The recovery semantics after a server failure are based on whether the process is invoked synchronously and asynchronously. The following describes the behavior based on invocation and process type:

  • Synchronous Invocation (Request): For synchronous requests, an error is thrown back to the client in the case of a server failure. It is the client's responsibility to handle the error message and take appropriate action such as retrying the request. This holds true for both Transient and Durable processes.

    Note:

    In the case of durable processes, the message is persisted to the dehydration store at certain points. It is possible to recover the message from the dehydration store and replay the process using Enterprise Manager, however, the client cannot be notified of the response. Therefore, this is not a recommended option. It is preferable to handle all recovery from the client.
  • Asynchronous Invocation (Post): There are two types of asynchronous invocations - one that starts a new process and one that is a callback to an existing process. In the case of a callback, the engine (after recognizing that it is a continue operation) attempts to resolve the subscribing process, first by conversation ID, and then by a correlation set if defined. Messages associated with asynchronous requests are persisted to the dehydration store as part of the client call. If there is a failure prior to the persistence, the client receives an error message and has to handle the error in the same way as an error for a synchronous invocation. If the persistence is successful, the client call returns and further processing is done outside the context of the client call. In the case of a server failure, one-way processes that were invoked asynchronously can be restarted using Enterprise Manager.

Figure 5-10 Enterprise Manager BPEL Engine Recovery

Enterprise Manager BPEL Engine Recovery
Description of "Figure 5-10 Enterprise Manager BPEL Engine Recovery"

See the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Developers Guide for details on the synchronous and asynchronous models supported by BPEL engine.

5.3.1.4 Oracle BPEL Process Manager Configuration Artifacts

Starting with Oracle Fusion Middleware 11g Release 1 (11.1.1.2), the configuration parameters for Oracle BPEL PM are stored in the SOA database. These parameters are configurable from Oracle Enterprise Manager Fusion Middleware Control.

To configure these parameters, go to the Oracle Enterprise Manager Fusion Middleware Control, and in the navigation tree, select SOA, soa-infra, (server_name). Right-click SOA Administration, and select BPEL Properties, or use the MBean Browser to navigate to the appropriate property.

These properties are specific to each WebLogic domain directory that contains WebLogic Servers running Oracle Fusion Middleware SOA. Other configuration options at the container level, such as data sources, JTA configuration, and persistent stores are maintained as part of the WebLogic Server Domain configuration, and are synchronized across a cluster of WebLogic Servers by the WebLogic Server core infrastructure. See the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite for more details on the configuration for the BPEL engine.

Figure 5-11, Shows Oracle WebLogic Server Administration Console where you can configure Oracle BPEL PM properties

Figure 5-11 Oracle BPEL PM Configuration Properties

Oracle BPEL PM Configuration Properties
Description of "Figure 5-11 Oracle BPEL PM Configuration Properties"

If you are using Oracle SOA Suite in a clustered environment, any configuration property changes you make in Oracle Enterprise Manager on one node must be made on all nodes. Configuration properties are set in Oracle Enterprise Manager through the following options of the SOA Infrastructure menu:

Administration > System MBean Browser

SOA Administration > any property selections.

5.3.2 Oracle BPEL Process Manager High Availability Architecture and Failover Considerations

Figure 5-6 describes a Oracle SOA Service Infrastructure two-node cluster running on two WebLogic Servers. Oracle BPEL PM is deployed as part of Oracle SOA Service Infrastructure.

5.3.2.1 Oracle BPEL Process Manager Protection from Failures and Expected Behavior

For information about Oracle SOA Service Infrastructure protection from failures and expected behavior, see Section 5.2.2.1, "Oracle SOA Service Infrastructure Protection from Failures and Expected Behavior".

The BPEL engine system is protected from all process failures by the WebLogic Server infrastructure. The following process failure considerations apply to Oracle BPEL:

  • If the managed servers crash, Node Manager attempts to restart them locally. If Whole Server Migration is configured and repeated restarts fail, the WebLogic Server infrastructure attempts to perform server migration of the managed server to the other node in the cluster, if it is configured. Once the server on the other node is restarted, Oracle HTTP Server resumes routing any incoming requests to it. The migrated server reads the SOA database, resumes any pending processing, and resume transactions from the transaction logs in shared storage.

  • The BPEL PM Service Engine or the entire SOA Service Infrastructure may be unavailable because of errors related to JDBC data-sources, or Coherence configuration, even after a successful managed server startup. Therefore, it is necessary to monitor SOA Service Infrastructure logs for errors. For the location of server logs, see Section 5.2.1.6, "Oracle SOA Service Infrastructure Log File Locations".

In addition, for the handling of failures in the BPEL Engine itself with the WebLogic Server infrastructure, you can define and perform fault recovery actions on BPEL process faults identified as recoverable in Oracle Enterprise Manager Fusion Middleware Control. The recovery actions you perform on faults are based on actions you defined in your fault recovery policy files for BPEL process service components.

Three types of faults can be displayed in Oracle Enterprise Manager Fusion Middleware Control:

  • Business: Application-specific faults that are generated when there is a problem with the information being processed (for example, a social security number is not found in the database).

  • System: Network and other types of errors, such as a database server being completely unavailable, or a Web service being unreachable.

  • Oracle Web Service Manager (OWSM): Errors on policies attached to SOA composite applications, service components, or binding components.

Fault recovery policies are defined at design time. The fault policies files included with the composites, define the actions to take should a failure occur. For more information on how to define and use Fault recovery, see the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite.

Missing Instances

The inbound payload for one-way invocation requests are stored in the dehydration store and committed with the current global transaction context. If the caller has already started a global transaction, the invocation message is saved when the caller commits the transaction; if no incoming transaction is present, a new transaction is started to commit the invocation message. If a BPEL instance cannot be found in Enterprise Manager, the invocation message may be present in the manual recovery console (the recovery console lists all of the incoming messages that have not been delivered yet). If the invocation message is not found in the recovery console, check the status of the transaction from the caller side.

Transactional issues with endpoints

BPEL asynchronously saves the audit trail for instances that cannot be dehydrated within the global transaction in which the request was started. If any transactional services that are referenced from a BPEL component happen to rollback the transaction, the BPEL instance audit trail should still appear in Enterprise Manager. If there is a problem with the dehydration store, the entire global transaction is rolled back and no audit trail is saved. In this case, a message or activity can be recovered to pick up processing from the last known dehydrated point.

If the dehydration point is an Oracle RAC database with multiple nodes and the Java EE server on which the BPEL engine is hosted loses its JDBC connection to the Oracle RAC node, the BPEL engine attempts to retry the transaction (thereby rolling back whatever changes were made in the current transaction). If a new connection cannot be established to a new Oracle RAC node, the BPEL message or activity can be recovered using the recovery console.

Logging

BPEL engine loggers are set to INFO level by default. When trying to track down a message or instance, you may find it helpful to set loggers to TRACE level to enable more output to the WebLogic Server logs. For dehydration log messages, enable oracle.soa.bpel.engine.data. For thread or dispatcher log messages, enable oracle.soa.bpel.engine.dispatcher. To capture all engine log messages, enable oracle.soa.bpel.engine.

5.3.2.1.1 Recovering Failed BPEL and Mediator Instances

In the case of a server failure, in-flight one-way processes that were invoked asynchronously may require manual recovery. Such processes are marked as Recoverable and can be restarted using Enterprise Manager as shown in Figure 5-12.

Figure 5-12 BPEL PM Instance Recovery using Oracle Enterprise Manager

BPEL PM Instance Recovery using Oracle Enterprise Manager

5.3.2.2 Oracle BPEL Process Manager Cluster-Wide Configuration Changes

The standard Java EE artifacts that BPEL engine uses are configured as part of Oracle WebLogic Domain in which SOA is installed. Oracle WebLogic Clusters provide automatic configuration synchronization for artifacts, such as data-sources, persistent stores, and JMS modules across the WebLogic Server domain.

As explained in the Section 5.3.1, "Oracle BPEL Process Manager Single-Instance Characteristics," BPEL engine-specific configuration is stored in the SOA database. Changes are applied once (per SOA server) but affect all BPEL instances in the same SOA domain. For example, If WLS_SOA1 and WLS_SOA2 are part of the same SOA cluster and the number of Dispatcher threads for BPEL in WLS_SOA1 is changed to 20, WLS_SOA2's dispatcher threads are set to 20 as well.

5.4 Oracle Mediator and High Availability Concepts

The information in this section guides you through the issues and considerations necessary for configuring Oracle Mediator for high availability.

5.4.1 Oracle Mediator Single-Instance Characteristics

Oracle Mediator is a service engine within the Oracle SOA Service Infrastructure. Oracle Mediator provides the framework to mediate between various providers and consumers of services and events. The Mediator service engine runs in-place with the SOA Service Infrastructure Java EE application. The runtime state for execution started by asynchronous interactions or involving parallel routing rules is maintained in the SOA runtime database. For details about administering Oracle Mediator, see the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite.

  • Synchronous and Asynchronous Interactions

    Oracle Mediator provides support for synchronous and asynchronous request-response interactions. In a synchronous interaction, the client making the request remains blocked, awaiting the response. In an asynchronous interaction, the client invokes the service but does not wait for the response. You can specify a time out period for an asynchronous interaction.

  • Sequential and Parallel Message Routing

    Oracle Mediator can route messages to their destinations either sequentially, or in parallel.

    • In a sequential routing, processing of data takes place in one single transaction.

    • In a parallel routing scenario, one transaction is used for en-queueing information, and another one for de-queuing it.

5.4.1.1 Oracle Mediator Component Characteristics

If a composite contains a Mediator component, SOA Service Infrastructure targets the component to the Mediator engine for deployment. None of the services provided by the Mediator engine system are singletons, therefore, Oracle Mediator engines can run in full active-active mode. The processing of messages by the worker threads in Oracle Mediator is transactional and relies on Oracle WebLogic Server transaction control service. Configure the appropriate transaction stores as recommended by WebLogic Server guidelines to guarantee recovery across failures in the WebLogic Server container. Additionally, Oracle Mediator's engine does not contain any stateful Web modules or stateful session beans, therefore you are not required to configure any sort of session replication when running Oracle Mediator in active-active mode. The state of work and work-to-be processed is maintained by Oracle Mediator in the database. Therefore, it is critical that Oracle Mediator's database be highly available. This requires configuring multi data sources for the SOA data source as described in Section 5.11.5, "Running Oracle Fusion Middleware Configuration Wizard on APPHOST1 to Create the SOA Domain". For information about multi data source configuration with RAC and the MDS repository, see Section 4.1.2, "Using Multi Data Sources with Oracle RAC."

External Dependencies

Oracle Mediator depends on the following components:

  • SOA database for Mediator message and message state persistence

  • MDS repository for composite metadata store

Both components must be available for Oracle Mediator to start or run properly.

5.4.1.2 Oracle Mediator Startup and Shutdown Lifecycle

When the Oracle SOA Service Infrastructure application starts, it initializes the Oracle Mediator engine and loads the composites from the MDS repository. If the composite contains any Oracle Mediator components, it targets them to the Oracle Mediator engine. At runtime, Mediator routing rules can be invoked through an inbound binding component or by another service engine. Graceful shutdown of the Mediator engine is initiated by the SOA Service Infrastructure and involves sending signals to in-flight instances and unloading of loaded components.

Figure 5-13 illustrates Oracle Mediator Startup lifecyle.

Figure 5-13 Oracle Mediator Startup Lifecyle

Oracle Mediator Startup Lifecyle
Description of "Figure 5-13 Oracle Mediator Startup Lifecyle"

5.4.1.3 Oracle Mediator Request Flow

The recovery semantics of Oracle Mediator after a server failure are based on client interaction type and routing rule type.

Oracle Mediator provides support for synchronous and asynchronous request-response interactions. The following describes the behavior based on interaction type:

  • Synchronous Interaction: For synchronous interactions, an error is thrown back to the client in the case of a server failure. It is the client's responsibility to handle the error message and take appropriate action such as retrying the request.

  • Asynchronous Interaction: There are two types of asynchronous invocations - one that starts a new routing rule execution and one that is a callback to an existing rule execution. In the case of a callback, the engine attempts to resolve the subscribing instance through a correlation id. If there is a failure in handling the callback, the client receives an error message and has to handle the error appropriately. The client invocation must be transactional in order to guarantee reliable handling of a callback in the case of a server failure.

Oracle Mediator can route messages either sequentially or in parallel. Only messages processed in parallel need manual recovery in the case of a server failure. The following describes the behavior based on routing rule type:

  • Sequential Routing Rule: For sequential routing rule, complete rule is executed in a single transaction. In the case of server failure, Mediator relies on the underlying transaction manager for recovery.

  • Parallel Routing Rule: For parallel routing rule, mediator uses store-and-forward paradigm that involves two separate transactions - one transaction for persisting the message and a second one for executing the routing rule. If there is a failure prior to the persistence, the client receives an error message and has to handle the error in the same way as in a sequential routing rule. If the persistence is successful, the client call returns and further processing is done outside the context of the client call. In the case of a server failure, Mediator will initiate recovery upon server restart after a configurable time interval specified by ContainerIdLeaseRefresh

5.4.1.4 Oracle Mediator Configuration Artifacts

Starting with Oracle Fusion Middleware 11g Release 1 (11.1.1.2), the configuration parameters for Oracle Mediator are stored in the SOA MDS database. You can configure these parameters using Oracle Enterprise Manager Fusion Middleware Control.

To configure these parameters, go to the Oracle Enterprise Manager Fusion Middleware Control, and in the navigation tree, select SOA, soa-infra, (server_name). Right-click SOA Administration, and select Mediator Properties, or use the MBean Browser to navigate to the appropriate property.

5.4.2 Oracle Mediator High Availability Architecture and Failover Considerations

Mediator being an embedded service engine leverages the same high availability architecture as the SOA Service Infrastructure. Figure 5-6 describes a two-node Oracle SOA Service Infrastructure cluster running on two WebLogic Servers. Oracle Mediator is deployed as part of Oracle SOA Service Infrastructure composite application.

In a clustered configuration, Oracle Mediator can run in an active-active mode as there are no singleton services and all state is stored in the SOA runtime database.

5.4.2.1 Oracle Mediator Protection from Failures and Expected Behavior

For information about Oracle SOA Service Infrastructure protection from failures and expected behavior, see Section 5.2.2.1, "Oracle SOA Service Infrastructure Protection from Failures and Expected Behavior."

During the execution of a Parallel Routing Rule, Mediator obtains a logical lock on a batch of messages in the SOA runtime database. This lock contains a reference to the container ID that uniquely identifies the Oracle Mediator engine. The container ID is assigned at startup time. The user can configure the batch size using DeferredMaxRowsRetrieved. A smaller batch size ensures that there are a lower number of locked rows requiring recovery (as explained above) in the case of a server failure.

In case of unplanned outages, you must wait as much time as specified as the ContainerIdLeaseRefresh interval, after restarting the server. This enables the server to complete the instances still in the running state.

In a multi-node cluster environment, if Mediator is used for an asynchronous message exchange pattern, there could be a possibility that the callback will not be handled by Mediator if it arrives before the request has completed. This could happen in a scenario where the request initiated by Mediator to the target service takes longer to complete and callback from the target service arrives before the request completes.

Process Failure

Oracle Mediator is protected from all process failures by the WebLogic Server infrastructure. The following process failure considerations apply to Oracle Mediator:

  • If the managed servers crash, Node Manager attempts to restart them locally. If Whole Server Migration is configured and repeated restarts fail, the WebLogic Server infrastructure attempts perform server migration of the managed server to the other node in the cluster, if it is configured. Once the server on the other node is restarted, Oracle HTTP Server resumes routing any incoming requests to it. The migrated server read the SOA database, resume any pending processing, and resume transactions from the transaction logs in shared storage.

  • The SOA Service Infrastructure application where Mediator Engine runs may be down due to failure in accessing resources, errors caused by the coherence infrastructure, or other issues unrelated to the status of the managed server where it is located. Therefore, monitor Oracle SOA Service Infrastructure application to watch for errors caused by the application in the managed server logs. For the location of server logs, see Section 5.2.1.6, "Oracle SOA Service Infrastructure Log File Locations").

  • During recovery after a server crash, Oracle Mediator attempts to redeliver messages that were partially processed. If the automatic retry fails, Oracle Mediator enqueues the message to the error hospital for manual recover.

Node Failure

In case of a node failure, server migration is triggered after the available server verifies the time stamp in the database leasing system. If the crashed server was the coherence cluster master, the available server become the new master, and the Mediator Engine remains available for processing messages from binding components. After the time stamp for leasing is verified, the Node Manager in the remaining node attempts to migrate the VIP used by the failed managed server, restarts it locally. This effectively result in the Mediator Engine having two instances running in the same node.

Cluster-Wide Configuration Changes

The standard Java EE artifacts that Oracle Mediator engine uses are configured as part of Oracle WebLogic's domain in which SOA is installed. Oracle WebLogic Clusters provide automatic configuration synchronization for artifacts, such as data-sources, persistent stores, and JMS modules across the WebLogic Server domain. At the same time, the WebLogic Server cluster is in charge of synchronizing the deployments, libraries used by Oracle Mediator's engine.

As explained in Section 5.4.1, "Oracle Mediator Single-Instance Characteristics," Oracle Mediator engine-specific configuration is stored in the SOA MDS database. Changes are applied once (per SOA server), but affect all Mediator instances in the same SOA domain. For example, if WLS_SOA1 and WLS_SOA2 are part of the same SOA cluster and the heartbeat period for Mediator in WLS_SOA1 is changed to 20, WLS_SOA2's heartbeat period is also set to 20.

5.4.2.1.1 Recovering Failed Mediator Instances

During recovery from a server failure, under certain circumstances, messages with in-flight routing rules may be placed in the error hospital for manual recovery. These messages can be recovered using Enterprise Manager as shown in Appendix 5.

Figure 5-14 Mediator Instance Recovery using Oracle Enterprise Manager

BPEL PM Instance Recovery using Oracle Enterprise Manager

5.4.2.2 Troubleshooting Oracle Mediator High Availability

To debug Mediator failures, check the database tables to determine which container failed. To identify requests that were in progress when Mediator failed, find the rows that are still locked, and unlock them. You may also view the payload as it is stored as a blob

The poll interval for Mediator Instance Manager should be the same across all the nodes. The timed stamp used is that database time stamp.

The possible stages of messages in the database are:

  • Ready

  • Locked

  • Completed

  • Error

5.5 Oracle Human Workflow and High Availability Concepts

The information in this section guides you through the issues and considerations necessary for configuring Oracle Human Workflow for high availability.

5.5.1 Oracle Human Workflow Single-Instance Characteristics

Oracle Human Workflow is a service engine running in Oracle SOA Service Infrastructure that allows the execution of interactive human driven processes. A human workflow provides the human interaction support such as approve, reject, and reassign actions within a process or outside of any process. The Human Workflow service consists of a number of services that handle various aspects of human interaction with a business process.

All human task metadata is stored and managed in the Metadata Service (MDS) repository. The Human Workflow engine consists of a Service Engine running within the SOA Service Infrastructure and additional Java EE applications for DefaultToDoTaskFlow and Worklist applications. A human workflow leverages an internal JMS queue for notifications related to a human task.

For details about administering Oracle Human Workflow, see the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite.

5.5.1.1 Oracle Human Workflow Startup and Shutdown Lifecycle

The Human Workflow startup consists of two phases, loading of the Java EE applications, and initialization of the service engine. The Java EE applications are loaded as part of the application server startup. The service engine initialization and shutdown is controlled by SOA Service Infrastructure. Post initialization, composites that contain Human Workflow components are targeted to the Human Workflow service engine by SOA Service Infrastructure.

5.5.1.2 Oracle Human Workflow Request Processing

A human workflow can be initiated by an invocation from another SOA engine (such as Oracle BPEL PM). The message is routed to the engine by SOA Service Infrastructure and is persisted by the Workflow engine within its runtime schema. The message becomes available for human actions through the browser based UI after the client transaction that is associated with the invocation is committed. Each update on the message or the runtime state through a user action is a separate transaction.

As soon as Workflow commits its transaction, the control passes back to BPEL which almost instantaneously commits its transaction. Between this window, if the RAC instance goes down, on failover, the message is retried and can cause duplicate tasks.

5.5.1.3 Oracle Human Workflow Configuration Artifacts

Starting with Oracle Fusion Middleware 11g Release 1 (11.1.1.2), the configuration parameters for Oracle Human Workflow are stored in the SOA MDS database. You can configure these parameters using Oracle Enterprise Manager Fusion Middleware Control.

To configure these parameters, go to the Oracle Enterprise Manager Fusion Middleware Control, and in the navigation tree, select SOA, soa-infra, (server_name). Right-click SOA Administration, Workflow Notification/Task Service Properties, or use the MBean Browser to navigate to the appropriate property.

5.5.1.3.1 Managing the URI of the Human Task Service Component Task Details Application

You can add or remove the URI of the task details application used in human workflow.

To manage the URI of the human task service component task details application:

  1. Access this page through one of the following options:

    From the SOA Infrastructure Menu... From the SOA Folder in the Navigator...
    1. Select Home.
    2. Select the Deployed Composites tab.

    3. In the Composite section, select a specific SOA composite application.

    1. Under soa-infra, select a specific SOA composite application.

  2. Select the human task service component in the Component Metrics table.

  3. Click Administration.

    The Administration page shows the URI for the task details application.

    Description of hwf_comp_admin.gif follows
    Description of the illustration hwf_comp_admin.gif

  4. Click the Add icon to specify the following details for the URI:

    • Application name

    • Host name

    • HTTP port

    • HTTPS port (optional)

    • URI

  5. Click Apply.

5.5.2 Oracle Human Workflow High Availability Architecture and Failover Considerations

Figure 5-6 describes a two-node Oracle SOA Service Infrastructure cluster running on two WebLogic Servers. Oracle Human Workflow is deployed as part of Oracle SOA Service Infrastructure composite application.

5.5.2.1 Oracle Human Workflow Protection from Failures and Expected Behavior

Oracle Human Workflow's engine uses transactional EJBs for persistence and JMS queues for user notification. All state is stored in the database and the JMS queue, and there is no in-memory session state to be replicated for recovery. Therefore, Oracle Human Workflow's service engine and the associated Java EE applications are run in an active-active topology on a WebLogic cluster. In the case of a server or hardware crash, whole server migration must be configured for recovering pending transactions and JMS messages stored on the local queue. Notifications are not sent out until the server is restarted.

5.5.2.2 Manual Recovery Required for Human Workflow Task in Rejected MSG Table

The FabricInstanceManager.persistCompositeInstanceBean API (or other InstanceManager APIs) does not have built-in retry logic. When there are transient failures caused by Oracle RAC failover, the WS invocation is rejected directly. This manual recovery of these rejected messages using Oracle Enterprise Manager Fusion Middleware Control.

To manually recover the rejected messages, in the Oracle Enterprise Manager Fusion Middleware Control, select SOA, soa-infra(soa_server1), <composite_name>, Faults and Rejected Messages, select the message, and choose Recover With Options. This causes the workflow request to be retried.

5.5.3 Troubleshooting Oracle Human Workflow High Availability

Human Workflow works like a standard Java EE application with a browser based client. Once a message is persisted to its runtime schema, it can be processed using the workflow UIs. If a server crashes in the middle of the transaction, you must recover the server for successful recovery or rollback. The following loggers are relevant for debugging errors associated with human workflow:

  • oracle.soa.services.common

  • oracle.soa.services.notification

  • oracle.soa.services.workflow

  • oracle.soa.services.workflow.common

  • oracle.soa.services.workflow.evidence

  • oracle.soa.services.workflow.metadata

  • oracle.soa.services.workflow.persistency

  • oracle.soa.services.workflow.query

  • oracle.soa.services.workflow.runtimeconfig

  • oracle.soa.services.workflow.soa

  • oracle.soa.services.workflow.task

  • oracle.soa.services.workflow.task.dispatch

  • oracle.soa.services.workflow.task.routing

5.6 Oracle B2B and High Availability Concepts

The information in this section guides you through the issues and considerations necessary for configuring Oracle B2B for high availability.

5.6.1 Oracle B2B Single-Instance Characteristics

Oracle B2B connects SOA composite applications to external services, applications, and technologies. Oracle B2B offers a multi-protocol gateway that supports industry-recognized B2B standards. Oracle B2B extends Oracle SOA Suite with business protocol standards, such as electronic data interchange (EDI), ebXML, HL7, and RosettaNet. For details about Oracle B2B's functionality, see the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite.

Oracle B2B is a binding component within SOA Service Infrastructure. B2B meta-data consists of trading partner details along with their supported documents and delivery channels which is stored within the MDS repository. The SOA composite has a reference to this meta-data.

Oracle B2B receives messages from different channels. A listener thread logs the messages in a table in the SOA database and sends the corresponding events to a JMS queue. Event handlers listen for events to process the requests in the JMS queues.

Oracle B2B supports the following transport protocols for communication with business entities:

  • HTTP(S)

  • Oracle Advanced Queue

  • Email (SMTP 1.0, IMAP 1.0, POP3)

  • File

  • FTP and SFTP (SSH FTP)

  • TCP/IP

  • JMS

For HTTP protocol, Oracle B2B uses the system's HTTP listener. For FTP and Email, Oracle B2B uses the external FTP and email server configured in the system. Even in high availability mode, only one B2B instance polls for incoming message for FTP, File, and email protocols.

Note:

MLLP & TCP/IP protocols are not supported in a clustered environment.

5.6.1.1 Oracle B2B Component Characteristics

Oracle B2B runs in-place with SOA Service Infrastructure Java EE application. Oracle B2B UI, is a Web application that is deployed as a standalone war file on the same managed server as the SOA Service Infrastructure. Oracle B2B UI application is stateful and stores information in the HTTP Session.

Oracle B2B stores state information within JMS queues and the SOA run-time database. Whole Server Migration is required for automatic JMS message and transaction recovery after a server failure.

Oracle B2B uses JMS intensively. Oracle B2B uses Oracle WebLogic Server Migration feature for protecting Oracle B2B JMS resources against failures. Oracle B2B's engine uses EJBs and Servlets, which are stateless. All state information is persisted in JMS queues and the database.

High availability of B2B depends on the high availability of the run-time database. Hence, the SOA runtime data sources should be configured as multi-data sources for high availability. For information about multi data source configuration with RAC and the MDS repository, see Section 4.1.2, "Using Multi Data Sources with Oracle RAC."

External Dependencies

Oracle B2B depends on the following components:

  • Oracle SOA database for Oracle B2B message and message state persistence

  • MDS repository for Oracle B2B metadata store

  • FTP and email servers if the corresponding adapters are used

The SOA database and the MDS repository must be available for Oracle B2B to start and run properly.

5.6.1.2 Oracle B2B Startup and Shutdown Lifecycle

When Oracle SOA Service Infrastructure application starts, it initializes Oracle B2B's engine. Oracle B2B meta-data deployment should occur before composites are deployed to the SOA Service Infrastructure. Oracle B2B end-points are defined as Channels which are started as part of the engine initialization. Based on the protocol, each configured channel has external dependencies, such as:

  • If FTP is used, the appropriate FTP server must be started.

  • If email is used, the appropriate email server must be started.

  • If HTTP/HTTPS is used, the HTTP server front ending the B2B system must be started

5.6.1.3 Oracle B2B Request Flow

In an Oracle B2B system all processing is done asynchronously. When a request arrives, the corresponding message is inserted in the SOA database. A notification is inserted in a JMS queue as a placeholder for the pending work. The event handler threads listen for events in these queues and process the work sequentially.

Figure 5-15 Oracle B2B Request Flow

Oracle B2B Request Flow
Description of "Figure 5-15 Oracle B2B Request Flow"

5.6.1.4 Oracle B2B Configuration Artifacts

You can enable and disable metrics for Oracle B2B's Server using Oracle Enterprise Manager Fusion Middleware Control. This is the only property directly exposed by Oracle Enterprise Manager Fusion Middleware Control. The configuration for Oracle B2B's Server is maintained in the SOA database and most of it is exposed by Oracle B2B User Interface Application. For details about Oracle B2B configuration, see the Oracle Fusion Middleware User's Guide for Oracle B2B.

5.6.2 Oracle B2B High Availability Architecture and Failover Considerations

Figure 5-6 describes a two-node Oracle SOA Service Infrastructure cluster running on two WebLogic Servers. Oracle B2B is deployed as part of Oracle SOA Service Infrastructure composite application. The following high availability characteristics are specific to an Oracle B2B high availability deployment:

  • Oracle B2B User Interface application runs inside each one of Oracle WebLogic Server servers, and as part of the same cluster as Oracle SOA Service Infrastructure application.

  • Oracle B2B's server maintains the partners, documents, and channels definitions in the SOA database using an Oracle RAC database.

5.6.2.1 Oracle B2B Protection from Failures and Expected Behavior

This section describes how an Oracle B2B high availability cluster deployment protects components from failure. This section also describes expected behavior in the event of component failure.

Oracle B2B UI Failure

Oracle B2B's User Interface application maintains some navigation information in memory. When a failure happens in one of the managed servers running Oracle B2B's User Interface, users' requests are redirected to another active WebLogic Server running the application. Ongoing requests from Oracle HTTP Server time out according the time out setting in Oracle HTTP Server's configuration. Failover requires those users accessing the failed instance to re-login, since the application is not enabled for serializing the session information.

B2B UI in-memory data is non-transactional and some steps may need to be revisited in the case of failover.

For general information about Oracle SOA Service Infrastructure process failure, see Section 5.2.2.1, "Oracle SOA Service Infrastructure Protection from Failures and Expected Behavior"

Node Failure

If a node failure occurs, or if the local oracle WebLogic Server Node Manager reaches the maximum restart tries on the failed managed server, whole server migration is triggered after the available server verifies the time stamp in the database leasing system. The other oracle B2B engine remains available for processing new messages from the different channels. At the same time, the remaining Oracle B2B server should resume the pending operations for singleton channels, such as FTP, email, or file, after the default timeout period is reached in the time stamp the failed node sets in the database (two minutes by default). For Oracle B2B's User Interface application, ongoing requests from Oracle HTTP Server time out according to Oracle HTTP Server configuration.

Database Failure

For information about Oracle B2B database failure, see Section 5.2.2.1, "Oracle SOA Service Infrastructure Protection from Failures and Expected Behavior"

5.6.2.2 Oracle B2B Cluster-Wide Configuration Changes

The standard Java EE artifacts that Oracle B2B engine uses are configured as part of the Oracle WebLogic's domain in which Oracle SOA Service Infrastructure is installed. Oracle WebLogic Clusters provide automatic configuration synchronization for artifacts, such as data-sources, persistent stores, and JMS modules across the WebLogic Server domain. At the same time, the WebLogic Server cluster controls synchronization of deployments and libraries used by Oracle B2B's engine.

As explained in Section 5.6.1, "Oracle B2B Single-Instance Characteristics," all Oracle B2B server-specific configuration is maintained in the database, and configuration changes are applied to all the SOA Servers running in a WebLogic Server domain. Therefore, configuration properties, such as Payload Size, and Outbound Dispatcher Counts are applied cluster-wide, meaning they are used by all instances in the Oracle SOA cluster.

To set up file, FTP, or email transports in a high availability environment, specify a unique name for each instance in b2b.HAInstanceName. If you use ServerName for the value, Oracle B2B retrieves the WebLogic Server name as the HAInstanceName.

For information about setting up file, FTP, or email transports, see Section B.4, Setting Up File, FTP, or Email in an HA Environment, in the Oracle Fusion Middleware User's Guide for Oracle B2B

5.6.2.3 Oracle B2B deployments in a Cluster

Using the command line utility for deploying, purging, or importing metadata in B2B may cause inconsistencies and errors in the B2B system. For B2B, deploy agreements and purge or import metadata ONLY from the GUI available in the B2B console, instead of using the command line utility.

5.6.2.4 Troubleshooting Oracle B2B Active-Active Configuration

This section provides troubleshooting tips and possible resolutions for Oracle B2B active-active configurations.

5.6.2.4.1 Purge, Import, or Deployment of B2B Metadata

When performing purge, import, or deployment of B2B metadata in a cluster, error timing and load balancing may cause exceptions that are unlikely to happen if a retry of the operation is performed.

There is no clean up nor other additional steps required. If errors, such as [java] MDS-02202: Content of the metadata object appear for deployment or "postTransfer: MDS-00521: error while reading document... appear for purge or import, retry the operation.

5.6.2.4.2 Error While Retrieving Oracle B2B Document Definitions

Problem: Error happens when trying to retrieve a document definition XSD from Oracle B2B. B2B resides in a cluster and is accessed through a load balancer. B2B console report the following:

An error occured while loading the document definitions.
 java.lang.IllegalArgumentException: Cluster address must be set when clustering
 is enabled.

Solution: This occurs if you do not set the frontend HTTP host and port for the Oracle WebLogic cluster where Oracle B2B resides. To eliminate this error, set the front end address for the SOA Cluster:

  1. In the WebLogic Server Administration Console, in the Change Center section, click Lock & Edit.

  2. In the left pane, choose the Environment in the Domain Structure window and then choose Clusters. The Summary of Clusters page appears.

  3. Select the WLS_SOA cluster.

  4. Select HTTP.

  5. Set the values for the following:

    • Frontend Host: soa.mycompany.com

    • Frontend HTTPS Port: 443

    • Frontend HTTP Port: 80

  6. Click Save.

  7. To activate the changes, click Activate Changes in the Change Center section of the Administration Console.

  8. Restart the servers to make the Frontend Host directive in the cluster effective.

5.7 Oracle Web Service Manager and High Availability Concepts

The information in this section guides you through the issues and considerations necessary for configuring Oracle WSM for high availability.

5.7.1 Oracle WSM Single-Instance Characteristics

Oracle Web Services Manager (Oracle WSM) provides a policy framework to manage and secure Web services consistently across your organization. It provides capabilities to build, enforce, execute and monitor WebService policies including security, WSRM, MTOM and addressing policies. It typically gets deployed along with SOA Service Infrastructure.

Oracle Web Services Manager is made up of the following components:

  • Policy Manager reads and writes security and management policies including predefined and custom policies from the MDS repository. It is typically deployed on Oracle SOA Service Infrastructure managed servers. However, you can deploy Policy Manager on separate managed servers.

    Policy Manager is a stateless Java EE application. It exposes its capabilities through stateless session beans. Policy Manager does not cache any data, (all accesses go to the MDS repository).

  • Agent is responsible for policy enforcement, execution and gathering of runtime statistics. Agent is available on all Oracle Fusion Middleware managed servers. It is configured on the same server as the application which it protects.

    Agent is made up of a set of jar files, which are a part of underlying WebService stack. It does not have any session state. Agent maintains an in-memory policy cache, which is populated at the Agent startup time. It does not use any JTA or JMS.

    Agent is made up of the following two pieces:.

    • Policy Access Point (PAP) interacts with Policy Manager. Agent communicates with the Policy Manager through EJB invocations over T3 (or T3s if ssl is enabled) protocol.

    • Policy Interceptor is generated when a WebService is deployed and activated, or when a policy is attached to a WebService using Enterprise Manager. If new WebServices are protected using Oracle WSM, an additional instance of the interceptor is generated for each new WebService. Interceptor is responsible for policy enforcement.

  • Metadata Store Policies are stored in the MDS repository. It is typically backed by an Oracle database. For high availability purposes, Oracle recommends using an Oracle RAC database as the back end for MDS repository.

  • Enterprise Manager is used to configure Oracle WSM. It also displays different WebServices metrics gathered by Oracle WSM.

For more details of Oracle WSM architecture, please refer to the Oracle Fusion Middleware Administration Guide Oracle WSM.

Figure 5-16 Oracle WSM Single-Instance Architecture

Oracle WSM Single-Instance Architecture
Description of "Figure 5-16 Oracle WSM Single-Instance Architecture"

5.7.1.1 Oracle WSM Component Characteristics

Oracle WSM Agent is a set of.jar files available on every Oracle Fusion Middleware managed server in a WebServices stack.

Policy Manager is contained in the wsm-pm.ear file. None of the services provided by Oracle WSM are singletons, therefore, it can run in full active-active mode. Oracle WSM services can be validated by http://APPHOSTx:port/wsm-pm/validator. This validator displays Oracle WSM policies.

Agent and Oracle Enterprise Manager interact with Policy Manager using the EJB interfaces. The EJBs used in Oracle WSM are stateless and can be deployed in a clustered environment. Therefore, there is no requirement to enable state replication in the cluster.

The agent and Policy Manager need not be co-located. However, for proper working the agent would expect policy manager to be deployed on at least one node of the domain. The WSM agent has capabilities to auto-discover Policy Managers deployed in the domain.

Neither Agent nor Policy Manager use JTA or JMS messaging. The MDS-based policy store also does not support JTA.

External Dependencies

Oracle WSM Policy Manager depends on the following components:

  • MDS repository for storing the policies

  • Oracle WSM Agent depends only on Oracle WSM Policy Manager.

Both components must be available for Oracle WSM to start and run properly.

5.7.1.2 Oracle WSM Startup and Shutdown Lifecycle

The following key Oracle WSM components are involved in the startup lifecycle for Oracle WSM

  • Oracle WSM Policy Manager

  • Oracle WSM Agent

Policy Manager is a stateless application which does not perform any caching. There is no special application level startup sequence performed when managed server where Policy Manager is deployed starts up. Policy Manager communicates with the MDS repository to retrieve policies. The MDS repository can be stored in an Oracle RAC database to provide MDS high availability.

When a managed server on which an Agent is configured comes up, the Agent connects to Policy Manager to get latest revision of policies. If it succeeds, the changes to the policies are downloaded and cached. Once the Agent is up and running, it periodically attempts a cache refresh at a configurable interval. The default time is every one minute.

Oracle WSM Agent communicates with the Policy Manager through EJB invocations over T3 (or T3s if SSL is enabled) protocol. If Policy Manager is deployed on different nodes and some of them have SSL enabled and others don't, Agent communicates only with the nodes with SSL connections.

If the Policy Manager to which Agent is connected becomes unavailable, the underlying infrastructure automatically connects to another Policy Manager instance running elsewhere in the cluster. This is achieved through Oracle WebLogic EJB clustering.

If a managed server has Web services deployed, which are protected by Oracle WSM, and Agent is not able to communicate with any of the Policy Managers at startup time, Web service invocation fails.

5.7.1.3 Oracle WSM Request Flow

When a protected WebService is accessed by a client application, Agent queries the policy cache and enforces the applicable policies. Based on the policies, the request is authenticated, encrypted, decrypted, authorized or logged. It does not connect to Policy Manager for any of these operations. Runtime availability of Policy Manager does not affect the functioning of Agent, unless there is a configuration change, such as new Webservices, which are protected by Oracle WSM, being deployed, or new policies attached to existing Webservices. If there is such a configuration change, then Agent must connect to Policy Manager to get the applicable policies. If it cannot connect after initial startup, it continues to operate based on the cached policies.

5.7.1.4 Oracle WSM Configuration Artifacts

Oracle WSM Agent configuration resides in policy-accessor-config.xml. This file is located in the DOMAIN_HOME/config/fmwconfig directory. With this configuration file you can specify:

  • Policy Manager URL (if configured)

  • Cache Refresh Interval

  • Clock skew, to allow for difference in system clock of the client and servers

These options are also available from Oracle Enterprise Manager Fusion Middleware Control and are specific to each Oracle WSM Agent installation.

Policy Manager also depends on adf-config.xml for MDS repository location.

Other configuration options at the container level, such as data sources for MDS repository location, and application targeting, are maintained as part of Oracle WebLogic Server Domain configuration, and are synchronized across a cluster of Oracle WebLogic Servers by Oracle WebLogic Server core infrastructure.

5.7.2 Oracle WSM High Availability Architecture and Failover Considerations

Figure 5-6 describes a Oracle SOA Service Infrastructure two-node cluster running on two WebLogic Servers. Oracle WSM is deployed as part of Oracle SOA Service Infrastructure composite application. The following high availability characteristics are specific to an Oracle WSM high availability deployment:

  • Oracle WSM Policy Manager, as well as Oracle WSM Agents are deployed on WLS_SOA_INFRA. In addition, Oracle WSM Agents are available on any custom WLS cluster, in case there is a need to protect any custom WebServices deployed on them.

  • Oracle WSM Policy Manager and Agents runs in a WebLogic Server Infrastructure managed servers, that are part of a WebLogic Server cluster. The WebLogic Server cluster synchronizes configuration for common artifacts of WebLogic Server used by Oracle SOA Service Infrastructure (JDBC).

  • Oracle WSM Policy Manager EJBs leverage clustering and high availability capabilities of the WebLogic Server cluster.

  • All Oracle WSM Policy Manager instances in the cluster point to the same MDS repository.

  • The MDS repository where Oracle WSM policies are stored is configured in an Oracle RAC database to protect from database failure.

5.7.2.1 Oracle WSM Protection from Failures and Expected Behavior

Since OWSM Agent is deployed on the same managed server as the application is deployed, hence it will be available again, as soon as the application becomes available due to server restart/migration. Following two sections describe the failover for Policy Manager.

Process Failure

Oracle WSM components are protected from process failures by the WebLogic Server infrastructure:

  • If the managed servers crash, Node Manager attempts to restart them locally. If Whole Server Migration is configured and repeated restarts fail, the WebLogic Server infrastructure attempts perform server migration of the managed server to the other node in the cluster, if it is configured. Once the server on the other node is restarted, Oracle HTTP Server resumes routing any incoming requests to it. At startup time, Oracle WSM Agent and Policy Manager go through the startup lifecycle as described in Section 5.7.1.2, "Oracle WSM Startup and Shutdown Lifecycle".

  • If the managed server running Policy Manager is restarted or migrated, the failover is transparent to the Agents connected to it. Policy Manager leverages underlying EJB clustering and the failover infrastructure of Oracle WebLogic Server.

Node Failure

If node failure occurs, and Whole Server Migration is not configured, Oracle WSM Agent fails over transparently to the other Policy Manager instance.

If Whole Server Migration is configured, server migration is triggered after the available server verifies the time stamp in the database leasing system. After the time stamp for leasing is verified, the Node Manager in the node that still remains available attempts to migrate the VIP used by the failed managed server, and restarts it locally. This effectively results in Oracle WSM Policy Manager application having two instances running in the same node. Refer to the Section 3.9, "Whole Server Migration" for more details on the failover process. The agents in this situation load balance against both Policy Manager instances running on the same Node.

5.7.2.2 Oracle WSM Cluster-Wide Configuration Changes

The standard Java EE artifacts that Policy Manager uses are configured as part of Oracle WebLogic Domain in which Oracle SOA Service Infrastructure is installed. Oracle WebLogic Clusters provide automatic configuration synchronization for artifacts, such as data-sources, across the WebLogic Server domain. At the same time, the WebLogic Server cluster control synchronizing the deployments and libraries used by different Oracle WSM components.

As explained in the single-instance section, Oracle WSM instance specific aspects are configured individually per WebLogic Server domain directory (typically one per node) through the policy-accessor-config.xml file, as well as the adf-config.xml file. These files are included in the .jar file that is created when using Oracle WebLogic Server Pack utility, therefore it is propagated to other nodes when you run pack/unpack to set up high availability for oracle SOA Service Infrastructure. However, ongoing changes to policy-accessor-config.xml or adf-config.xml are not replicated automatically to the other nodes. This implies that for updates, you must modify each node individually, for example, if you updated the Policy Manager URL or Cache Refresh Interval. These modifications must be manually performed in all the domain directories across a high availability topology. You can edit the policy-accessor -config.xml or adf-config.xml files directly, or use Oracle Enterprise Manager Fusion Middleware Control to make the configuration change on each managed server.

Oracle WSM Agents are capable of automatically discovering the deployed Oracle WSM Policy Manager deployments. If you want to over-ride the behavior and some point to a specific Policy Manager instance, you can do so by editing the policy-accessor-config.xml file.

5.7.2.3 Configuring the Java Object Cache for Oracle WSM

The Java Object Cache (JOC) should be configured among all the servers running Oracle WSM. This local cache is provided to increase the performance of Oracle WSM.

The Java Object Cache can be configured using the MW_HOME/oracle_common/bin/configure-joc.py script. This is a Python script which can be used to configure JOC in the managed servers. The script runs in WLST online mode and expects the Administration Server to be up and running.

When configuring JOC ports for Oracle products, Oracle recommends using ports in the 9988 to 9998 range.

Note:

After configuring the Java Object Cache using the wlst commands or configure-joc.py script, all affected managed servers should be restarted for the configurations to take effect.

Usage

  1. Connect to the Administration Server using the command-line Oracle WebLogic Scripting Tool (WLST), for example:

    MW_HOME/soa/common/bin/wlst.sh
    $ connect()
    

    Enter the Oracle WebLogic Administration user name and password when prompted.

  2. After connecting to the Administration Server using wlst, start the script using the execfile command, for example:

    wls:/mydomain/serverConfig>execfile('MW_HOME/oracle_common/bin/configure-joc.py')
    
  3. Configure JOC for all the managed servers for a given cluster.

    Enter 'y' when the script prompts whether you want to specify a cluster name, and also specify the cluster name and discover port, when prompted. This discovers all the managed servers for the given cluster and configure the JOC. The discover port is common for the entire JOC configuration across the cluster. For example:

    Do you want to specify a cluster name (y/n) <y>
    Enter Cluster Name : WSM_Cluster
    Enter Discover Port : 9991
    

    Here is a walkthrough for using configure-joc.py for HA environments:

    execfile('MW_HOME/oracle_common/bin/configure-joc.py')
    .
    Enter Hostnames (eg host1,host2) : SOAHOST1, SOAHOST2
    .
    Do you want to specify a cluster name (y/n) <y>y
    .
    Enter Cluster Name : WSM_Cluster
    .
    Enter Discover Port : 9991
    .
    Enter Distribute Mode (true|false) <true> : true
    .
    Do you want to exclude any server(s) from JOC configuration (y/n) <n> n
    

The script can also be used to perform the following JOC configurations:

  • Configure JOC for all specified managed servers.

    Enter 'n' when the script prompts whether you want to specify a cluster name, and also specify the managed server and discover port, when prompted. For example:

    Do you want to specify a cluster name (y/n) <y>n
    Enter Managed Server and Discover Port (eg WLS_WSM1:9998, WLS_WSM1:9998) : WLS_WSMM1:9991,WLS_WSM2:9991
    
  • Exclude JOC configuration for some managed servers.

    The script allows you to specify the list of managed servers for which the JOC configuration "DistributeMode" will be set to 'false'. Enter 'y' when the script prompts whether you want to exclude any servers from JOC configuration, and enter the managed server names to be excluded, when prompted. For example:

    Do you want to exclude any server(s) from JOC configuration (y/n) <n>y
    Exclude Managed Server List (eg Server1,Server2) : WLS_WSM1,WLS_WSM3
    
  • Disable the distribution mode for all managed servers.

    The script allows you to disable the distribution to all the managed servers for a specified cluster. Specify 'false' when the script prompts for the distribution mode. By default, the distribution mode is set to 'true'.

Verify JOC configuration using the CacheWatcher utility. See Appendix I, "Running CacheWatcher to verify Java Object Cache."

You can configure the Java Object Cache (JOC) using the HA Power Tools tab in the Oracle WebLogic Administration Console as described in Chapter 13, "Using HA Power Tools."

5.8 Oracle User Messaging Service and High Availability Concepts

The information in this section guides you through the issues and considerations necessary for configuring Oracle User Messaging Service for high availability.

5.8.1 Oracle User Messaging Service Single-Instance Characteristics

Oracle User Messaging Service (Oracle UMS) enables two way communication between users and deployed applications. It has support for a variety of channels, such as email, IM, SMS, and text-to-voice messages. Oracle UMS is integrated with Oracle Fusion Middleware components, such as Oracle BPEL PM, Oracle Human Workflow, Oracle BAM and Oracle WebCenter. It is typically deployed along with Oracle SOA Service Infrastructure.

Oracle UMS is made up of the following components:

  • UMS Server is a Java EE application that runs on Oracle WebLogic Server. The UMS Server orchestrates message flows between applications and users. The server routes outbound messages from a client application to the appropriate driver, and routes inbound messages to the correct client application. The server also maintains a repository of previously sent messages in a persistent store, and correlates delivery status information with previously sent messages.

  • UMS Drivers connect UMS to the messaging gateways, adapting content to the various protocols supported by UMS. Drivers can be deployed or undeployed independently of one another depending on the messaging channels available in a given installation.

    UMS Drivers adapt sent and received content to and from external protocols, such as email, XMPP, and SMPP. UMS Drivers are also Java EE applications deployed to an Oracle WebLogic Server.

  • UMS Client applications implement the business logic of sending and receiving messages. Examples of a UMS client application include an Oracle SOA application that sends messages as one step of an Oracle BPEL PM workflow, or an Oracle WebCenter Spaces application that can send messages from a Web interface.

    UMS client applications have either a UMS-specific EJB module embedded in them, or interact as standard Web service clients.

Figure 5-17 illustrates the services and dependencies of an Oracle UMS single-instance architecture.

Figure 5-17 Oracle User Messaging Service Single-Instance architecture

Oracle User Messaging Service Single-Instance architecture
Description of "Figure 5-17 Oracle User Messaging Service Single-Instance architecture"

5.8.1.1 Oracle User Messaging Service Component Characteristics

This section describes the characteristics of Oracle UMS components.

UMS Server is made up of the following:

  • Message Driven Beans (MDBs) that dequeue messages from JMS queues

  • Stateless Session Beans to implement messaging business logic

  • JAX-WS servlets to implement the messaging Webservices

  • A simple Oracle ADF Faces User Interface component for managing end user messaging preferences.

UMS Drivers typically contain the following:

  • JCA resource adapter (embedded within the EAR) in order to interface with external protocol gateways

  • Two MDBs, one for inbound, and one for outbound, that interface between the resource adapter and the JMS queues

  • Some UMS Drivers that implement an HTTP-based protocol, such as VoiceXML, have a servlet module as well.

UMS Client Applications that use the EJB interface have a stateless session bean that provides the API used by client business logic, and an MDB to asynchronously receive inbound messages.

UMS depends heavily on JMS. JMS queues are also used to buffer content between clients and servers, and between servers and drivers.

Each typical messaging operation (sending an outbound message or receiving an inbound message), involves two JMS queues, one between the client and server, and one between the server and driver.

UMS messaging state is stored in the database and in persistent JMS queues.

External Dependencies

UMS depends on an external database repository to maintain message and configuration state. It shares this state between clustered instances. The UMS Server accesses the database using a Java EE JDBC data source provisioned at installation time. This data source is a non-XA data source. Therefore, UMS does not depend on the JTA capabilities of the underlying infrastructure.

UMS uses JMS to deliver messages among messaging applications. By default it is configured to use a file-based persistent JMS store, therefore it depends on the storage device where those files are located.

5.8.1.2 Oracle User Messaging Service Startup and Shutdown Lifecycle

As Java EE applications, all UMS components are started by Oracle WebLogic Server container.

UMS Server startup

At server startup time, UMS Server initializes a TopLink session that creates a connection to the database repository. UMS Server then begins listening on Web service endpoints. EJBs also become available for invocation of functionality such as sending messages or retrieving delivery status.

UMS Driver startup

When a UMS driver is deployed to a cluster, it sends a registration message to the local UMS server. The UMS server records the registration information in the database, making it available to all UMS servers in the cluster. Once this occurs, any UMS server can route messages to any UMS driver in the cluster. This happens when new drivers are deployed, or when existing drivers are restarted following a configuration change.

At the time of server startup, UMS drivers typically establish a connection to the external gateway for which they are configured. Some of these are persistent socket-level connections, such as SMPP and XMPP. Some connections are established and torn down for each request, such as SMTP and IMAP connections. Drivers then make a remote EJB call to the UMS server to register themselves.

UMS Client Application startup

When the managed server where the clients are deployed starts up, EJB clients typically make a remote EJB call to register themselves with the UMS server. Web service clients do not have explicit registration mechanisms.

5.8.1.3 Oracle User Messaging Service Request Flow

UMS clients make a series of short-lived requests to the UMS server. Web service clients make requests using SOAP or HTTP. EJB clients make remote EJB calls to the server for initialization and configuration management, in addition to interacting using JMS.

After initial startup and registration, subsequent UMS request flows can be categorized as follows:

  • Outbound messaging: For an outbound message, a client application wishes to send a message to a user. The client application either makes a Web service request to the UMS Server, or enqueues a message in the UMS Server's sending JMS queue. The UMS Server receives the request, records the messaging state, and selects an appropriate driver. The UMS Server enqueues the message in a different outbound JMS queue. The corresponding driver, which is listening to the outbound queue, dequeues the message, converts it to the appropriate format for the given protocol, and delivers it to an external gateway (such as an SMTP server or IM gateway.)

  • Inbound messaging: For an inbound message, the end user wishes to send a message back to the client application. The user sends a message from a device, such as an IM client or phone. The message passes from the external gateway to the driver. The driver adapts the message to the UMS format, and enqueues it in a JMS queue. The server dequeues the message from the JMS queue, records the message state, and enqueues the message in the appropriate application's inbound JMS queue. The client application then dequeues the message and processes it as needed.

5.8.1.4 Oracle User Messaging Service Configuration Artifacts

UMS Server and Drivers each have an XML configuration file that is the artifact of a configuration Mbean, implemented using Oracle Fusion Middleware JMX Framework. You can perform configuration changes using Oracle Enterprise Manager Fusion Middleware Control.

Figure 5-18 Configuring Oracle User Messaging Service using Oracle Enterprise Manager Fusion Middleware Control

Configuring Oracle User Messaging Service
Description of "Figure 5-18 Configuring Oracle User Messaging Service using Oracle Enterprise Manager Fusion Middleware Control"

All UMS components use standard Java EE deployment descriptors and Oracle WebLogic proprietary descriptors for configuring Java EE components. These descriptors can be configured using standard Oracle WebLogic tools, such as Oracle WebLogic Server Administration Console or WLST. For more information, see UMS administration topics in Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite.

If you are using Oracle SOA Suite in a clustered environment, any configuration property changes you make in Oracle Enterprise Manager on one node must be made on all nodes. Configuration properties are set in Oracle Enterprise Manager through the following options of the SOA Infrastructure menu:

Administration > System MBean Browser

SOA Administration > any property selections.

5.8.2 Oracle User Messaging Service High Availability Architecture and Failover Considerations

See section Section 5.2.2, "Oracle SOA Service Infrastructure High Availability Architecture and Failover Considerations" for a description of Oracle SOA Service Infrastructure two-node high availability characteristics. Figure 5-6 illustrates an Oracle SOA Service Infrastructure cluster running on two WebLogic Servers. Oracle User Messaging Service is deployed as part of Oracle SOA Service Infrastructure composite application. The following high availability characteristics are specific to an Oracle User Messaging Service high availability deployment:

  • All UMS components are deployed to Oracle WebLogic Service Infrastructure managed servers that are part of an Oracle WebLogic Server cluster. Oracle WebLogic Server cluster synchronizes configuration for common artifacts of Oracle WebLogic Server used by UMS, such as JDBC data sources.

  • All Oracle UMS components are stateless.

  • UMS Server and Client stateless EJBs leverage clustering and high availability capabilities of Oracle WebLogic Server cluster

  • UMS Server relies on a shared database repository for persistent storage.

  • UMS relies on JMS distributed destinations for load-balancing and availability across cluster nodes. It also relies on the JMS connection factory's capability to failover to a different JMS server in the event of a failure.

  • The user messaging preferences user interface does not require session stickiness. It remains available through the use of a basic load balancing. There are no sticky session routing requirements, as all session state is persisted in the database and shared across the clustered instances.

  • UMS does not participate in any global transactions.

  • UMS uses Oracle WebLogic Server multi data source to connect to the back-end Oracle RAC database.

5.8.2.1 Oracle User Messaging Service Protection from Failures and Expected Behavior

Oracle UMS is typically deployed on the same managed server as Oracle SOA Service Infrastructure. Oracle SOA Service Infrastructure is protected from process and node failures using Oracle WebLogic Server Whole Server Migration. Whole Server Migration also provides failover capabilities for JMS usage.

For information about Oracle UMS protection from failures and expected behavior, see Section 5.2.2.1, "Oracle SOA Service Infrastructure Protection from Failures and Expected Behavior".

Process Failure

This section describes specific considerations for process failure in an Oracle User Messaging Service high availability configuration:

  • Node Manager attempts to restart the managed servers locally if a crash occurs.

  • If Whole Server Migration is configured for Oracle WebLogic Server managed server to which Oracle UMS components are deployed, and the restart count threshold is exceeded, Oracle WebLogic Server infrastructure attempts to perform a server migration of the managed server to another node in the cluster. After the server migration completes successfully, at the startup time, UMS Server and Drivers go through the startup cycle as previously described, including driver registration. Driver registration is an independent operation and does not have any affect on other available instances. See Chapter 3, "High Availability for WebLogic Server" for more information about server migration.

  • At restart, (on the same or different node), the UMS JMS server in the managed server starts producing and consuming messages from its JMS store.

  • If a managed server running UMS Server and Drivers is restarted or migrated, the failover is transparent to the connected UMS Clients. The failover is transparent because UMS components are stateless. Once the server's restart is finished, the Web server starts routing requests to it for Web service clients. Similarly, EJB clients become aware of the server availability and start routing requests to it. This is made possible by Oracle WebLogic clustering infrastructure.

Node Failure

For information about Oracle UMS node failure, see Section 5.2.2.1, "Oracle SOA Service Infrastructure Protection from Failures and Expected Behavior"

Database Failure

For information about Oracle UMS database failure, see Section 5.2.2.1, "Oracle SOA Service Infrastructure Protection from Failures and Expected Behavior"

Protection From External Messaging Gateway Failures

Before attempting a message delivery, UMS first persists the message to the database. If an external Messaging Gateway becomes unavailable, the corresponding UMS driver periodically attempts to reconnect to the gateway and deliver any undelivered messages persisted in the database. Alternatively, if the messages are not delivered, administrators can manually resend the messages using the UMS server's Message Status Page in Oracle Fusion Middleware Enterprise Manager.

5.8.2.2 Oracle User Messaging Service Cluster-Wide Configuration Changes

UMS configuration is file-based, standard Java EE deployment descriptors and JMX configuration Mbeans, using a standard JMX framework. Changes are propagated using standard Oracle WebLogic Server Mbean server mechanisms. There are no cluster-wide configuration capabilities. As a result, configuration changes must be repeated on every member of a cluster

UMS uses standard Java EE artifacts, configured as part of Oracle WebLogic's domain in which Oracle UMS is installed. Oracle WebLogic Server clusters provide automatic configuration synchronization for artifacts, such as data sources, across Oracle WebLogic Server domain. At the same time, Oracle WebLogic Server cluster controls synchronization of deployments and libraries used by different Oracle UMS components.

5.9 Oracle JCA Adapters and High Availability Concepts

The information in this section guides you through the issues and considerations necessary for configuring Oracle JCA Adapters for high availability.

5.9.1 Oracle JCA Adapters Single-Instance Characteristics

Oracle JCA Adapters are JCA binding components that allow the Service Infrastructure to communicate to endpoints using different protocols. Oracle JCA Adapters are deployed as a JCA resource (RAR) and are not part of Oracle SOA Service Infrastructure.

The run-time component of Oracle JCA Adapters is the J2CA 1.5 resource adapter for the specific back-end application. Oracle JCA Adapters are deployed in J2CA container of Oracle WebLogic Server. Oracle Fusion Middleware integrates with these J2CA 1.5 adapters through the JCA Binding Component, which converts Web service messages into J2CA interactions and back. Adapters can, therefore, fully leverage the scalability and high availability of the underlying Oracle WebLogic Server platform.

Figure 5-19 illustrates the services and dependencies of Oracle JCA Adapters single-instance architecture.

Figure 5-19 Oracle JCA Adapters Single-Instance Architecture

Oracle JCA Adapters Single-Instance architecture
Description of "Figure 5-19 Oracle JCA Adapters Single-Instance Architecture"

5.9.1.1 Oracle JCA Adapters Component Lifecycle

The lifecycle of an adapter is controlled by Oracle SOA Service Infrastructure. There are three high-level steps in the initialization of an Oracle JCA Adapter:

  • Deployment: Oracle JCA Adapters are deployed during installation as J2CA 1.5 resource adapters (RAR files) within the same Oracle WebLogic Server container as Oracle Fusion Middleware. The physical deployment of adapters involves using the RAR file and registering the adapters as connectors with the underlying Oracle WebLogic Server, or the middle tier platform. The RAR file contains the ra.xml file, which contains declarative information about the contract between Oracle WebLogic Server and the resource adapter. In addition, the RAR file contains weblogic-ra.xml which is the deployment descriptor file containing deployment-specific information about the resource adapter.

  • Loading: Loading refers to the process of creating in-memory resources and creating connections to the configured end-point. Although Oracle JCA Adapters are physically deployed as J2CA 1.5 resource adapters, their logical deployment involves creating the Connection Factory entries for the J2CA 1.5 resource adapter by editing the weblogic-ra.xml file. This file is Oracle WebLogic Server-specific deployment descriptor for a resource adapter. It contains configurations for deploying resource adapters to Oracle WebLogic Server, which includes the back-end application connection information as specified in the deployment descriptor of the resource adapter, Java Naming and Directory Interface (JNDI) name to be used, connection pooling parameters, resource principal mapping mechanism, and configurations.

  • Activation: Activation refers to initiation of a JCA binding component (Service and Reference) within a Composite. Listeners are started for the endpoint referenced by the Adapter configuration within the Composite.

Property Changes During Oracle JCA Adapters Runtime

Oracle JCA Adapters expose the underlying back-end operation-specific properties as header properties and allow the manipulation of these elements within a business process.The underlying properties are as follows:

  • interactionspec or activationspec Properties - These properties require the adapter endpoint to be recycled (re-activated).

  • Endpoint Properties - Changes to these properties are notified to the adapter without requiring the endpoint to be restarted.

5.9.1.2 Oracle JCA Adapters Reliability and Transactional Behavior

Oracle JCA Adapters support global transactions based on the JCA 1.5 XA contracts that leverage the underlying application server transaction manager. Adapters supporting XA transactions include Oracle Adapters for Oracle Applications, database, Advanced Queuing, JMS and MQSeries. Non-transactional adapters include Oracle File Adapter and Oracle FTP Adapter.

Inbound Transactions

For a synchronous process, the global transaction initiated by the adapter spans message delivery and composite execution.

For an asynchronous service entry point, a transactional adapter initiates a global JTA transaction before sending an inbound message to a composite. When control returns to the adapter, it commits the JTA transaction, executing the following set of actions as an atomic unit of work:

  1. Commit the removal of the message from the inbound adapter endpoint, for example, table and queue.

  2. Commit the execution of the composite instance.

If anything fails during this process, both of these actions are rolled back based on XA guarantees.

Outbound Transactions

For transactional adapters, outbound JCA interactions (the invoke activities) are scoped with the global JTA transaction of the Composite instance. This means that all composite activities, including Oracle JCA adapter invocations, are part of a global transaction, and as such all activities are either committed or rolled back if an error occurs. Therefore, one can guarantee exactly-once message delivery when both inbound and outbound adapters are transactional and the connection factories have been configured to support XA global transactions.

Nontransactional

The Oracle File Adapter picks up a file from an inbound directory, processes the file, and sends the processed file to an output directory. However, during this process if a failover occurs in the Oracle RAC backend or in an SOA managed server, then the file is processed twice because of the nontransactional nature of Oracle File Adapter. As a result, there can be duplicate files in the output directory.

5.9.1.3 Oracle JCA Adapters - Rejected Message Handling

The messages that error out before being posted to Oracle SOA Service Infrastructure mesh are referred to as rejected messages. For example, Oracle File Adapter picks a file having data in CSV format and tries to translate it to the XML format using NXSD. Now, if there is any errors in the translation, the message is rejected and is not posted to the target composite.

All rejected messages are stored in the database with payload. There are two destinations for rejected messages:

  • Adapter Rejection Table - Messages are stored in the Adapter Rejection table when there is a binding error related to a corrupt message.

    In case the database hosting the Rejected Message Table is offline, rejected messages are temporarily stored in the local file system (as a backup) and eventually loaded back into the database when it is detected as being online again.

  • Composite Instance Failure Table - This table is part of instance tracking, populated by each component detecting a failure.

In the case of errors being thrown by the SOA Service Infrastructure layer, the Adapter framework behavior depends on whether or not the error is marked as retriable. If the error is not retriable, the message is treated as a rejected message and persisted to the Rejected Message Table.

Configuring Rejection Handlers

Rejection handlers are defined and configured by using fault policies as described in the following steps:

  1. Define a fault policy for the rejected messages in the fault-policies.xml file, which is stored along with composite.xml in the Oracle JDeveloper project directory.

  2. Associate the fault policy with a service endpoint of the composite in the fault-bindings.xml file.

  3. Copy the fault-policies.xml file and the fault-bindings.xml file to Oracle SOA Composite project directory.

  4. Deploy the Oracle SOA Composite project.

Retry frequency and retry interval are both configurable. Service engines can mark an invocation as retriable. In a clustered environment, if roll back occurs, it may be picked up by another instance. If another node picks up the message, it generates a unique ID. The number of retries are configured within the composite, under the binding.jca element. For more information, see the Oracle Fusion Middleware User's Guide for Technology Adapters.

System-Defined Rejected Message Handlers

The following system-defined error handlers can be configured in fault policies:

  • Web Service Handler - Predefined WSDL interface must be implemented by the target service

  • Custom Java Handler - A predefined Java interface must be implemented by the target class.

  • JMS Queue - The rejected message is enqueued to a queue as a JMS message with the appropriate context and payload.

  • File - The rejected message is stored in the file system with the proper context and payload.

Payload Persistence

For resubmitting rejected messages, payload persistence is imperative. Payloads are stored in the database and be viewed through Oracle Enterprise Manager Fusion Middleware Control. Error handlers, invoked during automatic error handling also receive the payload during their invocation. Error payload persistence in the database is enabled by default.

5.9.2 Oracle JCA Adapters High Availability Architecture and Failover Considerations

This section describes an Oracle JCA Adapters high availability considerations for configuring Oracle JCA Adapters to run on an Oracle SOA cluster.

5.9.2.1 Oracle JCA Adapters High Availability Error Handling

This section describes Oracle JCA High Availability error handling.

Connection Errors

Oracle JCA Adapters and Oracle Adapters for Oracle Applications handle connection errors for the following interactions:

  • Outbound Interaction: Oracle JCA Adapters and Oracle Adapter for Oracle Applications raise the oracle.tip.adapter.api.exception.PCRetriableResourceException exception for transient connection errors, which are recoverable connection errors. For example, a database listener may not have started, resulting in connection errors. You can define the maximum number of attempts to reconnect made using the fault-policy.xml file. The parameters for attempts to reconnect can be specified using this file. After the configured number of retries is reached, the fabricInvocationException exception is thrown.

    Fault handling and retry properties are defined as part of the binding level configuration. When the binding level retry has expired, the fault is handled based on policies specified within the fault-policy.xml. For more information on binding properties, see the Oracle Fusion Middleware User's Guide for Technology Adapters.

  • Inbound Interaction: Oracle JCA Adapters, Oracle Adapter for Oracle Applications, and legacy adapters support a poll model for connecting to the back-end application for receiving events. In case of unrecoverable connection failures, the adapters recycle old connections, and send out alerts or notifications to Oracle SOA Service Infrastructure. The inbound interaction connection errors are written to a log, and can be viewed through Oracle WebLogic Server Administration Console.

5.9.2.2 Oracle File and FTP Adapters High Availability

The Oracle File and FTP Adapters can be configured for high availability using an active-active topology. A cluster-based high availability configuration is supported for both inbound and outbound operations. File and FTP Adapters do not support XA and hence can only guarantee at-least-once delivery of messages. Thus, it is possible to have duplicate messages after recovery from a server crash.

Prerequisites for High Availability

The following list describes prerequisites for Oracle File and FTP Adapters high availability:

  • In a clustered configuration, inbound adapters across managed servers must point to the same physical directory, a directory on a shared drive.

  • Connection-factories must specify the same shared folder as the control directory, and their names must match. For example, if the deployment descriptor for one connection-factory has /shared/control_dir as the value for controlDir, the other deployment descriptor must also have the same value.

The Oracle File and FTP Adapters must ensure that only one node processes a particular file in a distributed topology. You can use the database table as a coordinator to ensure that Oracle File and FTP Adapters are highly available for inbound operations. See the Oracle Fusion Middleware User's Guide for Technology Adapters for details on configuring a database table as a coordinator.

The Oracle File and FTP Adapters must ensure that if multiple references write to the same directory, then these do not overwrite each other. The following locking capabilities can be used to make Oracle File and FTP Adapters highly available for outbound operations:

  • Database mutex

  • User-defined mutex

See the Oracle Fusion Middleware User's Guide for Technology Adapters for details on configuring a database mutex.

Oracle File and FTP Adapters High Availability Configuration

A cluster-based high availability configuration is supported for both inbound and outbound operations. However, consider the following:

  • Inbound Operations - Oracle File and FTP Adapters must ensure that only one node processes a particular file in a distributed topology.

  • Outbound Operations - Oracle File and FTP Adapters must ensure that if multiple references write to the same directory, these do not overwrite each other.

Database-based mutexes are used as coordinators to ensure these behaviors in a clustered topology.

Configuring a Database Mutex

Configure a database table as a coordinator by modifying Oracle File and FTP Adapter deployment descriptor for the connection-instance corresponding to eis/HAFileAdapter from Oracle WebLogic Server Administration Console:

  1. Click FileAdapter under Summary of Deployments on Oracle WebLogic Server Administration Console.

  2. Click the Outbound Connection Pools tab, and expand javax.resource.cci.ConnectionFactory to see the configured connection factories.

  3. Click eis/HAFileAdapter. The Outbound Connection Properties for the connection factory corresponding to high availability is displayed.

  4. Update the connection factory properties.

Update the Adapter configuration to use the connection factory as shown in the following example:

<adapter-config name="FlatStructureOut" adapter="File Adapter" xmlns="http://platform.integration.oracle/blocks/adapter/fw/metadata">
              <connection-factory location="eis/HAFileAdapter" adapterRef=""/>
              <endpoint-interaction portType="Write_ptt" operation="Write">
            <interaction-spec className="oracle.tip.adapter.file.outbound.FileInteractionSpec">
                  <property../>
                  <property../>
                </interaction-spec>
              </endpoint-interaction>
            </adapter-config>

Note:

The location attribute is set to eis/HAFileAdapter for the connection factory.

The new parameters in connection factory for Oracle File and FTP Adapters are as follows:

  • controlDir - Set this parameter to the directory structure where you want the control files to be stored. Set it to a shared location if multiple Oracle WebLogic Server instances are running in a cluster.

  • inboundDataSource - Set this parameter to jdbc/SOADataSource. This is the data source where the schemas corresponding to high availability are pre-created. The pre-created schemas are located under MW_HOME/AS11gR1SOA/rcu/integration/soainfra/sql/adapter/createschema_adapter_oracle.sql. To create the schemas elsewhere, use this script. Set the inboundDataSource property accordingly if you choose a different schema.

  • outboundDataSource - Set this parameter to jdbc/SOADataSource. This is the data source where the schemas corresponding to high availability are pre-created. The pre-created schemas can be found under MW_HOME/AS11gR1SOA/rcu/integration/soainfra/sql/adapter/createschema_adapter_oracle.sql. To create the schemas elsewhere, use this script. Set the outboundDataSource property if you do create the schemas elsewhere.

  • outboundLockTypeForWrite - Set this parameter to oracle if you are using Oracle database. By default Oracle File and FTP Adapters use an in-memory mutex to lock outbound write operations. Choose one of the following values for synchronizing write operations:

    • memory - Oracle File and FTP Adapters use an in-memory mutex to synchronize access to the file system.

    • oracle - The adapter uses Oracle database sequence.

    • db - The adapter uses a pre-created database table (FILEADAPTER_MUTEX) as the locking mechanism. Use this option only if you are using a schema other than the Oracle database schema.

    • user-defined - The adapter uses a user-defined mutex. In order to configure the user-defined mutex, implement the mutex interface: oracle.tip.adapter.file.Mutex and then configure a new binding-property with the name oracle.tip.adapter.file.mutex and value as the fully qualified class name for the mutex for the outbound reference.

Note:

For large payloads, increase transaction timeout for the SOA DataSource by adding the following:

<xa-set-transaction-timeout>true</xa-set-transaction-timeout>

<xa-transaction-timeout>1000</xa-transaction-timeout>

Increase global transaction timeouts if a database is used as the coordinator.

5.9.2.3 Oracle Database Adapters High Availability

The Oracle Database Adapter supports high availability in an active-active setup. In an active-active setup, distributed polling techniques can be used for inbound Database Adapters to ensure that the same data is not retrieved more than once. For more information, see Section 9.3.7.1, "Distributed Polling Best Practice: MarkReservedValue." of the Adapter Guide. Similar to other adapters, an Oracle Database Adapter can also be configured for singleton behavior within an active-passive setup. This allows a high performance multi-threaded inbound Oracle Database Adapter instance running in an active-passive setup, to follow a fan out pattern and invoke multiple composite instances across a cluster. The Oracle Database Adapter also supports the high availability feature when there is a database failure or restart. The DB adapter picks up again without any message loss.For information about how an inbound rejected message is handled by using fault policy, see the Oracle Fusion Middleware User's Guide for Technology Adapters.

Surviving Database Restart

The Oracle Database Adapter can survive a database down scenario and pick up again without any message loss. For avoiding data loss, the data-source needs to be XA enabled and configured for RAC (multi-data source). For information on configuring a datasource for high availability, see Appendix B, "Recommended Multi Data Sources."

5.9.2.4 Oracle JMS Adapters High Availability

Oracle JMS Adapters support multiple provider including WebLogic JMS, MQ Series and Oracle AQ. JMS Adapters support exactly-once message delivery with no message loss during a server crash. In order to guarantee exactly-once delivery of messages, the JMS connection factory should be configured to support XA. For details on configuring connection factories for different adapters please refer to Section 8.4 of the Oracle Fusion Middleware User's Guide for Technology Adapters.

For Oracle AQ, it is also necessary to configure the data-source for XA and RAC. For information on configuring a datasource for high availability, see Appendix B, "Recommended Multi Data Sources."

5.9.2.5 Oracle JCA Adapters Log File Locations

You can view the logs for Oracle JCA Adapters as follows:

Oracle JCA Adapters and Oracle Adapter for Oracle Applications: For both outbound and inbound interactions, the log files are redirected to the soa-diagnostic.log file.The log files for Oracle Fusion Middleware SOA Suite that is deployed to the server-soa managed server are located in:

MW_HOME/user_projects/domains/domain_name/servers/server-soa/logs/soa-diagnostic.log

Packaged-application adapters: These adapters do not implement the LogManager interface because it is not part of the J2CA 1.5 standard. Therefore, for OPMN-managed components the log outputs are redirected to

ORACLE_INSTANCE\diagnostics\logs\component_type\component_name.

For outbound interactions, the logs are directed to the same location. On the other hand, for inbound interactions, logs are redirected to soa-diagnostic.log.

Legacy adapters: In addition to the J2CA resource adapter, legacy adapters consists of Oracle Connect, which consists of native adapters for communicating with the mainframe application and data stores. Oracle Connect logs can be viewed using Oracle Studio, which is the mainframe adapter design-time tool and Oracle Connect management tool. Oracle Connect generates various types of logs, such as the daemon log, workspace log, and server process log.

5.10 Oracle Business Activity Monitoring and High Availability Concepts

The information in this section guides you through the issues and considerations necessary for configuring Oracle BAM for high availability.

5.10.1 Oracle Business Activity Monitoring Single-Instance Characteristics

Oracle BAM provides the tools for monitoring business services and processes in the enterprise. It allows correlating of market indicators to the actual business process and to changing business processes quickly or taking corrective actions if the business environment changes. Oracle Business Activity Monitoring (Oracle BAM) provides the necessary tools and runtime services for creating dashboards that display real-time data inflow and define rules to send alerts under specified conditions.

Figure 5-20 illustrates the main services and dependencies that characterize an Oracle BAM instance.

Figure 5-20 Oracle Business Activity Monitoring Single-Instance Architecture

BAM Single-Instance Architecture
Description of "Figure 5-20 Oracle Business Activity Monitoring Single-Instance Architecture"

5.10.1.1 Oracle Business Activity Monitoring Component Characteristics

Oracle BAM is made up of the following components:

  • Oracle BAM Server is a set of runtime components that handle incoming data from different data sources. Oracle BAM components are also used to evaluate conditions for sending alerts to users and triggering the required actions configured for these alerts. The following are the main components in an Oracle BAM Server:

    • Active Data Cache is designed and optimized to handle large amounts of data in a real-time solution. To make data readily accessible and deliverable, it maintains real-time views of the data. The data feed to the Active Data Cache is a combination of business data sources, from data warehouse information to transactional feeds and other enterprise sources. The various data streaming technologies integrated with Oracle BAM send this information to the Active Data Cache in a continuous stream as data changes occur.

      The Active Data Cache hosts and runs the data objects, the view sets and the active view sets. It receives transactions (insert, update, delete, and upsert) to its data objects, and these data objects notify other data objects which are linked to them through lookups. Active view sets which are monitoring these data objects are notified of the changes and produce active data.

    • Event Engine: Event Engine monitors complex data conditions and implements specified rules. Rules can include a series of conditions and actions attached to an event. The Event Engine continuously monitors the information in the Active Data Cache for certain conditions and executes the related actions defined in associated rules.

      The Event Engine is responsible for tracking events based on date, time or data changes. The design of the Event Engine uses a satellite concept, in which there are four different systems (satellites), with which event clauses can be registered and in which they can be tracked.

    • Report Cache off loads the burden of maintaining the view set snapshot in memory from the Active Data Cache. The Report Cache opens view sets and active view sets in the Active Data Cache for the Report Server in Oracle BAM Web Applications set of components. It then caches the snapshot (in chunks) and the active data before sending it to the Report Server. This allows for random access into the snapshot and recovery from losing internet connectivity. The Report Cache also allows the Report Server to be stateless, and with the Active Data Cache it supports view set sharing.

  • Real Time Data Streaming Clients: In an Oracle BAM system clients can feed data into Oracle BAM server by using different types of protocols and APIs. The following are the available mechanisms to send data to the Oracle BAM server:

    • Oracle BAM Adapter is a JCA-compliant Adapter, which Java EE applications can use to send data to the Oracle BAM Server.

    • Enterprise message sources are used by applications to provide direct Java Message Service (JMS) connectivity to the Oracle BAM server by mapping messages directly to Oracle BAM data objects. The Oracle BAM server can read data directly from any JMS based message queue or topic. This option offers guaranteed messaging.

      In an Oracle BAM high availability environment, involving Enterprise Message Sources, the Enterprise Message Source property Start when BAM Server starts should be set to Yes.

      Durable Subscriber Name must be set in the Enterprise Message Source to ensure that no messages are lost during the failover.

    • Oracle Data Integrator is the ETL tool that is used with Oracle BAM to perform rigorous data transformations. The Oracle BAM Server has been implemented as an ODI Technology (for example, DB2, SQL Server are ODI Technologies) and Oracle BAM has ODI Knowledge Modules which let ODI perform all of the operations on the Oracle BAM Server to facilitate reading and writing data in various ways, including Change Data Capture.

    • Web Services API interacts directly with Oracle BAM data objects from a remote client.

  • Oracle BAM Web Applications are the Web user interfaces (Java EE Web applications) for viewing Oracle BAM data. Oracle BAM Web Applications also allow for building data models and creating dashboards and alerts. Oracle BAM Web Applications are the browser based interfaces provided for building reports and administrating the users that have access to the BAM User Interface system. As part of the Web applications, a Report Server applies the report definitions to the data sets retrieved from the Active Data Cache for presentation in a browser.

  • ICommand is a command line interface for administrating the data in the Active Data Cache. It provides a set of commands to export, import, rename, clear, and delete items from Active Data Cache.

The BAM application runs separately from the rest of Oracle Fusion Middleware SOA Suite components. Oracle BAM Web Applications are deployed together with the BAM Server. Oracle BAM Server uses EJBs and DAOs for implementing its services and persisting information to the database. All of the EJBs are stateless. However, the Oracle BAM Server is a singleton: only one active Oracle BAM server is used for maintaining fed data and for pushing it into the Web Application Reports. Oracle WebLogic Server Migration feature is used to protect Oracle BAM server from failures. Oracle BAM server uses JMS intensively. However, all Oracle BAM JMS queues are internal and do not require to be configured in a high availability configuration. When failover occurs, the queues are recreated on the new active Oracle BAM instance.

Oracle BAM Web Application Modules can run in full active-active mode connecting to the available Oracle BAM server. The information is retrieved using RMI protocol to access the EJBs that provide the information to remote consumers. Oracle BAM Web Applications are stateless, except for the Reports Server, which maintains a session ID for reports or users accessing the systems. This requires enabling session replication for the WebLogic Managed Server where Oracle BAM Web Applications run.

The processing of work by the EJBs and threads in Oracle BAM is non transactional and does not require Oracle WebLogic Server special transaction logs configuration. At the same time, Oracle BAM uses the database intensively, therefore it is important that Oracle BAM's database access is prepared for failure in the database. This requires configuring multi data sources for Oracle BAM data source as described in Section 5.12, "Configuring High Availability for Oracle BAM". For information about multi data source configuration with RAC and the MDS repository, see Section 4.1.2, "Using Multi Data Sources with Oracle RAC."

External Dependencies

Oracle BAM engine system depends only on the Oracle BAM database (which contains Oracle BAM schemas) for Oracle BAM data and report metadata. The database must be available for Oracle BAM system to start and run properly.

5.10.1.2 Oracle Business Activity Monitoring Startup/Shutdown Lifecycle

As shown in Figure 5-21, when the WebLogic Server is started, the Oracle BAM Server application is initialized. During startup, the Active Data Cache loads all the data from the repository. The Reports Cache is initialized with the data required for the available systems. The Event Engine starts analyzing the data and preparing notifications. Oracle BAM Web Applications are configured at boot time with an Oracle BAM Server's address. When Oracle BAM Web Applications are started, they connect to the Active Data Cache in Oracle BAM server and retrieve the corresponding viewsets through the Reports Server. Once initialization is done, the system is ready to receive data from clients and raise events and update reports dynamically.

Figure 5-21 Oracle Business Activity Monitoring Startup/Shutdown Lifecycle

BAM Startup/Shutdown Lifecycle
Description of "Figure 5-21 Oracle Business Activity Monitoring Startup/Shutdown Lifecycle"

5.10.1.3 Oracle Business Activity Monitoring Startup and Shutdown of Processes

Oracle BAM Server is an application, oracle-bam, that operates independently from the rest of the SOA Service Infrastructure. It runs in the managed server where Oracle BAM is installed. It gets started by default with Oracle WebLogic Managed Server to which Oracle BAM is deployed. Use Oracle WebLogic Server Administration Console to verify Oracle BAM Server's status and to start and stop it. You can also use WebLogic Server WLST command line to control the application.

Figure 5-22 Startup and Shutdown of Oracle Business Activity Monitoring using WebLogic Server

Startup and Shutdown of Oracle Business Activity Monitoring
Description of "Figure 5-22 Startup and Shutdown of Oracle Business Activity Monitoring using WebLogic Server"

Oracle Enterprise Manager Fusion Middleware Control allows multiple operations and configuration of the BAM Server as well as monitoring its status. For details about monitoring and controlling the Oracle BAM engine, Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite.

Figure 5-23 Startup and Shutdown of Oracle Business Activity Monitoring using Fusion Middleware Control

Startup and Shutdown of Oracle Business Activity Monitoring
Description of "Figure 5-23 Startup and Shutdown of Oracle Business Activity Monitoring using Fusion Middleware Control "

The BAM Web Applications run collocated by default in the same managed server as BAM Server. for example, the BAM server and BAM Web Applications are part of the same deployment in WebLogic Server Administration Console. They can't be stopped and managed separately from BAM Server by default. It is possible though, as explained in later sections, to manipulate WebLogic Server application targeting to separate both components. The Oracle WebLogic Administration Console allows starting and stopping Oracle BAM. This can also be done using WLST commands.

Oracle Enterprise Manager Fusion Middleware Control Console allows multiple operations and configuration of the BAM Web Applications

Figure 5-24 Oracle Enterprise Manager Fusion Middleware Control

Oracle Enterprise Manager Fusion Middleware Control
Description of "Figure 5-24 Oracle Enterprise Manager Fusion Middleware Control"

Note:

Although Enterprise Manager offers separate start and stop for Oracle BAM Server and Oracle BAM Web Applications, this is in reality a stop operations that affects both components, meaning if Oracle BAM Server is stopped from Enterprise Manager, the corresponding Oracle BAM Web Applications are also stopped, and the reverse is also true. This also applies for start operations.

5.10.1.4 Oracle Business Activity Monitoring Configuration Artifacts

Oracle Enterprise Manager Fusion Middleware Control exposes some of the configuration options for Oracle BAM Server.

Figure 5-25 Configuring Oracle Business Activity Monitoring

Configuring Oracle Business Activity Monitoring
Description of "Figure 5-25 Configuring Oracle Business Activity Monitoring "

The properties exposed by Oracle Enterprise Manager are a mix of the information contained in two different files: BAMServerConfig.xml and BAMCommonConfig.xml. These files are located under the DOMAIN_HOME/servers/BAM_Server_Name/tmp/_WL_user/oracle-bam_11.1.1/yhryfp/APP-INF/classes/config directory (notice that yhryfp is the random directory generated at installation time for deploying BAM applications). This is the random directory generated at installation time for deploying Oracle BAM applications. The properties in these files are modifiable by using the Mbeans exposed in Oracle Enterprise Manager Fusion Middleware Control. For details on the configuration options for Oracle BAM Servers, refer to the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Fusion Middleware User's Guide for Oracle Business Activity Management.

Similarly, some properties are exposed for Oracle BAM Web Applications by Oracle Enterprise Manager Fusion Middleware Control as shown in Figure 5-26.

Figure 5-26 Oracle Business Activity Monitoring Configuration Properties

Oracle Business Activity Monitoring Configuration Properties
Description of "Figure 5-26 Oracle Business Activity Monitoring Configuration Properties"

Configuration options at the container level, such as data sources and persistent stores, are maintained as part of Oracle WebLogic Server Domain configuration and are synchronized across a cluster of Oracle WebLogic Servers by Oracle WebLogic Server core infrastructure. See the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite for details on configuring Oracle BAM.

5.10.2 Oracle Business Activity Monitoring High Availability Architecture and Failover Considerations

Figure 5-27 describes a two-node Oracle BAM cluster running on two Oracle WebLogic Servers. Oracle WebLogic Servers are front ended by Oracle HTTP Servers which load balances incoming requests to them. The following are the main characteristics of this configuration:

  • Oracle BAM Web Applications run on two clustered WebLogic Server managed servers. The WebLogic Server Cluster synchronizes configuration for common artifacts of WebLogic Server used by Oracle BAM Web Applications, such as data sources, persistent store, and definitions. Oracle BAM Server is targeted by any of the servers where BAM Web applications are running. Only one WebLogic server runs Oracle BAM Server.

  • The architecture uses Oracle WebLogic Server Migration feature to protect Oracle WebLogic Server that runs both Oracle BAM Web Applications and Oracle BAM Server (in APPHOST1) against failures. This provides protection for Oracle BAM Server which is a singleton. The WebLogic Managed Server in which Oracle BAM Server runs is listening on a Virtual IP that gets migrated to another node when the failover occurs. This is the address that Oracle BAM Web Applications in APPHOST2 use to connect to an Oracle BAM Server. Plan appropriately to account for the scenario where Oracle BAM Server and two instances of Oracle BAM Web Applications are running on APPHOST2. For more information on Server Migration features, see Chapter 3, "High Availability for WebLogic Server."

  • Oracle BAM's database is configured with Oracle Real Application Clusters (RAC) to protect from database failures. Oracle BAM Server performs the appropriate reconnection and operations retries if database instance failure occurs.

Figure 5-27 Oracle BAM High Availability Architecture

Oracle BAM High Availability Architecture
Description of "Figure 5-27 Oracle BAM High Availability Architecture"

5.10.2.1 Oracle Business Activity Monitoring Protection from Failures and Expected Behavior

Oracle BAM Server and Oracle BAM Web applications are protected from all process failures by the WebLogic Server infrastructure. This section also describes expected behavior in the event of component failure.

Process Failure

Oracle BAM Server and BAM Web Applications are protected from all process failures by the WLS infrastructure. This section describes process failure considerations for Oracle BAM.

  • If the WLS_BAMx server crashes, Node Manager attempts to restart it locally. It attempts to restart the servers according to the configured restart count threshold.

    • For failures related to the WebLogic Server on which the BAM Server is running, if repeated restarts fail, the WebLogic Server infrastructure attempts to perform a server migration to the other node in the cluster. Ongoing requests from the clients time out during failover. Once the server's restart completes on the other node, the clients should be restarted to continue routing to it. For opened reports running in the Web browser, if a request is delivered though the BAM WebApps running in the same managed server as BAM Server, a Reconnecting message appears in the BAM WebApps report. If the request is being delivered through the BAM WebApps running in the other node, no message appears. In this situation, and during failover, you may be viewing stale data (while a failover takes place or while connection from webapps to servers is reestablished). The BAMWebApps Servlet remains available, and reconnects as soon as BAM Server comes up. For new reports or requests, BAM Web Applications in APPHOST2 reconnect once the VIP is migrated and the managed server is restarted. BAM Web Applications listening on VIP1 become functional once the server migration completes. At this point, the HTTP Server restarts routing HTTP requests to the Managed Server.

    • Failures affecting the WebLogic Server where only the BAM Web Applications are running, do not affect the BAM system. Other BAM Web Applications (running in APPOHOST1) remain available and maintain session information by using the WebLogic Server Session replication cluster. Failover should be transparent.

  • The oracle-bam application where BAM Server and BAM Web Applications run may be down due to failure in accessing resources, errors in reading the database, or other issues unrelated to the status of the managed server where it is located. Therefore, you should monitor Oracle SOA Service Infrastructure application and watch for errors caused by the application in the managed server logs, for log file locations, see Section 5.2.1.6, "Oracle SOA Service Infrastructure Log File Locations".

  • Failover may not succeed for an open report if when the report is opened, only one BAM managed server in the cluster was up, and after starting an additional managed server in the cluster, no other operations were performed. One trigger for session state replication is opening a report. Session state replication is not triggered by active data updates.

Node Failure

For node failures in APPHOST2, the behavior in case of a node failure is equivalent to the process failure scenario: Oracle BAM Web Applications in the other node remain available and can serve requests. Sessions are preserved by the session replication framework provided by WebLogic Server and failover to the other node should be transparent.

For node failures in APPHOST1, server migration is triggered after the available server verifies the time stamp in the database leasing system. While failover occurs, clients are unable to feed data into the system and retry appropriately. Oracle BAM Web Applications connecting to the server attempt to reconnect until the VIP is migrated and the server is restarted.

Database Failure

For information about Oracle BAM database failure, see Section 5.2.2.1, "Oracle SOA Service Infrastructure Protection from Failures and Expected Behavior"

5.10.2.2 Oracle Business Activity Monitoring Cluster-Wide Configuration Changes

The standard Java EE artifacts that Oracle BAM Server and Oracle BAM Web Application use are configured as part of Oracle WebLogic Domain in which Oracle BAM is installed. Oracle WebLogic Clusters provide automatic configuration synchronization for artifacts such as data-sources, persistent stores, and JMS modules, across the WebLogic Server domain. At the same time, the WebLogic Server cluster is in charge of synchronizing the deployments, and libraries used by Oracle BAM Web Applications and Oracle BAM Server.

As explained in the single instance section Oracle BAM Server's and Oracle BAM Web Applications configuration are maintained in the DOMAIN_HOME/servers/BAM_Server_Name/tmp/_WL_user/oracle-bam_11.1.1/yhryfp/APP-INF/classes/config directory (notice that yhryfp is the random directory generated at installation time for deploying BAM applications). The properties in these files can be modified by using the Mbeans exposed in Oracle Enterprise Manager Fusion Middleware Control. The properties exposed through MBeans are specific to each server. The properties exposed through Enterprise Manager-specific screens are cluster-wide and are only modified on one server. All properties, whether applied in Enterprise Manager or in an MBean browser require a restart of Oracle WebLogic Servers where Oracle BAM runs. For details on the configuration options for BAM Server and BAM Web Applications see the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Fusion Middleware User's Guide for Oracle Business Activity Management.

One of the configuration options related to high availability environments is the Application URL which is used to determine the front end host used by the system in a cluster configuration. This option is used to produce the copy shortcut URL for reports and alerts. The other relevant parameter for Oracle BAM high availability configuration is the Server Name in OracleBAM Web configuration screen. This parameter is used by Oracle Web Application to determine Oracle BAM server to which it connects for accessing the Active Data Cache.

For SOA high availability installations frontended by Oracle HTTP Server, monitoring should be done on the on the Oracle HTTP Server ports of the real backend servers. This is the case when a deployment is using all the components deployed to the SOA Managed Server. A simple HTTP monitor that pings the HTTP/HTTPS port and expects a pre-determined response in turn should suffice. If only a specific SOA component is being used (such as B2B), then a monitor that does a deeper level check all the way to the Managed server can be considered to validate the health of the component in use. Please check with your load balancer vendor on setting up the HTTP monitors with your load balancer.

5.11 Configuring High Availability for Oracle SOA Service Infrastructure and Component Service Engines

The procedures described in this section include setting up the Service engines contained in Oracle SOA Service Infrastructure system, such as Oracle BPEL PM, Oracle Mediator, Oracle Human Workflow and Oracle Decision Services, as well as Oracle B2B and Oracle User Messaging Service.

Note:

Oracle strongly recommends reading the release notes for any additional installation and deployment considerations prior to starting the setup process.

Figure 5-28 represents the example architecture that the configuration steps in this section create.

Figure 5-28 Oracle SOA Service Infrastructure High Availability Architecture

Service Infrastructure High Availability
Description of "Figure 5-28 Oracle SOA Service Infrastructure High Availability Architecture"

Figure 5-28 describes a two-node SOA cluster running on two Oracle WebLogic Servers. Oracle Servers are front ended by an Oracle HTTP Server, which load balances incoming requests. A separate Oracle WebLogic Server is used for custom logic and application deployment. This configuration uses an Oracle RAC database for storing metadata and SOA schemas, and shared storage for transaction and JMS stores. Virtual IP addresses (VIPs) provide manual failover for the Administration Server and for Oracle SOA Servers (for Server Migration). For more details about the components contained in this architecture, see the individual component sections in this chapter.

For information about configuring virtual IPs for the Administration Server and configuring the Administration Server for high availability, see Section 10.2.2.3, "Transforming the Administration Server for Cold Failover Clusters."

5.11.1 Preparing the Environment: Prerequisite Steps Before Setting up a SOA High Availability Configuration

The following sections provide prerequisite steps before setting up an Oracle SOA Service Infrastructure high availability configuration.

For information about platform-specific commands, see the Oracle Fusion Middleware Installation Guide for Oracle SOA Suite.

5.11.1.1 Database Prerequisites

Oracle SOA Service Infrastructure supports the following database versions:

  • Oracle Database 10g (10.2.0.4 or later for non-XE database)

  • Oracle Database 11g (11.1.0.7 or later for non-XE database)

To determine the database version, execute this query:

SQL>select version from sys.product_component_version where product like 'Oracle%';

5.11.1.2 VIP and IP Prerequisites

As shown in Table 5-1, you configure the Administration Server and the SOA managed servers to listen on different virtual IPs. This requires the provisioning of the corresponding VIP in the node and related host names in the DNS system in your network. Make sure that the different VIPS are available and are reachable before running the installation.

Table 5-1 Virtual Hosts

Virtual IP VIP Maps to... Description

VIP0

APPHOST1VHN0

APPHOST1VHN0 is the virtual host name that is the listen address for the Administration Server and fails over with manual failover of the Administration Server. It is enabled on the node where the Administration Server process is running (APPHOST1 by default).

VIP1

APPHOST1VHN1

APPHOST1VHN1 is the virtual host name that maps to the listen address for WLS_SOA1 and fails over with server migration of this managed server. It is enabled on the node where WLS_SOA1 process is running (APPHOST1 by default).

VIP2

APPHOST2VHN2

APPHOST2VHN1 is the virtual host name that maps to the listen address for WLS_SOA2 and fails over with server migration of this managed server. It is enabled on the node where WLS_SOA2 process is running (APPHOST2 by default).


5.11.1.3 Shared Storage Prerequisites

For proper recovery in case of failure, store both JMS and transaction logs in a location that is accessible to all the nodes that can resume the operations after a failure in a managed server. This requires a shared storage location that can be referenced by multiple nodes. Table 5-2 lists the contents of shared storage.

Table 5-2 Contents of Shared Storage

Server Type of Data Vol in Shared Storage Directory Files

WLS_SOA1

Tx Logs

VOL1

ORACLE_BASE/admin/domain_name/soa_cluster_name/tlogs

Common location (stores decided by WebLogic Server)

WLS_SOA2

Tx Logs

VOL1

ORACLE_BASE/admin/domain_name/soa_cluster_name/tlogs

Common location (stores decided by WebLogic Server)

WLS_SOA1

JMS Stores

VOL1

ORACLE_BASE/admin/domain_name/soa_cluster_name/jms

Common location but Individual store per server (for example: SOAJMSStore1, UMSJMSStore1)

WLS_SOA2

JMS Stores

VOL1

ORACLE_BASE/admin/domain_name/soa_cluster_name/jms

Common location but Individual store per server (for example: SOAJMSStore2, UMSJMSStore2)


The shared storage can be a NAS or SAN device. The following is an example command based on a NAS device. Your options may be different from the ones specified in this section:

APPHOST1> mount nasfiler:/vol/volX/FMWshared
MW_HOME -t nfs -o
rw,bg,hard,nointr,tcp,vers=3,timeo=300,rsize=32768,wsize=32768

5.11.1.4 Installing and Configuring an LDAP Provider

For production environments, it is a mandatory requirement for Oracle SOA Suite high availability topologies to have an external LDAP policy store. For more information on the supported policy stores as well as instructions on configuring LDAP, see the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite.

5.11.1.5 Synchronizing System Clocks

Synchronizing system clocks on each of the cluster nodes for high availability SOA deployments is required.

5.11.1.6 Terminology for Directories and Directory Environment Variables

The follow list describes the directories and variables used in this chapter:

  • ORACLE_BASE: This environment variable and related directory path refers to the base directory under which Oracle products are installed.

  • MW_HOME: This environment variable and related directory path refers to the location where Fusion Middleware (FMW) resides.

  • WL_HOME: This environment variable and related directory path contains installed files necessary to host a WebLogic Server.

  • ORACLE_HOME: This environment variable and related directory path refers to the location where Oracle FMW SOA Suite is installed.

  • ORACLE_COMMON_HOME: This environment variable and related directory path refers to the Oracle home that contains the binary and library files required for the Oracle Enterprise Manager Fusion Middleware Control and Java Required Files (JRF).

    There can be only one Oracle Common home within each Middleware home.

  • DOMAIN Directory: This directory path refers to the location where the Oracle WebLogic Domain information (configuration artifacts) is stored.

  • ORACLE_INSTANCE: An Oracle instance contains one or more system components, such as Oracle Web Cache, Oracle HTTP Server, or Oracle Internet Directory. An Oracle instance directory contains updateable files, such as configuration files, log files, and temporary files.

The values used and recommended for consistency for this directories are:

ORACLE_BASE:

/u01/app/oracle

MW HOME (Apptier):

ORACLE_BASE/product/fmw

WL_HOME:

MW_HOME/wlserver_10.3

ORACLE_HOME:

MW_HOME/soa

ORACLE_COMMON_HOME:

MW_HOME/oracle_common

Location for JMS file based stores and Tlogs:

ORACLE_BASE/admin/domain_name/soa_cluster_name/jms
ORACLE_BASE/admin/domain_name/soa_cluster_name/tlogs

Mount point is:

ORACLE_BASE/admin/domain_name/soa_cluster_name/

Shared Storage location:

Shared storage location: ORACLE_BASE/admin/<domain_name>/aserver

5.11.1.7 Installing and Configuring the Database Repository

This section describes how to install and configure the database repository.

Oracle Clusterware

  • For 10g Release 2 (10.2), see Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide.

  • For 11g Release 1 (11.1), see the Oracle Clusterware Installation Guide.

Automatic Storage Management

  • For 10g Release 2 (10.2), see Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide.

  • For 11g Release 1 (11.1), see Oracle Real Application Clusters Installation GuideOracle Real Application Clusters Installation Guide.

  • When you run the installer, select the Configure Automatic Storage Management option in the Select Configuration page to create a separate Automatic Storage Management home.

Oracle Real Application Clusters

  • For 10g Release 2 (10.2), see Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide.

  • For 11g Release 1 (11.1), see Oracle Real Application Clusters Installation Guide.

You must install the 11g (11.1.1) Oracle Fusion Middleware Repository into a Real Application Cluster database before you install the Oracle Fusion Middleware components. Oracle Fusion Middleware provides a tool, Oracle Fusion Middleware Repository Creation Utility (RCU), to create the component schemas in an existing database.You install RCU in its own, separate Middleware home.

See Oracle Fusion Middleware Repository Creation Utility User's Guide for more information about installing RCU.

Database Initialization Parameters

Ensure that the following initialization parameter is set to the required value. It is checked by Repository Creation Assistant.

Table 5-3 Required Initialization Parameters

Parameter Required Value Parameter Class

PROCESSES

300 or greater

Static


To check the value of the initialization parameter using SQL*Plus, you can use the SHOW PARAMETER command.

As the SYS user, issue the SHOW PARAMETER command as follows:

SQL> SHOW PARAMETER processes

Set the initialization parameter using the following command:

SQL> ALTER SYSTEM SET processes=300 SCOPE=SPFILE

Restart the database.

Note:

The method that you use to change a parameter's value depends on whether the parameter is static or dynamic, and on whether your database uses a parameter file or a server parameter file. See the Oracle Database Administrator's Guide for details on parameter files, server parameter files, and how to change parameter values.

5.11.1.8 Using Oracle Fusion Middleware Repository Creation Utility to Load the Fusion Middleware Schemas in the Database

Install the 11g (11.1.1) Oracle Fusion Middleware Metadata Store and Oracle SOA schemas into a Real Application Cluster database before you install Oracle Fusion Middleware SOA components. Oracle Fusion Middleware provides a tool, Oracle Fusion Middleware Repository Creation Utility (RCU), to create the component schemas in an existing database. See Oracle Fusion Middleware Installation Concepts and Repository Creation Utility User's Guide for more information about installing RCU

5.11.1.8.1 Running RCU

Run RCU to install the required metadata for Oracle Fusion Middleware 11g.

  1. For linux, go to RCU_Home/bin and use the following command:

    ./rcu
    
  2. In the Welcome screen, click Next.

  3. In the Create Repository screen, select Create to load component schemas into a database, click Next.

  4. In the Database Connection Details screen, enter connection information for your database:

    • Database Type: Select Oracle Database.

    • Host Name: Enter the name of the node that is running the database. For an Oracle RAC database, specify the VIP name, or one of the node names as the host name: SOADBHOST1VIRTUAL.

    • Port: The port number for the database: 1521

    • Service Name: Enter the service name of the database: soaha.mycompany.com

    • Username: SYS

    • Password: Enter the password for the SYS user.

    • Role: SYSDBA

  5. Click Next.

  6. If you receive the following warning message:

    The database you are connecting is with non-UTF8 charset, if you are going to use this DB for multilingual support, you may have data loss. If you are not using for multilingual support you can continue, otherwise we strongly recommend using UTF-8 database.

  7. Click Ignore or Stop.

  8. In the Select Components screen, select Create a New Prefix, and enter a prefix to use for the database schemas, for example, SOAHA

    Write down the schema names so they are available in later procedures.

    • Select the following schemas:

      • Under AS Common Schemas, select Metadata Services.

      • Under SOA Service Infrastructure, select SOA Service Infrastructure and User Messaging.

  9. Click Next.

  10. In the Schema Passwords screen, enter passwords for the main and additional (auxiliary) schema users, and click Next.

  11. In the Map Tablespaces screen, choose the tablespaces for the selected components, and click Next.

  12. In the Summary screen, click Create.

  13. In the Completion Summary screen, click Close.

See the Oracle Fusion Middleware Repository Creation Utility User's Guide for more information about using RCU.

5.11.1.8.2 Configuring SOA Schemas for Transactional Recovery Privileges

You need the appropriate database privileges to allow the Oracle WebLogic Server transaction manager to query for transaction state information and issue the appropriate commands, such as commit and rollback, during recovery of in-flight transactions after a WebLogic Server container crash.

To configure the SOA schemas for transactional recovery privileges:

  1. Log on to sqlplus as a user with sysdba privileges. For example:

    sqlplus "/ as sysdba"
    
  2. Grant select on sys.dba_pending_transactions to soaha_soainfra.

  3. Grant force any transaction to soaha_soainfra.

    Note:

    These privileges should be granted to the owner of the soainfra schema, as determined by the RCU operations.

5.11.2 Installing Oracle HTTP Server on WebHost1

To install Oracle HTTP Server on WEBHOST1:

  1. Verify that the servers meet the following requirements:

    • The system, patch, kernel, and other requirements meet the requirements specified in the Oracle Fusion Middleware Installation Guide for Oracle SOA Suite.

    • Port 7777 is not used by any service on the nodes. You can verify this by running the following command:

      Unix:

      netstat -an | grep LISTEN | grep "7777"
      

      Windows:

      netstat -an | findstr "LISTEN" | findstr ":7777"
      

    If the ports are in use, make them available.

  2. On UNIX platforms, if the /etc/oraInst.loc or /var/opt/oracle/oraInst.loc file exists, check that its contents are correct. Specifically, check that the inventory directory is correct, and that you have write permissions for that directory.

    If the /etc/oraInst.loc file does not exist, skip this step.

  3. Start Oracle Universal Installer for Oracle Fusion Middleware 11g Web Tier Utilities CD installation as follows:

    For UNIX, run this command: runInstaller.

    For Windows, double-click setup.exe.

  4. In the Welcome screen, click Next.

  5. In the Select Installation Type screen, select Install and Configure, and click Next.

  6. In the Prerequisite Checks screen, ensure that all the prerequisites are met, and click Next.

  7. In the Specify Installation Location screen, set the location to:

    /u01/app/oracle/product/11.1.1/ohs_1

  8. Click Next.

  9. In the Configure Components screen:

    • Select Oracle HTTP Server.

    • Do not select Associate Selected Components with WebLogic Domain.

    • Click Next.

  10. In the Specify Component Details screen, enter the following values:

    • Instance Home Location: Instance Home

    • Instance Home Location: /u01/app/oracle/product/11.1.1/ohs_1/instances/ohs_instance1

    • Instance Name: ohs_instance1

    • OHS Component Name: ohs1

  11. Click Next.

  12. In the Specify Web Tier Port Details screen:

    • Select Specify Custom Ports. If you specify a custom port, select Specify Ports using Configuration File, and use the Browse function to select the file.

    • Enter the Oracle HTTP Server port, for example, 7777.

  13. Click Next.

  14. In the Configuration Summary screen, ensure that the selections are correct, and click Install.

  15. In the Installation Progress screen:

    For UNIX systems, a dialog box appears prompting you to run the oracleRoot.sh script. Open a command window and run the script, following the prompts.

    Click Next.

  16. In the Configuration screen, several configuration assistants are launched in succession. When the configuration assistants are finished, the Configuration Completed screen appears.

  17. In the Configuration Completed screen, click Finish to exit.

5.11.2.1 Validating Oracle HTTP Server

To verify that Oracle HTTP Server is set up correctly, access the root URL context of the server by using the following URL a browser:

HTTP://WebHost1:7777/

If Oracle HTTP Server is set up correctly, the Hello World page appears in the browser.

5.11.3 Installing Oracle Fusion Middleware Home

This section describes the procedure for installing Oracle Fusion Middleware on all nodes in the application tier that run Oracle WebLogic Server and Oracle Fusion Middleware SOA Suite.

Install the following Oracle Fusion Middleware components:

5.11.3.1 Installing Oracle WebLogic Server

To install Oracle WebLogic Server on all nodes in the application tier:

  1. Start Oracle WebLogic Server Installer

    On UNIX (Linux in the following example):

    APPHOST1> wls1032_linux32.bin
    

    On Windows:

    APPHOST1> wls1032_win32.exe
    
  2. In the Welcome screen, click Next.

  3. In the Choose Middleware Home Directory screen:

    • Select Create a New Middleware Home.

    • For the Middleware Home Directory field, enter MW_HOME.

    • Click Next.

  4. In the Register for Security Updates screen, enter your contact information for security update notifications, and click Next.

  5. In the Choose Install Type screen, select Custom, and click Next.

  6. In the Choose Products and Components screen, click Next.

  7. In the JDK Selection screen, select only Oracle JRockit 1.6.0_14 SDK, and click Next.

  8. In the Choose Product Installation Directories screen, accept the following directory:

    WL_HOME

    Click Next.

  9. In the Installation Summary screen, click Next.

  10. In the Installation Complete screen, deselect Run QuickStart, and click Done.

5.11.3.2 Installing Oracle Fusion Middleware for Oracle SOA

On Linux platforms, if the /etc/oraInst.loc file exists, check that its contents are correct. Specifically, check that the inventory directory is correct and that you have write permissions for that directory.

If the /etc/oraInst.loc file does not exist, you can skip this step.

  1. Start Oracle Fusion Middleware 11g Oracle SOA Suite Installer:

    On UNIX:

    APPHOST1> runInstaller
    

    On Windows:

    APPHOST1> setup.exe
    

    When Oracle Fusion Middleware 11g Oracle SOA Suite Installer prompts you for a JRE/JDK location enter Oracle SDK location created in Oracle WebLogic Server installation, for example, MW_HOME/jrockit_160_14_R27.6.4-18.

  2. In the Specify Inventory Directory screen, do the following:

    1. Enter HOME/oraInventory, where HOME is the home directory of the user performing the installation (this is the recommended location).

    2. Enter the OS group for the user performing the installation.

    3. Click Next.

    4. Follow the instructions on the screen to execute /createCentralInventory.sh as root.

    5. Click OK.

  3. In the Welcome screen, click Next.

  4. In the Prerequisite Checks screen, verify that the checks complete successfully, and click Next.

  5. In the Specify Installation Location screen:

    • For Middleware Home, enter MW_HOME.

    • For Oracle Home Directory, enter soa.

    Click Next.

  6. In the Installation Summary screen, click Install.

  7. In the Installation Complete screen, click Finish.

5.11.4 Enabling VIP1 in APPHOST1 and VIP2 APPHOST2

The SOA domain uses virtual hostnames as the listen addresses for the SOA managed servers. You must enable VIP mapping each of these hostnames on the two SOA Machines, (VIP1 on APPHOST1 and VIP2 on APPHOST2), and must be correctly resolve the virtual hostnames in the network system used by the topology (either by DNS Server, hosts resolution).

For information about configuring virtual IPs for the Administration Server and configuring the Administration Server for high availability, see Section 10.2.2.3, "Transforming the Administration Server for Cold Failover Clusters." Refer to Section 5.11.15, "Configuring Server Migration for the WLS_SOA Servers" for more details on configuring server migration for the SOA servers.

5.11.5 Running Oracle Fusion Middleware Configuration Wizard on APPHOST1 to Create the SOA Domain

Run Oracle Fusion Middleware Configuration Wizard from the SOA home directory to create a domain containing the Administration Server and Oracle SOA components. Ensure that the database where you installed the repository is running. For RAC databases, all the instances must be running.

  1. Change the directory to the location of Oracle WebLogic Server Configuration Wizard, located in the Middleware home, SOA directory:

    APPHOST1> cd ORACLE_HOME/common/bin
    
  2. Start Oracle WebLogic Server Configuration Wizard:

    For Linux:

    APPHOST1> ./config.sh
    

    For Windows:

    APPHOST1> config.cmd
    
  3. In the Welcome screen, select Create a New WebLogic Domain, and click Next.

  4. In the Select Domain Source screen:

    • Select Generate a domain configured automatically to support the following products.

    • Select the following products:

      • Basic Weblogic Server Domain - 10.3.1.0 [wlserver_10.3] (Selected by Default)

      • Oracle SOA Suite - 11.1.1.0 [soa]

      • Oracle Enterprise Manager - 11.1.1.0 [soa] (Selected by Default)

      • Oracle JRF - 11.1.1.0 [soa] (Selected by Default)

    If you accidentally deselect some of the targets, make sure that the following selections are made in this screen:

    • Oracle Enterprise Manager

    • Oracle WSM Policy Manager

    • Oracle JRF

    Click Next.

  5. In the Specify Domain Name and Location screen, make the following entries:

    • Domain Name: soadomain

    • Domain Location: accept the default

    • Application Location: accept the default

    Click Next.

  6. In the Configure Administrator Username and Password screen, enter the username and password to be used for the domain's administrator, and click Next.

  7. In the Configure Server Start Mode and JDK screen, make the following selections:

    • WebLogic Domain Startup Mode: select Production Mode

    • JDK Selection: select Oracle JRockit 1.6.0_14 SDK.

    Click Next.

  8. In the Configure JDBC Component Schema screen:

    1. Select all the component schemas that appear in the table at the bottom: SOA Infrastructure, User Messaging Service, OWSM MDS Schema and SOA MDS Schema.

    2. Select Configure selected component schemas as RAC multi data source schemas in the next panel.

    3. Ensure that the following data source appears on the screen. The user names shown in Table 5-4, assume that soaha was used as the prefix for schema creation from RCU.

      For information about multi data source configuration with RAC and the MDS repository, see Section 4.1.2, "Using Multi Data Sources with Oracle RAC."

      Table 5-4 Configuring Values for Data Sources

      Multi Data Source Schema Schema Owner

      SOA Infrastructure

      SOAHA_SOAINFRA

      User Messaging Service

      SOAHA_ORASDPM

      OWSM MDS Schema

      SOAHA_MDS

      SOA MDS Schema

      SOAHA_MDS


    4. Click Next.

  9. In the Configure RAC Multi Data Source Component Schema screen, enter values for the following fields, specifying the connect information for the RAC database that was seeded with RCU:

    Figure 5-29 Configure RAC Multi Data Source Component Schema Screen

    Configure RAC Multi Data Sources
    1. Driver: Select Oracle driver (Thin) for RAC Service-Instance connections, Versions:10, 11.

    2. Service Name: Enter the service name of the database, for example, soaha.mycompany.com.

    3. Username prefix: Enter the prefix for the schemas. The user names shown in Table 5-4 assume that soaha was used as prefix for schema creation from RCU.

    4. Password and Confirm Password: Enter the password for access to the schemas.

    5. Click Add, and enter the details for the first RAC instance.

    6. Update each multi data source schema by selecting one data source at a time, adding the appropriate details.

      Make sure that the information is entered for all the multi data source schemas: SOA Infrastructure, User Messaging Service, OWSM MDS Schema, and SOA MDS Schema.

    7. Click Next.

  10. In the Test JDBC Data Sources screen, the connections are tested automatically. The Status column displays the results. Ensure that all connections were successful. If not, click Previous to return to the previous screen and correct your entries.

    Click Next when all the connections are successful.

  11. In the Select Optional Configuration screen, select the following:

    Figure 5-30 Select Optional Configuration Screen

    Select Optional Configuration
    • Administration Server

    • Managed Servers, Clusters and Machines

  12. In the Customize Server and Cluster Configuration screen, select Yes, and click Next.

  13. In the Configure the Administration Server screen, enter the following values:

    • Name: AdminServer

    • Listen Address: Enter the hostname for the VIP0 virtual IP

    • Listen Port: 7001

    • SSL listen port: N/A

    • SSL enabled: leave unchecked

    Click Next.

  14. In the Configure Managed Servers screen, add the following managed servers:

    Table 5-5 Configuring Managed Servers

    Name Listen Address Listen Port SSL Listen Port SSL Enabled

    WLS_SOA1

    APPHOST1VHN1

    8001

    n/a

    No

    WLS_SOA2

    APPHOST2VHN2

    8001

    n/a

    No


    Do not delete any server that appears. You can modify the servers. If you delete a server and add a new one, targeting fails.

    Note:

    Although the standard recommendation is to run custom applications and other systems in a separate WebLogic Managed Server, the creation of the custom WLS managed servers described in Figure 5-28 is not addressed here.

    Click Next.

  15. In the Configure Clusters screen, add the following cluster:

    • Name: SOA_Cluster

    • Cluster Messaging Mode: unicast

    • Multicast Address: N/A

    • Multicast Port: N/A

    • Cluster Address: Leave empty

    Click Next.

  16. In the Assign Servers to Clusters screen, assign the following servers to SOA_Cluster:

    • WLS_SOA1

    • WLS_SOA2

    Click Next.

  17. In the Configure Machines screen:

    • Delete the LocalMachine that appears by default.

    • Click the Unix Machine tab, and add the following machines:

    Table 5-6 Configuring Machines

    Name Node Manager Listen Address

    APPHOST1

    Hostname of APPHOST1

    APPHOST2

    Hostname of APPHOST2


    Leave all other fields to their default values, and click Next.

  18. In the Assign Servers to Machines screen, assign servers to machines as follows:

    Figure 5-31 Assign Servers to Machines Screen

    Assign Servers to Machines screen
    • APPHOST1: AdminServer, WLS_SOA1

    • APPHOST2: WLS_SOA2

    Click Next.

  19. In the Review WebLogic Domain screen, click Next.

  20. In the Configuration Summary screen, click Create.

  21. In the Creating Domain screen, click Done.

Note:

The multicast and unicast addresses are different from the ones used by the WebLogic Server cluster for cluster communication. SOA guarantees that composites are deployed to members of a single WebLogic Server cluster even though the communication protocol for the two entities (the WebLogic Server cluster and the groups to which composites are deployed) are different.

5.11.6 Creating boot.properties for the Administration Server on APPHOST1

This is an optional step for enabling the Administration Server to start up without prompting you for the administrator username and password. Create a boot.properties file for the Administration Server on APPHOST1.

For the Administration Server:

  1. Create the following directories:

    APPHOST1> mkdir ORACLE_BASE/product/fmw/user_
    projects/domains/soadomain/servers/ 
    
    APPHOST1> mkdir ORACLE_BASE/product/fmw/user_
    projects/domains/soadomain/servers/AdminServer/
    
    APPHOST1> mkdir ORACLE_BASE/product/fmw/user
    _projects/domains/soadomain/servers/AdminServer/security
    
  2. Use a text editor to create a file called boot.properties in the security directory created in the previous step, and enter the following lines in the file:

    username=adminuser
    password=password
    

    Note:

    When you start up the Administration Server, the username and password entries in the file are encrypted.

    For security reasons, minimize the time the entries in the file are left unencrypted. After you edit the file, start up the server as soon as possible in order for the entries to be encrypted.

5.11.7 Starting the System in APPHOST1

This section describes procedures for starting the system in APPHOST1

5.11.7.1 Starting the Administration Server on APPHOST1

To Start the Administration Server on APPHOST1 run the following commands:

APPHOST1> cd MW_HOME/user_projects/domains/soadomain/bin

APPHOST1> ./startWebLogic.sh

5.11.7.2 Validating the Administration Server

To verify that the Administration Server is properly configured:

  1. In a browser, go to http://vip0:7001/console.

  2. Log in as the administrator.

  3. Verify that the WLS_SOA1 and WLS_SOA2 managed servers are listed.

  4. Verify that the SOA_Cluster cluster is listed.

  5. Verify that you can access Enterprise Manager at http://vip0:7001/em.

5.11.7.3 Disabling Host Name Verification for the Administration Server and the WLS_SOA1 Managed Server

This step is required if you have not set up the appropriate certificates for hostname verification between the Administration Server and Node Manager. If SSL is not set up, you receive an error message unless you disable host name verification.

You can re-enable host name verification when you have set up SSL communication between the Administration Server and the Node Manager.

To disable host name verification:

  1. In Oracle WebLogic Server Administration Console, select Servers, and then AdminServer.

  2. Select SSL, and then Advanced.

  3. Set Hostname Verification to None.

  4. In Oracle WebLogic Server Administration Console, select WLS_SOA1, SSL, and then Advanced.

  5. Set Hostname Verification to None.

  6. Save and activate the changes.

  7. Restart the Administration Servers and any Managed Server that is affected.

5.11.7.4 Starting Node Manager on APPHOST1

Perform these steps to start Node Manager on APPHOST1:

  1. Run the setNMProps.sh script, which is located in the ORACLE_COMMON_HOME/common/bin directory, to set the StartScriptEnabled property to true before starting Node Manager

    SOAHOST1> cd ORACLE_COMMON_HOME/common/bin
    SOAHOST1> ./setNMProps.sh
    

    Note:

    You must use the StartScriptEnabled property to avoid class loading failures and other problems.
  2. Start Node Manager:

    WL_HOME/server/bin
    
    APPHOST1> ./startNodeManager.sh
    

5.11.7.5 Using Oracle Coherence for Deploying Composites

Although deploying composites uses multicast communication by default, Oracle recommends using unicast communication for SOA high availability. Use unicast if you disable multicast communication for security reasons.

Note:

An incorrect configuration of the Oracle Coherence framework that is used for deployment may prevent the SOA system from starting. The deployment framework must be properly customized for the network environment on which the SOA system runs. Oracle recommends the following configuration described in this section.

Enabling Communication within Clusters Using Unicast Communication

Multicast communication enables Oracle Fusion Middleware SOA to discover all of the members of a cluster to which to it deploys composites dynamically. However, unicast communication does not enable nodes to discover other cluster members in this way. Consequently, you must specify the nodes that belong to the cluster. You do not need to specify all of the nodes of a cluster, however. You need only specify enough nodes so that a new node added to the cluster can discover one of the existing nodes. As a result, when a new node has joined the cluster, it is able to discover all of the other nodes in the cluster. Additionally, in a configuration where multiple IPs are available in the same node, you must configure Oracle Coherence to use a specific hostname to create Oracle Coherence cluster.

Tip:

To guarantee high availability during deployments of SOA composites, specify enough nodes so that at least one of them is running at any given time.

Specify the nodes using the tangosol.coherence.wka<n> system property, where <n> is a number between 1 and 9. You can specify up to 9 nodes. Start the numbering at 1. This numbering must be sequential and must not contain gaps. In addition, specify the hostname used by Oracle Coherence to create a cluster through the tangosol.coherence.localhost system property. This hostname should be the virtual hostname used by the SOA servers that maps the corresponding listener addresses (VIP1 and VIP2). Set this property by adding the -Dtangosol.coherence.localhost parameters to the Arguments field of Oracle WebLogic Server Administration Console's Server Start tab (Figure 5-32).

Figure 5-32 Setting the Hostname Using the Start Server Tab of Oracle WebLogic Server Administration Console

Setting the Hostname
Description of "Figure 5-32 Setting the Hostname Using the Start Server Tab of Oracle WebLogic Server Administration Console"

Specifying the hostname

To add the hostname used by Oracle Coherence:

  1. Log into Oracle WebLogic Server Administration Console.

  2. In the Domain Structure window, expand the Environment node.

  3. Click Servers. The Summary of Servers page appears.

  4. Click the name of the server (represented as a hyperlink) in Name column of the table. The settings page for the selected server appears.

  5. Click the Server Start tab (illustrated in Figure 5-32).

  6. Enter the following for WLS_SOA1 and WLS_SOA2 into the Arguments field.

    For WLS_SOA1, enter the following:

    -Dtangosol.coherence.wka1=apphost1vhn1 -Dtangosol.coherence.wka2=apphost2vhn1 -Dtangosol.coherence.localhost=apphost1vhn1 
    

    For WLS_SOA2, enter the following:

    -Dtangosol.coherence.wka1=apphost1vhn1 -Dtangosol.coherence.wka2=apphost2vhn1 -Dtangosol.coherence.localhost=apphost2vhn1
    
  7. Click Save and activate the changes.

  8. This change requires the SOA servers to be restarted.

Note:

The multicast and unicast addresses are different from the ones used by the WebLogic Server cluster for cluster communication. SOA guarantees that composites are deployed to members of a single WebLogic Server cluster even though the communication protocol for the two entities (the WebLogic Server cluster and the groups to which composites are deployed) are different.

5.11.7.6 Running the soa-createUDD.py Script

Before you can start the WLS_SOA1 managed server on APPHOST1, run the soa-createUDD.py script. This scripts transforms the regular JMS destination in highly available and load balanced Uniform Distribute Destinations making the system more scalable and robust.

You run the script using WLST offline:

  1. Set up the environment.

    APPHOST1>. ORACLE_BASE/product/fmw/
    user_projects/domains/soadomain/bin/setDomainEnv.sh
    
  2. Run the soa-createUDD.py script. This script is located at ORACLE_HOME/bin/soa-createUDD.py.

    APPHOST1> WL_HOME/jrockit_160_14_R27.6.4-18/bin/java weblogic.WLST
             ORACLE_HOME/bin/soa-createUDD.py
             --domain_home ORACLE_BASE/product/fmw/user_projects/domains/soadomain/
             --soacluster SOA_Cluster
    
  3. Restart the Administration Server.

  4. Verify that the following modules are listed in the Oracle WebLogic Server Administration Console:

    • SOAJMSModuleUDDs

      Verify this module by first expanding the Services node in the Domain Structure window and then expanding the Messaging node in the Oracle WebLogic Server Administration Console. Select JMS Modules. Select SOAJMSMOduleUDDS (represented as a hyperlink) in the Names column of the table. Click the Subdeployment tab. Click SOAJMSSubDM and then select Targets.

      Verify that the module is targeted to SOAJMSServer_auto_1 and SOAJMSServer_auto_2.

    • UMSJMSSystemResource

      Verify this module by first expanding the Services node in the Domain Structure window and then expanding the Messaging node in the Oracle WebLogic Server Administration Console. Select JMS Modules. Select UMSJMSSystemResource (represented as a hyperlink) in the Names column of the table. Click the Subdeployment tab. Click SOAJMSSubDM and then select Targets.

      Verify that the module is targeted to UMSJMSServer_auto_1 and UMSJMSServer_auto_2.

Note:

Whenever the script is run after domain extension use the --extend true option, for example if the domain is extended to include Oracle BAM.

For example:

MW_HOME/jrockit_160_14_R27.6.4-18/bin/java weblogic.WLST 
MW_HOME/soa/bin/soa-createUDD.py --domain_home MW_HOME/
user_projects/domains/soadomain --bamcluster BAM_Cluster --extend true

5.11.7.7 Starting and Validating the WLS_SOA1 Managed Server

To start up the WLS_SOA1 managed server and check that it is configured correctly:

  1. Start the WLS_SOA1 managed server using Oracle WebLogic Server Administration Console.

  2. When WLS_SOA1 is started, the following URLs become available:

    • http://APPHOST1VHN1:8001/b2b

      • Verify login to B2B console

    • http://APPHOST1VHN1:8001/integration/worklistapp

      • Verify login to worklist console

    • http://APPHOST1VHN1:8001/wsm-pm

      • Verify the policy validator link

5.11.8 Install WebLogic Server and Oracle SOA on APPHOST2

Repeat the procedures for installing WebLogic Server and Oracle SOA for APPHOST2. The directory paths for binary files and domains used when installing new nodes must be exactly the same as those used for first node. If these paths and domains are not exactly the same as those used for the first node, failover does not occur.

5.11.9 Propagating the Domain Configuration to APPHOST2 with pack/unpack Utilities

Follow these steps to propagate the domain configuration to APPHOST2 using Pack/Unpack utilities:

  1. Run the following pack command on APPHOST1 to create a template pack:

    APPHOST1> cd ORACLE_COMMON_HOME/common/bin
    APPHOST1> ./pack.sh -managed=true
    -domain=ORACLE_BASE/product/fmw/user_projects/domains/soadomain/
    -template=soadomaintemplate.jar 
    -template_name=soa_domain_template
    
  2. Run the following command on APPHOST1 to copy the template file created in the previous step to APPHOST2:

    APPHOST1> scp soadomaintemplate.jar
     oracle@node2:ORACLE_COMMON_HOME/common/bin
    
  3. Run the unpack command on APPHOST2 to unpack the propagated template:

    APPHOST2> cd ORACLE_COMMON_HOME/common/bin
    APPHOST2> ./unpack.sh
     -domain=ORACLE_BASE/product/fmw/user_projects/domains/soadomain/
     -template=soadomaintemplate.jar
    

    Note:

    When you run the Fusion Middleware Configuration Wizard after a Pack/unpack procedure, products that already exist in the original domain, and appear in the Select Extension Source screen, are not selected or greyed out.

    Templates created with the pack command are considered user-created templates. They are treated differently from default templates. User-created templates are self-contained and do not contain any information related to the creation of the original domain.

5.11.10 Extracting XEngine Files in the Second Node

To enable B2B's XEngine in the second node, it is required to extract the content of the ZEngine tar manually:

SOAHOST2>cd ORACLE_HOME/soa/thirdparty/edifecs
SOAHOST2>tar xzvf XEngine.tar.gz

5.11.11 Starting the System in APPHOST2

This section describes procedures for starting the system in APPHOST2.

5.11.11.1 Disabling Host Name Verification for the WLS_SOA2 Managed Server

Before you can start and verify the WLS_SOA2 managed server, you need to disable host name verification. You can re-enable it when you have set up SSL communication between the Administration Server and the Node Manager.

To disable host name verification:

  1. In Oracle WebLogic Server Administration Console, select WLS_SOA2, SSL , and then Advanced.

  2. Set Hostname Verification to None.

5.11.11.2 Starting Node Manager on APPHOST2

To start the Node Manager on APPHOST2, repeat the steps from Section 5.11.7.4, "Starting Node Manager on APPHOST1" on APPHOST2.

5.11.11.3 Starting and Validating the WLS_SOA2 Managed Server

To start up the WLS_SOA2 managed server and verify that it is configured correctly:

  1. Start the WLS_SOA2 managed server using Oracle WebLogic Server Administration Console

    When WLS_SOA2 is started, the following URLs become available:

    http://VIP2:8001/b2b

    Verify login to B2B console.

    http://VIP2:8001/integration/worklistapp

    Verify login to worklist console.

    http:/VIP2:8001/wsm-pm

    Verify the policy validator link.

5.11.12 Configuring Oracle HTTP Server for the Administration Server and the WLS_SOAn Managed Servers

Enable Oracle HTTP Server to route to the Administration Server, that contains the WLS_SOAn managed servers, by setting the WebLogicCluster parameter to the list of nodes in the cluster.

  1. Add the following lines to the OHS_HOME/instances/ohs_instance1/config/OHS/ohs1/mod_wl_ohs.conf file:

    # SOA soa-infra app
    <Location /soa-infra>
        SetHandler weblogic-handler
        WebLogicCluster apphost1vhn1:8001,apphost2vhn1:8001
        WLLogFile /tmp/web_log.log
    </Location>
     
    # Worklist
    <Location /integration/>
        SetHandler weblogic-handler
        WebLogicCluster apphost1vhn1:8001,apphost2vhn1:8001
        WLLogFile /tmp/web_log.log
    </Location>
     
    <Location /DefaultToDoTaskFlow/>
        SetHandler weblogic-handler
        WebLogicCluster SOAHOST1VHN2:8001,SOAHOST2VHN1:8001
    </Location>
    
    # B2B
    <Location /b2b>
        SetHandler weblogic-handler
        WebLogicCluster apphost1vhn1:8001,apphost2vhn1:8001
        WLLogFile /tmp/web_log.log
    </Location>
     
    # UMS prefs
    <Location /sdpmessaging/userprefs-ui >
        SetHandler weblogic-handler
        WebLogicCluster apphost1vhn1:8001,apphost2vhn1:8001
        WLLogFile /tmp/web_log.log
    </Location>
    
    #Required if attachments are added for workflow tasks
     <Location /ADFAttachmentHelper> 
        SetHandler weblogic-handler 
        WebLogicCluster SOAHOST1VHN2:8001,SOAHOST2VHN1:8001 
     </Location>
    
    # SOA composer application 
     <Location /soa/composer> 
         SetHandler weblogic-handler 
         WebLogicCluster SOAHOST1VHN2:8001,SOAHOST2VHN1:8001 
     </Location>
    
  2. Restart Oracle HTTP Server on WEBHOST1:

    WEBHOST1> OHS_HOME/instances/ohs_instance1/bin/opmnctl restartproc ias-component=ohs1
    

5.11.12.1 Validating Access through Oracle HTTP Server

Verify the following URLS to ensure that appropriate routing and failover is working from the HTTP Serer to the SOA_Cluster.

  1. While WLS_SOA2 is running, stop WLS_SOA1 from Oracle WebLogic Server Administration Console.

  2. Access the following URLs and verify the appropriate functionality:

    • WebHost1:7777/soa-infra

    • WebHost1:7777/b2b

    • WebHost1:7777/integration/worklistapp

    • WebHost1:7777/sdpmessaging/userprefs-ui

    • WebHost1:7777/wsm-pm

  3. Start WLS_SOA1 from Oracle WebLogic Server Administration Console.

  4. Stop WLS_SOA2.

  5. Access the following URLs and verify the appropriate functionality:

    • WebHost1:7777/soa-infra

    • WebHost1:7777/b2b

    • WebHost1:7777/integration/worklistapp

    • WebHost1:7777/sdpmessaging/userprefs-ui

5.11.12.2 Configuring JMS Persistence Store as Shared Across the Servers

Configure the location for all persistence stores to a directory visible from both nodes. Change all persistent stores to use this shared base directory.

From the Fusion Middleware Administration Console, select Services, Persistence Store, Persistence_Store_Name, and then Directory.

To enable resume of pending JMS messages, you must specify a location on a persistent storage solution (NAS, SAN) that is available to other servers in the cluster. Therefore, the directory that you enter must be accessible by both WLS_SOA1 and WLS_SOA2. This directory must exist before the server is restarted.

5.11.12.3 Configuring a Default Persistent Store for Transaction Recovery

Each server has a transaction log, which stores information about committed transactions coordinated by the server that may not have been completed. WebLogic Server uses the transaction log when recovering from system crashes or network failures. To take advantage of the migration capability of the Transaction Recovery Service for servers in a cluster, you must store the transaction log in a location that is available to a server and its backup servers, preferably on a dual-ported SCSI disk, or a Storage Area Network (SAN). To configure the default persistent store:

  1. In the Domain Structure tree, expand Environment and select Servers.

  2. Select the server you want to modify.

  3. Select the Configuration, Services tab.

  4. Under Default Store, in the Directory field, enter the path to the folder where you want the default persistent store to store its data files.

    To enable migration of the Transaction Recovery Service, you must specify a location on a persistent storage solution that is available to other servers in the cluster. Therefore, the directory that you enter must be accessible by both WLS_SOA1 and WLS_SOA2. This directory must exist before the server is restarted.

5.11.13 Setting the Front End HTTP Host and Port

You need to set the front end HTTP host and port for Oracle WebLogic Server cluster:

  1. In the WebLogic Server Administration Console, in the Change Center section, click Lock & Edit.

  2. In the left pane, select Environment > Clusters.

  3. Select the WLS_SOA cluster.

  4. Select HTTP.

  5. Set the values for the following:

    • Frontend Host: webhost1

      Frontend HTTP Port: 7777

    • Frontend HTTPS Port: leave it blank

    Note:

    Make sure this address is correct and available (the hTTP server is up). An incorrect value, for example, http:// in the address, or trailing / in the hostname, may prevent the SOA system from being accessible even when using the virtual IPs to access it.
  6. Click Save.

  7. To activate the changes, click Activate Changes in the Change Center section of the Administration Console.

  8. If you have started the server before, notice this change requires a restart of the participants in the cluster.

    Note:

    The SOA system calculates the callback URL as follows:
    • If a request to SOA originates from an external or internal service is originating a request to SOA, then SOA uses the callback URL specified by the client.

    • If a request to an external or internal service originates from SOA, the callbackURL cannot be populated in the SOA request dynamically because SOA is the originator. Instead, callbackServerURL is used if it is specified as a binding property for the specific reference. (You can set this when modeling the composite or at runtime using the MBeans accessed through Oracle Enterprise Manager console.) This allows different service calls to have different callback URLs. That is, a callback URL from an external service is different than one to an internal service.

      However, if the callbackServer URL is not specified as a binding property for that reference, then the system uses the callback URL as specified in soa-infra-config. If the callback URL is not specified in soa-infra-config, then the system uses the callback URL as the front end host specified in WLS. If the front end host is not specified in WLS, the system uses the callback URL as the local host name as provided by WLS MBean APIs.

For SOA high availability installations frontended by Oracle HTTP Server, monitoring should be done on the on the Oracle HTTP Server ports of the real backend servers. This is the case when a deployment is using all the components deployed to the SOA Managed Server. A simple HTTP monitor that pings the HTTP/HTTPS port and expects a pre-determined response in turn should suffice. If only a specific SOA component is being used (such as B2B), then a monitor that does a deeper level check all the way to the Managed server can be considered to validate the health of the component in use. Please check with your load balancer vendor on setting up the HTTP monitors with your load balancer.

If you do not set the front end HTTP host and port, you get the following message when trying to retrieve a document definition XSD from Oracle B2B:

An error occured while loading the document definitions.
java.lang.IllegalArgumentException: Cluster address must be set when
clustering is enabled.

5.11.14 Deploying Applications

You can deploy SOA composite applications from Oracle Enterprise Manager Fusion Middleware Control Console with the Deploy SOA Composite wizard. Use the Deploy SOA Composite wizard to deploy any of the following:

  • A new SOA composite application for the first time

  • A new revision (for example, 2.0) alongside an older revision (for example, 1.0) without impacting the latter. The revision deployed last becomes the new default revision of that composite (unless you specify otherwise at a later step during deployment).

  • A bundle (ZIP file) containing multiple SOA composite application revisions (for example, revisions 2.0, 3.0, and 4.0) of a SOA composite application that already has a different revision that is currently deployed (for example, 1.0). This option enables you to deploy revisions 1.0, 2.0, 3.0, and 4.0 at the same time. The bundle can also contain revisions of different composites. There is no restriction that all revisions must be of the same composite application.

Deployment extracts and activates the composite application in SOA Service Infrastructure. Once an application is deployed, you can perform administration tasks, such as creating instances, configuring properties, monitoring performance, managing instance life cycles, and managing policies and faults.

Note:

If you want to redeploy an existing revision of an application, do not use this wizard. Instead, use the Redeploy SOA Composite wizard.

To deploy applications:

  1. Access the Deploy SOA Composite wizard through one of the following options:

    From SOA Infrastructure Menu... From the SOA Folder in the Navigator... From the SOA Infrastructure Home Page... From the SOA Composite Menu...
    1. Select SOA Deployment > Deploy.
    1. Right-click soa-infra.
    2. Select SOA Deployment > Deploy.

    1. Click the Deployed Composites tab.
    2. Above the Composite table, click Deploy.

    1. Select SOA Deployment > Deploy Another Composite.

    The Select Archive page appears.

    Description of sca_deploy.gif follows
    Description of the illustration sca_deploy.gif

  2. In the Archive or Exploded Directory section, specify the archive of the SOA composite application to deploy. The archive contains the project files of the composite to be deployed (for example, HelloWorld_rev1.0.jar for a single archive or OrderBooking_rev1.0.zip for multiple archives).

  3. In the Configuration Plan section, optionally specify the configuration plan to include with the archive. The configuration plan enables you to define the URL and property values to use in different environments. During process deployment, the configuration plan is used to search the SOA project for values that must be replaced to adapt the project to the next target environment.

  4. Click Next.

    The Select Target page appears.

  5. Select the WebLogic Server or cluster to which to deploy the SOA composite application archive. You can deploy to multiple servers and clusters.

  6. Click Next.

    The Confirmation page appears.

  7. Review your selections.

  8. Select whether or not to deploy the SOA composite application as the default revision. The default revision is instantiated when a new request comes in.

  9. Click Deploy.

    Processing messages are displayed.

  10. When deployment has completed, click Close.

See Also:

Oracle Fusion Middleware Developer's Guide for Oracle SOA Suite for instructions on creating configuration plans and deploying applications from Oracle JDeveloper

5.11.15 Configuring Server Migration for the WLS_SOA Servers

The high availability architecture for a SOA system uses server migration to protect some singleton services against failures. For more information on Whole Server Migration see Oracle Fusion Middleware Using Clusters for Oracle WebLogic Server.

The WLS_SOA1 managed server is configured to be restarted on APPHOST2 in case of failure, and the WLS_SOA2 managed server is configured to be restarted on APPHOST1 in case of failure. For this configuration the WLS_SOA1 and WLS_SOA2 servers listen on specific floating IPs that are failed over by WLS Server Migration. To configure server migration for the WLS_SOAn managed servers, follow these steps:

Step 1   Set up user and tablespace for the Server Migration leasing table
  1. Create a tablespace called leasing.

    Example: Log on to SQL*Plus as the sysdba user and run the following command:

    SQL> create tablespace leasing
            logging datafile 'DB_HOMEs/oradata/orcl/leasing.dbf'
            size 32m autoextend on next 32m maxsize 2048m extent management local;
    
  2. Create a user named leasing and assign to it the leasing tablespace.

    SQL> create user leasing identified by welcome1;
    
    SQL> grant create table to leasing;
    
    SQL> grant create session to leasing;
    
    SQL> alter user leasing default tablespace leasing;
    
    SQL> alter user leasing quota unlimited on LEASING;
    
  3. Create the leasing table using the leasing.ddl script.

    1. Copy the leasing.ddl file, located in the WL_HOME/server/db/oracle/817 or WL_HOME/server/db/oracle/920 directories, to your database node.

    2. Connect to the database as the leasing user.

    3. Run the leasing.ddl script in SQL*Plus.

      SQL> @copy_location/leasing.ddl;
      
Step 2   Create a Multi Data Source from Oracle WebLogic Server Administration Console

The second step is to create a multi data source for the leasing table from the Oracle WebLogic Server Administration Console:

You create a data source to each of the Oracle RAC database instances during the process of setting up the multi data source, both for these data sources and the global leasing multi data source. When you create a data source:

  • Make sure that this is a non-xa data source.

  • The names of the multi data sources are in the following format: <MultiDS>-rac0, <MultiDS>-rac1

  • Use Oracle's Driver (Thin) Version 9.0.1, 9.2.0, 10, 11.

  • Use Supports Global Transactions, One-Phase Commit, and specify a service name for your database.

  • Target these data sources to the SOA cluster.

  • Make sure the datasources' connection pool initial capacity is set to 0. To do this, select Services, JDBC, and then Datasources. In the Datasources screen, click the Datasource Name, then click the Connection Pool tab, and enter 0 in the Initial capacity field.

To create a multi data source:

  1. From Domain Structure window in the Oracle WebLogic Server Administration Console, expand the Services node, then expand the JDBC node.

  2. Click Multi Data Sources. The Summary of JDBC Multi Data Source page appears.

  3. Click New. The Create a New JDBC Multi Data Source page appears.

  4. Click Lock and Edit.

  5. Enter leasing as the Name.

  6. Enter jdbc/leasing as the JNDI name.

  7. Select Failover as algorithm (default).

  8. Click Next.

  9. Select non-XA driver (the default).

  10. Click Next.

  11. Click Create New Data Source.

  12. Enter leasing-rac0 as name. Enter jdbc/leasing-rac0 as JNDI name. Enter oracle as the database type. For the driver type, enter Oracle Driver (Thin) for RAC server-Instance connection Version 10,11.

  13. Click Next.

  14. Deselect Supports Global Transactions.

  15. Click Next.

  16. Enter the service name, database name, host port, and password for your leasing schema.

  17. Click Next.

  18. Click Test Configuration and verify the connection works.

  19. Target the data source to the SOA cluster.

  20. Select the data source and add it to the right screen.

  21. Click Create a New Data Source and repeat the steps for the second instance of your RAC database.

  22. Add the second data source to your multi data source.

  23. Save and Activate the changes.

Step 3   Edit the Node Manager's properties file

For information on Node manager and Whole Server Migration, see Section 3.9, "Whole Server Migration."

The nodemanager.properties file is located in the WL_HOME/common/nodemanager directory. For server migration to work properly, you need to add the properties listed in this section:

  • Interface=eth0

    Note:

    Do not specify the sub interface, such as eth0:1 or eth0:2. This interface is to be used without the :0, or :1. The Node Manager's scripts traverse the different :X enabled IPs to determine which to add or remove. For example, the valid values in Linux environments are eth0, eth1, or, eth2, eth3, ethn, depending on the number of interfaces configured.

    This property specifies the interface name for the floating IP (eth0, for example).

    Be sure that the interface provided is the public interface for this node. On multi home nodes, this interface should be the one on which the floating IP can be enabled.

    Note:

    For Windows, the Interface should be set to the Network Interface Name. For example: Interface="Local Area Connection".
  • NetMask=255.255.255.0

    This property specifies the net mask for the interface for the floating IP. The netmask provided (255.255.255.0) is just an example. The actual value depends on your network.

  • UseMACBroadcast=true

    This property specifies whether or not to use a node's MAC address when sending ARP packets, that is, whether or not to use the -b flag in the arping command.

After starting Node Manger, verify in Node Manager's output (shell where Node Manager is started) that these properties are being used, or problems may arise during migration. You should see something like this in the Node Manager's output:

StateCheckInterval=500

Interface=eth0

NetMask=255.255.255.0

UseMACBroadcast=true

Note:

The steps in this section are not required if the server properties (start properties) have been properly set and the Node Manager can start the servers remotely.
  1. Set the following property in the nodemanager.properties file.

    • StartScriptEnabled

      Set this property to true.

  2. Start the Node Manager on Node 1 and Node 2 by running the startNodeManager.sh script located in the WL_HOME/server/bin directory.

  3. Validate the changes to the nodemanager.properties file by checking the nodemanager.log file.

Step 4   Set environment and superuser privileges for the wlsifconfig.sh script
  1. Grant sudo privilege to the WebLogic user ('oracle') with No Password restriction, and grant execute privilege on /sbin/ifconfig and /sbin/arping binaries.

    Make sure the script is executable by the WebLogic user ('oracle'). The following is an example of an entry inside /etc/sudoers granting sudo execution privilege for oracle and also over the ifconfig and arping:

    oracle ALL=NOPASSWD: /sbin/ifconfig,/sbin/arping
    
Step 5   Configure Server Migration Targets

Configuring Cluster Migration sets the DataSourceForAutomaticMigration property to true. Follow the steps in this section to configure cluster migration in a migration in a cluster:

  1. Log into Oracle WebLogic Server Administration Console

  2. In the left pane, expand Environment and select Clusters.

  3. Select the cluster for which you want to configure migration (SOA_Cluster).

  4. Click Migration.

  5. In the Available field, select the machine to which to allow migration and click the right arrow. In this case, select APPHOST1 and APPHOST2.

  6. Select the data source to be used for automatic migration. In this case select the leasing data source.

  7. Click Save.

  8. Set the Candidate Machines for Server Migration. This needs to be done for all the managed servers.

    To do this, follow these steps:

    1. In the left pane of Oracle WebLogic Server Administration Console, expand Environment and select Servers.

    2. Select the server for which you want to configure migration.

    3. Click the Migration tab.

    4. In the Available field, located in the Migration Configuration section, select the machines to which to allow migration and click the right arrow. For WLS_SOA1, select APPHOST2. For WLS_SOA2, select APPHOST1.

    5. Select the checkbox for Automatic Server Migration Enabled. This enables the Node Manager to start a failed server on the target node automatically.

    6. Click Save and activate the changes.

Step 6   Test Server Migration

To verify that Server Migration is working properly, follow these steps:

From Node 1:

  1. Force stop the WLS_SOA1 managed server.

    To do this, run this command:

    APPHOST1> kill -9 <pid>
    

    pid specifies the process ID of the managed server. You can identify the pid in the node by running this command:

    APPHOST1> ps -ef | grep WLS_SOA1
    

    Note:

    For Windows, the Managed Server can be terminated by using the taskkill command. For example:
    taskkill /f /pid <pid>
    

    Where <pid> is the process Id of the Managed Server.

    To determine the process Id of Managed server run the following command and identify the pid of the WLS_SOA Managed Server.

    MW_HOME\jrockit_160_14_R27.6.4-18\bin\jps -l -v
    
  2. Watch the Node Manager console: you should see a message indicating that WLS_SOA1's floating IP has been disabled.

  3. Wait for the Node Manager to try a second restart of WLS_SOA1. Node Manager waits for a fence period of 30 seconds before trying this restart.

  4. Once Node Manager restarts the server, stop it again. Now Node Manager should log a message indicating that the server will not be restarted again locally.

From Node2:

  1. Watch the local Node Manager console. After 30 seconds since the last try to restart WLS_SOA1on Node 1, Node Manager on Node 2 should prompt that the floating IP for WLS_SOA1 is being brought up and that the server is being restarted in this node.

  2. Access the soa-Service Infrastructure console in the same IP.

Migration can also be verified in Oracle WebLogic Server Administration Console:

  1. Log into Oracle WebLogic Server Administration Console.

  2. Click on Domain on the left pane.

  3. Click on the Monitoring tab and then on the Migration sub-tab.

    The Migration Status table provides information on the status of the migration.

    Figure 5-33 Migration Status Screen in Oracle WebLogic Server Administration Console

    Migration Status screen in the Administration Console

Migration Status screen in Oracle WebLogic Server Administration Console

Note:

On Windows, when you manually shut down multiple servers at the same time, on the same machine, and then, on another machine, attempt to start one of the servers that was shut down, the IP bind may not work. This happens because the original machine still has claim to the IP address, even though netsh has reported that the IP address has been removed.

To resolve this, you must check the network configuration either by using the ipconfig utility or Windows Network Configuration. Either of these may show that one of the virtual/floating IP addresses is still configured even though the servers have been shut down. You can then use Windows Network Configuration to remove the IP address using the following procedure:

  1. From Windows Control Panel, select Network Connections.

  2. Select the appropriate network interface, right-click, and select Properties.

  3. Select Internet Protocol (TCP/IP) and click the Properties button.

  4. Select Advanced.

  5. Select the appropriate IP address and click the Remove button.

5.11.16 Scaling the Topology

You can scale out or scale up the enterprise topology. When you "scale up" the topology, you add new managed servers to nodes that are already running one or more managed servers. When you "scale out" the topology, you add new managed servers to new nodes.

5.11.16.1 Scaling Up the Topology (Adding Managed Servers to Existing Nodes)

In this case, you already have a node that runs a managed server configured with SOA components. The node contains a Middleware home, an Oracle HOME (SOA) and a domain directory for existing managed servers.

You can use the existing installations (the Middleware home, and domain directories) for creating new WLS_SOA servers. There is no need to install SOA binaries in a new location, or to run pack and unpack.

Follow these steps for scaling up the topology:

  1. Using the Administration Console, clone WLS_SOA1 into a new managed server. The source managed server to clone should be one that already exists on the node where you want to run the new managed server.

    To clone a managed server:

    1. Select Environment -> Servers from the Administration Console.

    2. Select the managed server that you want to clone (for example, WLS_SOA1).

    3. Select Clone.

    Name the new managed server WLS_SOAn, where n is a number to identify the new managed server.

    The rest of the steps assume that you are adding a new server to APPHOST1, which is already running WLS_SOA1.

  2. For the listen address, assign the host name or IP to use for this new managed server. If you are planning to use server migration as recommended for this server, this should be the VIP (also called a floating IP) to enable it to move to another node. The VIP should be different from the one used by the managed server that is already running.

  3. Create JMS Servers for SOA and UMS on the new managed server.

    1. Use the Oracle WebLogic Server Administration Console to create a new persistent store for the new SOAJMSServer and name it, for example, SOAJMSFileStore_N. Specify the path for the store. This should be a directory on shared storage, as recommended in section Section 5.11.1.3, "Shared Storage Prerequisites":

      Note:

      This directory must exist before the managed server is started or the start operation fails.
      ORACLE_BASE/admin/DOMAIN_NAME/cluster_name/jms/SOAJMSFileStore_N
      
    2. Create a new JMS Server for SOA, for example, SOAJMSServer_N. Use the SOAJMSFileStore_N for this JMSServer. Target the SOAJMSServer_N Server to the recently created Managed Server (WLS_SOAn).

    3. Create a new persistence store for the new UMSJMSServer, for example, UMSJMSFileStore_N Specify the path for the store. This should be a directory on shared storage as recommended in sectionSection 5.11.1.3, "Shared Storage Prerequisites":

      ORACLE_BASE/admin/domain_name/cluster_name/jms/UMSJMSFileStore_N.
      

      Note:

      This directory must exist before the managed server is started or the start operation fails.

      You can also assign SOAJMSFileStore_N as store for the new UMS JMS Servers. For the purpose of clarity and isolation, individual persistent stores are used in the following steps.

    4. Create a new JMS Server for UMS, for example, UMSJMSServer_N. Use the UMSJMSFileStore_N for this JMSServer. Target the UMSJMSServer_N Server to the recently created Managed Server (WLS_SOAn).

    5. Update the SubDeployment targets for the SOA JMS Module to include the recently created SOA JMS Server. To do this, expand the Services node and then expand the Messaging node. Choose JMS Modules from the Domain Structure window of the Oracle WebLogic Server Administration Console. The JMS Modules page appears. Click SOAJMSModuleUDDs (represented as a hyperlink in the Names column of the table). The Settings page for SOAJMSModuleUDDs appears. Click the SubDeployments tab. The SOAJMSSubDM subdeployment appears.

      Note:

      This subdeployment module results from updating the JMS configuration for the first two servers (WLS_SOA1 and WLS_SOA2) with the Uniform Distributed Destination Script (soa-createUDD.py) which is required for the initial HA topology set up.

      Click the SOAJMSSubDM subdeployment. Add the new JMS Server for SOA called SOAJMSServer_N to this subdeployment. Click Save.

    6. Target the UMSJMSSystemResource to the SOA_Cluster as it may have changed during extend operations. To do this, expand the Services node and then expand the Messaging node. Choose JMS Modules from the Domain Structure window of the Oracle WebLogic Server Administration Console. The JMS Modules page appears. Click UMSJMSSytemResource and click the Targets Tab. Make sure all of the servers in the SOA_Cluster appear selected (including the recently cloned WLS_SOAn).

    7. Update the SubDeployment Targets for UMSJMSSystemResource to include the recently created UMS JMS Server. To do this, expand the Services node and then expand the Messaging node. Choose JMS Modules from the Domain Structure window of the Oracle WebLogic Server Administration Console. The JMS Modules page appears. Click UMSJMSSystemResource (represented as a hyperlink in the Names column of the table). The Settings page for UMSJMSSystemResource appears. Click the SubDeployments tab. The UMSJMSSubDMSOA subdeployment appears.

      Note:

      This subdeployment module results from updating the JMS configuration for the first two servers (WLS_SOA1 and WLS_SOA2) with the Uniform Distributed Destination Script (soa-createUDD.py) which is required for the initial HA topology set up.

      Click the SOAJMSSubDM subdeployment. Add the new JMS Server for UMS called UMSJMSServer_N to this subdeployment. Click Save.

  4. Configure TX persistent store for the new server. This should be a location visible from other nodes as indicated in the recommendations about shared storage for the EDG.

    From the Administration Console, select Server_name > Services tab. Under Default Store, in Directory, enter the path to the folder where you want the default persistent store to store its data files.

  5. Start and test the new managed server from the Administration Console.

    1. Shut down the existing managed servers in the cluster.

    2. Ensure that the newly created managed server, WLS_SOAn, is up.

    3. Access the application on the newly created managed server (http://vip:port/soa-infra). The application should be functional.

  6. Configure Server Migration for the new managed server.

    Note:

    Since this is a scale-up operation, the node should already contain a Node Manager and environment configured for server migration. The floating IP for the new SOA managed server should also be already present.

    Configure server migration following these steps:

    1. Log into the Administration Console.

    2. In the left pane, expand Environment and select Servers.

    3. Select the name of the new managed server for which you want to configure migration.

    4. Click the Migration tab.

    5. In the Available field, in the "Migration Configuration" section, select the machines to which to allow migration and click the right arrow. Select the same migration targets as for the servers that already exist on the node.

      For example, for new managed servers on APPHOST1, which is already running WLS_SOA1, select APPHOST2. For new managed servers on APPHOST2, which is already running WLS_SOA2, select APPHOST1.

      Make sure the appropriate resources are available to run the managed servers concurrently during migration.

    6. Select the "Automatic Server Migration Enabled" option. This enables the Node Manager to start a failed server on the target node automatically.

    7. Click Save.

    8. Restart the Administration Server, managed servers, and Node Manager.

  7. Test server migration for this new server. Follow these steps from the node where you added the new server:

    1. Stop the WLS_SOAn managed server.

      To do this, run "kill -9 <pid>" on the PID of the managed server. You can identify the PID of the node using "ps -ef | grep WLS_SOAn".

    2. Watch the Node Manager Console: you should see a message indicating that WLS_SOA1's floating IP has been disabled.

    3. Wait for the Node Manager to try a second restart of WLS_SOAn. Node Manager waits for a fence period of 30 seconds before trying this restart.

    4. Once Node Manager restarts the server, stop it again. Now Node Manager should log a message indicating that the server will not be restarted again locally.

5.11.16.2 Scaling Out the Topology (Adding Managed Servers to New Nodes)

When you scale out the topology, you add new managed servers configured with SOA to new nodes.

Before performing the steps in this section, check that you meet these requirements:

  • There must be existing nodes running managed servers configured with SOA within the topology.

  • The new node can access the existing home directories for WebLogic Server and SOA. (Use the existing installations in shared storage for creating a new WLS_SOA or WLS_WSM managed server. You do not need to install WebLogic Server or SOA binaries in a new location but you do need to run pack and unpack to bootstrap the domain configuration in the new node.)

    Note:

    If there is no existing installation in shared storage, installing WebLogic Server and SOA in the new nodes is required as described in Section 5.11, "Configuring High Availability for Oracle SOA Service Infrastructure and Component Service Engines."

    Note:

    When an ORACLE_HOME or WL_HOME is shared by multiple servers in different nodes, Oracle recommends keeping the Oracle Inventory and Middleware home list in those nodes updated for consistency in the installations and application of patches. To update the oraInventory in a node and "attach" an installation in a shared storage to it, use ORACLE_HOME/oui/bin/attachHome.sh. To update the Middleware home list to add or remove a WL_HOME, edit the user_home/bea/beahomelist file. See the following steps.

Follow these steps for scaling out the topology:

  1. On the new node, mount the existing Middleware home, which should include the SOA installation and the domain directory, and ensure that the new node has access to this directory, just like the rest of the nodes in the domain.

  2. To attach ORACLE_HOME in shared storage to the local Oracle Inventory, execute the following command:

    APPHOSTn>cd ORACLE_BASE/product/fmw/soa/
    APPHOSTn>./attachHome.sh -jreLoc ORACLE_BASE/fmw/jrockit_160_14_R27.6.4-18
    

    To update the Middleware home list, create (or edit, if another WebLogic installation exists in the node) the MW_HOME/bea/beahomelist file and add ORACLE_BASE/product/fmw to it.

  3. Log in to the Oracle WebLogic Administration Console.

  4. Create a new machine for the new node that will be used, and add the machine to the domain.

  5. Update the machine's Node Manager's address to map the IP of the node that is being used for scale out.

  6. Use the Oracle WebLogic Server Administration Console to clone WLS_SOA1 into a new managed server. Name it WLS_SOAn, where n is a number.

    Note:

    These steps assume that you are adding a new server to node n, where no managed server was running previously.
  7. Assign the host name or IP to use for the new managed server for the listen address of the managed server.

    If you are planning to use server migration for this server (which Oracle recommends) this should be the VIP (also called a floating IP) for the server. This VIP should be different from the one used for the existing managed server.

  8. Create JMS servers for SOA and UMS on the new managed server.

    Note:

    These steps are not required for scaling out the WSM_PM managed server, only for WLS_SOA managed servers. They are not required either to scale up the BAM Web Applications system.

    To create JMS servers for SOA and UMS

    1. Use the Oracle WebLogic Server Administration Console to create a new persistent store for the new SOAJMSServer (which is created in a later step) and name it, for example, SOAJMSFileStore_N. Specify the path for the store. This should be a directory on shared storage as recommended in Section 5.11.1.3, "Shared Storage Prerequisites."

      Note:

      This directory must exist before the managed server is started or the start operation fails.
    2. Create a new JMS Server for SOA, for example, SOAJMSServer_N. Use the SOAJMSFileStore_N for this JMSServer. Target the SOAJMSServer_N Server to the recently created managed server (WLS_SOAn).

    3. Create a new persistence store for the new UMSJMSServer, and name it, for example, UMSJMSFileStore_N. Specify the path for the store . This should be a directory on shared storage as recommended in Section 5.11.1.3, "Shared Storage Prerequisites."

      ORACLE_BASE/admin/domain_name/cluster_name/jms/UMSJMSFileStore _N
      

      Note:

      This directory must exist before the managed server is started or the start operation fails.

      Note:

      It is also possible to assign SOAJMSFileStore_N as the store for the new UMS JMS Servers. For the purpose of clarity and isolation, individual persistent stores are used in the following steps.
    4. Create a new JMS Server for UMS: for example, UMSJMSServer_N. Use the UMSJMSFileStore_N for this JMS Server. Target the UMSJMSServer_N Server to the recently created managed server (WLS_SOAn).

    5. Update the SubDeployment targets for the SOA JMS Module to include the recently created SOA JMS Server. To do this, expand the Services node and then expand the Messaging node. Choose JMS Modules from the Domain Structure window of the Oracle WebLogic Server Administration Console. The JMS Modules page appears. Click SOAJMSModuleUDDs (represented as a hyperlink in the Names column of the table). The Settings page for SOAJMSModuleUDDs appears. Open the SubDeployments tab. The SOAJMSSubDM subdeployment appears.

      Note:

      This subdeployment module results from updating the JMS configuration for the first two servers (WLS_SOA1 and WLS_SOA2) with the Uniform Distributed Destination Script (soa-createUDD.py), which is required for the initial high availability topology setup.

      Click the SOAJMSSubDM subdeployment. Add the new JMS Server for SOA called SOAJMSServer_N to this subdeployment. Click Save.

    6. Target the UMSJMSSystemResource to the SOA_Cluster as it may have changed during extend operations. To do this, expand the Services node and then expand the Messaging node. Choose JMS Modules from the Domain Structure window of the Oracle WebLogic Server Administration Console. The JMS Modules page appears. Click UMSJMSSytemResource and open the Targets tab. Make sure all of the servers in the SOA_Cluster appear selected (including the recently cloned WLS_SOAn).

    7. Update the SubDeployment targets for UMSJMSSystemResource to include the recently created UMS JMS Server. To do this, expand the Services node and then expand the Messaging node. Choose JMS Modules from the Domain Structure window of the Oracle WebLogic Server Administration Console. The JMS Modules page appears. Click UMSJMSSystemResource (represented as a hyperlink in the Names column of the table). The Settings page for UMSJMSSystemResource appears. Open the SubDeployments tab. The UMSJMSSubDMSOA subdeployment appears

      Note:

      This subdeployment module results from updating the JMS configuration for the first two servers (WLS_SOA1 and WLS_SOA2) with the Uniform Distributed Destination Script (soa-createUDD.py), which is required for the initial HA topology setup.

      Click the UMSJMSSubDMSOA subdeployment. Add the new JMS Server for UMS called UMSJMSServer_N to this subdeployment. Click Save.

  9. Run the pack command on APPHOST1 to create a template pack as follows:

    APPHOST1> cd ORACLE_COMMON_HOME/common/bin
     
    APPHOST1> ./pack.sh -managed=true -domain= MW_HOME/user_projects/domains/soadomain/
    -template=soadomaintemplateScale.jar -template_name=soa_domain_templateScale
    

    Run the following command on APPHOST1 to copy the template file created to APPHOSTN:

    APPHOST1> scp soadomaintemplateScale.jar oracle@APPHOSTN:/ ORACLE_BASE/product/fmw/soa/common/bin
    

    Run the unpack command on APPHOSTN to unpack the template in the managed server domain directory as follows:

    APPHOSTN> cd ORACLE_BASE/product/fmw/soa/common/bin
     
    APPHOSTN> ./unpack.sh -domain= ORACLE_BASE/product/fmw/user_projects/domains/soadomain/ -template=soadomaintemplateScale.jar
    
  10. Start the Node Manager on the new node. To start the Node Manager, use the installation in shared storage from the existing nodes, and start Node Manager by passing the host name of the new node as a parameter as follows:

    APPHOSTN> WL_HOME/server/bin/startNodeManager <new_node_ip>
    
  11. Disable host name verification for the new managed server. Before starting and verifying the WLS_SOAN managed server, you must disable host name verification. You can re-enable it after you have configured server certificates for the communication between the Oracle WebLogic Administration Server and the Node Manager in APPHOSTn.

    To disable host name verification:

    1. In the Oracle Enterprise Manager Console, select Oracle WebLogic Server Administration Console.

    2. Expand the Environment node in the Domain Structure window.

    3. Click Servers. The Summary of Servers page appears.

    4. Select WLS_SOAn in the Names column of the table.

      The Settings page for server appears.

    5. Click the SSL tab.

    6. Click Advanced.

    7. Set Hostname Verification to None.

    8. Click Save.

  12. Start and test the new managed server from the Oracle WebLogic Server Administration Console:

    1. Shut down all the existing managed servers in the cluster.

    2. Ensure that the newly created managed server, WLS_SOAn, is running.

    3. Access the application on the newly created managed server (http://vip:port/soa-infra). The application should be functional.

  13. Configure the appropriate coherence "well known addresses list" as start parameters for the server as described in Section 5.11, "Configuring High Availability for Oracle SOA Service Infrastructure and Component Service Engines."

  14. Configure server migration for the new managed server.

    Note:

    Since this new node is using an existing shared storage installation the node is already using a Node Manager and an environment configured for server migration that includes netmask, interface, wlsifconfig script superuser privileges. The floating IP for the new SOA Managed Server is already present in the new node.

    To log into the Administration Console and configure server migration:

    1. In the left pane, expand Environment and select Servers.

    2. Select the server (represented as hyperlink) for which you want to configure migration from the Names column of the table. The Setting page for that server appears.

    3. Click the Migration tab.

    4. In the Available field, in the Migration Configuration section, select the machines to which to allow migration and click the right arrow.

      Note:

      Specify the least-loaded machine as the migration target for the new server. The required capacity planning must be completed so that this node has enough available resources to sustain an additional managed server.
    5. Select the Automatic Server Migration Enabled option. This enables the Node Manager to start a failed server on the target node automatically.

    6. Click Save.

    7. Restart the Administration Server, managed servers, and Node Manager.

5.12 Configuring High Availability for Oracle BAM

This section describes how to set up a 2 node Highly Available configuration for BAM. This includes the BAM Server and the BAM Web Applications. The architecture targeted for the configuration steps is as follows:

Note:

Oracle strongly recommends reading the release notes for any additional installation and deployment considerations prior to starting the setup process.

Figure 5-34 represents the example architecture that configuration steps in this section create.

Figure 5-34 Oracle SOA BAM High Availability Architecture

Oracle SOA BAM High Availability Architecture
Description of "Figure 5-34 Oracle SOA BAM High Availability Architecture "

Figure 5-34 illustrates a two-node cluster running Oracle BAM Web Applications. One of them is also running Oracle BAM Server. The WebLogic Servers are front ended by Oracle HTTP Server which loads balances incoming requests to BAM Web Applications. An Oracle RAC database is used for storing metadata and BAM schemas.

5.12.1 Preparing the Environment: Prerequisite Steps Before Setting up a High Availability Configuration for Oracle BAM

The following sections provide prerequisite steps before setting up an Oracle SOA Service Infrastructure high availability configuration with Oracle BAM:

5.12.1.1 Database Prerequisites

Oracle SOA Service Infrastructure supports the following database versions:

  • Oracle Database 10g Release 2 (10.2.x)

  • Oracle Database 11g Release 1 (11.1.x)

To determine the database version, execute this query:

SQL>select version from sys.product_component_version where product like 'Oracle%';

5.12.1.2 VIP and IPs Prerequisites

As shown in Table 5-7, you configure the Administration and the BAM Server in APPHOST1 to listen on two different virtual IPs, as shown in the architecture diagram. This requires the provisioning of the corresponding VIP in the box and related host names in the DNS system in your network. Make sure that the different VIPS are available and are reachable before running the installation.

Table 5-7 Virtual Hosts

Virtual IP VIP Maps to... Description

VIP0

APPHOST1VHN0

APPHOST1VHN0 is the virtual host name that is the listen address for the Administration Server and fails over with manual failover of the Administration Server. It is enabled on the node where the Administration Server process is running (APPHOST1 by default).

VIP1

APPHOST1VHN1

APPHOST1VHN1 is the virtual host name that maps to the listen address for WLS_BAM1 and fails over with server migration of this managed server. It is enabled on the node where WLS_BAM1 process is running (APPHOST1 by default).


5.12.1.3 Installing and Configuring the Database Repository

This section describes how to install and configure the database repository.

Oracle Clusterware

  • For 10g Release 2 (10.2), see the Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide.

  • For 11g Release 1 (11.1), see Oracle Clusterware Installation Guide.

Automatic Storage Management

  • For 10g Release 2 (10.2), see Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide.

  • For 11g Release 1 (11.1), see Oracle Clusterware Installation Guide.

  • When you run the installer, select the Configure Automatic Storage Management option in the Select Configuration page to create a separate Automatic Storage Management home.

Oracle Real Application Clusters

Database Initialization Parameters

Ensure that the following initialization parameter is set to the required value. It is checked by Repository Creation Assistant.

Table 5-8 Required Initialization Parameters

Parameter Required Value Parameter Class

PROCESSES

300 or greater

Static


To check the value of the initialization parameter using SQL*Plus, you can use the SHOW PARAMETER command.

As the SYS user, issue the SHOW PARAMETER command as follows:

SQL> SHOW PARAMETER processes

Set the initialization parameter using the following command:

SQL> ALTER SYSTEM SET processes=300 SCOPE=SPFILE

Restart the database.

Note:

The method that you use to change a parameter's value depends on whether the parameter is static or dynamic, and on whether your database uses a parameter file or a server parameter file. See the Oracle Database Administrator's Guide for details on parameter files, server parameter files, and how to change parameter values.

5.12.1.4 Using Oracle Fusion Middleware Repository Creation Utility to Load Oracle Fusion Middleware schemas

Install the 11g (11.1.1) Oracle Fusion Middleware Metadata Store and SOA schemas into a Real Application Cluster database before you install Oracle Fusion Middleware SOA components. Oracle Fusion Middleware provides a tool, Oracle Fusion Middleware Repository Creation Utility (RCU), to create the component schemas in an existing database. See Oracle Fusion Middleware Installation Concepts and Repository Creation Utility User's Guide for more information about installing RCU.

5.12.1.4.1 Running RCU

Run RCU to install the required metadata for Oracle Fusion Middleware 11g.

  1. Start up RCU using the following command:

    RCU_HOME/bin/rcu &
    
  2. In the Welcome screen, click Next.

  3. In the Create Repository screen, select Create to load component schemas into a database, click Next.

  4. In the Database Connection Details screen, enter connection information for your database:

    • Database Type: Select Oracle Database.

    • Host Name: Enter the name of the node that is running the database. For an Oracle RAC database, specify the VIP name, or one of the node names as the host name: SOADBHOST1VIRTUAL.

    • Port: The port number for the database: 1521

    • Service Name: Enter the service name of the database: soaha.mycompany.com

    • Username: SYS

    • Password: Enter the password for the SYS user.

    • Role: SYSDBA

  5. Click Next.

  6. If you receive the following warning message:

    The database you are connecting is with non-UTF8 charset, if you are going to use this DB for multilingual support, you may have data loss. If you are not using for multilingual support you can continue, otherwise we strongly recommend using UTF-8 database.

  7. Click Ignore or Stop.

  8. In the Select Components screen, select Create a New Prefix, and enter a prefix to use for the database schemas, for example, SOAHA

    Write down the schema names so they are available in later procedures.

    • Select the following schemas:

      • AS Common Schemas:

        Metadata Services

      • SOA Infrastructure:

        Business Activity Monitoring

        User Messaging

  9. Click Next.

  10. In the Schema Passwords screen, enter passwords for the main and additional (auxiliary) schema users, and click Next.

  11. In the Map Tablespaces screen, choose the tablespaces for the selected components, and click Next.

  12. In the Summary screen, click Create.

  13. In the Completion Summary screen, click Close.

See Oracle Fusion Middleware Installation Concepts and Repository Creation Utility User's Guide for more information about using RCU.

5.12.2 Installing Oracle HTTP Server on WebHost1

To install Oracle HTTP Server on WEBHOST1:

  1. Verify that the servers meet the following requirements:

    • The system, patch, kernel, and other requirements meet the requirements specified in the Oracle Fusion Middleware Installation Guide for Oracle SOA Suite.

    • Port 7777 is not used by any service on the nodes. You can verify this by running the following command:

      Unix:

      netstat -an | grep LISTEN | grep ":7777"
      

      Windows:

      netstat -an | findstr "LISTEN" | findstr ":7777"
      

    If the ports are in use, make them available.

  2. On UNIX platforms, if the /etc/oraInst.loc or /var/opt/oracle/oraInst.loc file exists, check that its contents are correct. Specifically, check that the inventory directory is correct, and that you have write permissions for that directory.

    If the /etc/oraInst.loc file does not exist, skip this step.

  3. Start Oracle Universal Installer for Oracle Fusion Middleware 11g Web Tier Utilities CD installation as follows:

    For UNIX, run this command: runInstaller.

    For Windows, double-click setup.exe.

  4. In the Welcome screen, click Next.

  5. In the Select Installation Type screen, select Install and Configure, and click Next.

  6. In the Prerequisite Checks screen, ensure that all the prerequisites are met, and click Next.

  7. In the Specify Installation Location screen, set the location to:

    /u01/app/oracle/product/11.1.1/ohs_1

  8. Click Next.

  9. In the Configure Components screen:

    • Select Oracle HTTP Server.

    • Do not select Associate Selected Components with WebLogic Domain.

    • Click Next.

  10. In the Specify Component Details screen, enter the following values:

    • Instance Home Location: corrected value. See bug 8199458

    • Instance Home Location: /u01/app/oracle/product/11.1.1/ohs_1/instances/ohs_instance1

    • Instance Name: ohs_instance1

    • OHS Component Name: ohs1

  11. Click Next.

  12. In the Configure Ports screen, do the following:

    • Select Specify Ports using Configuration File and copy the staticports.ini template file from your installation disk (the file is located in the /Disk1/stage/Response directory) to your user's home. Then use the Browse button to select this file.

    • Click View/Edit File to open the staticports.ini file in an editor.

    • Change the Oracle HTTP Server port in that file to 7777.

    • Save the file.

  13. Click Next.

  14. In the Configuration Summary screen, ensure that the selections are correct, and click Install.

  15. In the Installation Progress screen:

    For UNIX systems, a dialog box appears prompting you to run the oracleRoot.sh script. Open a command window and run the script, following the prompts.

    Click Next.

  16. In the Configuration screen, several configuration assistants are launched in succession. When the configuration assistants are finished, the Configuration Completed screen appears.

  17. In the Configuration Completed screen, click Finish to exit.

5.12.2.1 Validating Oracle HTTP Server

To verify that Oracle HTTP Server is set up correctly, access the root URL context of the server by using the following URL a browser:

HTTP://WebHost1:7777/

If Oracle HTTP Server is set up correctly, the Hello World page appears in the browser.

5.12.3 Installing Oracle Fusion Middleware Home

This section describes how to install Oracle Fusion Middleware on all nodes in the application tier that runs Oracle WebLogic Server and Oracle Fusion Middleware SOA Suite.

Install the following Oracle Fusion Middleware components:

5.12.3.1 Installing Oracle WebLogic Server

To install Oracle WebLogic Server on all nodes in the application tier:

  1. Start Oracle WebLogic Server Installer

    On UNIX (Linux in the following example):

    APPHOST1> wls1032_linux32.bin
    

    On Windows:

    APPHOST1> wls1032_win32.exe
    
  2. In the Welcome screen, click Next.

  3. In the Choose Middleware Home Directory screen:

    • Select Create a New Middleware Home

    • For the Middleware Home Directory field, enter MW_HOME

    • Click Next.

  4. In the Register for Security Updates screen, enter your contact information for security update notifications, and click Next.

  5. In the Choose Install Type screen, select Custom, and click Next.

  6. In the Choose Products and Components screen, click Next.

  7. In the JDK Selection screen, select only Oracle JRockit 1.6.0_14 SDK, and click Next.

  8. In the Choose Product Installation Directories screen, accept the following directory:

    WL_HOME

    Click Next.

  9. In the Installation Summary screen, click Next.

  10. In the Installation Complete screen, deselect Run QuickStart, and click Done.

5.12.3.2 Installing Oracle Fusion Middleware for Oracle SOA

To install Oracle Fusion Middleware for Oracle SOA on all the nodes in the application tier:

  1. On Linux platforms, if the /etc/oraInst.loc file exists, check that its contents are correct. Specifically, check that the inventory directory is correct and that you have write permissions for that directory.

    If the /etc/oraInst.loc file does not exist, you can skip this step.

  2. Start Oracle Fusion Middleware 11g Oracle SOA Suite Installer:

    On UNIX:

    APPHOST1> runInstaller
    

    On Windows:

    APPHOST1> setup.exe
    

    When Oracle Fusion Middleware 11g Oracle SOA Suite Installer prompts you for a JRE/JDK location enter Oracle SDK location created in Oracle WebLogic Server installation, for example, MW_HOME/jrockit_160_14_R27.6.4-18.

  3. In the Welcome screen, click Next.

  4. In the Prerequisite Checks screen, verify that the checks complete successfully, and click Next.

  5. In the Specify Installation Location screen:

    • For Middleware Home, enter MW_HOME

    • For Oracle Home Directory, enter soa

    Click Next.

  6. In the Installation Summary screen, click Install.

  7. In the Installation Complete screen, click Finish.

5.12.4 Enabling VIP0 and VIP1 on APPHOST1

For information about configuring virtual IPs for the Administration Server and configuring the Administration Server for high availability, see Section 10.2.2.3, "Transforming the Administration Server for Cold Failover Clusters."

5.12.5 Running Oracle Fusion Middleware Configuration Wizard on APPHOST1 to Create the WebLogic Server Oracle BAM Domain

Run Oracle Fusion Middleware Configuration Wizard from the SOA home directory to create a domain containing the Administration Server and Oracle SOA components. Ensure that the database where you installed the repository is running. For RAC databases, all the instances must be running.

  1. Change the directory to the location of Oracle WebLogic Server Configuration Wizard, located in the SOA home directory:

    APPHOST1> cd ORACLE_HOME/common/bin
    
  2. Start Oracle WebLogic Server Configuration Wizard

    For UNIX:

    APPHOST1> ./config.sh
    

    For Windows:

    APPHOST1> config.cmd
    
  3. In the Welcome screen, select Create a New WebLogic Domain, and click Next.

  4. In the Select Domain Source screen, select Generate a domain configured automatically to support the following products, and select the following products:

    • Oracle Business Activity Monitoring

    • Oracle WSM-PM (selected by default)

    • Enterprise Manager

    • JRF (selected by default)

    Click Next.

  5. In the Specify Domain Name and Location screen, make the following entries:

    • Domain Name: bamdomain

    • Domain Location: accept the default

    • Application Location: accept the default

    Click Next.

  6. In the Configure Administrator Username and Password screen, enter the username and password to be used for the domain's administrator, and click Next.

  7. In the Configure Server Start Mode and JDK screen, make the following selections:

    • WebLogic Domain Startup Mode: select Production Mode

    • JDK Selection: select Oracle SDK1.6.0_05.

    Click Next.

  8. In the Configure JDBC Component Schema screen:

    1. Ensure that the following data source appears on the screen. The user names shown in Table 5-9, assume that soaha was used as the prefix for schema creation from RCU.

      Table 5-9 Configuring Values for Data Sources

      Data Source User Name

      BAMDataSource

      soaha_orabam

      OraSDPMDataSource

      soaha_orasdpm

      mds-owsm

      soaha_mds


    2. Select Configure selected component schemas as RAC multi data source schemas in the next panel.

    3. Click Next.

  9. In the Configure RAC Multi Data Sources screen enter the following:

    1. With all schemas selected enter values for the following fields, specifying the connect information for the Oracle RAC database that was seeded with RCU.

      Driver: Oracle driver (Thin) for RAC Service-Instance connections, Versions:10, 11.

      Service Name: Enter the service name of the database, for example, soaha.mycompany.com.

      Username: Enter the complete user name (including the prefix) for the schemas.

      Password: Enter the password to use to access the schemas.

    2. Enter the host name, instance name, and port.

    3. Click Add.

    4. Repeat for each Oracle RAC instance.

    5. Select the BAM Schema Only and enter the User Name and Password (soaha_orabam/passwd).

    6. Select the User Messaging Service Schema Only and enter the User Name and Password (soaha_orasdpm/passwd)

    7. Select the OWSM MDS Schema Only and enter the User Name and Password (soaha_mds/passwd)

    8. Click Next.

  10. In the Test JDBC Data Sources screen, the connections are tested automatically. The Status column displays the results. Ensure that all connections were successful. If not, click Previous to return to the previous screen and correct your entries.

    Click Next when all the connections are successful.

  11. In the Select Optional Configuration screen, select Administration Server, Managed Servers, Cluster and Machines and Deployment and services, and click Next.

  12. In the Configure the Administration Server screen, enter the following values:

    • Name: AdminServer

    • Listen Address: VIP0

    • Listen Port: 7001

    • SSL listen port: N/A

    • SSL enabled: leave unchecked

    Click Next.

  13. In the Configure Managed Servers screen, add the following managed servers:

    Table 5-10 Configuring Managed Servers

    Name Listen Address Listen Port SSL Listen Port SSL Enabled

    WLS_BAM1

    APPHOST1VHN1

    8001

    n/a

    No

    WLS_BAM2

    APPHOST2 (BAM Server WLS_BAM2 does not use server migration)

    8001

    n/a

    No


    Note:

    Although the standard recommendation is to run custom applications and other systems in a separate WebLogic Managed Server, the creation of custom WLS managed servers described in Figure 5-34 is not addressed here.

    Click Next.

  14. In the Configure Clusters screen, add the following cluster:

    • Name: BAM_Cluster

    • Cluster Messaging Mode: unicast

    • Multicast Address: N/A

    • Multicast Port: N/A

    • Cluster Address: Leave empty

    Click Next.

  15. In the Assign Servers to Clusters screen, assign the following servers to BAM_Cluster:

    • WLS_BAM1

    • WLS_BAM2

    Click Next.

  16. In the Configure Machines screen:

    • Delete the LocalMachine that appears by default.

    • Click the Unix Machine tab, and add the following machines:

    Table 5-11 Configuring Machines

    Name Node Manager Listen Address

    APPHOST1

    APPHOST1

    APPHOST2

    APPHOST2


    Leave all other fields to their default values, and click Next.

  17. In the Assign Servers to Machines screen, assign servers to machines as follows:

    • APPHOST1: AdminServer, WLS_BAM1

    • APPHOST2: WLS_BAM2

    Click Next.

  18. In the Target Deployments to Cluster or Services screen, ensure the following targets:

    • User Messaging Deployments should be targeted only to BAM_Cluster. (The usermessaging-xmpp, usermessaging-smpp, and usermessaging-voicexml applications are optional.)

    • wsm-pm should be targeted only to BAM_Cluster.

    • The DMS Application should be targeted to BAM_Cluster and Administration Server.

    • The oracle.rules.*, oracle.sdp.* and oracle.bam.* deployments should be targeted only to BAM_Cluster.

    • The oracle.wsm.seedpolicies library should be targeted only to the BAM_Cluster.

  19. In the Target Services to Cluster or Servers screen, ensure the following targets:

    • WSM Startup Class should be targeted only to BAM_Cluster.

    • mds-wsm, mds-wsm-rac0, and mds-wsm-rac1 should be targeted to both BAM_Cluster and AdminServer.

    • OrasDPMDatasource, OrasDPMDatasource-rac0, and OrasDPMDatasource-rac1 should be targeted to the BAM_Cluster.

    • OWSM Startup Class is only targeted to BAM_Cluster *DMS Startup is targeted both to BAM_Cluster and AdminServer*mds-wsm, mds-wsm-rac0, and mds-wsm-rac1 should be targeted to both BAM_Cluster and AdminServer

    • mds-soa, mds-soa-rac0, and mds-soa-rac1 should be targeted to both BAM_Cluster and AdminServer.

  20. In the Configuration Summary screen, click Create.

  21. In the Creating Domain screen, click Done.

5.12.6 Creating boot.properties for the Administration Server and for WLS_BAM1 on APPHOST1

This is an optional step for enabling the Administration Server to start up without prompting you for the administrator username and password. Create a boot.properties file for the Administration Server on APPHOST1.

For the Administration Server:

  1. Create the following directories:

    mkdir ORACLE_BASE/product/fmw/user_projects/domains/bamdomain/servers/
    
    mkdir ORACLE_BASE/product/fmw/user_projects/domains/bamdomain/servers/AdminServer
    
    mkdir ORACLE_BASE/product/fmw/user_projects/domains/bamdomain/servers/AdminServer/security
    
  2. Use a text editor to create a file called boot.properties in the directory created in the previous step, and enter the following lines in the file:

    username=adminuser
    password=password
    

    Note:

    When you start up the Administration Server, the username and password entries in the file are encrypted.

    For security reasons, minimize the time the entries in the file are left unencrypted. After you edit the file, start up the server as soon as possible in order for the entries to be encrypted.

5.12.7 Starting the Administration Server on APPHOST1

To start the Administration Server on APPHOST1 run the following commands:

For Linux:

APPHOST1> cd MW_HOME/user_projects/domains/bamdomain/bin

APPHOST1> ./startWebLogic.sh

For Windows:

startWebLogic.cmd

To verify that the Administration Server is properly configured:

  1. In a browser, go to http://VIP0:7001/console.

  2. Log in as the administrator.

  3. Verify that the WLS_BAM1 and WLS_BAM2 managed servers are listed.

  4. Verify that the BAM_Cluster cluster is listed.

  5. Verify that you can access Enterprise Manager at http://VIP0:7001/em.

5.12.8 Disabling Host Name Verification for the Servers

This step is required if you have not set up the appropriate certificates to authenticate the different nodes with the Administration Server. If you have not configured the server certificates, you receive errors when managing the different WebLogic Servers. To avoid these errors, disable host name verification while setting up and validating the topology, and enable them again once the high availability topology configuration is complete.

To disable host name verification:

  1. In Oracle WebLogic Server Administration Console, select Administration Server, SSL, and then Advanced.

  2. Set Hostname Verification to None.

  3. In Oracle WebLogic Server Administration Console, select WLS_BAM1, SSL, and then Advanced.

  4. Set Hostname Verification to None.

  5. In Oracle WebLogic Server Administration Console, select WLS_BAM2, SSL, and then Advanced.

  6. Set Hostname Verification to None.

  7. Restart the Administration Server for the changes to take affect.

  8. Save and activate the changes.

5.12.9 Configuring a JMS Persistence Store for BAM UMS

Configure the location for all of the persistence stores as a directory that is visible from both APPHOST1 and APPHOST2. This is required in order to enable the resume of transactions when a server is migrated to a different node. By using a shared location for the persistent stores from both nodes, it is guaranteed that no messages are lost should a failover take place.

  1. Log into the Oracle WebLogic Server Administration Console.

  2. In the Domain Structure window, expand the Services node and then click the Persistence Stores node.

    The Summary of Persistence Stores page appears.

  3. Select the a persistent store (represented as a hyperlink) from the Name column of the table.

    The Settings page for the persistence store appears.

  4. In the Configuration tab, enter the location on a persistent storage solution (such as NAS or SAN) that is available to other servers in the cluster in the Directory field. Specifying this location enables pending JMS messages to be sent.

  5. Click Save.

  6. Repeat these steps for all persistent stores.

  7. Save and Activate the changes.

5.12.10 Configuring a Default Persistence Store for Transaction Recovery

Each server has a transaction log that stores information about committed transactions that are coordinated by the server that may not have been completed. The WebLogic Server uses this transaction log for recovery from system crashes or network failures. To leverage the migration capability of the Transaction Recovery Service for the servers within a cluster, store the transaction log in a location accessible to a server and its backup servers.

Note:

Preferably, this location should be a dual-ported SCSI disk or on a Storage Area Network (SAN).

To set the location for the default persistence store:

  1. Log into the Oracle WebLogic Server Administration Console.

  2. In the Domain Structure window, expand the Environment node and then click the Servers node.

    The Summary of Servers page appears.

  3. Click the name of the server (represented as a hyperlink) in Name column of the table.

    The settings page for the selected server appears and defaults to the Configuration tab.

  4. Click the Services tab.

  5. In the Default Store section of the page, enter the path to the folder where the default persistent stores will store its data files.

  6. Save and Activate the changes.

    Note:

    To enable migration of the Transaction Recovery Service, specify a location on a persistent storage solution that is available to other servers in the cluster. This directory must also exist before you restart the server.

5.12.11 Untargeting the BAM Server System from APPHOST2

Because the BAM server component in BAM is a singleton, you must untarget it from one of the WLS_BAM servers before you configure it for server migration. Otherwise the system would use two active BAM Servers which could cause different data inconsistencies. This ways BAM Web applications run in both APPHOST1 and APPHOST2 but BAM Server is initially active only in APPHOST1.

In this step, you remove the BAM server runtime from WLS_BAM2. To untarget the BAM server artifacts from WLS_BAM2:

  1. Log into the Oracle WebLogic Administration Console at http://APPHOST1VHN0:7001/console.

  2. In the Domain Structure window, choose Environment and then Servers.

    The Summary of Servers page appears.

  3. Select WLS_BAM2 in Name column of the table.

    The Settings page for WLS_BAM2 appears.

  4. Click the Deployments tab.

  5. Select the oracle-bam application from the Name column of the table.

    The Settings page for the oracle-bam application appears.

  6. Click the Targets tab.

  7. Click Lock and Edit.

  8. Change the targets for the modules as described in Table 5-12.

    Save and Activate the changes

    Note:

    You must target all of these components as described in Table 5-12, as incorrect targeting can prevent the BAM system from starting.

    Table 5-12 oracle-bam Component Target Types

    Component Type Target

    oracle-bam(11.1.1)

    Enterprise Application

    BAM_Cluster

    /oracle/bam

    WEBAPP

    WLS_BAM1

    oracle-bam-adc-ejb.jar

    EJB

    WLS_BAM1

    oracle-bam-ems-ejb.jar

    EJB

    WLS_BAM1

    oracle-bam-eventengine-ejb.jar

    EJB

    WLS_BAM1

    oracle-bam-reportcache-ejb.jar

    EJB

    WLS_BAM1

    oracle-bam-statuslistener-ejb.jar

    EJB

    WLS_BAM1

    OracleBAM

    WEBAPP

    BAM_Cluster

    OracleBAMWS

    WEBAPP

    BAM_Cluster

    sdpmessagingclient-ejb.jar

    EJB

    WLS_BAM1


5.12.12 Propagating the Domain Configuration from APPHOST1 with pack/unpack Utilities

Follow these steps to propagate the domain configuration to APPHOST1 using Pack/Unpack utilities:

  1. Run the following pack command on APPHOST1 to create a template pack:

    APPHOST1> cd ORACLE_COMMON_HOME/common/bin
    APPHOST1> ./pack.sh -managed=true
    domain=ORACLE_BASE/product/fmw/user_projects/domains/bamdomain/
    -template=bamdomaintemplate.jar 
    -template_name=bam_domain_template
    
  2. Run the unpack command on APPHOST2 to unpack the propagated template:

    APPHOST2> cd
     ORACLE_COMMON_HOME/common/bin
    APPHOST2> ./unpack.sh
     -domain=MW_HOME/user_projects/domains/bamdomain/
     -template=bamdomaintemplate.jar
    

5.12.13 Starting Node Manager on APPHOST1 and APPHOST2

To start Node Manager on APPHOST1 and APPHOST2:

  1. Run the setNMProps.sh script, located in the ORACLE_COMMON_HOME/common/bin directory.

    This script sets the StartScriptEnabled property to true before starting Node Manager, and allows staring servers from the Administration Console (the environment variables required for the BAM Servers are set in the default start script in the domain directory):

    APPHOST1> cd ORACLE_COMMON_HOME/common/bin
    APPHOST1> ./setNMProps.sh
    

    Note:

    You must use the StartScriptEnabled property to avoid class loading failures and other problems.
  2. Start Node Manager using the following command:

    APPHOST1> cd WL_HOME/server/bin
    APPHOST1> ./startNodeManager.sh
    
  3. Repeat step1 and 2 for NodeManager in APPHOST2.

5.12.14 Starting the Oracle BAM System

To start the WLS_BAM1 managed server on APPHOST1:

  1. Start the WLS_BAM1 managed servers:

    1. Log into the Oracle WebLogic Server Administration Console using the following URL:

      http://apphost1vhn0:7001/console

    2. In the Domain Structure window, expand the Environment node and then select Servers.

      The Summary of Servers page appears.

    3. Click the Control tab.

    4. Select WLS_BAM1 from the Servers column of the table.

    5. Click Start.

  2. Access http://apphost1vhn1:9001/OracleBAM to verify status of WLS_BAM1.

  3. Start the WLS_BAM2 managed servers:

    1. Log into the Oracle WebLogic Server Administration Console at http://apphost1vhn0:7001/console.

    2. In the Domain Structure window, expand the Environment node and then select Servers.

      The Summary of Servers page appears.

    3. Click the Control tab.

    4. Select WLS_BAM2 from the Servers column of the table.

    5. Click Start.

  4. Access http://apphost2:9001/OracleBAM to verify status of WLS_BAM2.

If the managed servers fails to start with the following message:

Listener refused the connection with the following error:
ORA-12519, TNS:no appropriate service handler found
The Connection descriptor used by the client was <db_connect_string>

Verify that the PROCESSES initialization parameter for the database is set to a high enough value.

5.12.15 Configuring RAC Failover for the WLS_BAM Servers

Oracle BAM allows customizing the behavior of a BAM server when Oracle RAC is used as the repository for the BAM schemas and a failure occurs in the database. The properties that allow this customization can be adjusted depending on the application and based on the desired expected behavior for each BAM system. The properties are configured in the Fusion Middleware Control Console System MBean Browser, or in the corresponding Oracle BAM-specific XML configuration file.

If you want to completely disable Oracle BAM's failover to the database in a RAC configuration set UseDBFailover to false in the Fusion Middleware Control Console System MBean Browser for your BAM server. The default value of this property is true, therefore, failover is provided. You can also increase or decrease the number of retries in the access to the database when there is a database instance failure (for BAM to retry the in flight transactions). To adjust the number of retries change the MaxDBNodeFailoverRetries in the Fusion Middleware Control Console System MBean Browser. The default value for MaxDBNodeFailoverRetries is 5 times. See the Oracle Fusion Middleware Administrator's Guide for Oracle SOA Suite for more details on the properties that can be configured for Oracle BAM.

5.12.16 Configuring the BAM Web Applications to Use the BAM Server in APPHOST1

To configure the OracleBamWeb(WLS_BAM1) and OracleBamWeb(WLS_BAM2) applications to use the BAM server in APPHOST1:

  1. Access Oracle Enterprise Manager Fusion Middleware Control at the following URL

    http:// apphost1vhn0:7001/em
    
  2. Expand BAM in the navigation tree.

  3. Right-click OracleBamWeb(WLS_BAM1).

    • Select BAM Web Properties from the context menu.

      The BAM Web Properties page appears.

    • Define the following properties:

      • Enter webhost:7777 for the application URL.

      • Enter APPOST1VHN1 for the server name.

  4. Select OracleBamWeb(WLS_BAM2) from the navigation tree and repeat these steps.

5.12.17 Configuring the ADCServer to Use the Appropriate BAMServer Address

To configure the ADCServer to use the correct BAMServer address:

  1. Access Oracle Enterprise Manager Fusion Middleware Control using the followingURL:

    http://apphost1vhn0:7001/em
    
  2. Expand BAM in the navigation tree.

  3. Right-click OracleBamServer(WLS_BAM1).

  4. Select System MBean Browser from the context menu.

  5. In the System MBean Browser, expand oracle.bam.server, located within Application Defined MBeans.

  6. Click the BamServerConfig MBean.

  7. Update the ADCServerName attribute by entering APPHOST1VHN1 (server address of WLS_BAM1) as the value.

  8. Update the ADCServerPort attribute to the listening port of the WLS server where the BAM server is running (9001).

  9. Click Apply.

5.12.18 Configuring Oracle HTTP Server for the Administration Server and the WLS_BAMn Managed Servers

Enable Oracle HTTP Server to route to the Administration Server and the WSM-PM_Cluster, which contain the WLS_BAMn managed servers, by setting the WebLogicCluster parameter to the list of nodes in the cluster.

Add the following lines to the mod_wl_ohs.conf file, located in the OHS_HOME/instances/ohs_instance1/config/OHS/ohs1 directory:

# BAM Web app
<Location /OracleBAM>
    SetHandler weblogic-handler
    WebLogicCluster apphost1vn1:9001,apphost2:9001
</Location>
 
# WSM-PM
<Location /wsm-pm>
    SetHandler weblogic-handler
    WebLogicCluster apphost1vn1:9001,apphost2:9001
</Location>

Restart Oracle HTTP Server on WEBHOST1:

WEBHOST1> OHS_HOME/instances/ohs_instance1/bin/opmnctl restartproc ias-component=ohs1

5.12.19 Validating Access through Oracle HTTP Server

Verify the following URLs to ensure that appropriate routing and failover is working from the HTTP Server to the SOA_Cluster.

  1. While WLS_BAM2 is running, stop WLS_BAM1 from Oracle WebLogic Server Administration Console.

  2. Access the following URLs and verify the appropriate functionality:

    • WebHost1:7777/wsm-pm

    • WebHost1:7777/OracleBAM

  3. Start WLS_BAM1 from Oracle WebLogic Server Administration Console.

  4. Stop WLS_BAM2.

  5. Access the following URLs and verify the appropriate functionality:

    • WebHost1:7777/wsm-pm

    • WebHost1:7777/OracleBAM

5.12.20 Configuring Server Migration for the WLS_BAM Servers

The high availability architecture for Oracle BAM uses server migration to protect singleton services against failures. The WLS_BAM1 managed server is configured so that it can be restarted on APPHOST2 in case of failure. For this configuration, WLS_BAM1 listens on a specific floating IP address that is failed over by Oracle WebLogic Server Migration. To configure server migration for the WLS_BAM1 managed servers, follow these steps:

Note:

If server migration was configured previously for SOA, the BAM stem can use the same data sources and database schemas. In that case, steps 1 through 5 may not be required, but you must also target the corresponding server-migration/leasing datasources to the BAM Cluster.

5.12.20.1 Setting Up the User and Tablespace for the Server Migration Leasing Table

To create the user and tablespace:

  1. Create a tablespace called leasing. For example, log on to SQL*Plus as the sysdba user and run the following command:

    Example: Log on to SQL*Plus as the sysdba user and run the following command:

    SQL> create tablespace leasing
            logging datafile 'DB_HOME/oradata/orcl/leasing.dbf'
            size 32m autoextend on next 32m maxsize 2048m extent management local;
    
  2. Create a user named leasing and assign to it the leasing tablespace.

    SQL> create user leasing identified by welcome1;
    
    SQL> grant create table to leasing;
    
    SQL> grant create session to leasing;
    
    SQL> alter user leasing default tablespace leasing;
    
    SQL> alter user leasing quota unlimited on LEASING;
    
  3. Create the leasing table using the leasing.ddl script.

    1. Copy the leasing.ddl file located in the WL_HOME/server/db/oracle directory to your database node.

    2. Connect to the database as the leasing user.

    3. Run the leasing.ddl script in SQL*Plus.

      SQL> @copy_location/leasing.ddl;
      

5.12.20.2 Creating a Multi-Data Source from the WebLogic Server Administration Console

Create a multi-data source for the leasing table from the Oracle WebLogic Server Administration Console:

You create a data source to each of the RAC database instances during the process of setting up the multi-data source, both for these data sources and the global leasing multi-data source. When you create a data source:

  • Make sure that this is a non-xa data source

  • The names of the multi-data sources are in the format of <MultiDS>-rac0, <MultiDS>-rac1, and so on

  • Use Oracle's Driver (Thin) Version 9.0.1, 9.2.0, 10, 11

  • Use Supports Global Transactions, One-Phase Commit, and specify a service name for your database

  • Target these data sources to the BAM cluster

  • Make sure the datasources' connection pool initial capacity is set to 0. To do this, select Services, JDBC, and then Datasources. In the Datasources screen, click the Datasource Name, then click the Connection Pool tab, and enter 0 in the Initial capacity field.

Creating a Multi-Data Source

To create a multi-data source, complete these steps:

  1. From Domain Structure window in the Oracle WebLogic Server Administration Console, expand the Services node, then expand the JDBC node.

  2. Click Multi Data Sources. The Summary of JDBC Multi Data Source page appears.

  3. Click Lock and Edit.

  4. Click New. The Create a New JDBC Multi Data Source page appears.

  5. Enter leasing as the Name

  6. Enter jdbc/leasing as the JNDI name.

  7. Select Failover as algorithm (default).

  8. Click Next.

  9. Select SOA_Cluster as the target.

  10. Click Next.

  11. Select non-XA driver (the default).

  12. Click Next.

  13. Click Create New Data Source.

  14. Enter leasing-rac0 as name. Enter jdbc/leasing-rac0 as JNDI name. Enter oracle as the database type. For the driver type, enter Oracle Driver (Thin) for RAC server-Instance connection Version 10,11.

    Note:

    When creating the multi-datasources for the leasing table, enter names in the format of <MultiDS>-rac0, <MultiDS>-rac1, and so on.
  15. Click Next.

  16. Deselect Supports Global Transactions.

  17. Click Next.

  18. Enter the service name, database name, host port, and password for your leasing schema.

  19. Click Next.

  20. Click Test Configuration and verify the connection works.

  21. Target the data source to the SOA cluster.

  22. Select the data source and add it to the right screen.

  23. Click Create a New Data Source second instance of your RAC database, target it to SOA_Cluster, repeating the steps for the second instance of your RAC database.

  24. Add the second data source to your multi-data source.

  25. Click Save.

  26. Click Activate Changes.

5.12.20.3 Edit the Node Manager's Properties File

The nodemanager.properties file is located in the WL_HOME/server/bin directory.

Interface=eth0
NetMask=255.255.255.0
UseMACBroadcast=true
  • Interface=eth0

    This property specifies the interface name for the floating IP (eth0, for example).

    Note:

    Do not specify the sub interface, such as eth0:1 or eth0:2. This interface is to be used without the :0, or :1. The Node Manager's scripts traverse the different :X enabled IPs to determine which to add or remove. For example, the valid values in Linux environments are eth0, eth1, or, eth2, eth3, ethn, depending on the number of interfaces configured.
  • NetMask

    This property specifies the net mask for the interface for the floating IP.

  • UseMACBroadcast

    This property specifies whether or not to use a node's MAC address when sending ARP packets, that is, whether or not to use the -b flag in the arping command.

Perform this configuration in the two nodes APPHOST1 and APPHOST2. Verify in the output of Node Manager (the shell where the Node Manager is started) that these properties are in use. Otherwise, problems may occur during migration. The output should be similar to the following:

...
StateCheckInterval=500
Interface=eth0
NetMask=255.255.255.0
...

5.12.20.4 Set Environment and Superuser Privileges for the wlsifconfig.sh Script

To set the environment and superuser privileges for the wlsifconfig.sh script:

  1. Ensure that your PATH environment variable includes the files listed in Table 5-13.

    Table 5-13 Files Required to be in the PATH Environment Variable

    File Located in this directory

    wlsifconfig.sh

    MW_HOME/user_projects/domains/bamdomain/bin/server_migration

    wlscontrol.sh

    WL_HOME/common/bin

    nodemanager.domains

    WL_HOME//common


  2. Grant sudo configuration for the wlsifconfig.sh script.

    • Configure sudo to work without a password prompt.

    • For security reasons, sudo should be restricted to the subset of commands required to run the wlsifconfig.sh script. For example, to set the environment and superuser privileges for the wlsifconfig.sh script, complete these steps:

      • Grant sudo privilege to the WebLogic user ('oracle') with no password restriction, and grant execute privilege on the /sbin/ifconfig and /sbin/arping binaries.

      • Make sure the script is executable by the WebLogic user ('oracle'). The following is an example of an entry inside /etc/sudoers granting sudo execution privilege for oracle and also over ifconfig and arping:

        oracle ALL=NOPASSWD: /sbin/ifconfig,/sbin/arping
        

        Note:

        Ask the system administrator for the sudo and system rights as appropriate to this step.

5.12.20.5 Configure Server Migration Targets

Configuring Cluster Migration sets the DataSourceForAutomaticMigration property to true. Follow these steps to configure cluster migration in a cluster:

  1. Log into the Oracle WebLogic Server Administration Console.

  2. In the Domain Structure window, expand Environment and select Clusters.

    The Summary of Clusters page appears.

  3. Click the cluster for which you want to configure migration (BAM_Cluster) in the Name column of the table.

  4. Click the Migration tab.

  5. In the Available field, select the machine to which to allow migration and click the right arrow. In this case, select APPHOST1 and APPHOST2.

  6. Select the data source to be used for automatic migration. In this case select the leasing data source.

  7. Click Save.

  8. Set the Candidate Machines for Server Migration.

    You must perform this task for WLS_BAM1:

    1. In Domain Structure window of the Oracle WebLogic Server Administration Console, expand Environment and select Servers.

    2. Select the server for which you want to configure migration.

    3. Click the Migration tab.

    4. In the Available field, located in the Migration Configuration section, select the machines to which to allow migration and click the right arrow. Select APPHOST2 for WLS_BAM1.

    5. Select Automatic Server Migration Enabled.

      This enables the Node Manager to start a failed server on the target node automatically.

    6. Click Save and Activate the changes.

    7. Restart the Administration Server and the WLS_BAM1 server.

5.12.20.6 Test Server Migration

To verify that Server Migration is working properly, follow these steps:

From Node 1:

  1. Kill the WLS_BAM1 managed server using the following command:

    APPHOST1> kill -9 <pid>
    

    pid specifies the process ID of the managed server. You can identify the pid in the node by running the following command:

    APPHOST1> ps -ef | grep BAM_SOA1
    
  2. In the Node Manager console, a message appears indicating that WLS_BAM1's floating IP has been disabled.

  3. Wait for the Node Manager to try a second restart of WLS_BAM1.

    Node Manager waits for a fence period of thirty seconds before trying this restart.

  4. Once Node Manager restarts the server, stop it again.

    Node Manager should now log a message indicating that the server will not be restarted again locally.

From APPHOST2:

  1. Watch the local Node Manager console.

    After thirty seconds since the last try to restart WLS_BAM1on Node 1, Node Manager on APPHOST2 should prompt that the floating IP for WLS_BAM1 is being brought up and that the server is being restarted in this node.

  2. Access the Oracle BAM console using APPHOST1VHN1/OracleBAM and WebHost1:7777/OracleBAM.

Verification From the Administration Console

You can verify Migration in the Administration Console:

  1. Log into the Administration Console.

  2. Click on Domain on the left console.

  3. Click the Monitoring tab and then on the Migration tab.

The Migration Status table provides information on the status of the migration.