This chapter describes considerations for developing Fusion web applications that you intend to deploy to a high availability, clustered server environment.
This chapter includes the following topics:
Section 48.2, "Configuring Oracle ADF for High Availability"
Section 48.3, "Troubleshooting Fusion Web Applications for High Availability"
High availability for the Fusion web application is the ability of the application to be available when it is needed.
A high availability environment ensures that users can access the application without loss of service. Deploying the Fusion web application to a high availability environment minimizes the time when the application is down, or unavailable, and maximizes the time when it is running, or available.
High availability comes from redundant systems and components. You can categorize high availability solutions by their level of redundancy into active-active solutions and active-passive solutions.
An active-active solution deploys two or more active servers and can be used to improve scalability and provide high availability. In active-active deployments, all instances handle requests concurrently. Oracle recommends active-active solutions for all single-site middleware deployments. Active-passive solutions deploy an active instance that handles requests and a passive instance that is on standby.
For more information about high availability environments, see the High Availability Guide.
At runtime, ADF objects such as the binding container and managed beans are instantiated. Each of these objects has a defined life span set by its scope attribute.
For more information about the life span of ADF objects, see Section 27.3, "About Object Scope Lifecycles."
An Oracle WebLogic cluster provides application high availability. If one member of the cluster is unavailable, any other available member of the cluster is able to handle the request. You should familiarize yourself with failover requirements and the expected behavior of an application in the event of a failover.
For seamless failover of a Fusion web application, the application must meet the following conditions:
The application is in a cluster and at least one member of the application cluster is available to serve the request.
For stateful applications, state replication is configured correctly, as described in Section 48.2, "Configuring Oracle ADF for High Availability."
If you use Oracle HTTP Server, the server is configured with the WebLogicCluster directive to balance among all available application instances.
If you use a hardware load balancer, the load balancer is:
Routing traffic to all available instances
Configured correctly with a health monitor to mark unavailable instances
Configured to support persistence of session state
Expected Behavior for Application Failover
If the environment is configured correctly, application users do not notice when an application instance in a cluster becomes unavailable. The sequence of events in an application failover is, for example, as follows:
A user makes a request and is routed by a hardware load balancer to Instance A of the application.
Instance A of the application becomes unavailable because of node failure, process failure, or network failure.
The hardware load balancer marks Instance A as unavailable.
The user makes a subsequent request. The request is routed to Instance B.
Instance B is configured as a replication partner of Instance A and has the user's session state.
The application resumes using the session state on Instance B and the user continues working without interruption.
When you configure the ADF application module to access a highly available database system, such as redundant databases or Oracle Real Application Clusters (Oracle RAC) as the backend, the data source must be container-defined. In this scenario, you must use a GridLink data source or a multi data source. However, from the standpoint of the application module configuration, the naming convention for the multi data source or GridLink data source is the same as it is for a non-multi data source or GridLink data source. This naming convention ensures that the correct data source is used at runtime.
To more information about configuring GridLink data sources or multi data sources for high availability applications, see the "Database Considerations" chapter of the High Availability Guide.
For information about high availability considerations related to modifying the mdsDS data source URL post deployment of the Fusion web application, see the "Configuring High Availability for Other Components" chapter of the High Availability Guide.
To support automatic replication and failover for web applications in a clustered environment, Oracle WebLogic Server supports mechanisms for replicating HTTP session state across clusters. You can configure Oracle ADF to ensure the Fusion web application's state can be restored from any server in the cluster.
An application module is the transactional component that the Fusion web application uses to work with application data. It defines an updateable data model and top-level procedures and functions, called service methods, related to a logical unit of work related to an end-user task. An application module supports passivating, or storing, its transaction state as a snapshot in the database. It also supports the reverse operation of activating the transaction state from one of these saved snapshots.
For more information on application module state management, see Section 50.1, "About Fusion Web Application State Management."
To enable support for ADF Business Components failover, set the jbo.dofailover parameter to true so that the application module state is saved on release. This enables Oracle ADF to restore the application module state from a snapshot saved from a previous check in. By contrast, when the failover feature is disabled, which it is by default, then application module state is saved only when the application is reused by a subsequent user session and only when the application module pool cannot find an unused application module.
You can set this parameter in your application module configuration on the Pooling and Scalability tab of the Edit Configuration dialog.
To configure application modules for high availability:
Launch JDeveloper and open the application.
In the Applications window, expand the project that contains the data model and double-click the application module.
In the overview editor for the application module, select the Configurations navigation tab and double-click the desired configuration from the list.
In the Edit Configuration dialog, click the Pooling and Scalability tab.
Select the Failover Transaction State Upon Managed Release checkbox.
Click OK.
To enable support for replicating HTTP session state, you must assign a value to the persistent-store-type element in the Oracle WebLogic Server weblogic.xml file. The value replicated_if_clustered ensures that the in-effect persistent store type will be replicated so that sessions on the clustered environment are stored in accordance with the value set for the cluster of servers to which this server belongs.
Note:
Oracle ADF applications such as the Oracle WebCenter Portal Suite are preconfigured and do not need additional configuration.To configure the weblogic.xml file for high availability:
Launch JDeveloper and open the application.
In the Applications window, expand the project that contains the web application and expand the WEB-INF folder.
Double-click the weblogic.xml file, and click the Source tab to edit the file.
In the file, add the persistent-store-type definition to the session-descriptor element:
<weblogic-web-app>
<session-descriptor>
<persistent-store-type>
replicated_if_clustered
</persistent-store-type>
</session-descriptor>
</weblogic-web-app>
When you design an application to run in a clustered environment, you must ensure that Oracle ADF is aware of changes to managed beans stored in ADF scopes (view scope and page flow scope).
When a value within a managed bean in either view scope or page flow scope is modified, the application must notify Oracle ADF so that it can ensure the bean's new value is replicated.
To enable ADF Controller to track changes to ADF memory scopes and replicate the page flow scope and view scope within the server cluster, you must set the ADF Controller parameter <adf-scope-ha-support> in the application's adf-config.xml file to true. For example, when set to true for an application and that application adds or removes a bean from a page flow scope during a request, the change will automatically replicated within a cluster.
The adf-config.xml file is the central configuration file for all ADF components. The file contains sections to configure the runtime behavior for ADF components, including, ADF Controller.
Note:
If your application uses MDS and will use an Oracle database that supports failover, Oracle recommends enabling MDS retry on failover. To do this, add the followingretry-connection entry to the MDS configuration section of adf-config.xml.
<persistence-config>
<metadata-namespaces>...
<metadata-store-usages>...
<external-change-detection enabled="false" />
<read-only-mode enabled="true"/>
<retry-connection enabled="true"/>
</persistence-config>
To configure the adf-config.xml file for high availability:
In the Application window, expand the Application Resources.
Select Descriptors, and then select the ADF META-INF node.
Double-click the adf-config.xml file, and click the Source tab to edit the file.
Add the following to the file:
<adf-controller-config xmlns="http://xmlns.example.com/adf/controller/config"> <adf-scope-ha-support>true</adf-scope-ha-support> </adf-controller-config>
For more information about using the adf-config.xml file to configure ADF, see Section A.9, "adfc-config.xml."
JSF Application Scope beans are backed by a servlet context which can only have one instance per cluster node. Therefore, for each design time configured bean, there will be a runtime instance of the bean for each node in a cluster. Because there is no synchronization across nodes, each instance of the application running on different nodes will have separate instances of that bean with separate values. Therefore, application scoped managed beans should not be used to carry application-wide values that can be updated to be shared across all nodes in a cluster. They can, however, be used to carry constants, read-only values, or values to reflect the state of a single node within the cluster (such as the number of users on a node).
The org.apache.myfaces.trinidad.CHECK_FILE_MODIFICATION context parameter provides optimization for developing and testing the Fusion web application in the JDeveloper. Leaving this parameter set to true when the application runs in a high availability environment can lead to errors after failover occurs. Note that this limitation should not affect your work because you typically use this parameter in a development environment only, not a high availability environment.
To configure the CHECK_FILE_MODIFICATION parameter for high availability:
In the Applications window, expand the user interface project and expand the WEB-INF folder.
Double-click the web.xml file and click the Application navigation tab.
In the overview editor, expand the Context Initialization Parameters section and confirm that the org.apache.myfaces.trinidata.CHECK_MODIFICATION parameter definition is set to false.
The web.xml file definition will be:
<web-app>
...
<context-param>
<param-name>org.apache.myfaces.trinidad.CHECK_FILE_
MODIFICATION</param-name>
<param-value>false</param-value>
</context-param>
</web-app>
This section describes procedures for troubleshooting possible issues with Fusion web applications at design time and after deployment.
When running or debugging an application that uses failover support (jbo.dofailover is enabled) within the JDeveloper environment, you are frequently starting and stopping the application server. The ADF failover mechanism has no way of knowing whether you stopped the server to simulate an application server failure, or whether you stopped it because you want to retest something from scratch in a fresh server instance. If you intend to do the latter, exit out of your browser before restarting the application on the server. This eliminates the chance that you will be confused by the correct functioning of the failover mechanism when you didn't intend to be testing that aspect of your application.
When you develop the Fusion web application in Oracle JDeveloper, the integrated development environment provides support for detecting potential high availability issues. The warnings that JDeveloper provides are generated by the audit framework and are triggered to appear in the JDeveloper source editors. The warnings the editors show are based on the audit rules for high availability applications.
The high availability audit rules that JDeveloper enables by default are:
ADF Controller Configuration - High Availability for ADF Scopes is not Enabled warns the developer that the adf-scope-ha-support flag in the adf-config.xml file is set is not set to true. This audit rule fires only when the <adf-controller-config> element is in the ADF application-level configuration file (adf-config.xml).
ADF Page Flows - Bean in Scope Map is Modified warns the developer when the some code calls a setter method on a bean to indicate that the code did not subsequently call the ControllerContext.markScopeDirty() method. This audit rule fire only when the adf-scope-ha-support flag in the adf-config.xml file is set to true.
ADF Page Flows - EL Bean is Modified warns the developer when some code evaluates an EL expression that mutates a bean to indicate that the code did not subsequently call the ControllerContext.markScopeDirty() method. This audit rule fire only when the adf-scope-ha-support flag in the adf-config.xml file is set to true.
ADF Page Flows - Managed Bean Class Not Serializable warns the developer that a managed bean has a non-serializable class defined in viewScope, pageFlowScope, or sessionScope. This audit rule fire only when the adf-scope-ha-support flag in the adf-config.xml file is set to true.
You can modify the high availability audit rule settings using the Preference dialog in JDeveloper. From the JDeveloper toolbar, choose Tools - Preferences, under Audit - Profiles expand ADF Controller Configuration or ADF Pages Flows and make the desired audit rule selections.
You can also trigger the audit by choosing Build - Audit project.jpr from the JDeveloper toolbar.
There are two actions to take if you encounter Fusion web application deployment issues: verify the JRF Runtime and verify the status of application deployments.
The first step to troubleshooting an Fusion web application deployment is to verify that the JRF Runtime is installed on the WebLogic Server domain. ADF applications require the JRF and ADF runtime. You cannot run ADF applications with a standalone WebLogic Server domain or just the WebLogic Server portion of the Application Development product; you must extend it with the JRF extension template.
For more information on the JRF Template, see "Oracle JRF and ADF Templates" in Domain Template Reference.
Fusion web applications deploy when the Managed Server first starts. Use the Administration Console to check that all application deployments were successful:
Click Deployments in the left hand pane. The right hand pane shows the application deployments and their status. The state of all applications, assuming all the servers are running, should be ACTIVE.
If an application deployment has failed, the server logs may provide some indication of why the application was not deployed successfully. The server logs are located in the DOMAIN_HOME/servers/server_name/logs directory. Common issues include:
Unavailability of external resources, such as database resources. Examine the error, fix it, and attempt to redeploy the application.
The appropriate applications or libraries are not targeted correctly to the right Managed Server or Cluster.
State Replication is most prominent in failover scenarios. A user working on one server may discover that, upon failover:
Windows may close or the state might reset.
Screens may require a reset.
The application may redirect to the logon screen.
To diagnose and troubleshoot state replication issues.
Confirm that this is not a known replication issue.
See Section 48.1.2, "Oracle ADF Failover and Expected Behavior Considerations"for possible expected behaviors. Before proceeding to further diagnose the issue, first confirm that the failover behavior is not an expected behavior.
Check load balancer settings.
For replication and failover to function correctly, the load balancer must be configured with the appropriate persistence settings. For more details on configuring Hardware Load Balancers for Oracle WebLogic Server, see Administering Clusters for Oracle WebLogic Server.
Check the cluster status.
Replication occurs within the context of a cluster. For failover to be successful, there must be at least one other healthy member of the cluster available. You can check cluster status in one of two ways:
Check the cluster status using the Oracle WebLogic Server Administration Console - In the Left-hand pane, click on Servers. Verify the state of all servers in the cluster.
Check the cluster status using the weblogic.Admin utility. You can use the weblogic.Admin utility to query the state of all servers in a specific cluster. For example:
$ java weblogic.Admin -url Adminhost:7001 -username <username> -password <password> CLUSTERSTATE -clustername Spaces_Cluster
This example returns:
There are 2 server(s) in cluster: Spaces_Cluster The alive servers and their respective states are listed below: Application Server---RUNNING Managed Server---RUNNING
Check cluster communications.
Even if all cluster members are running, there may be communication issues which prevent them from communicating replication information to each other. There are two types of cluster communication configurations. Troubleshooting depends on the cluster type:
Checking Unicast cluster communications - For Unicast clusters, Managed Servers must be able to access each other's hosts and each other's default listening port.
Ensure that all individual Managed Servers have their Listen Address set correctly. You can find this setting by selecting Configuration, General for each Managed Server.
Checking Multicast cluster communications - For multicast clusters, servers must be able to intercept the same multicast traffic. Ensure that multicast is configured correctly by running the WebLogic utility utils.MulticastTest on each machine. For example:
$ java utils.MulticastTest -H
Confirm Oracle WebLogic Server application configuration.
Oracle WebLogic Server is not configured by default for failover. In-memory replication takes place only with the proper setting in the weblogic.xml file:
<session-descriptor> <persistent-store-type>replicated_if_clustered</persistent-store-type> </session-descriptor>
A persistent-store-type of replicated is also acceptable. This setting can be made in JDeveloper, as described in Section 48.2.2, "How to Enable Support for Replicating HTTP Session State."
Confirm Oracle ADF Business Components configuration.
Oracle ADF is not configured by default for failover. Failover is supported only with the proper setting in the ADF Business Components configuration file (bc4j.xcfg):
<AppModuleConfig ... <AM-Pooling jbo.dofailover="true"/> </AppModuleConfig>
This setting is made in JDeveloper through the Edit Business Components Configuration dialog, as described in Section 48.2.1, "How to Configure Application Modules for High Availability."
Confirm Oracle WebLogic Server connection pool parameter.
Set an appropriate value for the weblogic-application.xml deployment descriptor parameter inactive-connection-timeout-seconds on the element <connection-check-params> pool-params.
When enabling application module state passivation, a failure can occur when Oracle WebLogic Server is configured to forcibly release connections back into the pool. The failure creates an exception "Connection has already been closed" that gets saved to the server log. The user interface does not show this exception.
Set inactive-connection-timeout-seconds to several minutes. In most cases, this setting avoids forcing the inactive connection timeout and passivation failure. Adjust the setting as needed for your environment.
Confirm ADF Controller configuration.
Oracle ADF is not configured by default to replicate changes to ADF objects in ADF memory scopes. ADF object replication is supported only with the proper setting in the ADF application-level configuration file (adf-config.xml):
<adfc:adf-controller-config> <adfc:adf-scope-ha-support>true</adfc:adf-scope-ha-support> </adfc:adf-controller-config>
This setting is made in JDeveloper through the source editor. See Section 48.2.1, "How to Configure Application Modules for High Availability."
Check default logger messages.
By default the ADF log shows high-level messages (INFO level). The default logging often reports problems with serialization and replication without the need to enable more detailed log messages.
Enable log messages for ADF high availability applications.
Configure the ADF logger to output runtime messages for high availability. By default the ADF log shows high-level messages (INFO level). You enable high availability diagnostics for ADF Controller by setting the logging level in Fusion Middleware Control to FINE.
When enabled, the logger outputs a warning if the adfc:adf-scope-ha-support setting in the adf-config.xml file is not set.
Enable debug.
Check the server logs for any unusual messages on Managed Server startup. In particular, if the Managed Server is unable to locate other members of the cluster. The server logs are located in the DOMAIN_HOME/servers/SERVER_NAME/logs directory.
For further debugging, enable the flags DebugCluster, DebugClusterAnnouncements, DebugFailOver, DebugReplication, and DebugReplicationDetails. You can enable each flag with the weblogic.Admin utility:
$ java weblogic.Admin -url Adminhost:7001 -username <username> -password <password> SET -type ServerDebug -property DebugCluster true
Enable component state serialization checking.
Enable server checking to ensure no unserializable state content on session attributes is detected. This check is disabled by default to reduce runtime overhead. Serialization checking is supported by the Java server system property org.apache.myfaces.trinidad.CHECK_STATE_SERIALIZATION.
Table 48-1 shows the options you can use with the property. Use commas to delimit the options.
Table 48-1 CHECK_STATE_SERIALIZATION Options
| Option | Description |
|---|---|
|
tree |
Checks whether the entire component tree is serializable. This is the fastest component check. Most testing should be performed with this flag enabled. |
|
component |
Checks each component individually for serializability. This option is much slower than "tree." It is typically turned on only after testing with "tree" reports an error. This option narrows down the problematic component. |
|
property |
Checks each component attribute individually for serializability. This is slower than "component" and narrows down the specific problematic component attribute after a failure has been detected in the "tree" or "component" modes. |
|
session |
Checks that all attributes in the JSF Session Map that are marked as Serializable are serializable. |
|
application |
Checks that all attributes in the JSF Application Map that are marked as Serializable are serializable. |
|
beans |
Checks that any serializable object in the appropriate map has been marked as dirty if the serializable content of the object changes during the request. |
|
all |
Checks everything. |
For high availability testing, start off by validating that the Session and JSF state is serializable by launching the application server with the system property:
-Dorg.apache.myfaces.trinidad.CHECK_STATE_SERIALIZATION=session,tree
Add the beans option to check that any serializable object in the appropriate map has been marked as dirty if the serialized content of the object has changed during the request:
-Dorg.apache.myfaces.trinidad.CHECK_STATE_SERIALIZATION=session,tree,beans
If a JSF state serialization failure is detected, relaunch the application server with the system property to enable component and property flags and rerun the test:
-Dorg.apache.myfaces.trinidad.CHECK_STATE_SERIALIZATION=all
These are Java system properties and you must specify them when you start the application server.