Tuning the Fault Monitor for Oracle Solaris Cluster HA for Sun Java System Application Server

The fault monitor for the DAS component of Oracle Solaris Cluster HA for Sun Java System Application Server supported versions as of 9.1 is contained in a resource whose resource type is SUNW.jsas.

System properties and extension properties of the resource types control the behavior of the fault monitors. The default values of these properties determine the preset behavior of the fault monitors. The preset behavior should be suitable for most Oracle Solaris Cluster installations. Therefore, you should tune the fault monitors only if you need to modify this preset behavior.

Tuning these fault monitors involves the following tasks:

• Setting the interval between fault monitor probes

• Setting the timeout for fault monitor probes

• Defining the criteria for persistent faults

• Specifying the failover behavior of a resource

Perform these tasks when you register and configure HA for Sun Java System Application Server, as described in Registering and Configuring Oracle Solaris Cluster HA for Sun Java System Application Server Supported Versions as of 9.1.

For detailed information about these tasks, see Tuning Fault Monitors for Oracle Solaris Cluster Data Services in Oracle Solaris Cluster Data Services Planning and Administration Guide.

This section provides the following information:

• Description of the fault monitor's probing functionality.

• Conditions, messages, and recovery actions associated with unsuccessful probing.

• Conditions and messages associated with successful probing.

Operations by the Fault Monitor for the DAS Component During a Probe

The fault monitor probe for the DAS component of Oracle Solaris Cluster HA for Sun Java System Application Server sends a request to the server to query the health of the Application Server server. The operations of the fault monitor depend on the setting of the Monitor_Uri_List extension property.

Fault Monitor Operations When the Monitor_Uri_List Property Is Set

When the Monitor_Uri_List extension property is set to a single URI or a list of URIs, the fault monitor probe executes the following steps.

1. The fault monitor probes the Application Server instance according to the time-out value set with the Probe_timeout resource property.

2. The probe connects to the Application Server server and performs an HTTP 1.1 GET check by sending a HTTP request and receiving a response to each of the URIs in Monitor_Uri_List.

   The result of each HTTP request is either failure or success. If all of the requests successfully receive a reply from the Application Server server, the probe returns and continues the next cycle of probing and sleeping.

   Heavy network traffic, heavy system load, and misconfiguration can cause the HTTP GET probe to fail. Misconfiguration of the Monitor_Uri_List property can cause a failure if a URI in the Monitor_Uri_List includes an incorrect port or hostname. For example, if the application server instance is listening on logical host schost-1 and the URI was specified as http://schost-2/servlet/monitor, the probe will try to contact schost-2 to request /servlet/monitor.

3. The probe records a failure in the history log if the reply to the probe is not received within the Probe_timeout limit. The probe considers this scenario a failure on the part of the Application Server data service. A Application Server probe failure can be a complete failure or a partial failure.

   If the reply to the probe is received within the Probe_timeout limit, the HTTP response code is checked. If the response code is 500 Internal Server Error, the probe is considered a complete failure. All other response codes are ignored.

   The following are complete probe failures.

   • The following error message is received upon failure to connect to the server. The %s indicates the hostname and %d indicates the port number.

     Failed to connect to the host <%s> and port <%d>. Receiving a
     response code of 500 Internal Server Error HTTP GET
     Response Code for probe of %s is 500. Failover will be in
     progress

   • The following error message is received upon failure to successfully send the probe string to the server. The first %s indicates the hostname, the %d indicates the port number, and the second %s indicates further details about the error.

     Write to server failed: server %s port %d: %s.

4. The monitor accumulates partial failures that occur within the Retry_interval resource property setting until they equal a complete failure.

   The following are partial probe failures:

   • The following error message is received when there is a failure to disconnect before the Probe_timeout setting lapses. The %d indicates the port number and the %s indicates the resource name.

     Failed to disconnect from port %d of resource %s.

   • Failure to complete all probe steps within Probe_timeout time is a partial failure.

   • The following error message is received upon failure to read data from the server for other reasons. The first %s indicates the hostname, the %d indicates the port number, and the second %s indicates further details about the error.

     Failed to communicate with server %s port %d: %s

5. Based on the history of failures and the setting of the probe parameters, a failure can cause either a local restart or a failover of the data service.

Fault Monitor Operations When the Monitor_Uri_List Property Is Not Set

When the Monitor_Uri_List extension property is not set, the fault monitor probe executes the following steps.

1. The fault monitor probes the Application Server instance according to the time-out value set with the Probe_timeout resource property.

2. The probe uses the asadmin command to get the status of the DAS, as follows:

   $INSTALL_DIR/appserver/bin/asadmin list-domains --domaindir $DOMAIN_DIR

   The following environment variables are set.

   • INSTALL_DIR is the installation location that is set using the Confdir_list extension property.

   • DOMAIN_DIR is the complete path to the domain directory.

   • DOMAIN_NAME is the name of the domain.

3. If the probe determines that the DAS is not running, a complete failure is issued.

4. Based on the history of failures and the setting of the probe parameters, a failure can cause either a local restart or a failover of the data service.