Sun Java System Application Server Enterprise Edition 8.1 2005Q2 High Availability Administration Guide

Chapter 4 Configuring Load Balancing and Failover

This section describes the HTTP load balancer plug-in. It includes the following topics:

How the Load Balancer Works

The load balancer attempts to evenly distribute the workload among multiple Application Server instances (either stand-alone or clustered), thereby increasing the overall throughput of the system.

Using a load balancer also enables requests to fail over from one server instance to another. For HTTP session information to persist, configure HTTP session persistence. For more information, see Chapter 8, Configuring High Availability Session Persistence and Failover

For complete instructions on configuring load balancing, see the Sun Java System Application Server High Availability Administration Guide.

Use the asadmin tool, not the Admin Console, to configure HTTP load balancing.

See Also:

Assigned Requests and Unassigned Requests

When a request first comes in from an HTTP client to the load balancer, it is a request for a new session. A request for a new session is called an unassigned request. The load balancer routes this request to an application server instance in the cluster according to a round-robin algorithm.

Once a session is created on an application server instance, the load balancer routes all subsequent requests for this session only to that particular instance. A request for an existing session is called an assigned or a sticky request.

HTTP Load Balancing Algorithm

The Sun Java System Application Server load balancer uses a sticky round robin algorithm to load balance incoming HTTP and HTTPS requests. All requests for a given session are sent to the same application server instance. With a sticky load balancer, the session data is cached on a single application server rather than being distributed to all instances in a cluster.

Therefore, the sticky round robin scheme provides significant performance benefits that normally override the benefits of a more evenly distributed load obtained with a pure round robin scheme.

When a new HTTP request is sent to the load balancer plug-in, it’s forwarded to an application server instance based on a simple round robin scheme. Subsequently, this request is “stuck” to this particular application server instance, either by using cookies, or explicit URL rewriting.

From the sticky information, the load balancer plug-in first determines the instance to which the request was previously forwarded. If that instance is found to be healthy, the load balancer plug-in forwards the request to that specific application server instance. Therefore, all requests for a given session are sent to the same application server instance.

The load balancer plug-in uses the following methods to determine session stickiness:

Sample Applications

The following directories contain sample applications that demonstrate load balancing and failover:

install_dir/samples/ee-samples/highavailability
install_dir/samples/ee-samples/failover

The ee-samples directory also contains information for setting up your environment to run the samples.

Setting Up HTTP Load Balancing

This section describes how to set up the Load Balancer plug-in and includes the following sections:

Prerequisites for Setting Up Load Balancing

Before configuring your load balancer, you must:

HTTP Load Balancer Deployments

You can configure your load balancer in different ways, depending on your goals and environment, as described in the following sections:

Using Clustered Server Instances

The most common way to deploy the load balancer is with a cluster or clusters of server instances. By default all the instances in a cluster have the same configuration and the same applications deployed to them. The load balancer distributes the workload between the server instances and requests fail over from an unhealthy instance to a healthy one. If you’ve configured HTTP session persistence, session information persists when the request is failed over.

If you have multiple clusters, requests are only load balanced and failed over between the instances in a single cluster. Use multiple clusters in a load balancer to easily enable rolling upgrades of applications. For more information, see Upgrading Applications Without Loss of Availability.

Using a Single, Stand-Alone Instance with Load Balancer Used as a Reverse-Proxy Plug-in

You can also configure your load balancer to use stand-alone server instance instead of a cluster. This configuration results in the load balancer plug-in working as a reverse-proxy plug-in (sometimes called a pass-through plug-in). When the web server receives requests for applications enabled in the load balancer, it forwards the requests directly to the Application Server.

Use the same procedures to configure the load balancer for a pass-through plug-in as you use to configure it for a cluster of server instances.

Using Multiple Stand-Alone Instances

It is also possible to configure your load balancer to use multiple stand-alone instances, and load balance and fail-over requests between them. However, in this configuration, you must manually ensure that the stand-alone instances have homogenous environments and the same applications deployed to them. Because clusters automatically maintain a homogenous environment, for most situations it is better and easier to use clusters.

Procedure to Set Up Load Balancing

Use the asadmin tool to configure load balancing in your environment. For more information on the asadmin commands used in these steps, see Configuring the Load Balancer

ProcedureTo Set Up Load Balancing

  1. Create a load balancer configuration using the asadmin command create-http-lb-config.

  2. Add a reference to a cluster or stand-alone server instance for the load balancer to manage using asadmin create-http-lb-ref.

    If you created the load balancer configuration with a target, and that target is the only cluster or stand-alone server instance the load balancer references, skip this step.

  3. Enable the cluster or stand-alone server instance referenced by the load balancer using asadmin enable-http-lb-server.

  4. Enable applications for load balancing using asadmin enable-http-lb-application.

    These applications must already be deployed and enabled for use on the clusters or stand-alone instances that the load balancer references. Enabling for load balancing is a separate step from enabling them for use.

  5. Create a health checker using asadmin create-health-checker.

    The health checker monitors unhealthy server instances so that when they become healthy again, the load balancer can send new requests to them.

  6. Generate the load balancer configuration file using asadmin export-http-lb-config .

    This command generates a configuration file to use with the load balancer plug-in shipped with the Sun Java System Application Server.

  7. Copy the load balancer configuration file to your web server config directory where the load balancer plug-in configuration files are stored.

Configuring Web Servers for Load Balancing

The load balancer plug-in installation program makes a few modifications to the web server’s configuration files. The changes made depend upon the web server.


Note –

The load balancer plug-in can be installed either along with Sun Java System Application Server Enterprise Edition, or separately, on a machine running the supported web server. For complete details on the installation procedure, see Chapter 1, Installing Application Server Software, in Sun Java System Application Server Enterprise Edition 8.1 2005Q2 Installation Guide (if using the standalone Application Server) or Sun Java Enterprise System 2005Q5 Installation Guide (if using Java Enterprise System).


Modifications to Sun Java System Web Server

The installation program adds the following entries to the Sun Java System Web Server’s configuration files:

To the web server instance’s magnus.conf file, it adds:

##EE lb-pluginInit 
fn="load-modules"
shlib="web_server_install_dir/plugins/lbplugin/bin/libpassthrough.so" 
funcs="init-passthrough,service-passthrough,name-trans-passthrough" Thread="no"
Init fn="init-passthrough"
##end addition for EE lb-plugin

To the web server instance’s obj.conf file, it adds:

<Object name=default>
NameTrans fn="name-trans-passthrough" name="lbplugin" 
config-file="web_server_install_dir/web_server_instance/config/loadbalancer.xml"
<Object name="lbplugin">
  ObjectType fn="force-type" type="magnus-internal/lbplugin"
  PathCheck fn="deny-existence" path="*/WEB-INF/*"
  Service type="magnus-internal/lbplugin" 
  fn="service-passthrough"
  Error reason="Bad Gateway" 
  fn="send-error" 
  uri="$docroot/badgateway.html"
</object>

In the above code, lbplugin is a name that uniquely identifies the Object, and web_server_install_dir/ web_server_instance/config/loadbalancer.xml is the location of the XML configuration file for the virtual server on which the load balancer is configured to run.

After installing, configure the load balancer as described in Setting Up HTTP Load Balancing

Using Apache Web Server

To use Apache Web Server, you must perform certain configuration steps before installing the load balancer plug-in. The load balancer plug-in installation also makes additional modifications to the Apache Web Server. After the plug-in is installed, you must perform additional configuration steps.


Note –

On Apache 1.3, when more than one Apache child processes runs, each process has its own load balancing round robin sequence. For example, if there are two Apache child processes running, and the load balancer plug-in load balances on to two application server instances, the first request is sent to instance #1 and the second request is also sent to instance #1. The third request is sent to instance #2 and the fourth request is sent to instance #2 again. This pattern is repeated (instance1, instance1, instance2, instance2, etc.)This behavior is different from what you might expect, that is, instance1, instance2, instance1, instance2, etc. In Sun Java System Application Server, the load balancer plug-in for Apache instantiates a load balancer instance for each Apache process, creating an independent load balancing sequence.

Apache 2.0 has multithreaded behavior if compiled with the --with-mpm=worker option.


Requirements for Using Apache Web Server

For the Apache Web Server, your installation must meet the minimum requirements, depending on the version of Apache.

Requirements for Apache 1.3

With Apache 1.3, the load balancer plug-in requires:


Note –

To use C compiler other than gcc, set the path of the C compiler and make utility in the PATH environment variable. For example, with the sh shell: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:appserver_installdir/lib


Minimum Requirements for Apache 2

With Apache 2.0, the load balancer plug-in requires:

The software sources are available at http://www.sunfreeware.com

In addition, before compiling Apache:


Note –

To use a C compiler other than gcc, set the path of the C compiler and make utility in the PATH environment variable. For example, with the sh shell:export LD_LIBRARY_PATH= app_server_install_dir/lib:$LD_LIBRARY_PATH.


Configuration before Installing the Load Balancer Plug-in

Before installing the load balancer plug-in for Apache, install the Apache Web Server. The Apache source must be compiled and built to run with SSL. This section describes the minimum requirements and high-level steps needed to successfully compile Apache Web Server to run the load balancer plug-in. These requirements and steps only apply to the Solaris and Linux versions of the software. For information on the Windows version of Apache, see the Apache web site.

ProcedureTo Install SSL-aware Apache

Before You Begin

You must have already downloaded and uncompressed the Apache software.

  1. Download and unpack the OpenSSL source.

  2. Compile and build OpenSSL.

    This step is not required on the Linux platform if OpenSSL 0.9.7.e is installed.

    Enter these commands:


    cd openssl-0.9.7e
    make
    make install

    For more information about OpenSSL, see the http://www.openssl.org/.

  3. Follow one of these procedures, depending on the version of Apache:

    • For Apache 1.3, configure Apache with mod_ssl with the following steps:

      1. Unpack the mod_ssl source.

      2. cd mod_ssl-2.8.14–1.3.x

      3. ./configure –with-apache=../apache_1.3.x --with-ssl=../openssl-0.9.7e --prefix=install_path --enable-module=ssl --enable-shared=ssl --enable-rule=SHARED_CORE --enable-module=so

      In the above commands, x is the Apache version number, and install_path is the directory in which to install Apache.

      For more information on mod_ssl, see http://www.modssl.org.

    • For Apache 2.0, configure the source tree:

      1. cd http-2.0_ x.

      2. Run ./configure --with-ssl= open_ssl_install_path --prefix= install_path --enable-ssl --enable-so

        In the above commands, x is the Apache version number, open_ssl_install_path is the directory where OpenSSL is installed, and install_path is the directory in which to install Apache.

  4. For Apache on Linux 2.1, before compiling:

    1. Open src/MakeFile and find the end of the automatically generated section.

    2. Add the following lines after the first four lines after the automatically generated section:

      LIBS+= -licuuc -licui18n -lnspr4 -lpthread -lxerces-c 
      -lsupport -lnsprwrap -lns-httpd40
      LDFLAGS+= -L/appserver_installdir/lib -L/opt/sun/private/lib

      Note that -L/opt/sun/private/lib is only required if you installed Application Server as part of a Java Enterprise System installation.

      For example:

      ## (End of automatically generated section)
      ## 
      CFLAGS=$(OPTIM) $(CFLAGS1) $(EXTRA_CFLAGS)
      LIBS=$(EXTRA_LIBS) $(LIBS1)
      INCLUDES=$(INCLUDES1) $(INCLUDES0) $(EXTRA_INCLUDES)
      LDFLAGS=$(LDFLAGS1) $(EXTRA_LDFLAGS)
      "LIBS+= -licuuc -licui18n -lnspr4 -lpthread 
      -lxerces-c -lsupport -lnsprwrap -lns-httpd40
      LDFLAGS+= -L/appserver_installdir /lib -L/opt/sun/private/lib
    3. Set environment variable LD_LIBRARY_PATH.

      With all installations, set it to: appserver_install_dir/lib

      With Java Enterprise System Installations, set it to appserver_install_dir/lib:opt/sun/private/lib .

  5. Compile Apache as described in the installation instructions for the version you are using.

    For more information, see the http://httpd.apache.org/

    In general the steps are:

    1. make

    2. make certificate (Apache 1.3 only)

    3. make install

      The command make certificate asks for a secure password. Remember this password as it is required for starting secure Apache.

  6. Configure Apache for your environment.

Modifications Made by the Application Server Installer

The load balancer plug-in installation program extracts the necessary files to a directory in the web server’s root directory:

It adds the following entries to the web server instance’s httpd.conf file:

<VirtualHost machine_name:443>
##Addition for EE lb-plugin
LoadFile /usr/lib/libCstd.so.1
LoadModule apachelbplugin_module libexec/mod_loadbalancer.so
#AddModule mod_apachelbplugin.cpp
<IfModule mod_apachelbplugin.cpp> 
  config-file webserver_instance/conf/loadbalancer.xml
  locale en
</IfModule>
<VirtualHost machine_ip_address>
  DocumentRoot "webserver_instance/htdocs"
  ServerName server_name
</VirtualHost>
##END EE LB Plugin ParametersVersion 7

ProcedureTo configure Apache Security Files to work with the Load Balancer

Apache Web Server must have the correct security files to work well with the load balancer plug-in.

  1. Create a directory called sec_db_files under apache_install_dir.

  2. Copy application_server_domain_dir /config/*.db to apache_install_dir /sec_db_files.

  3. Depending on the platform, perform additional configuration.

    • On the Solaris platform:

      Add the path /usr/lib/mps/secv1 to LD_LIBRARY_PATH in the apache_install_dir/bin/apachectl script. The path must be added before /usr/lib/mps.

    • On Linux:

      Add the path /opt/sun/private/lib to LD_LIBRARY_PATH in the apache_install_dir/bin/apachectl script. The path must be added before /usr/lib.

    • On Microsoft Windows:

      1. Add a new path to the Path environment variable.

        Click Start->Settings->Control Panel->System->Advanced->Environment Variables->System Variables.

        Add application_server_install_dir /bin to the Path environment variable.

      2. Set the environment variable NSPR_NATIVE_THREADS_ONLY to 1.

        In the Environment Variables window, under System Variables, click New. Enter Variable name of NSPR_NATIVE_THREADS_ONLY and Variable value of 1.

      3. Restart the machine.

Modifications to Microsoft IIS

To configure Microsoft Internet Information Services (IIS) to use the load balancer plug-in, modify certain properties in Windows Internet Services Manager. The Internet Services Manager is located in the Administrative Tools folder in the Control Panel folder.

Make these modifications after installing the Sun Java System Application Server.

ProcedureTo Configure Microsoft IIS to use the Load Balancer Plug-in

  1. Open the Internet Services Manager.

  2. Select the web site for which you want to enable the plug-in.

    This web site is typically named the Default Web Site.

  3. Right click on the web site and select Properties to open the Properties notebook.

  4. Add a new ISAPI filter, following these steps:

    1. Open the ISAPI Filters tab.

    2. Click Add.

    3. In the Filter Name field, enter Application Server

    4. In the Executable field, type C:\Inetpub\wwwroot\sun-passthrough\sun-passthrough.dll

    5. Click OK, and close the Properties notebook.

  5. Create and configure a new virtual directory:

    1. Right click on the default web site, select New, and then Virtual Directory.

      The Virtual Directory Creation Wizard opens.

    2. In the Alias field, type sun-passthrough .

    3. In the Directory field, type C:\Inetpub\wwwroot\sun-passthrough.

    4. Check the Execute Permission checkbox.

      Leave all other permission-related check boxes are left unchecked.

    5. Click Finish.

  6. Add the path of sun-passthrough.dll file and application_server_install_dir /bin to the system’s PATH environment variable.

  7. Restart the machine.

  8. Stop and start the web server for the new settings to take effect.

    To stop the web server, right click on the web site and select Stop . To start the web server, right click on the web site and select Start.

  9. Verify that the web server, load balancer plug-in, and Application Server are operating correctly.

    Type the following in a web browser to access the web application context root: http://webserver_name/web_application, where webserver_name is the host name or IP address of the web server and web_application is the context root that you listed in the C:\Inetpub\wwwroot\sun-passthrough\sun-passthrough.properties file.

Automatically configured Sun-passthrough properties

The installer automatically configures the following properties in sun-passthrough.properties. You can change the default values.

Property 

Definition 

Default Value 

lb-config-file 

Path to the load balancer configuration file 

IIS_www_root\sun-passthrough\loadbalancer.xml

log-file 

Path to the load balancer log file 

IIS_www_root\sun-passthrough\lb.log

log-level 

Log level for the web server 

INFO 

Configuring Multiple Web Server Instances

The Sun Java System Application Server installer does not allow the installation of multiple load balancer plug-ins on a single machine. To have multiple web servers with the load balancer plug-in on a single machine, in either a single cluster or multiple clusters, a few manual steps are required to configure the load balancer plug-in.

ProcedureTo Configure Multiple Web Server Instances

  1. Configure the new web server instance to use the load balancer plug-in.

    Follow the steps in Modifications to Sun Java System Web Server or Using Apache Web Server , or Installation

  2. Copy the DTD file.

    Copy sun-loadbalancer_1_1.dtd from the existing web server instance’s config directory to the new instance’s config directory.

  3. Set up the load balancer configuration file. Either:

    • Copy the existing load balancer configuration.

      Use an existing load balancer configuration, copy the loadbalancer.xml file from the existing web server instance’s config directory to the new instance’s config directory.

    • Create a new load balancer configuration:

      1. Use asadmin create-http-lb-config to create a new load balancer configuration.

      2. Export the new configuration to a loadbalancer.xml file using asadmin export http-lb-config.

      3. Copy that loadbalancer.xml file to the new web server’s config directory.

        For information on creating a load balancer configuration and exporting it to a loadbalancer.xml file, see Creating an HTTP Load Balancer Configuration

Configuring the Load Balancer

A load balancer configuration is a named configuration in the domain.xml file. Load balancer configuration is extremely flexible:

This section describes how to create, modify, and use a load balancer configuration, including the following topics:

Creating an HTTP Load Balancer Configuration

Create a load balancer configuration using the asadmin command create-http-lb-config. Creating an HTTP Load Balancer Configuration describes the parameters. For more information see the documentation for create-http-lb-config, delete-http-lb-config, and list-http-lb-configs.

Table 4–1 Load Balancer Configuration Parameters

Parameter 

Description 

response timeout 

Time in seconds within which a server instance must return a response. If no response is received within the time period, the server is considered unhealthy. The default is 60. 

HTTPS routing

Whether HTTPS requests to the load balancer result in HTTPS or HTTP requests to the server instance. For more information, see Configuring HTTPS Routing.

reload interval 

Interval between checks for changes to the load balancer configuration file loadbalancer.xml. When the check detects changes, the configuration file is reloaded. A value of 0 disables reloading.For more information, see Enabling Dynamic Reconfiguration

monitor 

Whether monitoring is enabled for the load balancer. 

routecookie

Name of the cookie the load balancer plug-in uses to record the route information. The HTTP client must support cookies. If your browser is set to ask before storing a cookie, the name of the cookie is JROUTE. 

target 

Target for the load balancer configuration. If you specify a target, it is the same as adding a reference to it. Targets can be clusters or stand-alone instances.

Creating an HTTP Load Balancer Reference

When you create a reference in the load balancer to a stand-alone server or cluster, the server or cluster is added to the list of target servers and clusters the load balancer controls. The referenced server or cluster still needs to be enabled (using enable-http-lb-server) before requests to it are load balanced. If you created the load balancer configuration with a target, that target is already added as a reference.

Create a reference using create-http-lb-ref. You must supply the load balancer configuration name and the target server instance or cluster.

To delete a reference, use delete-http-lb-ref. Before you can delete a reference, the referenced server or cluster must be disabled using disable-http-lb-server .

For more information, see the documentation for create-http-lb-ref and delete-http-lb-ref.

Enabling Server Instances for Load Balancing

After creating a reference to the server instance or cluster, enable the server instance or cluster using enable-http-lb-server . If you used a server instance or cluster as the target when you created the load balancer configuration, you must enable it.

For more information, see the documentation for enable-http-lb-server .

Enabling Applications for Load Balancing

All servers managed by a load balancer must have homogenous configurations, including the same set of applications deployed to them. Once an application is deployed and enabled for access (this happens during or after the deployment step) you must enable it for load balancing. If an application is not enabled for load balancing, requests to it are not load balanced and failed over, even if requests to the servers the application is deployed to are load balanced and failed over.

When enabling the application, specify the application name and target. If the load balancer manages multiple targets (for example, two clusters), enable the application on all targets.

For more information, see the online help for enable-http-lb-application .

If you deploy a new application, you must also enable it for load balancing and export the load balancer configuration again.

Creating the HTTP Health Checker

The load balancer’s health checker periodically checks all the configured Application Server instances that are marked as unhealthy. A health checker is not required, but if no health checker exists, or if the health checker is disabled, the periodic health check of unhealthy instances is not performed.

The load balancer’s health check mechanism communicates with the application server instance using HTTP. The health checker sends an HTTP request to the URL specified and waits for a response. A status code in the HTTP response header between 100 and 500 means the instance is healthy.

Creating a Health Checker

To create the health checker, use the asadmin create-http-health-checke r command. Specify the following parameters:

Table 4–2 Health Checker Parameters

Parameter 

Description 

Default 

url 

Specifies the listener’s URL that the load balancer checks to determine its state of health.  

“/” 

interval 

Specifies the interval in seconds at which health checks of instances occur. Specifying 0 disables the health checker. 

30 seconds 

timeout 

Specifies the timeout interval in seconds within which a response must be obtained for a listener to be considered healthy.  

10 seconds 

If an application server instance is marked as unhealthy, the health checker polls the unhealthy instances to determine if the instance has become healthy. The health checker uses the specified URL to check all unhealthy application server instances to determine if they have returned to the healthy state.

If the health checker finds that an unhealthy instance has become healthy, that instance is added to the list of healthy instances.

For more information see the documentation for create-http-health-checker and delete-http-health-checker.

Additional Health Check Properties for Healthy Instances

The health checker created by create-http-health-checker only checks unhealthy instances. To periodically check healthy instances set some additional properties in your exported loadbalancer.xml file.


Note –

These properties can only be set by manually editing loadbalancer.xml after you’ve exported it. There is no equivalent asadmin command to use.


To check healthy instances, set the following properties:

Table 4–3 Health-checker Manual Properties

Property 

Definition 

active-healthcheck-enabled

True/false flag indicating whether to ping healthy server instances to determine whether they are healthy. To ping server instances, set the flag to true. 

number-healthcheck-retries

Specifies how many times the load balancer’s health checker pings an unresponsive server instance before marking it unhealthy. Valid range is between 1 and 1000. A default value to set is 3. 

Set the properties by editing the loadbalancer.xml file. For example:

<property name="active-healthcheck-enabled" value="true"/>
<property name="number-healthcheck-retries" value="3"/>

If you add these properties, then subsequently edit and export the loadbalancer.xml file again, you must add these properties to the file again, since the newly exported configuration won’t contain them.

Exporting the Load Balancer Configuration File

The load balancer plug-in shipped with Sun Java System Application Server uses a configuration file called loadbalancer.xml. Use the asadmin tool to create a load balancer configuration in the domain.xml file. After configuring the load balancing environment, export it to a file.

ProcedureTo export the load balancer configuration

  1. Export a loadbalancer.xml file using the asadmin command export-http-lb-config.

    Export the loadbalancer.xml file for a particular load balancer configuration. You can specify a path and a different file name. If you don’t specify a file name, the file is named loadbalancer.xml. load_balancer_config_name. If you don’t specify a path, the file is created in the application_server_install_dir /domains/domain_name /generated directory.

    To specify a path on Windows, use quotes around the path. For example, "c:\sun\AppServer\loadbalancer.xml" .

  2. Copy the exported load balancer configuration file to the web server’s configuration directory.

    For example, for the Sun Java System Web Server, that location might be web_server_root/config .

    The load balancer configuration file in the web server’s configuration directory must be named loadbalancer.xml. If your file has a different name, such as loadbalancer.xml. load_balancer_config_name, you must rename it.

Changing the Load Balancer Configuration

If you change a load balancer configuration by creating or deleting references to servers, deploying new applications, enabling or disabling servers or applications, and so on, export the load balancer configuration file again and copy it to the web server’s config directory. For more information, see Exporting the Load Balancer Configuration File

The load balancer plug-in checks for an updated configuration periodically based on the reload interval specified in the load balancer configuration. After the specified amount of time, if the load balancer discovers a new configuration file, it starts using that configuration.

Enabling Dynamic Reconfiguration

With dynamic reconfiguration, the load balancer plug-in periodically checks for an updated configuration.

To enable dynamic reconfiguration:


Note –

If the load balancer encounters a hard disk read error while attempting to reconfigure itself, then it uses the configuration that is currently in memory. The load balancer also ensures that the modified configuration data is compliant with the DTD before over writing the existing configuration.

If a disk read error is encountered, a warning message is logged to the web server’s error log file.

The error log for Sun Java System Web Server’ is at: web_server_install_dir/webserver_instance /logs/.


Disabling (Quiescing) a Server Instance or Cluster

Before stopping an application server for any reason, you want the instance to complete serving requests. The process of gracefully disabling a server instance or cluster is called quiescing.

The load balancer uses the following policy for quiescing application server instances:

ProcedureTo disable a server instance or cluster

  1. Run asadmin disable-http-lb-server, setting the timeout (in minutes).

  2. Export the load balancer configuration file using asadmin export-http-lb-config.

  3. Copy the exported configuration to the web server config directory.

  4. Stop the server instance or instances.

Disabling (Quiescing) an Application

Before you undeploy a web application, you want to the application to complete serving requests. The process of gracefully disabling an application is called quiescing. When you quiesce an application, you specify a timeout period. Based on the timeout period, the load balancer uses the following policy for quiescing applications:

ProcedureTo disable an application

  1. Use asadmin disable-http-lb-application, specifying the following:

    • Timeout (in minutes).

    • Name of the application to disable.

    • Target cluster or instance on which to disable it.

  2. Export the load balancer configuration file using asadmin export-http-lb-config.

  3. Copy the exported configuration to the web server config directory.

Configuring HTTP and HTTPS Failover

The load balancer plug-in fails over HTTP/HTTPS sessions to another application server instance if the original application server instance to which the session was connected becomes unavailable. This section describes how to configure the load balancer plug-in to enable HTTP/HTTPS routing and session failover.

This section discusses the following topics:

HTTPS Routing

The HTTP Secure (HTTPS) protocol uses Secure Sockets Layer (SSL) to provide encryption an decryption of HTTP requests for secure communication. For HTTPS routing to work, one or more HTTPS listeners must be configured.

The load balancer plug-in routes all incoming HTTP or HTTPS requests to application server instances. However, if HTTPS routing is enabled, an HTTPS request will be forwarded by the load balancer plug-in to the application server using an HTTPS port only. HTTPS routing is performed on both new and sticky requests.

If an HTTPS request is received and no session is in progress, then the load balancer plug-in selects an available application server instance with a configured HTTPS port, and forwards the request to that instance.

In an ongoing HTTP session, if a new HTTPS request for the same session is received, then the session and sticky information saved during the HTTP session is used to route the HTTPS request. The new HTTPS request is routed to the same server where the last HTTP request was served, but on the HTTPS port.

Configuring HTTPS Routing

The httpsrouting option of the create-http-lb-config command controls whether HTTPS routing is turned on or off for all the application servers that are participating in load balancing. If this option is set to false, all HTTP and HTTPS requests are forwarded as HTTP. Set it to true when creating a new load balancer configuration, or change it later using the asadmin set command.


Note –

If https-routing is set to true, and a new or a sticky request comes in where there are no healthy HTTPS listeners in the cluster, then that request generates an error.


Known Issues

The Load Balancer has the following limitations with HTTP/HTTPS request processing.

Configuring Idempotent URLs

An idempotent request is one that does not cause any change or inconsistency in an application when retried. In HTTP, some methods (such as GET) are idempotent, while other methods (such as POST) are not. Retrying an idempotent URL must not cause values to change on the server or in the database. The only difference is a change in the response received by the user.

Examples of idempotent requests include search engine queries and database queries. The underlying principle is that the retry does not cause an update or modification of data.

To enhance the availability of deployed applications, configure the environment to retry failed idempotent HTTP requests on all the application server instances serviced by a load balancer. This option is used for read-only requests, for example, to retry a search request.

Configure idempotent URLs in the sun-web.xml file. When you export the load balancer configuration, idempotent URL information is automatically added to the loadbalancer.xml file.

For more information on configuring idempotent URLs, see Configuring Idempotent URL Requests in Sun Java System Application Server Enterprise Edition 8.1 2005Q2 Developer’s Guide.

Upgrading Applications Without Loss of Availability

Upgrading an application to a new version without loss of availability to users is called a rolling upgrade. Carefully managing the two versions of the application across the upgrade ensures that current users of the application complete their tasks without interruption, while new users transparently get the new version of the application. With a rolling upgrade, users are unaware that the upgrade occurs.

Application Compatibility

Rolling upgrades pose varying degrees of difficulty depending on the magnitude of changes between the two application versions.

If the changes are superficial, for example, changes to static text and images, the two versions of the application are compatible and can both run at once in the same cluster. Compatible applications must:

You can perform a rolling upgrade of a compatible application in either a single cluster or multiple clusters. For more information, see Upgrading In a Single Cluster

If the two versions of an application do not meet all the above criteria, then the applications are considered incompatible. Executing incompatible versions of an application in one cluster can corrupt application data and cause session failover to not function correctly. The problems depend on the type and extent of the incompatibility. It is good practice to upgrade an incompatible application by creating a “shadow cluster” to which to deploy the new version and slowly quiesce the old cluster and application. For more information, see Upgrading Incompatible Applications

The application developer and administrator are the best people to determine whether application versions are compatible. If in doubt, assume that the versions are incompatible, since this is the safest approach.

Upgrading In a Single Cluster

You can perform a rolling upgrade of an application deployed to a single cluster, providing the cluster’s configuration is not shared with any other cluster.

ProcedureTo upgrade an application in a single cluster

  1. Save an old version of the application or back up the domain.

    To back up the domain use the asadmin backup-domain command.

  2. Turn off dynamic reconfiguration (if enabled) for the cluster.

    To do this with Admin Console:

    1. Expand the Configurations node.

    2. Click the name of the cluster’s configuration.

    3. On the Configuration System Properties page, uncheck the Dynamic Reconfiguration Enabled box.

    4. Click Save

    Alternatively, use this command:

    asadmin set --user user --passwordfile password_file cluster_name -config.dynamic-reconfiguration-enabled=false

  3. Redeploy the upgraded application to the target domain.

    If you redeploy using the Admin Console, the domain is automatically the target. If you use asadmin, specify the target domain. Because dynamic reconfiguration is disabled, the old application continues to run on the cluster.

  4. Enable the redeployed application for the instances using asadmin enable-http-lb-application.

  5. Quiesce one server instance in the cluster from the load balancer.

    Follow these steps:

    1. Disable the server instance using asadmin disable-http-lb-server.

    2. Export the load balancer configuration file using asadmin export-http-lb-config.

    3. Copy the exported configuration file to the web server instance’s configuration directory.

      For example, for Sun Java System Web Server, the location is web_server_install_dir/ https- host-name /config/loadbalancer.xml . To ensure that the load balancer loads the new configuration file, be sure that dynamic reconfiguration is enabled by setting the reloadinterval in the load balancer configuration.

    4. Wait until the timeout has expired.

      Monitor the load balancer’s log file to make sure the instance is offline. If users see a retry URL, skip the quiescing period and restart the server immediately.

  6. Restart the disabled server instance while the other instances in the cluster are still running.

    Restarting causes the server to synchronize with the domain and update the application.

  7. Test the application on the restarted server to make sure it runs correctly.

  8. Re-enable the server instance in load balancer.

    Follow these steps:

    1. Enable the server instance using asadmin enable-http-lb-server.

    2. Export the load balancer configuration file using asadmin export-http-lb-config.

    3. Copy the configuration file to the web server’s configuration directory as described in Upgrading In a Single Cluster of Upgrading In a Single Cluster .

  9. Repeat steps 5 through 8 for each instance in the cluster.

  10. When all server instances have the new application and are running, you can re-enable dynamic reconfiguration for the cluster again.

Upgrading in Multiple Clusters

ProcedureTo upgrade a compatible application in two or more clusters:

  1. Save an old version of the application or back up the domain.

    To back up the domain use the asadmin backup-domain command.

  2. Turn off dynamic reconfiguration (if enabled) for all clusters.

    To do this with Admin Console:

    1. Expand the Configurations node.

    2. Click the name of one cluster’s configuration.

    3. On the Configuration System Properties page, uncheck the Dynamic Reconfiguration Enabled box.

    4. Click Save

    5. Repeat for the other clusters

    Alternatively, use this command:

    asadmin set --user user --passwordfile password_file cluster_name-config.dynamic-reconfiguration-enabled=false

  3. Redeploy the upgraded application to the target domain.

    If you redeploy using the Admin Console, the domain is automatically the target. If you use asadmin, specify the target domain. Because dynamic reconfiguration is disabled, the old application continues to run on the clusters.

  4. Enable the redeployed application for the clusters using asadmin enable-http-lb-application.

  5. Quiesce one cluster from the load balancer

    1. Disable the cluster using asadmin disable-http-lb-server.

    2. Export the load balancer configuration file using asadmin export-http-lb-config.

    3. Copy the exported configuration file to the web server instance’s configuration directory.

      For example, for Sun Java System Web Server, the location is web_server_install_dir/ https- host-name /config/loadbalancer.xml . Dynamic reconfiguration must be enabled for the load balancer (by setting the reloadinterval in the load balancer configuration), so that the new load balancer configuration file is loaded automatically.

    4. Wait until the timeout has expired.

      Monitor the load balancer’s log file to make sure the instance is offline. If users see a retry URL, skip the quiescing period and restart the server immediately.

  6. Restart the disabled cluster while the other clusters are still running.

    Restarting causes the cluster to synchronize with the domain and update the application.

  7. Test the application on the restarted cluster to make sure it runs correctly.

  8. Re-enable the cluster in load balancer:

    1. Enable the cluster using asadmin enable-http-lb-server.

    2. Export the load balancer configuration file using asadmin export-http-lb-config.

    3. Copy the configuration file to the web server’s configuration directory.

  9. Repeat steps 5 through 8 for the other clusters.

  10. When all server instances have the new application and are running, you can reenable dynamic reconfiguration for all clusters again.

Upgrading Incompatible Applications

For information on what makes applications compatible, see Application Compatibility the new version of the application is incompatible with the old. Also, you must upgrade incompatible application in two or more clusters. If you have only one cluster, create a “shadow cluster” for the upgrade, as described below.

When upgrading an incompatible application:

ProcedureTo upgrade an incompatible application by creating a second cluster

  1. Save an old version of the application or back up the domain.

    To back up the domain use the asadmin backup-domain command.

  2. Create a “shadow cluster” on the same or a different set of machines as the existing cluster.

    1. Use the Admin Console to create the new cluster and reference the existing cluster’s named configuration.

      Customize the ports for the new instances on each machine to avoid conflict with existing active ports.

    2. For all resources associated with the cluster, add a resource reference to the newly created cluster using asadmin create-resource-ref.

    3. Create a reference to all other applications deployed to the cluster (except the current redeployed application) from the newly created cluster using asadmin create-application-ref.

    4. Configure the cluster to be highly available using asadmin configure-ha-cluster.

    5. Create reference to the newly-created cluster in the load balancer configuration file using asadmin create-http-lb-ref.

  3. Give the new version of application a different name from the old version.

  4. Deploy the new application with the new cluster as the target. Use a different context root or roots.

  5. Enable the deployed new application for the clusters using asadmin enable-http-lb-application.

  6. Start the new cluster while the other cluster is still running.

    The start causes the cluster to synchronize with the domain and be updated with the new application.

  7. Test the application on the new cluster to make sure it runs correctly.

  8. Disable the old cluster from the load balancer using asadmin disable-http-lb-server.

  9. Set a timeout for how long lingering sessions survive.

  10. Enable the new cluster from the load balancer using asadmin enable-http-lb-server.

  11. Export the load balancer configuration file using asadmin export-http-lb-config.

  12. Copy the exported configuration file to the web server instance’s configuration directory.

    For example, for Sun Java System Web Server, the location is web_server_install_dir/ https- host-name /config/loadbalancer.xml . Dynamic reconfiguration must be enabled for the load balancer (by setting the reloadinterval in the load balancer configuration), so that the new load balancer configuration file is loaded automatically.

  13. After the timeout period expires or after all users of the old application have exited, stop the old cluster and delete the old application.

Monitoring the HTTP Load Balancer Plug-in

Configuring Log Messages

The load balancer plug-in uses the web server’s logging mechanism to write log messages. The default log level on the Application Server is set to the default logging level on Sun Java System Web Server (INFO), Apache Web Server (WARN) and Microsoft IIS (INFO). The application server log levels - FINE, FINER, and FINEST map to the DEBUG level on the web server.

These log messages are written to the web server log files, and are in the form of raw data that can be parsed using scripts, or imported into spreadsheets to calculate required metrics.

Types of Log Messages

The load balancer plug-in generates the following types of log messages:

Load Balancer Configurator Log Messages

These messages will be logged when you are using idempotent URLs and error page settings.

An output for idempotent URL pattern configuration contains the following information:

Request Dispatch and Runtime Log Messages

These log messages are generated while a request is being load balanced and dispatched.

Configurator Error Messages

These errors appear if there are configuration problems, for example, if the custom error page referenced is missing.

Enabling Load Balancer Logging

The load balancer plug-in logs the following information:


Note –

When load balancer logging is enabled, and if the web server logging level is set to DEBUG or to print verbose messages, the load balancer writes HTTP session IDs in the web server log files. Therefore, if the web server hosting the load balancer plug-in is in the DMZ, do not use the DEBUG or similar log level in production environments.

If you must use the DEBUG logging level, turn off load balancer logging by setting require-monitor-data property to false in loadbalancer.xml.


ProcedureTo turn on load balancer logging

  1. Set the logging options in the web server. The procedure depends on the web server:

    • With Sun Java System Web Server

      In the server’s admin console, go to the Magnus Editor tab and set the Log Verbose option to On.

    • For Apache Web Server, set the log level to DEBUG.

    • For Microsoft IIS, set the log level to FINE in the sun-passthrough.properties file.

  2. Set the load balancer configuration’s monitor option to true.

    Use the asadmin create-http-lb-config command to set monitoring to true when you initially create the load balancer configuration, or use the asadmin set command to set it to true later. Monitoring is disabled by default.

Understanding Monitoring Messages

The format of the load balancer plug-in log messages is as follows.