Sun ONE Application Server 7, Enterprise Edition Administrator's Guide |
Chapter 16
Configuring Load BalancingThis chapter describes how to setup load balancing of HTTP requests, configure, and administer the load balancer plug-in.
This chapter includes the following topics:
About Load BalancingSun Open Net Environment (ONE) Application Server 7, Enterprise Edition, provides load balancing and HTTP session persistence. The goal of load balancing is to evenly distribute the workload between multiple Sun ONE Application Server instances, thereby increasing the overall throughput of the system.
The load balancer plug-in is an enhanced version of the HTTP reverse proxy plug-in used in Sun ONE Application Server 7, Standard Edition. For more information on the reverse proxy plug-in, see Chapter 7, “Configuring the Web Server Plugin,” in Sun ONE Application Server 7, Standard Edition Administrator’s Guide.
This section addresses the following topics:
Load Balancing Algorithm
Sun ONE Application Server load balancer uses sticky round robin algorithm to load balance incoming HTTP and HTTPS requests. This means that all requests for a given session are sent to the same application server instance. With a sticky load balancer, the session data can be cached on a single application server rather than having to be distributed to all instances in a cluster.
Therefore, the sticky round robin scheme provides significant performance benefits that normally override the benefits of a more evenly distributed load that can be obtained with pure round robin.
About Sticky Round Robin Load Balancing Algorithm
When a new HTTP request is sent to the load balancer plug-in, it’s forwarded to an application server instance based on a simple round robin scheme. Subsequently, this request is “stuck” to this particular application server instance, either by using cookies, or explicit URL rewriting.
From the sticky information, the load balancer plug-in first determines the instance to which the request was previously forwarded. If that instance is found to be healthy, the load balancer plug-in forwards the request to that specific application server instance. Therefore, all requests for a given session are sent to the same application server instance.
The load balancer plugin uses the following methods to determine session stickiness:
Cookie Based Method
In the cookie based method, the load balancer plug-in uses a separate cookie to record the route information.
Explicit URL Rewriting Method
In the explicit URL rewriting method, the sticky information is appended to the URL. This method works even if the browser does not support cookies.
Requirements for Load Balancing
You must meet the following requirements before you can use the load balancer plug-in:
Configuring Load Balancer Plug-inThis section discusses the following topics:
Configuration Changes to Web Server
The load balancer plug-in installation program makes a few modifications to the web server’s configuration files. This section details the changes to the following web server’s configuration files:
Modifications to Sun ONE Web Server Configuration
The installation program makes the following changes to the Sun ONE Web Server’s configuration files:
- Adds the following load balancer plug-in specific entries to the web server instance’s magnus.conf file:
##EE lb-plugin
Init fn="load-modules" shlib="webserver_install_dir/webserver_instance/plugins/lbplugin/bin/libpassthrough.so" funcs="init-passthrough,service-passthrough,name-trans-passthrough" Thread="no"Init fn="init-passthrough"
##end addition for EE lb-plugin
- Adds the following entries specific to the load balancer plug-in to the web server instance’s obj.conf file:
<Object name=default>
NameTrans fn="name-trans-passthrough" name="lbplugin" config-file="webserver_install_dir/webserver_instance/config/loadbalancer.xml"
<Object name="lbplugin">
ObjectType fn="force-type" type="magnus-internal/lbplugin"
PathCheck fn="deny-existence" path="*/WEB-INF/*"
Service type="magnus-internal/lbplugin" fn="service-passthrough"
Error reason="Bad Gateway" fn="send-error" uri="$docroot/badgateway.html"
</object>lbplugin is a name that uniquely identifies the Object, and webserver_install_dir/webserver_instance/config/loadbalancer.xml is the location of the XML configuration file for the virtual server on which the load balancer is configured to run.
You must now configure the load balancing plug-in, by editing the loadbalancer.xml configuration file. For more information on editing the loadbalancer.xml file, see "Creating the Load Balancer Configuration File".
Modifications to Apache Web Server
Before installling the load balancer plug-in on Apache, see information on compiling and configuring Apache in Appendix C, "Compiling and Configuring Apache Web Server."
The load balancer plug-in installation program extracts the necessary files to the modules folder under the web server’s root directory.It adds the following entries specific to the load balancer plug-in to the web server instance’s httpd.conf file:
<VirtualHost machine_name:443>
##Addition for EE lb-plugin
LoadFile /usr/lib/libCstd.so.1
LoadModule apachelbplugin_module libexec/mod_loadbalancer.so
#AddModule mod_apachelbplugin.cpp
<IfModule mod_apachelbplugin.cpp>
config-file webserver_instance/config/loadbalancer.xml
locale en
</IfModule><VirtualHost machine_ip_address>
DocumentRoot "webserver_instance/apache/htdocs"
ServerName server_name
</VirtualHost>##END EE LB Plugin ParametersVersion 7
This completes the installation of load balancer plug-in on Apache Web Server.
This following section describes how to create a loadbalancer.xml file and specify the necessary configuration details.
Creating the Load Balancer Configuration File
The loadbalancer.xml file should be created based on the sun-loadbalancer_1_0.dtd and loadbalancer.xml.example files installed during load balancer installation. These files are available in the following location:
For more information on the elements and their attributes, see "Load Balancer Configuration File".
Sample loadbalancer.xml file
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE loadbalancer PUBLIC "-//Sun Microsystems Inc.//DTD Sun ONE Web Server 6.0//EN" "file:///webserver_install_dir/webserver_instance/config/sun-loa dbalancer_1_0.dtd">
<loadbalancer>
<cluster name="cluster1">
<instance name="server1" enabled="true" listeners="http://austen.sun.com:80 https://http://austen.sun.com:443"/>
<instance name="server2" enabled="true" listeners="http://polar.sun.com:80 https://polar.sun.com:443"/>
<web-module context-root="webapps-simple" enabled="true"/>
</cluster><property name="response-timeout-in-seconds" value="120"/>
<property name="reload-poll-interval-in-seconds" value="180"/>
<property name="https-routing" value="true"/>
</loadbalancer>
- Save the changes made to the loadbalancer.xml file and restart your web server.
Note
- For Apache Web Server, the DTD file will be in the conf directory. Therefore, the target location of the DTD file should be specified as given in the following example: "file:///webserver_install_dir/webserver_instance/conf/sun-loadbalancer_1_0.dtd">
- For more information on creating and configuring application server instances, see Chapter 4, "Using Application Server Instances."
- The response-timeout-in-seconds property specifies the timeout interval within which a response must be obtained for a request that is load balanced, or an error page will be displayed.
- For information on clustering, see Chapter 17, "Managing Clusters."
For more information on loadbalancer.xml, see "Load Balancer Configuration File".
Configuring Multiple Web Server Instances
The Sun ONE Application Server 7, Enterprise Edition, installer will not allow the installation of multiple load balancer plug-ins on a single machine. If you need to have multiple web servers (with the load balancer plug-in), on a single machine, in either a single cluster or multiple clusters, a few manual steps are required to configure the load balancer plug-in.
Follow the procedure listed in this section to configure the load balancer plug-in for multiple web server instances.
Configuring Multiple Sun ONE Web Server Instances
- Make the following changes to the magnus.conf file of the web server instance on which you want to configure the load balancer plug-in:
Init fn="load-modules" shlib="webserver_install_dir/webserver_instance/plugins/lbplugin/bin/libpassthrough.so" funcs="init-passthrough,service-passthrough,name-trans-passthrough" Thread="no"
Init fn="init-passthrough"
- Make the following changes to the obj.conf file of the web server instance on which you want to configure the load balancer plug-in:
<Object name=default>
NameTrans fn="name-trans-passthrough" name="lbplugin" config-file="webserver_install_dir/webserver_instance/config/loadbalancer.xml"
<Object name="lbplugin">
ObjectType fn="force-type" type="magnus-internal/lbplugin"
PathCheck fn="deny-existence" path="*/WEB-INF/*"
Service type="magnus-internal/lbplugin" fn="service-passthrough"
Error reason="Bad Gateway" fn="send-error" uri="$docroot/badgateway.html"
</object>- Copy the sun-loadbalancer_1_0.dtd and loadbalancer.xml file from the existing (which is configured with the load balancer plug-in) web server instance’ config directory to new, load balancer configured, instance’s config directory.
- Edit the loadbalancer.xml file as described in "Creating the Load Balancer Configuration File".
Configuring Multiple Apache Web Server Instances
- Add the following entries specific to the load balancer plug-in to the web server instance’s httpd.conf file:
<VirtualHost machine_name:443>
##Addition for EE lb-plugin
LoadFile /usr/lib/libCstd.so.1
LoadModule apachelbplugin_module libexec/mod_loadbalancer.so
#AddModule mod_apachelbplugin.cpp
<IfModule mod_apachelbplugin.cpp>
config-file webserver_instance/config/loadbalancer.xml
locale en
</IfModule><VirtualHost machine_ip_address>
DocumentRoot "webserver_instance/apache/htdocs"
ServerName server_name
</VirtualHost>##END EE LB Plugin Parameters
- Copy the sun-loadbalancer_1_0.dtd and loadbalancer.xml file from the existing (which is configured with the load balancer plug-in) web server instance’ conf directory to new, load balancer configured, instance’s conf directory.
- Edit the loadbalancer.xml file as described in "Creating the Load Balancer Configuration File".
Configuring Response Timeout
The load balancer configuration file, loadbalancer.xml, contains a response-timeout-in-seconds property that specifies the timeout interval within which a response must be obtained for a request that is load balanced, or an error page will be displayed. The default value is 60 seconds.
While configuring response-timeout-in-seconds, you must take into account the maximum time-outs for all the applications that are running. The value should be set to the maximum response time of all the applications.
Configuring HTTPS Routing
This section describes how to configure the load balancer plug-in to enable HTTP/HTTPS routing and session fail over between the web server and Sun ONE Application Server. The load balancer plug-in fails over HTTP /HTTPS sessions over to another application server instance if the original application server instance to which the session was connected, becomes unavailable.
The following topics are discussed in this section:
About HTTPS Routing
All incoming requests, whether HTTP or HTTPS (secure HTTP), are routed by the load balancer plug-in to application server instances. However, if HTTPS routing is enabled, a HTTPS request will be forwarded by the load balancer plug-in to the application server using an HTTPS port only. Note that HTTPS routing is performed on both new and sticky requests.
If an HTTPS request is received and no session is in progress, then the load balancer plug-in selects an available application server instance with a configured HTTPS port, and forwards the request to that instance.
In an ongoing HTTP session, if a new HTTPS request for the same session is received, then the session and sticky information saved during the HTTP session is used to route the HTTPS request. The new HTTPS request is routed to the same server where the last HTTP request was served, but on the HTTPS port.
Configuring HTTPS Routing
The https-routing property controls whether HTTPS routing is turned on or off for all the application servers that are participating in load balancing. If this property is set to false, then all HTTP and HTTPS requests are forwarded as HTTP.
To configure HTTPS routing and failover, perform the following steps:
The https-routing property controls whether HTTPS routing is turned on or off for all the application servers that are participating in load balancing. If this property is set to false, then all HTTP and HTTPS requests are forwarded as HTTP.
Configuring the Health-Checker
The load balancer periodically checks all the configured Sun ONE Application Server instances that are marked as unhealthy, based on the values specified in the health-checker element. Enabling the health checker is optional. If the health-checker is not enabled, periodic health check of unhealthy instances will not be performed.
The load balancer’s health check mechanism communicates with the application server instance using HTTP. The health checker sends an HTTP request to the URL specified and waits for a response. The status code in the HTTP response header should be between 100 and 500 to consider the instance to be healthy.
To enable the health checker, edit the following properties in the loadbalancer.xml file:
- url
Specifies the listener’s URL that the load balancer will check to determine its state of health.
- interval-in-seconds
Specifies the interval at which health checks of instances occur. The default is 30 seconds.
- timeout-in-seconds
Specifies the timeout interval within which a response must be obtained for a listener to be considered healthy. The default is 10 seconds.
If an application server instance is marked as unhealthy, the health checker polls the unhealthy instances to determine if the instance has become healthy. The health checker uses the url attribute to check all unhealthy application server instances to check if they have returned to the healthy state.
If the health checker finds that an unhealthy instance has become healthy, that instance is added to the list of healthy instances.
Configuring Quiescence
Quiescing is the process of gracefully disabling an application server instance or web application. You can set a duration, called the quiescing period, during which time the load balancer plug-in fails over sticky requests to another healthy instance in the cluster.
If you want to stop an application server instance for any reason (for example to add a JDBC resource), you would want this instance to complete serving these requests before you stop it. Similarly, if you want to undeploy a web application from a cluster, you would want to this application to complete serving the requests before undeploying it. You can achieve this by using the quiescing process.
For detailed information on quiescence, see Chapter 17, "Managing Clusters."
You can choose to quiesce an application or an application server instance. These two options are described in the following sections:
Quiescing an Instance
The load balancer uses the following policy for quiescing application server instances:
- If an instance is disabled, and the time out has not expired, then sticky requests will continue to be delivered to that instance. New requests, however, will not be sent to the disabled instance.
- When the time-out expires, the instance is disabled. The load balancer does not send any requests to this instance, even if all sessions sticking to this instance were not invalidated. All open connections from the load balancer to the instance are closed. Instead, the load balancer will fail over sticky requests to another healthy instance in the same cluster.
Follow these steps to quiesce an instance:
- In the loadbalancer.xml file, set the enabled sub-element under the instance element to false.
- Set an appropriate value for the disable-timeout-in-minutes sub-element.
The disable-timeout-in-minutes sub-element represents the elapsed time after which the load balancer will disable the specified instance. The default is 31 minutes.
Quiescing an Application
The load balancer uses the following policy for quiescing applications:
- If an application is disabled, and the time out has not expired, new requests to the disabled applications will not be forwarded by the load balancer. These requests will be returned to the web server. Sticky requests will continue to be forwarded till the time-out expires.
- When the time out expires, the application is disabled. The load balancer will not accept any requests for this application, including sticky requests.
Follow these steps to quiesce an application:
- In the loadbalancer.xml file, set the enabled sub-element under the web-module element to false.
- Set an appropriate value for the disable-timeout-in-minutes sub-element.
The disable-timeout-in-minutes sub-element represents the elapsed time after which the load balancer will disable the specified application. The default is 31 minutes.
About Dynamic ReconfigurationThe load balancer plug-in detects changes to its configuration by examining the time stamp of the loadbalancer.xml file. If a change has been made to the loadbalancer.xml file, then the load balancer automatically reconfigures itself.
The time interval after which the load balancer checks the time stamp of the loadbalancer.xml file can be specified in the reload-poll-interval-in-seconds property in the loadbalancer.xml file.By default, this value is set to 0 seconds, and therefore, dynamic reconfiguration is not enabled.
Monitoring the Load Balancer Plug-inThis section discusses the following topics:
Configuring Log Messages
The load balancer plug-in uses the web server logging mechanism to write log messages. The load balancer plug-in uses the default logging level on Sun ONE Web Server (INFO), and Apache Web Server (WARN). The application server log levels - FINE, FINER, and FINEST map to the DEBUG level on the web server.
These log messages are written to the web server log files, and are in the form of raw data that can be parsed using scripts, or imported into spreadsheets to calculate the metrics you need.
Configuring Monitoring
Perform the following steps to turn on load balancer plug-in log messages:
- From Sun ONE Web Server’s admin console, go to the Magnus Editor tab.
- Set the Log Verbose option to On.
For Apache Web Server, set the log level to DEBUG.
- Set the value of the require-monitor-data in loadbalancer.xml property to true.
For example: <property name="require-monitor-data" value="true" />
The load balancer plug-in logs record the following information:
- Request start/stop information is logged for every request.
- Failed over request information is logged when the request fails over from an unhealthy instance to a healthy instance.
- List of unhealthy instances is logged at the end of every health check cycle.
Understanding Monitoring Messages
The format of the load balancer plug-in log messages is as follows.
Load Balancer Configuration FileThe loadbalancer.xml file contains the configuration information for load balancing Sun Open Net Environment (ONE) Application Server instances. This file is part of the load balancer plugin for the front-end web server. Although located in the web server, it is described in this manual because of its importance and relevance to the Sun ONE Application Server.
The encoding of the loadbalancer.xml file is UTF-8 to maintain compatibility with regular UNIX text editors. A schema file, sun-loadbalancer_1_0.dtd, determines the format and content of the loadbalancer.xml file.
This chapterdescribes loadbalancer.xml and sun-loadbalancer_1_0.dtd files in these sections:
The sun-loadbalancer_1_0.dtd File
The sun-loadbalancer_1_0.dtd file defines the structure of the loadbalancer.xml file, including the elements it can contain and the subelements and attributes these elements can have.
Note
Do not edit the sun-loadbalancer_1_0.dtd file; its contents change only with new versions of Sun ONE Application Server Enterprise Edition.
For general information about DTD files and XML, see the XML specification at:
Each element defined in a DTD file (which may be present in the corresponding XML file) can contain the following:
Subelements
Elements can contain subelements. For example, the following file fragment defines the cluster element.
<!ELEMENT cluster (instance*, web-module*, health-checker?)>
The ELEMENT tag specifies that a cluster element can contain instance, web-module, and health-checker elements in that order.
The following table shows how optional suffix characters of subelements determine the requirement rules, or number of allowed occurrences, for the subelements. The left column lists the subelement ending character, and the right column lists the corresponding requirement rule.
If an element cannot contain other elements, you see EMPTY or (#PCDATA) instead of a list of element names in parentheses.
Data
Some elements contain character data instead of subelements. These elements have definitions of the following format:
<!ELEMENT element-name (#PCDATA)>
For example:
<!ELEMENT description (#PCDATA)>
In the loadbalancer.xml file, white space is treated as part of the data in a data element. Therefore, there should be no extra white space before or after the data delimited by a data element. For example:
<description>response timeout property</description>
Attributes
Elements that have ATTLIST tags contain attributes (name-value pairs). For example:
<!ATTLIST instance name CDATA #REQUIRED
enabled %boolean; "true"
disable-timeout-in-minutes CDATA "31"
listeners CDATA #REQUIRED>An instance element can contain name, enabled, disable-timeout-in-minutes, and listeners attributes.
The #REQUIRED label means that a value must be supplied. The #IMPLIED label means that the attribute is optional, and that Sun ONE Application Server generates a default value. Wherever possible, explicit defaults for optional attributes (such as "true") are listed.
Attribute declarations specify the type of the attribute. For example, CDATA means character data, and %boolean is a predefined enumeration.
Elements in the loadbalancer.xml File
This section describes the XML elements in the loadbalancer.xml file.
Note
Subelements must be defined in the order in which they are listed under each Subelements heading unless otherwise noted.
loadbalancer
Defines a load balancer. This is the root element; there can only be one loadbalancer element in a loadbalancer.xml file.
Subelements
The following table describes subelements for the loadbalancer element. The left column lists the subelement name, the middle column indicates the requirement rule, and the right column describes what the element does.
Table 16-2 loadbalancer subelements
Element
Required
Description
zero or more
Defines a cluster of server instances.
zero or more
Specifies a load balancer property.
Attributes
none
property
Specifies a load balancer property.
Subelements
The following table describes subelements for the property element. The left column lists the subelement name, the middle column indicates the requirement rule, and the right column describes what the element does.
Table 16-3 property subelements
Element
Required
Description
zero or one
Contains a text description of this element.
Attributes
The following table describes attributes for the property element. The left column lists the attribute name, the middle column indicates the default value, and the right column describes what the attribute does.
Table 16-4 property attributes
Attribute
Default
Description
name
none
Specifies the name of the property.
value
none
Specifies the value of the property.
Properties
The following table describes load balancer properties. The left column lists the property name, the middle column indicates the default value if the property is not present, and the right column describes what the property does.
description
Contains a text description of the parent element.
Subelements
none
Attributes
none
cluster
Defines a cluster of server instances.
Subelements
The following table describes subelements for the cluster element. The left column lists the subelement name, the middle column indicates the requirement rule, and the right column describes what the element does.
Table 16-6 cluster subelements
Element
Required
Description
zero or more
Defines a server instance.
zero or more
Defines a web module.
zero or one
Configures the cluster’s health checker.
Attributes
The following table describes attributes for the cluster element. The left column lists the attribute name, the middle column indicates the default value, and the right column describes what the attribute does.
Table 16-7 cluster attributes
Attribute
Default
Description
name
none
Specifies the name of the cluster. Within a load balancer, cluster names must be unique.
instance
Defines a server instance.
Subelements
none
Attributes
The following table describes attributes for the instance element. The left column lists the attribute name, the middle column indicates the default value, and the right column describes what the attribute does.
web-module
Defines a web module.
Subelements
none
Attributes
The following table describes attributes for the web-module element. The left column lists the attribute name, the middle column indicates the default value, and the right column describes what the attribute does.
health-checker
Configures the cluster’s health checker.
Subelements
none
Attributes
The following table describes attributes for the health-checker element. The left column lists the attribute name, the middle column indicates the default value, and the right column describes what the attribute does.
Known Issues in Load Balancing RequestsThe following list discusses the limitations in Load Balancer with respect to HTTP/HTTPS request processing.
- If a session uses a combination of HTTP and HTTPS requests, then the first request has to be a HTTP Request. If the first request is an HTTPS request, it cannot be followed by an HTTP request. This is because the cookie associated with the HTTPS session is not returned by the browser. The browser interprets the two different protocols as two different servers, and initiates a new session.