BEA Logo BEA WebLogic Server Release 1.1

  Corporate Info  |  News  |  Solutions  |  Products  |  Partners  |  Services  |  Events  |  Download  |  How To Buy

   Using WebLogic Server Clusters:   Previous topic   |   Next topic   |   Contents   |  Index

 

Administering WebLogic Clusters

 

Overview

This document describes how to configure a WebLogic Cluster, and how to start a WebLogic Server so that it joins a cluster.

Requirements

There are a number of constraints placed on the networking environment and on the cluster configuration, as described below.

Network requirements

The machine(s) you will be using as WebLogic hosts for the cluster must have permanently assigned, static IP addresses. You cannot use dynamically-assigned IP addresses in a clustering environment.

The WebLogic Servers in a cluster must all be located on the same local area network (LAN) and must be reachable via IP multicast.

Licensing

To run a WebLogic Cluster, you need a special cluster license. The standard download of WebLogic Server does not include the clustering feature. Contact your sales representative to obtain a cluster license.

weblogic.properties files

A WebLogic Server cluster uses a hierarchy of three weblogic.properties files to set configuration parameters: a global properties file for all WebLogic Servers, a cluster-wide properties file for all servers in the cluster, and a per-server properties file for individual server instances. All WebLogic Servers that participate in a cluster must have access to the same global and cluster-wide properties files. You can ensure that all servers access these files in one of two ways:

Using a shared filesystem for cluster configuration introduces a single point of contention for the system. All clustered servers must compete to access the filesystem (and possibly to write individual log files). Moreover, should the shared filesystem fail, you may be unable to start clustered servers.

To avoid having a single point of failure, we recommend against using a shared filesystem to store global and cluster-wide weblogic.properties files. Instead, create a master copy of each weblogic.properties file, and copy that file into the installation directory of each clustered server. If you need to make changes to either file, simply propagate those changes (by recopying the master files) into each installation directory. This method provides a simple and efficient way to maintain cluster-wide configurations, and does not introduce a single point of failure to the system.

If you do use a shared file system for your cluster, ensure that the file system is stored on fault tolerant hardware. There are a variety of approaches to fault tolerance for disks:

Off-the-shelf file servers can provide fault tolerance out-of-the-box, with operating systems such as Solaris and Windows NT providing the software support. Consult with your hardware supplier for further information on installing and running such a system.

Server Configuration

All WebLogic Servers in a cluster must be the same version. Configure the servers in your cluster depending on the particular mix of services that you are offering:

In practice, this often results in WebLogic Servers in the cluster being identically configured in all areas to do with WebLogic services, class files, and external resources (such as databases).

Configuration is the most complicated part of setting up a cluster. This document contains step-by-step instructions to help you set up your file system and the various properties files.

Configuring a WebLogic Server Cluster

In general, before you start a WebLogic Server that can join a cluster, you need to carry out the following steps to set up the environment:

  1. Acquire a set of addresses and configure DNS. You need an IP address for each WebLogic Server and a multicast address for the cluster. Note that you cannot set up a cluster on a single machine with one IP address by running multiple servers listening on different ports; all servers must be listening on different IP addresses at the same port.

  2. Set up weblogic.properties files for the cluster

  3. Locate classes and register classes in the properties files

  4. Specify deployment and startup class properties in the properties files

  5. Set the CLASSPATH for each WebLogic host

In this section, we cover each of these steps in greater detail.

Setting up the Network

  1. Acquire the following addresses:

  2. Configure DNS.

    For information on DNS, there are many excellent online resources. A search on any of the popular Internet search engines will provide many useful links. One of the most highly regarded works on DNS is DNS and BIND, published by O'Reilly & Associates, Inc.

    You should configure DNS so that a single host name directs incoming requests to a cluster of servers, rather than directing incoming requests to a single server. Cluster-wide load balancing and failover both depend upon DNS round-robining. You should configure:

Setting up the weblogic.properties files

Each WebLogic Server installation in a given cluster should have the directories and weblogic.properties files described below. The global and cluster-wide weblogic.properties files should be exactly the same for each server in the cluster.

If you are using a shared filesystem, you need only create one directory hierarchy and one copy of each weblogic.properties file for servers to access.

To setup weblogic.properties files:

  1. Create weblogic/. If you are using a shared filesystem, create a common WebLogic home directory and name it weblogic/.

    Note that you can name the directory whatever you want, but if the directory is not named weblogic/ you will not be able to take advantage of all the startup defaults. In this document, we refer to this directory exclusively as weblogic/.

  2. Install the WebLogic distribution in the weblogic/. Your file system will now appear something like this:

    /weblogic/
    /weblogic/classes/
    /weblogic/lib/
    /weblogic/myserver/

  3. Create a directory named mycluster/ in the weblogic/ directory. If you have used WebLogic before, you can think of the mycluster/ directory as analogous to the myserver/ directory, except it supports a cluster of WebLogic Servers rather than a single server.

    Here is an example of this directory structure that shows the classes/, the standard myserver/ directory, and the lib/ directories that were installed with the distribution, as well as the mycluster/ directory that you have created:

    /weblogic/
    /weblogic/classes/
    /weblogic/lib/
    /weblogic/mycluster/
    /weblogic/myserver/

    Note that the name "mycluster" is a standard default. You can name the mycluster/ directory something else and change your configuration parameters accordingly, but here we use the default for simplicity. We suggest that you follow this naming convention during your initial configuration until you are familiar with how the configuration details relate to each other.

  4. Copy the public_html/ directory and all of its contents from the weblogic/myserver/ directory into the weblogic/mycluster directory. Your file system should now appear as:

    /weblogic/
    /weblogic/classes/
    /weblogic/lib/
    /weblogic/mycluster/
    /weblogic/mycluster/public_html/
    /weblogic/myserver/

  5. Copy any administrative files (like security certificates) that all cluster members will share into the mycluster/ directory. Here is an example:

    /weblogic/
    /weblogic/classes/
    /weblogic/lib/
    /weblogic/mycluster/
    /weblogic/mycluster/ca.der
    /weblogic/mycluster/ca.pem
    /weblogic/mycluster/democert.pem
    /weblogic/mycluster/demokey.pem
    /weblogic/mycluster/public_html/
    /weblogic/myserver/

Note that as you start servers that join this cluster, WebLogic will look in the mycluster/ directory for a per-server directory for each server that joins the cluster. If the per-server directory does not exist, one will be created in the mycluster/ directory. The default naming scheme for the individual servers is myserver/<last three numbers of the IP address>. The per-server directory contains per-server administrative files like the HTTP access logs, the weblogic.log, and the JDBC log, and it may contain a per-server properties file.

Here is an example of one cluster configuration used for testing at WebLogic consisting of 5 Solaris machines, most of which are multihomed, and all of which have DNS names set up.

The computers used in the cluster include:

The common file system is resident on node sunglass, and nfs-mounted to the other machines in the cluster. WebLogic Server has been installed onto this shared file system.

Netscape Enterprise Server with the WebLogic plug-in is installed on node Mellen as the Web Server.

If you are not using a shared filesystem, make sure that each WebLogic Server uses the directory hierarchy described in the above steps.

You have completed the first step of cluster setup. The next step is configuration, which is done with a set of weblogic.properties files.

weblogic.properties file contents

With clustering, another level of control is added to the properties hierarchy. There are now three levels of properties files: global, cluster, and per-server. The global properties file is read first; its properties apply to all WebLogic Servers. The cluster properties is read next, and its properties apply to servers in a particular cluster. The per-server properties file is read last; its properties apply only to the individual server instance that is starting.

What happens when you start a server that will join a cluster? If no other server in the cluster is running, the server you start becomes the first member of the cluster.

The server goes through the following steps:

  1. Open and read the global properties file, located in the WebLogic home directory (installed by default as weblogic/).

  2. Open and read the cluster properties file (installed by default as weblogic/mycluster). Any properties set in the cluster properties file will override the same properties that were set from the global properties file.

  3. Look for a subdirectory whose name matches the name you assign to the Server when you start it, by setting the weblogic.system.name property on the command line. The server name installed by default is weblogic/<clusterdir>/<server><last three digits of the IP address>.

    If you specify a server name on the command line and its directory is not found, the server will quit with an appropriate error message.

    If the directory is found, the server will look for a per-server properties file and will read that. Any properties set in the per-server properties file will override the same properties that were set from the cluster or global properties files.

  4. Load all of its startup classes and deployments.

  5. Start listening for login requests.

An identical property set in a properties file will override the setting for the same property in any properties file read earlier in the startup sequence. For example, properties in each per-server properties file will override identical properties already set in the global or cluster properties files. Finally, any property set on the command line will override all other settings for that property.

The properties files are nested:

To set up the properties file hierarchy, begin with the weblogic.properties file that was installed from the distribution in the weblogic/ directory.

Here is a typical configuration, with the locations shown of the various properties files:

Default and global properties

 /weblogic/weblogic.properties

Cluster properties

 /weblogic/<mycluster>/weblogic.properties

All paths that you configure in the cluster properties file, such as servlet classpath, document root, etc., should point to the correct directory on the shared file system.

Per-server properties

 /weblogic/<mycluster>/server154/weblogic.properties

Note that if a per-server properties file exists in any of the server directories beneath the cluster directory (mycluster/), it should not contain any properties that exist in the global or cluster properties files, except those properties that you want to override settings established in the global or cluster properties files. Make sure you delete such properties from the per-server properties file, since the presence of a property in the per-server property overrides the identical property in global and cluster properties files.

Setting CLASSPATH

For instructions on setting your CLASSPATH, see the Administrators Guide, Setting classpath.

Setting up HTTP servlets

WebLogic Server supports clustering servlets across a WebLogic Cluster. This section describes the issues involved and how to set up a servlet cluster.

Initial HTTP servlet requests are passed to an arbitrary server node in a cluster. When a servlet request is part of an ongoing session, subsequent requests must have access to the HTTP session data. You set up your WebLogic Server Cluster to accomplish this using either in-memory state replication or JDBC session persistence or file persistence.

Using In-Memory State Replication with a Proxy Server

This feature provides better performance, adequate failover, and does not use a persistent store. It (unlike JDBC session persistence) does require a WebLogic Server Cluster license.

When using in-memory state replication, your WebLogic Server Cluster must live behind one or more proxy servers. The proxy servers are smart enough to send servlet requests, belonging to the same HTTP session, back to the same server in the cluster that holds the session data. This eliminates the need to write the session data to a shared database. You may use Apache, Netscape Server, MS-IIS or WebLogic Server as a proxy server.

Failover is provided by replicating each session on a secondary node in the cluster. If the primary node fails, the secondary node takes over, becoming the new primary node for the session data. Another node is selected as the secondary node, and a backup of the session is made.

Setting up the Proxy Server

For details on setting up Netscape, Microsoft-IIS, or Apache as proxy server, see:

This section shows you how to set up a WebLogic Server as a proxy server for clusters using HttpClusterServlet.

Requests intended for the WebLogic Cluster (for example JSP pages and servlets) must be handled by the HttpClusterServlet on the proxy server. The HttpClusterServlet knows how to proxy the request to the appropriate server in the WebLogic Cluster. Set up your DNS to point to the proxy server, not directly to a server in the WebLogic Cluster.

There are two ways to configure the proxy server: you can set the default servlet as the file servlet, or the cluster servlet.

In the first option, the proxy server acts as the file server for all static content, and only registered dynamic content is handled by the cluster. This configuration is clear and direct, because only JSP files and registered servlets are passed to the cluster servlet, while HTML pages, images, and other static content are delivered by the proxy server's file servlet.

The following properties in the proxy server's weblogic.properties file configure the FileServlet as the default servlet.

  weblogic.allow.execute.weblogic.servlet=everyone
weblogic.httpd.register.file=\
weblogic.servlet.FileServlet
weblogic.http.defaultServlet=file

Requests that are intended for the WebLogic Cluster can be registered with the cluster servlet as shown below:


weblogic.httpd.register.*.jsp=\
weblogic.servlet.internal.HttpClusterServlet
weblogic.httpd.initArgs.*.jsp=\
defaultServers=server1:9001|server2:9001
weblogic.httpd.register.*.servlet=\
weblogic.servlet.internal.HttpClusterServlet
weblogic.httpd.initArgs.*.servlet=\
defaultServers=server1:9001|server2:9001
weblogic.httpd.register.myfavoriteservlet=\
weblogic.servlet.internal.HttpClusterServlet
weblogic.httpd.initArgs.myfavoriteservlet=\
defaultServers=server1:9001|server2:9001

The initialization parameter defaultServers lists the addresses of servers in the WebLogic Cluster.

In the second option, the proxy server's default is the cluster servlet and static requests can be handled by the file servlet. The main reason for choosing this configuration over the first option would be a large number of dynamic requests with different file extensions. Setting the cluster servlet as the default servlet in the proxy eliminates the need to individually register servlets, which may have arbitrary file extensions. Instead, static content should be registered.

The following properties set HttpClusterServlet as the default servlet.


weblogic.allow.execute.weblogic.servlet=everyone
weblogic.httpd.register.cluster=\
weblogic.servlet.internal.HttpClusterServlet
weblogic.httpd.initArgs.cluster=\
defaultServers=server1:9001|server2:9001
weblogic.httpd.defaultServlet=cluster

Requests that are intended for the WebLogic FileServlet can be registered as shown below for the .html, .jpg, and .gif MIME types. Other MIME file extensions can be added in the same manner.


weblogic.httpd.register.*.html=\
weblogic.servlet.FileServlet
weblogic.httpd.register.*.jpg=\
weblogic.servlet.FileServlet
weblogic.httpd.register.*.gif=\
weblogic.servlet.FileServlet

We recommend that you remove all other servlet registrations from the proxy server's weblogic.properties file, except the chosen default servlet (FileServlet in option 1, and ClusterServlet in option 2). That is, remove all other properties that begin with

  weblogic.httpd.register ...

This will avoid confusion where the proxy server may attempt to directly respond to a request if it matches a registration, or where the proxy servlet may fail due to an error in the cluster.

Setting up the WebLogic Cluster

In addition to setting up the proxy server, you need to set up a WebLogic Cluster as described below, including the weblogic.properties files. In addition to setting the normal properties for a clustered server and an HTTP servlet, you must set the following properties in the cluster-wide <my_cluster>weblogic.properties file to enable clustered HTTP sessions using in-memory replication:

  weblogic.httpd.clustering.enable=true
weblogic.httpd.session.persistence=true
weblogic.httpd.session.persistentStoreType=replicated

Using JDBC Session Persistence

Note: JDBC session persistence (unlike in-memory state replication) does not require a WebLogic Cluster license.

Each cluster node accesses servlet session data via a shared JDBC persistent store. For details on setting up persistent storage for servlet sessions, see the Developers Guide Using WebLogic HTTP Servlets: Making sessions persistent. WebLogic supports servlet clustering with EJB entity bean-managed persistence so the frequency of physical writes to disk can be carefully controlled.

Each time you add or change data in the HttpSession the change is written immediately to the persistent store database, allowing any other servers access to the updated data. This method provides good failover, but each change to the session involves a write to the database, which can be a substantial performance reduction compared to using no persistence. We recommend instead that you use in-memory state replication.

Note that persistence is only for failover; previous session data is available only if you use a database or an entity EJB to make it persistent beyond the session.

Follow these steps to set up servlet clustering using JDBC session persistence:

  1. As described earlier, set up the DNS hosts file for your LAN so that there is a single IP entry that points to all the server hosts that make up the cluster.

  2. Set up separate physical machines, each installed with the same version of WebLogic Server and the same directory structure. Note that you cannot run different versions of WebLogic in any kind of interdependent environment.

  3. Set up the same connection pools for each WebLogic Server to support JDBC-based persistence.

  4. Make sure the servlet-related properties (for sessions, connection pools, and ACLs) are set to the same values in all of the properties files.

  5. Install your servlets on each machine, each in the proper directory structure.

  6. Start the WebLogic Servers.

  7. Your clients should request servlets using the cluster IP address rather than an IP address of an individual host.

Setting Up Enterprise JavaBeans

In addition to the information on Enterprise JavaBeans provided in Deploying EJBs Using DeployerTool, and in the General Requirements section of this chapter, to use the EJB hot deployment feature in a cluster you must:

  1. Undeploy the jar unit on all machines in the cluster

  2. Deploy the new jar unit on all machines in the cluster

This keeps the version of the bean present in the cluster the same at all times. There is a window where bean operations can fail and beans are unavailable, causing transactions to roll back or bean homes not to be found. You should write your clients to retry the EJB in the event it can not be found to cover this case, as described in our Tech Support Guide Troubleshooting your WebLogic EJB application. For more information on hot deploy, see Using WebLogic Server hot deploy.

Starting the WebLogic Server and Joining a Cluster

You should have completed the items listed in General Requirements before you start a server that will join a cluster.

Starting with Defaults

For general information on starting a WebLogic Server, see the Administrators Guide Setting up and starting the WebLogic Server. We assume that you have read this and have started a WebLogic Server in a single server mode and are familiar with the use of the supplied scripts.

In the examples below on starting the Server, we have left out much of the command, and only shown the commands that are relevant to the cluster.

You can start a cluster using defaults and a couple of command-line options. The defaults assume that the host is not multihomed. The simplest way to start a WebLogic server that will join a cluster is to set the property weblogic.cluster.enable to true on the command line, as shown here:

 $ java -Dweblogic.cluster.enable=true
... weblogic.Server

Using this property alone on the command line will invoke a server with the following default settings:

Cluster name defaults to "mycluster" (and matches the name of the directory that contains your properties file and security files) IP address for this WebLogic Server defaults to its assigned IP address Listen port defaults to "7001" Server name defaults to the pattern "server" plus the last three digits of the IP address, as in "server154" Multicast address defaults to 237.0.0.1

Details on Starting WebLogic in a Clustered Environment

This section provides more details on the properties you can set for starting the Server in a Cluster, on the command line, in the weblogic.properties file, the cluster-wide properties file and the per-server properties file.

Other standard startup options are detailed in the Administrators Guide document, Setting up and starting the WebLogic Server, which details options for single-server configuration.

Command Line Properties

When you start each server that will join the cluster, you must set certain properties on the command line - with the -D option - unless you want to use the defaults. Here is a complete list of command line properties that affect server configuration, their defaults, and dependencies. Cluster-specific properties that can be specified in the cluster-wide properties and the per-server properties file are included.

weblogic.cluster.name=clustername, where clustername is the name of the cluster this WebLogic Server should join on startup, and matches the name of the "mycluster/" directory. If you do not supply this command-line property, the cluster will start with the "mycluster" name, and will look in the weblogic/mycluster/ directory for its cluster property file, security-related files, and its document root. The cluster name property must be the same for every server in the cluster and must be set only on the command line.

For example:

 $ java -Dweblogic.cluster.name=megacluster
... weblogic.Server

weblogic.system.name, to specify the server name. If you do not use the standard pattern for establishing the server name ("server" + last 3 digits of the IP address), you must set the server name on the command line. Of course, the server name must be unique for each server in the cluster.

For example:

 $ java -Dweblogic.system.name=bizserver
... weblogic.Server

weblogic.cluster.enable should be set to true. This property must always be supplied on the command line.

weblogic.system.listenPort. This property must be the same for every server in a cluster. If you do not specify a listen port, the default used is "7001". If you specify a non-default listen port for one server, you must specify the same port for every other server in the cluster. (Note that you cannot set up a cluster by running multiple servers on the same IP address listening at different ports; all servers must be listening on different IP addresses at the same port.) The listen port can be set on the command line or in the global or cluster-wide properties files.

weblogic.system.bindAddr, if the machine is multihomed and you want to specify the IP address for this server process. If you do not specify an IP address when you start WebLogic on a multihomed machine, WebLogic will choose one of its host's IP addresses; the algorithm for determining which IP address is chosen is operating-system dependent. You should set the bindAddr property if the machine is multihomed to avoid unpredictable results. You can either set this property on the command line or - if you have specified the server name, as described above - in the appropriate per-server properties file.

For example:

 $ java -Dweblogic.system.bindAddr=192.168.0.55 (or DNS name)
... weblogic.Server

Here is a sample of properties to be set for the WebLogic Server "server154" to join "mycluster". The multicast address for this cluster is 237.5.6.7, and the machine is multihomed, with IP addresses 227.15.15.154 and 204.198.18.6, and the WebLogic Server would be running on the 227.15.15.154 address.

  weblogic.cluster.name=mycluster
weblogic.cluster.multicastAddress=237.0.0.5
weblogic.system.bindAddr=227.15.15.154
weblogic.system.listenPort=7001
weblogic.cluster.enable=true

Advanced Properties

In addition to the above properties, you can set these properties in the cluster-wide properties file:

weblogic.cluster.multicastAddress, to specify the multicast address that the servers in a cluster will use to communicate with each other. If you do not specify an address, WebLogic will attempt to use use the default multicast address "237.0.0.1". The multicast address must be the same for every server that joins the cluster. Consequently, it is best if you set it only in the cluster-wide properties file.

weblogic.cluster.multicastTTL, the time-to-live (number of hops) for the cluster's multicast messages. The default is 1, with a range of 1-255.

weblogic.cluster.defaultLoadAlgorithm, the default load balancing algorithm to be used between replicated services if none was specified for that particular service. The default is "round-robin", with acceptable values of "random", "round-robin", and "weight-based". Load balancing is explained in Writing and Deploying Clustered Objects.

Per-Server Properties

There are also cluster-related properties that apply only to a specific server; you set these properties in the per-server properties file:

weblogic.system.weight, the weight of the individual server for the weight-based load-balancing. The default is 100, with a range of 0-100. Larger numbers increase the amount of traffic routed to the server.

Stopping a Server in a Cluster

Use the Admin SHUTDOWN command to halt a WebLogic Server in a cluster, just as you would a WebLogic Server running in stand-alone mode. The other members of the Cluster will determine that the halted server is no longer available and the cluster-wide JNDI tree will be updated accordingly.

Special Considerations for Clients of a Cluster

Performance of your JAVA client can be degraded when the number of socket readers is lower than the number of WebLogic Servers in your cluster. You can improve the performance by setting the following properties from the command line while starting your client:

 $ java -Dweblogic.system.executeThreadCount=<number of threads>  

$ java -Dweblogic.system.percentSocketReaders=<percentage of threads to be used as socket readers>

Setting the number of threads and the percentage of socket readers to ensure an equal number of socket readers and instances of WebLogic Server will help to maintain high performance levels.