|
|
This chapter provides guidelines on how to prevent cluster problems or troubleshoot them if they do occur.
You can do a number of things to help prevent problems before you boot the cluster.
All servers in the cluster must have the same major version number, but can have different minor version numbers and service packs.
The cluster’s Administration Server is typically not configured as a cluster member, but it should run the same major version of WebLogic Server used on the managed servers.
A problem with the multicast address is one of the most common reasons a cluster does not start or a server fails to join a cluster.
A multicast address is required for each cluster. The multicast address can be an IP number between 224.0.0.0 and 239.255.255.255, or a host name with an IP address within that range.
You can check a cluster’s multicast address and port on its Configuration-->Multicast tab in the Administration Console.
For each cluster on a network, the combination of multicast address and port must be unique. If two clusters on a network use the same multicast address, they should use different ports. If the clusters use different multicast addresses, they can use the same port or accept the default port, 7001.
Before booting the cluster, make sure the cluster’s multicast address and port are correct and do not conflict with the multicast address and port of any other clusters on the network.
The errors you are most likely to see if the multicast address is bad are:
Unable to create a multicast socket for clustering
Multicast socket receive error
Make sure the value of CLASSPATH is the same on all managed servers in the cluster. CLASSPATH is set by the setEnv script, which you run before you run startManagedWebLogic to start the managed servers.
By default, setEnv sets this value for CLASSPATH (as represented on Windows systems):
set WL_HOME=C:\bea\wlserver_10.00
set JAVA_HOME=C:\bea\jdk131
.
.set CLASSPATH=%JAVA_HOME%\lib\tools.jar;
%WL_HOME%\server\lib\weblogic_sp.jar;
%WL_HOME%\server\lib\weblogic.jar;
%CLASSPATH%
If you change the value of CLASSPATH on one managed server, or change how setEnv sets CLASSPATH, you must change it on all managed servers in the cluster.
Each server instance in the cluster has a default execute queue, configured with a fixed number of execute threads. To view the thread count for the default execute queue, choose the Configure Execute Queue command on the Advanced Options portion of the Configuration> General tab for the server. The default thread count for the default queue is 15, and the minimum value is 5. If the value of Thread Count is below 5, change it to a higher value so that the Managed Server does not hang on startup.
If the cluster fails to start, or a server fails to join the cluster, the first step is to check any commands you have entered, such as startManagedWebLogic or a java interpreter command, for errors and misspellings.
Before contacting BEA Technical Support for help with cluster-related problems, collect diagnostic information. The most important information is a log file with multiple thread dumps from a Managed Server. The log file is especially important for diagnosing cluster freezes and deadlocks.
Remember: a log file that contains multiple thread dumps is a prerequisite for diagnosing your problem.
% java -ms64m -mx64m-verbose:gc-classpath $CLASSPATH
-Dweblogic.domain=mydomain -Dweblogic.Name=clusterServer1
-Djava.security.policy==$WL_HOME/lib/weblogic.policy
-Dweblogic.admin.host=192.168.0.101:7001
weblogic.Server>> logfile.txt
Redirecting both standard error and standard output places thread dump information in the proper context with server informational and error messages and provides a more useful log.
kill -3 or <Ctrl>-<Break> to create the necessary thread dumps to diagnose your problem. Make sure to do this several times on each server, spaced about 5-10 seconds apart, to help diagnose deadlocks.| Note: | If you are running the JRockit JVM under Linux, see Getting a JRockit Thread Dump Under Linux. |
% tar czf logfile.tar logfile.txt
- or zip it using a Windows utility.
If you use the JRockit JVM under Linux, use one of the following methods to generate a thread dump.
weblogic.admin THREAD_DUMP command.Xmanagement option), you can generate a thread dump using the JRockit Management Console. Kill -3 PID, where PID is the root of the process tree. To obtain the root PID, perform a:
using a grep argument that is a string that will be found in the process stack that matches the server startup command. The first PID reported will be the root process, assuming that the ps command has not been piped to another routine.
Under Linux, each execute thread appears as a separate process under the Linux process stack. To use Kill -3 on Linux you supply must match PID of the main WebLogic execute thread, otherwise no thread dump will be produced.
If you are experiencing cluster problems, you should also check the garbage collection on the managed servers. If garbage collection is taking too long, the servers will not be able to make the frequent heartbeat signals that tell the other cluster members they are running and available.
If garbage collection (either first or second generation) is taking 10 or more seconds, you need to tune heap allocation (the msmx parameter) on your system.
You can verify that multicast is working by running utils.MulticastTest from one of the managed servers. See
“Using the WebLogic Server Java Utilities” in WebLogic Server Command Reference.
If you have problems with running multicast, see Debugging Multicast.
For more information on troubleshooting multicast within a cluster, see Troubleshooting Multicast Configuration.
|