|
|
Using IP multicasting, WebLogic Server instances in a cluster can share a single IP address and port number. This capability enables all members of a cluster to be treated as a single entity and enables members of the cluster to communicate among themselves.
The following sections provide suggestions for troubleshooting IP multicast configuration problems:
For general information on using and configuring multicast within a cluster, see Cluster Configuration and config.xml.
For information on configuring multicast address from the console, see Multicast in the Administration Console Online Help.
The first step in troubleshooting multicast problems is to verify that you have configured the multicast address and port correctly. A multicast address must be correctly configured for each cluster.
Multicast address and port configuration problems are among the most common reasons why a cluster does not start or a server fails to join a cluster. The following considerations apply to multicast addresses:
The following types of errors commonly occur due to multicast configuration problems:
To check the multicast address and port:
<cluster> element in config.xml.
After you verify that the multicast address and port are configured correctly, determine whether network problems are interfering with multicast communication.
Ensure that no physical problems exist in your network.
Address conflicts within a network can disrupt multicast communications.
On UNIX systems, you may encounter the UnkownHostExceptions error. This error can occur at random times even when the server is not under a heavy load. Check /etc/nsswitch.conf and change the order to ‘files,DNS,NIS’ to avoid this error.
For more information, see the nsswitch.conf man page for your system.
After you verify that the multicast address and port are configured correctly and there are no physical or configuration problems with your network, you can use utils.MulticastTest to verify that multicast is working and to determine if unwanted traffic is occurring between different clusters.
If MulticastTest fails and the machine is multihomed, ensure that the primary address is being used. See Multicast and Multihomed Machines.
The following sections describe how to tune various features of WebLogic Server to work with multicasting.
Multicast timeouts can occur during a Network Interface Card (NIC) failover. Timeouts can result in an error message like the following:
<Error><Cluster><Multicast socket receive error:
java.io.InterruptedIOException: Receive timed out>
When this error occurs, you can:
igmp snooping switch. This switch is part of the Internet Group Management Protocol (IGMP) and is used to prevent multicast flood problems on the managed switch.MulticastTTL=32
For more information, see Configure Multicast Time-To-Live (TTL).
Each WebLogic Server instance in a cluster uses multicast to broadcast regular heartbeat messages that advertise its availability. By monitoring heartbeat messages, server instances in a cluster determine when a server instance has failed.
The following sections describe possible solutions when cluster heartbeat problems occur.
Multicast Send Delay specifies the amount of time the server waits to send message fragments through multicast. This delay helps to avoid OS-level buffer overflow. This can be set via the MulticastSendDelay attribute of the Cluster MBean. For more information, see the Cluster Mbean Javadocs.
If problems still occur after setting the Multicast Send Delay, you may need to set the following operating system parameters related to UDP settings:
If these parameters are set to a lower value (8K for example) there may be a problem if the multicast packet size is set to the maximum allowed (32K). Try setting these parameters to 64K.
A multicast storm is the repeated transmission of multicast packets on a network. Multicast storms can stress the network and attached stations, potentially causing end-stations to hang or fail.
Increasing the size of the multicast buffers can improve the rate at which announcements are transmitted and received, and prevent multicast storms. See Configure Multicast Buffer Size.
The following considerations apply when using multicast in a multihomed environment:
UnixMachine instance from the WebLogic Server console and have specified an InterfaceAddress for each Server instance to handle multicast traffic./usr/sbin/ifconfig -a to check the MAC address of each machine in the multihomed environment. Ensure that each machine has a unique MAC address. If machines use the same MAC address, this can cause multicast problems.If multicast problems occur when cluster members are in different subnets you should configure Multicast-Time-To-Live. The value of the Multicast Time-To-Live (TTL) parameter for the cluster must be high enough to ensure that routers do not discard multicast packets before they reach their final destination.
The Multicast TTL parameter sets the number of network hops a multicast message makes before the packet can be discarded. Configuring the Multicast TTL parameter appropriately reduces the risk of losing the multicast messages that are transmitted among server instances in the cluster.
For more information, see Configure Multicast Time-To-Live (TTL).
If you are still having problems with the multicast address after performing the troubleshooting tips above, gather debugging information for multicast.
The following utilities can help you debug multicast configuration problems.
MulticastMontior is a stand-alone Java command line utility that monitors multicast traffic on a specific multicast address and port. The syntax for this command is:
java weblogic.cluster.MulticastMonitor <multicast_address> <multicast_port> <domain_name> <cluster_name>
The MulticastTest utility helps you debug multicast problems when you configure a WebLogic cluster. The utility sends out multicast packets and returns information about how effectively multicast is working on your network.
For more information on MulticastTest utility, see “Using the WebLogic Server Java Utilities” in WebLogic Server Command Reference.
The following debug flags are specific to multicast:
Set these flags from the command line during server startup by adding the following options:
Set debug flags using the weblogic.Admin utility. For example:
java weblogic.Admin -url t3://localhost:7001 - username weblogic -password weblogic SET -type ServerDebug -property DebugCluster true
Turn off debug flags by running the same command and setting the parameter to false.
The following sections describe miscellaneous multicast issues you may encounter.
AIX version 5.1 does not support IPv4 mapped multicast addresses. If you are using an IPv4 multicast address, you cannot join a multicast group even if you are switching to IPv6. When running MulticastTest on AIX, use the order on the command line specified in the following example:
java -Djava.net.preferIPv4Stack=true utils.Multicast <options>
Additionally, verify the following settings on AIX to properly configure cluster operations:
Depending on the operating system, there may be problems with the number of file descriptors open. On UNIX, you can use loses to determine how many files on disk a process has open. If a problem occurs, you may need to increase the number of file descriptors on the machine.
For more information on troubleshooting cluster problems, see Troubleshooting Common Problems.
The following resources may be helpful in resolving multicast problems:
|