Operations Guide

     Previous  Next    Open TOC in new window    View as PDF - New Window  Get Adobe Reader - New Window
Content starts here

Configuring SNMP

The following sections describe how to configure and manage SNMP services with WebLogic SIP Server:

 


Overview of WebLogic SIP Server SNMP

WebLogic SIP Server includes a dedicated SNMP MIB to monitor activity on engine tier and data tier server instances. The WebLogic SIP Server MIB is available on both Managed Servers and the Administration Server of a domain. However, WebLogic SIP Server engine and data tier traps are generated only by the Managed Server instances that make up each tier. If your Administration Server is not a target for the sipserver custom resource, it will generate only WebLogic Server SNMP traps (for example, when a server in a cluster fails). Administrators should monitor both WebLogic Server and WebLogic SIP Server traps to evaluate the behavior of the entire domain.

Note: WebLogic SIP Server MIB objects are read-only. You cannot modify a WebLogic SIP Server configuration using SNMP.

 


Browsing the MIB

Use a Web browser to view the WebLogic SIP Server SNMP MIB Reference on the BEA e-docs Web site. Because the MIB Reference uses Javascript and DHTML to provide browsing capabilities that are similar to a MIB browser, you must use one of the following Web browsers:

 


Configuring SNMP

To enable SNMP monitoring for the entire WebLogic SIP Server domain, follow these steps:

  1. Login to the Administration Console for the WebLogic SIP Server domain.
  2. In the left pane, select the Services->SNMP node.
  3. Select the Enabled check box to enable SNMP.
  4. Note: WebLogic SIP Server instances ignore the SNMP port number specified on this page. See SNMP Port Binding for WebLogic SIP Server.
  5. Click Apply to apply your changes.

 


SNMP Port Binding for WebLogic SIP Server

If you run multiple Managed Server instances on the same machine, each server instance would normally attempt to bind to the same configured SNMP port (for example, port 161). WebLogic SIP Server instances automatically manage SNMP port conflicts by automatically attempting to bind to port 1610, and incrementing the port number as needed if the current port is unavailable. This helps to avoid a SNMP startup failure when multiple WebLogic SIP Server instances are deployed on the same server hardware.

You can also manually override the starting port number that WebLogic SIP Server attempts to bind to by supplying the -DWLSS.SNMPAgentPort=port_number startup argument.

WARNING: If you specify the -DWLSS.SNMPAgentPort option, ensure that the starting port number and subsequent numbers are unused on your system. The default starting port of 1610 was selected because no services are commonly bound to the 1610 port range.

 


Understanding and Responding to SNMP Traps

The following sections describe the WebLogic SIP Server SNMP traps in more detail. Recovery procedures for responding to individual traps are also included where applicable.

Files for Troubleshooting

The following WebLogic SIP Server log and configuration files are frequently helpful for troubleshooting problems, and may be required by your technical support contact:

General information that can help the technical support team includes:

Trap Descriptions

Table 2-1 lists the WebLogic SIP Server SNMP traps and indicates whether the trap is generated by servers in the engine tier or data tier. Each trap is described in the sections that follow.

Table 2-1 WebLogic SIP Server SNMP Traps
Server Node in which Trap is Generated
Trap Name
Engine Tier Servers
Engine and Data Tier Servers, if servers are members of a cluster
Data Tier Servers

connectionLostToPeer

Description

This trap is generated by an engine tier server instance when it loses its connection to a replica in the data tier. It may indicate a network connection problem between the engine and data tiers, or may be generated with additional traps if a data tier server fails.

Recovery Procedure

If this trap occurs in isolation from other traps indicating a server failure, it generally indicates a network failure. Verify or repair the network connection between the affected engine tier server and the data tier server.

If the trap is accompanied by additional traps indicating a data tier server failure (for example, dataTierServerStopped), follow the recovery procedures for the associated traps.

connectionReestablishedToPeer

Description

This trap is generated by an engine tier server instance when it successfully reconnects to a data tier server after a prior failure (after a connectionLostToPeer trap was generated). Repeated instances of this trap may indicate an intermittent network failure between the engine and data tiers.

Recovery Procedure

See connectionLostToPeer.

dataTierServerStopped

Description

WebLogic SIP Server data tier nodes generate this alarm when an unrecoverable error occurs in a WebLogic Server instance that is part of the data tier. Note that this trap may be generated by the server that is shutting down, by another replica in the same partition, or in some cases by both servers (network outages can sometimes trigger both servers to generate the same trap).

Recovery Procedure

See the Recovery Procedure for serverStopped.

overloadControlActivated, overloadControlDeactivated

Description

Weblogic SIP Server engine tier nodes use a configurable throttling mechanism that helps you control the number of new SIP requests that are processed. After a configured overload condition is observed, WebLogic SIP Server destroys new SIP requests by responding with “503 Service Unavailable” to the caller. The server continues to destroy new requests until the overload condition is resolved according to a configured threshold control value. This alarm is generated when the throttling mechanism is activated. The throttling behavior should eventually return the server to a non-overloaded state, and further action may be unnecessary. See Overload in the Configuration Reference Manual.

Recovery Procedure

1. Check other servers to see if they are nearly overloaded.

2. Check to see if the load balancer is correctly balancing load across the application servers, or if it is overloading one or more servers. If additional servers are nearly overloaded, Notify Tier 4 support immediately.

3. If the issue is limited to one server, notify Tier 4 support within one hour.

Additional Overload FAQs

Question: How can I monitor load using the Administration Server? How can I tell when I'm near a threshold?

Answer: If you set the queue length as an incoming call overload control, you can monitor the length of the queue using the Administration Console. If you specify a session rate control, you cannot monitor the session rate using the Administration Console. (The Administration Console only displays the current number of SIP sessions, not the rate of new sessions generated.)

replicaAddedToPartition

Description

WebLogic SIP Server data tier nodes generate this alarm when a server instance is added to a partition in the data tier.

Recovery Procedure

This trap is generated during normal startup procedures when data tier servers are booted.

replicaRemovedFromPartition

Description

WebLogic SIP Server data tier nodes generate this alarm when a server is removed from the data tier, either as a result of a normal shutdown operation or because of a failure. There must be at least one replica remaining in a partition to generate this trap; if a partition has only a single replica and that replica fails, the trap cannot be generated. In addition, because engine tier nodes determine when a replica has failed, an engine tier node must be running in order for this trap to be generated.

Recovery Procedure

If this trap is generated as a result of a server instance failure, additional traps will be generated to indicate the exception. See the recovery procedures for traps generated in addition to replicaRemovedFromPartition.

serverStopped

Description

This trap indicates that the WebLogic Server instance is now down. This trap applies to both engine tier and data tier server instances, but only when the servers are members of a named WebLogic Server cluster. If this trap is received spontaneously and not as a result of a controlled shutdown, follow the steps below.

Recovery Procedure
  1. Use the following command to identify the hung process:
  2. ps – ef | grep java

    There should be only one PID for each WebLogic Server instance running on the machine.

  3. After identifying the affected PID, use the following command to kill the process:
  4. kill -3 [pid]
  5. This command generates the actual thread dump. If the process is not immediately killed, repeat the command several times, spaced 5-10 seconds apart, to help diagnose potential deadlock problems, until the process is killed.
  6. Attempt to restart WebLogic SIP Server immediately. See Restarting a Failed Managed Server in the WebLogic Server 9.2 documentation.
  7. Make a backup copy of all SIP logs on the affected server to aid in troubleshooting. The location of the logs varies based on the server configuration.
  8. Copy each log to assist Tier 4 support with troubleshooting the problem.
  9. Note: WebLogic SIP Server logs are truncated according to your system configuration. Make backup logs immediately to avoid losing critical troubleshooting information.
  10. Notify Tier 4 support and include the log files with the trouble ticket.
  11. Monitor the server closely over next 24 hours. If the source of the problem cannot be identified in the log files, there may be a hardware or network issue that will reappear over time.
Additional Shutdown FAQs

Question: If the server shuts down, are all SNMP traps for the server lost?

Answer: The Administration Console generates SNMP messages for managed WebLogic Server instances only until the ServerShutDown message is received. Afterwards, no additional messages are generated.

sipAppDeployed

Description

WebLogic SIP Server engine tier nodes generate this alarm when a SIP Servlet is deployed to the container.

Recovery Procedure

This trap is generated during normal deployment operations and does not indicate an exception.

sipAppUndeployed

Description

WebLogic SIP Server engine tier nodes generate this alarm when a SIP application shuts down, or if a SIP application is undeployed. This generally occurs when WebLogic SIP Server is shutdown while active requests still exist.

Recovery Procedure

During normal shutdown procedures this alarm should be filtered out and should not reach operations. If the alarm occurs during the course of normal operations, it indicates that someone has shutdown the application or server unexpectedly, or there is a problem with the application. Notify Tier 4 support immediately.

sipAppFailedToDeploy

Description

WebLogic SIP Server engine tier nodes generate this trap when an application deploys successfully as a Web Application but fails to deploy as a SIP application.

Recovery Procedure

The typical failure is caused by an invalid sip.xml configuration file and should occur only during software installation or upgrade procedures. When it occurs, undeploy the application, validate the sip.xml file, and retry the deployment.

Note: This alarm should never occur during normal operations. If it does, contact Tier 4 support immediately.

  Back to Top       Previous  Next