Skip navigation.

Understanding and Responding to SNMP Traps

Overview

The following sections describe the WebLogic SIP Server SNMP traps in more detail. Recovery procedures for responding to individual traps are also included where applicable.

Files for Troubleshooting

The following WebLogic SIP Server log and configuration files are frequently helpful for troubleshooting problems, and may be required by your technical support contact:

In addition, the following WebLogic Server log and configuration files are helpful in diagnosing problems with WebLogic SIP Server:

General information that can help the technical support team includes:

Trap Descriptions

The following sections provide detailed information about the following WebLogic SIP Server SNMP traps:

OkiSipAsStop

Description: This trap is generated during normal shutdown procedures.  Because all traps are filtered during shutdown, this trap should never reach operations.  The trap is designated as Critical in the unlikely event that it does occur during normal operations, which would could indicate that shutdown was initiated incorrectly.  In this case, restart the server.

OkiSipAsFailure

Description: The SIP servlet container shut down either as a result of a normal WebLogic Server shutdown procedure, or because the SIP container encoutered a fatal problem.

Note: Because the SIP servlet container does not support graceful shutdown, this trap is generated any time the WebLogic Server shuts down.

Recovery Procedure:

If this trap was generated as part of a normal WebLogic SIP Server shutdown procedure, no action is necessary. If the trap is generated but the WebLogic Server instance is still running, follow this recovery procedure:

  1. Use the following command to identify the hung process:

    ps –ef | grep java

    There should be only one PID for each WebLogic Server instance running on the machine.

  2. After identifing the affected PID, use the following command to kill the process:

    kill -3 [pid]

  3. This command generates the actual thread dump. If the process is not immediately killed, repeat the command several times, spaced 5-10 seconds apart, to help diagnose potential deadlock problems, until the process is killed.
  4. Attempt to restart WebLogic SIP Server immediately. See Restarting Failed Server Instances in the WebLogic Server 8.1 documentation.
  5. Make a backup copy of all SIP logs on the affected server to aid in troubleshooting. The location of the logs varies based on the server configuration. The default location is the /logs directory of the WebLogic SIP Server domain directory. Specific logs to backup include:
  6. Copy each log to assist Tier 4 support with troubleshooting the problem.

    Note: WebLogic SIP Server logs are truncated according to your system configuration. Make backup logs immediately to avoid losing critical troubleshooting information.

  7. Notify Tier 4 support and include the log files with the trouble ticket.
  8. Monitor the server closely over next 24 hours. If the source of the problem cannot be identified in the log files, there may be a hardware or network issue will reappear over time.

OkiSipAsStatusChanged

Description: WebLogic SIP Server configuration files should be changed only during regulary-scheduled maintenance period (for example, when the server instance is shut down). This trap indicates that changes were made to configuration files during normal operations, and may indicate that someone has changed a configuration property or has accidentally changed, moved, or overwritten a configuration file. 

Recovery Procedure:

Notify Tier 4 support.

OkiSipAsLicenseViolation

Description: WebLogic SIP Server generates this trap when it detects a license violation. This trap can occur if server usage or access exceeds the limitations specified in the license file, or if the file is accidentally modified. Never modify, move, or delete the license file during normal operations.

License violations may cause one or more of the following behaviors if the license file has been modified or corrupted:

If usage reaches the maximum values set in the license file (max-sessions, max-registers or max-users) WebLogic SIP Server continues to run but rejects requests that exceed the defined limits. The following behaviors may be observed when usage limits are reached:

  1. If the number of sessions has reached max-sessions and there is a request to create a new session:
  2. If there is an attempt to register a user beyond the maximum number specified in max-users:
  1. If the number of connected terminals has reached max-registers:

The license file is an XML document located at $BEA_HOME/sip-license.xml. The following sample shows the contents of an evaluation license:

<?xml version="1.0" encoding="UTF-8" ?>
 
<license-group format="1.0"
               product="WebLogic SIP Server"
               edition="Design Partner Release">

               release="2.0.1.0"
  <license component="SIP Servlet Engine"
           licensee="BEA Evaluation Customer"
           type="EVAL"
           signature="WTDWfnJeDLYnFgggVtols2AHcBs=">
    <ip-address>any</ip-address>
    <expiration>2005-08-30</expiration>
    <max-sessions>any</max-sessions>
  </license>
  <license component="Subscribers Manager"
           licensee="BEA Evaluation Customer"
           type="EVAL"
           signature="WTDWfnJeELYnFgggVtols2AHcBs">
    <max-users>any</max-users>
  </license>
  <license component="Registrar Servlet"
           licensee="BEA Evaluation Customer"
           type="EVAL"
           signature="WTDWfnJeKCYnFgggVtols2AHcBs">
    <max-registers>any</max-registers>
  </license>
</license-group>

Recovery Procedure:

  1. Check the license file to insure that it has not been accidentally removed, changed or corrupted. Note that WebLogic SIP Server checks the license every four hours, so repairs may not be registered immediately.
  2. Check the expiration date in the license, and confirm that an EVAL license was not accidentally installed over the permanent license.
  3. Notify Tier 4 Support of the condition, and send them a copy of the license file.

Additional License FAQs

Question: What IP should be used for licensing purposes?

Question: Each box has multiple IP addresses. Which IP should be assigned to the license?

Answer: Use the IP address that is returned with get local host command.

 

Question: I've upgraded my hardware system or need to move WebLogic SIP Server to a new machine. How do I modify the license file to use a new IP address?

Answer: Contact BEA Support with the updated IP address. BEA will generate a new license for you. You can then replace the license file with the updated file immediately without stopping WebLogic SIP Server.

OkiSipAsRegulateThreshExceeded

Description: Weblogic SIP Server uses a configurable throttling mechanism that helps you control the number of new SIP requests that are processed. After a configured overload condition is observed, WebLogic SIP Server destroys new SIP requests by responding with “503 Service Unavailable” to the caller. The server continues to destroy new requests until the overload condition is resolved according to a configured threshold control value. This alarm is generated when the throttling mechanism is activated. The throttling behavior should eventually return the server to a non-overloaded state, and further action may be unnecessary.

Recovery Procedure:

  1. Check other servers to see if they are nearly overloaded.
  2. Check to see if the load balancer is correctly balancing load across the application servers, or if it is overloading one or more servers. If additional servers are nearly overloaded, Notify Tier 4 support immediately. 
  3. If the issue is limited to one server, notify Tier 4 support within one hour.

Additional Overload FAQs

Question: How can I monitor load using the Administration Server? How can I tell when I'm near a threshold?

Answer: If you set the queue length as an incoming call overload control, you can monitor the length of the queue using the Administration Console. If you specify a session rate control, you cannot monitor the session rate using the Administration Console. (The Administration Console only displays the current number of SIP sessions, not the rate of new sessions generated.)

OkiSipAppShutdown

Description: WebLogic SIP Server generates this alarm when a SIP application shuts down, or if a SIP application is undeployed. This generally occurs when WebLogic SIP Server is shutdown while active requests still exist, in which case it is accompanied by an OkiSipAsStop alarm.

Recovery Procedure:

During normal shutdown procedures this alarm should be filtered out and should not reach operations. If the alarm occurs during the course of normal operations, it indicates that someone has shutdown the application or server unexpectedly, or there is a problem with the application. Notify Tier 4 support immediately.

OkiSipAppFailure

Description: WebLogic SIP Server generates this trap when an application deploys successfully as a Web Application but fails to deploy as a SIP application.

Recovery Procedure:

The typical failure is caused by an invalid sip.xml configuration file and should occur only during software installation or upgrade procedures.  When it occurs, undeploy the application, validate the sip.xml file, and retry the deployment.

Note: This alarm should never occur during normal operations.  If it does, contact Tier 4 support immediately.

OkiSipAppThreshExceeded

Description: This trap indicates that the number of application sessions was exceeded. The application session regulation function is used to throttle session creation for each application. When the number of application sessions exceeds the specified maximum value, the WebLogic SIP Server rejects the creation of subsequent application sessions with a “503 Application Busy” response.

Note: The trap is triggered once every five minutes that the system is in excess of the threshhold. There is no clearing trap and the trap is no longer generated after the system is operating within the threshhold value.

To enable this function, place the extension configuration file sse.xml in the WEB-INF subdirectory of each application. If the file does not exist, the function is not enabled and the trap is never generated. The following example shows a throttle configuration for a maximum of 100 application sessions:

 

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE sse-app SYSTEM "file:/sse_1_0.dtd">
<sse-app>
    <session-config>
        <max-app-session-count>100</max-app-session-count>
    </session-config>
</sse-app>

Recovery Procedure:

  1. Check other servers and applications to determine if they, too, are nearly overloaded. An application overload condition may be caused by unexpected peak traffic.
  2. Check to see if the load balancer is correctly balancing load across the application servers. If one or more of the servers is nearly overload because of poor load balancing, Notify Tier 4 support immediately. 
  3. If the issue is limited to one server, notify Tier 4 support within one hour.

Additional Overload FAQs

Question: Is there a correlation between application sessions and SIP requests?

Answer: SIP application sessions may include SIP sessions, and SIP sessions may include SIP requests. SIP requests have the same Call-ID header of the SIP protocol in one SIP session.

 

Question: Can this trap be ignored or is action required?

Answer: BEA recommends setting a threshold value for applications based on the application priority. This prevents all resources from being consumed by a single SIP application’s burst traffic.

ServerShutDown

Description: This trap indicates that the WebLogic Server instance is now down. If this trap is received spontaneously and not as a result of a controlled shutdown, follow the steps below. This trap is not always generated with WebLogic SIP server traps.

Recovery Procedure:

  1. Use the following command to identify the hung process:

    ps –ef | grep java

    There should be only one PID for each WebLogic Server instance running on the machine.

  2. After identifing the affected PID, use the following command to kill the process:

    kill -3 [pid]

  3. This command generates the actual thread dump. If the process is not immediately killed, repeat the command several times, spaced 5-10 seconds apart, to help diagnose potential deadlock problems, until the process is killed.
  4. Attempt to restart WebLogic SIP Server immediately. See Restarting Failed Server Instances in the WebLogic Server 8.1 documentation.
  5. Make a backup copy of all SIP logs on the affected server to aid in troubleshooting. The location of the logs varies based on the server configuration. The default location is the /logs directory of the WebLogic SIP Server domain directory. Specific logs to backup include:
  6. Copy each log to assist Tier 4 support with troubleshooting the problem.

    Note: WebLogic SIP Server logs are truncated according to your system configuration. Make backup logs immediately to avoid losing critical troubleshooting information.

  7. Notify Tier 4 support and include the log files with the trouble ticket.
  8. Monitor the server closely over next 24 hours. If the source of the problem cannot be identified in the log files, there may be a hardware or network issue will reappear over time.

Additional Shutdown FAQs

Question: If the server shuts down, are all SNMP traps for the server lost?

Response: The Administration Console generates SNMP messages for managed WebLogic Server instances only until the ServerShutDown message is received. Afterwards, no additional messages are generated.