18 Converged Application Server Debugging and Tuning

This chapter describes how to debug and tune Oracle Communications Converged Application Server.

Debugging Issues in the Runtime Environment

At times, issues can arise in your runtime environment such as when a call session fails or a server fails because the server configuration did not load correctly. You can resolve the issues in the runtime environment with the help of the debug attributes that Converged Application Server supports.

About the Runtime Debug Process

When you encounter an issue in the runtime environment, review the debug attributes that Converged Application Server supports in the Administration Console. To further diagnose an issue, select one or more of the relevant debug attributes that would help to reproduce the scenario.

You can isolate the debug process by enabling the selected debug attributes in one server only. Or you can attempt to view the behavior by enabling the selected attributes in all servers.

After enabling the relevant debug attributes in the Administration Console, rerun the scenario.

By default, Converged Application Server prints the debug log information to standard output stream, stdout. To parse the debug information, pipe the stdout data to a file. When the issue is resolved, be sure to disable the debug flag settings in the Administration Console. You can redirect the Java virtual machine (JVM) output to a log file. For more information, see the description about "Redirecting JVM Output" in Oracle Fusion Middleware Configuring Log Files and Filtering Log Messages for Oracle WebLogic Server.

If you pipe stdout content to a file, manage the rotated log files in your production or development environment. For more information about rotating log files, see "Configuring Log File Rotation" in Converged Application Server Developer's Guide.

WARNING:

Debug flags can result in gigabytes of log output in heavy traffic cases. Make sure that you use targeted debug flags as described in Table 18-1.

Revert the setting for the debug attributes as soon as the issue you are tracing is reproduced.

About the Debug Attributes Configuration Method

Use only the Administration Console to enable or disable debug attributes in Converged Application Server.

Converged Application Server provides the serverdebug.xml file for your reference. This file is located in the domain_home/config/custom directory, where domain_home represents the directory in which Converged Application Server domain is created.

Important:

  • Do not modify the serverdebug.xml file manually.

  • Do not enter a debug setting as a startup command option.

Recommended Debug Log Settings

Table 18-1 lists the various scenarios that you may encounter and the recommended debug attributes to select and enable in the Administration Console for information on debugging each issue.

Table 18-1 Converged Application Server Debug Attributes

Debug Attribute Issues for which this Attribute Provides Information

wlss.Admin

Details about Sip Server startup and the loading of its modules

wlss.AppRouter

Application router issues

wlss.CallState

Call state issues

wlss.coherenceStore

Information related to Oracle Coherence

wlss.concurrent

Concurrent service issues

wlss.Deployment

Application deployment issues

wlss.Diameter

Diameter protocol handling within the container issues

wlss.Dns

Issues related to Domain Naming Service

wlss.Filters

Filter module that filters SIP message issues

wlss.Geo

Geo redundancy handling issues

wlss.Headers

Sip (message) header issues

wlss.History

Issues related to call state history

wlss.instrumentation

Issues related to diagnostics, SIP or Diameter message properties

wlss.MHDebug

Message handler issues

wlss.Security

Security-related issues

wlss.SipEngine

Issues related to SIP servers

wlss.SipEngineConfig

Issues related to the loading of a server configuration

wlss.SipRequest

Issues related to SIP requests

wlss.SipSession

Issues related to SIP sessions

wlss.Status

Performance and garbage collection issues

wlss.Status.Timer

Timer statistics related to collections performance

wlss.Store

Call state and other cache issues

wlss.Timer

Internal timer issues

wlss.Traffic

Issues related to Sip traffic information

wlss.Transaction

Issues related to client and server transactions created within the container to process the call flow

wlss.Transport

Transport-level information for UDP, TCP, or TLS protocols.

wlss.Wrapping

Issues related to Enterprise JavaBeans (EJB) business object wrapping

Issues that Require the Enabling of Multiple Debug Attributes

Several issues require the enabling of multiple flags. The most common issues are described here.

SIP Specific Issues Involving Calls

In general for any SIP specific issue involving calls, enable the following debug flags on the engine servers and retrieve the server and stdout logs:

  • wlss.Transaction

  • wlss.SipSession

Transport-level Issues

In general for transport-level issues, enable the following debug flags on the engine servers and retrieve the server and stdout logs:

  • wlss.Transport

  • wlss.SipSession

Server Does not Process SIP Messages

For SIP specific issues, a Wireshark trace indicates that a SIP message reached the server but the message was not processed. Enable the following debug flags on the engine servers and retrieve the server and stdout logs:

  • wlss.Admin

  • wlss.MHDebug

  • wlss.Transaction

  • wlss.SipSession

Locking and Timer-related Issues

For locking and timer-related issues, enable following debug flags in combination or one at a time:

  • wlss.CallState

  • wlss.SipEngine

  • wlss.Timer

Message Validation Issues

For message handling details, enable following debug flags in combination or one at a time:

  • wlss.SipRequest

  • wlss.Headers

Enabling the Runtime Debug Attributes

To debug and resolve issues at run time, you can enable the appropriate debug attributes for one or all servers through the Converged Application Server Administration Console.

To set the debug attributes through the Administration Console:

  1. Use your browser to access the URL:

    http://address:port/console

    where address is the Administration Server's listen address and port is the listen port.

    Note:

    The default Administration Console port for Converged Application Server is 7001.

  2. Select the Sip Server node in the left pane.

    The right pane of the console provides two levels of tabbed pages that are used for configuring and monitoring Converged Application Server.

  3. Select the Server Debug tab under Configuration.

  4. In the List of Servers in the Domain table, click on the name of the server.

    The Server debug attributes table appears.

  5. Select the check box next to the debug attribute that you want to enable or disable. For information about the attributes, see "Recommended Debug Log Settings".

  6. From the command buttons above or below the table, do one of the following:

    • To enable the selected debug attributes in the selected server, click Enable.

    • To disable the selected debug attributes in the selected server, click Disable.

    • To enable the selected debug attributes in all servers, click Enable in all Servers.

    • To disable the selected debug attributes in all servers, click Disable in all Servers.

For more information, see the description about "Define debug settings" in Oracle Fusion Middleware Oracle WebLogic Server Administration Console Online Help.

Server Performance Tuning Recommendations

The following recommendations can help improve Converged Application Server performance.

Manage SIP Application Session Timeout

To prevent a caller or callee from being logged out during a session, make sure that the SIP application session timeout is conservatively set based upon the upper bound of a call duration. To set the session timeout, add the following entry to the sip.xml, where n determines the timeout in minutes:

<session-config>
  <javaee:session-timeout>n</javaee:session-timeout>
</session-config>

The session timeout is set to 3 minutes by default. As a rule of thumb, if the maximum call duration is, for example, 3 minutes, a rational setting for the session timeout would be 5 to 7 minutes.

Max Application Session Timeout

While the SIP application session timeout is dictated by the session-timeout element in sip.xml as described above, the maximum application session lifetime is configured at the container level by the max-application-session-lifetime element in sipserver.xml. The max-application-session-lifetime element essentially limits the maximum value a deployer can set for the session-timeout element in sip.xml. Configuring this element helps resource management by ensuring that a deployer is forced to set a timeout value within specific boundaries. If a value is not specified for max-application-session-lifetime, a deployer can set any value in sip.xml.

Specifying the Minimum Thread Pool Size

Depending upon the number of concurrent users for your environment, you should adjust the value of the server parameter SelfTuningThreadPoolMinSize.

The self-tuning thread pools start with a default size, which grows and shrinks automatically as required. The default size for the administration server is 15, and 100 for engines in each cluster. You can increase the number of threads to improve throughput, but the minimum will never fall below the default. However, an excessive number of threads increases memory use, and could cause garbage collection related performance issues. Oracle recommends 100 as a working maximum. If you have a low number of concurrent users, you can use a lower value.

To configure SelfTuningThreadPoolMinSize:

  1. Use your browser to access the URL http://address:port/console where address is the Administration Server's listen address and port is the listen port.

    Note:

    The default Administration Console port for Converged Application Server is 7001.

  2. If your domain is running in Production mode, click Lock & Edit.

  3. Expand the Environment node in the left pane and then, select Servers.

    The Summary of Servers page appears.

  4. Select the Configuration tab. Then, select the server name from the Servers table.

    The Settings for Server_name page appears.

  5. Select the Tuning subtab and expand the Advanced pane at the bottom.

  6. Enter a value for Self Tuning Thread Minimum Pool Size.

  7. Click Save to save your configuration changes.

  8. If your domain is running in Production mode, click Activate Changes.

  9. Restart the server.

Files for Troubleshooting

The following Converged Application Server log and configuration files are frequently helpful for troubleshooting problems. Your technical support contact generally requests the following files from you:

  • domain_home/config/custom/coherence.xml

  • domain_home/config/coherence/Coherence-Default/Coherence-Default.xml

  • domain_home/config/coherence/Coherence-Default/Custom-Default.xml

  • domain_home/config/config.xml

  • domain_home/config/custom/sipserver.xml

  • domain_home/server_name/*.log (Server and message logs)

  • Located in the /WEB-INF subdirectory of the application

  • Located in the /WEB-INF subdirectory of the application

By default, domain_home represents the directory in which Converged Application Server domain is created and server_name is the name of the server.

General information that can help the technical support team includes:

  • The specific versions of:

    • Converged Application Server

    • Java SDK

    • Operating System

  • Thread dumps for hung Converged Application Server processes

  • Network analyzer logs