1. Maintenance and Troubleshooting Guide
  2. Performance Management
  3. Monitoring Your System in Real-Time

Monitoring Your System in Real-Time

This section explains how to monitor your system in real-time by using the monitor media and monitor sessions commands.

  • monitor media: real-time media statistics
  • monitor sessions: real-time SIP statistics

    Note:

    The ACLI statistics displays use standard VT100 escape sequences to format the display. Therefore, your terminal emulator or terminal itself must support VT100.

Displaying the Statistics

The following information explains how to work with the statistics display.

Changing the Refresh Rate

At any point, you can press any numerical digit (0-9) to change the number of seconds for the refresh rate (the rate at which the display is updated). By default, the statistics refresh every second. For example, while viewing the statistics, you can press 6 to cause the system statistics to refresh every 6 seconds. While viewing the statistics via the ACLI, you can press any key to automatically refresh the statistics upon key press.

Quitting the Display

Pressing <q> or <Q> allows you to exit the statistics display and returns you to the ACLI system prompt (for example, ORACLE#). From that point, you can continue with any other task you choose.

Viewing Real-Time Media Statistics

Display real-time media statistics for your running system by using the monitor media command. 

acmepacket# monitor media
17:31:00-160
MBCD Status                -- Period -- -------- Lifetime --------
                 Active    High   Total      Total  PerMax    High
Client Sessions     143     182    1930    1218332    4225     683
Client Trans          0      18    5744    2500196    8439     625
Contexts            144     182    1930     834745    2783    2001
Flows               296     372    3860    1669498    5566    3689
Flow-Port           286     362    3860    1669488    5566    3679
Flow-NAT            294     365    3788    1658668    5563    2051
Flow-RTCP             0       0       0          0       0       0
Flow-Hairpin          0       0       0          0       0       0
Flow-Released         0       0       0          0       0       0
MSM-Release           0       0       0          0       0       0
NAT Entries         295     365    3791    1658671    5563    2051
Free Ports         7430    7518    7828    3346410   11604    8002
Used Ports          572     724    7724    3338980   11132    8000
Port Sorts            -       -       0      14796    4156
MBC Trans          1141    1234    5748    2503147    8440    2974
MBC Ignored           -       -       0          0       0

ARP Trans 0 0 0 8 8 1

Real-time statistics for the following categories appear on the screen:

  • Client Sessions
  • Client Trans
  • Contexts
  • Flows
  • Flow-Port
  • Flow-NAT
  • Flow-RTCP
  • Flow-Hairpin
  • Flow-Release
  • MSM-Release
  • NAT Entries
  • Free Ports
  • Used Ports
  • Port Sorts
  • MBC Trans
  • MBC Ignored
  • ARP Trans

By default, the statistics refresh every second. Press any numerical digit (0-9) to change the refresh rate. For example while viewing the statistics, you can press 6 to cause the system statistics to refresh every 6 seconds.

Pressing q or Q allows you to exit the statistics display and returns you to the ACLI system prompt.

Viewing Real-Time SIP Session Statistics

If you have Superuser access, display real-time monitoring of your running system for sessions. This table displays information similar to that which is displayed for the show sipd command, except that the information in the monitor sessions table is real-time and updates automatically. 

ORACLE# show sipd
14:16:43-149
SIP Status                -- Period -- -------- Lifetime --------
                Active    High   Total      Total  PerMax    High
Sessions             0       0       0          0       0       0
Subscriptions        0       0       0          0       0       0
Dialogs              0       0       0          0       0       0
CallID Map           0       0       0          0       0       0
Rejections           -       -       0          0       0
ReINVITEs            -       -       0          0       0
Media Sessions       0       0       0          0       0       0
Media Pending        0       0       0          0       0       0
Client Trans         0       0       0          0       0       0
Server Trans         0       0       0          0       0       0
Resp Contexts        0       0       0          0       0       0
Saved Contexts       0       0       0          0       0       0
Sockets              0       0       0          0       0       0
Req Dropped          -       -       0          0       0
DNS Trans            0       0       0          0       0       0
DNS Sockets          0       0       0          0       0       0
DNS Results          0       0       0          0       0       0
Session Rate = 0.0
Load Rate = 0.0

Real-time statistics for the following categories appear on the screen:

  • Dialogs
  • Sessions
  • CallID Map
  • Rejections
  • ReINVITES
  • Media Sessions
  • Media Pending
  • Client Trans
  • Server Trans
  • Resp Contexts
  • Sockets
  • Reqs Dropped
  • DNS Trans
  • DNS Sockets
  • DNS Results

By default, the statistics refresh every second. Press any numerical digit (0-9) to change the refresh rate. For example, while viewing the statistics, you can press 6 to cause the system statistics to refresh every 6 seconds.

Pressing q or Q allows you to exit the statistics display and returns you to the ACLI system prompt.

Thread Level Load Monitoring and Alarms

The Oracle Communications Session Border Controller provides a thread-level monitoring for CPU usage, specifically including three critical traffic processes: SIP, ATCP and MBCD.

Several mechanisms are available for monitoring CPU usage on a per-thread basis: ACLI commands, alarms, HDR, traps and MIBs. The thread usage table MIB object is found in ap-usbcsys.mib. It supports the output of process and thread utilization information. HDR information is produced as a comma separated value file whose data can be displayed in a formatted fashion via command line. The system sends SNMP traps when any of the SIP, ATCP worker threads or MBCD tasks exceed configured thresholds. Users can construct a Threshold Crossing Alarm (TCA) which issues minor, major and critical system alarms when the thread usage level exceeds pre-configured values. These Thread Overload Alarms follow the example in the Configurable Alarm Thresholds and Traps section.

ACLI

The following commands display thread-level load statistics:
  • show processes: add sipd, atpcd and overload arguments
  • show queues atcpd
  • show queues sipd

Alarms

The user-configurable alarm may be created to notify the user of any problematic usage of CPU resources by specific processes at a thread-level basis.

To create alarms, the user sets the alarm-threshold, type with the following values to create the corresponding threshold crossing alarm.

  • cpu-sipd
  • cpu-atcp
  • cpu-mbcd

When a thread alarm is active due to crossing a pre-configured threshold, the system sends the apUsbcSysThreadUsageExceededTrap trap.

When the thread's load falls below the threshold that triggered the alarm, the system sends the apUsbcSysThreadUsageClearTrap trap.

The user can get this information using the display-alarms command.

Historic Data Recording (HDR)

There are two HDR groups available to record Thread Level Load Monitoring information:

  • thread-event: reports pending and dropped events per protocol as well as calculating latency
  • thread-usage: reports CPU thread usage per protocol and an overload condition

The data captured by these two HDR groups corresponds to the show queues atcpd and show queues sipd ACLI command output.

SNMP MIBs and Traps

Thread Level Load Monitoring information can be retrieved via SNMP by from MIB objects in the ap-usbcsys.mib. In addition, traps are available to send off-system notifications.

There are two types of traps that provide process-level thread threshold indicators.

  • Traps managed by the process-level thread alarm configurations, which use the alarm configuration as triggers. These traps include:
    • apUsbcSysThreadUsageExceededTrap
    • apUsbcSysThreadUsageClearTrap
  • Traps managed by Symmetric Multi-processing (SMP)-aware task load limiting function's configurations, which use the use the SMP Transport, SIP and Media limiting configurations as triggers. These traps include:
    • apUsbcSysThreadUsageOverloadEnableTrap
    • apUsbcSysThreadUsageOverloadDisableTrap

Information in overload enable/disable traps include the threshold type, the overload alarm exceeded or cleared as well as the overload method activated or de-activated.

The system follows the SMP-Aware Task Load Limiting rules described in the Oracle® Communications Session Border Controller Troubleshooting and Maintenance Guide to determine the action(s) it takes when crossing these thresholds.

The system manages these traps by applying a function similar to CPU overloading limit to smooth the output and avoid issuing too many traps. Each trap contains the thread name. The usage exceeded trap also contains current thread usage.

For SIP, ACTP and MBCD, traps display the threshold type, the overload alarm exceeded or cleared, or the overload method activated or de-activated.