Application Performance Monitoring Metrics

You can monitor the health, capacity, and performance of your applications by using metrics, alarms, and notifications.

Overview

Application Performance Monitoring data sources such as APM Java agents and APM Browser agents collect and upload key application metrics to the Oracle Cloud Infrastructure Monitoring service. For information on Application Performance Monitoring data sources, see Application Performance Monitoring Data Sources.

Here are the Monitoring service metric namespaces related to Application Performance Monitoring:

  • oracle_apm_rum: Metrics related to Real User Monitoring (RUM), specifically session metrics for all configured web applications as well as metrics for all monitored user activities (page activities, AJAX calls, and script errors).
  • oracle_apm_synthetics: Metrics related to Synthetic Monitoring, specifically the monitor's availability, the number of connections established, the number of DNS lookups, and so on.
  • oracle_apm_monitoring: All other Application Performance Monitoring metrics, such as the metrics stemming from application servers and custom metrics.

For information on the Monitoring service and its features, see Monitoring Overview.

Prerequisites

IAM policies: To monitor resources, you must be given the required type of access in a policy written by an administrator, whether you're using the Console or the REST API with an SDK, CLI, or other tool. The policy must give you access to the monitoring services as well as the resources being monitored. If you try to perform an action and get a message that you don’t have permission or are unauthorized, confirm with your administrator the type of access you've been granted and which compartment you should work in. For more information on user authorizations for monitoring, see the Authentication and Authorization section for the related service: Monitoring or Notifications.

Available Metrics: oracle_apm_rum

The RUM metrics are available after an APM Browser agent is configured and uploads user experience-related data. You don't need to enable monitoring on the APM domain to get these metrics.

RUM metrics include the following dimensions:

  • Generic Dimensions
    • ApdexLevel: The level of user satisfaction. For example, Satisfied or Frustrated.
    • ApmrumType: The type of monitored activity. For example, Page, AJAX call or Script Error.
    • ApmrumPageUpdateType: The subtype of the ApmrumType dimension, which is related to user activity. For example, Page Load, Full Update or Click.
  • User Session-Based Dimensions
    • BrowserName: The name of the web browser derived from the user agent. For example, Chrome.
    • DeviceType: The type of device used by the user. For example, Personal Computer.
    • OsFamily: The Operating System family derived from the user agent. For example, Linux.
    • ResourceId: The OCID of the APM domain.
    • Type: The type of metric. For example, Gauge or Counter.
    • WebApplicationName: The name of the web application as specified in instrumentation. If no value is specified for this dimension, then "Default WebApp" is assigned.
Note

When building metric queries for the oracle_apm_rum namespace, you must include the general dimension, MetricGroupName = "No Group" to ensure correctness of data. Similarly, the WebApplicationName dimension with the value All Web Applications or the name of the required web application must be added. For information on adding dimensions, see Build Application Performance Monitoring Metric Queries.

The following table lists RUM metrics:

Metric Description Unit
ActiveSessionsPerMin

The number of active sessions per minute (in the context of web applications).

AjaxCalls

The number of AJAX calls observed.

AjaxDownloadTime

The time taken to download a response.

Milliseconds

AjaxErrors

The number of AJAX errors observed.

AjaxFirstByteTime

The time taken to receive the first byte of an AJAX call response after the request is sent to the server. This is mainly impacted by network latency and server response time.

Milliseconds

AjaxInitTime

The time taken to initialize an AJAX request inside the browser, from the creation of fetch/xhr till the request is sent.

Milliseconds

AjaxResponseTime

The time taken to process the AJAX call response, which is the sum of AjaxInitTime, AjaxFirstByteTime, and AjaxDownloadTime.

Milliseconds

ApdexScore

The Apdex value between 0 and 1.

ConnectCount

The number of connections established.

ConnectTime

The average time taken to establish server connections.

Milliseconds

DNSLookups

The number of DNS lookups.

DNSTime

The average time taken to perform DNS lookups.

Milliseconds

FrustratedPageViews

The number of page views with a 'frustrating' performance.

PageClicks

The number of clicks on a page.

PageDownloadTime

The time from the browser sending a request for the page URL till the full content is received.

Milliseconds

PageFirstByteTime

The time from the browser sending the request for the page URL till the first byte of response from the server is sent for the page URL. This is mainly impacted by server response time and network latency.

Milliseconds

PageInitTime

The time from the browser receiving the navigation trigger till the request for the page URL is started, which includes connection setup time. This time is calculated for Page Loads and Page Updates.

Milliseconds

PageInteractiveTime

The time taken for the page to become interactive.

Milliseconds

PageRenderTime

The time from the browser receiving full HTML till the content is rendered.

Milliseconds

PageResponseTime

The time taken from the beginning of navigation till the loading of script is complete. This is the sum of PageInitTime, PageFirstByteTime, PageDownloadTime, and PageRenderTime.

Milliseconds

PageViews

The number of page views or page updates.

RedirectTime

The average time spent in handling HTTP redirects.

Milliseconds

SatisfiedPageViews

The number of page views with a 'satisfying' performance.

ScriptErrors

The number of JavaScript errors observed.

SSLTime

The average time taken to establish secure server connections (as part of ConnectTime).

Milliseconds

ToleratingPageViews

The number of page views with a 'tolerable' performance.

Available Metrics: oracle_apm_synthetics

The Synthetic Monitoring metrics are emitted as soon as the monitor is created, although there may be a delay of a few minutes before the metrics are displayed initially. You don't need to enable monitoring on the APM domain to get these metrics.

Synthetic Monitoring metrics include the following dimensions:

  • CustomMarker: Indicates the custom target name using which an operation is added to a script.
  • ErrorCategory: The category of the error, if an error occurs when the script is executed.
  • Genre: Indicates that the metric is a Synthetic Monitoring metric. The default value is Synthetics.
  • Host: The host of the network timing metrics. The network timing metrics such as SSLTime and DNSTime are averaged by this host value.
  • IsAPMAgentMonitored: Indicates if the application is also being monitored by another Application Performance Monitoring agent.
  • IsHarAvailable: Indicates if the HAR .zip file is available.
  • IsLogAvailable: Indicates if the log .zip file is available.
  • IsNetworkDataAvailable: Indicates if the network .zip file is available.
  • IsRetryExecution: Indicates if monitor execution has been tried more than once.
  • IsScreenshotAvailable: Indicates if screenshots are available.
  • MonitorId: The OCID of the monitor created in Synthetic Monitoring.
  • MonitorName: The name assigned to the monitor.
  • MonitorType: The type of monitor.
  • RequestType: The type of request. The values are CSS, Image, JavaScript, AJAX, or Others.
  • ResourceId: The OCID of the APM domain in which the monitor is created.
  • Target: The base URL specified in the uploaded script for the Scripted Browser or Scripted REST monitor type or specified in the user interface when creating a Browser or REST monitor type.
  • UserAgent: The type of agent used. For the Browser and Scripted Browser monitor types, Chrome is the default value. For the REST and Scripted REST monitor types, the respective monitor type is the default value.
  • VantagePoint: The vantage point on which the monitor is running.
  • VantagePointDisplayName: The display name of the vantage point on which the monitor is running.

The following table lists the common Synthetic Monitoring metrics for all monitor types:

Metric Name Description Unit
Availability

The availability of the monitor.

1 for success, 0 for failure

AverageBytesPerRequest

The average number of bytes loaded per request for this request type.

AverageLoadTimePerRequest

The average time taken to load the resources of this request type.

Milliseconds

ConnectCount

The number of connections established.

ConnectTime

The average time taken to establish server connections.

Milliseconds

CustomMetric

Records the custom markers added to the scripts.

Milliseconds

DNSLookups

The number of DNS lookups.

DNSTime

The average time taken to perform DNS lookups.

Milliseconds

Failure

Indicates if the monitor execution failed.

0 for success, 1 for failure

HTTP4xxFailureCount

The total number of 4xx request failures during monitor execution.

HTTP5xxFailureCount

The total number of 5xx request failures during monitor execution.

Latency

The average of the round-trip packet time.

Milliseconds

LatencyDeviation

The standard deviation of latency.

Milliseconds

MonitorExecutionTime

The total time taken to execute the monitor.

Milliseconds

MonitorLoadTime

The total time taken to execute the monitor, excluding the pauses given in the script.

Milliseconds

PacketLossPercentage

The percentage of probe packets lost.

Percentage

RequestCount

The total number of requests for this request type.

SSLTime

The average time taken to establish secure server connections (as part of ConnectTime).

Milliseconds

Success

Indicates if the monitor execution was successful.

1 for success, 0 for failure

TotalRequestFailures

The total number of requests that failed to receive a response.

TotalSteps

The total number of clicks or enter keys that are captured during monitor execution.

The following table lists the Synthetic Monitoring metric for the REST monitor type:

Metric Name Description Unit
CertificateExpiry

The number of days left before the certificate expires.

Days

Available Metrics: oracle_apm_monitoring

The generic Application Performance Monitoring metrics include default and custom (user-defined) metrics uploaded by the APM Java agent. You don't need to enable monitoring on the APM domain to get these metrics.

This section provides information on the default Application Performance Monitoring metrics. For information on how to create custom metrics, see Custom Metrics.

Default Application Performance Monitoring metrics include the following dimensions:

  • Application Server Dimensions
    • Appserver: The flag to indicate if an application server has been discovered. When the application server type is java-jmx or java-no-jmx, then this dimension is set to false.
    • AppserverDisplayPort: The port number of the application server. The default is the lowest HTTPS port, if available, or the lowest HTTP port.
    • AppserverDomainName: The domain name of the application server.
    • AppserverId: The hash of significant application server resources, which indicates if there is resource change in the application server over time.
    • AppserverName: The name of the application server.
    • AppserverPorts: The port numbers of the application server. Usually, there is only one port number for the Oracle WebLogic server and multiple for the Apache Tomcat server. In case of multiple port numbers, the values are comma separated.
    • AppserverServerName: The name of the application server instance.
    • AppserverType: The type of application server. For example, weblogic or tomcat.
    • AppserverVersion: The version of the application server software.
  • Generic Dimensions
    • ApmVersion: The version of the Application Performance Monitoring agent.
    • DisplayName: The display name of the application server.
    • ServiceInstanceId: The observer ID of the agent.
    • ServiceName: The name you've assigned to the service.
  • Host Dimensions
    • HostAddress: The IPv4 address resolved from the host name.
    • HostAddresses: The list of IPv4 addresses in the network interfaces of the host.
    • Hostname: The name of the host.
    • HostnameCanonical: The canonical name (FQDN) of the host.
    • Hostnames: The list of host names in the network interfaces of the host.
  • Java Virtual Machine (JVM) Dimensions
    • ProcessId: The process ID of the application server.
    • VmName: The name of the virtual machine.
    • VmVendor: The vendor of the virtual machine.
    • VmVersion: The version of the virtual machine.
    • WorkingDirectory: The working directory of the application server.
  • Kubernetes Dimensions
    • KubernetesNamespace: The namespace in the Kubernetes cluster the pod is running in.
    • KubernetesNodeName: The name of the Kubernetes node the pod is running in.
    • KubernetesPodAnnotations: The annotations you've assigned to the pod, if any.
    • KubernetesPodLabels: The labels you've assigned to the pod, if any.
    • KubernetesPodName: The name of the pod (container) in the Kubernetes cluster.
  • Oracle Cloud Infrastructure Dimensions
    • OciAvailabilityDomain: The Oracle Cloud Infrastructure availability domain the compute instance is running in.
    • OciCompartmentId: The OCID of the compartment in which the compute instance resides.
    • OciComputeShape: The shape of the compute instance.
    • OciDisplayName: The display name of the compute instance or Kubernetes node.
    • OciFaultDomain: The name of the Oracle Cloud Infrastructure fault domain the compute instance is running in.
    • OciInstanceId: The OCID of the compute instance.
    • OciRegion: The Oracle Cloud Infrastructure region that contains the availability domain the compute instance is running in.
  • Oracle Cloud Infrastructure Container Engine for Kubernetes (OKE) Dimensions
    • OkeClusterId: The OCID of the OKE cluster.
    • OkeClusterLabel: The cluster label, which are the last 11 characters of the OCID of the OKE cluster.
    • OkeCompartmentName: The Oracle Cloud Infrastructure compartment where the OKE cluster resides.
    • OkeKubernetesVersion: The Kubernetes version.
    • OkeNodePoolId: The OCID of the OKE node pool.
    • OkeNodepoolLabel: The node pool label, which are the last 11 characters of the OCID of the OKE node pool.
    • OkeTenancyId: The OCID of the OKE tenancy.
  • Operating System Dimensions
    • OsAvailableProcessors: The number of processors available to the JVM.
    • OsName: The name of the Operating System.
    • OsVersion: The version of the Operating System.

The following table lists the default Application Performance Monitoring metrics:

Metric Description Unit
GcAfterOldCommitted

The heap committed after old generation garbage collection.

Bytes

GcAfterOldUsed

The heap used after old generation garbage collection.

Bytes

GcAfterYoungCommitted

The heap committed after young generation garbage collection.

Bytes

GcAfterYoungUsed

The heap used after young generation garbage collection.

Bytes

GcOldTotalCount

The number of old generation garbage collections since the JVM started.

GcOldTotalTime

The total time used for old generation garbage collection since the JVM started.

Milliseconds

GcYoungTotalCount

The number of young generation garbage collections since the JVM started.

GcYoungTotalTime

The total time used for young generation garbage collection since the JVM started.

Milliseconds

HeapCommitted

The current system memory allocated for heap usage.

Bytes

HeapUsed

The current heap memory usage. This value is less than or equal to the HeapCommitted value.

Bytes

NonHeapCommitted

The current system memory allocated for non-heap usage.

Bytes

NonHeapUsed

The current non-heap memory usage. This value is less than or equal to the NonHeapCommitted value.

Bytes

ProcessCpuLoad

The current CPU load of the JVM process.

ProcessCpuTime

The total CPU time of the JVM process.

Nanoseconds

SystemCpuLoad

The current CPU load of the system.

SystemPhysicalMemoryFree

The free memory remaining in the system.

Bytes

SystemSwapFree

The free swap disk space remaining in the system.

Bytes

SystemVirtualMemoryCommitted

The committed virtual memory of the JVM process.

Bytes

ThreadBlockedCount

The number of threads in "BLOCKED" state in the JVM.

Note: BLOCKED is the state in which a thread is blocked waiting for a monitor lock. For more information, see Thread States: BLOCKED in Java® Platform, Standard Edition & Java Development Kit Version 16 API Specification.

ThreadCount

The total number of threads in the JVM.

ThreadDeadlockCount

The number of deadlocked threads in the JVM.

Note: DEADLOCK occurs when two or more threads form a cyclic dependency with each other.

ThreadTimedWaitingCount

The number of threads in "TIMED_WAITING" state in the JVM.

Note: TIMED_WAITING is the state in which a thread is waiting for another thread to perform a particular action for a specified waiting time. For more information, see Thread State: TIMED_WAITING in Java® Platform, Standard Edition & Java Development Kit Version 16 API Specification.

ThreadWaitingCount

The number of threads in "WAITING" state in the JVM.

Note: WAITING is the state in which a thread is waiting indefinitely for another thread to perform a particular action. For more information, see Thread State: WAITING in Java® Platform, Standard Edition & Java Development Kit Version 16 API Specification.

TotalTraceCount

The total number of traces evaluated.

TotalSpanCount

The total number of spans evaluated.

TotalTraceSampledCount

The total number of traces sampled.

This value will increment at the same pace as TotalTraceCount metric if there's no sampling configured or enabled.

TotalSpanSampledCount

The total number of spans sampled.

This value will increment at the same pace as TotalSpanCount metric if there's no sampling configured or enabled.

Uptime

The time that has elapsed since the JVM started.

Milliseconds

WeblogicJDBCActiveConnectionsAverageCount The average number of active connections in this instance of the data source.
WeblogicJDBCActiveConnectionsCurrentCount The number of connections currently in use by applications.
WeblogicJDBCActiveConnectionsHighCount The highest number of active database connections in this instance of the data source since the data source was instantiated.
WeblogicJMSConnectionsCurrentCount The current number of connections to this WebLogic Server.
WeblogicJMSConnectionsHighCount The peak number of connections to this WebLogic Server since the last reset.
WeblogicJMSConnectionsTotalCount The total number of connections made to this WebLogic Server since the last reset.
WeblogicJMSServersCurrentCount The current number of JMS servers that are deployed on this WebLogic Server instance.
WeblogicJMSServersHighCount The peak number of JMS servers that were deployed on this WebLogic Server instance since the server was started.
WeblogicJMSServersTotalCount The number of JMS servers that were deployed on this WebLogic Server instance since the server was started.
WeblogicJTAActiveTransactionsTotalCount The number of active transactions on the server.

WeblogicJTATransactionAbandonedTotalCount The total number of transactions that were committed.
WeblogicJTATransactionCommittedTotalCount The total number of transactions committed since the server was started.

WeblogicJTATransactionHeuristicsTotalCount The number of transactions that completed with a heuristic status since the server was started.

WeblogicJTATransactionRolledBackAppTotalCount The number of transactions that were rolled back due to an application error.

WeblogicJTATransactionRolledBackResourceTotalCount The number of transactions that were rolled back due to a resource error.
WeblogicJTATransactionRolledBackSystemTotalCount The number of transactions that were rolled back due to an internal system error.
WeblogicJTATransactionRolledBackTimeoutTotalCount The number of transactions that were rolled back due to a timeout expiration.
WeblogicJVMHeapFreeCurrent The current amount of memory that is available in the JVM heap. Bytes
WeblogicJVMHeapFreePercent The percentage of the JVM heap that is free. Percentage
WeblogicJVMHeapSizeCurrent The current size of the JVM heap. Bytes
WeblogicJVMHeapSizeMax The maximum size of the JVM heap. Bytes
WeblogicServerOpenSocketsCurrentCount The current number of sockets registered for socket muxing on this server.

WeblogicServerState
The current state of the server as an integer. It can be:
  • 0: Shutdown
  • 1: Starting
  • 2: Running

WeblogicThreadPoolCompletedRequestCount

The number of completed requests in the priority queue.

WeblogicThreadPoolExecuteThreadIdleCount The number of idle threads in the pool. This count does not include standby threads and stuck threads. The count indicates threads that are ready to pick up new work when it arrives.

WeblogicThreadPoolExecuteThreadTotalCount The total number of threads in the pool.

WeblogicThreadPoolHoggingThreadCount The threads that are currently being held by a request right now. These threads will either be declared as stuck after the configured timeout or will return to the pool before that. The self-tuning mechanism will backfill if necessary.

WeblogicThreadPoolPendingUserRequestCount

The number of pending user requests in the priority queue. The priority queue contains requests from internal subsystems and users. This is just the count of all user requests.

WeblogicThreadPoolQueueLength The number of pending requests in the priority queue. This is the total of internal system requests and user requests.

WeblogicThreadPoolStandbyCount The number of threads in the standby pool. Threads that are not needed to handle the present work load are designated as standby and added to the standby pool. These threads are activated when more threads are needed.

WeblogicThreadPoolStuckCount The number of stuck threads in the thread pool.

WeblogicThreadPoolThroughput The mean number of requests completed per second. Request per second
WeblogicWebAppOpenSessionsCurrentCount The count of the current total number of open sessions in this module.

Using the Console

Build Application Performance Monitoring Metric Queries

You can build Application Performance Monitoring metric queries in Metrics Explorer and monitor your applications by various dimensions. For example, you can build a query using the oracle_apm_rum namespace, ScriptErrors metric, WebApplicationName dimension to monitor the script errors in your application for a specified period of time. You can also add another dimension, BrowserName, to determine if the script errors are browser-related.

  1. Sign in to the Oracle Cloud Infrastructure console.
  2. Open the navigation menu, click Observability & Management. Under Monitoring, click Metrics Explorer.

    The Metrics Explorer page displays an empty chart with fields to create a query.

  3. Select a time period using the Start time and End time or Quick Selects fields on the top of the page.
  4. Scroll down and fill in the following fields for your query:
    • Compartment: Ensure that the compartment in which your APM domain resides is selected.
    • Metrics namespace: Select one of the following Application Performance Monitoring namespaces:
      • oracle_apm_rum
      • oracle_apm_synthetics
      • oracle_apm_monitoring
    • Metrics name: Select a metric within the selected Metric Namespace.
    • Interval: Select an aggregation window.
    • Statistic: Select an aggregation function.
    • Metric dimensions: Select a dimension in the Dimension name field and a value for the specified dimension in the Dimension value field to filter the metric data. Optionally, you can add an additional dimension.
      Note

      In addition to the dimensions you want to select, all the metric queries for the oracle_apm_rum namespace must include the following dimensions to ensure correctness of data:
      • MetricGroupName = "No Group"
      • WebApplicationName = All Web Applications or WebApplicationName = <name of the individual web application>
  5. Click Update Chart.

The chart will be updated to display the metrics that have been requested.

For more information on Metrics Explorer and how to build a metric query, see Building Metric Queries.

Create an Alarm for Application Performance Monitoring Metrics

You can create an alarm using the Oracle Cloud Infrastructure Monitoring service to be notified if an Application Performance Monitoring metric breaches the specified threshold. For example, if you've built a query to monitor the JavaScript errors in browsers, you can create an alarm to be notified if the errors for any browser go over 5%.

After you create an alarm for an Application Performance Monitoring metric, you can go to the Application Performance Monitoring Home page and monitor firing alarms that require your attention in the Alarms widget and click an alarm to navigate to the Alarms Definition page.

For information on how to create a threshold alarm for Application Performance Monitoring metrics, see Using the Console to Create an Alarm.

Using the API

Use the following APIs for monitoring: