Note:
- This tutorial requires access to Oracle Cloud. To sign up for a free account, see Get started with Oracle Cloud Infrastructure Free Tier.
- It uses example values for Oracle Cloud Infrastructure credentials, tenancy, and compartments. When completing your lab, substitute these values with ones specific to your cloud environment.
Monitor Applications using OCI Application Performance Monitoring and OpenTelemetry
Introduction
Monitoring applications that vary in stack components can often be challenging. A unified approach to monitor these components allows for end-to-end visibility and insight into issues that may arise. OpenTelemetry is an industry standard for collecting trace, metric, and log data for most programming languages. It can provide powerful insights when combined with Oracle Cloud Infrastructure (OCI) Observability and Management services such as OCI Application Performance Monitoring and OCI Logging Analytics.
OCI Application Performance Monitoring provides deep visibility into the performance of applications and provides the ability to diagnose issues quickly to deliver a consistent level of service. This includes the monitoring of the multiple components and application logic spread across clients, third-party services, and back-end computing tiers, on-premises or on the cloud.
OCI Logging Analytics is a cloud solution in OCI that index, enrich, aggregate, explore, search, analyze, correlate, visualize, and monitor all log data from applications and system infrastructure. Gain powerful insights from log data using curated AI/ML models built within OCI Logging Analytics.
OpenTelemetry has various implementations for different programming languages and is constantly being updated or developed. The purpose of using this framework is to have an unified and standardized way to collect and send monitoring data.
In this tutorial, you will learn how to instrument an application with OpenTelemetry and export the collected data to OCI services such as OCI Application Performance Monitoring and OCI Logging Analytics.
Objectives
-
Configure OpenTelemetry for a non-Oracle stack application.
-
Send traces and metrics to OCI Application Performance Monitoring.
-
Send log data to OCI Logging Analytics.
Prerequisites
-
An OCI tenancy with the initial administrator account.
-
A
Node.js
application (GitHub repository example: oci-observability-and-management). -
A basic understanding of JavaScript.
GitHub Example
If you would like to implement monitoring for MERN/MEAN/MEVN application(s), use the GitHub repository oci-observability-and-management. This repository includes files to:
-
Instrument the application frontend and backend.
-
Send custom metrics to OCI Application Performance Monitoring (APM).
-
Send log messages to OCI Logging Analytics.
-
A sample application to experiment with the OpenTelemetry implementation.
Task 1: Configure OCI Application Performance Monitoring
-
Create an OCI Application Performance Monitoring (APM) domain
Note: Review the OCI Application Performance Monitoring documentation for prerequisite tasks.
-
Log in to the OCI Console and navigate to Observability & Management, Application Performance Management, Administration to open OCI Application Performance Monitoring administration page.
-
Click Create APM Domain and enter a domain name. Note the following information found in the OCI Application Performance Monitoring domain page.
-
Data upload endpoint: URL for sending data to OCI Application Performance Monitoring.
-
Public data key: Used with the OCI Application Performance Monitoring browser agent.
-
Private data key: To connect with data collectors such as OpenTelemetry.
-
-
-
Instrument an Application
To monitor an application’s frontend, instrument the OCI Application Performance Monitoring browser agent. For more information, see Browser/Client Instrumentation Steps.
Server Instrumentation
-
To get end-to-end traces, context propagation is needed from the frontend traces of a browser/client to the server. One thing to keep in mind when doing so is ensuring there are HTTP headers to provide the context. By default, OpenTelemetry uses W3C (traceparent) to automatically propagate the context. There are other header types which can be used but need to be accounted in the application code.
-
Be sure to add the following code with the snippet of JavaScript that runs the browser/client instrumentation.
window.apmrum.traceSupportingEndpoints = [ { headers: [ 'W3C'], hostPattern: '.*' } ];
-
Instrument the automatic instrumentation or manual backend depending on what is available, see steps for your programming language.
Examples in GitHub
-
Task 2: Configure OCI Logging Analytics
-
Send Application Logs
Send application logs to OCI Logging Analytics which enables to correlate traces, metrics and logs to gain complete visibility of the application. Logging can be enabled using standard logging libraries and custom log appender. Log appender must be built to send the logs using OCI provided SDK or REST endpoints as shown.
-
Create an API Signing Key
-
Go to the OCI Console and navigate to User, User Settings.
-
Select API Keys under Resources.
-
Click Add API Key, Generate API Key Pair, Download Private Key and Add.
-
Copy the content from Configuration File Preview and click Close.
-
-
Create a Configuration File
Create a directory (
.oci
) and a config file with content from configuration file preview and path to private key file. Below is an example of the configuration file.[DEFAULT] user= [User OCID] fingerprint= [API Key Fingerprint] tenancy= [Tenancy OCID] region= [Region] key_file= [Path to Private Key File]
-
Create a Log Parser
-
Go to the OCI Console and navigate to Observability & Management, Logging Analytics, Administration.
-
Click Parsers, Create Parser and select Type as
JSON
. -
Enter example
JSON
log content, it will parse and extract fields and map it to specific field names and click Save Changes.
-
-
Create a Log Source
-
Go to the OCI Console and navigate to Logging Analytics, Administration, Sources , Create Source.
-
Enter Source Type as
File
and Entity Types asHost (Linux)
. -
In Specific Parser, select the parser created in Task 2.4 and click Create Source.
-
-
Create a Log Group
-
Go to the OCI Console and navigate to Logging Analytics, Administration, Log Groups, Create Log Group.
-
Enter Name, Description of log group and click Create.
Note: Note the Oracle Cloud Identity (OCID) of log group which will be used later.
-
-
Get the Namespace Details
-
Go to the OCI Console and navigate to Identity, Compartments and click on compartment where the log source is created and copy OCID of the compartment.
-
Open Cloud Shell and run the following command to get the namespace.
oci os ns get -c compartmentID
-
-
Create a Log Appender
A custom log appender can be built using OCI Logging Analytics SDKs provided for different programming languages like Java, Python, .Net, TypeScript/JavaScript, Go and Ruby or use OCI API Rest endpoints. For more information, see Software Development Kits (SDKs) and API Reference and Endpoints.
Example in GitHub: Log Appender – JavaScript (MERN Stack).
Make a note of the following parameters required to initialize and send logs to OCI Logging Analytics from the above steps.
- [PATH]/config: Path to config file.
- [PROFILE]: Profile in config file to be used for OCI authentication.
- [NAMESPACE]: Namespace.
- [UPLOADNAME]: User defined name for uploads.
- [LOGSOURCENAME]: Log source name created in OCI Logging Analytics.
- [LOGFILENAME]: Log file name to indicate log messages are related to specific log file.
- [LOGGROUPID]: Log group ID created in OCI Logging Analytics to group the log messages.
Once the log records are created for different log levels (debug, info, warn and error) through log appender, it will be sent to OCI Logging Analytics, and you can view the log records in Log Explorer as shown below.
Task 3: Correlate Traces and Logs
OpenTelemetry includes TraceId
and SpanId
in the log records and this allows to directly correlate logs and traces that correspond to the same execution context. Application traces and spans flows into OCI Application Performance Monitoring and logs into OCI Logging Analytics service. OCI Application Performance Monitoring provides an easy way to navigate from traces and spans to logs in OCI Logging Analytics in one click using a drilldown configuration.
Drilldowns are links to other services in OCI or other custom services using customizable URLs including attributes from spans (for example: loganalytics/explorer?search=<OciInstanceId>
where ociInstanceId
is a span attribute). Follow the steps below to configure drilldowns in OCI Application Performance Monitoring.
Task 4: Create a Custom Dashboard
Once tracing data and metrics are in OCI, use that data to visually represent it. Creating a custom dashboard is easy, just drag-and-drop the widgets needed and modify the source data (metric, trace, or log data).
For more information on custom dashboards, see:
- Creating custom APM dashboards
- Creating custom Logging Analytics dashboards
- Customize and display trace data in Application Performance Monitoring dashboards using widgets to create custom widgets from Application Performance Monitoring’s Trace Explorer queries.
Example dashboard that was created for a MERN application.
Related Links
Acknowledgments
- Author - Zyaad Khader (Strategic Customer Program Engineer)
More Learning Resources
Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.
For product documentation, visit Oracle Help Center.
Monitor Applications using OCI Application Performance Monitoring and OpenTelemetry
F91894-01
January 2024
Copyright © APMOT, Oracle and/or its affiliates.