Here's information on configuring the OpenTelemetry (OTEL) data sources (such as instrumentation libraries and OpenTelemetry Collector) to upload monitoring data (metrics and traces) to Application Performance Monitoring.

const opentelemetry = require("@opentelemetry/sdk-node");
const {
 getNodeAutoInstrumentations,
} = require("@opentelemetry/auto-instrumentations-node");
const {
 OTLPTraceExporter,
} = require("@opentelemetry/exporter-trace-otlp-http");

  const sdk = new opentelemetry.NodeSDK({
traceExporter: new OTLPTraceExporter({
  url: "https://aaabbbb.example.us-phoenix-1.oci.oraclecloud.com/20200101/opentelemetry/private/v1/traces",
  headers: {"Authorization": "dataKey AAAAABBBBBCCCC"},
 }),

  instrumentations: [getNodeAutoInstrumentations()]
});

 sdk.start()
(base) [ssirajud@ssirajud node_with_opentelemetry]$

export OTEL_TRACES_EXPORTER=otlp
export OTEL_SERVICE_NAME=my-service
export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf
export OTEL_EXPORTER_OTLP_TRACES_HEADERS="authorization=dataKey 1111aaaabbbccc"
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://aaabbbb.example.us-phoenix-1.oci.oraclecloud.com/20200101/opentelemetry/private/v1/traces/
export OTEL_METRICS_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=https://aaabbbb.example.us-phoenix-1.oci.oraclecloud.com/20200101/opentelemetry/v1/metrics
export OTEL_EXPORTER_OTLP_METRICS_HEADERS="authorization=dataKey 1111aaaabbbccc"
export OTEL_EXPORTER_OTLP_METRICS_PROTOCOL=http/protobuf

Here's information on the changes that must be made to the code of a Java application that uses the Jaeger client for distributed tracing, and use it to upload traces to Application Performance Monitoring.

To use the Jaeger tracer to upload traces from a Java application to Application Performance Monitoring, you must configure the Jaeger Java client for Zipkin compatibility (report in Zipkin v2 format) and add the Application Performance Monitoring collector URL. To do so, you must make the following changes to your service's code:

1. Along with the required Jaeger client dependencies, add the following dependencies to the pom.xml file.

   <dependency>
       <groupId>io.zipkin.reporter2</groupId>
       <artifactId>zipkin-sender-urlconnection</artifactId>
       <version>2.15.0</version>
   </dependency>
   <dependency>
       <groupId>io.jaegertracing</groupId>
       <artifactId>jaeger-zipkin</artifactId>
       <version>1.3.2</version>
   </dependency>

2. Use the ZipkinV2Reporter to adapt a Zipkin 2 reporter to the Jaeger reporter interface and convert Jaeger spans to the JSON-encoded Zipkin v2 format. Ensure that you've specified the Application Performance Monitoring collector URL to upload spans to Application Performance Monitoring, and the name of your service or application.

   import io.jaegertracing.Configuration;
   import io.jaegertracing.zipkin.ZipkinV2Reporter;
   import zipkin2.reporter.AsyncReporter;
   import zipkin2.reporter.urlconnection.URLConnectionSender;
    
   /* .... */
    
   ZipkinV2Reporter reporter = new ZipkinV2Reporter(AsyncReporter.create(URLConnectionSender.create("<Application Performance Monitoring collector URL>")));
   JaegerTracer tracer = Configuration
       .fromEnv("<service-name>")
       .getTracerBuilder()
       .withReporter(reporter)
       .withSampler(new ConstSampler(true))
       .build();
    
   /* .... */

3. Register B3TextMapCodec to use B3 propagation, which is a specification for the b3 header. These headers are used for trace context propagation across service boundaries.

   import io.jaegertracing.Configuration;
   import io.jaegertracing.internal.propagation.B3TextMapCodec;
   import io.opentracing.propagation.Format;
    
   /* .... */
    
   B3TextMapCodec b3Codec = new B3TextMapCodec.Builder().build();       
   JaegerTracer tracer = Configuration
       .fromEnv("<service-name>")
       .getTracerBuilder()
       .registerInjector(Format.Builtin.HTTP_HEADERS, b3Codec)
       .registerExtractor(Format.Builtin.HTTP_HEADERS, b3Codec)
       .build();
    
   /* .... */

   Note that if the Jaeger tracer is created using environment configuration (Configuration.fromEnv()) , setting the JAEGER_PROPAGATION environment variable to b3c sets the Zipkin (B3) context propagation mode without code changes.

Here's information on the changes that must be made to the code of a Node.js application that uses the Zipkin client for distributed tracing, and use it to upload traces to Application Performance Monitoring.

To do so, make the following changes to the Zipkin tracer initialization code:

1. Ensure that you've made the necessary changes to report spans in the JSON-encoded Zipkin v2 format.

   For information about the Zipkin v2 format, see Zipkin 2.

2. Add the Application Performance Monitoring collector URL, to upload spans to Application Performance Monitoring, and the name of your service or application.
3. Disable the Zipkin "span joining" feature, if it's enabled. OpenTracing specification does not allow Span ID sharing between different spans within the same trace and this feature must be disabled.

Node.js Example

Here's a sample of the Zipkin tracer initialization code, and the changes detailed in the procedure are in bold:

const {
  Tracer,
  BatchRecorder,
  jsonEncoder: {JSON_V2}
} = require('zipkin');
 
const CLSContext = require('zipkin-context-cls');
const {HttpLogger} = require('zipkin-transport-http');
 
// Setup the tracer
const tracer = new Tracer({
  ctxImpl: new CLSContext('zipkin'), // implicit in-process context
  recorder: new BatchRecorder({
    logger: new HttpLogger({
      endpoint: '<Application Performance Monitoring collector URL>', //Span collection endpoint URL setting
      jsonEncoder: JSON_V2 //Span format and encoding setting
    })
  }), // batched http recorder
  localServiceName: '<service-name>', // name of the application/service
  supportsJoin: false //Span join disable setting
});

The Application Performance Monitoring collector URL is required when configuring open-source tracers to communicate with Application Performance Monitoring service. This topic provides details about the APM Collector URL format.

The Application Performance Monitoring collector URL has the following format:

<dataUploadEndpoint>/<API version>/observations/<observationType>?dataFormat=<dataFormat>&dataFormatVersion=<dataFormatVersion>&dataKey=<dataKey>

Obtain the <dataUploadEndpoint> value from the APM domain details page.

https://aaaabbbb.example.us-ashburn-1.oci.oraclecloud.com/20200101/observations/public-span?dataFormat=zipkin&dataFormatVersion=2&dataKey=11115Q7IGN

spring.zipkin.baseUrl =  https://aaaabbbb.example.us-ashburn-1.oci.oraclecloud.com
spring.zipkin.apiPath = /20200101/observations/public-span?dataFormat=zipkin&dataFormatVersion=2&dataKey=11115Q7IGN

https://aaaabbbb.example.us-phoenix-1.oci.oraclecloud.com/20200101/observations/private-span?dataFormat=otlp-span&dataFormatVersion=1&dataKey=11112222UZ5

Configure context propagation to get the trace context information propagated in the appropriate format, and add it to one or more HTTP request headers. The trace context enables the correlation of spans that come from different microservices, but belong to the same transaction (trace) and are identified by a common Trace ID.

APM Agent/Tracer

The APM Java Agent/Tracer can be configured to work with other OpenTracing tracers (such as Jaeger and Zipkin) in order to form a complete trace.

When a HTTP call occurs, the APM Java Agent/Tracer will attempt to add HTTP headers to the call which can then be read on the other end to help connect spans together.

You can change this format via the com.oracle.apm.agent.tracer.propagation.type property, which can be found in the AgentConfig.properties file of the agent's config directory.

You can utilize the following propagation types by setting the property to the name:

Set the appropriate propagation type in order to have your various tracers being able to work together to form a complete trace.

OpenTelemetry, Zipkin & Jaeger