Note:

Send Oracle Cloud Infrastructure Logs to New Relic using Vector

Introduction

Oracle Cloud Infrastructure (OCI) generates various logs that need to be stored, analyzed, and monitored efficiently.

Vector is an open-source, high-performance observability pipeline that collects, transforms, and routes logs to various destinations, including New Relic. Vector acts as a log shipper, efficiently processing logs before forwarding them. It can be deployed as an agent-based solution on compute instances or used as a centralized aggregator to handle logs from multiple sources. Vector is an open-source tool developed by Datadog for collecting, transforming, and forwarding logs and other observability data. Think of it as a flexible pipeline that connects your logs from source to destination, making the flow efficient and customizable. For more information, see Vector.

New Relic brings your logs to life with real-time visibility and powerful context, helping teams troubleshoot faster and surface insights instantly. By unifying logs, metrics, and traces in one platform, you reduce noise and focus on what matters. With built-in SIEM capabilities, New Relic empowers security teams to detect, investigate, and respond to threats quickly—turning observability into a force multiplier for security operations. For more information, see New Relic.

This tutorial explains how to set up logs and connectors to send them from OCI Logging and OCI Streaming. It also includes Vector as a log shipper for reading data from OCI Streaming and sending them securely to New Relic.

The following image shows the architecture for how the logs will flow from OCI and New Relic.

Edit button

Objectives

This setup is lightweight, extensible, and secure, perfect for modern observability needs.

Prerequisites

Task 1: Configure OCI to Forward Logs to OCI Streaming

  1. Enable logs in OCI Logging.

    For this tutorial, we will use Audit logs. You can also enable service or custom logs based on your use case. For more information, see Logging Overview.

  2. Create a stream.

    1. To create a stream, see Creating a Stream.

      Edit button

    2. Note down the stream name, username from stream pool and generated auth token for the user. For more information, see auth token.

      We will need all these in our vector configuration file.

      Edit button

  3. Create an OCI Connector Hub.

    To create a connector hub, enter the following information.

    • Source: Select Logging.
    • Destination: Select Streaming (select the stream created in step 2).
    • Select Create policies automatically to generate required OCI IAM policies.

    For more infromation, see Creating a Connector with a Logging Source.

    Edit button

    Edit button

    Edit button

Task 2: Install Vector

Download the latest Vector release from here: Vector v0.46.1 and install in any virtual machine. In this tutorial, we are using OCI Compute instance.

Run the following sample commands to download and install Vector.

wget https://github.com/vectordotdev/vector/releases/download/v0.45.0/vector-0.45.0-1.aarch64.rpm
sudo dnf install -y vector-0.45.0-1.aarch64.rpm
sudo systemctl start vector
sudo systemctl enable vector

Task 3: Configure Vector to Read Logs from OCI Streaming

Create and edit the Vector configuration file at /etc/vector/vector.yaml.

sources:
  oci_streaming:
    type: kafka
    bootstrap_servers: "cell-1.streaming.us-ashburn-1.oci.oraclecloud.com:9092" #your stream endpoint
    topics: ["vector-stream"] #your stream name
    group_id: "vector-log-processor"
    sasl:
      enabled: true
      mechanism: "PLAIN"
      username: "${STREAM_KAFKA_USERNAME}" #your stream username
      password: "${STREAM_KAFKA_PASSWORD}" #your stream password
    tls:
      enabled: true
      verify_certificate: false
      ca_file: "/etc/vector/oci_ca_cert.pem"

sinks:
  new_relic:
    type: http
    inputs: ["oci_streaming"]
    uri: https://log-api.newrelic.com/log/v1 #your new relic api log ingestion endpoint
    method: post
    auth:
      strategy: bearer
      token: "${NEW_RELIC_API_KEY}" #your new relic license/api key
    encoding:
      codec: "json"
    tls:
      verify_certificate: true
    compression: "gzip"
    batch:
      max_bytes: 1048576

Note: Modify the stream endpoint, stream name, new relic api log ingestion endpoint, NEW_RELIC_API_KEY, STREAM_KAFKA_USERNAME, STREAM_KAFKA_PASSWORD based on your setup.

Key Sections:

Task 4: Verify Logs in New Relic

After starting Vector, run the following command to check the logs.

sudo journalctl -u vector -f

You should see logs flowing into New Relic. Log in to your New Relic dashboard and navigate to Logs to confirm ingestion.

Edit button

Acknowledgments

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.