Using Vector for Log Streaming

The Vector service allows you to send the Private Cloud Appliance logging information from Loki to an external location that you specify.

Beginning with Private Cloud Appliance software version 3.0.2-b1261765, Vector is installed, configured, and enabled on the appliance by default.

To specify which data you want and where you want the data sent, sign in to the currently active management node as the root user and customize the Vector configuration file It is stored in /nfs/shared_storage/log_streaming/pca_vector.yaml.

Edit the configuration file to customize the sinks section. See the Vector Sinks Reference. The following is a sample pca_vector.yaml file:

# Copyright (c) 2024, Oracle and/or its affiliates.

# DO NOT TURN API OFF
# otherwise livenessProbe will fail
api:
  enabled: true
  # Bind to 0.0.0.0. Otherwise the API will not be exposed outside the container.
  address: "0.0.0.0:8686"

sources:
  fluentd_source:
    type: fluent
    address: "0.0.0.0:8080"
    mode: tcp
    encoding:
      codec: json

transforms:
  log_event:
    type: remap
    inputs:
      - fluentd_source
    source: |
      log(.)

sinks:
  loki_sink:
    type: loki
    inputs:
      - fluentd_source
    endpoint: http://your_external_location:3100
    encoding:
      codec: json
    labels:
      job: "vector"
      namespace: "default"
      system: "pca_name.example.com"
      filename: "{{tag}}"

In the endpoint value, your_external_location can be an IP address or a domain name. At this location, you could install Grafana or use other tools to filter, manipulate, and display the data.

The value of the filename label that is shown in the example causes the name of the source log file to be shown in the Vector Loki Sink output. You can then use that file name as a label to search within Loki and Grafana.

The following is an example Splunk sink:

sinks:
  splunk_sink:
    type: splunk_hec_logs
    inputs:
      - source_id
    endpoint: https://splunk_endpoint
    token: splunk-hec-token
    encoding:
      codec: json
    tls:
      ca_file: "/path/to/ca.pem"

The splunk-hec-token is required to send logs to Splunk. The ca_file is optional if you are using HTTPS. For more information about HTTP Event Collector and how to configure and use Splunk, see the Splunk Documentation.

The following command reports the status of the log streaming pod (the Vector service):

# kubectl get pods -n log-streaming
NAME                           READY   STATUS    RESTARTS   AGE
log-streamer-bc4d65d78-ndrsk   1/1     Running   0          14d

The following command prints the logs from the log-streamer-bc4d65d78-ndrsk pod, which has only one container. For more information about the kubectl logs command, see kubectl logs on the Kubernetes site.

# kubectl logs log-streamer-bc4d65d78-ndrsk -n log-streaming

Oracle Cloud Infrastructure Documentation

Using Vector for Log Streaming