Oracle Cloud Infrastructure Documentation

Scenario: Archive Logs to Object Storage

This topic explains how to archive log data to a bucket in Object Storage.

This scenario involves creating a service connector. The service connector  processes and moves log data from Logging to Object Storage.

Required IAM Policy

To use Oracle Cloud Infrastructure, you must be granted security access in a policy  by an administrator. This access is required whether you're using the Console or the REST API with an SDK, CLI, or other tool. If you get a message that you don’t have permission or are unauthorized, verify with your administrator what type of access you have and which compartment  you should work in.

If you're a member of the Administrators group, you already have the required access to execute this scenario. Otherwise, you need access to Monitoring, Notifications, and Object Storage.

The workflow for creating the service connector includes a default policy when needed to provide permission for writing to the target service.

Setting Up This Scenario

Setup is easy in the Console. Alternatively, you can use the Oracle Cloud Infrastructure CLI or API, which lets you execute the individual operations yourself.

Using the Console

This example walks through using the Console to create a service connector that receives subnet log data from Logging and then moves this data to a bucket in Object Storage.

For help with troubleshooting, see Troubleshooting Service Connectors.

Create the service connector
  1. Open the navigation menu. Under Data and AI, click Service Connector Hub.
  2. Choose the Compartment where you want to create the service connector.
  3. Click Create Service Connector.

  4. On the Create Service Connector page, send subnet log data to a bucket:

    • Type a Connector Name, such as "Archive Logs." Avoid entering confidential information.
    • Select the Resource Compartment where you want to store the new service connector.
    • Under Configure Service Connector, select your source and target services to move log data to a metric:
      • Select Source: Logging
      • Select Target: Object Storage
    • Under Configure source connection, select your subnet log:
      • Compartment: The compartment containing the log data you want.
      • Log Group: The log group containing the log data you want.
      • Logs: The log object name for the log data you want.
    • Under Configure target connection, select the bucket where you want to archive this log data:
      • Select the Service Compartment where you want to store the received (filtered) log data.
      • Select the Bucket you want.
  5. If prompted to create a policy (required for access to create or update a service connector), click Create.
  6. Click Create.

Using the CLI

This example walks through using the CLI to create a service connector that receives subnet log data from Logging and then moves this data to a bucket in Object Storage.

For information about using the API and signing requests, see REST APIs and Security Credentials. For information about SDKs, see Software Development Kits and Command Line Interface.

  • Create a service connector: Open a command prompt and run the oci sch service-connector create command: 

    oci sch service-connector create --display-name
"<display_name>" --compartment-id <compartment_OCID> --source [<source_in_JSON>] --target [<targets_in_JSON>]

For help with troubleshooting, see Troubleshooting Service Connectors.

Using the API

This example walks through using the API to create a service connector that receives subnet log data from Logging and then moves this data to a bucket in Object Storage.

For information about using the API and signing requests, see REST APIs and Security Credentials. For information about SDKs, see Software Development Kits and Command Line Interface.

Use the following operations:

  • CreateServiceConnector: Create a service connector.

    Example CreateServiceConnector request
    POST /20200909/serviceConnectors
Host: service-connector-hub.us-phoenix-1.oraclecloud.com
<authorization and other headers>
{
  "compartmentId": "<compartment_OCID>",
  "description": "My service connector description",
  "displayName": "My Service Connector",
  "source": {
    "kind": "logging",
    "logSources": [
      {
        "compartmentId": "<compartment_OCID>",
        "logGroupId": "<log_group_OCID>",
        "logId": "<log_OCID>"
      }
    ]
  },
  "target": {
    "kind": "objectStorage",
    "bucketName": "MyBucket",
    "objectNamePrefix": "myprefix"
  }
}

For help with troubleshooting, see Troubleshooting Service Connectors.