Oracle Cloud Infrastructure Documentation

Scenario: Archiving Logs to Object Storage

Archive log data to a bucket in Object Storage.

This scenario involves creating a connector. The connector  processes and moves log data from Logging to Object Storage.

Required IAM Policy

If you're a member of the Administrators group, you already have the required access to execute this scenario. Otherwise, you need access to Monitoring , Notifications , and Object Storage .

The workflow for creating the connector includes a default policy when needed to provide permission for writing to the target service. If you're new to policies, see IAM Policies Overview.

Setting Up This Scenario

This example walks through creating a connector that receives subnet log data from Logging and then moves this data to a bucket in Object Storage.

Setup is easy in the Console. Alternatively, you can use the Oracle Cloud Infrastructure CLI or API, which lets you execute the individual operations yourself.

For help with troubleshooting, see Troubleshooting Connectors.

  • Note

    For complete descriptions of fields on the Create connector page, see Creating a Connector.

    On the Connectors list page, select Create connector. If you need help finding the list page, see Listing Connectors.

    1. Basic Connector Information

    Enter identifying information.

    • Connector name: Enter a user-friendly name for the new connector and an optional description. Avoid entering confidential information. Example: Archive Logs
    • Select a compartment: Select the compartment to store the new connector in.

    Select Next.

    2. Configure Connector Source

    Select your subnet log.

    • Select source: Select Logging.
    • Configure source: Select the compartment, log group, and your subnet log.

    Select Next.

    3. Configure Connector Task

    Select Next (skip this section).

    4. Configure Connector Target

    Select the bucket where you want to archive the log data.

    • Target: Select Object Storage.
    • Configure target: Select the compartment and bucket where you want to store the received log data.
    • Create policy: Select to accept the default policy provided for the entered target configuration.

    Select Next.

    Connector Preview

    Review the connector configuration and then select Create.

    The creation process begins, and its progress is displayed. On completion, the connector's details page opens.

  • For information about using the API and signing requests, see REST API documentation and Security Credentials. For information about SDKs, see SDKs and the CLI.

    • Create a connector: Open a command prompt and run the oci sch service-connector create command: 

      oci sch service-connector create --display-name
"<display_name>" --compartment-id <compartment_OCID> --source [<source_in_JSON>] --target [<targets_in_JSON>]

    For help with troubleshooting, see Troubleshooting Connectors.

  • For information about using the API and signing requests, see REST API documentation and Security Credentials. For information about SDKs, see SDKs and the CLI.

    Use the following operations:

    • CreateServiceConnector: Create a connector.

      Example CreateServiceConnector request
      POST /20200909/serviceConnectors
Host: service-connector-hub.us-phoenix-1.oraclecloud.com
<authorization and other headers>
{
  "compartmentId": "<compartment_OCID>",
  "description": "My connector description",
  "displayName": "My Connector",
  "source": {
    "kind": "logging",
    "logSources": [
      {
        "compartmentId": "<compartment_OCID>",
        "logGroupId": "<log_group_OCID>",
        "logId": "<log_OCID>"
      }
    ]
  },
  "target": {
    "kind": "objectStorage",
    "bucketName": "MyBucket",
    "objectNamePrefix": "myprefix"
  }
}

    For help with troubleshooting, see Troubleshooting Connectors.