Scenario: Analyze Logs

This topic explains how to send log data to Logging Analytics.

This scenario involves creating a log group and a service connector. The service connector (Service Connector Hub)  processes and moves log data from Logging to the log group in Logging Analytics.

Required IAM Policy

To use Oracle Cloud Infrastructure, you must be granted security access in a policy  by an administrator. This access is required whether you're using the Console or the REST API with an SDK, CLI, or other tool. If you get a message that you don’t have permission or are unauthorized, verify with your administrator what type of access you have and which compartment  to work in.

If you're a member of the Administrators group, you already have the required access to execute this scenario. Otherwise, you need access to Logging Analytics to create the log group and access to Service Connector Hub to create the service connector.

The workflow for creating the service connector includes a default policy when needed to provide permission for writing to the target service.

Setting Up This Scenario

Setup is easy in the Console. Alternatively, you can use the Oracle Cloud Infrastructure CLI or API, which lets you execute the individual operations yourself.

Using the Console

This section walks through creating a log group and a service connector using the Console.

For help with troubleshooting, see Troubleshooting Service Connectors.

Task 1: Create the log group

Use Logging Analytics to create the log group. For instructions, see Create Log Groups.

Task 2: Create the service connector

This example walks through using the Console to create a service connector that sends log data from Logging to the log group you created using Logging Analytics. In this example, the service connector filters VCN flow log data.

  1. Open the navigation menu and click Analytics & AI. Under Messaging, click Service Connector Hub.
  2. Choose the Compartment where you want to create the service connector.
  3. Click Create Service Connector.
  4. On the Create Service Connector page, filter VCN flow log data to your log group:

    • Type a Connector Name such as "VCN Flow Log Error Analysis."
    • Select the Resource Compartment where you want to store the new service connector.
    • Under Configure Service Connector, select your source and target services to move log data to the log group:
      • Select Source: Logging
      • Select Target: Logging Analytics
    • Under Configure source connection, select your VCN flow log:
      • Compartment: The compartment containing the VCN flow log data.
      • Log Group: The log group containing the VCN flow log data.
      • Logs: The log object name for your VCN flow logs.
    • Under Configure task, filter the log data to rejected traffic:

      • Property: data.action
      • Operator: =
      • Value: REJECT

      If you are interested in rejected traffic for a particular port or address, add another filter. For example, select the property data.destinationPort or data.destinationAddress.

    • Under Configure target connection, enter the log group that you want to send the filtered log data to:
  5. If prompted to create a policy (required for access to create or update a service connector), click Create.
  6. Click Create.

Using the CLI

This section walks through creating the log group and service connector using the CLI.

For information about using the API and signing requests, see REST APIs and Security Credentials. For information about SDKs, see Software Development Kits and Command Line Interface.
  1. Create a log group: Open a command prompt and run the oci log-analytics log-group create command:

    oci log-analytics log-group create --display-name "<display_name>" --compartment-id <compartment_OCID> --namespace-name "<namespace_name>"
  2. Create a service connector: Open a command prompt and run the oci sch service-connector create command:

    oci sch service-connector create --display-name "<display_name>" --compartment-id <compartment_OCID> --source [<source_in_JSON>] --tasks [<tasks_in_JSON>] --target [<targets_in_JSON>]

For help with troubleshooting, see Troubleshooting Service Connectors and Troubleshooting Notifications.

Using the API

This section walks through creating the log group and service connector using the API.

For information about using the API and signing requests, see REST APIs and Security Credentials. For information about SDKs, see Software Development Kits and Command Line Interface.

Use the following operations:

  1. CreateLogAnalyticsLogGroup: Create a log group.

    Example CreateLogAnalyticsLogGroup request
    post /20200601/namespaces/<namespaceName>/logAnalyticsLogGroups
    Host: loganalytics.us-phoenix-1.oci.oraclecloud.com
    <authorization and other headers>
    {
      "compartmentId": "<compartment_OCID>",
      "displayName": "My Log Group"
    }
  2. CreateServiceConnector: Create a service connector.

    Example CreateServiceConnector request
    POST /20200909/serviceConnectors
    Host: service-connector-hub.us-phoenix-1.oraclecloud.com
    <authorization and other headers>
    {
      "compartmentId": "<compartment_OCID>",
      "description": "My service connector description",
      "displayName": "My Service Connector",
      "source": {
        "kind": "logging",
        "logSources": [
          {
            "compartmentId": "<compartment_OCID>",
            "logGroupId": "<log_group_OCID>",
            "logId": "<log_OCID>"
          }
        ]
      },
      "target": {
        "compartmentId": "<compartment_OCID>",
        "kind": "loggingAnalytics",
        "logGroupId": "<logging_analytics_log_group_OCID>"
      },
      "tasks": [
        {
          "condition": "data.action='REJECT'",
          "kind": "logRule"
        }
      ]
      }
    }

For help with troubleshooting, see Troubleshooting Service Connectors and Troubleshooting Notifications.