Oracle Cloud Infrastructure Documentation

Scenario: Alarming on Log Data

Set up alarms for log data using Connector Hub, Logging, and Monitoring services.

This scenario involves creating a connector and an alarm. The connector (Connector Hub)  processes and moves log data from Logging to Monitoring while the alarm  fires when triggered by received log data.

Required IAM Policy

If you're a member of the Administrators group, you already have the required access to execute this scenario. Otherwise, you need access to Monitoring and Notifications .

The workflow for creating the connector includes a default policy when needed to provide permission for writing to the target service. If you're new to policies, see IAM Policies Overview.

Setting Up This Scenario

Setup is easy in the Console. Alternatively, you can use the Oracle Cloud Infrastructure CLI or API, which lets you execute the individual operations yourself.

This example walks through creating a connector that filters VCN flow log from Logging for rejected traffic and then moves this log data to a new metric in Monitoring.

For help with troubleshooting, see Troubleshooting Connectors and Troubleshooting Notifications.

  • This section walks through creating a connector and an alarm using the Console and then updating the topic created with the alarm.

    Note

    Another workflow for this scenario involves creating your topic and subscriptions first, then selecting this topic when you create your alarm.
    Task 1: Create the connector

    Task 1: Create the connector

    Note

    For complete descriptions of fields on the Create connector page, see Creating a Connector.

    On the Connectors list page, select Create connector. If you need help finding the list page, see Listing Connectors.

    1. Basic Connector Information

    Enter identifying information.

    • Connector name: Enter a user-friendly name for the new connector and an optional description. Avoid entering confidential information. Example: VCN Flow Log Errors
    • Select a compartment: Select the compartment to store the new connector in.

    Select Next.

    2. Configure Connector Source

    Select your VCN flow log.

    • Select source: Select Logging.
    • Configure source: Select the compartment, log group, and log for your VCN flow logs.
    • Log filter task: Filter the log data to rejected traffic.
      • Property: data.action
      • Operator: =
      • Value: REJECT

      If you're interested in rejected traffic for a particular port or address, add another filter. For example, select the property data.destinationPort or data.destinationAddress.

    Select Next.

    3. Configure Connector Task

    Select Next (skip this section).

    4. Configure Connector Target

    Enter the metric namespace and metric name that you want to use for the filtered log data.

    • Target: Select Monitoring.
    • Configure target: Enter the following information.
      • Compartment: Select the compartment where you want to store the metric data points corresponding to the VCN flow log data for rejected traffic.
      • Metric Namespace: vcnlogs
        Note

        When typing a new metric namespace, press ENTER to submit it.
      • Metric: rejectedtraffic
    • Create policy: Select to accept the default policy provided for the entered target configuration.

    Select Next.

    Connector Preview

    Review the connector configuration and then select Create.

    The creation process begins, and its progress is displayed. On completion, the connector's details page opens.

    Task 2: Create the alarm

    Task 2: Create the alarm

    This example walks through using the Console to create an alarm that sends a message when Monitoring receives metric data points corresponding to VCN flow log data for rejected traffic. During this process, you'll create a topic and an email subscription.

    1. Open the navigation menu  and select Observability & Management. Under Monitoring, select Alarm Definitions.

    2. Select Create Alarm.

    3. On the Create Alarm page, under Define alarm, set up your threshold: 

      • Metric description: 
        • Compartment: (select the compartment  specified in the previous task, where you are storing the metric data points corresponding to the VCN flow log data)
        • Metric namespace: (enter the metric namespace that you specified in the connector)
        • Metric name: (enter the metric name that you specified in the connector)
        • Interval: 1m
        • Statistic: Count

      • Trigger rule:

        • Operator: greater than
        • Value: 0
        • Trigger delay minutes: 1
    4. Under Notifications, Destinations, set up an email notification:
      • Destination Service:  Notifications Service
      • Compartment: (select the compartment  where you want to create the topic and associated subscriptions)

      • Topic: Select Create a topic

        Note

        If you already created your topic and email subscription, you can select that topic here instead of creating a new one.
        • Topic name: Rejected Traffic Topic
        • Subscription protocol: Email
        • Subscription Email: (type your email address here)

    5. Select Save alarm.

  • This section walks through creating the connector, topic, subscriptions, and alarm using the CLI.

    For information about using the API and signing requests, see REST API documentation and Security Credentials. For information about SDKs, see SDKs and the CLI.

    1. Create a connector: Open a command prompt and run the oci sch service-connector create command: 

      oci sch service-connector create --display-name
"<display_name>" --compartment-id <compartment_OCID> --source [<source_in_JSON>] --tasks [<tasks_in_JSON>] --target [<targets_in_JSON>]

    2. Create a topic: Open a command prompt and run the oci ons topic create command: 

      oci ons topic create --name "Alarm Topic" --compartment-id "<compartment-ocid>"

    3. To this topic, add a subscription referencing your email address. Open a command prompt and run the oci ons subscription create command:

      oci ons subscription create --compartment-id "<compartment-ocid>" --topic-id "<topic-ocid>" --protocol "EMAIL" --subscription-endpoint "john.smith@example.com"

    4. Create an alarm that defines the error threshold and references this topic as the destination: Open a command prompt and run the oci monitoring alarm create command: 

      oci monitoring alarm create --display-name "My Alarm" --compartment-id "<compartment-ocid>" --metric-compartment-id "<compartment-ocid>" --namespace "oci_computeagent" --query-text "<yourmetric>[1m].count() > 0" --severity "INFO" --destinations "<topic-ocid>" --is-enabled true

    For help with troubleshooting, see Troubleshooting Connectors and Troubleshooting Notifications.

  • This section walks through creating the connector, topic, subscription, and alarm using the API.

    For information about using the API and signing requests, see REST API documentation and Security Credentials. For information about SDKs, see SDKs and the CLI.

    Use the following operations:

    1. CreateServiceConnector: Create a connector.

      Example CreateServiceConnector request
      POST /20200909/serviceConnectors
Host: service-connector-hub.us-phoenix-1.oraclecloud.com
<authorization and other headers>
{
  "compartmentId": "<compartment_OCID>",
  "description": "My connector description",
  "displayName": "My Connector",
  "source": {
    "kind": "logging",
    "logSources": [
      {
        "compartmentId": "<compartment_OCID>",
        "logGroupId": "<log_group_OCID>",
        "logId": "<log_OCID>"
      }
    ]
  },
  "target": {
    "compartmentId": "<compartment_OCID>",
    "kind": "metrics",
    "metric": "<yourmetric>",
    "metricNamespace": "<yournamespace>"
  },
  "tasks": [
    {
      "condition": "data.action='REJECT'",
      "kind": "logRule"
    }
  ]
  }
}

    2. CreateTopic: Create a topic.

      Example CreateTopic request
      POST /20181201/topics
Host: notification.us-phoenix-1.oraclecloud.com
<authorization and other headers>
{
  "name": "Alarm Topic",
  "compartmentId": "<compartment_OCID>"
}

    3. CreateSubscription: To this topic, add a subscription referencing your email address.

      Example CreateSubscription request: Email
      POST /20181201/subscriptions
Host: notification.us-phoenix-1.oraclecloud.com
<authorization and other headers>
{
  "topicId": "<topic_OCID>",
  "compartmentId": "<compartment_OCID>",
  "protocol": "EMAIL",
  "endpoint": "john.smith@example.com"
}

    4. CreateAlarm: Create an alarm that defines the memory threshold and references this topic.

      Example CreateAlarm request
      POST /20180401/alarms
Host: telemetry.us-phoenix-1.oraclecloud.com
<authorization and other headers>
{
  "displayName": "My Alarm",
  "compartmentId": "<compartment_OCID>",
  "metricCompartmentId": "<compartment_OCID>",
  "namespace": "<yournamespace>",
  "query": "<yourmetric>[1m].count() > 0",
  "severity": "INFO",
  "destinations":
  [
    "<topic_OCID>"
  ],
  "isEnabled": true
}

    For help with troubleshooting, see Troubleshooting Connectors and Troubleshooting Notifications.