About Streaming OCI Logs Using OCI Streaming to LogRhythm Kafka Beat

Stream Oracle Cloud Infrastructure (OCI) logging data to LogRhythm SIEM (security information and event management) to leverage the capabilities of a SIEM system such as event correlation, security incident management, and compliance requirements. In this solution, we walkthrough how to integrate OCI logs with LogRhythm SIEM using OCI Streaming and Kafka Beat Open Collector.

Before You Begin

Before you begin to integrate OCI Logging with LogRhythm SIEM, validate the supported versions to use with Kafka Open Collector and Kafka Beat for LogRhythm in the Open Collector Installation and User Guide.

Review Documentation

Before you begin, ensure you are familiar with the following resources.

OCI Connector Hub:

• Creating a Stream
• Creating a Stream Pool
• Creating a Connector with a Logging Source
• Overview of Connector Hub

LogRhythm:

• LogRhythm Open Collector Installation and User Guide
• LogRhythm Supported Log Source Types
• Initialize Kafka Beat

Architecture

In a third-party SIEM reference architecture, OCI Logging captures logs from different sources such as audit logs, service logs, and custom logs. OCI Logging is connected to the stream with a service connector hub that writes the logs inside the OCI Streaming service.

OCI Streaming has built-in support for Kafka connect using Connect Harness. Kafka Connect uses sink and source connectors to move data from Kafka topics or send data to Kafka topics.

It can then interface with a third-party SIEM platform, such as LogRhythm, which collects the streamed data for further analysis. We use LogRhythm open collector for Kafka beat to move data from Oracle Streaming to LogRhythm.

The following diagram illustrates the workflow of this reference architecture.

Description of the illustration stream-oci-logs-logrhythm-kafka-beat.png

stream-oci-logs-logrhythm-kafka-beat.zip

The flow of the architecture resembles:

1. OCI Connector Hub reads log from OCI Logging.
2. OCI Connector Hub writes log data to OCI Streaming.
3. Kafka Beat Open Collector reads data from OCI Streaming.
4. LogRhythm parses and processes data for operational activities.

This architecture supports the following components:

• Logging
  Logging is a highly scalable and fully managed service that provides access to the following types of logs from your resources in the cloud:

  • Audit logs: Logs related to events emitted by the Audit service.
  • Service logs: Logs emitted by individual services such as API Gateway, Events, Functions, Load Balancing, Object Storage, and VCN flow logs.
  • Custom logs: Logs that contain diagnostic information from custom applications, other cloud providers, or an on-premises environment.
• Service connectors

  Oracle Cloud Infrastructure Connector Hub is a cloud message bus platform that orchestrates data movement between services in OCI. You can use service connectors to move data from a source service to a target service. Service connectors also enable you to optionally specify a task (such as a function) to perform on the data before it is delivered to the target service.

  You can use Oracle Cloud Infrastructure Connector Hub to quickly build a logging aggregation framework for security information and event management (SIEM) systems.

• Streaming

  Oracle Cloud Infrastructure Streaming provides a fully managed, scalable, and durable storage solution for ingesting continuous, high-volume streams of data that you can consume and process in real time. You can use Streaming for ingesting high-volume data, such as application logs, operational telemetry, web click-stream data; or for other use cases where data is produced and processed continually and sequentially in a publish-subscribe messaging model.

About Required Services and Roles

This solution requires the following services:

• OCI Logging
• OCI Connector Hub
• LogRhythm Open Collector
• LogRhythm Kafka Beat

These are the roles needed for each service.

Service Name: Resource Type	Required to...
OCI Logging: log-groups	Create and manage log groups and log objects.
OCI Connector Hub: ConnectorUsers	Configure and manage connectors.
LogRhythm Open Collector	Install and configure open collector for LogRhythm.
LogRhythm Kafka Beat	Configure and initialize Kafka Beat.

• OCI Logging Policies
• OCI Connector Hub Policies

See Oracle Products, Solutions, and Services to get what you need. See Initialize Kafka Beat to learn about LogRhythm's required roles.