Note:

Move Logs from Oracle Cloud Infrastructure to IBM QRadar

Introduction

Oracle Cloud Infrastructure (OCI) is an Infrastructure-as-a-Service (IaaS) and Platform-as-a-Service (PaaS) trusted by large-scale enterprises. It offers a comprehensive array of managed services encompassing hosting, storage, networking, databases, and so on.

The OCI Observability and Management platform is designed to align with our customers preferences. Many have adopted established operational practices utilizing third-party observability tools. Our goal is to ensure seamless integration with these tools, empowering our customers to leverage their existing investments alongside OCI.

In this tutorial, we will walk you through how you can move logs from OCI to IBM QRadar.

Now, let us look at the high-level representation of the solution architecture as shown in the following image.

Architecture Diagram

OCI Connector Hub reads log data from OCI Logging and sends the logs to the OCI Streaming service. IBM QRadar has an integrated Kafka consumer that can connect with the OCI Streaming service to read this data.

Objectives

Prerequisites

Task 1: Configure the Logs to Capture

OCI Logging service is a highly scalable and fully managed single pane of glass for all the logs in your tenancy. OCI Logging provides access to logs from OCI resources. A log is a first-class OCI resource that stores and captures log events collected in a given context. A log group is a collection of logs stored in a compartment. Log groups are logical containers for logs. Use log groups to organize and streamline management of logs by applying Oracle Cloud Infrastructure Identity and Access Management (OCI IAM) policy or grouping logs for analysis.

To get started, enable a log for a resource. Services provide log categories for the different types of logs available for resources. For example, the OCI Object Storage service supports the following log categories for storage buckets: read and write access events. Read access events capture download events, while write access events capture write events. Each service can have different log categories for resources.

  1. Log in to the OCI Console, navigate to Observability & Management, Logging and Log Groups.

  2. Select your compartment, click Create Log Group and enter the following information.

    • Name: Enter QRadar_log_group.
    • Description (Optional): Enter the description.
    • Tags (Optional): Enter the tags.

  3. Click Create to create a new log group.

  4. Under Resources, click Logs.

  5. Click Create custom log or Enable service log as desired.

    For example, to enable write logs for an OCI Object Storage bucket, follow the steps:

    1. Click Enable Service Log.

    2. Select your resource compartment and enter Object Storage in the Search services.

    3. Click Enable Logs and select your OCI Object Storage bucket name in the Resource.

    4. Select log group (QRadar_log_group) created in Task 1.2 and Write Access Events in the Log Category. Optionally, enter QRadar_bucket_write as Log name.

    5. Click Enable to create your new OCI log.

Task 2: Create a Stream using OCI Streaming

OCI Streaming service is a real-time, serverless, Apache Kafka-compatible event streaming platform for developers and data scientists. It provides a fully managed, scalable, and durable solution for ingesting and consuming high-volume data streams in real-time such as logs. we can use OCI Streaming for any use case in which data is produced and processed continually and sequentially in a publish-subscribe messaging model.

  1. Go to the OCI Console, navigate to Analytics & AI, Messaging and Streaming.

  2. Click Create Stream to create stream.

  3. Enter the following information and click Create.

    • Name: Enter the stream name. For this tutorial, it is Qradar_Stream.
    • Stream Pool: Select existing stream pool or create a new one with public endpoint.
    • Retention (in hours): Enter the number of hours to retain messages in this stream.
    • Number of Partitions: Enter the number of partitions for the stream.
    • Total Write Rate and Total Read Rate: Enter based on the amount of data you need to process.

    You can start with default values for testing. For more information, see Partitioning a Stream.

Task 3: Set up an OCI Connector Hub

OCI Connector Hub orchestrates data movement between services in OCI. OCI Connector Hub provides a central place for describing, executing and monitoring data movements between services, such as OCI Logging, OCI Object Storage, OCI Streaming, OCI Logging Analytics, and OCI Monitoring. It can also trigger OCI Functions for lightweight data processing and OCI Notifications to set up alerts.

  1. Go to the OCI Console, navigate to Observability & Management, Logging and Connectors.

  2. Click Create Connector to create the connector.

  3. Enter the following information.

    • Name: Enter QRadar_SC.
    • Description (Optional): Enter the description.
    • Compartment: Select your compartment.
    • Source: Select Logging.
    • Target: Select Streaming.

  4. Under Configure Source Connection, select a Compartment name, Log Group, and Log (log group and log created in Task 1).

  5. If you also want to send Audit Logs, click +Another Log and select the same compartment while replacing _Audit as your log group.

  6. Under Configure target, select a Compartment, and Stream (stream created in Task 2).

  7. To accept default policies, click the Create link provided for each default policy. Default policies are offered for any authorization required for this connector to access source, task, and target services.

  8. Click Create.

Task 4: Set Up Access Control for IBM QRadar to Retrieve Logs

To allow IBM QRadar to access data from an OCI stream, create a user and grant stream-pull permissions for retrieving logs.

  1. Create an OCI user. For more information, see Managing Users.

  2. Create an OCI group named QRadar_User_Group and add the OCI user to the group. For more information, see Managing groups.

  3. Create the following OCI IAM policy.

    Allow group <QRadar_User_Group> to use stream-pull in compartment <compartment_of_stream>

Task 5: Configure IBM QRadar

  1. Log in to the IBM QRadar Console, click Admin and QRadar Log Source Management.

    QRadar Log Source Management

  2. Click New Log source and select Single Log Source.

    New Log source

    single log source

  3. Select Log Source Type as Universal DSM, Protocol Type as Apache Kafka and click Configure Log source parameters.

    Log Source Type

    Protocol Type

  4. In the Configure the Log Source Parameters window, enter parameters according to your requirements and environment and click Configure Protocol Parameters. This step is specific to your use case and self-explanatory.

    Configure log source parameters

  5. The parameters in the Configure Protocol Parameters section can be found in the OCI Console. Enter the following parameters and click Finish.

    1. Go to the OCI Console, navigate to Home, Streaming, Stream Pools, Stream Pool Details and click Kafka Connection Settings. You can find the Bootstrap server and username details. The password is the Auth token of the user.

      Kafka Connection Settings

    2. The Topic List is your Stream Name.

      OCI Stream

    3. Disable Use Client Authentication. When using SASL authentication without client authentication, a copy of the server certificate is required to be placed in /opt/qradar/conf/trusted_certificates/.

      To copy a certificate to the /opt/qradar/conf/trusted_certificates directory, choose one of the following options:

      a. Use SSH to log in to the QRadar Console or managed host and retrieve the certificate by typing the following command.

      /opt/qradar/bin/getcert.sh <FQDN of Streaming Endpoint>

      A certificate is downloaded from the specified host name or IP address and placed into the /opt/qradar/conf/trusted_certificates directory in the appropriate format.

      b. Alternatively, Use the following command to fetch the server certificate and add it to the location /opt/qradar/conf/trusted_certificates/.

      openssl s_client -showcerts -connect <bootstrap_server>:9092 < /dev/null | openssl x509 -outform DER > <certificate_name>.der

      Server Certificate

      configure protocol parameters

  6. Click Deploy changes for the changes to take effect.

    Deploy changes

  7. In QRadar log source Management, click view to check the log source status. The status should be OK and Connected: Waiting for Events….

    Log source status

    Log source summary

  8. In QRadar log source Management, click Events to view the logs ingested from OCI tenancy.

    Events

    Events

    Note: As per the feature description in the IBM QRadar Console, when Use As A Gateway Log Source feature is enabled, IBM QRadar processes the collected events through its traffic analysis engine, which automatically detects and assigns the log source name, often appearing as Custom Rule Engine-8::Hostname. When this feature is disabled, the events retain their original log source name, such as Oracle Cloud Infrastructure Logs. Ensure you filter for both log sources when verifying log ingestion from the OCI tenancy.

    Events

  9. After completing all the steps, if the logs do not appear in QRadar, you may need to take the following actions:

    1. Restart the ingress service (if feasible). Restarting the ingress service may help resolve the issue. However, consult your administrator or assess the potential impact on your environment before running the following command.

      systemctl restart ecs-ec-ingress

    2. Disable and re-enable the log source.

Next Steps

This tutorial has demonstrated the process of integrating OCI and IBM QRadar. On the Security Information and Event Management (SIEM) side, it is essential to define dashboards to capture critical metrics and configure alerts to trigger when predefined thresholds are exceeded. Additionally, defining specific queries is crucial for detecting malicious activities and identifying patterns within your OCI tenancy. These actions will further enhance your security posture and enable proactive monitoring of your cloud environment.

Acknowledgments

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.