Note:

This tutorial requires access to Oracle Cloud. To sign up for a free account, see Get started with Oracle Cloud Infrastructure Free Tier.
It uses example values for Oracle Cloud Infrastructure credentials, tenancy, and compartments. When completing your lab, substitute these values with ones specific to your cloud environment.

Ingest Oracle Cloud Infrastructure logs into Microsoft Azure Sentinel using OCI Functions

Introduction

Oracle Cloud Infrastructure (OCI) Functions is a fully managed, multi-tenant, highly scalable, on-demand, Functions-as-a-Service platform. It is built on enterprise grade Oracle Cloud Infrastructure and powered by the Fn Project open source engine.

Service Connector Hub is a cloud message bus platform that offers a single pane of glass for describing, executing, and monitoring interactions when moving data between Oracle Cloud Infrastructure services. Use the Service Connector Hub service to transfer data between services in Oracle Cloud Infrastructure.

By using the Service Connector and Functions services, you can connect the generation of logs from your OCI cloud environment with the automatic and scalable sending of all information security records from your environment to your instance of Security information and event management (SIEM) Microsoft Azure Sentinel.

Objective

Ingest OCI logs into Microsoft Azure Sentinel using OCI Functions.

Prerequisites

Access to an Oracle Cloud tenancy
A Virtual Cloud Network setup in your tenancy
All the policy setups for Logs, Streaming and Functions.

Task 1: Start the OCI Instance

Create a new OCI instance via the Web Console and activate any of its plugins by default, go to the Instances page and click Create Instance.
Fill in all the information required for instance creation.
Go to the end of the page and click Show advanced options.
Some tabs will appear on the screen. Click Oracle Cloud Agent and select the plugins you want to be active after instance creation is complete.
All plugins selected on this tab will be automatically activated once the instance is ready for use.
After selecting the plugins, click Create at the end of the page.

Task 2: Configure the OCI Logging settings

Task 2.1: Configure Windows Instance custom logs

After creating the Windows instance, you must configure your OCI environment to capture Event Viewer logs via Oracle Agent installed on the server.

Before you start, make sure the “Custom Logs” plugin is active on the server you want to collect logs from. In your OCI console, go to Compute, Instances.
Select the instance to be monitored.
Click on the Oracle Cloud Agent tab, and make sure the plugin named Custom Logs Monitoring is enabled.
- If the plugin is not enabled in your instance, enable it. After that, let’s configure the OCI logging so that Windows Event Viewer events can be captured by this agent.

Task 2.2: Configure the Dynamic Group of servers

In order to capture Event Viewer logs from one or more Windows servers, a Dynamic Group must be created. This group will contain all Windows instances that should have their logs captured.

To create the Dynamic Group, click on the main menu of your OCI console and go to Identity & Security, Identity.
From the Identity menu, access the *Domains” submenu.
Click on your Identity Domain (in this tutorial example, we use Default).
Now, inside the identity domain, click on the Dynamic groups menu so that we can create the Dynamic Group that will contain all the Windows servers to be monitored. Once this is done, click Create dynamic group.
A new page will open. Fill in the information regarding the creation of the Dynamic Group (name and description) and, in Matching rules, click on the Rule builder link to be able to create your rule for selecting instances that will be grouped in this Dynamic Group.
The example rule that we will create will group all existing servers in our Compartment. To do this, just capture the Compartment ID and insert it in the Value field as shown in the image below.
- For more information about creating Dynamic Groups, see the official documentation: Creating Dynamic Groups and Policies to Access Other Oracle Cloud Infrastructure Resources from Notebook Sessions.
- When the creation is finished, a page containing the Dynamic Group information will be available.
- The Dynamic Group is created and ready to be used to centralize the logs of your OCI environment.

Note: You can use advanced filters and create Dynamic Groups to only capture instances with certain Tags. For this, we recommend reading the official OCI documentation at: Writing Matching Rules to Define Dynamic Groups.

Task 2.3: Create a Logging Group

You will also need to create a logging group to be able to concentrate all the logs sent by the agents installed on the Windows servers in your environment.

Click on the main menu of your OCI console and go to Observability & Management, Log Groups.
When accessing the service page, click Create Log Group.
Inside the log group creation page, fill in the requested information and click Create.

Once this step is finished, let’s start creating the Event Viewer log capture by Oracle Cloud Agent installed in the OCI instance.

Task 2.4: Configure OCI Logging

Once the instance, the Dynamic Group and the Log Group are configured, go to Observability & Management, and click Logging.
Within the OCI Logging page, click Agent Configurations, so that we can create the configuration that will allow the sending of Windows Event Viewer logs to OCI Logging. Once inside the agent configuration page, click Create agent config.
Fill in all the necessary fields on the page and select the Dynamic Group you created in the previous step.
- When filling out these initial items, scroll down the page and fill in the Agent Configuration fields, informing that you want to capture the “Windows event log” of each of the available channels (Application, Security and System). Here you will need to repeat the process of creating log inputs for each of the channels.
- Once the configuration is finished, your agent config page will be available and will look like this.
  
  Note: The agent config above was created to capture both Event Viewer logs and logs generated in the /var/log/messages file on Linux servers.
- Once your Agent Config is in Active status, wait approximately 5 minutes until the first log collection can be done. To view the logs of the monitored instances being collected, click on the Explore log menu as shown in the image below.

Task 3: Note the Microsoft Azure Sentinel details

After setting up the centralized logging environment in your OCI region, go to your Microsoft Azure subscription and collect the following information:
- Workspace ID: The Log Analytics Workspace ID used by Microsoft Azure Sentinel within your Microsoft Azure Subscription.
- Shared Key: Your Log Analytics Workspace access key.
These values can be found within the Agents Management menu as shown in the image below at your Azure Subscription:
Note down these values.

Important: It is not the scope of this documentation to provide guidance on creating and configuring Microsoft Azure Sentinel within your Microsoft Azure subscription. To do so, access the official product documentation on the Microsoft website.

Task 4: Create your OCI Vault

Now that we already have the Microsoft Azure Log Analytics Workspace data belonging to SIEM Sentinel, let’s create an OCI Vault to store the secret value copied in the previous task. This secret must be stored in the OCI Vault so you don’t need to insert it inside the OCI Functions code that we will create in the next step.

To create your OCI Vault, go to the main menu of your OCI console and go to Identity & Security, Vault.
Once the Vault page is loaded, click Create Vault, select the desired compartment and name your Vault.
After creating the Vault, you will need to create your Master Encryption Key. This key will be used to encrypt all secrets in your Vault. To do this, click Create Key as shown in the image below.
When you finish creating your Master Key, click on the Secrets menu and create the Microsoft Azure secret that you noted in the previous step.
Fill the necessary fields to create your Secret in the Vault with the primary or secondary key data from your Microsoft Azure Sentinel Log Analytics Workspace as shown in Task 3.
The other items (Sentinel Workspace ID and the name of the table where the logs will be stored in Sentinel) may or may not be inserted in the Vault. Here, let’s insert them inside the code of our Function.

Important: Take note of the OCID of the Secret you create within the Vault.

Task 5: Create and configure OCI Functions

After setting up the centralized logging environment in your OCI region, the Vault and your Azure environment are ready, it’s time to create the OCI Function that will send the Custom Logs and Audit Logs from your OCI region to Microsoft Azure Sentinel.

Task 5.1: Create the OCI Function

Go to the main menu of your OCI console and go to Developer Services, Functions.
Once the Functions page is loaded, click Create application.
Fill in the required fields and click the Create.
After creation, note that your Function will not be protected by any network security group. You must add it to a network security group.
Important Points:
1. As your Function will access the public Microsoft Azure endpoint for sending logs to Microsoft Azure Sentinel, this Function must be created in a subnet that has access to the Internet.
2. Also validate your VNET Security Lists to ensure that none of them prevent your OCI Function from accessing the Internet.

Task 5.2: Configure OCI Functions and Startup

Once the OCI Function has been created, you will see on the screen the commands necessary for configuring the function in the environment. Here, as an example, we will follow the step by step via Cloud Shell, but it is also possible to carry out these settings directly on your equipment.

Open Cloud Shell directly in the web console of the OCI region to be configured (in this tutorial, the region is Brazil East).
Follow the steps indicated on your Function page and execute the requested commands inside your Cloud Shell terminal.
The first 3 commands to be executed do not require changes, however, in the fourth command you will need to provide the name of the repository where the application image will be stored. In our example, the name entered was repo-ocilog-to-azuresiem.

$ fn update context registry gru.ocir.io/xxxxxxxxxa/repo-ocilog-to-azuresiem
Generate an Auth Token within your Oracle Cloud Infrastructure Identity and Access Management (OCI IAM) profile. The Auth Token is the access credential that will allow you to interact with the OCI repository where your Function’s Docker image will be stored. Click on your user icon in the OCI Web console and then click My profile.
Once inside your user page, scroll down and click Auth tokens.
Within the Auth Tokens section, click Generate Token.
Fill in the name for your Auth Token and click Generate token.

Important: Write down this token and store it in a safe place. This token is your access password to the OCI Registry where your Function images will be stored.
Return to the Functions page by accessing the main menu of the OCI web console and accessing Developer Services, Functions.
On the Functions page, access your Function created in the previous steps.
Click on the Getting started menu and continue step by step after the Auth Token creation step.
Run the command docker login and as password use the string you just copied from the creation of the Auth Token in the last step executed.
Now in the next steps we will create, deploy and invoke the Function. To initialize the function in your Cloud Shell, change the command suggested in the example shown on the screen. This change is necessary because the Runtime Environment we are going to use is in Python (and not in Java as shown in the hello-world example suggested by creating the Function).

$ fn init --runtime python func-ocilog-to-azuresiem

Task 5.2.1: Writing the source code of your OCI Function

Once the init command is executed, access the newly created Function directory.

$ cd func-ocilog-to-azuresiem/
- Note that inside the directory you just accessed there is a Python file (func.py), a YAML file (func.yaml) and a requirements file (requirements.txt). We will need to edit two of these files.

Edit the func.py file, delete all content and insert the below content into it.

import io
import json
import requests
import datetime
import hashlib
import hmac
import base64
import logging
import oci
import base64
import sys
from fdk import response

# Global variable
logger = logging.getLogger()

# Get Resource Principal Credentials
# OCI Functions get permission to read vault from OCI IAM
signer = oci.auth.signers.get_resource_principals_signer()

# Retrieve secret from OCI Vault
def get_binary_secret(secret_ocid):
    try:
        client = oci.secrets.SecretsClient({}, signer=signer)
        secret_content = client.get_secret_bundle(secret_ocid).data.secret_bundle_content.content.encode('utf-8')
        secret_content_decoded = base64.b64decode(secret_content).decode('UTF-8')
    except Exception as ex:
        logger.critical("ERROR: failed to retrieve the secret content from OCI Vault. ", ex)
        raise
    return secret_content_decoded

# Build the API signature
def  build_signature(customer_id, shared_key, date, content_length, method, content_type, resource):
    x_headers =  'x-ms-date:'  + date
    string_to_hash = method +  "\n"  + str(content_length) +  "\n"  + content_type +  "\n"  + x_headers +  "\n"  + resource
    bytes_to_hash = bytes(string_to_hash, encoding= "utf-8" )
    decoded_key = base64.b64decode(shared_key)
    encoded_hash = base64.b64encode(hmac.new(decoded_key, bytes_to_hash, digestmod=hashlib.sha256).digest()).decode()
    authorization =  "SharedKey {}:{}" .format(customer_id,encoded_hash)
    return  authorization

# Build and send a request to the POST API (Microsoft Azure Sentinel public endpoint)
def  post_data(customer_id, shared_key, body, log_type, logger):
    method =  'POST'
    content_type =  'application/json'
    resource =  '/api/logs'
    rfc1123date = datetime.datetime.utcnow().strftime( '%a, %d %b %Y %H:%M:%S GMT' )
    content_length = len(body)
    signature = build_signature(customer_id, shared_key, rfc1123date, content_length, method, content_type, resource)
    uri =  'https://'  + customer_id +  '.ods.opinsights.azure.com'  + resource +  '?api-version=2016-04-01'

    headers = {
        'content-type' : content_type,
        'Authorization' : signature,
        'Log-Type' : log_type,
        'x-ms-date' : rfc1123date
    }

    response = requests.post(uri,data=body, headers=headers)
    if (response.status_code >= 200 and  response.status_code <=  299 ):
        logger.critical( '===> Upload to Microsoft Azure Sentinel was accepted %s', content_length )
    else:
        logger.critical( "Error during upload.  Response code: {}" .format(response.status_code))

####
## PART 3
####

#
# Entrypoint and initialization
#
def handler(ctx, data: io.BytesIO=None):
    logger.info("OCI Function - Handler function init fired.")

    # Update the customer ID to your Log Analytics workspace ID
    customer_id =  ‘<PUT_HERE_YOUR_AZURE_LOG_ANALYTICS_WORKSPACE_ID>'

    # For the shared key, use either the primary or the secondary Connected Sources client authentication key
    shared_key_OCID = "<PUT_HERE_THE_OCID_OF_YOUR_OCI_VAULT_SHARED_KEY_SECRET>"
    shared_key = get_binary_secret(shared_key_OCID)

    # The log type is the name of the event that is being submitted
    log_type =  '<SENTINEL_TABLE_NAME>'

    try:
        log_body = data.getvalue()
        post_data(customer_id, shared_key, log_body, log_type, logger)
    except  Exception  as  err:
        logger.error( "Error in main process: {}" .format(str(err)))
    return  response.Response(
        ctx,
        response_data=json.dumps({ "status" :  "Success" }),
        headers={ "Content-Type" :  "application/json" }
    )

This is the Python script that will connect to the Azure SIEM and send the OCI logs to the remote endpoint.
In addition to copying and pasting this script, you will need to change 3 fields within it
1. _customer_id: Log Analytics Workspace ID within your Azure subscription, in the Agents Management menu.
2. _shared_key: Primary or Secondary key that can also be obtained from the Agents Management menu.
3. _log_type: Name of the Log Analytics Workspace that SIEM Sentinel uses. All logs coming into this Workspace are analyzed by Sentinel. Here in our example, we´ve used OCILogging.
Below is an example print of the screen where you can get the information above.

Note: It is outside the scope of this document to configure Log Analytics Workspace within your Azure Subscription. For this, look for more information in the official Microsoft documentation.

After changing the func.py file, it’s time to edit the requirements.txt file. We will insert a requirement where the OCI Functions package manager will install the necessary Python dependencies for our script to work correctly. For this case, insert the word requests in the last line of the file:
Once that’s done, run your Function’s deploy step.

$ fn -v deploy --app app-ocilog-to-azuresiem

After a few minutes, the configuration of both the Function and the creation of the Docker image will be successfully completed.

Required Python modules

Task 5.2.2: Provide the required permissions for the OCI Function

For the OCI Function to run smoothly, it will need permissions to read the Azure secret inside the Vault. For this, we will need to create a Dynamic Group for the Function we created and, for this Dynamic Group, we will give the necessary permission (least-privilege) so that our code can read the Vault and capture the shared key from Azure in a secure way.

So, in the same way we did in item 2.2 above, where we created a Dynamic Group to aggregate the servers that will collect the logs, retrace the steps and create a Dynamic Group to include our Function.

During the creation of the Dynamic Group, you will need a Matching Rule that will be used to capture your Function. Here, in our example, we will create a Matching Rule that will list all the Functions of the Compartment that we are using exclusively for this tutorial.

ALL {resource.type = 'fnfunc', resource.compartment.id = 'ocid1.compartment.oc1..aaaaaxxxxgjxxxxglxxxxiqxxxxjdxxxxnxxxxmxxxxm3xxxxmoxxxxsxxx'}

Capture your Compartment’s OCID and insert it into your Matching Rule and create your Dynamic Group.
After creating your Dynamic Group, go to Identity & Security, Policies to create the policy that will allow the Function to access the Vault in read mode.
- Once the Dynamic Group is created, create the policy that will allow the Function to access the Vault.
Use the policy below to grant the minimum required access to the Function to run your Vault credential reading activity.

Allow dynamic-group <FUNCTION\_DYNAMIC\_GROUP\_NAME> to read secret-bundles in compartment <COMPARTMENT\_NAME>

Task 5.2.3: Run your OCI Function

Run your function to validate that it is working correctly.

$ fn -v deploy --app app-ocilog-to-azuresiem

Now, to finalize, we need to configure the OCI Service Connectors to capture all OCI logs from your region and send them to Azure SIEM using the Function we just created.

Task 6: Configure the Service Connectors

In order to be able to send the logs of the regions of your OCI tenant for the analysis of the Microsoft Azure Sentinel, it is necessary to configure the Service Connectors.

Basically this service will capture the OCI logs that we have already configured and send it to our Function which, in turn, will forward all the logs to the Azure subscription.

Go to the main menu of your OCI Console and then go to Observability & Management, **Service Connectors.
Once inside the Service Connectors, click Create Service Connector.
Fill in the required name and description fields, select the compartment where you want to create your Service Connector.
Now, select the source of data that will be transmitted via the Service Connector. Here, use Logging. And, as target, that is, where the data will be transitioned, use Functions.
To finalize the configuration, select in Configure source, which logs will be sent to OCI Functions. In this step we must select the \_Audit log and, adding one more log, we will select the log group that we created in the initial steps of this procedure.
- If you want to add another log group, click + Another log.
Having finished this step, we now need to define the Target parameters, that is, our OCI Function that will send the data to the Azure subscription. To do this, select your compartment where the function is, select the App and, after that, the name of your function. Then, just click Create as shown in the image below.

The process is finished. At this point, just wait until the logs start to be sent to Microsoft Azure automatically.

You can monitor the operation of the service Connector created by the “Metrics” menu.

Monitor your Service Connector metrics

Task 7: Read the logs at Microsoft Azure Sentinel

After a few moments, logs from all servers will be sent to the Microsoft Azure subscription as shown in the image below.
According to your Microsoft Azure Sentinel configuration, incidents will start showing up in the interface as the SIEM rules are triggered.
- Incident opened for Brazil East region server (we’ve created some custom rule called “[Lab] Regex Incident” to generate our example incidents).
- Open incident for Us-Phoenix region server.

Acknowledgments

Author - Rodrigo Pace de Barros (Oracle LAD A-Team Cloud Security Solution Engineer)

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.

Title and Copyright Information

Ingest Oracle Cloud Infrastructure logs into Microsoft Azure Sentinel using OCI Functions

F80858-01

April 2023

Ingest Oracle Cloud Infrastructure logs into Microsoft Azure Sentinel using OCI Functions

Introduction

Objective

Prerequisites

Task 1: Start the OCI Instance

Task 2: Configure the OCI Logging settings

Task 2.1: Configure Windows Instance custom logs

Task 2.2: Configure the Dynamic Group of servers

Task 2.3: Create a Logging Group

Task 2.4: Configure OCI Logging

Task 3: Note the Microsoft Azure Sentinel details

Task 4: Create your OCI Vault

Task 5: Create and configure OCI Functions

Task 5.1: Create the OCI Function

Task 5.2: Configure OCI Functions and Startup

Task 5.2.1: Writing the source code of your OCI Function

Task 5.2.2: Provide the required permissions for the OCI Function

Task 5.2.3: Run your OCI Function

Task 6: Configure the Service Connectors

Task 7: Read the logs at Microsoft Azure Sentinel

Related Links

Acknowledgments

More Learning Resources