Implement SIEM Using a Serverless Pipeline
A security information and event management (SIEM) system is a critical operations tool to manage the security of your cloud resources. Detect, prevent, and respond to threats to your cloud deployments by setting up an efficient SIEM system that analyzes logs generated by the Oracle Cloud Infrastructure Audit service.
Architecture
This architecture leverages the Audit, Events, Streaming, Notifications, and Functions services to deploy a low-cost, serverless, and scalable framework to process and export audit data to a SIEM system.
The Audit service automatically records calls to all the supported public API endpoints, as log events. The Events and Notifications services enable you to track changes in the state of resources in your tenancy. Oracle Functions is a serverless platform that this architecture uses to automate the processing of streamed data.
The following diagram shows the process flow:
Description of the illustration siem-taskflow-oci-png.png
The following diagram illustrates the architecture:
Description of the illustration siem-infra-oci-png.png
- Region
An Oracle Cloud Infrastructure region is a localized geographic area that contains one or more data centers, called availability domains. Regions are independent of other regions, and vast distances can separate them (across countries or even continents).
- Virtual cloud network (VCN) and subnets
A VCN is a customizable, software-defined network that you set up in an Oracle Cloud Infrastructure region. Like traditional data center networks, VCNs give you complete control over your network environment. A VCN can have multiple non-overlapping CIDR blocks that you can change after you create the VCN. You can segment a VCN into subnets, which can be scoped to a region or to an availability domain. Each subnet consists of a contiguous range of addresses that don't overlap with the other subnets in the VCN. You can change the size of a subnet after creation. A subnet can be public or private.
The functions in this architecture are deployed in a private subnet.
- Oracle services network
The Oracle services network (OSN) is a conceptual network in Oracle Cloud Infrastructure that is reserved for Oracle services. These services have public IP addresses that you can reach over the internet. Hosts outside Oracle Cloud can access the OSN privately by using Oracle Cloud Infrastructure FastConnect or VPN Connect. Hosts in your VCNs can access the OSN privately through a service gateway.
- Service gateway
The service gateway provides access from a VCN to other services, such as Oracle Cloud Infrastructure Object Storage. The traffic from the VCN to the Oracle service travels over the Oracle network fabric and never traverses the internet.
- Network address translation (NAT) gateway
A NAT gateway enables private resources in a VCN to access hosts on the internet, without exposing those resources to incoming internet connections.
- Schedule-based autoscaling
A schedule-based autoscaling policy triggers an event at configurable intervals, using
cronexpressions. In this architecture, scheduled-based autoscaling policies are used to trigger scaling events. - Events
Oracle Cloud Infrastructure services emit events, which are structured messages that describe the changes in resources. Events are emitted for create, read, update, or delete (CRUD) operations, resource lifecycle state changes, and system events that affect cloud resources.
In this architecture, the Events service is used to respond to the scaling action event that the scheduled autoscaling policy generates.
- Functions
Oracle Functions is a fully managed, multitenant, highly scalable, on-demand, Functions-as-a-Service (FaaS) platform. It is powered by the Fn Project open source engine. Functions enable you to deploy your code, and either call it directly or trigger it in response to events. Oracle Functions uses Docker containers hosted in Oracle Cloud Infrastructure Registry.
- Notifications
The Oracle Cloud Infrastructure Notifications service broadcasts messages to distributed components through a publish-subscribe pattern, delivering secure, highly reliable, low latency, and durable messages for applications hosted on Oracle Cloud Infrastructure.
- Streaming
Oracle Cloud Infrastructure Streaming provides a fully managed, scalable, and durable storage solution for ingesting continuous, high-volume streams of data that you can consume and process in real time. You can use Streaming for ingesting high-volume data, such as application logs, operational telemetry, web click-stream data; or for other use cases where data is produced and processed continually and sequentially in a publish-subscribe messaging model.
Recommendations
- VCN
When you create a VCN, determine the number of CIDR blocks required and the size of each block based on the number of resources that you plan to attach to subnets in the VCN. Use CIDR blocks that are within the standard private IP address space.
Select CIDR blocks that don't overlap with any other network (in Oracle Cloud Infrastructure, your on-premises data center, or another cloud provider) to which you intend to set up private connections.
After you create a VCN, you can change, add, and remove its CIDR blocks.
- Subnets
When you design the subnets, consider your traffic flow and security requirements. Attach all the resources within a specific tier or role to the same subnet, which can serve as a security boundary.
Deploy all the functions in a private regional subnet.
- Security lists
Use security lists to define ingress and egress rules that apply to the entire subnet.
- Route tablesDefine the following routing rules:
- Route traffic to services such as Streaming and Notifications through the service gateway.
- Route traffic from the private subnet bound for the public internet through the NAT gateway.
- Functions
Oracle Functions supports Python, Java, Node, Go, and Ruby. Use a programming language of your choice. Always enable logging to allow debugging if an error occurs.
- Notifications
The Notifications service is used to decouple functions from one another in the pipeline. Instead of invoking a function directly from another function, we use notifications to trigger the subsequent functions. This approach allows each function to end without waiting for the subsequent functions to be completed.
- Streaming
Streaming allows publishing of messages in an append-only format using a distributed commit log. Functions are stateless; but the Streaming service allows us to store the state reliably and preserve the context as the functions run and end. Use multiple partitions if your functions exceed the 1-Mbps write limit and the 2-Mbps read limit per partition.
Considerations
When implementing a SIEM system using this architecture, consider your requirements for the following parameters:
- Performance
The architecture scales based on the number of audit events generated by the tenancy. Oracle Functions is a highly scalable service.
Streaming, too, is a highly scalable service, and is used as a temporary conduit to store audit event information retrieved using the Audit API. It also acts as a load balancer.
- Extensibility
The solution needs to scale as the tenancy grows. By using functions to dynamically retrieve the list of regions and compartments, you ensure that new compartments and regions are picked up as they’re added and configured.
- Security
The functions are hosted in a private subnet. The cloud native services such as Streaming and Notifications are accessed through a service gateway. The internet access required to export data to Splunk happens over a secure TLS connection through a NAT gateway, making the entire architecture secure. The Streaming service, which stores the audit logs, is encrypted using Oracle-managed keys, making the data at rest secure. All the API requests sent by the functions are made using dynamic groups and instance principals.
- Availability
Oracle ensures high availability of the Functions, Streaming, and Notifications services, which are cloud native and fully managed.
- Cost
Every component used in the architecture is a pay-per-use component. None of the components are charged unless used, even after being provisioned. The billing rates of all the components are tied to usage. For example, the billing for Streaming is based on the amount of data processed, and the billing for Functions is based on the number of invocations.