This architecture combine the abilities of a data lake and a data warehouse to process and analyze streaming data from production workloads in a highly available and redundant environment. A linear sequence of steps spans the diagram:

1. Discover
2. Ingest
3. Transform
4. Curate
5. Analyze, learn, and predict
6. Measure and act

The following components apply to the entire Oracle Cloud Infrastructure region:

• Security, Identity, and Access Management
• Governance
• Discovery Lab and Sandbox

The region is divided into functional layers that house physical or functional components:

• Data Sources layer: Data sources can and will be anything. Those shown include Enterprise Applications, Devices, End Users, Event Data, Sensors, and Any Digital Asset that makes up Social Voice.

• Data Refinery layer: Streaming event data moves into a block labeled Streaming Ingest (Kafka, OCI Streaming). Streaming data moves to and from the ingest block to an analytics block that spans both the Data Persistence Platform layer and the Access and Interpretation layer.

• Data Persistence Platform layer: Ingested data enters a processing block that is labeled Streaming Analytics (GoldenGate Stream Analytics) in the Data Persistence Platform layer and that includes native Kafka, Spark, ATP, File Storage in the Access and Interpretation layer.

  Data moves between the analytics block and the blocks labeled Serving (Oracle Autonomous Data Warehouse), Streaming Processing (Spark), and Cloud Storage (Object Storage). Data also moves between the Streaming Processing block and the Serving, and Cloud Storage blocks.

  Dotted lines show data optionally passing from the Serving and Cloud Storage blocks to a block labeled Governance (Data Catalog).

• Access and Interpretation layer: Events data moves from the Streaming Analytics block to the Data Consumers layer. A dotted line shows processed streaming data optionally passing from the Serving block into a block labeled Visualize/Learn (Oracle Analytics Cloud) and from there to the Data Consumers layer.

• Data Consumers layer: Analytics data, labeled Information, passes on to People and Partners. Event data, labeled Events, passes on to a group labeled Things.