This diagram shows the conversion of enterprise applications, flat file, events and sensors data from multiple sources through components in an Oracle Cloud Infrastructure region to usable insights for data consumers.

The following components apply to the entire Oracle Cloud Infrastructure region:

• Data Governance (Data Catalog) as part of a general Governance layer
• Data Security (Data Safe, Audit, Logging, Vault) as part of both a general Governance layer and an Infrastructure, Network Security, Identity and Access Management (IAM) layer

The region is divided into functional layers that house physical or functional components:

• Data Sources layer: Data share provider, devices, end users, events, sensors, files, any data source to any type, and enterprise applications. Metadata from the enterprise applications databases is harvested and flows into the block Governance (Data Catalog).

• Connect, Ingest, Transform layer: Data is refined in batch, API, and streaming ingest, leveraging different services depending on the use case.

  A block labeled Batch Ingest (OCI Data Integration, Data Integrator, Data Transforms) handles data that is ingested and transformed in batches or micro batches. Raw data is stored in object storage.

  A block labeled API-based Ingest (OIC, ORDS, API Gateway, Functions) handles data that is ingested APIs. Raw data is stored in object storage.

  A block labeled Real Time Ingest (GoldenGate Service/OGG Marketplace) handles data that is ingested in near real time. Data ingested in real time is also processed and refined with the capabilities provided by the batch ingest component. Raw and refined application data pass to the Persist, Curate, Create layer along separate paths. Raw data is stored in object storage.

  A block labeled Bulk Transfer (FastConnect, Data Transfer, Storage Gateway, CLI, SDK, API) handles bulk (file) data which then passes on to the Cloud Storage block.

  Streaming data is ingested by a block labeled Streaming Ingest (OCI Streaming, Service Connector Hub, Kafka Connect), which then flows into Cloud Storage (Object Storage), Streaming Processing (OCI Golden Gate, Streaming Analytics) or Streaming Analytics (Golden Gate Stream Analytics).

• Persist, Curate, Create: Data is persisted in the lakehouse in Oracle Autonomous Data Warehouse or Object Storage or both. It is accessed by APIs and is used for analytics, visualization, and data science.

  Processed data and data from a data share provide moves into a block within the lakehouse labeled Serving (Oracle Autonomous Data Warehouse). The lakehouse also includes Cloud/Storage/Date Lake (Object Storage).

  Additional processing is provided by Batch Processing (Data Flow, OCI Data Integration, Data Integrator), by Open Source Ecosystem (Big Data Service), and Streaming Processing (OCI GoldenGate Stream Analytics, OCI Data Flow). Metadata passes to the block labeled Data Governance (Data Catalog).

• Analyze, Learn, Predict layer: Lakehouse data is used by Oracle Analytics Cloud for analytics and visualization, by Data Science and AI for leaning and predictions, by AI and Generative AI services, by Data Labeling for data enrichment, by OCI Search to provide results for people and partners and to applictions and IoT consumers that use an API.

  Lakehouse and streaming data can use Reverse ETL Writeback to provide data to applications and IoT consumers.

• Measure, Act layer: Data Share Recipients use shared data, people and partners use analytics data while applications and IoT consumers use refined data.