Internal Sources
Oracle AI Data Platform supports ingestion from internal Oracle sources using built-in ingestion connectors. These connectors enable users to seamlessly extract data using Spark-based notebooks and integrate it into their workflows and data pipelines.
Ingestion connectors abstract the complexities of connection setup, providing optimized access patterns for both batch and near real-time ingestion from Oracle-native services.
AI Data Platform provides sample code templates in this ZIP file to support ingesting data from several internal sources using Spark in notebooks.
Table 14-1 Internal Sources
| Source | Access Type | Integration Method | Decription | External Catalog Support | Sample Code Available |
|---|---|---|---|---|---|
| Fusion | Extract Only | Preconfigured Spark Templates | Extracts data from Fusion SaaS applications via BICC into AI Data Platform tables or volumes. | No | Yes |
| REST Endpoints | Read Only | JDBC via Spark Notebook | Reads from APIs for ingesting semi-structured data like JSON. | No | Yes |
| MySQL HeatWave | Read Only | JDBC via Spark Notebook | Move data between AI Data Platform and MySQL HeatWave using JDBC. | No | Yes |
| Autonomous Data Warehouse (ADW) | Read/Write + Zero-Copy | JDBC or External Catalog | Ingest from or register ADW as an external catalog for querying data directly without duplication. | Yes | Yes |
| Autonomous Transaction Processing | Read/Write + Zero-Copy | JDBC or External Catalog | Ingest from or register ADW as an external catalog for querying data directly without duplication. | Yes | Yes |
| Oracle DB | Read/Write | JDBC or External Catalog | Supports data ingestion from on-prem or OCI Oracle Databases. | Yes | Yes |
| Exadata | Read/Write | JDBC or External Catalog | Access Exadata systems for high-performance reads and writes using JDBC. | No | Yes |
Table 14-2 Spark SQL to Oracle Database, Autonomous Database, Exadata Data Type mapping
| Spark SQL Type | Oracle Database, Autonomous Database, Exadata Data Type |
|---|---|
| ByteType | NUMBER(38,10) |
| ShortType | NUMBER(38,10) |
| IntegerType (INT) | NUMBER(38,10) |
| LongType | NUMBER(38,10) |
| FloatType | FLOAT(126) |
| DoubleType | NUMBER(38,10) |
| DecimalType(p,s) | NUMBER(p,s) |
| StringType | VARCHAR2(4000 CHAR) |
| BinaryType | BLOB |
| BooleanType | VARCHAR2(4000 CHAR) |
| DateType | DATE |
| TimestampType | TIMESTAMP(6) |
| ArrayType | VARCHAR2(4000 CHAR) |
| MapType | Not supported |
| StructType | VARCHAR2(4000 CHAR) |
| CalendarIntervalType | Supported if converted to String/VARCHAR2 |