Iceberg Tables

Iceberg is an open table format for managing large analytical datasets stored in distributed object stores. It is not a folder convention. It separates data storage from metadata management, enabling efficient schema evolution, snapshot-based transactions, and interoperability across multiple query engines.

Iceberg tables store data in immutable data files, typically in Parquet format, and
metadata separately using metadata JSON files, manifest files, and manifest list files. Each successful transaction creates a new snapshot that references the current set of data files.

This snapshot-based architecture provides transactional consistency, supports concurrent access by multiple query engines, and enables schema evolution and time travel.

Query planning begins with table state and snapshot metadata, then selects manifests and data files. Performance therefore depends on metadata health and physical files.

Iceberg Scan Planning

Iceberg scan-planning flow: Catalog or metadata URI current snapshot manifest list relevant manifests relevant data files projected columns and row groups.

Iceberg snapshots identify a table state. Manifest lists summarize manifests, and manifests list data or delete files with partition data and file metrics. A query engine can skip metadata and files that cannot satisfy the predicate. See Apache Iceberg specification.

Layer	Keep efficient by	Failure pattern
Snapshot metadata	Retaining only required history according to governance and recovery policy.	Metadata grows continuously and planning or maintenance becomes slower.
Manifest lists and manifests	Keeping useful partition and column bounds and using only Oracle- supported maintenance documented for the access path.	Many manifests must be opened even for selective filters.
Data files	Avoiding small-file proliferation, preserving useful partition boundaries, and producing quality Parquet metadata.	High file-open overhead and weak parallel efficiency.
Delete files	Using only delete-file modes supported by the Oracle release and access path.	Extra merge work or query failure when a reader does not support the delete-file mode.

Note - Format capability is not Oracle feature support. The Iceberg specification supports partition evolution, schema evolution, snapshots, delete files, and time travel. A specific Oracle release and access path may support only a subset.
For supported Iceberg read paths and Iceberg operations, see Chapter 4.

Partition and Schema Evolution

An Iceberg partition spec defines transforms from source columns to partition values. Queries filter source columns; the engine derives predicates for each applicable partition spec. When the spec evolves, old files retain the old spec and new files use the new one.

Partition specs are metadata, not folder contracts Example - If new writes evolve from day(order_date) to month(order_date) , applications continue filtering order_date . Older files retain their earlier partition spec and new files use the new spec. Oracle behavior still depends on the release and access path.

Non-Iceberg folders	Iceberg partition specs
Pruning depends on path conventions known to the external table.	Partition fields, transforms, and spec IDs are stored in Iceberg metadata and manifests.
Changing layout usually/ requires changing paths, table metadata, or both.	Partition evolution can keep old files unchanged while new writes use a new spec.
Users often filter explicit path-derived columns such as year and month.	Users filter business columns; hidden transforms can derive partition predicates.
Over-partitioning creates many folders and files.	Poor transforms or frequent spec changes can increase split planning and metadata work.

Choose the specifications that match stable query patterns and produce useful file grouping. Do not evolve a spec simply to imitate a folder hierarchy. See Apache Iceberg Evolution.

Schema Evolution

Iceberg tracks columns by stable field IDs, but Oracle object behavior depends on the access path:

A direct-metadata Iceberg external table has a fixed Oracle schema and fixed metadata.json entry point. Recreate it when the referenced schema changes.
A catalog-managed external table can resolve a current snapshot through the catalog, subject to the documented catalog and schema restrictions.
A mounted catalog table and a native Iceberg table use catalog metadata, but supported DDL, DML, partitioning, and evolution remain release-specific.

Metadata and File Maintenance

Use only Oracle AI Database 26ai maintenance operations documented for the selected release and Iceberg access path. Do not delete manifests, metadata JSON, snapshots, or data files directly from object storage.

Oracle Maintenance Action	Use it for	Procedure and scope
Refresh the catalog metadata cache	Make catalog-side table or schema changes visible when Oracle catalog metadata is stale.	Use `DBMS_CATALOG.FLUSH_CATALOG_CACHE` or `DBMS_CATALOG.PREFILL_CATALOG_CACHE` as applicable. These procedures refresh only the catalog metadata cache; they do not refresh Lake Cache data, rewrite Iceberg metadata, or compact files.
Refresh Lake Cache Where available and eligible for the deployment	Align locally cached data with new, modified, or retired source files.	Policy-based caches do not refresh automatically; repopulate modified files with the documented `DBMS_EXT_TABLE_CACHE` population operations. Automatic caches refresh on schedule and can be refreshed on demand with `DBMS_CACHE.REFRESH`. See Lake Cache.
Regather optimizer statistics for a persistent local external table	Correct estimates after material changes to the source contents or file layout used by the stored Oracle external-table object.	Use `DBMS_STATS` for the persistent external table. This does not update Iceberg manifests, catalog statistics, or files, and it does not apply to a query-scoped inline external table.

Metadata Cache and Lake Cache

Cache	Stores	Refresh when	Does not replace
`DBMS_CATALOG` Metadata cache	Remote catalog schemas, tables, and related metadata used for discovery and catalog access.	A new table, schema, snapshot, or catalog change is not visible as expected.	Data-file caching or Iceberg table maintenance.
Lake Cache	Eligible external or mounted-catalog table data stored locally in Autonomous AI Database.	Source files are new, modified, or retired according to the cache policy.	Catalog metadata freshness, file compaction, or statistics.

-- Refresh mounted catalog metadata after a catalog-side change.
EXEC DBMS_CATALOG.FLUSH_CATALOG_CACHE('ICEBERG_CAT');
EXEC DBMS_CATALOG.PREFILL_CATALOG_CACHE('ICEBERG_CAT');

-- Inspect Lake Cache objects in the current schema.
SELECT external_table_name, cached, disabled
FROM user_external_tab_caches;

CACHE_ENABLED and CACHE_DURATION control catalog metadata caching where supported. Lake Cache uses DBMS_EXT_TABLE_CACHE for policy-based management or DBMS_CACHE for automatic management. See Manage Catalogs with DBMS_CATALOG, and Lake Cache.

Oracle Iceberg Access Paths

Identify the Oracle object type before selecting a cache or refresh procedure.

Oracle access path	Metadata resolution	Freshness behavior	Relevant cache	Primary tuning owner
Local persistent external table	A persistent external table created with `DBMS_CLOUD`.`CREATE_EXTERNAL_TABLE`, or a `CREATE TABLE ... ORGANIZATION EXTERNAL` definition that uses the `ORACLE_BIGDATA` access driver, points to a metadata.json URI or uses a supported catalog-backed access protocol.	A direct URI remains on the referenced table version. Catalog-backed access can resolve a newer snapshot, but the Oracle column definition remains fixed and must stay compatible with the current Iceberg schema. Recreate the external table when the documented access path requires it after schema evolution.	Lake Cache where available and eligible. This path does not use the `DBMS_CATALOG` metadata cache unless the catalog is mounted separately.	The database schema owns the persistent external-table definition and its local statistics and cache. The source catalog governs Iceberg metadata and snapshots; object storage holds the metadata and data files.
Local inline external table query	A query-scoped EXTERNAL definition that uses the `ORACLE_BIGDATA` access driver points to a metadata.json URI or uses a supported catalog-backed access protocol.	Metadata resolution occurs for the statement. The Oracle columns declared in the inline definition must remain compatible with the resolved Iceberg schema.	No persistent statistics object or Lake Cache applies to the query-scoped definition. It does not use the `DBMS_CATALOG` metadata cache unless the catalog is mounted separately.	There is no persistent table definition, statistics object, or cache owner. The executing schema supplies the required privileges and credentials.
Mounted catalog table `schema. table@catalog`	`DBMS_CATALOG` exposes a remote catalog table without a separately created local external table.	Catalog metadata can be cached for a configured duration and flushed or prefilled.	`DBMS_CATALOG` metadata cache plus Lake Cache for eligible data.	The source catalog governs Iceberg metadata. The Oracle owner manages the catalog metadata cache and Lake Cache and uses only operations documented for the Oracle access path.
Native Iceberg table	Created and addressed through a mounted external catalog using Oracle Iceberg DDL.	Catalog transactions and visibility follow the supported Oracle and catalog feature set.	Verify Lake Cache eligibility and catalog metadata behavior for the release.	The Oracle owner uses documented Native Iceberg DDL, DML, caching, and maintenance operations. The catalog administrator manages catalog availability and authorization.

Note - After external maintenance, if the table is missing or stale, refresh catalog metadata. If the table is visible but reads old cached files, refresh Lake Cache. If the plan estimates are wrong, regather Oracle statistics. These are three independent actions.