Handle DDL and Schema Evolution

Expected sequence

When a table definition changes and DDL capture is enabled, the stream can deliver the DDL record and then updated object metadata and table-specific DML schema before subsequent records that use the new definition. The exact batching and order across WebSocket messages are implementation details; consumers must handle a schema or metadata record whenever it appears.

• Cache schemas by their id, not only by table name.

• Cache object metadata by qual_table and replace it when a newer record arrives.

• Do not assume that the schema received at connection time remains valid for the life of the connection.

• Pause or dead-letter a DML record if its required schema is unknown and the downstream system cannot accept schemaless data.

• Apply compatible changes automatically where policy allows; route incompatible changes to a controlled migration workflow.

Compatible and incompatible changes

Change	Typical consumer action
Add nullable column	Accept the new field and update the downstream schema asynchronously or dynamically.
Add non-null column	Confirm default/backfill behavior before applying records.
Drop column	Stop writing the field; preserve historical downstream data according to retention policy.
Rename column	Treat as drop plus add unless a governed mapping states otherwise.
Change JSON primitive type or source precision	Validate conversion, precision, and downstream compatibility before continuing.
Change key columns	Rebuild idempotency, merge, and delete logic before consuming records under the new key.

DDL handling policy

• Record the DDL pos, object identity, statement, and operation timestamp in an audit log.

• Do not execute unreviewed source DDL directly against a downstream system unless that behavior is explicitly approved and secured.

• Use maxDDLTextSize large enough for the organization’s DDL, and alert when truncation is detected or suspected.

• Expect multiple schema versions for the same table over time. Keep the schema identifier with archived events.

Schema registry integration

A consumer can publish in-band JSON Schema records to an enterprise schema registry. Preserve the original id, associate it with the Data Stream and deployment, and version changes rather than overwriting historical schema definitions. The registry complements the runtime schema; it does not replace checkpoint or source-trail recovery.