Payload and Schema Reference

Data Streams is self-describing when JSON schema and metadata records are enabled. The exact record shape can change when output fields are renamed, optional metadata is enabled, the record model changes, or source metadata changes. The generated AsyncAPI YAML and in-band JSON Schema records are the authoritative runtime definitions.

Classification	Identification	Processing guidance
Schema record	Contains `id`, `$schema`, `type`, and `properties`	Cache by `id`; do not route as business data.
Stream metadata	`op_type` is `STREAM_METADATA`	Record producer and source context; refresh diagnostics when it reappears.
Object metadata	`op_type` is `OBJECT_METADATA`	Cache columns, primitive types, nullability, and key flags by `qual_table`.
DML	`op_type` is `INSERT`, `UPDATE`, `DELETE`, or `UPSERT`	Apply the row change according to before/after semantics.
DDL	`op_type` is `DDL` and `ddl` is present	Process the object and statement, then expect refreshed metadata/schema when applicable.

JSON Schema records

By default, Data Streams emits a JSON Schema record before sending a record type that requires that schema. Schema records use JSON Schema draft 4 and contain the following fields:

Field	Meaning
`id`	Stable identifier for the schema type and, for table-specific schemas, the qualified object.
`title`	Human-readable schema title.
`$schema`	JSON Schema dialect, `http://json-schema.org/draft-04/schema#`.
`type`	Top-level JSON type, `object`.
`definitions`	Reusable column or metadata definitions when required.
`properties`	Allowed fields and their constraints.
`required`	Fields required for instances of the schema.
`additionalProperties`	Whether fields not declared by the schema are accepted.

Consumer rule: Do not expect a schema record to contain op_type. Detect it by schema structure and id. A schema can be sent again when source metadata or stream configuration changes.

Stream metadata record

Field	Subfields	Description
`op_type`	-	`STREAM_METADATA`.
`producer`	`name`, `version`	Data Stream producer identity and software version.
`database`	`name`, `version`, `charset`, `timeZone`	Source database context. Treat values as informational and source-dependent.
`machine`	`node`, `system`, `release`, `version`, `hardware`	Host operating system context.
`source`	`file`, `uri`, `uriHistory` when present, `creationTime`, `compatibility`	Source trail or process context.

Object metadata record

An object metadata record describes the current table definition used to encode DML. It has op_type: OBJECT_METADATA, qual_table, and a columns object. Each column contains:

Column attribute	Description
`type`	JSON primitive classification such as `string`, `number`, or `boolean`. Use the table-specific JSON Schema for the complete accepted types, including `null`.
`null`	Whether the source column can be null according to the captured metadata.
`key`	Whether the column is part of the key known to GoldenGate for the object.

{
  "op_type": "OBJECT_METADATA",
  "qual_table": "SALES.ORDERS",
  "columns": {
    "ORDER_ID": {"type": "number", "null": false, "key": true},
    "STATUS": {"type": "string", "null": false, "key": false},
    "TOTAL_AMOUNT": {"type": "number", "null": true, "key": false}
  }
}

DML record

Default field	Required by default	Meaning
`qual_table`	Yes	Fully qualified table name. The name and presence can be changed with `metaFields.qualifiedTable`.
`op_type`	Yes	Operation string. Defaults are `INSERT`, `UPDATE`, `DELETE`, `UPSERT`; strings can be customized.
`op_ts`	Yes	Source operation timestamp in ISO 8601 form in current Data Streams schemas.
`xid`	Yes	Source transaction identifier represented as a string. Format is source-dependent.
`pos`	Yes	Opaque recovery position. Store exactly; do not parse.
`before`	Conditional	Before image for update and delete operations when the required columns were captured. Name can be customized.
`after`	Conditional	After image for insert and update operations when the required columns were captured. Name can be customized.
Optional metadata	No	`csn`, operation sequence, transaction indicator, primary keys, catalog/schema/table, processing timestamps, tokens, environment values, and static fields.

Before and after image matrix

Operation	`before`	`after`	Notes
INSERT	Normally absent	New row image	Only columns available in the trail are emitted.
UPDATE	Original values	New values	The trail and source logging determine whether images are full or partial.
DELETE	Deleted row image	Normally absent	Ensure key columns and required audit columns are captured.
UPSERT	Runtime-schema dependent	Resulting row image	Use the generated schema and operation contract for the source pipeline.

DML examples

{
  "qual_table": "SALES.ORDERS",
  "op_type": "INSERT",
  "op_ts": "2026-07-13T04:01:01.123456Z",
  "xid": "example-transaction-1001",
  "pos": "G-QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVo=1001",
  "after": {
    "ORDER_ID": 1001,
    "STATUS": "NEW",
    "TOTAL_AMOUNT": 125.50
  }
}

{
  "qual_table": "SALES.ORDERS",
  "op_type": "UPDATE",
  "op_ts": "2026-07-13T04:02:12.654321Z",
  "xid": "example-transaction-1002",
  "pos": "G-R0hJSktMTU5PUFFSU1RVVldYWVo=1002",
  "before": {"ORDER_ID": 1001, "STATUS": "NEW"},
  "after": {"ORDER_ID": 1001, "STATUS": "APPROVED"}
}

{
  "qual_table": "SALES.ORDERS",
  "op_type": "DELETE",
  "op_ts": "2026-07-13T04:03:15.000000Z",
  "xid": "example-transaction-1003",
  "pos": "G-SktMTU5PUFFSU1RVVldYWVo=1003",
  "before": {"ORDER_ID": 1001, "STATUS": "APPROVED"}
}

DDL record

Field	Meaning
`op_type`	`DDL`.
`op_ts`	DDL operation timestamp.
`xid`	Transaction identifier associated with the DDL record.
`pos`	Opaque recovery position for the DDL record.
`ddl.object.catalog`	Catalog name when the source provides one; can be an empty string.
`ddl.object.schema`	Schema or owner name.
`ddl.object.object`	Affected object name.
`ddl.statement`	DDL text, truncated if it exceeds `maxDDLTextSize`.

{
  "op_type": "DDL",
  "op_ts": "2026-07-13T04:04:30.000000Z",
  "xid": "example-transaction-1004",
  "pos": "G-TU5PUFFSU1RVVldYWVo=1004",
  "ddl": {
    "object": {
      "catalog": "",
      "schema": "SALES",
      "object": "ORDERS"
    },
    "statement": "ALTER TABLE SALES.ORDERS ADD CUSTOMER_TIER VARCHAR2(20)"
  }
}

CloudEvents format

When cloudEventsFormat is enabled, the GoldenGate record is carried in the CloudEvents JSON data member. Standard CloudEvents attributes identify the event. Treat the values of type, source, and id as runtime contract values and inspect the generated YAML and actual records rather than hardcoding examples from another stream.

{
  "specversion": "1.0",
  "type": "<runtime event type>",
  "source": "<runtime event source>",
  "id": "<runtime event id>",
  "datacontenttype": "application/json",
  "data": {
    "qual_table": "SALES.ORDERS",
    "op_type": "INSERT",
    "op_ts": "2026-07-13T04:01:01.123456Z",
    "xid": "example-transaction-1001",
    "pos": "G-QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVo=1001",
    "after": {"ORDER_ID": 1001, "STATUS": "NEW"}
  }
}