Data Streams

Oracle GoldenGate Data Streams provides a WebSocket endpoint that continuously publishes committed change records from an Oracle GoldenGate trail. A client creates or uses a named Data Stream resource, connects to the generated endpoint, receives JSON records, processes each record, and saves the last successfully processed position for recovery.

Data Streams uses an AsyncAPI 3.0 document to describe the endpoint, channel, query parameters, message shape, and record schemas. The generated AsyncAPI YAML for the specific stream is the authoritative contract for that stream. Use it instead of copying a generic host name, channel path, or payload definition from another deployment.

Benefits for Developers and Data Scientists

• Enhanced Data Ingestion: The Publish or Subscribe model powered by AsyncAPI enables applications to efficiently receive real-time data updates.

• Flexible Data Formatting: You can select a preferred format for seamless integration with existing tools.

• Streamlined Integration: AsyncAPI fosters smooth interaction with various applications and tools commonly used by developers and data scientists.

• Guaranteed Data Integrity: Inheriting the core strength of Oracle GoldenGate, Data Streams ensure data durability by replicating changes as they are committed in the source database.

Note: The Data Streams functionality is licensed with Oracle GoldenGate for Distributed Applications and Analytics or with Oracle GoldenGate for Big Data Targets. Refer to the Oracle GoldenGate for Distributed Applications and Analytics in the Oracle GoldenGate Licensing Guide for details.

Runtime sequence

1. Extract captures committed source changes and writes them to a local or remote Oracle GoldenGate trail.

2. An administrator creates a Data Stream resource that identifies the source trail and optional filtering and JSON output settings.

3. The service generates an AsyncAPI YAML document for the Data Stream.

4. A client establishes a WebSocket connection and supplies begin=now, begin=earliest, an ISO 8601 timestamp, or a saved pos value.

5. The service pushes JSON messages continuously. The protocol does not wait for a per-record acknowledgment from the client.

6. The client processes records and durably saves the pos from the last successfully processed DML or DDL record.

What the service sends

• JSON Schema records that describe records in the stream.

• A stream metadata record that identifies the producer, source database, host, and source trail information.

• Object metadata records that describe table columns, JSON primitive types, nullability, and key membership.

• DML records for INSERT, UPDATE, and DELETE .

• DDL records that identify the affected object and contain the DDL statement.

• Optional CloudEvents JSON envelopes whose data member contains the GoldenGate record.