Manage GG Change Data Stream

Starting a GoldenGate Change Stream

To start a GG Change Data stream:

Go to the Catalog page and hover the mouse over the GG Change Data stream that you want to start.
Click the Start GG Change Data icon that appears to your right side on the screen.
On the warning dialog box, click OK.

Note:

When you start a GG Change Data replicat process, it creates kafka topics, and starts pushing changed data to the new topics. For example, if there are 10 tables in the extract process that you chose while creating the GG Change Data, 10 new topics will be created.

The names of the topics created are in the following format:

GGChangeDataName_fullyQualifiedTableName

You can use these topics to create a new stream (with Goldengate as stream type), and in pipelines, similar to using a Kafka stream.

Stopping a GG Change Data Stream

To stop a GG Change Data stream:

Go to the Catalog page and hover the mouse over the GG Change Data stream that you want to stop.
Click the Stop GG Change Data icon that appears to your right side on the screen..
On the warning dialog box, click OK.

Purging the GoldenGate Trail Files

The trail files are not needed once the replicat has finished processing them. You can purge the trail files to save disk space.

The default settings are as follows: PURGEOLDEXTRACTS /location-of-trail-files MINKEEPHOURS 1, FREQUENCYMINUTES 10.

The trail files, after being processed completely by the replicat process, and after one hour of inactivity, will be purged. The files will be checked for purging every 10 minutes.

You can modify the above rule, in an OCI GGSA VM, following the steps below:

Stop the manager process by running the command sudo systemctl stop ggbd-mgr.
Modify the rules in the file /u01/app/ggbd/OGG_BigData_Linux_x64_19.1.0.0.0/dirprm/mgr.prm.
Start the manager process by running the command sudo systemctl start ggbd-mgr.

For more information on rules about purging the trail files, see PURGEOLDEXTRACTS for Manager.

Streaming GoldenGate Full Records

The GoldenGate Extract process captures either full data records or transactional data changes, depending on the configuration parameters. To minimize the overhead or performance impact on the transactional database, GGSA users configure the Extract to capture only the transactional changes. This also helps to reduce payload size needs to transfer over the network, thus increasing the performance and security. But a few customers also need the unchanged columns (full data records), making them available to the processes that require up-to-date data feed, or to replicate this data to various big data targets for analysis.

To enable streaming of full data record (value of all fields), GGSA provides the Generate Full Records option while creating a GoldenGate stream. Enable this option to stream all the records, irrespective of the database transactional changes made to a single row, a subset or all the columns of a row.

Note:

Full Records option is not supported in the GGSA marketplace instance.