Add an Extract for MongoDB

Extract is a process that runs agains the source data source connection and extracts, or captures, data. Learn to add an Extract for MongoDB.

Related Topics

Add an Extract (26ai)

Before you begin, ensure that you:
  • Review the details specific to this source connection type.
  • Configure a MongoDB replicat set:
    • OCI GoldenGate for Big Data uses the operations log (oplog) to read Change Data Capture (CDC) records. The oplog is a capped collection in MongoDB that maintains a rolling record of all operations that modify the database. Oplog files are created when a replica set is enabled in MongoDB.
    • MongoDB Atlas comes preconfigured with a Replicat set. For on-premises MongoDB deployments, you must manually configure and deploy a Replicat set to enable oplog and CDC functionality. For on premises MongoDB, you need to deploy a Replica set.
  • Have MongoDB version 3.6 or later.
  • Have OCI GoldenGate support to capture the following operations: INSERT, UPDATE, DELETE.
To add an Extract in for MongoDB:
  1. In the OCI GoldenGate deployment console navigation menu, click Extracts.
  2. In the Add Extract panel, on the Extract Information page, complete the following fields as needed, and then click Next:
    • Enter a Process Name, no more than 8 characters long.
    • Enter a Description (optional) to help you distinguish this process from others.
    • Select a Source from the dropdown.
  3. On the Extract Options page, complete the following fields as needed, and then click Next:
    1. For Extract Trail:
      1. Enter a Name for the Extract process.
      2. Enter a Subdirectory name to set a custom location for the generated Trail file.
      3. Enter Trail Sequence to set the starting number for Trail files.
      4. Enter a Trail Size to set the max size for the generated trail file.
      5. Select an Encryption Profile. The Local Wallet profile is selected by default if an encryption profile wasn't created.
      6. Select an Encryption Algorithm:
        • NONE
        • AES256
        • AES192
        • AES128
    2. For Source Credentials, select the:
      1. Domain
      2. Alias
      3. When/where to Begin
  4. On the Managed Options page, complete the following optional fields as needed, and then click Next:
    • Profile Name
    • Critical to deployment health
    • Auto Start
    • Auto Restart

    Note:

    Adding a profile and configuring Auto Start and Auto Restart options enables your deployment to restart automatically after a network disruption. See Configure managed processes to learn more.

  5. On the Parameter File page, update the source mapping with TABLE source.*;

    Note:

    This is the source database/collection mapping. TABLE *.*; results in the process extracting from all databases/collections.
  6. Click Create and Run to create and start the Extract. If you click Create, then you can manually start the Extract later from the Extracts page.

Add an Extract (21c)

Learn to add an Extract process for a MongoDB source in OCI GoldenGate

To add an Extract for MongoDB:
  1. In the OCI GoldenGate deployment console, ensure that you're on the Administration Service Overview page, and then click Add Extract (plus icon).
  2. On the Add Extract page, for Extract type select one of the following, and then click Next:
    • Initial Load
    • Change Data Capture
  3. On the Extract Options page, complete the fields as follows, and then click Next:
    1. For Process Name, enter a name for the Extract.
    2. For Connection Alias, select the connection alias from the dropdown.
    3. For Source, select File.
    4. For File Name, enter three characters at minimum for the filename.
    5. (Optional) Under Managed Options, you can configure the following:
      • Profile Name
      • Auto Start
      • Auto Restart

      Note:

      Adding a profile and configuring Auto Start and Auto Restart options enables your deployment to restart automatically after a network disruption. See Configure managed processes for more information.
  4. On the Parameter File page, update the source mapping with TABLE source.*;

    Note:

    This is the source database/collection mapping. TABLE *.*; results in the process extracting from all databases/collections.
  5. Click Create and Run.
You return to the Administration Service Overview page. Click the Extract name to view details and reports of the Extract.

Known issues

Replicats fail when using Trail file from MongoDB Extract with BINARY_JSON_FORMAT

When a Replicat uses a Trail file generated from a MongoDB Extract with BINARY_JSON_FORMAT in the Extract parameter file, the Replicat fails with the following error: 

ERROR 2023-08-04 17:13:13.000421 [main] - Unable to decode column 0 : Input length = 1
      java.nio.charset.MalformedInputException: Input length = 1 at
      java.nio.charset.CoderResult.throwException(CoderResult.java:281) ~[?:1.8.0_311]at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816) ~[?:1.8.0_311] at
      oracle.goldengate.datasource.UserExitDataSource.createColumnValue(UserExitDataSource.java:1106)
      [ggdbutil-21.9.0.0.3.001.jar:21.9.0.0.3.001] Exception in thread “main”
      oracle.goldengate.util.GGException: Unable to decode column 0 : Input length = 1 at
      oracle.goldengate.datasource.UserExitDataSource.createColumnValue(UserExitDataSource.java:1203)

Workaround: When BINARY_JSON_FORMAT is removed from the Extract parameters, the Replicat runs successfully and documents are represented in Extended JSON format.