Migrate MongoDB to Oracle AI Database with zero downtime

Discover how to use OCI GoldenGate to MongoDB to Oracle AI Database with zero downtime.

OCI GoldenGate Big Data supports migrations from MongoDB and MongoDB Atlas to the following targets with zero downtime:

  • Oracle Autonomous AI JSON Database
  • Oracle Autonomous AI Database
  • Oracle AI Database

Autonomous AI JSON Database and Autonomous AI Database come with preconfigured Oracle API for MongoDB connection strings that OCI GoldenGate uses to connect to target Oracle AI Database systems. Refer to Using Oracle Database API for MongoDB for more information.

If your target is an on-premise Oracle AI Database, you can use Oracle Rest Data Services to enable Oracle Database API for MongoDB with your on-premise Oracle AI Database.

Before you begin

To successfully complete this quickstart, ensure you have:

  • A MongoDB source, version 5 or higher
  • Created an OCI GoldenGate Big Data deployment, version 23.7 or higher
  • MongoDB Database Tools, including mongodump and mongostore, installed and respective directory paths added to PATH environment variable

Set up and run the migration

  1. In your source MongoDB, run the MongoDB Dump Utility.
    1. Run mongodump with --oplog option to create a snapshot of the source MongoDB database:
      $ ./bin/mongodump --uri="mongodb://localhost:27021" --oplog -v

      The command returns the following:

      <date+timestamp>    getting most recent oplog timestamp
<date+timestamp>    writing admin.system.version to dump/admin/system.version.bson
<date+timestamp>    done dumping admin.system.version (1 document)
<date+timestamp>    dumping up to 4 collections in parallel
<date+timestamp>    writing testDB.coll1 to dump/testDB/coll1.bson
<date+timestamp>    writing testDB.coll2 to dump/testDB/coll2.bson
<date+timestamp>    writing testDB.coll3 to dump/testDB/coll3.bson
<date+timestamp>    done dumping testDB.coll3 (10000 documents)
<date+timestamp>    done dumping testDB.coll1 (10000 documents)
<date+timestamp>    done dumping testDB.coll2 (10000 documents)
<date+timestamp>    writing captured oplog to
<date+timestamp>    dumped 7 oplog entries
      This generates a dump folder containing the binary archive data file for all databases and collections in the following location: 
      dump/database-name/collection-name/collection-name.bson
      It also creates an oplog.bson directly under dump folder.
    2. To inspect the contents of the oplog.bson file (which is in binary format), you can convert it to JSON using the bsondump utility:
      $ ./bin/bsondump --pretty --outFile /path/to/oplog.json dump/oplog.bson

      The command returns the following:

      <date+timestamp>    7 objects found
  2. Extract first and last operation timestamps from oplong.bson:
    1. Download the OplogLSN.sh script
    2. Run the OplongLSN.sh script in your source MongoDB in the same directory where mongodump is located, passing the location to oplog.bson as an argument as follows:
      $./oplogLSN.sh /path/to/dump/oplog.bson

      The command returns the following:

      <date+timestamp> 1 objects found
First LSN: 1740663867.1
Last LSN: 1740663946.211
    3. Inspect Oplog entries. If there were any incoming operations during the dump, then the oplog.bson file contains entries for those operations, each with a timestamp. You can use OplogLSN.sh to capture the first and last operation timestamps or convert it to a JSON file for manual inspection, as shown in the previous step.
  3. Run the MongoDB Restore Utility. Use the mongorestore utility to restore the selected collections from the dump to the target MongoDB instance:
    $ ./mongorestore --uri="mongodb://localhost:27021" --nsInclude=testDB.coll1 --nsInclude=testDB.coll2 /path/to/dump -v

    The command returns the following:

    <date+timestamp>    using write concern: &{majority <nil> 0s}
<date+timestamp>    using default 'dump' directory
<date+timestamp>    preparing collections to restore from
<date+timestamp>    found collection admin.system.version bson to restore to admin.system.version
<date+timestamp>    found collection metadata from admin.system.version to restore to admin.system.version
<date+timestamp>    don't know what to do with file "dump/oplog.json", skipping...
<date+timestamp>    found collection testDB.coll1 bson to restore to testDB.coll1
<date+timestamp>    found collection metadata from testDB.coll1 to restore to testDB.coll1
<date+timestamp>    found collection testDB.coll2 bson to restore to testDB.coll2
<date+timestamp>    found collection metadata from testDB.coll2 to restore to testDB.coll2
<date+timestamp>    reading metadata for testDB.coll1 from dump/testDB/coll1.metadata.json
<date+timestamp>    reading metadata for testDB.coll2 from dump/testDB/coll2.metadata.json
<date+timestamp>    creating collection testDB.coll1 with no metadata
<date+timestamp>    creating collection testDB.coll2 with no metadata
<date+timestamp>    restoring testDB.coll1 from dump/testDB/coll1.bson
<date+timestamp>    restoring testDB.coll2 from dump/testDB/coll2.bson
<date+timestamp>    finished restoring testDB.coll1 (10000 documents, 0 failures)
<date+timestamp>    finished restoring testDB.coll2 (10000 documents, 0 failures)
<date+timestamp>    no indexes to restore for collection testDB.coll1
<date+timestamp>    no indexes to restore for collection testDB.coll2
<date+timestamp>    20000 document(s) restored successfully. 0 document(s) failed to restore.

    The command restores the data, metadata, and index definitions for the specified collection to the target MongoDB/ORDS instance. For example, coll1 and coll2 in database testDB as shown in the output above.

  4. Create and run a Change Data Capture (CDC) Extract on your source Big Data deployment for MongoDB. Start the MongoDB CDC Extract from the first operation timestamp (First LSN) obtained in Step 2b. This ensures the CDC Extract captures operations that occur after the dump process starts.
  5. Create and run a MongoDB Replicat.
    1. Use the generated CDC Trail File from Step 4.
    2. Set oplongReplayLastLsn to the last operation timestamp (Last LSN) obtained in Step 2b, or the path to oplog.bson and the Last LSN is obtained automatically. This ensures the Replicat runs in oplong-replay mode, avoiding collisions and guaranteeing precise initiation with no data loss or duplication. After the last timestamp is processed, the Replicat continues in normal mode.

Tips

Here are some tips to ensure your migration goes smoothly:

  • It's recommended to use mongodb-database-tools version 100.10.0 or below.
  • Before you run mongodump, clean up the the existing dump folder to delete inconsistent data.
  • MongoDB restore: Multiple collections of a database can be replicated using multiple --nsInclude options in mongorestore command. However, the ORDS multiple databases cannot be restored using mulitple --nsInclude commands. You must use multiple restore commands, one for each database to be restored.