Migrate from Classic to Microservices Architecture Using the Migration Utility

The Oracle GoldenGate Classic to Microservices Migration Utility is used to migrate from an Oracle GoldenGate Classic deployment to a Microservices deployment. This utility is an interactive utility that migrates Extracts, Replicats from a Classic deployment to the Microservices deployment and converts pumps to a distribution path. It also migrates trail files, checkpoints, and other associated artifacts.

The Migration Utility is available in the My Oracle Support (MOS) portal as Patch 37274898. Updates to the patch are released when required. The patch description includes details with the latest build number.

Prerequisites for Using the Migration Utility

Following are the prerequisites for using the Migration Utility:

  • Oracle GoldenGate 19c and higher with Bundle Patch 200714 or later.

    Note:

    The Migration Utility can also be used with Oracle GoldenGate 21c, if the underlying database/system already supports Oracle GoldenGate Microservices Architecture in the earlier version, such as Oracle GoldenGate for DAA. If the underlying database does not support Microservices Architecture, then you can use Oracle GoldenGate 23ai for heterogeneous databases.

  • Java version 1.8 JRE or later (Same as provided by Oracle GoldenGate).

  • New deployments created using Microservices Architecture should be local to the Classic Architecture deployments or the Classic Architecture home file system must be available on the same host.

  • New Microservices Architecture deployment created and all services running.

  • Classic Architecture deployment must be stopped during migration. Make sure that the server collectors (if any) are also stopped. Abended processes must be restarted and gracefully shutdown.

  • The Migration Utility needs to run from a shell/command prompt that has the environmental variables set for the Microservices Architecture deployment.

    The environment variables for OGG_HOME must be set to the Oracle GoldenGate Microservices software directory location.

    On Microsoft Windows, the PATH variable must point to OGG_HOME\lib.

    On UNIX and Linux platforms, the LD_LIBRARY_PATH needs to point to OGG_HOME/lib.

Migration Utility Restrictions

The migration utility cannot migrate any processes that have the following

  • Extract Pump processes that perform transformations or has a DBLOGIN.

  • Classic Extract (Deprecated in Oracle GoldenGate 19c and desupported in version 21c for any target MA greater than 19.1.)

  • Processes that use the following parameters:

    • MACRO

    • OBEY

    • MACROCHAR

  • Use of Remote Trail Files, Remote Tasks or Initial Load.

  • Absolute path names for trail files. Classic trail file MUST reside in ./dirdat (or a subdirectory within it). If the trail file reside in another directory, they can be moved to ./dirdat and then modify the checkpoint using the convchk utility for the Extract. Then modify the parameter for any Replicat that is affected. See convchk in the Parameters and Functions Reference for Oracle GoldenGate for Oracle GoldenGate 21c.

  • INCLUDE Restrictions:

    • Nested INCLUDES

    • Only in Extract and Replicats, not in Extract Pump.

  • Oracle GoldenGate for DAA (Only): For each Extract and Replicat group, one should manually inspect the corresponding .properties file for properties whose value refers to a file in the Classic file system. For each of these properties, do the following manual migration steps:

    1. Copy the referenced file to the Microservices deployment file system.

    2. Modify the property value to refer to the new file path in the Microservices deployment file system.

Run the Migration Utility

Make sure that you have completed the steps mentioned in Prerequisites for Using the Migration Utility before you begin the tasks in this section.

Use the following steps on the command prompt to run the Migration Utility:

  1. Run the following command to start the Migration Utility:

    java -jar ogg-migration-package-23.4.0.0.0.jar -classicHome /scratch/ogg/home –
    -dryrun java -jar ogg-migration-package-23.4.0.0.0.jar -classicHome /scratch/ogg/home
    –debug java -jar ogg-migration-package-23.4.0.0.0.jar –help
    The options available with this command are:

    • -classicHome path: (Required) Must point to the location of the Oracle GoldenGate Classic software directory.

    • -dryrun: (Optional) Indicates that the migration is simulated (Do a Dry Run) to check for possible issues.

    • -debug: This is optional but it is recommended that you use this option with the utility. Enables debug level logging and diagnostics. It provides more detailed log data and will produce a zip file with artifacts from both the classic and MA when a migration fails.

    • -skipPumpURI: Specifies the Accept Default URI on Pump Migrate

    • -dbtype: Specifies the database name. Valid values are 'ORACLE', 'MYSQL', 'MSSQL', 'POSTGRESQL', 'SYBASE', 'DB2400', 'DB2ZOS', 'DB2LUW', 'TERADATA', 'TIMESTEN'

    • -help: Shows usage.

  2. The utility prompts for the user credentials for the Service Manager of the MA deployment:
    ***************************************************************************
**      Oracle GoldenGate Classic to Microservices Migration Utility              **  
**        Version 23.4.0.0.0 Build 66 at Fri Jan 22 17:24:41 UTC 2024             **
**  Copyright (C) 1995, 2024, Oracle and/or its affiliates. All rights reserved.  **
**                                                                                **    
**         Operating system character set identified as UTF-8.                    **
************************************************************************************

Command Entered: -classicHome /ogg/payroll -debug
Log File for this session is /ogg/payroll/./Migration081321105419.log. Debug Mode is Enabled.
Enter Service Manager Credentials: User Id: dba
Password:
  3. The utility scans for a Service Manager on the host and provides a choice of deployments to migrate to if there are more than one deployment and then prompts to continue.
    Scanning for Microservices Deployments...
Using Deployment payroll [cf4c7c93-3a1c-4cbc-b1ab-bc942bab7645]
The following processes will be migrated to deployment payroll: 
Extracts: EXTE
Replicats: REPW
Pumps: DPE
Do you wish to continue? (yes/no):>

    Enter yes to continue.

  4. The utility will migrate the database credentials from the Classic deployment to the target Microservices deployment and then copy all the trail files in the Classic deployment and migrate all the Extracts and Replicats processes.
    Migrating Credentials from /ogg/payroll/./dircrd
Migrating Credentials for Domain Payroll with an alias of ogg. 
Migrating Credentials for Domain Payroll with an alias of user.

Copying Trail Files from /ogg/payroll/dirdat to /ogg/Microservices/payroll/var/lib/data.
235 trail files copied.

Migrating EXTRACT EXTE to http://10.20.0.221:9901. 
Checkpoint File(s) Copied and Converted Successfully. 
Parameter File EXTE.prm Saved Successfully.
Migration of Extract EXTE Completed Successfully.

Migrating REPLICAT REPW to http://10.20.0.221:9901. 
Parameter File REPW.prm Saved Successfully.
Checkpoint File(s) Copied and Converted Successfully. 
Migration of Replicat REPW Completed Successfully.

  5. The utility then migrates all the Extract Pump processes and prompt for the URL for the target Receiver Service for each. To accept the default value, press enter.

    If you are using a different target, enter the URL. By default, the utility assumes that the classic pump is sending to another classic deployment server process. The default URL allows the Distribution Service to link to the classic target. If you are also planning to migrate the target Classic deployment at a later time, you will need to modify the newly created Distribution Path at the same time. See Different Use Cases for Migration for details.

    Note:

    The loopback pumps are common in a test environment where a Classic pump sends to the same Manager on the same deployment. This loopback pump is automatically be routed to the Receiver Service on the migrated MA deployment. 
    Migrating PUMP PMPW2E to http://10.20.0.221:9902.

Use Default Pump Target URI of ogg://ushost1:9903/services/v2/targets?trail=./bb? 
Press Enter to Accept or type Alternate URI:>
Creating Distribution Path for Pump PMPW2E.
Distribution Path Created Successfully. 
Setting trail sequence to 235 with an RBA of 1319472691.

Migration of Pump PMPW2E Completed Successfully. 

Migrating Manager tasks to http://10.20.0.221:9901.
Creating Task for Parameter PurgeTrail-aa. Task Created Successfully.
Creating Task for Parameter Purge-Extract. Task Created Successfully.
2 Tasks were migrated, 2 Successfully, 0 with errors.

Migrating Lag Report.
Creating Task for Parameter LagReport. Task Created Successfully.
Migration pf Manager Tasks Completed Successfully.
  6. Next, the utility asks if it should migrate parameters in the GLOBALS configuration file and report on any issues encountered.
    Do you wish to migrate your GLOBALS parameters? (yes/no):> y 
GLOBAL parameters successfully migrated, 3 parameters merged.

  7. The utility prompts to copy existing wallet entries if they exist and issues a warning if entries exist on the target wallet. These target wallet entries will be overwritten. After this task completes, the utility terminates.

    Your Classic Deployment has existing wallet entries. 
Do you wish to copy them? (yes/no):> y

Copying Oracle Wallet Entries.

WARNING: Target deployment has existing wallet entries and will be overwritten!

Do you wish to continue? (yes/no):> y 
Wallet entries successfully copied.

Migration Processing Complete, 3 processes were migrated successfully.

    If the migration was successful, it will rename the checkpoint files in the Classic dirchk directory by appending .migrated to the name. This will prevent the Classic deployment processes from being started inadvertently.

    If the utility encounters an issue and is not able to complete the migration process, then it will perform a rollback of all of the work it has performed and restore the Classic deployment to its original state. It will produce a diagnostic zip file containing the log file, classic configuration and target configuration to help diagnose the issue.

Different Use Cases for Migration

Oracle GoldenGate Classic Architecture deployments are usually not self-contained. Deployments typically send trail data to other deployments using the Pump process that have very complex interconnected topologies. When a deployment is migrated to Microservices, it still may be sending trail data to another Classic or Microservices deployment. A major feature of Microservices Architecture is the ability to secure the deployment using established security protocols such as TLS/SSL and secured web sockets.

When migrating deployments, a well thought out approach should be used. Usually, it is preferred that source and target deployments are migrated on an individual basis and not all at once. The Migration Utility allows you to migrate deployments one by one. As the source deployment uses a non-secured protocol to communicate with Classic Server/Collector or the Receiver Service, the migrated Distribution Path will use the legacy ogg protocol in the path. While this will be functional, it does not take advantage of the security capabilities of Microservices Architecture in creating a fully secure deployment.

It is recommended that you replace the Distribution Path generated by the Migration Utility, with one using the wss or ws protocols after all deployments have been migrated to Microservices Architecture or whenever the source and target deployments have been migrated.

To allow phased migration of deployments and processes to Microservices Architecture, source and target Classic Architecture deployments can be independently migrated.

The following lists different use cases and the associated Distribution protocol used while setting up the distribution path.

Source Target Distribution Protocol Description

Hub architecture (no distribution trails)

Migrate Classic to Microservices

Not Applicable

Not Applicable

Not Applicable

Migrate Classic to Microservices

Classic Deployment

ogg protocol to target classic deployment

-

Migrate Classic to Microservices

Microservices Deployment

ogg protocol

Optionally, drop and recreate DISTPATH to use ws or wss protocol (if target is secured with a reverse proxy)

Classic deployment

Migrate Classic to Microservices

Pump to target Receiver Service

Source Pump must be modified:

Receiver Service Port and Remote Trail

Microservices deployment

Migrate to Classic to Microservices

ogg protocol

Drop and recreate DISTPATH and use ws or wss protocol (if target is secured with a reverse proxy)

Migrating Source from Classic to Microservices with Target as Microservices

In this use case, you can drop and recreate the Distribution Path to use the ws or wss protocol (if target is secured with a reverse proxy) by specifying the sequence no (seqno) and RBA values. This is optional as the target deployment is using Microservices Architecture.

Run the Admin Client and connect to the source deployment to get the SEQNO and RBA values:
OGG (http://north:9001)> INFO DISTPATH dpe, DETAIL
Path Name: dpe Status: running
Source: trail://north:9002/services/v2/sources?trail=ea 
Target: ogg://south:9003/services/v2/targets?trail=ra 
Authentication Method: USERIDALIAS ggsca domain OracleGoldenGate
[…]
Checkpoints:
Input Checkpoint: 
  Current:
     Name: ea 
     Sequence: 14
     Offset: 2579
     Path: /u01/ogg_deployments/depl_north/var/lib/data/
[…]

Convert the distribution path to ws or wss protocol by recreating and repositioning the Distribution path that was created by the Migration Utility: 

OGG (http://south:9001)> DELETE DISTPATH dpe, DETAIL

##If using Secure Web Socket Protocol (wss)
OGG (http://south:9001)> ADD DISTPATH dpm 
                         SOURCE trail://src_host:9002/services/v2/sources?trail=ea 
                         TARGET wss://trg_host:9003/services/v2/targets?trail=ra 
                         AUTHENTICATION CERTIFICATE default
##If using Web Socket Protocol (ws)
OGG (http://south:9001)> ADD DISTPATH dpm
                         SOURCE trail://src_host:9002/services/v2/sources?trail=ea 
                         TARGET ws://trg_host:9003/services/v2/targets?trail=ra 
                         AUTHENTICATION USERIDALIAS ggsca DOMAIN OracleGoldenGate
OGG (http://south:9001)> ALTER DISTPATH dpm, BEGIN SEQNO 14, RBA 2579
OGG (http://south:9001)> START DISTPATH dpm

Migrate Target from Classic to Microservices where Source is Microservices

In this use case, it is mandatory to drop and recreate the Distribution Path to use ws or wss protocol, if the target is secured with a reverse proxy. Follow the steps in Migrating Source from Classic to Microservices with Target as Microservices to drop and recreate the Distribution Path on the target deployment.

Migrating Target from Classic to Microservices with Source as Classic

When migrating the target deployment to Microservices Architecture while the source deployment is on Classic Architecture, you must perform the following additional steps:

  • Modify the Pump port at source routing it to the target Receiver Service.

  • The dirdat directory is not needed in the target Microservices file hierarchy. Modify RMTTRAIL file from ./dirdat/ra to ./ra in Extract Pump parameter file, as shown in the following example:

As shown in the following example, the target manager port number has been changed to the Receiver Service port number 9903 and the RMTTRAIL file path has been modified in the Extract Pump parameter file. 

GGSCI> STOP EXTRACT PMPW2E
GGSCI> EDIT PARAMS PMPW2E
The following is the Extract Pump parameter file:
EXTRACT PMPW2E 
USERIDALIAS ggeast, 
target uri: host1:9903/services/v2/targets?trail=./bb? ENCRYPT AES192, 
RMTTRAIL ./rt SEQUENCE hr.employees_seq; TABLE hr.*;
Restart Extract Pump after completing the modifications:
GGSCI> START EXTRACT PMPW2E

Note:

No trail files are delivered to the target.

Before the migration begins, stop all processes at the source gracefully.