Migrate Data Integration Platform Cloud GoldenGate Configuration to Oracle Cloud Marketplace

Pre-Migration Preparation Phase

Before migrating your Data Integration Platform Cloud GoldenGate configuration to the Oracle Cloud Infrastructure Marketplace, you must make the necessary changes to your old GoldenGate environment in Data Integration Platform Cloud, and get ready to provision the new GoldenGate environment through the Oracle Cloud Infrastructure Marketplace.

Get set to migrate to the Oracle Cloud Infrastructure Marketplace, following these steps:

Freeze any configuration change to the old GoldenGate environment. You can still can start/stop any GoldenGate processes. Delete any unneccesary GoldenGate processes. Make sure all running GoldenGate processes are up to date, for example, no stopped processes with extremely large LAG/Gap.

Check running GoldenGate LAG
```
GGSCI>INFO ALL
```
Make sure that all GoldenGate processes are showing as running. If there are any showing as stopped or abend, you must check the reason. Dlete the processes if they are no longer needed any more.
```
GGSCI>LAG ER *
```
Make sure that the LAG time is small.
Provision the new GoldenGate environment through the Oracle Cloud Infrastructure Marketplace and choose the appropriate VCN that will allow access to the source/target Database. Copy over the GoldenGate credential and wallet File, GLOBALS, and MGR.prm File. Prepare the manager and pmsrvr process.

GoldenGate is fully provisioned on the Market with all the files ready. No process is running. Make sure that the new environment uses the same settings including the TCP and UDP ports, as the old environment.

Some of the default ports that are used by GoldenGate from the old environments are Manager port, 7809, from dirprm/MGR.prm file PMSRVR UDP PORT, 9930; PMSRVR HTTP PORT 7810, from GLOBALS file under the gghome directory in old environment. Copy these two files to the corresponding location on the new environment, and then start manager and pmsrvr by using the following command:
```
Ggsci>start manager
Ggsci>start pmsrvr -initds
```
Enter the following command to check those two processes are up and running:
```
Ggsci>info all
```
You should see the following result:
```
GGGSCI (wocphx) 1> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING                                           
JAGENT      STOPPED                                           
PMSRVR      RUNNING    
    
```
Check all the network connection and Database login/GG Credential on the new GoldenGate instance to make sure that after migration, those connection string or credential alias will work correctly.

If all GoldenGate processes were configured by Data Integration Platform Cloud tasks, then only make sure hostname is reachable, as Data Integration Platform Cloud tasks use Easy Connect format. Make sure that the hostname used in easy connect string is reachable. All the used host names can be found in the GoldenGate credentialstore. Run the following command in GGSCI to check the credentialstore:
```
Ggsci>info credential
```
Example:
```
GGSCI (wocphx) 30> info credentialstore
```
Reading from credential store:
```
Default domain: OracleGoldenGate
  Alias: GGALIASTGT_CLASSIC_WOCPHX_WDBTESTOCI
  Userid: c##ggs@//wdbtestoci:1521/wdboci2_pdb1.sub10292314462.wernervcn.oraclevcn.com

  Alias: GGALIASSRC_CDB_SRCWDBCSPDB
  Userid: c##ggs@//129.150.181.171:1521/ORCL.609376793.oraclecloud.internal
  Alias: GGALIASTGT_CLASSIC_TGTWTARGETPDB
  Userid: dipc_tgt@//10.0.0.3:1521/wpdb.sub10292314460.wernervcn.oraclevcn.com
  Alias: reptestuser
  Userid: dipc_tgt@//10.0.0.3:1521/wpdb.sub10292314460.wernervcn.oraclevcn.com
  Alias: GGALIASTGT_CLASSIC_TGTDBAASWTARGET
  Userid: c##ggs@//wdboci:1521/wpdb.sub10292314460.wernervcn.oraclevcn.com
  Alias: GGALIASSRC_CDB_WOCPHX_WDBOCI_PDB
  Userid: c##ggs@//wdboci:1521/WTARGET_phx14d.sub10292314460.wernervcn.oraclevcn.com
```

Here are the directories that are needed to be copied from the old environment to the new environment

dirwlt

Copy entire directory from the Data Integration Platform Cloud remote agent gghome to the new GoldenGate home on VM. This directory has GoldenGate encryption key files.

dircrd

Copy entire directory from the Data Integration Platform Cloud remote agent gghome to the new GoldenGate home on VM. This directory has all the credential store that holds credential userid and alias.

If you've created your own Extract/Replicat manually, and you are not using credential alias to log in to the Database, make sure that those sqlnet connection works fine to connect to the source or target database the original Extract or Replicat is running against. This might include updating the sqlnet.ora and tnsnames.ora file, using the information from the Data Integration Platform Cloud remote agent, if you've done any customized setting on the remote agent.

Note:

Special consideration is needed for any deployment using SSL connection to the database. Make sure that the wallet and certificate is transferred to the new VM properly. Use tnsping to verify that the new VM can talk to the same source database or target database through sqlnet. Use sqlplus or dblogin in GGSCI to verify the logins or credential alias work correctly to connect to the database.

Migration Phase

The migration phase involves working with the GoldenGate processes in the old to new environment.

To start your migration:

Check long running transactions on source database, and check Lag time on GoldenGate processes.
Also, check open transactions Extract is tracking:
```
Ggsci>send <extract>, showtrans
```
There will always be open transactions on production system. This is normal. However, you want to pay attention to the ‘Start time’ of the transaction, make sure it is not too old, such as a couple of days. You would typically see open transactions dating back to 1-2 hours ago. The older the open transaction start time shows, the longer the recovery time your new Extract will need upon start. Also, make sure that all the archived log files exist on your system starting from that time.
If you find open transactions dating back further, please check the gv$transaction view to track down that transaction if it is still alive, then contact your DBA to properly clean it up.
```
Sqlplus>select status, xid, xidusn, xidslot, xidsqn, start_scn, start_time from
        gv$transaction
```
Example :
```
GGSCI (wocphx) 5> send E0959068, showtrans

Sending SHOWTRANS request to EXTRACT E0959068 ...


------------------------------------------------------------
XID:                  3537721090.9.24.8685  
Items:                0        
Extract:              E0959068  
Redo Thread:          1      
Start Time:           2019-05-27:23:05:23  
SCN:                  0.85977095 (85977095)             
Redo Seq:             118
Redo RBA:             82133208            
Status:               Running             
Stop
```
Once you've verified the open transactions, make sure they are all recent transactions. Then, you can proceed to stop Extract.
```
SQL> select status, xid, xidusn, xidslot, xidsqn, start_scn, start_time from gv$transaction;

STATUS		 XID		      XIDUSN	XIDSLOT     XIDSQN  START_SCN
---------------- ---------------- ---------- ---------- ---------- ----------
START_TIME
--------------------
ACTIVE		 09001800ED210000	   9	     24       8685   85977095
05/27/19 23:05:23
```
Stop GoldenGate processes on the old environment.
First, stop source redo log reading Extract on the old environment. Assuming the Extract processes were created by Data Integration Platform Cloud tasks, the name will begin with “E”
```
 Ggsci>Stop E* 
```
If the Extract was created by user manually through GGSCI, it could be any naming convention. Please adjust the name properly.

After the redo Log Extract is stopped, make sure all the downstream components have reached EOF, for example, consumed all the exttrail files. Then stop them in order, Pump Extract (if exists) first, then Replicat. Follow the same manner, first check all the data in the trail file has been completely consumed, and then stop.

For pump Extract or Replicat, do:
```
Ggsci>send <pump extract>, status 
```
For Replicat, do:
```
Ggsci>send <replicat>, status
```
Note:
Make sure that the Sequence # and RBA # matches the last exttrail file seqno in the name and the size of the file.
Example:
```
GGSCI (wocphx as c##ggs@WTARGET/CDB$ROOT) 41> info R8346919

REPLICAT   R8346919  Last Started 2019-05-21 19:42   Status RUNNING
INTEGRATED
Checkpoint Lag       00:00:00 (updated 00:00:02 ago)
Process ID           14662
Log Read Checkpoint  File /mnt/wdisk/dicloud/gghome/dirdat/B8000000002
                     2019-05-21 19:12:49.000000  RBA 17461418


GGSCI (wocphx as c##ggs@WTARGET/CDB$ROOT) 42> sh ls -l /mnt/wdisk/dicloud/gghome/dirdat/B8000000002

-rw-r-----. 1 dipcdemo dipcdemo 17461418 May 21 19:35 /mnt/wdisk/dicloud/gghome/dirdat/B8000000002


GGSCI (wocphx as c##ggs@WTARGET/CDB$ROOT) 43> send R8346919, status

Sending STATUS request to REPLICAT R8346919 ...
  Current status: At EOF
  Sequence #: 2
  RBA: 17,461,418
  0 records in current transaction
```
After verifying the Pump Extract process has finished processing all the trail files it is reading, stop the Pump Extract processes. By default, the pump extract created by Data Integration Platform Cloud tasks are named beginning with P.
```
 Ggsci>Stop P*
```
If there are any customized pump extract processes you created, you must stop them using the same procedure.
After verifying the Replicat Process has finished processing all the trail files it is reading, stop the Replicat processes. By default, the Replicat process created by Data Integration Platform Cloud tasks are named beginning with R.
```
Ggsci>Stop R*
```
If there are any customized replicat processes you created, they must be stopped, following the same procedure. Once all GoldenGate components are stopped, you can use the following command to verify the checkpoint positions of the Extract and Replicat
```
Ggsci>info <extract name>, showch Ggsci>info <pump extract name>, detail Ggsci>info <replicat name>, detail 
```
Save the information of the output. You will need that to verify the migrated processes later.
Stop Manager and PMSRVR
```
 Ggsci>Stop manager
 Ggsci>Stop PMSRVR
```
Copy GoldenGate configuration files from the old environment to new environment.
dirprm
Copy only Extract and Replicat parameter files
dirchk
Copy only pce/pcb/pcr files
Use the commands below to check the GoldenGate processes checkpoint in the new environment and make sure they match the information you see in previous steps.
```
Ggsci>info <extract name>, showch
Ggsci>info <pump extract name>, detail
Ggsci>info <replicat name>, detail
```
The only difference that you will see is the Replicat ‘Log Read Checkpoint file,’ which was still using the old full path location. This will be taken care of in the step below.
Copy at least the last trail file that is being updated or read from old environment to new environment.
If the trail file is at default location ./dirdat, just copy the trail file over. Otherwise, you must create symbolic link for the trail file location to match original location. Copy at least last seqno trail file over to the symbolic link location.
Restart GoldenGate process in new environment
Redo Reading Extract Handling
Issue the following commands in GGSCI
```
Ggsci>start <extract name>
```
Pump Extract handling:

Make sure that the new environment has proper network connection to the target GoldeGate location (mgr port and collector port). Proper firewall or network access rule changes on the target are required, due to the change of source.
```
Ggsci>start <pump Extract>
```
Replicat handling:
You must position Replicat to start from the current dirdat directory, as Replicat uses absolute path. Also, you must create a checkpoint table, since the elevated task in Data Integration Platform Cloud doesn’t use checkpoint table for Replicat. This means you will have to delete the Replicat and add it back again. When you copied the Replicat checkpoint file over, you will see the Replicat in GGSCI. Take down the checkpoint information, so it can be used when you add it back.
```
Ggsci>dblogin useridalias <use the Replicat userid alias>
Ggsci>add checkpointtable <OGG_User>.<checkpoint table name>
Ggsci>info replicat <name>
Write down the checkpoint information, especially the read checkpoint position.
Ggsci>delete replicat <name>
Ggsci> add replicat <name>, exttrail ./dirdat/<trail name>, extseqno <current +1>, extrba 0, checkpointtable <OGG_User>.<checkpoint table name>

Ggsci>start <replicat name>
```
For example, if your 'info replicat’ command shows the following,
```
GGSCI (wocphx as c##ggs@WTARGET/CDB$ROOT) 41> info R8346919

REPLICAT   R8346919  Last Started 2019-05-21 19:42   Status STOPPED
INTEGRATED
Checkpoint Lag       00:00:00 (updated 00:00:02 ago)
Process ID           14662
Log Read Checkpoint  File /mnt/wdisk/dicloud/gghome/dirdat/B8000000002
                     			2019-05-21 19:12:49.000000  RBA 17461418
```

Take down 2 as the current seqno, it is from B8000000002.

Post-Migration Verification phase

After completing the migration, you must verify if the GoldenGate processes are all running without any errors.

Verify the success of your migration, following these steps:

Verify Goldengate process are up and running, and not throwing any error message.
```
Ggsci>info all
Ggsci>view report <Extract/Pump/Replicat name> 
```
Verify if GoldenGate checkpoint is moving forward.
```
Ggsci>send <Extract/Pumo/Replicat> status 
```

Other Considerations

To make sure that the migration is successful, please double-check if remote capture and/or remote delivery is feasible to your source and/or target when remote agent is used in your current Data Integration Platform Cloud implementation.

Remember to consider these points:

Keep in mind, GoldenGate instance provisioned from the Oracle Cloud Infrastructure Marketplace runs only inside Oracle Cloud Infrastructure. It can connect to the Database(s) running outside of Oracle Cloud Infrastructure to perform remote capture/remote delivery. This is done through sqlnet connection. Hence, appropriate security/firewall setting is needed to make the remote capture and remote delivery work. The following scenarios need special consideration.

The remote agent is deployed in Oracle Cloud at Customer (OCC).
A sqlnet connection is needed from GoldenGate instance running in Oracle Cloud Infrastructure to the Database running in OCC. GoldenGate will use remote capture/remote delivery when replacing the DIPC remote agent. If this sqlnet connection can’t be established, you can’t use GoldenGate Marketplace to replace Data Integration Platform Cloud.
The remote agent is deployed at customer on-premise or any third party Cloud.
A sqlnet connection is needed from GoldenGate instance running in Oracle Cloud Infrastructure to the Database running on premise or third party cloud. GoldenGate will use remote capture/remote delivery when replacing the Data Integration Platform Cloud remote agent. If this sqlnet connection can’t be established, you can’t use GoldenGate Marketplace to replace Data Integration Platform Cloud.
Data Integration Platform Cloud Classic customer runs GoldenGate directly on Data Integration Platform Cloud host.
This will be the similar use case for GGCS. You should be able to follow the GGCS migration steps. For remote agent use cases, it will be the same consideration as described above.
Customized GoldenGate Configuration under remote agent.
Any GoldenGate tasks/components configured manually under the remote agent, other than elevated tasks will not be covered by this migration guideline. For example, XAG integration, Downstream mining, GoldenGate Bigdata configuration, etc. Oracle suggests customer engage service team to review the migration plan.
Miscellaneous
If you have patched the GoldenGate manually under your Data Integration Platform Cloud or Data Integration Platform Cloud remote agent, i.e., not through the Data Integration Platform Cloud patching process, please double-check the fixed bug is included in the latest GoldenGate Marketplace. In most cases, it should be. If you are not sure, please open a service request (SR) to verify with Oracle GoldenGate support team.