Migrating Your Database with Zero Downtime Migration

4 Migrating Your Database with Zero Downtime Migration

Evaluate the database migration job, run the job, and perform other operations during and after a database migration.

See the Zero Downtime Migration Release Notes for the latest information about known issues, My Oracle Support notes, and runbooks.

Migrate the Database
Perform the database migration with Zero Downtime Migration using the following procedure.
Query Migration Job Status
You can query the migration job status while the job is running.
List Migration Job Phases
You can list the operation phases involved in the migration job.
Pause and Resume a Migration Job
You can pause a migration job at any point after the ZDM_SETUP_TGT phase, and resume the job at any time.
Rerun a Migration Job
If there are any unexpected errors in the migration workflow, you can correct them and rerun the migration job.
Terminate a Running Migration Job
If you want to resubmit a database migration job for a specified database, you must first terminate the running migration job.
Post-Migration Tasks
The following topics describe tasks that you do after you complete the database migration job.

Migrate the Database

Perform the database migration with Zero Downtime Migration using the following procedure.

Ensure that you have met all of the prerequisites and completed the required preparations described in Preparing for Database Migration before you begin the migration procedures in this topic.

Obtain the necessary access credentials required.
If Oracle Cloud Infrastructure Object Storage is used as the backup medium, obtain the Object Storage access credential. The user ID for the Oracle Cloud Infrastructure Console user and an auth token for Object Storage is required. If you are not using an existing auth token, a new auth token can be generated using the Oracle Cloud Infrastructure Console.
If the source database server is accessed with the root user, then you need the root user password. If the source and target database serves are accessed with a private key file, then you need the private key file. The SYS password for the source database environment is also required.

If Zero Data Loss Recovery Appliance is used as the backup medium, get the Zero Data Loss Recovery Appliance virtual private catalog (VPC) user credentials.
Prepare the Zero Downtime Migration response file.
The database migration is driven by a response file that captures the essential parameters for accomplishing the task. Use the sample $ZDM_HOME/rhp/zdm/template/zdm_template.rsp file for example entries needed to set up the response file for your particular source, target, and backup environments.
Evaluate the database migration process.
Before submitting the database migration job for the production database, perform a test migration to determine how the process may fare with your configuration and settings. It is highly recommended that for each migration you run migrate database in evaluation mode first. This evaluation allows you to correct any potential problems in the setup and configuration before performing the actual migration on a production database.
In evaluation mode, the migration process runs without effecting the changes. It is safe to run the command with the -eval option as many times as needed before running the actual migration job.

The command result output indicates the job ID for the evaluation migration job, which you can use to query the status of the job.

To run an evaluation of the migration process, run the ZDMCLI command migrate database with the -eval option, as shown in the following example.

Log in to the Zero Downtime Migration service host and switch to the zdmuser installed user.
```
su - zdmuser
```
If connectivity to the source database server is done through root credentials then the command would be the following:
```
zdmuser> $ZDM_HOME/bin/zdmcli migrate database -sourcedb source_db_unique_name_value
-sourcenode source_database_server_name -srcroot 
-targetnode target_database_server_name 
-backupuser Object_store_login_user_name  
-rsp response_file_location
-tgtauth zdmauth 
-tgtarg1 user:target_database_server_login_user_name 
-tgtarg2 identity_file:ZDM_installed_user_private_key_file_location
-tgtarg3 sudo_location:/usr/bin/sudo -eval
```
For the prompts, specify the source database SYS password and the source database server root user password. If the backup destination is Object Store (Bucket), then specify user swift authentication token. If the backup destination is Storage Classic (Container) then specify your tenancy login password.

For example,
```
zdmuser> $ZDM_HOME/bin/zdmcli migrate database -sourcedb zdmsdb -sourcenode ocicdb1
-srcroot -targetnode ocidb1 -backupuser backup_user@example.com
-rsp /u01/app/zdmhome/rhp/zdm/template/zdm_template_zdmsdb.rsp -tgtauth zdmauth 
-tgtarg1 user:opc -tgtarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk -tgtarg3
sudo_location:/usr/bin/sudo -eval

Enter source database zdmsdb SYS password:
Enter source user "root" password:
Enter user "backup_user@example.com" password:
```
If connectivity to the source database server is through SSH key, then the command would be:
```
zdmuser> $ZDM_HOME/bin/zdmcli migrate database -sourcedb source_db_unique_name_value
-sourcenode source_database_server_name -srcauth zdmauth 
-srcarg1 user:source_database_server_login_user_name 
-srcarg2 identity_file:ZDM_installed_user_private_key_file_location 
-srcarg3 sudo_location:/usr/bin/sudo -targetnode target_database_server_name 
-backupuser Object_store_login_user_name -rsp response_file_location 
-tgtauth zdmauth -tgtarg1 user:target_database_server_login_user_name 
-tgtarg2 identity_file:ZDM_installed_user_private_key_file_location  
-tgtarg3 sudo_location:/usr/bin/sudo -eval
```
For the prompts, specify the source database SYS password. If the backup destination is Object Store (Bucket), then specify user swift authentication token. If the backup destination is Storage Classic (Container), then specify your tenancy login password.
```
zdmuser> $ZDM_HOME/bin/zdmcli migrate database -sourcedb zdmsdb -sourcenode ocicdb1 -srcauth zdmauth 
-srcarg1 user:opc -srcarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk 
-srcarg3 sudo_location:/usr/bin/sudo -targetnode ocidb1 -backupuser backup_user@example.com 
-rsp /u01/app/zdmhome/rhp/zdm/template/zdm_template_zdmsdb.rsp -tgtauth zdmauth -tgtarg1 user:opc 
-tgtarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk -tgtarg3 sudo_location:/usr/bin/sudo -eval

Enter source database zdmsdb SYS password:
Enter user "backup_user@example.com" password:
```
Note that if a source single instance database is deployed without a Grid Infrastructure home, then in the above command use -sourcesid in place of -sourcedb.

Also, if a source database is configured for a PASSWORD based wallet, then add the -tdekeystorepasswd option to the command above, and for the prompt, specify the source database TDE keystore password value.

Note that the –backupuser argument takes the Object Storage access user or Zero Data Loss Recovery Appliance VPC user, and is skipped if NFS is the backup medium. For NFS, the source database user should have ‘rwx’ access to the NFS path provided.

The migration command checks for patch compatibility between the source and target home patch level, and expects the target home patch level to be equal to or higher than the source. If the target home patch level is not as expected, then the migration job is stopped and missing patches are reported. You can either patch the target home with the necessary patches or you can force continue the migration by appending the –ignore PATCH_CHECK or -ignore ALL option to the migration command.

The command result output indicates the job ID for the migration job, which you can use to query the status of the job.

If you want to run the command without providing passwords at the command line, see Provide Passwords Non-Interactively Using a Wallet.
Determine if the migration process needs to be paused and resumed before you start the database migration. Once the migration job is initiated the job system runs the job as configured.

If the migration job needs to pause and resume at a particular point, then see the topics List Migration Job Phases and Pause and Resume Migration Job (cross references below) for more details.
Start the database migration process.
The database migration job is submitted from the Zero Downtime Migration service host by the zdmuser user using the ZDMCLI command migrate database.
If connectivity to the source database server is through root credentials, then the command would be:
```
zdmuser> $ZDM_HOME/bin/zdmcli migrate database -sourcedb source_db_unique_name_value 
-sourcenode source_database_server_name -srcroot 
-targetnode target_database_server_name 
-backupuser Object_store_login_user_name 
-rsp response_file_location -tgtauth zdmauth 
-tgtarg1 user:target_database_server_login_user_name
-tgtarg2 identity_file:ZDM_installed_user_private_key_file_location  
-tgtarg3 sudo_location:/usr/bin/sudo
```
For the prompts, specify the source database SYS password and source database server root user password. If the backup destination is Object Store (Bucket), then specify user swift authentication token. If the backup destination is Storage Classic (Container), then specify your tenancy login password.

For example:
```
zdmuser> $ZDM_HOME/bin/zdmcli migrate database -sourcedb zdmsdb -sourcenode ocicdb1 -srcroot
-targetnode ocidb1 -backupuser backup_user@example.com -rsp /u01/app/zdmhome/rhp/zdm/template/zdm_template_zdmsdb.rsp
-tgtauth zdmauth -tgtarg1 user:opc -tgtarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk
-tgtarg3 sudo_location:/usr/bin/sudo

Enter source database zdmsdb SYS password:
Enter source user "root" password:
Enter user "backup_user@example.com" password:
```
If connectivity to the source database server is through SSH key, then the command would be:
```
zdmuser> $ZDM_HOME/bin/zdmcli migrate database -sourcedb source_db_unique_name_value 
-sourcenode source_database_server_name -srcauth zdmauth 
-srcarg1 user:source_database_server_login_user_name 
-srcarg2 identity_file:ZDM_installed_user_private_key_file_location 
-srcarg3 sudo_location:/usr/bin/sudo -targetnode target_database_server_name 
-backupuser Object_store_login_user_name -rsp response_file_location 
-tgtauth zdmauth -tgtarg1 user:target_database_server_login_user_name 
-tgtarg2 identity_file:ZDM_installed_user_private_key_file_location  
-tgtarg3 sudo_location:/usr/bin/sudo 
```
For the prompts, specify the source database SYS password. If the backup destination is Object Store (Bucket), then specify user swift authentication token. If the backup destination is Storage Classic (Container), then specify your tenancy login password.

For example,
```
zdmuser> $ZDM_HOME/bin/zdmcli migrate database -sourcedb zdmsdb -sourcenode ocicdb1 -srcauth zdmauth 
-srcarg1 user:opc -srcarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk 
-srcarg3 sudo_location:/usr/bin/sudo -targetnode ocidb1 -backupuser backup_user@example.com 
-rsp /u01/app/zdmhome/rhp/zdm/template/zdm_template_zdmsdb.rsp -tgtauth zdmauth -tgtarg1 user:opc 
-tgtarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk -tgtarg3 sudo_location:/usr/bin/sudo

Enter source database zdmsdb SYS password:
Enter user "backup_user@example.com" password:
```
If a source single instance is deployed without a Grid Infrastructure home, then in the command above use -sourcesid in place of -sourcedb.

If the source database is configured for a PASSWORD based wallet, then add the -tdekeystorepasswd option to the command above, and for the prompt, specify the source database TDE keystore password value.

Note that the –backupuser argument takes the Object Storage access user or Zero Data Loss Recovery Appliance VPC user and is skipped if NFS is the backup medium. For NFS, the source database user should have ‘rwx’ access to the NFS path provided.

The migration command checks for patch compatibility between the source and target home patch level, and expects the target home patch level to be equal to or higher than the source. If the target home patch level is not as expected, then the migration job is stopped and missing patches are reported. You can either patch the target home with the necessary patches or you can force continue the migration by appending the –ignore PATCH_CHECK or -ignore ALL option to the migration command.
The command result output indicates the job ID for the migration job, which you can use to query the status of the job.

If you want to run the command without providing passwords at the command line, see Provide Passwords Non-Interactively Using a Wallet.

Query Migration Job Status

You can query the migration job status while the job is running.

Query the status of a database migration job using the ZDMCLI query job command, specifying the job ID. The job ID is shown in the command output when the database migration job is submitted.

zdmuser> $ZDM_HOME/bin/zdmcli query job -jobid job-id

You can find the console output of the migration job in the file indicated (Result file path:) in the query job command output. You can see migration progress messages in the specified file

Parent topic: Migrating Your Database with Zero Downtime Migration

List Migration Job Phases

You can list the operation phases involved in the migration job.

To list the operation phases involved in the migration job, add the -listphases option in the ZDMCLI migrate command. This option will list the phases involved in the operation.

For example,

zdmuser> $ZDM_HOME/bin/zdmcli migrate database -sourcedb zdmsdb -sourcenode ocicdb1 -srcauth zdmauth
-srcarg1 user:opc -srcarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk -srcarg3 sudo_location:/usr/bin/sudo
-targetnode ocidb1 -backupuser backup_user@example.com -rsp /u01/app/zdmhome/rhp/zdm/template/zdm_template_zdmsdb.rsp
-tgtauth zdmauth -tgtarg1 user:opc -tgtarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk
-tgtarg3 sudo_location:/usr/bin/sudo -listphases

Parent topic: Migrating Your Database with Zero Downtime Migration

Pause and Resume a Migration Job

You can pause a migration job at any point after the ZDM_SETUP_TGT phase, and resume the job at any time.

To pause a migration job, specify the –pauseafter option in the ZDMCLI migrate command with a valid phase to be paused after.

In the following example, if you specify -pauseafter ZDM_SETUP_TGT, the migration job will pause after completing the ZDM_SETUP_TGT phase.

zdmuser> $ZDM_HOME/bin/zdmcli migrate database -sourcedb zdmsdb -sourcenode ocicdb1
-srcauth zdmauth -srcarg1 user:opc
-srcarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk
-srcarg3 sudo_location:/usr/bin/sudo -targetnode ocidb1 
-backupuser backup_user@example.com -rsp /u01/app/zdmhome/rhp/zdm/template/zdm_template_zdmsdb.rsp -tgtauth zdmauth 
-tgtarg1 user:opc -tgtarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk
-tgtarg3 sudo_location:/usr/bin/sudo -pauseafter ZDM_SETUP_TGT

Choosing a Migration Job Phase to Pause After

Choose a valid phase that is listed in the migrate database ... -listphases command output.

Note that the -pauseafter option allows only one phase to be specified.

Pausing the migration job after ZDM_SETUP_TGT is recommended.

If you use -pauseafter at phase ZDM_CONFIGURE_DG_SRC, then at the end of the execution of the phase, a standby is created at the target database and synchronization occurs between source and target databases.

Preserving Log Files During a Paused Migration Job

To prevent source and target database log files from getting cleaned up between pausing and resuming a migration job, log files are written to $ORACLE_BASE/zdm/zdm_db_unique_name_zdm_job_id/zdm/log in their respective source and target database servers.

Resuming a Migration Job

A paused job can be resumed any time by running the ZDMCLI resume job command, specifying the respective job ID.

zdmuser> $ZDM_HOME/bin/zdmcli resume job -jobid Job_ID 
[-pauseafter valid-phase]

To schedule another pause, specify the –pauseafter option in the resume command with a valid phase to be paused after. Choose a valid phase later than phase currently paused at, that is listed in the migrate database ... -listphases command output.

Parent topic: Migrating Your Database with Zero Downtime Migration

Rerun a Migration Job

If there are any unexpected errors in the migration workflow, you can correct them and rerun the migration job.

The errors are recorded in the job output, which can be queried using the ZDMCLI query job command. Upon resolving the error, the failed job can be continued from the point of failure.

Rerun the migration job by running the ZDMCLI resume job command, specifying the job ID of the job to be rerun, as shown here.

zdmuser> $ZDM_HOME/bin/zdmcli resume job -jobid Job_ID

Parent topic: Migrating Your Database with Zero Downtime Migration

Terminate a Running Migration Job

If you want to resubmit a database migration job for a specified database, you must first terminate the running migration job.

Zero Downtime Migration blocks attempts to rerun the MIGRATE DATABASE command for a specified database if that database is already part of an ongoing migration job.

If you want to resubmit a database migration job for a specified database, you must first terminate the running migration job in either EXECUTING or PAUSED state using the ZDMCLI ABORT JOB command.

zdmuser> $ZDM_HOME/bin/zdmcli abort job -jobid job-id

Parent topic: Migrating Your Database with Zero Downtime Migration

Post-Migration Tasks

The following topics describe tasks that you do after you complete the database migration job.

Run Datapatch on the Target Database
If the target database environment is at a higher patch level than the source database, you must run the datapatch utility on the target database. Skip this task if you set TGT_SKIP_DATAPATCH=FALSE in the response file and datapatch was run as part of the migration job.

Parent topic: Migrating Your Database with Zero Downtime Migration

Run Datapatch on the Target Database

If the target database environment is at a higher patch level than the source database, you must run the datapatch utility on the target database. Skip this task if you set TGT_SKIP_DATAPATCH=FALSE in the response file and datapatch was run as part of the migration job.

For example, if your source database is at Jan 2020 PSU/BP and the target is at April 2020 PSU/BP), you must run the datapatch utility. Before running datapatch on the target, ensure you apply the target patch level to the binaries at the source (standby) database.

If you are running a multitenant architecture, open the PDBs.
```
SQL> alter pluggable database all open;
```
It is recommended that you run datapatch on all of the PDBs; however, if you only want to open a subset of the PDBs in the CDB, you can use the following command instead. Datapatch only runs on the CDB and opened PDBs.
```
SQL> alter pluggable database PDB_NAME open
```
To run datapatch on a PDB later (previously skipped or newly plugged in), open the database using the alter pluggable database command and rerun the datapatch utility.
Go to the OPatch directory in ORACLE_HOME and run the datapatch utility.
```
% cd $ORACLE_HOME/OPatch
% ./datapatch -verbose
```
The datapatch utility runs the necessary apply scripts to load the modified SQL files into the database. An entry is added to the dba_registry_sqlpatch view indicating the patch application.
Check for errors.
Error logs are located in the $ORACLE_BASE/cfgtoollogs/sqlpatch/patch#/unique patch ID directory in the following format:
patch#_apply_database_SID_CDB_name_timestamp.log

where database_SID is the database SID, CDB_name is the name of the multitenant container database, and timestamp is in the format YYYYMMMDD_HH_MM_SS.

Parent topic: Post-Migration Tasks