4 Migrating Your Database with Zero Downtime Migration
Evaluate the database migration job, run the job, and perform other operations during and after a database migration.
See the Zero Downtime Migration Release Notes for the latest information about known issues, My Oracle Support notes, and runbooks.
Evaluate the Migration Job
Zero Downtime Migration provides options and tools for evaluating the migration job before you run it against the production database.
In addition be sure the following tasks are done:
-
Obtain the necessary access credentials required.
If Oracle Cloud Infrastructure Object Storage is used as the backup medium, obtain the Object Storage access credential. The user ID for the Oracle Cloud Infrastructure Console user and an auth token for Object Storage is required. If you are not using an existing auth token, a new auth token can be generated using the Oracle Cloud Infrastructure Console.
If the source database server is accessed with the root user, then you need the root user password. If the source and target database serves are accessed with a private key file, then you need the private key file. The SYS password for the source database environment is also required.
If Zero Data Loss Recovery Appliance is used as the backup medium, get the Zero Data Loss Recovery Appliance virtual private catalog (VPC) user credentials.
-
Prepare the Zero Downtime Migration response file.
The database migration is driven by a response file that captures the essential parameters for accomplishing the task.
Use the sample RSP templates in
$ZDM_HOME/rhp/zdm/template/
file for example entries needed to set up the response file for your particular source, target, and backup environments.
The ZDMCLI migrate database
command has options to let you test the
migration before you run it in production. The options are highlighted in the
following syntax examples.
ZDMCLI migrate database
syntax for an Autonomous
Database migration:
zdmuser> $ZDM_HOME/bin/zdmcli migrate database
-rsp file_path
-sourcedb source_db_unique_name_value
-sourcenode host
-srcauth zdmauth
-srcarg1 user:username
-srcarg2 identity_file:ssh_key_path
-srcarg3 sudo_location:sudo_path
-eval [-advisor [-ignoreadvisor] | -skipadvisor]]
ZDMCLI migrate database
syntax for a co-managed
database migration:
zdmuser> $ZDM_HOME/bin/zdmcli migrate database
-rsp file_path
-sourcedb source_db_unique_name_value
-sourcenode host
-srcauth zdmauth
-srcarg1 user:username
-srcarg2 identity_file:ssh_key_path
-srcarg3 sudo_location:sudo_path
-targetnode host
-tgtauth zdmauth
-tgtarg1 user:username
-tgtarg2 identity_file:ssh_key_path
-tgtarg3 sudo_location:sudo_path
-eval [-advisor [-ignoreadvisor] | -skipadvisor]]
migrate
database
options to evaluate your migration.
Using the ZDMCLI MIGRATE DATABASE -eval Option
Before submitting the database migration job for the production database, perform a test migration to determine how the process may fare with your configuration and settings.
It is highly recommended that for each migration you run migrate
database
in evaluation mode first. This evaluation allows you to
correct any potential problems in the setup and configuration before performing the
actual migration on a production database.
In evaluation mode, the migration process runs without effecting the
changes. It is safe to run the command with the -eval
option as
many times as needed before running the actual migration job.
The command result output indicates the job ID for the evaluation migration job, which you can use to query the status of the job.
To run an evaluation of the migration process, run the ZDMCLI command
migrate database
with the -eval
option, as
shown in the following example.
Log in to the Zero Downtime Migration service host and switch to the
zdmuser
installed user.
su - zdmuser
If connectivity to the source database server is done through root credentials then the command would be the following:
zdmuser> $ZDM_HOME/bin/zdmcli migrate database -sourcedb source_db_unique_name_value
-sourcenode source_database_server_name -srcroot
-targetnode target_database_server_name
-backupuser Object_store_login_user_name
-rsp response_file_location
-tgtauth zdmauth
-tgtarg1 user:target_database_server_login_user_name
-tgtarg2 identity_file:ZDM_installed_user_private_key_file_location
-tgtarg3 sudo_location:/usr/bin/sudo -eval
For the prompts, specify the source database SYS password and the source database server root user password. If the backup destination is Object Store (Bucket), then specify user swift authentication token. If the backup destination is Storage Classic (Container) then specify your tenancy login password.
For example,
zdmuser> $ZDM_HOME/bin/zdmcli migrate database -sourcedb zdmsdb -sourcenode ocicdb1
-srcroot -targetnode ocidb1 -backupuser backup_user@example.com
-rsp /u01/app/zdmhome/rhp/zdm/template/zdm_template_zdmsdb.rsp -tgtauth zdmauth
-tgtarg1 user:opc -tgtarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk -tgtarg3
sudo_location:/usr/bin/sudo -eval
Enter source database zdmsdb SYS password:
Enter source user "root" password:
Enter user "backup_user@example.com" password:
If connectivity to the source database server is through SSH key, then the command would be:
zdmuser> $ZDM_HOME/bin/zdmcli migrate database -sourcedb source_db_unique_name_value
-sourcenode source_database_server_name -srcauth zdmauth
-srcarg1 user:source_database_server_login_user_name
-srcarg2 identity_file:ZDM_installed_user_private_key_file_location
-srcarg3 sudo_location:/usr/bin/sudo -targetnode target_database_server_name
-backupuser Object_store_login_user_name -rsp response_file_location
-tgtauth zdmauth -tgtarg1 user:target_database_server_login_user_name
-tgtarg2 identity_file:ZDM_installed_user_private_key_file_location
-tgtarg3 sudo_location:/usr/bin/sudo -eval
For the prompts, specify the source database SYS password. If the backup destination is Object Store (Bucket), then specify user swift authentication token. If the backup destination is Storage Classic (Container), then specify your tenancy login password.
zdmuser> $ZDM_HOME/bin/zdmcli migrate database -sourcedb zdmsdb -sourcenode ocicdb1 -srcauth zdmauth
-srcarg1 user:opc -srcarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk
-srcarg3 sudo_location:/usr/bin/sudo -targetnode ocidb1 -backupuser backup_user@example.com
-rsp /u01/app/zdmhome/rhp/zdm/template/zdm_template_zdmsdb.rsp -tgtauth zdmauth -tgtarg1 user:opc
-tgtarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk -tgtarg3 sudo_location:/usr/bin/sudo -eval
Enter source database zdmsdb SYS password:
Enter user "backup_user@example.com" password:
Note that if a source single instance database is deployed without a Grid
Infrastructure home, then in the above command use -sourcesid
in
place of -sourcedb
.
Also, if a source database is configured for a PASSWORD
based wallet, then add the -tdekeystorepasswd
option to the command
above, and for the prompt, specify the source database TDE keystore password
value.
Note that the –backupuser
argument takes the Object
Storage access user or Zero Data Loss Recovery Appliance VPC user, and is skipped if
NFS is the backup medium. For NFS, the source database user should have ‘rwx’ access
to the NFS path provided.
The migration command checks for patch compatibility between the source
and target home patch level, and expects the target home patch level to be equal to
or higher than the source. If the target home patch level is not as expected, then
the migration job is stopped and missing patches are reported. You can either patch
the target home with the necessary patches or you can force continue the migration
by appending the –ignore PATCH_CHECK
or -ignore
ALL
option to the migration command.
The command result output indicates the job ID for the migration job, which you can use to query the status of the job.
If you want to run the command without providing passwords at the command line, see Provide Passwords Non-Interactively Using a Wallet.
Using the Cloud Premigration Advisor Tool
The Cloud Premigration Advisor Tool (CPAT) is a tool that performs analysis of the source database, looking for uses of database features and constructs that are problematic when migrating to one of Oracle's Autonomous Cloud offerings.
CPAT integrated with Zero Downtime Migration, and can be used with logical migration jobs. CPAT provides the following benefits:
- Warns you about any features used by you database that aren't supported in the target cloud environment
- Makes suggestions for remedial changes and/or parameters to use for the Data Pump export and import operations
When you run a logical migration using ZDMCLI MIGRATE
DATABASE
, CPAT is run by default as phase
ZDM_PRE_MIGRATION_ADVISOR
.
The following are options you can use with ZDMCLI MIGRATE
DATABASE
to customize how CPAT runs or to skip the CPAT phase.
advisor
only runs the minimum phases required for exclusively running Cloud Premigration Advisor Tool (CPAT) on the migration.ignoreadvisor
ignores problems or errors reported by CPATskipadvisor
skips the CPAT phase in a migration job
See migrate database for more information about the command options, and see Cloud Premigration Advisor Tool (CPAT) Analyzes Databases for Suitability of Cloud Migration (Doc ID 2758371.1) for more information about CPAT.
Migrate the Database
Perform the database migration with Zero Downtime Migration using the
ZDMCLI migrate database
command.
Ensure that you have met all of the prerequisites and completed the required preparations described in Preparing for Database Migration before you begin the migration procedures in this topic.
In particular be sure the following tasks are done:
-
Obtain the necessary access credentials required.
If Oracle Cloud Infrastructure Object Storage is used as the backup medium, obtain the Object Storage access credential. The user ID for the Oracle Cloud Infrastructure Console user and an auth token for Object Storage is required. If you are not using an existing auth token, a new auth token can be generated using the Oracle Cloud Infrastructure Console.
If the source database server is accessed with the root user, then you need the root user password. If the source and target database serves are accessed with a private key file, then you need the private key file. The SYS password for the source database environment is also required.
If Zero Data Loss Recovery Appliance is used as the backup medium, get the Zero Data Loss Recovery Appliance virtual private catalog (VPC) user credentials.
-
Prepare the Zero Downtime Migration response file.
The database migration is driven by a response file that captures the essential parameters for accomplishing the task.
Use the sample $ZDM_HOME/rhp/zdm/template/zdm_template.rsp file for example entries needed to set up the response file for your particular source, target, and backup environments.
-
Determine if the migration process needs to be paused and resumed before you start the database migration. Once the migration job is initiated the job system runs the job as configured.
If the migration job needs to pause and resume at a particular point, then see the topics List Migration Job Phases and Pause and Resume Migration Job (cross references below) for more details.
Physical Migration with Root Credentials
The database migration job is submitted from the Zero Downtime Migration service host
by the zdmuser
user using the ZDMCLI command migrate
database
.
For a physical migration, if connectivity to the source database server is through root credentials, then the command would be:
zdmuser> $ZDM_HOME/bin/zdmcli migrate database
-sourcedb source_db_unique_name_value
-sourcenode source_database_server_name
-srcroot
-targetnode target_database_server_name
-backupuser Object_store_login_user_name
-rsp response_file_location
-tgtauth zdmauth
-tgtarg1 user:target_database_server_login_user_name
-tgtarg2 identity_file:ZDM_installed_user_private_key_file_location
-tgtarg3 sudo_location:/usr/bin/sudo
For the prompts, specify the source database SYS password and source database server root user password. If the backup destination is Object Store (Bucket), then specify user swift authentication token. If the backup destination is Storage Classic (Container), then specify your tenancy login password.
For example:
zdmuser> $ZDM_HOME/bin/zdmcli migrate database
-sourcedb zdmsdb
-sourcenode ocicdb1
-srcroot
-targetnode ocidb1
-backupuser backup_user@example.com
-rsp /u01/app/zdmhome/rhp/zdm/template/zdm_template_zdmsdb.rsp
-tgtauth zdmauth
-tgtarg1 user:opc
-tgtarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk
-tgtarg3 sudo_location:/usr/bin/sudo
Enter source database zdmsdb SYS password:
Enter source user "root" password:
Enter user "backup_user@example.com" password:
Physical Migration with SSH Key
For a physical migration, if connectivity to the source database server is through SSH key, then the command would be:
zdmuser> $ZDM_HOME/bin/zdmcli migrate database
-sourcedb source_db_unique_name_value
-sourcenode source_database_server_name
-srcauth zdmauth
-srcarg1 user:source_database_server_login_user_name
-srcarg2 identity_file:ZDM_installed_user_private_key_file_location
-srcarg3 sudo_location:/usr/bin/sudo
-targetnode target_database_server_name
-backupuser Object_store_login_user_name
-rsp response_file_location
-tgtauth zdmauth
-tgtarg1 user:target_database_server_login_user_name
-tgtarg2 identity_file:ZDM_installed_user_private_key_file_location
-tgtarg3 sudo_location:/usr/bin/sudo
For the prompts, specify the source database SYS password. If the backup destination is Object Store (Bucket), then specify user swift authentication token. If the backup destination is Storage Classic (Container), then specify your tenancy login password.
For example,
zdmuser> $ZDM_HOME/bin/zdmcli migrate database
-sourcedb zdmsdb
-sourcenode ocicdb1
-srcauth zdmauth
-srcarg1 user:opc
-srcarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk
-srcarg3 sudo_location:/usr/bin/sudo
-targetnode ocidb1
-backupuser backup_user@example.com
-rsp /u01/app/zdmhome/rhp/zdm/template/zdm_template_zdmsdb.rsp
-tgtauth zdmauth
-tgtarg1 user:opc
-tgtarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk
-tgtarg3 sudo_location:/usr/bin/sudo
Enter source database zdmsdb SYS password:
Enter user "backup_user@example.com" password:
If a source single instance is deployed without a Grid Infrastructure
home, then in the command above use -sourcesid
in place of
-sourcedb
.
If the source database is configured for a PASSWORD
based wallet, then add the -tdekeystorepasswd
option to the command
above, and for the prompt, specify the source database TDE keystore password
value.
Note that the –backupuser
argument takes the Object
Storage access user or Zero Data Loss Recovery Appliance VPC user and is skipped if
NFS is the backup medium. For NFS, the source database user should have ‘rwx’ access
to the NFS path provided.
Logical Migration to Autonomous Database
For a logical migration to Autonomous Database, the command would be:
zdmuser> $ZDM_HOME/bin/zdmcli migrate database
-rsp file_path
-sourcedb source_db_unique_name_value
-sourcenode host
-srcauth zdmauth
-srcarg1 user:username
-srcarg2 identity_file:ssh_key_path
-srcarg3 sudo_location:sudo_path
-eval [-advisor [-ignoreadvisor] | -skipadvisor]]
Logical Migration to a Co-Managed Database
For a logical migration to a co-managed system, the command would be:
zdmuser> $ZDM_HOME/bin/zdmcli migrate database
-rsp file_path
-sourcedb source_db_unique_name_value
-sourcenode host
-srcauth zdmauth
-srcarg1 user:username
-srcarg2 identity_file:ssh_key_path
-srcarg3 sudo_location:sudo_path
-targetnode host
-tgtauth zdmauth
-tgtarg1 user:username
-tgtarg2 identity_file:ssh_key_path
-tgtarg3 sudo_location:sudo_path
[-ignoreadvisor | -skipadvisor]
Patch Compatibility
The migration command checks for patch compatibility between the source
and target home patch level, and expects the target home patch level to be equal to
or higher than the source. If the target home patch level is not as expected, then
the migration job is stopped and missing patches are reported. You can either patch
the target home with the necessary patches or you can force continue the migration
by appending the –ignore PATCH_CHECK
or -ignore
ALL
option to the migration command.
Job ID Value
The command result output indicates the job ID for the migration job, which you can use to query the status of the job.
Running Migrations Non-Interactively
If you want to run the command without providing passwords at the command line, see Provide Passwords Non-Interactively Using a Wallet.
Query Migration Job Status
You can query the migration job status while the job is running.
Query the status of a database migration job using the ZDMCLI query job
command, specifying the job ID. The job ID is shown in the command output when the database migration job is submitted.
zdmuser> $ZDM_HOME/bin/zdmcli query job -jobid job-id
You can find the console output of the migration job in the file indicated (Result file
path:) in the query job
command output. You can see migration progress
messages in the specified file
List Migration Job Phases
You can list the operation phases involved in the migration job.
To list the operation phases involved in the migration job, add the -listphases
option in the ZDMCLI migrate
command. This option will list the phases involved in the operation.
For example,
zdmuser> $ZDM_HOME/bin/zdmcli migrate database -sourcedb zdmsdb -sourcenode ocicdb1 -srcauth zdmauth
-srcarg1 user:opc -srcarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk -srcarg3 sudo_location:/usr/bin/sudo
-targetnode ocidb1 -backupuser backup_user@example.com -rsp /u01/app/zdmhome/rhp/zdm/template/zdm_template_zdmsdb.rsp
-tgtauth zdmauth -tgtarg1 user:opc -tgtarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk
-tgtarg3 sudo_location:/usr/bin/sudo -listphases
Pause and Resume a Migration Job
You can pause a migration job at any point after the
ZDM_SETUP_TGT
phase, and resume the job at any
time.
To pause a migration job, specify the –pauseafter
option in the
ZDMCLI
migrate
command with a valid phase to be paused after.
In the following example, if you specify -pauseafter
ZDM_SETUP_TGT
, the migration job will pause after
completing the ZDM_SETUP_TGT
phase.
zdmuser> $ZDM_HOME/bin/zdmcli migrate database -sourcedb zdmsdb -sourcenode ocicdb1
-srcauth zdmauth -srcarg1 user:opc
-srcarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk
-srcarg3 sudo_location:/usr/bin/sudo -targetnode ocidb1
-backupuser backup_user@example.com -rsp /u01/app/zdmhome/rhp/zdm/template/zdm_template_zdmsdb.rsp -tgtauth zdmauth
-tgtarg1 user:opc -tgtarg2 identity_file:/home/zdmuser/.ssh/zdm_service_host.ppk
-tgtarg3 sudo_location:/usr/bin/sudo -pauseafter ZDM_SETUP_TGT
Choosing a Migration Job Phase to Pause After
Choose a valid phase after ZDM_CONFIGURE_DG_SRC
that is listed in the migrate database ... -listphases
command output.
Note that the -pauseafter
option allows only one
phase to be specified.
The following are some cases where you may need to pause a migration job, and resume after some manual steps:
- Convert the Standby Database to TDE. If the source is not encrypted, of if ZDLRA is used as the data transfer medium, or the Oracle Database version is earlier than 12.2, you need to encrypt the target Cloud database.
- Enable Active Data Guard (optional)
- Monitor Data Guard Configuration Health, before the switchover
- Test the Cloud database (optional). You can convert the standby to the primary without doing the application switchover, which can be used to duplicate the source database for testing in the Cloud.
Best Practices for Pausing a Physical Migration Job
In a physical migration, if you use -pauseafter
at
phase ZDM_CONFIGURE_DG_SRC
, then at the end of the
execution of the phase, a standby is created at the target database and
synchronization occurs between source and target databases.
Pausing the migration job after
ZDM_CONFIGURE_DG_SRC
is recommended so that you can
do the following:
- Perform the application switchover to the cloud
- Manually encrypt the cloud database if ZDLRA is the backup method or your database release is earlier than Oracle Database 12.2
- Perform a failover to test the cloud database without making changes to the on-premises database
Preserving Log Files During a Paused Migration Job
To prevent source and target database log files from getting cleaned
up between pausing and resuming a migration job, log files are written to
$ORACLE_BASE/zdm/zdm_db_unique_name_zdm_job_id/zdm/log
in their respective
source and target database servers.
Ensure that all of the archive logs generated during and after the
ZDM_BACKUP_INCREMENTAL_SRC
phase are available,
preferably until switchover, or at least until the target and source
databases are in sync. Older archive logs, the archive logs generated prior
to ZDM_BACKUP_INCREMENTAL_SRC
are not needed.
Resuming a Migration Job
A paused job can be resumed any time by running the ZDMCLI
resume job
command, specifying the respective job ID.
zdmuser> $ZDM_HOME/bin/zdmcli resume job -jobid Job_ID
[-pauseafter valid-phase]
To schedule another pause, specify the –pauseafter
option in the resume
command with a valid phase to be
paused after. Choose a valid phase later than phase currently paused at,
that is listed in the migrate database ... -listphases
command output.
Suspend and Resume a Migration Job
You can suspend a migration job at any point, and resume the job at any time.
To suspend a migration job run the following command.
zdmuser> $ZDM_HOME/bin/zdmcli suspend job -jobid job_id
Resuming a Suspended Migration Job
A suspended job can be resumed any time by running the ZDMCLI
resume job
command, specifying the respective job ID.
zdmuser> $ZDM_HOME/bin/zdmcli resume job -jobid Job_ID
[-pauseafter valid-phase]
You can optionally schedule a pause, by specifying the
–pauseafter
option in the resume
command with a valid phase to be paused after. Choose a valid phase later
than phase currently paused at, that is listed in the migrate
database ... -listphases
command output.
Rerun a Migration Job
If there are any unexpected errors in the migration workflow, you can correct them and rerun the migration job.
The errors are recorded in the job output, which can be queried using the ZDMCLI query job
command. Upon resolving the error, the failed job can be continued from the point of failure.
Rerun the migration job by running the ZDMCLI resume job
command, specifying the job ID of the job to be rerun, as shown here.
zdmuser> $ZDM_HOME/bin/zdmcli resume job -jobid Job_ID
Terminate a Running Migration Job
If you want to resubmit a database migration job for a specified database, you must first terminate the running migration job.
Zero Downtime Migration blocks attempts to rerun the MIGRATE
DATABASE
command for a specified database if that database is already
part of an ongoing migration job.
If you want to resubmit a database migration job for a specified
database, you must first terminate the running migration job in either
EXECUTING
or PAUSED
state using the
ZDMCLI ABORT JOB
command.
zdmuser> $ZDM_HOME/bin/zdmcli abort job -jobid job-id
Zero Downtime Migration Centralized Fleet Migration Management
Zero Downtime Migration allows you to centrally monitor fleet level migration using the Object Storage Service PAR URL.
Zero Downtime centralized fleet migration management gives you the following capabilities.
- Fleet level migration monitoring
- Fleet level log aggregation for easy troubleshooting and data mining
- Centralized migration per job metrics collection, which can be leveraged for executive-level migration status dash-boards and future migration forecasting
Centralized fleet migration management also lets you to prohibit the
operation team from accessing the source or target database
server for migration failures. The operation team can be allowed
to troubleshoot the failure with the logs available in the
specified ZDM_LOG_OSS_PAR_URL
or logs further
staged to a different Oracle Cloud storage bucket for operation
troubleshooting.
Centralized fleet migration management is only supported for physical
migration jobs, and is enabled by setting the parameter
ZDM_LOG_OSS_PAR_URL
to a pre-authenticated
URL, for example:
https://objectstorage.us-region.oraclecloud.com/ … /DEV_ZDM_LOGS_RGN/o/
Zero Downtime Migration uploads the job specific data, shown in the table below, to the specified OSS PAR URL at regular interval while a migration job is in progress.
Table 4-1 Centralized Fleet Migration Job Data
Category | URL Name space Suffix (Appended to value of ZDM_LOG_OSS_PAR_URL) | Content |
---|---|---|
Job Metrics |
For example,
|
Job Metrics details the source and target database vitals and backup and restore statistics per phase involved in the work flow and error. This data can be used for fleet level migration statistics for executive dashboards. |
Job Status |
For example,
|
Status entries are listed in the format
For example,
Phase Status values are |
Phase Log |
PAR_URL/2/TIMESTAMP/POD_NAME/ZDM_HOST/JOB_ID/ACTION HOST/PHASE_NAME.log For example, 2/2011-11-07T17:58:34.049000/PRD_TEST_MIGRATION/2-zdm-01/1/cldx01/ZDM_BACKUP_FULL_SRC.log |
Zero Downtime Migration uploads the phase-specific logs at the end of each phase with a UTC time-stamped name space. On job rerun, the time stamp would change and the rerun-specific log would be identified by latest time stamp. |
Phase Status |
For example,
|
The file contents will be At the start of a migration job, Zero Downtime Migration creates all of
the phase-named files with Use the creation time of |
Progress Message |
For example,
|
Zero Downtime Migration updates Progress messages are formatted like this:
For example, |