Cross Platform Enterprise Manager Repository Database Migration

In the ever evolving data management landscape, businesses frequently encounter the need to migrate databases across different endian platforms. Database migration is a critical operation, often needed due to system upgrades, cloud adoption, or a shift in business requirements. In the context of cross platform and cross endian migration, the challenges are higher as data structures, byte orders, and database formats vary between systems. In the realm of database migration, using Oracle Data Pump technology enables very high-speed movement of data from one database to another across diverse platforms and endian architectures. Whether transitioning between on-premises servers and the cloud, or migrating between different database management systems, a robust data pump mechanism becomes crucial for maintaining data integrity, security, and system performance.

The Enterprise Manager Repository, being the heartbeat of Oracle Enterprise Manager, contains vital metadata and performance data essential for effective monitoring and management of an organization's IT ecosystem. When contemplating a migration across platforms or endian architectures, a meticulous strategy is paramount to ensure the integrity and accessibility of this critical repository.

Data pump technologies play a pivotal role in facilitating the seamless movement of data during these migrations. As we embark on this exploration, this section reviews the complexities of cross platform cross endian migrations within the context of the Enterprise Manager Repository database. It introduces the RepMigrate utility and defines the process that can be followed by administrators for successful and reliable migration of the repository database in minimum time and with maximum efficiency.

Oracle recommends using the RepMigrate utility for Enterprise Manager repository database migrations.

The overall strategy for migration depends on:

The source and target database version
The amount of data and/or size of the repository
Actual data to migrate

For Oracle database details about cross platform transportable tablespace, data pump, and export/import options, see the Oracle Database Documentation.

Topics:

Introduction to RepMigrate

Oracle recommends using RepMigrate utility for Enterprise Manager repository database migration.

RepMigrate is a wrapper utility that is written on top of Oracle Data Pump to ease the cross platform cross endian repository database migration for Enterprise Manager administrators. The utility underlying uses Oracle Data Pump, a feature of Oracle Database since release 10g and successor to the Oracle Export and Import (exp and imp) utilities in release 9i and earlier. Oracle Data Pump is useful for migrating data among schemas, databases of different versions, and on different operating systems, and from on-premises to on-premises and to Oracle Cloud.

How to Get RepMigrate Utility?

Starting with Enterprise Manager 13c Release 5 Update 22 (13.5.0.22), the Release Update file includes the RepMigrate utility (RepMigrate.zip file).

To obtain the RepMigrate utility do the following:

Download the Release Update file from My Oracle Support and extract it.
After extracting the Release Update, locate the RepMigrate.zip file.
Copy the RepMigrate.zip file to the source database.
Copy the file to a location in the repository database host (source database) and unzip it.

Note:
You also need to do the same: copy and unzip the file in the target database.
The RepMigrate utility version used for the Enterprise Manager repository database migration process must be on the same version as the Enterprise Manager Release Update (RU) deployed.
After unzipping the file, you can see the RepMigrate utility directory that you can use for the repository database migration process during the export and import steps as explained in the next sections.

To learn more about RepMigrate parameters, see RepMigrate Usage.

RepMigrate Usage

Starting with Enterprise Manager 13c Release 5 Update 22 (13.5.0.22), the RepMigrate utility is included in the Release Update file and available for performing Enterprise Manager repository database migration.

To list all the RepMigrate available options, run:

./RepMigrate.sh -help

List of Some Useful Parameters:

-datafileLocation: Specifies the datafile options.

This is a mandatory parameter for ASM and RAC environments.
-encryptionPassword: Specifies the password of the encrypted column data, metadata, or table data in the export dump file of the source database for TDE environment.

Ensure to pass the same password during the import operation in the target database.
-parallel: Specifies the maximum number of processes of active execution operating on behalf of the export and/or import job.
-prereqOnly: Specifies to run only the RepMigrate prerequisites check during the export operation. For details, see Step 3. Run Prerequisites Check.
-ignorePrereqList: Ignores the prerequisites check.
-repmigrate_prereq_dbpatch: Checks if the database patch has been applied or not.

For RAC environment, use the connect string-based connection on the first node for the RAC setup. Scan address is not supported.

For SSL configured repository, use RepMigrate.sh -help to review the options available.

The RepMigrate utility checks for TEMP and UNDO tablespaces. If there's not enough space available, it will fail.

Repository Migration Steps Using RepMigrate

The following sections discuss the steps for repository migration using RepMigrate utility:

Step 1. Prepare Databases

Complete the following prerequisites to migrate the repository database using RepMigrate:

Step 1.1. Perform Common Prerequisites for Source and Target Databases

The following lists the prerequisites for both the source and target databases when using RepMigrate utility for repository migration:

Database Patches Prerequisites

Apply all the database patches to both the source and target databases as necessary.

Note:

Both databases, source and target, should be at the same Database Release Update (DB RU) version with all the patches applied.

Install the database patch 30978304 on both source and target databases before starting the repository migration process. For database patch information, see My Oracle Support.
Apply the recommended Oracle Database 19 Release Update 19 (19.19) patches.

Note:
Some database patches may be included as part of the Database Release Update (DB RU) patches. Oracle recommends to check if the database patches are listed in your inventory. If you are missing database patches, check My Oracle Support and see if they are available. If database patches are not available for their release update (RU), you may need to request a one-off patch.

Oracle Database 19 Release Update 19 (19.19) patches:
- Patch 35042068: Database Release Update : 19.19.0.0.230418 (35042068)
- Patch 35261302: DATAPUMP BUNDLE PATCH 19.19.0.0.0
Oracle Database 19 Release Update (RU) patches specific to RU19 and the Data Pump utility:
- Patch 36205997: MAX PGA LIMIT FLAG RESET IN 19C.
- Patch 36018313: MERGE ON DATABASE RU 19.19.0.0.0 OF 35261302 35854529 (Patch for Bug 35854529).
- Patch 33421125: MEMORY LEAK IN 'KOLASLASSIGN' AND PGA INCREASES WHEN PGA LIMIT IS HIGHER (Part of DB RU19 Patch 35042068).
- Patch 35799058: SLOW EXPDP OF PARTITIONED TABLE EM_METRIC_VALUES_E (ENTERPRISE MANAGER REPOSITORY).

Timezone Prerequisite

The timezone of the source and target databases should be same.

For example, you can check the timezone in Linux by running: timedatectl

Tablespace Considerations

Tablespace creation: Tablespaces need to be explicitly created using SMALLFILE. Preallocate the necessary data files.
Tablespace size:
- Query the size of the tablespaces: MGMT_TABLESPACE, MGMT_ECM_DEPOT_TS and MGMT_AD4J_TS on the source database and recreate them on the target database.
- TEMP and UNDO tablespaces:
  - Max UNDO space used seen during import is 680 MB.
  - Max TEMP space used seen during import is 140 GB.

Note:

For instructions about creating tablespaces manually, see Step 2.2. Create Tablespaces in Target Database.

Other Recommendations

RAC environment: Use connect string-based connection on the first node for RAC environment. Scan address is not supported.
Parallel environment:
- Export: Use no more than half the cores. The default value is 16.
- Import: Use no more than the number of cores. The default value is 16.
- Parallel jobs database parameters recommendations:
  - For single instance: max_datapump_parallel_per_job=200
  - For CDB/PDB: Set the following parameters in both databases:
    - max_datapump_jobs_per_pdb=200
    - max_datapump_parallel_per_job=400

Step 1.2. Complete Source Database Prerequisites

See below the prerequisites for the source database (ASM and non-ASM) when using RepMigrate utility for repository migration:

The source and target databases should be both at the same database release update (DB RU) version and with the same database patches installed.
Confirm all the database patches have been applied to the source database as described in Step 1.1. Perform Common Prerequisites for Source and Target Databases.
For TDE only, ensure to pass the same RepMigrate -encryptionPassword parameter to the source and target databases. When running the RepMigrate export operation in the source database using the -encryptionPassword parameter, after you need to pass the same -encryptionPassword parameter value during the RepMigrate import operation in the target database. For RepMigrate information, see RepMigrate Usage.
Before starting the repository migration process, confirm the source host has enough space available to store the files (dumpfiles) that will get created during the export operation.
- Perform tablespace validation.
  Check if the destination directory specified for the dumpfiles location can accommodate the contents of the database and confirm that there's enough space in the source host.

Step 1.3. Complete Target Database Prerequisites

See below the prerequisites for the target database when using RepMigrate utility for repository migration:

The source and target databases should be both at the same Database Release Update (DB RU) version and with the same database patches installed.
Confirm all the database patches have been applied to the target database as described in Step 1.1. Perform Common Prerequisites for Source and Target Databases.
The Management Agent should be pushed to the target database and patched using the same Enterprise Manager Release Update (RU) version as the one used by the OMS before starting the repository migration.
For information about Enterprise Manager requirements, see Step 1.4. Perform Common Prerequisites for Source and Target Enterprise Manager Hosts.
For SSL configured databases, use DB Wallet created at the source database using the EM Oracle Home to ensure same JDK version is used.
For TDE, ensure to pass the same RepMigrate -encryptionPassword parameter to the source and target databases. When running the RepMigrate import operation in the target database, you need to pass the same -encryptionPassword parameter value that was passed during the RepMigrate export operation in the source database. For RepMigrate information, see RepMigrate Usage.
Before starting the repository migration process, confirm the target host has enough space available to store the files (dumpfiles) that will get used for the import operation.
- Perform tablespace validation.
  Check if the directory specified for the dumpfiles location for the import operation can accommodate the contents of the database and confirm that there's enough space in the target host.
To minimize downtime, get the target database ready before the repository migration process is performed. For details, see Step 2. Best Practices.

Step 1.4. Perform Common Prerequisites for Source and Target Enterprise Manager Hosts

The following lists the common Enterprise Manager prerequisites when using RepMigrate utility for repository migration:

Apply the necessary Enterprise Manager Release Update (RU) version to both the source and target OMS environments.
The Enterprise Manager Release Update version should be the same on the source and target OMS hosts and the RepMigrate utility. For information about RepMigrate, see How to Get RepMigrate Utility?.
Apply the necessary Enterprise Manager Release Update version to the centralized Management Agents.
The Enterprise Manager Management Agent on the source and target hosts must be at the same Enterprise Manager RU version.
Add the correct Enterprise Manager Release Update (RU) Agent to the primary target database node.

Step 2. Best Practices

Complete the following steps in the target database to minimize downtime:

Step 2.1. Check Database Objects

To minimize the downtime, it's important to confirm that the target database is ready for the repository migration.

Oracle recommends to check the following in the target database before starting the repository migration process:

Tablespaces already created.
Tablespaces sized according to your requirements.
Database parameters updated.
Non-Enterprise Manager roles and users already created.
Any external profiles to be kept should get created.
Legacy users either dropped on the source database or created on the target database.
DBA directories to be kept from the source database created on the target database.
Storage checked.
If using RAC services, jobs should be already created.

Step 2.2. Create Tablespaces in Target Database

To minimize downtime, it's important to perform this step in the target database before starting the repository migration.

Create tablespaces in the target database manually by doing the following:

Check the size of the tablespace in the source database for MGMT, ECM_DEPOT and AD4J tablespaces by using the below query as SYS user:

select sum(bytes)/1024/1024 SIZE_IN_MB from dba_segments
where tablespace_name='MGMT_TABLESPACE';

select sum(bytes)/1024/1024 SIZE_IN_MB from dba_segments
where tablespace_name='MGMT_ECM_DEPOT_TS';

select sum(bytes)/1024/1024 SIZE_IN_MB from dba_segments
where tablespace_name='MGMT_AD4J_TS';

Using above ouput as sizes, create tablespaces using below queries:

CREATE SMALLFILE TABLESPACE "MGMT_AD4J_TS" EXTENT MANAGEMENT LOCAL AUTOALLOCATE SEGMENT SPACE MANAGEMENT AUTO
DATAFILE '<datafileLocation>/mgmt_ad4j.dbf'  SIZE <MGMT_AD4J_TS_size_as_calculated>
REUSE AUTOEXTEND ON NEXT 50M MAXSIZE UNLIMITED;

CREATE SMALLFILE TABLESPACE "MGMT_ECM_DEPOT_TS" EXTENT MANAGEMENT LOCAL AUTOALLOCATE SEGMENT SPACE MANAGEMENT AUTO
DATAFILE  '<datafileLocation>/mgmt_depot.dbf' SIZE <MGMT_ECM_DEPOT_TS_size_as_calculated>
REUSE AUTOEXTEND ON NEXT 20M MAXSIZE UNLIMITED;

CREATE SMALLFILE TABLESPACE "MGMT_TABLESPACE" EXTENT MANAGEMENT LOCAL AUTOALLOCATE SEGMENT SPACE MANAGEMENT AUTO
DATAFILE  '<datafileLocation>/mgmt.dbf' SIZE <MGMT_TABLESPACE_size_as_calculated> 
REUSE AUTOEXTEND ON NEXT 50M  MAXSIZE UNLIMITED;

Note:

The value of the <datafileLocation> should be similar to what you used in source database host.

For TEMP and UNDO tablespaces, Oracle recommends to have size around the following:
- Max PDB Temp Usage = 140 GB
- Max PDB Undo Usage = 680 MB
RepMigrate utility checks for TEMP and UNDO tablespaces. If there's not enough space available, it will fail.

Step 2.3. RAC only: Create RAC Services

To minimize downtime, it's important to perform this step in the target database before starting the repository migration.

This is only applicable to RAC environments: If RAC services are used in the source database, you must create the required RAC services in the target database.

For information about sizing, see Software Configurations in the Enterprise Manager Advanced Installation and Configuration Guide.

For information about RAC service links creation, see Jobs in the Enterprise Manager Avanced Installation and Configuration Guide.

Step 3. Run Prerequisites Check

Complete the following steps in the source database to run the RepMigrate utility and check only the prerequisites prior to starting the repository migration (export operation) and proactively resolve any issues.

Note:

Before proceeding, confirm that the database prerequisites have been completed. For details, see Step 1.1. Perform Common Prerequisites for Source and Target Databases.

Run RepMigrate to only check the prerequisites in the source database using the -prereqOnly parameter.

For example:

./RepMigrate.sh -prereqOnly -dbUser sys -dbPassword <db password> -connectString "<connect string>" -dataPumpDir <data pump dir location> -dataPumpUser <datapump user> -dataPumpPassword <datapump password> -action expdp  -dbHome <db home location> -reposPassword <repos Password> [-parallel <number>]

Address any issues reported from the standard output.

Repeat step 1 until you are satisfied and correct all you need.

When using the -prereqOnly parameter, you can check for issues with external users, external profiles, external roles, external directories, legacy EM schemas, such as SYSMAN_BIPLATFORM and SYSMAN_APM, DBMS scheduler jobs and invalid objects in external schemas. After receiving the report, follow the instructions as shown by RepMigrate to resolve any issues. Then, execute the RepMigrate again (step 1). If there are still issues, you can decide to ignore them by using the -ignorePrereqList parameter. If you pass the -ignorePrereqList without following the instructions, the import operation will result into failures.

Note:

For RAC environment, use the connect string-based connection on the first node for the RAC setup. Scan address is not supported.

For SSL configured repository, use RepMigrate.sh -help to review the available options.

For more information about RepMigrate utility, see RepMigrate Usage.

Step 4. Perform Migration

After completing the previous steps, perform the following to migrate the repository database using RepMigrate:

Step 4.1. Shutdown OMS and Agent

To shutdown Enterprise Manager OMS and Management Agents, connect to the source host and do the following:

Stop the OMS from the $OMS_Home/bin directory.
```
emctl stop oms -all -force
```
Note:
If there are multiple OMS environments then stop each OMS and agent on all nodes.
Stop the Management Agent.
```
emctl stop agent
```
Stop all central Management Agents.
Purge recyclebin as SYSMAN user.
Connect to the source repository database as SYSMAN user and execute the following:
```
SQL> PURGE RECYCLEBIN;
```

Step 4.2. Restart Source Database

Shutdown and restart the source databases including pluggable databases.

If the OMS Repository is in a single instance database, then the database needs to be shutdown and restart.
If the OMS Repository is in a pluggable database, then only the pluggable database needs to be shutdown and restart.
If the OMS Repository is in a RAC environment, then shutdown and restart all database instances.

Step 4.3. Export from Source Database Using RepMigrate

You can now proceed with the export operation from the source database using the RepMigrate utility.

See below the steps that need to be executed in the source database for a successful export operation in cross platform cross endian Repository Database:

Invoke RepMigrate in command line.
```
./RepMigrate.sh -dbUser sys -dbPassword <db password> -connectString "<connect string>" -dataPumpDir <data pump dir location> -dataPumpUser <datapump user> -dataPumpPassword <datapump password> -action expdp -dbHome <db home location> -reposPassword <repos Password> [-parallel <number>]
```
Where:
- <db password> is the password of SYS database user.
- <connect string> is the Enterprise Manager Repository database connect string.
- <data pump dir location> is the destination location where the datapump export dump files will be copied to.
- <datapump user> is SYSTEM database user or user with EXP_FULL_DATABASE privilege.
- <datapump password> is the password of the datapump user. In this case, the SYSTEM user password.
- <db home location> is the database home of the Enterprise Manager Repository.
- <repos Password> is the password of SYSMAN database user.
- -parallel <number> is optional. It's the number of parallel workers. Default is 16.

Note:

For ASM environment, you need to pass the ASM specific location for the -dataPumpDir <data pumpdir location> parameter. For example:

-dataPumpDir +DATAC1/DP_EXPORT_DIR

For information about RepMigrate utility, see RepMigrate Usage.

Example

./RepMigrate.sh -dbUser sys -dbPassword abc -connectString "(DESCRIPTION=(ADDRESS=(PROTOCOL=abc)(HOST=abchostname)(PORT=0000))(CONNECT_DATA=(SERVER=DEDICATED)(SID=sidname)))" -dataPumpDir /abc/dirname -dataPumpUser system -dataPumpPassword abc -action expdp -dbHome /abc/db_home -reposPassword abc

Alternatively, you can invoke RepMigrate using a response file. For details, see RepMigrate Response Files.

Output

Output looks similar to the following:

Looking for suitable Agent.
_agentRUVersion=13.5.0.22
Arguments validated Successfully
RepMigrate invoked with command line parameters: -action expdp -dataPumpPassword  ******** -dataPumpUser system -dataPumpDir /abc/dirname  -dbUser sys-dbPassword  ******** -reposPassword ******** -script_dir  /abc/RepMigrate  -connectString (DESCRIPTION=(ADDRESS=(PROTOCOL=abc)(HOST=abchostname)(PORT=0000))(CONNECT_DATA=(SERVER=DEDICATED)(SID=sidname))) -dbHome /abc/db_home
RepMigrate parameters from command line: -action expdp -dataPumpPassword ********  -dataPumpUser system -dataPumpDir /abc/dirname  -dbUser sys -dbPassword  ******** -reposPassword ******** -script_dir  /abc/RepMigrate -connectString (DESCRIPTION=(ADDRESS=(PROTOCOL=abc)(HOST=abchostname)(PORT=0000))(CONNECT_DATA=(SERVER=DEDICATED)(SID=sidname))) -dbHome /abc/db_home
DB Patch 30978304applied.
External Role check passed.
External User check passed.
External Profile check passed.
Legacy User check passed.
DBMS Scheduler Jobs check passed.
Invalid Objects check passed.
DBA Directory check passed.
Directory Created Successfully
Directory Created Successfully
Stopping AQ...
         RCU Logfile: /abc/RepMigrate/logs/rcu.log
Processing command line ....
Repository Creation Utility - Checking Prerequisites
Checking Global Prerequisites  
Repository Creation Utility - Checking Prerequisites
Checking Component Prerequisites
Repository Creation Utility - Creating Tablespaces
Validating and Creating Tablespaces
Create tablespaces in the repository database
Repository Creation Utility - Create 
Repository Create in progress.
Executing pre create operations
        Percent Complete: 55
        ...
Creating DB Migration Utility(EM_REPOS_MIGRATE_EXPORT)
        Percent Complete: 84
Executing post create operations
        Percent Complete: 100

Repository Creation Utility: Create - Completion Summary

Database details:
-----------------------------
Connect Descriptor                           :(DESCRIPTION=(ADDRESS=(PROTOCOL=abc)(HOST=abchostname)(PORT=0000))(CONNECT_DATA=(SERVER=DEDICATED)(SID=sidname)))
Connected As                                 : SYS
RCU Logfile                                  :/abc/RepMigrate/logs/rcu.log

Component schemas created:
-----------------------------
Component                                    Status         Logfile

DB Migration Utility                         Success        /abc/RepMigrate/logs/em_repos_migrate_export.log

Repository Creation Utility - Create : Operation Completed
Please copy the logs folder from baseHome to a safe location if you plan to delete the baseHome folder where baseHome is the location where the RepMigrate Utility is unzipped.

Log Files

Check the log files in the <full_path_RepMigrate_location>/logs directory. For more details, see RepMigrate Log Files.

Step 4.4. Copy Dumpfiles from Source to Target

Once the export operation is completed, copy the contents from the -dataPumpDir location used during the export operation from the source to the target host.

Copy everything under -dataPumpDir location from source to target.
Copy admin_gen_grants_to_ro_user.sql and carry_info.txt files from $BASE_DIR where BASE_DIR value is where you unzipped the RepMigrate utility on the source host to the target host under $BASE_DIR where you unzipped the RepMigrate utility.
For ASM, do the following:
- Copy from the ASM source host to the ASM target host using the following:
```
asmcmd cp +DATAC1/DP_MIGRATE_DIR/* sys/<pwd>@<IPAddress>.+ASM1:+DATAC1/DP_MIGRATE_DIR
```
  Where <pwd> is the ASM SYS password.

Step 4.5. Import into Target Database Using RepMigrate

You can now proceed with the import operation into the target databaseusing the RepMigrate utility.

See below the steps that need to be executed in the target database for a successful import operation in cross platform cross endian Repository Database:

Invoke RepMigrate in command line.

./RepMigrate.sh -dbUser sys -dbPassword <db password> -connectString "<connect  string>" -dataPumpDir <data pump dir location> -dataPumpUser <datapump user> -dataPumpPassword <datapump password>  -action impdp -dbHome <db home location> -reposPassword <repos  Password> [-datafileLocation <Location To Create Datafiles>]  [-parallel <number>]

Where:

<db password> is the password of SYS database user.
<connect string> is the Enterprise Manager Repository database connect string.
<data pump dir location> is the destination location where the datapump export dump files will be copied to.
<datapump user> is SYSTEM database user or user with IMP_FULL_DATABASE prilvilege.
<datapump password> is the password of the datapump user. In this case, the SYSTEM user password.
<db home location> is the database home of the Enterprise Manager Repository.
<repos Password> is the password of SYSMAN database user.
The -reposPassword <repos Password> value must be the same as the one used for the source database.
-datafileLocation <Location To Create Datafiles>] is optional. It's the location to create the datafiles.
-parallel <number> is optional. It's the number of parallel workers. Default value is 16.

Note:

The RepMigrate utility sets job queue process and submits jobs on target by default. You can stop this by adding -keepQuiesce.
For ASM, provide an extra parameter: -datafileLocation <Location to Datafile>.

Example

./RepMigrate.sh -dbUser sys -dbPassword abc -connectString (DESCRIPTION=(ADDRESS=(PROTOCOL=abc)(HOST=123hostname)(PORT=0000)) (CONNECT_DATA=(SERVER=DEDICATED)(SID=sidname))) -dataPumpDir /abc/dirname -dataPumpUser system -dataPumpPassword abc -action impdp -dbHome /abc/db_home -reposPassword abc

Alternatively, you can invoke RepMigrate using a response files. For details, see RepMigrate Response Files.

Output

Output looks similar to the following:

Looking for suitable Agent.
_agentRUVersion=13.5.0.22
Arguments validated Successfully
RepMigrate invoked with command line parameters:  -action impdp -dataPumpPassword  ******** -dataPumpUser system -dataPumpDir /abc/dirname -dbUser sys  -dbPassword ******** -reposPassword ******** -connectString (DESCRIPTION=(ADDRESS=(PROTOCOL=abc)(HOST=abchostname)(PORT=0000))(CONNECT_DATA=(SERVER=DEDICATED)(SID=sidname))) -dbHome /abc/db_home
RepMigrate parameters from command line:  -action impdp -dataPumpPassword ******** -dataPumpUser system  -dataPumpDir /abc/dirname -dbUser sys -dbPassword ********  -reposPassword ******** -connectString (DESCRIPTION=(ADDRESS=(PROTOCOL=abc)(HOST=abchostname)(PORT=0000))(CONNECT_DATA=(SERVER=DEDICATED)(SID=sidname))) -dbHome /abc/db_home
DB Patch 30978304 applied.
Directory Created Successfully
Datafile Folder Validated
Directory Created Successfully
Folder already exists. Skipping creation of the folder.
         RCU Logfile: /abc/RepMigrate/logs/rcu.log 
Processing command line ....
Repository Creation Utility - Checking Prerequisites
Checking Global Prerequisites
Repository Creation Utility - Checking Prerequisites
Checking Component Prerequisites
Repository Creation Utility - Creating Tablespaces
Validating and Creating Tablespaces
Create tablespaces in the repository database
Repository Creation Utility - Create
Repository Create in progress.
Executing pre create
        operations
        Percent Complete: 55
        ...
Creating DB Migration Utility(EM_REPOS_MIGRATE_IMPORT)
        Percent Complete: 84
Executing post create operations
        Percent Complete: 100

Repository Creation Utility: Create - Completion Summary

Database details:
-----------------------------
Connect Descriptor                         : (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=em1111.subnetaaa.bbbb.oraclevcn.com)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SID=orcl)))
Connected As                                 : SYS
RCU Logfile                                  : /abc/RepMigrate/logs/rcu.log

Component schemas created:
-----------------------------
Component                                    Status         Logfile
DB Migration Utility                         Success       /abc/RepMigrate/logs/em_repos_migrate_import.log

Repository Creation Utility - Create : Operation Completed
Please copy the logs folder from baseHome to a safe location ifyou plan to delete the baseHome folder where baseHome is the location where the RepMigrate Utility is unzipped.
Starting AQ.

Log File

Check the log files in the <full_path_RepMigrate_location>/logs directory. For more details, see RepMigrate Log Files .

Step 4.6. Set Up New Repository

To set up the new repository, do the following:

Restart Enterprise Manager OMS with admin option.
- emctl start oms -admin_only

Change OMS configuration.

emctl config oms -store_repos_details -repos_conndesc '(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=<Host_NAME>)(PORT=<Port>)))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=<Service>)))' -repos_user SYSMAN -repos_pwd <pwd>

emctl config emrep -sysman_pwd <sysman password> -conn_desc "new DB descriptor"

Change all services used by Enterprise Manager.
For RAC, Enterprise Manager has a connect string for certain subsystems that use RAC services. If those are configured, they need to be changed. For information, see the Enterprise Manager Advanced Installation and Configuration Guide.
Stop OMS.
- emctl stop oms -all
Restart OMS and Management Agent.
- emctl start oms
- emctl start agent

Perform Post Migration Verification

These verification steps should be carried out post migration to ensure that the migration was completely successful:

Verify any discrepancy in objects by comparing source and target databases through Enterprise Manager.
Verify the migrated database through Enterprise Manager to determine whether the database is running without any issues.
Verify the repository operations, dbms jobs and whether any management system errors are reported.
Verify that all Enterprise Manager functionalities are working correctly after the migration.
Make sure Management Services and the Repository target is properly relocated by verifying it through Enterprise Manager.

RepMigrate Files and Troubleshooting

Starting with Enterprise Manager 13c Release 5 Update 22 (13.5.0.22), the RepMigrate utility is available for performing Enterprise Manager repository database migration.

This section provides additional information about the RepMigrate utility such as log and response files, and troubleshooting.

RepMigrate Log Files

This section provides information about the log files generated when using the RepMigrate utility.

Location

They are located in the <RepMigrate_location>/logs folder.

Export Log Files

The following are the contents of the RepMigrate export log files:

em_repos_migrate_export.log
error.txt
export/
- restOfSchemasExportOutput.log
- sysmanTypesDataOnlySchemaExportOutput.log
- sysmanUsersOnlyExportOutput.log
- tableExportOutput.log
- viewExportOutput.log
logger<number>.properties where <number> is a random number generated.
output.txt
rcu.log
repmigrate/
- m_<timestamp>.expdp where <timestamp> is date and time generated.
  - repmigrate_expdp.log

Import Log Files

The following are the contents of the RepMigrate import log files:

custom_comp_create_tbs.log
import/
- restOfSchemasImportOutput.log
- sysmanTypesDataOnlySchemaImportOutput.log
- sysmanUsersOnlyImportOutput.log
- tableImportOutput.log
- viewImportOutput.log
output.txt
repmigrate/
- m_<timestamp>.impdp where <timestamp> is date and time generated.
  - repmigrate_impdp.log
em_repos_migrate_import.log
error.txt
logger<number>.properties where <number> is a random number generated.
rcu.log
utlprp.sql

RepMigrate Response Files

This section provides information about invoking RepMigrate using response files.

You can create response files and use them when invoking RepMigrate utility.

Response Files Location

They can be created and saved under the RepMigrate home directory (location where RepMigrate was unzipped).

Export Response File

You can create a response file using a text editor, add the required parameters for the export operation and save it with any preferred name. For example, it can be saved as <RepMigrate_location>/repmigrate_exp.rsp file.

Response File Example for Export:

action=expdp
connectString=(DESCRIPTION=(ADDRESS=(PROTOCOL=abc)(HOST=abchostname)(PORT=0000))(CONNECT_DATA=(SERVER=DEDICATED)(SID=sidname)))
dbUser=SYS
dbPassword=welcome***
reposPassword=welcome***
dataPumpUser=SYSTEM
dataPumpPassword=welcome***
dataPumpDir=/abc/dirname
dbHome=/abc/db_home

For information about the above parameters and values, see Step 4.3. Export from Source Database Using RepMigrate.

Import Response File

You can create a response file using a text editor, add the required parameters for the import operation and save it with any preferred name. For example, it can be saved as <RepMigrate_location>/repmigrate_imp.rsp file.

Response File Example for Import:

action=impdp
connectString=(DESCRIPTION=(ADDRESS=(PROTOCOL=abc)(HOST=abchostname)(PORT=0000))(CONNECT_DATA=(SERVER=DEDICATED)(SID=sidname)))
dbUser=SYS
dbPassword=welcome****
reposPassword=welcome***
dataPumpUser=SYSTEM
dataPumpPassword=welcome***
dataPumpDir=/abc/dirname 
dbHome /abc/em/db_home

For information about the above parameters and values, see Step 4.5. Import into Target Database Using RepMigrate.

Invoke RepMigrate Using Response File

To invoke RepMigrate using a response file, run the following:

RepMigrate.sh -responsefile <full_path_response_file>

Export example:

RepMigrate.sh -responsefile /abc/repmigrate_exp.rsp

Troubleshoot RepMigrate

This section covers some typical issues and resolutions related to the RepMigrate utility.

Users may encounter various errors during the repository migration process. Causes and recommended actions for some common errors are listed below.

Agent home does not have the same RU version as that of OMS. Ensure that the agent is patched with the same RU version as the OMS and then retry the operation.

Cause: RepMigrate utility requires EM agent with the same release update (RU) level of the OMS to be running on the source and target database host. Utility provides the above error if the EM agent is not running on the source or target host.

Action: Push an EM agent with same RU as the OMS to the source or target host before performing the export ( expdp) or import (impdp) operation.

DB Patch 30978304 is not found. Please contact Oracle Support to resolve this error.

Cause: RepMigrate utility requires Data Pump patch 30978304 to be applied on both source and target hosts before the expdp or impdp operation. Utility throws this error if the patch is not applied on source or target host.

Possible Action: Contact My Oracle Support and apply the database patch 30978304 or the Data Pump patch specific to the database version on the source or target host. For database patch information, see My Oracle Support.

External User check failed. Create EM_ADMIN user on target DB before import.

Cause: RepMigrate utility checks for any external users other than EM users created on the source database. If it finds external users, it throws this error with all the names of external users in the source. In the above example, EM_ADMIN is an external user.

Possible Action: User must create the external users listed in the error on the target.

User should rerun the utility by passing -ignorePrereqList repmigrate_external_user_check parameter.

The same information would be displayed on the console along with the error as below:

Re-run the RepMigrate utility on the source database with the "-ignorePrereqList repmigrate_external_user_check" option to complete the export operation. If more than one prereq has failed pass them with comma (,) separated list. For example: "-ignorePrereqList repmigrate_<x>_check,repmigrate_<y>_check". Use RepMigrate -help on usage.

External Role check failed. Create SYSMAN_READ1 role on target DB before import.

Cause: RepMigrate utility checks for any external roles other than EM roles created on the source database. If it finds external roles, it throws this error with all the names of external roles in the source. In the above example, SYSMAN_READ1 is an external role.

Possible Action: User must create the external roles listed in the error on the target.

User should rerun the utility by passing -ignorePrereqList repmigrate_external_roles_check parameter.

The same information would be displayed on the console along with the error as below:

Re-run the RepMigrate utility on the source database with the "-ignorePrereqList repmigrate_external_role_check" option to complete the export operation. If more than one prereq has failed pass them with comma (,) separated list e.g. "-ignorePrereqList repmigrate_<x>_check,repmigrate_<y>_check". Use RepMigrate -help on usage.

External Profiles check failed. Create PGGD profile on target DB before import.

Cause: RepMigrate utility checks for any external profiles other than EM profiles created on the source DB . If it finds external profiles, it throws this error with all the names of external profiles in the source. In this example, PGDB is an external profile.

Possible Action: User must create the external profiles listed in the error on the target.

User should rerun the utility by passing -ignorePrereqList repmigrate_external_profile_check parameter.

The same information would be displayed on the console along with the error as below:

Re-run the RepMigrate utility on the source database with the "-ignorePrereqList repmigrate_external_profile_check" option to complete the export operation. If more than one prereq has failed pass them with comma (,) separated list e.g. "-ignorePrereqList repmigrate_<x>_check,repmigrate_<y>_check". Use RepMigrate -help on usage.

Recovery of Old Enterprise Manager

To recover old EM, run the below as SYS on source database:

ALTER SYSTEM SET job_queue_processes=<JOB_QUEUE_MAX>  sid='*';  where JOB_QUEUE_MAX can be checked from carry_info.txt file present in  <dataPumpDir>.

For ASM, the value of JOB_QUEUE_MAX can be checked from carry_info.txt file located in <BASEDIR>.

ALTER SESSION SET CURRENT_SCHEMA = SYSMAN;

```
@<baseDir>/rsc/sql/admin_submit_dbms_jobs.sql;
```
The <baseDir> is the location where RepMigrate is unzipped.

Run below as SYSMAN to start AQ:

 begin
for emqrec in (select name from dba_queues where owner='SYSMAN' and name not like 'AQ$%')        
loop       
DBMS_AQADM.start_queue(
        queue_name => emqrec.name,
        enqueue =>    true,
        dequeue => true);
        end loop;
        end;
 /

Start OMS
```
emctl start oms
```
Start Agent
```
emctl start agent
```

Recovery from Failure during Export

In case of failure during Export, do the following:

Run following on source database as SYS:

ALTER SYSTEM SET job_queue_processes=<JOB_QUEUE_MAX>  sid='*';  where JOB_QUEUE_MAX can be checked from carry_info.txt file present in  <dataPumpDir>.

For ASM, the value of JOB_QUEUE_MAX can be checked from carry_info.txt file present in <baseDir>.

ALTER SESSION SET CURRENT_SCHEMA = SYSMAN;

@<baseDir>/rsc/sql/admin_submit_dbms_jobs.sql;

The <baseDir> is the location where RepMigrate is unzipped.

Run below as SYSMAN to start AQ:

begin
for emqrec in (select name from dba_queues where owner='SYSMAN' and name not like 'AQ$%')
loop       
DBMS_AQADM.start_queue(
queue_name => emqrec.name,
enqueue =>    true,
dequeue => true);
end loop;
end; 
/

Delete the following:
- The <dumpfile folder> passed to utility in step 3 as provided in -dataPumpDir option.
- <baseDir/logs> folder.
- For ASM, in addition to above steps, delete the carry_info.txt and admin_gen_grants_to_ro_user.sql from <baseDir>.
- Retry Export.

Recovery from Failure during Import

Remove the old DB instance.
If the old database instance is a CDB and you are importing to a PDB, you can delete and recreate the PDB.
Create a new instance.
Delete <baseDir/logs> folder in <baseDir> before retrying the import.
The <baseDir> is the location where RepMigrate is unzipped.
For ASM, in addition to above steps, delete the DUMPSET using ASMCMD by doing the following:
- Login to ASMCMD.
- Got to the DUMPSET of the ASM Instance.
- Run the following:
```
rm sysman_*
rm view.dmp
rm table.dmp
```