Before upgrading to Enterprise Manager Cloud Control 13c Release 4, ensure to meet the following prerequisites:
Hardware, Software, and Platform Requirements
Ensure that you meet all the prerequisites, including the hardware requirements, listed in the chapter on installing Enterprise Manager Cloud Control, in the Oracle Enterprise Manager Cloud Control Basic Installation Guide.
Also ensure that you upgrade only on the supported platforms as listed in Supported Platforms for Upgrading to Enterprise Manager Cloud Control 13c Release 4.
Supported Database Release Requirements
Ensure that the existing database is a database certified for 13c Release 4. You can see a list of certified databases in the Enterprise Manager certification matrix available on My Oracle Support. For instructions, see Accessing the Enterprise Manager Certification Matrix in Oracle Enterprise Manager Cloud Control Basic Installation Guide.
If the existing database is not of the release that is supported for 13c Release 4, then upgrade it to the supported release before you start upgrading the OMS and the Management Repository. For example, if you are upgrading from 13c Release 2, then you might have an earlier release of the database that is not supported for 13c Release 4. In this case, ensure that you first upgrade your database to the minimum database release supported for 13c Release 4, and then upgrade the Enterprise Manager system to 13c Release 4.
Before upgrading the database, ensure that you stop the OMS instances of the earlier release that are connected to the database. While upgrading the database, if you change the listener port, then follow these steps:
Complete the database upgrade with the changed listener port.
Start the administration server of the first OMS of the earlier release, which is connected to the database, by running the following command:
emctl start oms -admin_only
Update the first OMS and also the additional OMS instances with the changed listener port by running the following command on all the OMS instances:
emctl config oms –store_repos_details -repos_conndesc <connect descriptor> -repos_user sysman
When the aforementioned command is run on each OMS, you will be prompted to stop the OMS and start it again to reflect the changes. Do so on each OMS as instructed. You will have to stop them all in any case before starting the upgrade to 13c.
Proceed with the upgrade of the entire Enterprise Manager system to 13c.
Ensure that OMS starts successfully and is able to access console after the connect descriptor change is completed. It is suggested to proceed with OMS upgrade only after starting OMS successfully.
Database Patch Requirements
Ensure that you apply the latest PSU on the supported database.
It is mandatory to apply the latest DB PSU on the supported database before upgrading to Enterprise Manager Cloud Control 18.104.22.168.
Optimizer Adaptive Features Disabling Requirements
If the Management Repository is using Oracle Database 22.214.171.124.0 and patch 22652097 has not been applied, ensure that you disable the optimizer adaptive features by connecting to the database as
SYSDBA and running the following:
alter system set optimizer_adaptive_features=false scope=both
To verify that the changes have taken effect, run the following command:
show parameter adaptive;
You should see the following output:
NAME TYPE VALUE --------------------------------------------------------------------- optimizer_adaptive_features boolean FALSE
If the Management Repository is using Oracle Database 126.96.36.199 with patch 22652097 applied or the Management Repository is using Oracle Database 12.2, 18.x or 19.x, you need to do the following:
Connect to the database as
SYSDBA and run the following:
alter system set "_optimizer_nlj_hj_adaptive_join"= FALSE scope=both sid='*'; alter system set "_optimizer_strans_adaptive_pruning" = FALSE scope=both sid='*'; alter system set "_px_adaptive_dist_method" = OFF scope=both sid='*'; alter system set "_sql_plan_directive_mgmt_control" = 0 scope=both sid='*'; alter system set "_optimizer_dsdir_usage_control" = 0 scope=both sid='*'; alter system set "_optimizer_use_feedback" = FALSE scope=both sid='*'; alter system set "_optimizer_gather_feedback" = FALSE scope=both sid='*'; alter system set "_optimizer_performance_feedback" = OFF scope=both sid='*';
Database Initialization Parameter in Oracle Database 188.8.131.52.0/184.108.40.206.0
Ensure that the database initialization parameter is (
_allow_insert_with_update_check=TRUE) set to true in the Oracle Database 220.127.116.11.0/18.104.22.168.0 or higher that houses the Management Repository. To do so, follow these steps:
Enable the database initialization parameter by setting the
TRUE.To do so, run the following SQL command:
alter system set _allow_insert_with_update_check=true scope=both
Restart the database.
Verify that the changes have taken effect. To do so, run the following SQL command:
show parameter _allow_insert_with_update_check;
You should see the following output:
NAME TYPE VALUE --------------------------------------------------------------------- _allow_insert_with_update_check boolean TRUE
Management Agent Patch Requirements
Ensure that you apply the Patch 20282974 for Oracle Management Agent 12c Release 4 (22.214.171.124), on the Management Agents running on IBM AIX operating systems.
Ensure that the ports used by Oracle Management Service (OMS) are not set to a value lower than or equal to 1024. If they are, then the upgrade will fail. Ports up to 1024 are typically reserved for root users (super users). Therefore, make sure the ports are greater than 1024.
Customization Removal Requirements
Ensure that you remove the following types of customization done to the OMS. Once the upgrade is complete, you can redo the customization.
Additional data source parameters configured in the Weblogic Server.
Smart card authentication for Enterprise Manager login.
Oracle BI Publisher Shutdown Requirements
Ensure that you stop the Oracle BI Publisher deployed to your previous release. To do so, use one of the following approaches:
If you are upgrading from Enterprise Manager Cloud Control 13c Release 2 (126.96.36.199) or 13c Release 3 (188.8.131.52) , then run
emctl stop oms -allon all the OMS instances, including the ones running a BI Publisher Server.
Database Service Instance Creation Request Verification Requirements
Ensure that you check for outstanding database service instance creation requests. If there are any requests in progress, then allow them to complete. For requests that are scheduled, suspend them.
To do so, follow these steps.
In Cloud Control, from the Enterprise menu, select Cloud, then select Self Service Portal.
On the Infrastructure Cloud Self Service Portal page, right under the page title, select My Databases to view only database requests.
In the Requests table, for requests that are in progress, allow them to complete. For requests that are scheduled, suspend them.
To suspend the scheduled requests, click the request name. Click the Deployment tab. Click the deployment procedure listed there, and suspend it.
Repository Table Snapshot Verification Requirements
Ensure that the tables in the Management Repository do not have any snapshots created.
To verify this, log in to the Management Repository and run the following SQL query as SYSMAN user:
select master , log_table from all_mview_logs where log_owner='<EM_REPOS_USER>';
select master , log_table from all_mview_logs where log_owner='SYSMAN';
If there are snapshots created in a table, then the query displays the master table and the snapshot details. For example,
SQL> master log_table
If there are snapshots, then drop them by running the following command as SYSMAN user:
SQL> Drop snapshot log on <master>;
SQL> Drop snapshot log on em-violations;
Logon and Logoff Trigger Setting Verification Requirements
Ensure that you do not have any logon or logoff triggers set in the Oracle Database that houses the Oracle Management Repository.
To verify this, log on to the database and run the following query. If the query results in anything other than zero or no rows selected, then manually disable the triggers. After completing the upgrade, you can enable them again.
Verify if any logon triggers are set:
SQL> SELECT COUNT (trigger_name) FROM sys.dba_triggers WHERE TRIGGERING_EVENT LIKE 'LOGON%' AND status='ENABLED';
SQL> SELECT trigger_name FROM sys.dba_triggers WHERE TRIGGERING_EVENT LIKE 'LOGON%' AND status='ENABLED';
Verify if any logoff triggers are set:
SQL> SELECT COUNT (trigger_name) FROM sys.dba_triggers WHERE TRIGGERING_EVENT LIKE 'LOGOFF%' AND status='ENABLED';
SQL> SELECT trigger_name FROM sys.dba_triggers WHERE TRIGGERING_EVENT LIKE 'LOGOFF%' AND status='ENABLED';
To disable the triggers, run the following query:
SQL> alter trigger <trigger_name> disable;
SQL> alter trigger EXPFIL_ALTEREXPTAB_MAINT disable;
Delete Target Operation Auditing Requirements
Ensure that you enable auditing of the Delete Target operation.
To view a list of operations, run the following command:
Some of the Delete operations include Delete Target, Delete Named Credential, Delete Role, Delete Rule, Delete Monitoring Template, Delete User.
To verify the Delete operations that are currently enabled, run the following command:
If the Delete Target operation is not already enabled, then run the following command to enable it.
-file_prefix="file_prefix"(To be used by the export service to create the file name where audit data has to be written. The default value is em_audit. You can change per your standards for all sites)
-file_size="file_size (Bytes)"(Maximum value of each file size. The default value for this is 5000000 bytes)
-data_retention_period="data_retention_period (Days)"(Maximum period the Enterprise Manager repository stores audit data. The default value is 365 days)
The aforementioned parameters help you to set up or configure an archive location for archiving the audit data from the Management Repository to a file system after the retention period.
Selectively Skipping Some Job Type Updates for Reduced Downtime of Your Enterprise Manager System
While upgrading the Enterprise Manager system, job types are registered. As part of the job type registration process, all active executions corresponding to a job type are automatically upgraded to the newly registered version of the job type. This job type upgrade process is skipped for all queued and waiting executions, thereby reducing the overall downtime of the Enterprise Manager system. However, in some cases, the Enterprise Manager system might experience a considerable backlog, and if such a backlog is not cleared before initiating the upgrade, then the downtime of the Enterprise Manager system can be much longer. To circumvent this issue, you can selectively skip or postpone the upgrade of certain job types so that they are upgraded only after the downtime.
To skip or postpone some job types from being upgraded, follow these steps:
Identify the job types that you want to exclude from being upgraded during the downtime.
To do so, as a SYSMAN user, log in to the database that houses the Oracle Management Repository, and run the following query. Particularly look for the job types that have a large number of active executions.
SELECT job_type, COUNT(1) as n_execs
JOIN MGMT_JOB_TYPE_INFO USING (job_type_id)
WHERE status NOT IN (3,4,5,8,18,19,23)
GROUP BY job_type
HAVING COUNT(1) > 5000
ORDER BY COUNT(1) DESC;
Exclude the other job types you identified.
To do so, run the following query to exclude a job type from the MGMT_PARAMETERS table. For each job type you want to exclude, you must run one INSERT statement. For example, if you have three job types to exclude, then run the INSERT statement three times, each with a job type you want to exclude.
INSERT INTO MGMT_PARAMETERS(parameter_name, parameter_value) VALUES ('mgmt_job_skip_job_type_upg.1', '<job type>');
EMKEY Copy Requirements
[IN CASE OF MULTI-OMS UPGRADE, PERFORM THIS STEP ONLY FOR THE FIRST OMS UPGRADE]
Ensure that you copy the emkey from the existing OMS to the existing Management Repository. To do so, run the following command on the OMS you are about to upgrade. Here,
<ORACLE_HOME> refers to the Oracle home of the OMS.
$<ORACLE_HOME>/bin/emctl config emkey -copy_to_repos [-sysman_pwd <sysman_pwd>]
/u01/software/em13c/mw/oms/bin/emctl config emkey -copy_to_repos [-sysman_pwd <sysman_pwd>]
To verify whether the emkey is copied, run the following command on the OMS you are about to upgrade. Here,
<ORACLE_HOME> refers to the Oracle home of the OMS:
$<ORACLE_HOME>/bin/emctl status emkey
/u01/software/em13c/mw/oms/bin/emctl status emkey
If the emkey is copied, then you will see the following message:
The EMKey is configured properly, but is not secure. Secure the EMKey by running "emctl config emkey -remove_from_repos".
Certificate Key Strength Requirements
Ensure that the certificate key strength is at least 1024 bits.
For 11g Release 1 or lower, the certificates are generated with key strength 512 bits, and the certificates are carried over during upgrade to 12c Release (184.108.40.206) and subsequent upgrade to 13c Release 2 and 13c Release 3. Therefore, if you are upgrading from 11g Release 1 or lower, then your certificates will continue to be with 512 bits, and this will eventually cause multiple communication issues. Therefore, ensure that the certificate key strength is at least 1024 bits.
For instructions to set the certificate key strength to at least 1024 bits, see My Oracle Support note 1611578.1.
Starting with 13c Release 4, the minimum certificate supported is SHA 512/1024. MD5 certificates are no longer supported. If you have any MD5 certificate, you will see the following message stopping the upgrade process:
The upgrade process has detected MD5 certificate usage for some agent-OMS communication. These hosts and related targets will continue to be managed post-upgrade. However, it is strongly recommended to reconfigure them to use SHA-based certificates, since future versions of Enterprise Manager may not support MD5 certificates. Review the file /tmp/OraInstall<time_stamp>/md5Target.txt to find the agents that use MD5-based certificates. Follow the instructions outlined in My Oracle Support note 2179909.1 to reconfigure them with SHA-based certificates.
After you address it, you can restart the upgrade process.
Out-of-Box Memory Settings Backup Requirements
If you have changed the default out-of-the-box memory settings for an OMS instance, then you can save the default values as a backup so that they are not lost during upgrade.
You can verify the default values using the command
ORACLE_HOME/bin/emctl get property -name property_name.
ORACLE_HOME/bin/emctl get property -name OMS_PERMGEN_MAX
Starting with 12c release 4 (220.127.116.11), Enterprise Manager has introduced following new parameters which can be set as OMS Specific JAVA heap memory arguments:
OMS_HEAP_MINfor minimum heap size
OMS_HEAP_MAXfor maximum heap size
OMS_PERMGEN_MINfor minimum permGen size
OMS_PERMGEN_MAXfor maximum permGen size
To save the default values for out-of-the-box memory settings for an OMS instance, follow these steps:
Use the following command to set the value for any of the JAVA heap memory properties.
ORACLE_HOME/bin/emctl set property -name <property_name> -value <number_followed_by_G_or_M>
ORACLE_HOME/bin/emctl set property -name OMS_PERMGEN_MAX -value 1024M
Run the following command to restart all the OMS instances. Here,
<ORACLE_HOME>refers to the Oracle home of the OMS.
$<ORACLE_HOME>/bin/emctl stop oms -all
$<ORACLE_HOME>/bin/emctl start oms
/u01/software/em13c/mw/oms/bin/emctl stop oms -all
/u01/software/em13c/mw/oms/bin/emctl start oms
Prerequisite Check and Environment Validation Requirements
Ensure that you run the EM Prerequisite Kit and meet all the repository-related prerequisites before starting the upgrade.
To run the EM Prerequisite Kit, follow the parameters described in Oracle Enterprise Manager Cloud Control Basic Installation Guide.
OMS Backup Requirements
Ensure that you back up the OMS (the middleware home and the inventory), Oracle Management Repository, and Oracle Software Library. The restoration requires Instance home (gc_inst) backup. In case the upgrade fails, you can always restore using the backup. For instructions to back up, refer to Oracle Enterprise Manager Cloud Control Administrator's Guide.
OMS Shutdown Requirements
Ensure that you shut down the OMS you are about to upgrade and also the other OMS instances that connect to it.
IMPORTANT: If you are upgrading a multi-OMS environment using the software-only upgrade approach, then skip this step. You can stop the OMS instances after installing the software binaries as described in Installing the Enterprise Manager Cloud Control 13c Release 4 Software Binaries in Graphical Mode or Installing the Enterprise Manager Cloud Control 13c Release 4 Software Binaries in Silent Mode.
The Step 1 is applicable only if you are upgrading from 12c to 13c.
Stop the JVMD and ADP engines explicitly:
To stop them in graphical mode, access the weblogic console, and stop the JVMD and ADP weblogic managed servers manually.
To stop them in silent mode, run the following command on the OMS you are about to upgrade. Here,
<ORACLE_HOME>refers to the Oracle home of the OMS.
$<ORACLE_HOME>/bin/emctl extended oms jvmd stop -all
$<ORACLE_HOME>/bin/emctl extended oms adp stop –all
/u01/software/em12c/mw/oms/bin/emctl extended oms jvmd stop -all
/u01/software/em12c/mw/oms/bin/emctl extended oms adp stop –all
Shut down the OMS you are about to upgrade and also the other OMS instances that connect to it. To do so, run the following command. Here,
<ORACLE_HOME>refers to the Oracle home of the OMS.
$<ORACLE_HOME>/bin/emctl stop oms -all
/u01/software/em12c/mw/oms/bin/emctl stop oms -all
Management Agent Shutdown Requirements
Ensure that you shut down the Management Agent that monitors the Management Services and Repository target, to prevent the Management Agent from connecting to the Management Repository for metric collections. Not shutting down this Management Agent might cause the OMS upgrade to fail.
IMPORTANT: If you are upgrading a multi-OMS environment using the software-only upgrade approach, then skip this step. You can stop the Management Agent after copying the software binaries as described in Installing the Enterprise Manager Cloud Control 13c Release 4 Software Binaries in Graphical Mode or Installing the Enterprise Manager Cloud Control 13c Release 4 Software Binaries in Silent Mode.
Management Agent Proxy Removal Requirements
Ensure that you remove any proxy that is configured for the standalone Management Agents. Otherwise, the upgrade of the standalone Management Agents will fail. Although this requirement is more for Management Agent upgrade and does not affect the OMS upgrade, Oracle recommends that you remove the Management Agent-side proxy before you begin the OMS upgrade.
Additional Preparation and Steps Required for Performing an Upgrade and Transition to DR Readiness
A new mode of the installer named Upgrade and Transition to DR Readiness has been created to assist with the process of transitioning to the Standby OMSs using Storage Replication DR architecture. In addition to the prerequisites for a standard upgrade, an Upgrade and Transition to DR Readiness upgrade requires a specific flow of preparation and post-upgrade steps that must be followed.
This new mode is:
Enabled by passing a parameter named UPGRADE_TRANSITION
Only supported for use on the first OMS
Only supported via a standard GUI installation
A software only install followed by ConfigureGC.sh does not provide support for Upgrade and Transition to DR Readiness. In addition, upgrading additional OMSs using Upgrade and Transition to DR Readiness is not supported. In a multi-OMS environment, the additional OMSs must first be deinstalled, the first OMS and related post-upgrade processes associated with the transition must be completed, and then additional OMSs can be deployed. For details on performing an Upgrade and Transition to DR Readiness instead of performing a standard upgrade, see Upgrading Enterprise Manager and Transitioning to DR Readiness.
Undeploy Obsolete Plug-ins
The support for following plug-ins is discontinued and are now obsolete.
Oracle Audit Vault (oracle.em.soav).
Oracle Virtual Networking (oracle.em.sovn).
Oracle Engineered System Healthchecks (oracle.em.sehc).
Oracle Ops Center Infrastructure stack (oracle.em.sooc).
You must make sure that these plug-ins are undeployed from the Oracle Management Agents and Oracle Management Services before proceeding with your upgrade.
It is recommended to close the installer before undeploying the obsolete plug-ins. It is also recommended that these plug-ins are removed from the Self Update as well. For more information, see Undeploying Plug-ins in the Oracle Enterprise Manager Cloud Control Administrator's Guide.