Before upgrading your Enterprise Manager Cloud Control 13c Release 2, 13c Release 1, or 12c Release 5 (12.1.0.5), to 13c Release 3, meet the following prerequisites:
Database Service Instance Creation Request Verification Requirements
Selectively Skipping Some Job Type Updates for Reduced Downtime of Your Enterprise Manager System
Additional Preparation and Steps Required for Performing an Upgrade and Transition to DR Readiness
Note:
You must ensure that SQL ALG is disabled in firewall. If not, the Enterprise Manager Upgrade process may terminate/hang in same operation for a long time.
Note:
If you have events setup for maximum memory usage limit, then you must disable them before upgrade.
For example, alter system set event='10261 trace name context forever, level 3145728','10262 trace name context forever, level 3145728' scope=spfile;
Note:
You must make sure that there is no foreign custom objects. For example, views or MVs, which are dependent on EM repository objects.
Note:
Currently, AOM upgrade is not supported. You must make sure that the previous version of AOM is uninstalled before installing the new version. For more information on instructions and files to backup before uninstalling, see Uninstalling Always-On Monitoring in the Enterprise Manager Cloud Control Administrator's Guide.
Ensure that you meet all the prerequisites, including the hardware requirements, listed in the chapter on installing Enterprise Manager Cloud Control, in the Oracle Enterprise Manager Cloud Control Basic Installation Guide.
Also ensure that you upgrade only on the supported platforms as listed in Supported Platforms for Upgrading to Enterprise Manager Cloud Control 13c Release 3.
Ensure that the existing database is a database certified for 13c Release 3. You can see a list of certified databases in the Enterprise Manager certification matrix available on My Oracle Support. To access the Enterprise Manager certification matrix, follow these steps in Oracle Enterprise Manager Cloud Control Basic Installation Guide.
If the existing database is not of the release that is supported for 13c Release 3, then upgrade it to the supported release before you start upgrading the OMS and the Management Repository. For example, if you are upgrading from 12c Release 5 (12.1.0.5), then you might have an earlier release of the database that is not supported for 13c Release 3. In this case, ensure that you first upgrade your database to the minimum database release supported for 13c Release 3, and then upgrade the Enterprise Manager system to 13c Release 3.
Before upgrading the database, ensure that you stop the OMS instances of the earlier release that are connected to the database. While upgrading the database, if you change the listener port, then follow these steps:
Complete the database upgrade with the changed listener port.
Start the administration server of the first OMS of the earlier release, which is connected to the database, by running the following command:
emctl start oms -admin_only
Update the first OMS and also the additional OMS instances with the changed listener port by running the following command on all the OMS instances:
emctl config oms –store_repos_details -repos_conndesc <connect descriptor> -repos_user sysman
When the aforementioned command is run on each OMS, you will be prompted to stop the OMS and start it again to reflect the changes. Do so on each OMS as instructed. You will have to stop them all in any case before starting the upgrade to 13c.
Proceed with the upgrade of the entire Enterprise Manager system to 13c.
Note:
Ensure that OMS starts successfully and is able to access console after the connect descriptor change is completed. It is suggested to proceed with OMS upgrade only after starting OMS successfully.
Ensure that you apply the latest PSU on the supported database.
It is mandatory to apply the latest DB PSU on the supported database before upgrading to Enterprise Manager Cloud Control 13.3.0.0.
Ensure that you disable the optimizer adaptive feature (optimizer_adaptive_features=FALSE) in the Oracle Database 12.1.0.2.0 that houses the Management Repository. To do so, follow these steps:
Disable the optimizer adaptive feature by setting the optimizer_adaptive_features parameter to FALSE. To do so, run the following SQL command:
alter system set optimizer_adaptive_features=false scope=both
Restart the database.
Verify that the changes have taken effect. To do so, run the following SQL command:
show parameter adaptive;
You should see the following output:
NAME TYPE VALUE --------------------------------------------------------------------- optimizer_adaptive_features boolean FALSE
Ensure that the database initialization parameter is (_allow_insert_with_update_check=TRUE) set to true in the Oracle Database 12.2.0.1.0/18.0.0.0.0 or later that houses the Management Repository. To do so, follow these steps:
Enable the database initialization parameter by setting the _allow_insert_with_update_check parameter to TRUE. To do so, run the following SQL command:
alter system set _allow_insert_with_update_check=true scope=both
Restart the database.
Verify that the changes have taken effect. To do so, run the following SQL command:
show parameter _allow_insert_with_update_check;
You should see the following output:
NAME TYPE VALUE --------------------------------------------------------------------- _allow_insert_with_update_check boolean TRUE
Ensure that you apply the Patch 20282974 for Oracle Management Agent 12c Release 4 (12.1.0.4), on the Management Agents running on IBM AIX operating systems.
Ensure that the ports used by Oracle Management Service (OMS) are not set to a value lower than or equal to 1024. If they are, then the upgrade will fail. Ports up to 1024 are typically reserved for root users (super users). Therefore, make sure the ports are greater than 1024.
Ensure that you remove the following types of customization done to the OMS. Once the upgrade is complete, you can redo the customization.
Additional data source parameters configured in the Weblogic Server.
Smart card authentication for Enterprise Manager login.
Ensure that you stop the Oracle BI Publisher deployed to your previous release. To do so, use one of the following approaches:
If you are upgrading from Enterprise Manager Cloud Control 12c Release 5 (12.1.0.5), then run emctl stop oms -all on all the OMS instances, including the ones running a BI Publisher Server.
If you are upgrading from Enterprise Manager Cloud Control 12c Release 3 (12.1.0.3) or lower, then stop the primary BI Publisher Managed Server named BIP using the WebLogic Administration Console.
Ensure that you check for outstanding database service instance creation requests. If there are any requests in progress, then allow them to complete. For requests that are scheduled, suspend them.
To do so, follow these steps.
In Cloud Control, from the Enterprise menu, select Cloud, then select Self Service Portal.
On the Infrastructure Cloud Self Service Portal page, right under the page title, select My Databases to view only database requests.
In the Requests table, for requests that are in progress, allow them to complete. For requests that are scheduled, suspend them.
To suspend the scheduled requests, click the request name. Click the Deployment tab. Click the deployment procedure listed there, and suspend it.
Ensure that the tables in the Management Repository do not have any snapshots created.
To verify this, log in to the Management Repository and run the following SQL query as SYSMAN user:
select master , log_table from all_mview_logs where log_owner='<EM_REPOS_USER>';
For example,
select master , log_table from all_mview_logs where log_owner='SYSMAN';
If there are snapshots created in a table, then the query displays the master table and the snapshot details. For example,
SQL> master log_table
em-violations em$violation_log
If there are snapshots, then drop them by running the following command as SYSMAN user:
SQL> Drop snapshot log on <master>;
For example,
SQL> Drop snapshot log on em-violations;
Ensure that you do not have any logon or logoff triggers set in the Oracle Database that houses the Oracle Management Repository.
To verify this, log on to the database and run the following query. If the query results in anything other than zero or no rows selected, then manually disable the triggers. After completing the upgrade, you can enable them again.
Verify if any logon triggers are set:
SQL> SELECT COUNT (trigger_name) FROM sys.dba_triggers WHERE TRIGGERING_EVENT LIKE 'LOGON%' AND status='ENABLED';
SQL> SELECT trigger_name FROM sys.dba_triggers WHERE TRIGGERING_EVENT LIKE 'LOGON%' AND status='ENABLED';
Verify if any logoff triggers are set:
SQL> SELECT COUNT (trigger_name) FROM sys.dba_triggers WHERE TRIGGERING_EVENT LIKE 'LOGOFF%' AND status='ENABLED';
SQL> SELECT trigger_name FROM sys.dba_triggers WHERE TRIGGERING_EVENT LIKE 'LOGOFF%' AND status='ENABLED';
To disable the triggers, run the following query:
SQL> alter trigger <trigger_name> disable;
For example,
SQL> alter trigger EXPFIL_ALTEREXPTAB_MAINT disable;
Ensure that you enable auditing of the Delete Target operation.
To view a list of operations, run the following command:
emcli show_operations_list
Some of the Delete operations include Delete Target, Delete Named Credential, Delete Role, Delete Rule, Delete Monitoring Template, Delete User.
To verify the Delete operations that are currently enabled, run the following command:
emcli show_audit_settings
If the Delete Target operation is not already enabled, then run the following command to enable it.
emcli update_audit_settings
-operations_to_enable="name_of_the_operations_to_enable._For_all_operations_use_ALL"
-audit_switch="ENABLE”
-directory="db_directory_name” (Should_be_configured_with_an_OS_directory_where_the_export_service_archives_the_audit_data_files)
-file_prefix="file_prefix" (To be used by the export service to create the file name where audit data has to be written. The default value is em_audit. You can change per your standards for all sites)
-file_size="file_size (Bytes)" (Maximum value of each file size. The default value for this is 5000000 bytes)
-data_retention_period="data_retention_period (Days)" (Maximum period the Enterprise Manager repository stores audit data. The default value is 365 days)
The aforementioned parameters help you to set up or configure an archive location for archiving the audit data from the Management Repository to a file system after the retention period.
While upgrading the Enterprise Manager system, job types are registered. As part of the job type registration process, all active executions corresponding to a job type are automatically upgraded to the newly registered version of the job type. This job type upgrade process is skipped for all queued and waiting executions, thereby reducing the overall downtime of the Enterprise Manager system. However, in some cases, the Enterprise Manager system might experience a considerable backlog, and if such a backlog is not cleared before initiating the upgrade, then the downtime of the Enterprise Manager system can be much longer. To circumvent this issue, you can selectively skip or postpone the upgrade of certain job types so that they are upgraded only after the downtime.
To skip or postpone some job types from being upgraded, follow these steps:
Identify the job types that you want to exclude from being upgraded during the downtime.
To do so, as a SYSMAN user, log in to the database that houses the Oracle Management Repository, and run the following query. Particularly look for the job types that have a large number of active executions.
SELECT job_type, COUNT(1) as n_execs
FROM MGMT_JOB_EXEC_SUMMARY
JOIN MGMT_JOB_TYPE_INFO USING (job_type_id)
WHERE status NOT IN (3,4,5,8,18,19,23)
GROUP BY job_type
HAVING COUNT(1) > 5000
ORDER BY COUNT(1) DESC;
Exclude the other job types you identified.
To do so, run the following query to exclude a job type from the MGMT_PARAMETERS table. For each job type you want to exclude, you must run one INSERT statement. For example, if you have three job types to exclude, then run the INSERT statement three times, each with a job type you want to exclude.
INSERT INTO MGMT_PARAMETERS(parameter_name, parameter_value) VALUES ('mgmt_job_skip_job_type_upg.1', '<job type>');
COMMIT;
[IN CASE OF MULTI-OMS UPGRADE, PERFORM THIS STEP ONLY FOR THE FIRST OMS UPGRADE]
Ensure that you copy the emkey from the existing OMS to the existing Management Repository. To do so, run the following command on the OMS you are about to upgrade. Here, <ORACLE_HOME> refers to the Oracle home of the OMS.
$<ORACLE_HOME>/bin/emctl config emkey -copy_to_repos [-sysman_pwd <sysman_pwd>]
For example:
/u01/software/em12c/mw/oms/bin/emctl config emkey -copy_to_repos [-sysman_pwd <sysman_pwd>]
To verify whether the emkey is copied, run the following command on the OMS you are about to upgrade. Here, <ORACLE_HOME> refers to the Oracle home of the OMS:
$<ORACLE_HOME>/bin/emctl status emkey
For example,
/u01/software/em12c/mw/oms/bin/emctl status emkey
If the emkey is copied, then you will see the following message:
The EMKey is configured properly, but is not secure. Secure the EMKey by running "emctl config emkey -remove_from_repos".
Ensure that the certificate key strength is at least 1024 bits.
For 11g Release 1 or lower, the certificates are generated with key strength 512 bits, and the certificates are carried over during upgrade to 12c Release (12.1.0.5) and subsequent upgrade to 13c Release 3. Therefore, if you are upgrading from 11g Release 1 or lower, then your certificates will continue to be with 512 bits, and this will eventually cause multiple communication issues. Therefore, ensure that the certificate key strength is at least 1024 bits.
For instructions to set the certificate key strength to at least 1024 bits, see My Oracle Support note 1611578.1.
If you have changed the default, out-of-the-box memory settings for an OMS instance, then preserve the changes so that they are not lost during upgrade.
Starting with 12c release 4 (12.1.0.4), Enterprise Manager has introduced following new parameters which can be set as OMS Specific JAVA heap memory arguments:
OMS_HEAP_MIN for minimum heap size
OMS_HEAP_MAX for maximum heap size
OMS_PERMGEN_MIN for minimum permGen size
OMS_PERMGEN_MAX for maximum permGen size
To preserve the changes, follow these steps:
Use the following command to set the value for any of the JAVA heap memory properties.
ORACLE_HOME/bin/emctl set property -name <property_name> -value <number_followed_by_G_or_M>
For example,
ORACLE_HOME/bin/emctl set property -name OMS_PERMGEN_MAX -value 1024M
Run the following command to restart all the OMS instances. Here, <ORACLE_HOME> refers to the Oracle home of the OMS.
$<ORACLE_HOME>/bin/emctl stop oms -all
$<ORACLE_HOME>/bin/emctl start oms
For example,
/u01/software/em12c/mw/oms/bin/emctl stop oms -all
/u01/software/em12c/mw/oms/bin/emctl start oms
Ensure that you run the EM Prerequisite Kit and meet all the repository-related prerequisites before starting the upgrade.
To run the EM Prerequisite Kit, follow the parameters described in Oracle Enterprise Manager Cloud Control Basic Installation Guide.
Ensure that you back up the OMS (the middleware home and the inventory), Oracle Management Repository, and Oracle Software Library. The restoration requires Instance home (gc_inst) backup. In case the upgrade fails, you can always restore using the backup. For instructions to back up, refer to Oracle Enterprise Manager Cloud Control Administrator's Guide.
Ensure that you shut down the OMS you are about to upgrade and also the other OMS instances that connect to it.
IMPORTANT: If you are upgrading a multi-OMS environment using the software-only upgrade approach, then skip this step. You can stop the OMS instances after installing the software binaries as described in Installing the Enterprise Manager Cloud Control 13c Release 3 Software Binaries in Graphical Mode or Installing the Enterprise Manager Cloud Control 13c Release 3 Software Binaries in Silent Mode.
Note:
The Step 1 is applicable only if you are upgrading from 12c to 13c.
Stop the JVMD and ADP engines explicitly:
To stop them in graphical mode, access the weblogic console, and stop the JVMD and ADP weblogic managed servers manually.
To stop them in silent mode, run the following command on the OMS you are about to upgrade. Here, <ORACLE_HOME> refers to the Oracle home of the OMS.
$<ORACLE_HOME>/bin/emctl extended oms jvmd stop -all
$<ORACLE_HOME>/bin/emctl extended oms adp stop –all
For example,
/u01/software/em12c/mw/oms/bin/emctl extended oms jvmd stop -all
/u01/software/em12c/mw/oms/bin/emctl extended oms adp stop –all
Shut down the OMS you are about to upgrade and also the other OMS instances that connect to it. To do so, run the following command. Here, <ORACLE_HOME> refers to the Oracle home of the OMS.
$<ORACLE_HOME>/bin/emctl stop oms -all
For example,
/u01/software/em12c/mw/oms/bin/emctl stop oms -all
Ensure that you shut down the Management Agent that monitors the Management Services and Repository target, to prevent the Management Agent from connecting to the Management Repository for metric collections. Not shutting down this Management Agent might cause the OMS upgrade to fail.
IMPORTANT: If you are upgrading a multi-OMS environment using the software-only upgrade approach, then skip this step. You can stop the Management Agent after copying the software binaries as described in Installing the Enterprise Manager Cloud Control 13c Release 3 Software Binaries in Graphical Mode or Installing the Enterprise Manager Cloud Control 13c Release 3 Software Binaries in Silent Mode.
Ensure that you remove any proxy that is configured for the standalone Management Agents. Otherwise, the upgrade of the standalone Management Agents will fail. Although this requirement is more for Management Agent upgrade and does not affect the OMS upgrade, Oracle recommends that you remove the Management Agent-side proxy before you begin the OMS upgrade.
A new mode of the installer named Upgrade and Transition to DR Readiness has been created to assist with the process of transitioning to the Standby OMSs using Storage Replication DR architecture. In addition to the prerequisites for a standard upgrade, an Upgrade and Transition to DR Readiness upgrade requires a specific flow of preparation and post-upgrade steps that must be followed.
This new mode is:
Enabled by passing a parameter named UPGRADE_TRANSITION
Only supported for use on the first OMS
Only supported via a standard GUI installation
A software only install followed by ConfigureGC.sh does not provide support for Upgrade and Transition to DR Readiness. In addition, upgrading additional OMSs using Upgrade and Transition to DR Readiness is not supported. In a multi-OMS environment, the additional OMSs must first be deinstalled, the first OMS and related post-upgrade processes associated with the transition must be completed, and then additional OMSs can be deployed. For details on performing an Upgrade and Transition to DR Readiness instead of performing a standard upgrade, see Upgrading Enterprise Manager and Transitioning to DR Readiness.
The support for following plug-ins is discontinued and are now obsolete.
Oracle Audit Vault (oracle.em.soav).
Oracle Virtual Networking (oracle.em.sovn).
Oracle Engineered System Healthchecks (oracle.em.sehc).
Oracle Ops Center Infrastructure stack (oracle.em.sooc).
You must make sure that these plug-ins are undeployed from the Oracle Management Agents and Oracle Management Services before proceeding with your upgrade.
It is recommended to close the installer before undeploying the obsolete plug-ins. It is also recommended that these plug-ins are removed from the Self Update as well. For more information, see Undeploying Plug-ins in the Oracle Enterprise Manager Cloud Control Administrator's Guide.