3 Prerequisites for Upgrading to Enterprise Manager Cloud Control 13c Release 4

Before upgrading to Enterprise Manager Cloud Control 13c Release 4, ensure to meet the following prerequisites:

Hardware, Software, and Platform Requirements

Ensure that you meet all the prerequisites, including the hardware requirements, listed in the chapter on installing Enterprise Manager Cloud Control, in the Oracle Enterprise Manager Cloud Control Basic Installation Guide.

Also ensure that you upgrade only on the supported platforms as listed in Supported Platforms for Upgrading to Enterprise Manager Cloud Control 13c Release 4.

Supported Database Release Requirements

Ensure that the existing database is a database certified for 13c Release 4. You can see a list of certified databases in the Enterprise Manager certification matrix available on My Oracle Support. For instructions, see Accessing the Enterprise Manager Certification Matrix in Oracle Enterprise Manager Cloud Control Basic Installation Guide.

If the existing database is not of the release that is supported for 13c Release 4, then upgrade it to the supported release before you start upgrading the OMS and the Management Repository. For example, if you are upgrading from 13c Release 2, then you might have an earlier release of the database that is not supported for 13c Release 4. In this case, ensure that you first upgrade your database to the minimum database release supported for 13c Release 4, and then upgrade the Enterprise Manager system to 13c Release 4.

Before upgrading the database, ensure that you stop the OMS instances of the earlier release that are connected to the database. While upgrading the database, if you change the listener port, then follow these steps:

  1. Complete the database upgrade with the changed listener port.

  2. Start the administration server of the first OMS of the earlier release, which is connected to the database, by running the following command:

    emctl start oms -admin_only

  3. Update the first OMS and also the additional OMS instances with the changed listener port by running the following command on all the OMS instances:

    emctl config oms –store_repos_details -repos_conndesc <connect descriptor> -repos_user sysman

    When the aforementioned command is run on each OMS, you will be prompted to stop the OMS and start it again to reflect the changes. Do so on each OMS as instructed. You will have to stop them all in any case before starting the upgrade to 13c.

  4. Proceed with the upgrade of the entire Enterprise Manager system to 13c.

Note:

Ensure that OMS starts successfully and is able to access console after the connect descriptor change is completed. It is suggested to proceed with OMS upgrade only after starting OMS successfully.

Database Patch Requirements

Ensure that you apply the latest PSU on the supported database.

It is mandatory to apply the latest DB PSU on the supported database before upgrading to Enterprise Manager Cloud Control 13.4.0.0.

Optimizer Adaptive Features Disabling Requirements

  • If the Management Repository is using Oracle Database 12.1.0.2.0 and database patch 22652097 has not been applied, ensure that you disable the optimizer adaptive features by connecting to the database as SYSDBA and running the following:

    alter system set optimizer_adaptive_features=false scope=both;

    To verify that the changes have taken effect, run the following command:

    show parameter adaptive;

    You should see the following output:

    NAME                          TYPE         VALUE
    ---------------------------------------------------------------------
    optimizer_adaptive_features   boolean      FALSE 
  • If the Management Repository is using Oracle Database 12.1.0.2 with database patch 22652097 applied, connect to the database as SYSDBA and run the following:
    alter system set "_optimizer_nlj_hj_adaptive_join"= FALSE scope=both sid='*'; 
    alter system set "_optimizer_strans_adaptive_pruning" = FALSE scope=both sid='*'; 
    alter system set "_px_adaptive_dist_method" = OFF scope=both sid='*'; 
    alter system set "_sql_plan_directive_mgmt_control" = 0 scope=both sid='*'; 
    alter system set "_optimizer_dsdir_usage_control" = 0 scope=both sid='*'; 
    alter system set "_optimizer_use_feedback" = FALSE scope=both sid='*'; 
    alter system set "_optimizer_gather_feedback" = FALSE scope=both sid='*'; 
    alter system set "_optimizer_performance_feedback" = OFF scope=both sid='*'; 
  • If the Management Repository is using Oracle Database 12.2, 18.x or 19.x, there's no need to set any parameters since they will be addressed when applying patch 30912308 during the upgrade process. To apply the patch during the upgrade process follow one of the recommended methods:
    Otherwise, if you plan not to follow any of the above recommended methods to apply the patch 30912308 during the upgrade process then you must to connect to the database as SYSDBA and run the following:
    alter system set "_optimizer_nlj_hj_adaptive_join"= FALSE scope=both sid='*';
    alter system set "_optimizer_strans_adaptive_pruning" = FALSE scope=both sid='*';
    alter system set "_px_adaptive_dist_method" = OFF scope=both sid='*';
    alter system set "_sql_plan_directive_mgmt_control" = 0 scope=both sid='*';
    alter system set "_optimizer_dsdir_usage_control" = 0 scope=both sid='*';
    alter system set "_optimizer_use_feedback" = FALSE scope=both sid='*';
    alter system set "_optimizer_gather_feedback" = FALSE scope=both sid='*';
    alter system set "_optimizer_performance_feedback" = OFF scope=both sid='*';

Database Initialization Parameter in Oracle Database 12.2.0.1.0/18.0.0.0.0

Ensure that the database initialization parameter is (_allow_insert_with_update_check=TRUE) set to true in the Oracle Database 12.2.0.1.0/18.0.0.0.0 or higher that houses the Management Repository. To do so, follow these steps:

  1. Enable the database initialization parameter by setting the _allow_insert_with_update_check parameter to TRUE. To do so, run the following SQL command:

    alter system set _allow_insert_with_update_check=true scope=both

  2. Restart the database.

  3. Verify that the changes have taken effect. To do so, run the following SQL command:

    show parameter _allow_insert_with_update_check;

    You should see the following output:

    NAME                          TYPE         VALUE
    ---------------------------------------------------------------------
    _allow_insert_with_update_check   boolean      TRUE 

Management Agent Patch Requirements

Ensure that you apply the Patch 20282974 for Oracle Management Agent 12c Release 4 (12.1.0.4), on the Management Agents running on IBM AIX operating systems.

Port Requirements

Ensure that the ports used by Oracle Management Service (OMS) are not set to a value lower than or equal to 1024. If they are, then the upgrade will fail. Ports up to 1024 are typically reserved for root users (super users). Therefore, make sure the ports are greater than 1024.

Customization Removal Requirements

Ensure that you remove the following types of customization done to the OMS. Once the upgrade is complete, you can redo the customization.

  • Additional data source parameters configured in the Weblogic Server.

  • Smart card authentication for Enterprise Manager login.

Oracle BI Publisher Shutdown Requirements

Ensure that you stop the Oracle BI Publisher deployed to your previous release. To do so, use one of the following approaches:

  • If you are upgrading from Enterprise Manager Cloud Control 13c Release 2 (13.2.0.0) or 13c Release 3 (13.3.0.0) , then run emctl stop oms -all on all the OMS instances, including the ones running a BI Publisher Server.

Database Service Instance Creation Request Verification Requirements

Ensure that you check for outstanding database service instance creation requests. If there are any requests in progress, then allow them to complete. For requests that are scheduled, suspend them.

To do so, follow these steps.

  1. In Cloud Control, from the Enterprise menu, select Cloud, then select Self Service Portal.

  2. On the Infrastructure Cloud Self Service Portal page, right under the page title, select My Databases to view only database requests.

  3. In the Requests table, for requests that are in progress, allow them to complete. For requests that are scheduled, suspend them.

    To suspend the scheduled requests, click the request name. Click the Deployment tab. Click the deployment procedure listed there, and suspend it.

Repository Table Snapshot Verification Requirements

Ensure that the tables in the Management Repository do not have any snapshots created.

To verify this, log in to the Management Repository and run the following SQL query as SYSMAN user:

select master , log_table from all_mview_logs where log_owner='<EM_REPOS_USER>';

For example,

select master , log_table from all_mview_logs where log_owner='SYSMAN';

If there are snapshots created in a table, then the query displays the master table and the snapshot details. For example,

SQL> master log_table

em-violations em$violation_log

If there are snapshots, then drop them by running the following command as SYSMAN user:

SQL> Drop snapshot log on <master>;

For example,

SQL> Drop snapshot log on em-violations;

Logon and Logoff Trigger Setting Verification Requirements

Ensure that you do not have any logon or logoff triggers set in the Oracle Database that houses the Oracle Management Repository.

To verify this, log on to the database and run the following query. If the query results in anything other than zero or no rows selected, then manually disable the triggers. After completing the upgrade, you can enable them again.

  • Verify if any logon triggers are set:

    SQL> SELECT COUNT (trigger_name) FROM sys.dba_triggers WHERE TRIGGERING_EVENT LIKE 'LOGON%' AND status='ENABLED';

    SQL> SELECT trigger_name FROM sys.dba_triggers WHERE TRIGGERING_EVENT LIKE 'LOGON%' AND status='ENABLED';

  • Verify if any logoff triggers are set:

    SQL> SELECT COUNT (trigger_name) FROM sys.dba_triggers WHERE TRIGGERING_EVENT LIKE 'LOGOFF%' AND status='ENABLED';

    SQL> SELECT trigger_name FROM sys.dba_triggers WHERE TRIGGERING_EVENT LIKE 'LOGOFF%' AND status='ENABLED';

To disable the triggers, run the following query:

SQL> alter trigger <trigger_name> disable;

For example,

SQL> alter trigger EXPFIL_ALTEREXPTAB_MAINT disable;

Delete Target Operation Auditing Requirements

Ensure that you enable auditing of the Delete Target operation.

  • To view a list of operations, run the following command:

    emcli show_operations_list

    Some of the Delete operations include Delete Target, Delete Named Credential, Delete Role, Delete Rule, Delete Monitoring Template, Delete User.

  • To verify the Delete operations that are currently enabled, run the following command:

    emcli show_audit_settings

  • If the Delete Target operation is not already enabled, then run the following command to enable it.

    emcli update_audit_settings

    -operations_to_enable="name_of_the_operations_to_enable._For_all_operations_use_ALL"

    -audit_switch="ENABLE”

    -directory="db_directory_name” (Should_be_configured_with_an_OS_directory_where_the_export_service_archives_the_audit_data_files)

    -file_prefix="file_prefix" (To be used by the export service to create the file name where audit data has to be written. The default value is em_audit. You can change per your standards for all sites)

    -file_size="file_size (Bytes)" (Maximum value of each file size. The default value for this is 5000000 bytes)

    -data_retention_period="data_retention_period (Days)" (Maximum period the Enterprise Manager repository stores audit data. The default value is 365 days)

    The aforementioned parameters help you to set up or configure an archive location for archiving the audit data from the Management Repository to a file system after the retention period.

Selectively Skipping Some Job Type Updates for Reduced Downtime of Your Enterprise Manager System

While upgrading the Enterprise Manager system, job types are registered. As part of the job type registration process, all active executions corresponding to a job type are automatically upgraded to the newly registered version of the job type. This job type upgrade process is skipped for all queued and waiting executions, thereby reducing the overall downtime of the Enterprise Manager system. However, in some cases, the Enterprise Manager system might experience a considerable backlog, and if such a backlog is not cleared before initiating the upgrade, then the downtime of the Enterprise Manager system can be much longer. To circumvent this issue, you can selectively skip or postpone the upgrade of certain job types so that they are upgraded only after the downtime.

To skip or postpone some job types from being upgraded, follow these steps:

  1. Identify the job types that you want to exclude from being upgraded during the downtime.

    To do so, as a SYSMAN user, log in to the database that houses the Oracle Management Repository, and run the following query. Particularly look for the job types that have a large number of active executions.

    SELECT job_type, COUNT(1) as n_execs

    FROM MGMT_JOB_EXEC_SUMMARY

    JOIN MGMT_JOB_TYPE_INFO USING (job_type_id)

    WHERE status NOT IN (3,4,5,8,18,19,23)

    GROUP BY job_type

    HAVING COUNT(1) > 5000

    ORDER BY COUNT(1) DESC;

  2. Exclude the other job types you identified.

    To do so, run the following query to exclude a job type from the MGMT_PARAMETERS table. For each job type you want to exclude, you must run one INSERT statement. For example, if you have three job types to exclude, then run the INSERT statement three times, each with a job type you want to exclude.

    INSERT INTO MGMT_PARAMETERS(parameter_name, parameter_value) VALUES ('mgmt_job_skip_job_type_upg.1', '<job type>');

    COMMIT;

EMKEY Copy Requirements

[IN CASE OF MULTI-OMS UPGRADE, PERFORM THIS STEP ONLY FOR THE FIRST OMS UPGRADE]

Ensure that you copy the emkey from the existing OMS to the existing Management Repository. To do so, run the following command on the OMS you are about to upgrade. Here, <ORACLE_HOME> refers to the Oracle home of the OMS.

$<ORACLE_HOME>/bin/emctl config emkey -copy_to_repos [-sysman_pwd <sysman_pwd>]

For example:

/u01/software/em13c/mw/oms/bin/emctl config emkey -copy_to_repos [-sysman_pwd <sysman_pwd>]

To verify whether the emkey is copied, run the following command on the OMS you are about to upgrade. Here, <ORACLE_HOME> refers to the Oracle home of the OMS:

$<ORACLE_HOME>/bin/emctl status emkey

For example,

/u01/software/em13c/mw/oms/bin/emctl status emkey

If the emkey is copied, then you will see the following message:

The EMKey  is configured properly, but is not secure.
Secure the EMKey by running "emctl config emkey -remove_from_repos".

Certificate Key Strength Requirements

Ensure that the certificate key strength is at least 1024 bits.

For 11g Release 1 or lower, the certificates are generated with key strength 512 bits, and the certificates are carried over during upgrade to 12c Release (12.1.0.5) and subsequent upgrade to 13c Release 2 and 13c Release 3. Therefore, if you are upgrading from 11g Release 1 or lower, then your certificates will continue to be with 512 bits, and this will eventually cause multiple communication issues. Therefore, ensure that the certificate key strength is at least 1024 bits.

For instructions to set the certificate key strength to at least 1024 bits, see My Oracle Support note 1611578.1.

Starting with 13c Release 4, the minimum certificate supported is SHA 512/1024. MD5 certificates are no longer supported. If you have any MD5 certificate, you will see the following message stopping the upgrade process:

The upgrade process has detected MD5 certificate usage for some agent-OMS communication. These hosts and related targets will continue to be managed post-upgrade. However, it is strongly recommended to reconfigure them to use SHA-based certificates, since future versions of Enterprise Manager may not support MD5 certificates. Review the file /tmp/OraInstall<time_stamp>/md5Target.txt to find the agents that use MD5-based certificates. Follow the instructions outlined in My Oracle Support note 2179909.1 to reconfigure them with SHA-based certificates.

After you address it, you can restart the upgrade process.

Out-of-Box Memory Settings Backup Requirements

If you have changed the default out-of-the-box memory settings for an OMS instance, then you can save the default values as a backup so that they are not lost during upgrade.

You can verify the default values using the command ORACLE_HOME/bin/emctl get property -name property_name.

For example,

ORACLE_HOME/bin/emctl get property -name OMS_PERMGEN_MAX

Starting with 12c release 4 (12.1.0.4), Enterprise Manager has introduced following new parameters which can be set as OMS  Specific JAVA heap memory arguments:

  • OMS_HEAP_MIN for minimum heap size

  • OMS_HEAP_MAX for maximum heap size

  • OMS_PERMGEN_MIN for minimum permGen size

  • OMS_PERMGEN_MAX for maximum permGen size

To save the default values for out-of-the-box memory settings for an OMS instance, follow these steps:

  1. Use the following command to set the value for any of the JAVA heap memory properties.

    ORACLE_HOME/bin/emctl set property -name <property_name> -value <number_followed_by_G_or_M>

    For example,

    ORACLE_HOME/bin/emctl set property -name OMS_PERMGEN_MAX -value 1024M

  2. Run the following command to restart all the OMS instances. Here, <ORACLE_HOME> refers to the Oracle home of the OMS.

    $<ORACLE_HOME>/bin/emctl stop oms -all

    $<ORACLE_HOME>/bin/emctl start oms

    For example,

    /u01/software/em13c/mw/oms/bin/emctl stop oms -all

    /u01/software/em13c/mw/oms/bin/emctl start oms

Prerequisite Check and Environment Validation Requirements

Ensure that you run the EM Prerequisite Kit and meet all the repository-related prerequisites before starting the upgrade.

To run the EM Prerequisite Kit, follow the parameters described in Oracle Enterprise Manager Cloud Control Basic Installation Guide.

OMS Backup Requirements

Ensure that you back up the OMS (the middleware home and the inventory), Oracle Management Repository, and Oracle Software Library. The restoration requires Instance home (gc_inst) backup. In case the upgrade fails, you can always restore using the backup. For instructions to back up, refer to Oracle Enterprise Manager Cloud Control Administrator's Guide.

OMS Shutdown Requirements

Ensure that you shut down the OMS you are about to upgrade and also the other OMS instances that connect to it.

IMPORTANT: If you are upgrading a multi-OMS environment using the software-only upgrade approach, then skip this step. You can stop the OMS instances after installing the software binaries as described in Installing the Enterprise Manager Cloud Control 13c Release 4 Software Binaries in Graphical Mode or Installing the Enterprise Manager Cloud Control 13c Release 4 Software Binaries in Silent Mode.

  1. Note:

    The Step 1 is applicable only if you are upgrading from 12c to 13c.

    Stop the JVMD and ADP engines explicitly:

    To stop them in graphical mode, access the weblogic console, and stop the JVMD and ADP weblogic managed servers manually.

    To stop them in silent mode, run the following command on the OMS you are about to upgrade. Here, <ORACLE_HOME> refers to the Oracle home of the OMS.

    $<ORACLE_HOME>/bin/emctl extended oms jvmd stop -all

    $<ORACLE_HOME>/bin/emctl extended oms adp stop –all

    For example,

    /u01/software/em12c/mw/oms/bin/emctl extended oms jvmd stop -all

    /u01/software/em12c/mw/oms/bin/emctl extended oms adp stop –all

  2. Shut down the OMS you are about to upgrade and also the other OMS instances that connect to it. To do so, run the following command. Here, <ORACLE_HOME> refers to the Oracle home of the OMS.

    $<ORACLE_HOME>/bin/emctl stop oms -all

    For example,

    /u01/software/em12c/mw/oms/bin/emctl stop oms -all

Management Agent Shutdown Requirements

Ensure that you shut down the Management Agent that monitors the Management Services and Repository target, to prevent the Management Agent from connecting to the Management Repository for metric collections. Not shutting down this Management Agent might cause the OMS upgrade to fail.

IMPORTANT: If you are upgrading a multi-OMS environment using the software-only upgrade approach, then skip this step. You can stop the Management Agent after copying the software binaries as described in Installing the Enterprise Manager Cloud Control 13c Release 4 Software Binaries in Graphical Mode or Installing the Enterprise Manager Cloud Control 13c Release 4 Software Binaries in Silent Mode.

Management Agent Proxy Removal Requirements

Ensure that you remove any proxy that is configured for the standalone Management Agents. Otherwise, the upgrade of the standalone Management Agents will fail. Although this requirement is more for Management Agent upgrade and does not affect the OMS upgrade, Oracle recommends that you remove the Management Agent-side proxy before you begin the OMS upgrade.

Additional Preparation and Steps Required for Performing an Upgrade and Transition to DR Readiness

A new mode of the installer named Upgrade and Transition to DR Readiness has been created to assist with the process of transitioning to the Standby OMSs using Storage Replication DR architecture. In addition to the prerequisites for a standard upgrade, an Upgrade and Transition to DR Readiness upgrade requires a specific flow of preparation and post-upgrade steps that must be followed.

This new mode is:

  • Enabled by passing a parameter named UPGRADE_TRANSITION

  • Only supported for use on the first OMS

  • Only supported via a standard GUI installation

A software only install followed by ConfigureGC.sh does not provide support for Upgrade and Transition to DR Readiness. In addition, upgrading additional OMSs using Upgrade and Transition to DR Readiness is not supported. In a multi-OMS environment, the additional OMSs must first be deinstalled, the first OMS and related post-upgrade processes associated with the transition must be completed, and then additional OMSs can be deployed. For details on performing an Upgrade and Transition to DR Readiness instead of performing a standard upgrade, see Upgrading Enterprise Manager and Transitioning to DR Readiness.

Undeploy Obsolete Plug-ins

The support for following plug-ins is discontinued and are now obsolete.

  • Oracle Audit Vault (oracle.em.soav).

  • Oracle Virtual Networking (oracle.em.sovn).

  • Oracle Engineered System Healthchecks (oracle.em.sehc).

  • Oracle Ops Center Infrastructure stack (oracle.em.sooc).

You must make sure that these plug-ins are undeployed from the Oracle Management Agents and Oracle Management Services before proceeding with your upgrade.

It is recommended to close the installer before undeploying the obsolete plug-ins. It is also recommended that these plug-ins are removed from the Self Update as well. For more information, see Undeploying Plug-ins in the Oracle Enterprise Manager Cloud Control Administrator's Guide.