6 Patching Oracle Database Appliance

To keep Oracle Database Appliance running with the latest software, check for and apply Oracle Database Appliance patch bundles when they are released.

About Patching Oracle Database Appliance

Use the Oracle Database Appliance Patch Bundle to patch your appliance.

Patches offer new features and may improve the functionality of existing features.

About Out of Place Patching

Starting with Oracle Database Appliance release 19.11, Oracle Database Appliance uses the out-of-place patching model. This means that a new software Oracle home is created when patching an Oracle Grid Infrastructure or Oracle Database home. Starting with Oracle Database Appliance release 19.11, the patches for Oracle Grid Infrastructure are available as images. All procedures remain the same.

About MySQL Server and DCS Agent

Starting with Oracle Database Appliance release 19.10, DCS Agent uses MySQL server as its metadata store. MySQL server is automatically installed and configured, during provisioning, when creating the appliance. Similarly, during patching, the odacli update-dcscomponents command automatically installs and configures MySQL server. MySQL server processes do not run with root operating system user account. Instead, MySQL server processes run with a new operating system user account called odamysql. Note that the odamysql operating system user is created only for ownership purposes and the odamysql user account owns only the MySQL directories. The odamysql operating system user cannot log into the appliance. The odamysql operating system user account for MySQL ensures role separation between the user owning Oracle databases on the appliance and the metadata used for DCS operations.

About Oracle Database Appliance Patch Bundles

The Oracle Database Appliance Patch Bundle contains the latest patches for DCS Admin, DCS Components, OAK, Oracle ILOM, firmware, enhancements, and fixes.

The Oracle Database Appliance Release Notes include information about the latest Oracle Database Appliance patch bundle and a list of component versions in the patch bundle. Oracle recommends updating the appliance with the latest Oracle Database Appliance software version to take advantage of new features, fixes, and the latest supported component versions. See the Oracle Database Appliance Release Notes for the upgrade paths.

Caution:

Do not patch Oracle Database Appliance using generic patches for Oracle Grid Infrastructure or Oracle Linux. Also do not use individual infrastructure patches, such as firmware patches. You must only use Oracle Database Appliance patches. If you use patches that are not intended for Oracle Database Appliance, or if you use OPatch, or a similar patching tool, then the metadata maintained by Oracle Database Appliance is not updated, and you cannot complete future patch updates. If you apply out-of-cycle Oracle Database Release Update (RU), then ensure that you also follow the recommendations described in the Readme for the RU.

The patch bundle provides all relevant patches for the entire system, including the following:

  • Oracle Database patches available as images
  • BIOS

  • Hardware drivers

  • Hardware Management Pack (HMP) and firmware drivers for various components

  • Oracle Appliance Manager

  • Oracle Linux

  • Oracle VM Server

  • Java Runtime Environment (JRE)

  • Oracle Integrated Lights Out Manager (Oracle ILOM)

  • Oracle Auto Service Request (Oracle ASR)

  • Oracle Intelligent Platform Management Interface (Oracle IPMI)

  • Network Card Patches for relevant hardware models

About Updating Oracle Linux RPMs

While not recommended, you can update some Oracle Linux RPMs for Oracle Database Appliance. Do not update or customize Oracle Linux kernel, Oracle Optimal Flexible Architecture, Oracle InfiniBand, or related software. Other software may be installed, updated, or customized. However, the Oracle Database Appliance update may not carry newer version dependencies of customized components. Therefore, you might be required to remove and subsequently reapply site-specific changes to successfully update Oracle Database Appliance in the future.

Caution:

For Oracle Database Appliance, do not update the following:

  • Oracle Linux Kernel (kernel*)

  • Oracle Optimal Flexible Architecture (ofa*)

  • Oracle RDMA packages (oracle-rdma-release)

For storage, do not apply any RPM updates.

About Patching Pre-Checks

Patching pre-checks are designed to detect and flag problems that might be encountered during patching.

The pre-checks view the state of the appliance, including the version and hardware specific upgrades, verify that the necessary conditions and checks are complete before attempting an upgrade, and identify conditions that might cause an upgrade to fail. You can avoid potential patching problems by correcting the conditions and resolving the issues before applying the patch.

The pre-checks commands are available on Oracle Database Appliance release 12.2.1.2.0 or later. Update the DCS Agent to ensure that the latest updated pre-checks commands are available when you run these commands.

Components Verified by Patching Pre-Checks

Check Component Verified
System patching pre-checks

Checks that the DCS Agent is online on all nodes

Checks that there is sufficient space for the update

Validates the minimum agent version and validates the patching tag

Confirms that a valid patch is in the repository and calculates the patch bundle md5 checksum

Server patching pre-checks
For operating system:
  • Validates supported versions
  • Validates minimum supported versions
  • Validates patching tag
  • Validates whether patch location is available
  • Verifies OS patch
  • Validates command execution
For Oracle ILOM:
  • Validates supported versions
  • Validates minimum supported versions
  • Validates patching tag
  • Validates whether patch location is available
  • Checks for Oracle ILOM patch version
  • Validates Patch location
  • Validates command execution
For Oracle Grid Infrastructure:
  • Validates supported Oracle Grid Infrastructure versions
  • Validates available space
  • Validates whether Oracle Clusterware is running
  • Validates patching tag
  • Validates whether system is provisioned
  • Validates whether Oracle ASM is online
  • Validates the minimum agent version
  • Validates the central inventory
  • Validates patching locks
  • Validates whether clones location exists
  • Validates DB start dependencies
  • Validates DB stop dependencies
  • Validates command execution
For Oracle ORAchk:
  • Validates whether orachk is running
  • Validates command execution

Confirms that the bare metal system is upgraded, in the case of DB systems.

Verifies and confirms that the database start/stop dependencies are correct.

Confirms that patching locks are not present.

Verifies that the central inventory is accurate.

Database home patching pre-checks

Confirms that Oracle Grid Infrastructure is installed

Confirms that the DCS Agent is at the required version

Confirms that Oracle Clusterware is running on all nodes

Confirms that Oracle Grid Infrastructure is upgraded

Validates that there is sufficient disk space available

Checks if the disk group for database homes on Oracle ACFS is configured

Checks whether the clone file is present in the repository

Storage pre-checks

Validates storage patch location and tag

Validates patching location and tag

Validates command execution

Validates Oracle ASM disk groups status

About Enabling NUMA on Oracle Database Appliance

Understand how you can enable NUMA on bare metal, KVM, and DB Systems on Oracle Database Appliance.

NUMA (Non-Uniform Memory Access) can help facilitate efficient use of the underlying NUMA hardware. On Oracle Database Appliance, you can enable NUMA for databases on bare metal and DB systems, CPU pools, and for KVM applications.

About Enabling NUMA During Provisioning of Oracle Database Appliance

When you provision Oracle Database Appliance with release 19.12, NUMA is automatically enabled on the database, DB system and CPU pools. Hence, when you create a database, DB system, or CPU pool on an Oracle Database Appliance release 19.12 deployment, NUMA is automatically enabled and no additional configuration is required. NUMA is not enabled by default for application VMs.

About Enabling NUMA After Patching Oracle Database Appliance

When you patch to Oracle Database Appliance release 19.12, the odacli update-server command sets NUMA at the system level. New databases created after patching are NUMA-enabled. For existing databases, enable NUMA by running the command alter system set "enable_numa_support"=true on the database.

After patching the bare metal system to release 19.12, as a one-time activity, you must run the command odacli remap-cpupools to enable NUMA on all existing CPU pools (BM, VM, dbsystem).

For existing DB systems, run the command odacli modify-dbsystem --enable-numa to enable NUMA for the dbsystem and the database of the DB system.

NUMA is not enabled by default for application VMs. For new and existing application VMs, run the command odacli modify-vm --enable-numa to enable NUMA for the VM configuration. You must also enable NUMA for the components inside the VM such as operating system and application running inside the VM.

See the topic Patching Oracle Database Appliance Bare Metal Systems Using the Command-Line for more details about the steps to patch your deployment and enable NUMA.

Minimizing Downtime When Patching Oracle Database Appliance

Understand best practices to minimize downtime when you apply patches to your Oracle Database Appliance deployment.

Oracle Database Appliance does not apply patches in a rolling manner for high-availability systems. However, there are recommendations to minimize downtime during patching.

Recommendations for Minimizing Downtime During Patching

  • To minimize database patching downtime, it is recommended that you use Oracle RAC or Oracle RAC One Node on an Oracle Database Appliance high-availability system.
  • If available, use Oracle Data Guard and patch the standby system first. After you patch the standby system, switch the workload to the standby system, patch the primary system, and then switch the workload back to the primary system.
  • Always use Oracle Database Appliance Backup and Recovery (ODABR) to back up the system disks to ensure easy restore, if the patching operation fails. ODABR restores the system disk to pre-patching state.
  • The odacli update-storage command has a --rolling option for storage devices (SSD, HDD, NVMe) to minimize patching downtime. There could be other shared storage component firmware that require a system restart and do not support rolling. Oracle Database Appliance storage patching can be deferred, but it is not recommended.

See Also:

Oracle Database High Availability Overview and Best Practices Guide in the Oracle Database 19c Documentation Library

Patching Oracle Database Appliance Bare Metal Systems Using the Command-Line

Follow these steps to apply patches to your Oracle Database Appliance bare metal deployment and existing Oracle Database homes, using CLI commands.

To patch your Oracle Database Appliance deployment and your existing database homes to the current release, you must download the Oracle Database Appliance Server patch, Oracle Grid Infrastructure clone file, and Oracle Database clone files and update the repository on the bare metal system.

Note:

Run the commands in this topic in the same order as documented. Run the odacli update-dcsadmin, odacli update-dcscomponents, and odacli update-dcsagent commands before running the odacli update-server command.
The Oracle Database Appliance Backup Recovery (ODABR) tool helps recover a system to the pre-patch state, should any potential errors occur during the patching process that may render the system unusable. Download and install the latest version of the Oracle Database Appliance Backup Recovery (ODABR) tool to enable creation of snapshots before patching. See My Oracle Support Note 2466177.1 for the procedure to install ODABR.

https://support.oracle.com/rs?type=doc&id=2466177.1

Follow these steps to apply patches to your Oracle Database Appliance and update existing database homes.
  1. Download the Oracle Database Appliance Server Patch for the ODACLI/DCS stack (patch 33152232), Oracle Grid Infrastructure clone files (patch 30403673), and Oracle Database clone files (patch 30403662) from My Oracle Support to a temporary location on an external client. Refer to the release notes for details about the patch numbers and software for the latest release.
    For example, download the server patch for 19.12:
    p33152232_1912000_Linux-x86-64.zip
  2. Unzip the software — it contains README.html and one or more zip files for the patch.
    unzip p33152232_1912000_Linux-x86-64.zip

    The zip file contains the following software file:

    oda-sm-19.12.0.0.0-date-server.zip
  3. Copy all the software files from the external client to Oracle Database Appliance. For High-Availability deployments, copy the software files to only one node. The software files are copied to the other node during the patching process. Use the scp or sftp protocol to copy the bundle.
    Example using scp command:
    # scp software_file root@oda_host:/tmp
    Example using sftp command:
    # sftp root@oda_host
    Enter the root password, and copy the files.
    put software_file
  4. Update the repository with the server software file:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-repository -f /tmp/software_file

    For example, for 19.12:

    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-repository -f /tmp/oda-sm-19.12.0.0.0-date-server.zip
  5. Confirm that the repository update is successful:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli describe-job -i job_ID
  6. Update DCS admin:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-dcsadmin -v 19.12.0.0.0
  7. Update the DCS components:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-dcscomponents -v 19.12.0.0.0

    If you are patching from Oracle Database Appliance release 19.9 or earlier, then the odacli update-dcscomponents command updates the Zookeeper, installs MySQL, migrates metadata from Derby to MySQL and updates other DCS components such as the DCS Agent, DCS CLI, and DCS Controller on Oracle Database Appliance.

    If you are patching from Oracle Database Appliance release 19.10 or later, then the odacli update-dcscomponents command updates the Zookeeper, MySQL, and updates other DCS components such as the DCS Agent, DCS CLI, and DCS Controller on Oracle Database Appliance.

    If the DCS components are updated, then the message "status" : "Success" is displayed on the command line. For failed updates, fix the error and then proceed with the update by re-running the odacli update-dcscomponents command. See the topic Resolving Errors When Updating DCS Components During Patching about more information about DCS components checks errors.

  8. Update the DCS agent:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-dcsagent -v 19.12.0.0.0
  9. Update the repository with the Oracle Grid Infrastructure clone file and the Oracle Database clone file:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-repository -f /tmp/software_file

    For example, for 19.12:

    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-repository -f /tmp/odacli-dcs-19.12.0.0.0-date-GI-19.12.0.0.zip
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-repository -f /tmp/odacli-dcs-19.12.0.0.0-date-DB-19.12.0.0.zip
  10. Run patching pre-checks. You must run the odacli create-prepatchreport command before you patch your server; otherwise, the odacli update-server command fails with an error message prompting you to run the patching pre-checks.
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli create-prepatchreport -s -v version

    For example, for 19.12:

    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli create-prepatchreport -s -v 19.12.0.0.0
  11. Verify that the patching pre-checks ran successfully:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli list-prepatchreports

    For example:

    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli list-prepatchreports
    3797791f-72ac-42b6-8bb8-ed5e5c9b32c1 Patch pre-checks for [OS, GI, ORACHKSERVER] June 30, 2020 6:29:30 AM UTC Running  

    Use the command odacli describe-prepatchreport to view details of the pre-patch report. Fix the warnings and errors mentioned in the report and proceed with the server patching.

    For high-availability deployments, the following error is displayed in the pre-patch report:
    Evaluate GI patching Failed Internal error encountered:
    /u01/app/19.12.0.0/gridDCS-10001:
    PRGO-1022 : Working copy "OraGrid191200" already exists..... 

    Ignore the error and complete the server patching with the -f option.

  12. Apply the server update. Note that the --local option is not available with the odacli update-server command in this release.
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-server -v version

    For example, for 19.12:

    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-server -v 19.12.0.0.0
    For high-availability deployments, in this release, you must use the -f option to override precheck failure listed in step 10 and complete the server patching.
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-server -v 19.12.0.0.0 -f
  13. Confirm that the server update is successful:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli describe-job -i job_ID

    Note:

    After successful server update, it is recommended that you delete any ODABR snapshot created before the update. If you want to retain any ODABR snapshot when you update the storage and other components, then consider that the presence of active ODABR snapshots may affect system performance. See My Oracle Support Note 2466177.1 for the procedure to create and delete ODABR snapshots: https://support.oracle.com/rs?type=doc&id=2466177.1
  14. Before you update the storage components, run the odacli create-prepatchreport command with the -st option.
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli create-prepatchreport -st -v version

    For example, for 19.12:

    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli create-prepatchreport -st -v 19.12.0.0.0
  15. Verify that the patching pre-checks ran successfully:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli list-prepatchreports

    For example:

    [root@oda1 opt]# odacli list-prepatchreports
    
    ID Description Created Status
    ---------------------------------------- -----------------------------------------------------------------
    b030d6e2-bb10-465c-81d3-ccaa48cea681 Patch pre-checks for [STORAGE] July 28, 2021 11:26:45 AM UTC Success  

    Use the command odacli describe-prepatchreport to view details of the pre-patch report. Fix the warnings and errors mentioned in the report and proceed with the storage components patching.

  16. Update the storage components.
    Specify the --rolling option to patch shared disks in a rolling fashion.
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-storage -v version --rolling

    For example, for 19.12:

    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-storage -v 19.12.0.0.0 --rolling
  17. Starting with Oracle Database Appliance release 19.11, Oracle Database homes are not created on the local disk. Oracle Database homes are created on an Oracle ACFS-managed file system. For a database user oracle, the new database homes are created under /u01/app/odaorahome/oracle/.
    If the database home is not already configured on Oracle ACFS, then before patching the database home, configure the database home storage with the odacli configure-dbhome-storage command. For example:
    [root@oda1 opt]# odacli configure-dbhome-storage -dg DATA
    For information about managing database homes on Oracle ACFS, see the topic Managing Database Home Storage.
  18. To patch existing Oracle Database Homes to release 19.12:
    1. Ensure that the repository is updated with the Oracle Database clone files. If you have not updated the repository with the Oracle Database clone files, then update the repository. Note that you must update the DCS agent before you update the repository with the Oracle Database clone files.
      [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-repository -f /tmp/DB_software_file
      [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-repository -f /tmp/odacli-dcs-19.12.0.0.0-date-DB-19.12.0.0.zip
    2. Run the patching pre-checks and review the prepatch report. You must run the odacli create-prepatchreport command before you patch your Oracle Database homes; otherwise, the odacli update-dbhome command fails with an error message prompting you to run the patching pre-checks.
      [root@oda1 opt]# /opt/oracle/dcs/bin/odacli create-prepatchreport --dbhome --dbhomeid DB_Home_ID -v release_number
      [root@oda1 opt]# odacli describe-prepatchreport -i Job_ID
      [root@oda1 opt]# /opt/oracle/dcs/bin/odacli create-prepatchreport --dbhome --dbhomeid 7c67c5b4-f585-4ba9-865f-c719c63c0a6e -v 19.12.0.0.0
      [root@oda1 opt]# odacli describe-prepatchreport -i 39ef1eeb-70d3-47ad-b3f5-48960ca0607b
      If the disk group for database homes on Oracle ACFS is not set up as described in step 16, then the prepatch report lists the error The system is not set up to create database homes on ACFS. Run the odacli configure-dbhome-storage command to configure the disk group for database homes on Oracle ACFS.
    3. For Oracle Database Appliance release 19.12, if the following error messages are displayed in the pre-patch report, then you must run the odacli update-dbhome command with the -f option:
      For high-availability deployments, the following errors are displayed in the pre-patch report:
      Evaluate DBHome patching with Failed Internal error encountered: Internal RHP error encountered: PRGO-1693 : The database patching cannot be completed
      in a rolling manner because the target patched home at "/u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_3" contains non-rolling bug fixes "32327201"
      compared to the source home at "/u01/app/oracle/product/19.0.0.0/dbhome_1"....
      
      Evaluate DBHome patching with Failed Internal error encountered: Internal RHP error encountered: PRCT-1003 : failed to run "rhphelper" on node "node1"
      PRCT-1014 : Internal error: RHPHELP12102_main-02... 

      Ignore the errors and complete the database home patching with the -f option.

      [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-dbhome --dbhomeid 7c67c5b4-f585-4ba9-865f-c719c63c0a6e -v 19.12.0.0.0 -f
    4. if the pre-patch report does not show any errors, then update the database home to the latest release:
      [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-dbhome --dbhomeid DB_Home_ID -v release_number

      For example, for 19.12:

      [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-dbhome --dbhomeid 7c67c5b4-f585-4ba9-865f-c719c63c0a6e -v 19.12.0.0.0
  19. Confirm that the patch jobs are successful:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli list-jobs
  20. After patching the bare metal systems and database homes, enable NUMA (Non-Uniform Memory Access) on existing databases of release 19.12 or earlier, by running the following SQL*Plus command:
    SQL> alter system set "_enable_numa_support"=true scope=spfile sid='*';

    Restart the database after running the command to ensure that the changes take effect.

    For more information about enabling NUMA, see the topic About Enabling NUMA on Oracle Database Appliance.

  21. If you have deployed DB systems, VMs, or CPU pools, perform the following steps:
    1. Run the odacli remap-cpupools command:
      [root@oda1 opt]# /opt/oracle/dcs/bin/odacli remap-cpupools
    2. If you have DB systems in your deployment, then run the odacli modify-dbsystem command. Or, if you have VMs in your KVM deployment then run the odacli modify-vm --enable-numa command for VMs. That is:

      For DB systems:

      odacli modify-dbsystem --enable-numa

      For VMs:

      odacli modify-vm --enable-numa

Patching Oracle Database Appliance DB Systems Using the Command-Line

Follow these steps to apply patches to your Oracle Database Appliance DB system deployment and existing Oracle Database home in the DB system, using CLI commands.

To patch your Oracle Database Appliance deployment and your existing database homes to the current release, you must download the Oracle Database Appliance Server Patch and update the repository.

Note:

Run the commands in this topic in the same order as documented. Run the odacli update-dcsadmin, odacli update-dcscomponents, and odacli update-dcsagent commands before running the odacli update-server command.
Ensure that you have patched your Oracle Database Appliance bare metal deployment to release 19.12 before you patch your DB system.
Follow these steps to apply patches to your Oracle Database Appliance DB system deployment and update the existing database home in the DB system.
  1. Ensure that the Oracle Database Appliance Server Patch for the ODACLI/DCS stack (patch 33152232), Oracle Grid Infrastructure clone files (patch 30403673), and Oracle Database clone files (patch 30403662) are available in the repository. Run the odacli describe-dbsystem-image command to view whether the clone files are available in the system.
    [root@oda1 opt]# odacli describe-dbsystem-image
  2. Connect to the DB system.
  3. Update DCS admin:
    [root@dbsystem1 opt]# /opt/oracle/dcs/bin/odacli update-dcsadmin -v 19.12.0.0.0
  4. Update the DCS components:
    [root@dbsystem1 opt]# /opt/oracle/dcs/bin/odacli update-dcscomponents -v 19.12.0.0.0

    If the DCS components are updated, then the message "status" : "Success" is displayed on the command line. For failed updates, fix the error and then proceed with the update by re-running the odacli update-dcscomponents command. See the topic Resolving Errors When Updating DCS Components During Patching about more information about DCS components checks errors.

  5. Update the DCS agent:
    [root@dbsystem1 opt]# /opt/oracle/dcs/bin/odacli update-dcsagent -v 19.12.0.0.0
  6. Run patching pre-checks. You must run the odacli create-prepatchreport command before you patch your server; otherwise, the odacli update-server command fails with an error message prompting you to run the patching pre-checks.
    [root@dbsystem1 opt]# /opt/oracle/dcs/bin/odacli create-prepatchreport -s -v version

    For example, for 19.12:

    [root@dbsystem1 opt]# /opt/oracle/dcs/bin/odacli create-prepatchreport -s -v 19.12.0.0.0
  7. Verify that the patching pre-checks ran successfully:
    [root@dbsystem1 opt]# /opt/oracle/dcs/bin/odacli list-prepatchreports

    For example:

    [root@dbsystem1 opt]# odacli list-prepatchreports
    ID Description Created Status
    ---------------------------------------- --------------------------------------------------------------------------- ----------------------------------- ----------
    67029f6b-db5d-4f65-9367-c5986c3050d4 Patch pre-checks for [OS, GI, ORACHKSERVER] August 18, 2021 8:44:24 AM GMT Success 

    Use the command odacli describe-prepatchreport to view details of the pre-patch report. Fix the warnings and errors mentioned in the report and proceed with the server patching.

  8. Apply the server update. Note that the --local option is not available with the odacli update-server command in this release.
    [root@dbsystem1 opt]# /opt/oracle/dcs/bin/odacli update-server -v version

    For example, for 19.12:

    [root@dbsystem1 opt]# /opt/oracle/dcs/bin/odacli update-server -v 19.12.0.0.0

    You can use the -f option to override precheck failures and complete the server patching.

  9. Confirm that the server update is successful:
    [root@dbsystem1 opt]# /opt/oracle/dcs/bin/odacli describe-job -i job_ID
  10. To patch existing Oracle Database Homes to release 19.12:
    1. On the bare metal system, run the command odacli describe-dbsystem-image to check that the repository is updated with the required Oracle Database clone files. If the database clone file is not displayed in the command output, then run the command odacli update-repository to update the repository with the database clone files:
      [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-repository -f /tmp/DB_software_file
      [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-repository -f /tmp/odacli-dcs-19.12.0.0.0-date-DB-19.12.0.0.zip
    2. Run the patching pre-checks and review the prepatch report. You must run the odacli create-prepatchreport command before you patch your Oracle Database homes; otherwise, the odacli update-dbhome command fails with the error message prompting you to run the patching pre-checks.
      [root@dbsystem1 opt]# /opt/oracle/dcs/bin/odacli create-prepatchreport --dbhome --dbhomeid DB_Home_ID -v release_number
      [root@dbsystem1 opt]# odacli describe-prepatchreport -i Job_ID
      [root@dbsystem1 opt]# /opt/oracle/dcs/bin/odacli create-prepatchreport --dbhome --dbhomeid 7c67c5b4-f585-4ba9-865f-c719c63c0a6e -v 19.12.0.0.0
      [root@dbsystem1 opt]# odacli describe-prepatchreport -i 39ef1eeb-70d3-47ad-b3f5-48960ca0607b
    3. Update the database home to the latest release:
      [root@dbsystem1 opt]# /opt/oracle/dcs/bin/odacli update-dbhome --dbhomeid DB_Home_ID -v release_number

      For example, for 19.12:

      [root@dbsystem1 opt]# /opt/oracle/dcs/bin/odacli update-dbhome --dbhomeid 7c67c5b4-f585-4ba9-865f-c719c63c0a6e -v 19.12.0.0.0
    4. For Oracle Database Appliance release 19.12, the following error messages are displayed in the pre-patch report. You must rerun the odacli update-dbhome command again with the -f option:
      The following errors are displayed in the pre-patch report:
      Evaluate DBHome patching with Failed Internal error encountered: Internal RHP error encountered: PRGO-1693 : The database patching cannot be completed
      in a rolling manner because the target patched home at "/u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_3" contains non-rolling bug fixes "32327201"
      compared to the source home at "/u01/app/oracle/product/19.0.0.0/dbhome_1"....
      
      Evaluate DBHome patching with Failed Internal error encountered: Internal RHP error encountered: PRCT-1003 : failed to run "rhphelper" on node "node1"
      PRCT-1014 : Internal error: RHPHELP12102_main-02... 

      Ignore the errors and complete the database home patching with the -f option.

      [root@dbsystem1 opt]# /opt/oracle/dcs/bin/odacli update-dbhome --dbhomeid 7c67c5b4-f585-4ba9-865f-c719c63c0a6e -v 19.12.0.0.0 -f
  11. Confirm that the patch jobs are successful:
    [root@dbsystem1 opt]# /opt/oracle/dcs/bin/odacli list-jobs

Patching Oracle Database Appliance Bare Metal Systems Using the Browser User Interface

Upload the Oracle Database Appliance Server Patch to the patch repository, deploy the patch bundle using the Browser User Interface, and then update the DCS agent and components using CLI commands.

Updating the Patch Repository Using the Browser User Interface

  1. Download the Oracle Database Appliance patches from My Oracle Support and save them in a directory on the appliance. See the Oracle Database Appliance Release Notes for a list of available patches and links to download the patches.

  2. Log into the Browser User Interface with the oda-admin user name and password.

    https://Node0–host-ip-address:7093/mgmt/index.html
  3. Click Patch Manager.

  4. In the Update Patch Repository page, enter the absolute file path and patch name, then click Update Repository to start the update repository job. You can list more than one file to update to the repository with a comma separated list, without spaces, or you can update each file, one at a time.

  5. Click Activity to monitor the progress. When the job completes successfully, the Patch Repository is updated.

Updating the DCS Admin and DCS Components After Updating the Repository and Before Patching

Before patching your appliance using the Browser User Interface, you must update the DCS admin and DCS components manually, through CLI commands.

Follow these steps:

  1. Update DCS admin:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-dcsadmin -v 19.12.0.0.0
  2. Confirm that the update is successful:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli describe-job -i job_ID
  3. Update the DCS components:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-dcscomponents -v 19.12.0.0.0
    If the DCS components are updated, then the message "status" : "Success" is displayed on the command line. For failed updates, fix the error and then proceed with the update.

    Note:

    You must log into the Browser User Interface again after running the odacli update-dcscomponents command.

Running Patching Prechecks and Patching Agent, Server, and Storage Using the Browser User Interface

  1. Do not have any jobs running or pending during the update window.

  2. Click the Appliance tab, and then click Patch Manager in the Browser User Interface.

  3. Wait for the Component Details to load. If you just updated the Patch Repository, click Refresh in the Component Details section of the page.

    After the patch is uploaded to the Patch Repository, the Component Details on the page are updated with the Installed Version and Available Version for each component.

  4. On the Patch Manager page, click Refresh to refresh the Component Details. Select Update Server, then click Apply Patch to begin the job to patch the server components. Select Precheck to run patching prechecks. You must run the patching prechecks before updating the server and other components.

    The DCS agent is automatically updated whenever the Server or Storage components are updated.

    When the job finishes, go to the next step. Click Activity for job status.

  5. On the Patch Manager page, click Refresh to refresh the Component Details. Select Update Storage, then click Apply Patch to begin the job to patch the storage components. Select Precheck if you want to run patching prechecks.

    For high-availability environment, you can select the Rolling check box to perform rolling patching of storage components.

  6. Click Activity to monitor the progress, and verify that the job completes successfully.

All patching-related information is logged in the dcs-agent log file at /opt/oracle/dcs/log/dcs-agent.log.

Patching Oracle Database Appliance DB Systems Using the Browser User Interface

Upload the Oracle Database Appliance Server Patch to the patch repository, deploy the patch bundle using the Browser User Interface, and then update the DCS agent and components using CLI commands.

Updating the Patch Repository Using the Browser User Interface

  1. Download the Oracle Database Appliance patches from My Oracle Support and save them in a directory on the appliance. See the Oracle Database Appliance Release Notes for a list of available patches and links to download the patches.

  2. Log into the Browser User Interface within the bare metal system, with the oda-admin user name and password.

    https://Node0–host-ip-address:7093/mgmt/index.html
  3. Click Patch Manager.

  4. In the Update Patch Repository page, enter the absolute file path and patch name, then click Update Repository to start the update repository job. You can list more than one file to update to the repository with a comma separated list, without spaces, or you can update each file, one at a time.

  5. Click Activity to monitor the progress. When the job completes successfully, the Patch Repository is updated.

Updating the DCS Admin and DCS Components After Updating the Repository and Before Patching

Before patching your DB system using the Browser User Interface, you must update the DCS admin and DCS components manually, through CLI commands.

Follow these steps:

  1. Update DCS admin:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-dcsadmin -v 19.12.0.0.0
  2. Confirm that the update is successful:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli describe-job -i job_ID
  3. Update the DCS components:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-dcscomponents -v 19.12.0.0.0
    If the DCS components are updated, then the message "status" : "Success" is displayed on the command line. For failed updates, fix the error and then proceed with the update.

    Note:

    You must log into the Browser User Interface again after running the odacli update-dcscomponents command.

Running Patching Prechecks and Patching Server Using the Browser User Interface

  1. Do not have any jobs running or pending during the update window.

  2. Log into the Browser User Interface within the DB system with the oda-admin user name and password.

    https://Node0–host-ip-address:7093/mgmt/index.html
  3. Click the Appliance tab, and then click Patch Manager in the Browser User Interface.

  4. Wait for the Component Details to load. If you just updated the Patch Repository, click Refresh in the Component Details section of the page.

    After the patch is uploaded to the Patch Repository, the Component Details on the page are updated with the Installed Version and Available Version for each component.

  5. Select Precheck to run patching prechecks.
  6. On the Patch Manager page, click Refresh to refresh the Component Details. Select Update Server, then click Apply Patch to begin the job to patch the server components.

    The DCS agent is automatically updated whenever the server components are updated.

    When the job finishes, go to the next step. Click Activity for job status.

  7. On the Patch Manager page, click Refresh to refresh the Component Details. Select Precheck if you want to run patching prechecks.

  8. Click Activity to monitor the progress, and verify that the job completes successfully.

Updating Oracle Database Appliance Repository with Database Clone Files For Bare Metal Systems

Follow these steps to update the Oracle Database Appliance repository with Oracle Database clone files for the latest release using CLI commands.

To create new databases of a specific release version, you must update the Oracle Database Appliance repository with the Oracle Database clone files for a release. Follow these steps to update the repository with Oracle Database clone files for the latest release.
  1. Download the Oracle Database clone software file for a release, for example, p30403662_1912000_Linux-x86-64.zip, and extract the Oracle Database clone software:
    For example, download the Oracle Database clone software patch for 19.12:
    unzip /tmp/p30403662_1912000_Linux-x86-64.zip
    odacli-dcs-19.12.0.0.0-date-DB-19.12.0.0.zip
  2. Copy all the software files from the external client to Oracle Database Appliance. For High-Availability deployments, copy the software files to only one node. The software files are copied to the other node during the patching process. Use the scp or sftp protocol to copy the bundle.
    Example using scp command:
    # scp software_file root@oda_host:/tmp
    Example using sftp command:
    # sftp root@oda_host
    Enter the root password, and copy the files.
    put software_file
  3. Update the repository with latest release database software:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-repository -f /tmp/odacli-dcs-db-clone-file.zip/

    For example, for 19.12:

    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-repository -f /tmp/odacli-dcs-19.12.0.0.0-date-DB-19.12.0.0.zip
    Follow the same steps to update the repository with the database clone software for other supported database versions such as 12.2, 12.1, and so on.
  4. Confirm that the repository update is successful:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli list-jobs
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli describe-job -i job_ID
  5. Delete the software zip files from the temporary location on the appliance to save space:
    [root@oda1 opt]# rm software_file
    rm: remove regular file software_file? y

Patching Existing Database Homes on Bare Metal Systems Using the Browser User Interface

Use the Browser User Interface to patch database homes to the latest release on bare metal systems.

Before patching the database home, upload the Oracle Database clone files for the database version, to the repository. See Updating Oracle Database Appliance Repository with Database Clone Files Using the CLI for the procedure to update the repository with the latest Oracle Database clone files.
  1. Log into the Browser User Interface with the oda-admin user name and password.
    https://Node0–host-ip-address:7093/mgmt/index.html
  2. Navigate to the Database Home tab, and then select the patch version for the database home.
  3. Select the database home you want to patch, and click Patch. Select Precheck to run pre-checks before patching the database.
    Click Activity for job status.
  4. On the Patch page, for the database to be patched, click Actions and select View Pre-patch reports to view the pre-check report. Fix any errors, and then select Action as Apply to patch the database.
  5. Click Ignore Precheck Failures to ignore failures reported in the prechecks reported. It is recommended that you fix errors reported in the precheck results.
  6. Click Ignore Missing Patches to ignore missing patches.
  7. Verify that the patching job completes successfully.

Patching Non-Provisioned Oracle Database Appliance Systems

Oracle Database Appliance supports patching of non-provisioned single-node or high availablity systems by running odacli update-server command.

A non-provisioned Oracle Database Appliance system does not have Oracle Grid Infrastructure or Oracle Database installed or running on it. You can upgrade Oracle ILOM and storage firmware on an unprovisioned system without running the odacli create-appliance command by reimaging the system using the bare metal ISO image of Oracle Database release 19.10 or later. Then patch the system with the server patches of the same release as the bare metal ISO image by running the odacli update-server command. When you reimage your system, the DCS infrastructure is updated automatically.
Follow these steps to update the firmware:
  1. Reimage the Oracle Database Appliance using the bare metal ISO image for Oracle Database Appliance release 19.10 or later.
  2. Update the repository with the server patch for the Oracle Database Appliance release you want to patch to, for example 19.10 or later.
  3. For running odacli update-server on Oracle Database Appliance imaged with ISO of release 19.10:
    odacli update-server -v 19.10.0.0.0 -sko -f

    For Oracle Database Appliance release 19.10, you can use the -f option to override precheck failures and complete the server patching.

  4. For running odacli update-server on Oracle Database Appliance imaged with ISO of release 19.11 or later, follow these steps:
    1. Create the pre-patch report for the odacli update-server command by specifying the -s option.
      odacli create-prepatchreport -v 19.11.0.0.0 -s

      Fix the warnings and errors mentioned in the report and proceed with the server patching.

    2. Update the server:
      odacli update-server -v 19.11.0.0.0 -f

      For Oracle Database Appliance release 19.11, you can use the -sko option to skip ORAchk failures and proceed with the server patching, and the -f option to override precheck failures and complete the server patching.

Applying Additional Patches and Updates on Bare Metal Systems

Configure additional patch repositories and patch your operating system and databases to ensure that your deployment has the latest updates and security fixes.

Enabling Kernel Updates

Oracle Ksplice enables you to update your systems with the latest kernel security and bug fix updates.

You can download and install the latest Oracle Ksplice updates (uptrack-updates) with either the local ULN repository or ULN channel.
Follow these steps to install Oracle KSplice:
  1. Register your server with Unbreakable Linux Network (ULN). By default, you are registered for the Oracle Linux Latest channel for your operating system and hardware.
  2. Log in to Unbreakable Linux Network:
  3. Ensure that you have access to the Internet on the server where you want to use Ksplice. For example, if you are using a proxy server, then set the proxy server and port values in the shell with commands similar to the following:
    # export http_proxy=http://proxy.company.com:port
    # export https_proxy=http://proxy.company.com:port
  4. Register the server:
    # uln_register
  5. Set up the local repository as described in the Oracle Ksplice user documentation. This enables you to download the latest Ksplice update packages at regular intervals without requiring a direct connection to the Oracle Uptrack server or to ULN.
  6. To update a system to a specific effective kernel version, install the uptrack-updates package for the current kernel.
    yum -y install uptrack-updates-'uname -r'
    You can also download the uptrack-updates RPM from the Ksplice channel and install the RPM manually. For example:
    # rpm -Uvh uptrack-updates-4.14.35-1902.11.3.1.el7uek.x86_64-20200325-0.noarch.rpm

See Also:

Oracle Linux Ksplice User's Guide for information about configuring Oracle Ksplice at:

https://docs.oracle.com/cd/E37670_01/E39380/html/ksplice-enhanced-offline.html

Applying Out-of-Cycle Database Patches

The out-of-cycle patching feature enables administrators to patch databases in the Oracle Database Appliance deployment with the latest Oracle Database Release Update, outside of the Oracle Database Appliance releases.

You can apply Oracle Database Release Update (RU) patches manually using OPatch. Refer to the relevant section in this topic to run the OPatch for your database version.

Note:

The OPatch commands and procedures described in this topic are examples, and may need modifications based on a specific Oracle Database Release Update (RU). Always refer to the Readme of the Oracle Database Release Update (RU) for additional commands and steps.

Note:

It is strongly recommended that you patch your deployment through the Oracle Database Appliance releases, whenever possible.

Note:

You can apply out-of-cycle patches to both baremetal and Virtualized Platform deployments using this procedure.

Using OPatchauto to Manually Patch Oracle Database Release 12c and Later

  1. Confirm that the dcs-agent and other components are updated to the latest release.
    # odacli describe-component
    # odacli list-dbhomes
  2. Update OPatch to the latest version, if available:
    1. Download the latest OPatch version from My Oracle Support.
    2. Copy the files to the local host and extract them to the ORACLE_HOME/OPatch directory.
    3. Update the OPatch tool as oracle user.
      /bin/rm -rf $ORACLE_HOME/OPatch/
      su - oracle -c /usr/bin/unzip -q -o OPatch_zip_location/p6880880_122010_Linux-x86-64.zip -d $ORACLE_HOME
  3. Download the latest database patches from My Oracle Support and copy the files to the local host.
    scp patchfilename root@hostname:directory
  4. Unzip patches to an empty directory, for example, /tmp/patchesDir as oracle user.

    This creates sub directories under /tmp/patchesDir with bug numbers. If the patch you are applying is a Release Update (RU), then the patch directory has multiple sub-directories.

    /usr/bin/unzip -o location_of_patches/p29301631_180000_Linux-x86-64.zip -d /tmp/patchesDir
  5. For Oracle Database Appliance High-Availability deployments, set up SSH Keys for secure connection.
  6. Analyze the patches.
    $ORACLE_HOME/OPatch/opatchauto apply directory_with_patches_extracted -analyze -oh $ORALCE_HOME -log /tmp/opatchAutoAnalyzePatch.log
  7. As root user, apply the patches.
    $ORACLE_HOME/OPatch/opatchauto apply directory_to_extract_patches -oh $ORACLE_HOME -inplace
  8. When patching is completed on both nodes, run utlrp.sql and catcon.pl scripts to recompile invalid objects in the database.
  9. Update the registry to ensure that system component values are registered.
    # odacli update-registry -n dbhome -f

Cleaning Up the Patch Repository

Use the Browser User Interface or Command Line Interface to delete obsolete or old patches from the repository.

About Cleaning Up the Patch Repository

After patching Oracle Database Appliance with the latest patches, old unused clone files may remain and consume space in the repository. Cleanup repository deletes the unused clone/patch files to reclaim space. You can clean up the repository using the Browser User Interface or ODACLI commands.

Cleaning Up the Patch Repository Using ODACLI command

# odacli cleanup-patchrepo [-cl clones] | [-v version [-comp component]]| [-l local|-n NodeID]

Specify the clone file names and the node from which you want to delete the old patches.

For details about the command options, see the chapter Oracle Database Appliance Command-Line Interface.

Cleaning Up the Patch Repository Using the Browser User Interface

Follow these steps to clean up the patch repository:
  1. In the Browser User Interface, click Patch Manager.
  2. Click the Cleanup Repository tab, and then click Refresh to list the available patches in the repository.
  3. Select the Components, such as old Oracle Grid Infrastructure or Oracle Database clone files that you want to clean up from the repository, and the Patch Version, for example, an older release of Oracle Database Appliance.
  4. Click Cleanup Clone Files to clean up clone files for selected components from the repository at /opt/oracle/oak/pkgrepos/orapkgs/clones/.
  5. Click Cleanup Repository to submit the cleanup repository job.
  6. Click Activity to monitor the progress. When the job completes successfully, the Patch Repository is updated.

Related Topics

About Upgrading to a Different Database Home

Understand how to upgrade to a different database home using either the Browser User Interface or odacli commands.

Prerequisites for Upgrading Database Homes

You can upgrade database homes of the following releases:
  • Oracle Database 12.2 to 19c
  • Oracle Database 12.1 to 19c
  • Oracle Database 12.1 to 12.2

Before upgrading to a different database home, upload the Oracle Database clone files for the database version, to the repository, and then create the database home. See Patching Oracle Database Appliance Using the CLI for the procedure to update the repository with the latest Oracle Database clone files.

About Upgrading Database Homes Using the Browser User Interface

Follow these steps to upgrade to a different database home using the Browser User Interface.

  1. Log into the Browser User Interface with the oda-admin user name and password.
    https://Node0–host-ip-address:7093/mgmt/index.html
  2. Click the Database tab, and then select a database, click Actions, and then click Upgrade.
  3. Verify that the database upgrade job completes successfully.

About Upgrading Database Homes Using the Command-Line Interface

Run the following command to upgrade to a different database home using the CLI:

odacli upgrade-database -i Comma separated list of database ids -from source dbhome id -to destination dbhome id [-j] [-h]

For more information about the options for the upgrade-database command, see the chapter Oracle Database Appliance Command-Line Interface.

Related Topics