7 Patching Oracle Database Appliance

To keep Oracle Database Appliance running with the latest software, check for and apply Oracle Database Appliance patch bundles when they are released.

About Patching Oracle Database Appliance

Use the Oracle Database Appliance Patch Bundle to patch your appliance.

Patches offer new features and may improve the functionality of existing features.

About Oracle Database Appliance Patch Bundles

The Oracle Database Appliance Patch Bundle contains the latest patches for DCS Admin, DCS Components, OAK, Oracle Grid Infrastructure, Oracle Database homes, enhancements, and fixes.

The Oracle Database Appliance Release Notes include information about the latest Oracle Database Appliance patch bundle and a list of component versions in the patch bundle. Oracle recommends updating the appliance with the latest Oracle Database Appliance software version to take advantage of new features, fixes, and the latest supported component versions. See the Oracle Database Appliance Release Notes for the upgrade paths.

Caution:

Do not patch Oracle Database Appliance using individual patches for Oracle Grid Infrastructure or Oracle Linux. Also do not use individual infrastructure patches, such as firmware patches. You must only use Oracle Database Appliance patches. If you use patches that are not intended for Oracle Database Appliance, or if you use OPatch, or a similar patching tool, then Oracle Database Appliance inventory is not updated, and you cannot complete future patch updates. If you apply out-of-cycle Oracle Database Release Update (RU), then ensure that you follow the recommendations described in the Readme for the RU.

The patch bundle provides all relevant patches for the entire system, including the following:

  • BIOS

  • Hardware drivers

  • Hardware Management Pack (HWM) and firmware drivers for various components

  • Oracle Appliance Manager

  • Oracle Linux

  • Oracle VM Server

  • Java Runtime Environment (JRE)

  • Oracle Integrated Lights Out Manager (Oracle ILOM)

  • Oracle Database Bundle Patch (BP), Oracle Database Patch Set Update (PSU), and Oracle Database Release Update (RU)

  • Oracle Auto Service Request (Oracle ASR)

  • Oracle Grid Infrastructure

  • Oracle Intelligent Platform Management Interface (Oracle IPMI)

  • Network Card Patches for relevant hardware models

About Updating Oracle Linux RPMs

While not recommended, you can update some Oracle Linux RPMs for database nodes. Do not update or customize Oracle Linux kernel, Oracle Optimal Flexible Architecture, Oracle InfiniBand, or related software. Other software may be installed, updated, or customized. However, the Oracle Database Appliance update may not carry newer version dependencies of customized components. Therefore, you might be required to remove and subsequently reapply site-specific changes to successfully update Oracle Database Appliance in the future.

Caution:

For database nodes, do not update the following:

  • Oracle Linux Kernel (kernel*)

  • Oracle Optimal Flexible Architecture (ofa*)

  • Oracle RDMA packages (oracle-rdma-release)

For storage, do not apply any RPM updates.

About Patching Pre-Checks

Patching pre-checks are designed to detect and flag problems that might be encountered during patching.

The pre-checks view the state of the appliance, including the version and hardware specific upgrades, verify that the necessary conditions and checks are complete before attempting an upgrade, and identify conditions that might cause an upgrade to fail. You can avoid potential patching problems by correcting the conditions and resolving the issues before applying the patch.

The pre-checks commands are available on Oracle Database Appliance release 12.2.1.2.0 or later. Update the DCS agent to support the pre-checks commands.

Note:

Patching pre-checks are available only when patching ODACLI stack. If you are patching and migrating from OAKCLI to ODACLI stack for bare metal system, then follow the procedure in the topic Patching Oracle Database Appliance Bare Metal Systems Using the Command-Line. For patching Virtualized Platform, see the topic Patching Oracle Database Appliance Virtualized Platform.

Components Verified by Patching Pre-Checks

Check Component Verified
System patching pre-checks

Checks that the agent is online on all nodes

Checks that there is sufficient space for the update

Validates the minimum agent version and validates the patching tag

Confirms that a valid patch is in the repository and calculates the patch bundle md5 checksum

Server patching pre-checks

Confirms that Oracle Grid Infrastructure patch is installed

Confirms that Oracle Clusterware is running on all nodes

Performs a check for required components using opatch prereq checkcomponents

Analyzes the patch using opatchauto

Performs a patch conflict check

Checks the available disk space

Confirms that the yum repository directory exists

Performs a yum check and yum check-update

Performs a test (dry run) yum update to check for dependencies in the RPMs

DBHome patching pre-checks

Confirms that the Oracle Grid Infrastructure is installed

Confirms that Oracle Clusterware is running on all nodes

Confirms that the Oracle Grid Infrastructure is upgraded before upgrading other components

Validates that there is sufficient space available in local mount

Performs a check for required components using opatch prereq checkcomponents

Analyzes the patch using opatchauto

Performs a patch conflict check and determines if a rollback of database patches is required

Storage pre-checks

Validates storage patch location and tag

Validates Oracle ASM disk groups status

Running Patching Pre-Checks Before Applying Patches

After updating the repository with patches, use the pre-checks to reduce potential update problems, and then apply patches.

Note:

Follow the procedure in this topic only when patching ODACLI stack. If you are patching and migrating from OAKCLI to ODACLI stack for bare metal system, then follow the procedure in the topic Patching Oracle Database Appliance Bare Metal Systems Using the Command-Line. For patching Virtualized Platform, see the topic Patching Oracle Database Appliance Virtualized Platform.
Before running the patching pre-checks, ensure that the dcs-agent is updated.
  1. Run the create-prepatchreport command to generate a pre-check report.
    odacli create-prepatchreport -s -v release_number
  2. Run the describe-prepatchreport command to display the pre-check report.
    odacli describe-prepatchreport -i job_ID

Patching Oracle Database Appliance Bare Metal Systems Using the Command-Line

Follow these steps to apply patches to your Oracle Database Appliance bare metal deployment and existing Oracle Database homes, using CLI commands.

To patch your Oracle Database Appliance deployment and your existing database homes to the current release, you must download the Oracle Database Appliance Server Patch and update the repository. To patch your existing databases to the latest release, you do not need the Oracle RDBMS clone files for the release. Download Oracle RDBMS clone files for a release only if you want to create new database homes for that release.

Note:

Run the commands in this topic in the same order as documented. Run the odacli update-dcsadmin and odacli update-dcscomponents commands before running the odacli update-server command.
Follow these steps to apply patches to your Oracle Database Appliance and update existing database homes.
  1. Download the Oracle Database Appliance Server Patch for the ODACLI/DCS stack (patch 31481816) from My Oracle Support to a temporary location on an external client. Refer to the release notes for details about the software for the latest release.
    For example, download the server patch for 19.8:
    p31481816_198000_Linux-x86-64_1of2.zip
    p31481816_198000_Linux-x86-64_2of2.zip
  2. Unzip the software — it contains README.html and one or more zip files for the patch.
    unzip p31481816_198000_Linux-x86-64_1of2.zip
    unzip p31481816_198000_Linux-x86-64_2of2.zip

    The zip file contains the following software files:

    oda-sm-19.8.0.0.0-200718-server1of2.zip
    oda-sm-19.8.0.0.0-200718-server2of2.zip
  3. Copy all the software files from the external client to Oracle Database Appliance. For High-Availability deployments, copy the software files to only one node. The software files are copied to the other node during the patching process. Use the scp or sftp protocol to copy the bundle.
    Example using scp command:
    # scp software_file root@oda_host:/tmp
    Example using sftp command:
    # sftp root@oda_host
    Enter the root password, and copy the files.
    put software_file
  4. Update the repository with the server software:
    # /opt/oracle/dcs/bin/odacli update-repository -f /tmp/software_file1,/tmp/software_file2

    For example, for 19.8:

    # /opt/oracle/dcs/bin/odacli update-repository -f /tmp/oda-sm-19.8.0.0.0-200718-server1of2.zip,/tmp/oda-sm-19.8.0.0.0-200718-server2of2.zip
  5. Confirm that the repository update is successful:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli describe-job -i job_ID
  6. Update the DCS agent:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-dcsagent -v 19.8.0.0.0
  7. Update DCS admin:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-dcsadmin -v 19.8.0.0.0
  8. Update the DCS components:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-dcscomponents -v 19.8.0.0.0

    This command updates the DCS components such as Zookeeper.

  9. Run patching pre-checks:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli create-prepatchreport -s -v version

    For example, for 19.8:

    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli create-prepatchreport -s -v 19.8.0.0.0
  10. Verify that the patching pre-checks ran successfully:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli list-prepatchreports

    For example:

    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli list-prepatchreports
    3797791f-72ac-42b6-8bb8-ed5e5c9b32c1 Patch pre-checks for [OS, ILOM, GI, ORACHKSERVER] June 30, 2020 6:29:30 AM UTC Running 

    Use the command odacli describe-prepatchreport to view details of the pre-patchreport.

  11. Apply the server update:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-server -v version

    For example, for 19.8:

    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-server -v 19.8.0.0.0
  12. Confirm that the server update is successful:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli describe-job -i job_ID
  13. Update the storage components.
    Specify the -rolling option to patch shared disks in a rolling fashion.
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-storage -v version --rolling

    For example, for 19.8:

    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-storage -v 19.8.0.0.0 --rolling
  14. To patch existing Oracle Database Homes to release 19.8:
    1. Run the patching pre-checks and review the prepatch report. For example:
      [root@oda1 opt]# /opt/oracle/dcs/bin/odacli create-prepatchreport --dbhome --dbhomeid DB_Home_ID -v release_number
      [root@oda1 opt]# odacli describe-prepatchreport -i Job_ID
      [root@oda1 opt]# /opt/oracle/dcs/bin/odacli create-prepatchreport --dbhome --dbhomeid 7c67c5b4-f585-4ba9-865f-c719c63c0a6e -v 19.8.0.0.0
      [root@oda1 opt]# odacli describe-prepatchreport -i 39ef1eeb-70d3-47ad-b3f5-48960ca0607b
    2. Update the database home to the latest release:
      [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-dbhome --dbhomeid DB_Home_ID -v release_number

      For example, for 19.8:

      [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-dbhome --dbhomeid 7c67c5b4-f585-4ba9-865f-c719c63c0a6e -v 19.8.0.0.0
  15. Confirm that the patch jobs are successful:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli list-jobs
  16. Delete the software zip files from the temporary location on the appliance to save space:
    [root@oda1 opt]# rm software_file
    rm: remove regular file software_file? y

Patching Oracle Database Appliance Bare Metal Systems Using the Browser User Interface

Upload the Oracle Database Appliance Server Patch to the patch repository, deploy the patch bundle using the Browser User Interface, and then update the DCS agent and components using CLI commands.

Updating the Patch Repository Using the Browser User Interface

  1. Download the Oracle Database Appliance patches from My Oracle Support and save them in a directory on the appliance. See the Oracle Database Appliance Release Notes for a list of available patches and links to download the patches.

  2. Log into the Browser User Interface with the oda-admin user name and password.

    https://Node0–host-ip-address:7093/mgmt/index.html
  3. Click Patch Manager.

  4. In the Update Patch Repository page, enter the absolute file path and patch name, then click Update Repository to start the update repository job. You can list more than one file to update to the repository with a comma separated list, without spaces, or you can update each file, one at a time.

  5. Click Activity to monitor the progress. When the job completes successfully, the Patch Repository is updated.

Running Patching Prechecks and Patching Agent, Server, and Storage Using the Browser User Interface

Note:

If your appliance has two nodes, you have the option to update both nodes at the same time or individually. If you update nodes individually, then update Node0 before updating Node1.
  1. Do not have any jobs running or pending during the update window.

  2. Click the Appliance tab, and then click Patch Manager in the Browser User Interface.

  3. Wait for the Component Details to load. If you just updated the Patch Repository, click Refresh in the Component Details section of the page.

    After the patch is uploaded to the Patch Repository, the Component Details on the page are updated with the Installed Version and Available Version for each component.

  4. Select Precheck, click Update Components, then click Submit to verify the patches in the repository and check for conflicts.

    When the job finishes, go to the next step. Click Activity for job status.

  5. On the Patch Manager page, click Refresh to refresh the Component Details. Select Update Server, click Update Components, then click Submit to begin the job to patch the server components.

    For multi-node systems, select the node to update: All Nodes (default), Node0, or Node1.

    The DCS agent is automatically updated whenever the Server or Storage components are updated.

    When the job finishes, go to the next step. Click Activity for job status.

  6. On the Patch Manager page, click Refresh to refresh the Component Details. Select Update Storage, click Update Components, then click Submit to begin the job to patch the storage components.

    For multi-node systems, select the node to update: All Nodes (default), Node0, or Node1.

    For high-availability environment, you can select the Rolling check box to perform rolling patching of storage components.

  7. Click Activity to monitor the progress, and verify that the job completes successfully.

All patching-related information is logged in the dcs-agent log file at /opt/oracle/dcs/log/dcs-agent.log.

Updating the DCS Admin and DCS Components After Patching

After patching your appliance using the Browser User Interface, you must update the DCS admin and DCS components manually, through CLI commands.

Follow these steps:

  1. Update DCS admin:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-dcsadmin -v 19.8.0.0.0
  2. Confirm that the update is successful:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli describe-job -i job_ID
  3. Update the DCS components:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-dcscomponents -v 19.8.0.0.0

Updating Oracle Database Appliance Repository with Database Clone Files For Bare Metal Systems

Follow these steps to update the Oracle Database Appliance repository with Oracle Database clone files for the latest release using CLI commands.

To create new databases of a specific release version, you must update the Oracle Database Appliance repository with the RDBMS Clone files for a release. Follow these steps to update the repository with Oracle Database clone files for the latest release.
  1. Download the RDBMS Clone software file for a release, for example, p30403662_198000_Linux-x86-64.zip, and extract the RDBMS Clone software:
    For example, download the RDBMS Clone software patch for 19.8:
    unzip /tmp/p30403662_198000_Linux-x86-64.zip
    odacli-dcs-19.8.0.0.0-200713-DB-19.8.0.0.zip
  2. Copy all the software files from the external client to Oracle Database Appliance. For High-Availability deployments, copy the software files to only one node. The software files are copied to the other node during the patching process. Use the scp or sftp protocol to copy the bundle.
    Example using scp command:
    # scp software_file root@oda_host:/tmp
    Example using sftp command:
    # sftp root@oda_host
    Enter the root password, and copy the files.
    put software_file
  3. Update the repository with latest release database software:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-repository -f /tmp/odacli-dcs-db-clone-file.zip/

    For example, for 19.8:

    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-repository -f /tmp/odacli-dcs-19.8.0.0.0-200713-DB-19.8.0.0.zip
    Follow the same steps to update the repository with the RDBMS Clone software for other supported database versions such as 12.2, 12.1, and so on.
  4. Confirm that the repository update is successful:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli list-jobs
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli describe-job -i job_ID
  5. Delete the software zip files from the temporary location on the appliance to save space:
    [root@oda1 opt]# rm software_file
    rm: remove regular file software_file? y

Patching Existing Database Homes on Bare Metal Systems Using the Browser User Interface

Use the Browser User Interface to patch database homes to the latest release on bare metal systems.

Before patching the database home, upload the RDBMS Clone Files for the database version, to the repository. See Updating Oracle Database Appliance Repository with Database Clone Files Using the CLI for the procedure to update the repository with the latest RDBMS Clone Files.
  1. Log into the Browser User Interface with the oda-admin user name and password.
    https://Node0–host-ip-address:7093/mgmt/index.html
  2. Click the Database tab, and then click Patch on the left pane.
  3. Click Patch, and then select the Action as Precheck to run pre-checks before patching the database.
    Click Activity for job status.
  4. On the Patch page, for the database to be patched, click View and select Pre-patch reports to view the pre-check report. Fix any errors, and then select Action as Apply to patch the database.
  5. Verify that the patching job completes successfully.

Upgrading the Operating System and Patching Oracle Database Appliance Virtualized Platform

When you patch your deployment to Oracle Database Appliance release 19.8, you also upgrade the operating system to Oracle Linux 7. Follow these steps to apply patches to Oracle Database Appliance OAKCLI Virtualized Platform using CLI commands.

Download the Oracle Database Appliance Server Patch for OAK Stack and Virtualized Platform and update the repository with the server zip files for Oracle Database Appliance release 19.8. Free up space in the patch repository, before you start patching your deployment. Ensure that at least 21 GB of space is available to download the server patch.
See the topic Backup and Recovery for Oracle Database Appliance Upgrades on Virtualized Platform in this guide for the procedure to take a backup and restore from backup for Oracle Database Appliance virtualized platform.
Follow these steps to upgrade the operating system to Oracle Linux 7 on Oracle Database Appliance Virtualized Platform deployment, using CLI commands. You must upgrade the operating system on the first node successfully. The upgraded node is rebooted automatically, and then you must upgrade the operating system on the other node.
  1. On both nodes, ensure that you free up space for the patch files on your appliance by deleting earlier Oracle Grid Infrastructure and Oracle Database clone files or patches that are not needed.
    Run the following commands and delete any old Oracle Grid Infrastructure clones and patches:
    # cd /opt/oracle/oak/pkgrepos/orapkgs/GI 
    # du -sh *
    Run the following commands and delete any old Oracle Database clones and patches:
    # cd /opt/oracle/oak/pkgrepos/orapkgs/DB 
    # du -sh *
  2. Download the Oracle Database Appliance Server Patch for OAK Stack and Virtualized Platforms (patch 31481822) from My Oracle Support to a temporary location on an external client. Refer to the Oracle Database Appliance Release Notes for 19.8, for details about the software for the latest release.
    For example, download the Oracle Database Appliance Server Patch for OAK Stack and Virtualized Platform for 19.8:
    OAKPatchBundle_19.8.0.0.0_200718_1of3.zip
    OAKPatchBundle_19.8.0.0.0_200718_2of3.zip
    OAKPatchBundle_19.8.0.0.0_200718_3of3.zip

    Note: Download OAKPatchBundle_19.8.0.0.0_200718_1of3.zip and OAKPatchBundle_19.8.0.0.0_200718_3of3.zip to upgrade the operating system and patch Oracle Database Appliance server on Virtualized Platform. Complete the operating system upgrade and server patching, and delete the patch files. To ensure that you have adequate space for patching your appliance, download OAKPatchBundle_19.8.0.0.0_200718_2of3.zip and unpack the software when you want to patch the databases in your deployment to the latest release.

  3. Copy the software files from the external client to Oracle Database Appliance. Ensure that you copy the software files to ODA_BASE on both nodes. Use the scp or sftp protocol to copy the bundle.
    Example using scp command:
    scp software_file root@oda_host:/tmp
    Example using sftp command:
    sftp root@oda_host
    Enter the root password, and copy the files.
    put software_file
  4. (Optional) Run this step if you want to save the input and output commands in a log file, so that they can be used for debugging, if required.
    1. Login to dom0.
    2. Run the command:
      script -a node-odabase-date.log
    3. Within the prompt, after running script, log into the ODA_BASE console:
      xm console oakDom1
    4. Run the next steps in this procedure. The typed commands and output are logged in node-odabase-date.log.
  5. Unpack the patch bundle. Run the command on both nodes.
    [root@hostname ~]# /opt/oracle/oak/bin/oakcli unpack -package path_to_patch_file

    For example, for 19.8:

    # /opt/oracle/oak/bin/oakcli unpack -package /tmp/OAKPatchBundle_19.8.0.0.0_200718_1of3.zip
    # /opt/oracle/oak/bin/oakcli unpack -package /tmp/OAKPatchBundle_19.8.0.0.0_200718_2of3.zip
    # /opt/oracle/oak/bin/oakcli unpack -package /tmp/OAKPatchBundle_19.8.0.0.0_200718_3of3.zip

    After the unpack operation completes successfully, remove the patch bundle to free up space.

  6. Run the oakcli update -patch command with the pre-patchreport option to generate the pre-patch report. Run the command on both nodes.
    [root@hostname ~]# /opt/oracle/oak/bin/oakcli update -patch 19.8.0.0.0 --pre-patchreport --local
    INFO: Running OS prepatch checks ...
    INFO: Validate supported versions
    SUCCESS: Validated minimum supported versions.
    INFO: Validate patching tag
    SUCCESS: Validated patching tag: 19.8.0.0.0
    INFO: Is patch location available
    SUCCESS: Patch location is available.
    INFO: Space checks for OS upgrade
    SUCCESS: Validated space checks.
    INFO: Install OS upgrade software
    INFO: This action might take a while if packages need to be uncompressed
    SUCCESS: Extracted OS upgrade patches into /root/oda-upgrade.
    Do not remove this directory untill OS upgrade completes.
    INFO: Verify OS upgrade by running preupgrade checks
    INFO: This operation will take long, please wait
    SUCCESS: Results stored in: '/root/preupgrade-results/preupg_results-200618165535.tar.gz' .
    Read complete report file '/root/preupgrade/result.html' before attempting OS upgrade.
    INFO: Validate custom rpms installed
    WARNING: Found additional RPMs installed
    Warning: user is responsible for updating custom RPM files
    SUCCESS: All OS prepatch tasks ran correctly
    INFO: Running Storage prepatch checks ...
    INFO: Validate patching tag
    SUCCESS: Validated patching tag: 19.8.0.0.0
    INFO: Patch location validation
    SUCCESS: Verified patch location.
    INFO: Patch tag validation
    SUCCESS: Verified patching tag.
    INFO: Verify ASM disks status
    SUCCESS: ASM disk are online.
    SUCCESS: All Storage prepatch tasks ran correctly
    SUCCESS: Pre patch report ended successfully
    See the topic Analyzing the Pre-Checks Report for Operating System Upgrades on Virtualized Platform in this guide, for detailed information about various checks in the report and the suggested actions for failed checks.
    Ensure that the pre-patch report checks complete successfully on both nodes. If there are any errors, then fix the errors before you proceed to step 6.
  7. On the local node, run the oakcli update -patch with the pre-osupgrade option to stop the Virtual Machines, shared repositories, Oracle Clusterware, and databases:
    [root@hostname ~]# /opt/oracle/oak/bin/oakcli update -patch 19.8.0.0.0 --pre-osupgrade --local
    INFO: Trying to ping dom0 agent
    INFO: dom0 agent is live
    INFO: dom0 agent is not updated
    INFO: Updating dom0 agent, this will take a while...
    SUCCESS: dom0 agent updated
    INFO: Stopping Vms and Repos on both nodes. This may take a while...
    SUCCESS: Repos and vms stopped
    INFO: Disabling start repo
    Start repo operation is now DISABLED on node 0
    Start repo operation is now DISABLED on node 1
    SUCCESS: Start repo disabled
    INFO: Stopping CRS in local node
    INFO: 2020-06-23 02:08:27: Clusterware is running on local node
    INFO: 2020-06-23 02:08:27: Attempting to stop clusterware and its resources locally
    SUCCESS: 2020-06-23 02:09:29: Successfully stopped the clusterware on local node
    SUCCESS: CRS stopped in local node
    SUCCESS: Loop devices check passed
    SUCCESS: Pre-OSupgrade finished successfully
    See the topic Analyzing the Preupgrade Checks Report for Operating System Upgrades on Virtualized Platform in this guide, for detailed information about various checks in the report and the suggested actions for failed checks.
    If you did not fix the failures described in the prepatch report, then running the above command may display an error. You can use the --force option to ignore errors and run the command. However, it is recommended that you fix all failures described in the prepatch report and then proceed with the operating system upgrade.
  8. Run the oakcli update -patch with the osupgrade option to upgrade the operating system to Oracle Linux 7:
    [root@hostname ~]# /opt/oracle/oak/bin/oakcli update -patch 19.8.0.0.0 --osupgrade --local
    
    INFO: Validating local resources are stopped
    2020-06-16--19:04:09 - Performing ODA pre-upgrade steps:
    Copy /boot/grub/grub.conf to /root/oda-upgrade/grub.conf
    2020-06-16--19:04:09 - Running pre-upgrade script /root/oda-upgrade/scripts/convert-cmdline-grubdefault.sh
    2020-06-16--19:04:09 - Running pre-upgrade script /root/oda-upgrade/scripts/convert-cmdline-envfile.sh
    2020-06-16--19:04:09 - Disabling repos in /etc/yum.repos.d/oracle-linux-ol6.repo
    Copy /boot/grub/grub.conf to /root/oda-upgrade/grub.conf
    2020-06-14--14:17:20 - Running pre-upgrade script /root/oda-upgrade/scripts/convert-cmdline-grubdefault.sh
    2020-06-14--14:17:20 - Running pre-upgrade script /root/oda-upgrade/scripts/convert-cmdline-envfile.sh
    2020-06-14--14:17:20 - Disabling repos in /etc/yum.repos.d/oracle-linux-ol6.repo
    2020-06-14--14:17:20 - Disabling repos in /etc/yum.repos.d/uek-ol6.repo
    Device "priv0" does not exist.
    2020-06-14--14:17:20 - Info: No need to migrate nic names for private interconnect.
    2020-06-14--14:17:20 - Stopping CRS
    CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'test1'
    CRS-2673: Attempting to stop 'ora.crf' on 'test1'
    CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'test1'
    CRS-2673: Attempting to stop 'ora.mdnsd' on 'test1'
    CRS-2673: Attempting to stop 'ora.gpnpd' on 'test1'
    CRS-2677: Stop of 'ora.crf' on 'test1' succeeded
    CRS-2673: Attempting to stop 'ora.gipcd' on 'test1'
    CRS-2677: Stop of 'ora.drivers.acfs' on 'test1' succeeded
    CRS-2677: Stop of 'ora.mdnsd' on 'test1' succeeded
    CRS-2677: Stop of 'ora.gpnpd' on 'test1' succeeded
    CRS-2677: Stop of 'ora.gipcd' on 'test1' succeeded
    CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'test1' has completed
    CRS-4133: Oracle High Availability Services has been stopped.
    2020-06-14--14:17:23 - Disable CRS autostart
    CRS-4621: Oracle High Availability Services autostart is disabled.
    2020-06-14--14:17:23 - Done performing ODA pre-upgrade steps.
    /tmp/osupgrade.expspawn /bin/sh /root/oda-upgrade/oda-upgrade.sh -u
    2020-06-14--14:17:23 - Running redhat-upgrade-tool:
    setting up repos...
    upgradeiso | 3.6 kB 00:00 ...
    upgradeiso/primary_db | 1.3 MB 00:00 ...
    Preupgrade assistant risk check found risks for this upgrade.
    You can run preupg --riskcheck --verbose to view these risks.
    Addressing high risk issues is required before the in-place upgrade
    and ignoring these risks may result in a broken upgrade and unsupported upgrade.
    Please backup your data.
    List of issues:
    preupg.risk.MEDIUM: We detected that some packages installed on the system were removed between Oracle Linux 6 and Oracle Linux 7. This might break the functionality of the packages that depend on the removed packages.
    ...
    ...
    HOOK-pkgdowngrades: INFO: DEP: enforcing package installation 'info.x86_64'
    HOOK-pkgdowngrades: INFO: DOWNGRADE: enforcing package installation 'grep.x86_64'
    HOOK-pkgdowngrades: INFO: DOWNGRADE: enforcing package installation 'kernel-uek-firmware.noarch'
    HOOK-pkgdowngrades: INFO: done
    Finished. Reboot to start upgrade.
    Finished. Reboot to start upgrade.
    2020-06-20--10:57:51 - Modifying /boot/grub/grub.conf to specify console to ttyS0
    2020-06-20--10:57:51 - Done running redhat-upgrade-tool.
    INFO: Ready to upgrade OS
    INFO: Reboot the system to upgrade OS
    Monitor the progress of the upgrade. After successful upgrade, the operating system restarts. 
    You can use the --force option to ignore errors and run the command. However, it is recommended that you fix all failures described in the preupgrade report and then proceed with the operating system upgrade.
    You can run the ODA_BASE upgrade from the console and monitor the progress of the upgrade. Start the console from dom0 with xm start oakDom1. Note that the xm start oakDom1 session is closed when each ODA_BASE node is rebooted after upgrade. You can restart the xm start oakDom1 session and monitor the progress of the upgrade.
  9. Review the progress of the upgrade operation. The output on the screen displays the status of the upgrade.
    When the operating system upgrade is completed successfully, the node is rebooted automatically.
  10. Run the oakcli update -patch with the post-osupgrade option to verify that the operating system upgrade succeeded:
    [root@hostname ~]# /opt/oracle/oak/bin/oakcli update -patch 19.8.0.0.0 --post-osupgrade --local
    INFO: OS has been upgraded to 4.14.35-1902.301.1.el7uek successfully.
    INFO: ODA software has been upgraded to 19.8.0.0.0 successfully.
    INFO: GI software has been upgraded to 18.8.0.0.191015(30112122,30581079) successfully.
    INFO: Starting CRS
    ...
    INFO: Patching post-osupgrade command finished successfully

    If you have installed custom RPMs on ODA_BASE, then review the RPMs listed in the directories /root/oda-upgrade/rpms-added-from-Oracle and /root/oda-upgrade/rpms-added-from-ThirdParty, and re-install the custom RPMs after the operating system is upgraded to Oracle Linux 7.

    See the topic Analyzing the Post Upgrade Checks Report for Operating System Upgrades on Virtualized Platform in this guide, for detailed information about various checks in the report and the suggested actions for failed checks.
    You can use the --force option to ignore errors and run the command. However, it is recommended that you fix all failures described in the post upgrade report.
  11. After the post checks complete successfully, upgrade the operating system on the second node by running steps 6 to 9 on the second node.
  12. Apply the server update:
    [root@hostname ~]# /opt/oracle/oak/bin/oakcli update -patch version --server

    For example, for 19.8:

    [root@hostname ~]# /opt/oracle/oak/bin/oakcli update -patch 19.8.0.0.0 --server
    After Oracle Grid Infrastructure patching completes successfully, you can free up space by detaching 18c and 12c grid homes from the inventory and deleting the grid homes as follows:
    1. Run the following from Node 0:
      # /u01/app/19.0.0.0/grid/oui/bin/runInstaller -nowait -waitforcompletion 
      -ignoreSysPrereqs -updateNodeList ORACLE_HOME=/u01/app/19.0.0.0/grid 
      "CLUSTER_NODES={NODE0,NODE1}" CRS=true LOCAL_NODE=NODE0 
      # /u01/app/19.0.0.0/grid/oui/bin/runInstaller -detachHome -silent 
      ORACLE_HOME=/u01/app/18.0.0.0/grid 
    2. You can also detach 12c grid homes and delete the Oracle Grid Infrastructure software from /u01:
      # rm -rf /u01/app/18.0.0.0 
    3. Delete the Oracle Grid Infrastructure clone:
      # cd /opt/oracle/oak/pkgrepos/orapkgs/GI/19.8.0.0.200714/Base/ 
      # rm grid19.tar.gz  
  13. Update the storage components.
    [root@hostname ~]# /opt/oracle/oak/bin/oakcli update -patch version --storage

    For example, for 19.8:

    [root@hostname ~]# /opt/oracle/oak/bin/oakcli update -patch 19.8.0.0.0 --storage
  14. To patch existing Oracle Database Homes to release 19.8:
    1. Download OAKPatchBundle_19.8.0.0.0_200718_2of3.zip to /tmp location and unpack the software:
      [root@hostname ~]# /opt/oracle/oak/bin/oakcli unpack -package /tmp/OAKPatchBundle_19.8.0.0.0_200718_2of3.zip
    2. Update the database homes with latest release database software:
      [root@hostname ~]# /opt/oracle/oak/bin/oakcli update -patch version --database

      For example, for 19.8:

      [root@hostname ~]# /opt/oracle/oak/bin/oakcli update -patch 19.8.0.0.0 --database
    3. Use the interactive command line to specify the database homes that you want to update to the latest release.
  15. Verify that the updates are applied.
    [root@hostname ~]# /opt/oracle/oak/bin/oakcli show version -detail
  16. Delete the software zip files from the temporary location on the appliance to save space:
    [root@hostname ~]# rm software_file
    rm: remove regular file software_file? y
  17. Enable the discovery of repositories from ODA_BASE.
    [root@hostname ~]# oakcli enable startrepo -node 0 
    [root@hostname ~]# oakcli enable startrepo -node 1 
  18. Restart the system:
    [root@hostname ~]# oakci restart oak
  19. After the upgrade completes successfully, verify that all the programs work correctly on ODA_BASE, and backup the logs and temporary files.
    Clean up the following directories used for patching, so that you can save space:
    /root/preupgrade 
    /root/oda-upgrade/
    /root/preupgrade-results  

Updating Oracle Database Appliance Repository with Database Clone Files For Virtualized Platform

Follow these steps to update the Oracle Database Appliance repository with Oracle Database clone files for Virtualized Platform for the latest release using CLI commands.

To create new databases of a specific release version, you must update the Oracle Database Appliance repository with the RDBMS Clone files for a release. Follow these steps to update the repository with Oracle Database clone files for the latest release.
  1. Download the RDBMS Clone software for Virtualized Platform for a release, for example, patch 30403653 for release 19.8, and extract the Oracle Database Clone software:
    For example, download the Oracle Database Clone software patch for 19.8:
    p30403653_198000_Linux-x86-64.zip
  2. Copy the software file from the external client to Oracle Database Appliance. For High-Availability deployments, copy the software file to only one node. The software file is copied to the other node during the patching process. Use the scp or sftp protocol to copy the bundle.
    Example using scp command:
    # scp software_file root@oda_host:/tmp
    Example using sftp command:
    # sftp root@oda_host
    Enter the root password, and copy the files.
    put software_file
  3. Unpack the software:
    [root@hostname ~]# /opt/oracle/oak/bin/oakcli unpack -package /tmp/p30403653_198000_Linux-x86-64.zip
  4. Use the oakcli create database command to create a database of a specific release.

Applying Additional Patches and Updates on Bare Metal Systems

Configure additional patch repositories and patch your operating system and databases to ensure that your deployment has the latest updates and security fixes.

Enabling Kernel Updates

Oracle Ksplice enables you to update your systems with the latest kernel security and bug fix updates.

You can download and install the latest Oracle Ksplice updates (uptrack-updates) with either the local ULN repository or ULN channel.
Follow these steps to install Oracle KSplice:
  1. Register your server with Unbreakable Linux Network (ULN). By default, you are registered for the Oracle Linux Latest channel for your operating system and hardware.
  2. Log in to Unbreakable Linux Network:
  3. Ensure that you have access to the Internet on the server where you want to use Ksplice. For example, if you are using a proxy server, then set the proxy server and port values in the shell with commands similar to the following:
    # export http_proxy=http://proxy.company.com:port
    # export https_proxy=http://proxy.company.com:port
  4. Register the server:
    # uln_register
  5. Set up the local repository as described in the Oracle Ksplice user documentation. This enables you to download the latest Ksplice update packages at regular intervals without requiring a direct connection to the Oracle Uptrack server or to ULN.
  6. To update a system to a specific effective kernel version, install the uptrack-updates package for the current kernel.
    yum -y install uptrack-updates-'uname -r'
    You can also download the uptrack-updates RPM from the Ksplice channel and install the RPM manually. For example:
    # rpm -Uvh uptrack-updates-4.14.35-1902.11.3.1.el7uek.x86_64-20200325-0.noarch.rpm

See Also:

Oracle Linux Ksplice User's Guide for information about configuring Oracle Ksplice at:

https://docs.oracle.com/cd/E37670_01/E39380/html/ksplice-enhanced-offline.html

Applying Out-of-Cycle Database Patches

The out-of-cycle patching feature enables administrators to patch databases in the Oracle Database Appliance deployment with the latest Oracle Database Release Update, outside of the Oracle Database Appliance releases.

You can apply Oracle Database Release Update (RU) patches manually using OPatch. Refer to the relevant section in this topic to run the OPatch for your database version.

Note:

The OPatch commands and procedures described in this topic are examples, and may need modifications based on a specific Oracle Database Release Update (RU). Always refer to the Readme of the Oracle Database Release Update (RU) for additional commands and steps.

Note:

It is strongly recommended that you patch your deployment through the Oracle Database Appliance releases, whenever possible.

Note:

You can apply out-of-cycle patches to both baremetal and Virtualized Platform deployments using this procedure.

Using OPatchauto to Manually Patch Oracle Database Release 12c and Later

  1. Confirm that the dcs-agent and other components are updated to the latest release.
    # odacli describe-component
    # odacli list-dbhomes
  2. Update OPatch to the latest version, if available:
    1. Download the latest OPatch version from My Oracle Support.
    2. Copy the files to the local host and extract them to the ORACLE_HOME/OPatch directory.
    3. Update the OPatch tool as oracle user.
      /bin/rm -rf $ORACLE_HOME/OPatch/
      su - oracle -c /usr/bin/unzip -q -o OPatch_zip_location/p6880880_122010_Linux-x86-64.zip -d $ORACLE_HOME
  3. Download the latest database patches from My Oracle Support and copy the files to the local host.
    scp patchfilename root@hostname:directory
  4. Unzip patches to an empty directory, for example, /tmp/patchesDir as oracle user.

    This creates sub directories under /tmp/patchesDir with bug numbers. If the patch you are applying is a Release Update (RU), then the patch directory has multiple sub-directories.

    /usr/bin/unzip -o location_of_patches/p29301631_180000_Linux-x86-64.zip -d /tmp/patchesDir
  5. For Oracle Database Appliance High-Availability deployments, set up SSH Keys for secure connection.
  6. Analyze the patches.
    $ORACLE_HOME/OPatch/opatchauto apply directory_with_patches_extracted -analyze -oh $ORALCE_HOME -log /tmp/opatchAutoAnalyzePatch.log
  7. As root user, apply the patches.
    $ORACLE_HOME/OPatch/opatchauto apply directory_to_extract_patches -oh $ORACLE_HOME -inplace
  8. When patching is completed on both nodes, run utlrp.sql and catcon.pl scripts to recompile invalid objects in the database.
  9. Update the registry to ensure that system component values are registered.
    # odacli update-registry -n dbhome -f

Using OPatch to Manually Patch Oracle Database Release 11.2.0.4

  1. Confirm that the dcs-agent and other components are updated to the latest release.
    # odacli describe-component
    # odacli list-dbhomes
  2. Update OPatch to the latest version, if available:
    1. Download the latest OPatch version from My Oracle Support.
    2. Copy the files to the local host and extract them to the ORACLE_HOME/OPatch directory.
    3. Update the OPatch tool as oracle user.
      /bin/rm -rf $ORACLE_HOME/OPatch
      su - oracle -c /usr/bin/unzip -q -o OPatch_zip_location/p29141056_112040_Linux-x86-64.zip -d $ORACLE_HOME
  3. Download the latest database patches from My Oracle Support and copy the files to the local host. Unzip the database patches to an empty directory, for example, /tmp/patchesDir as oracle user.
    This creates sub directories under /tmp/patchesDir with bug numbers. If the patch you are applying is an Oracle Database Release Update (RU), then the patch directory has multiple sub-directories.
    su - oracle -c '/usr/bin/unzip -o p29141056_112040_Linux-x86-64.zip -d /tmp/patchesDir'
  4. Delete bug#2015 if it exists in the inventory.
    1. Check if bug#2015 exists in the inventory:
      su - oracle 
      export ORACLE_HOME=path_to_the_11.2.0.4_ORACLE_HOME 
      $ORACLE_HOME/OPatch/opatch lspatches | grep -i "OCW" | cut -d ';' -f1
    2. The command returns a bug number, for example, 28729234. Navigate to the inventory:
      cd $ORACLE_HOME/inventory/oneoffs/bug# from above command/etc/config
    3. Check if inventory.xml contains a string such as 'bug number="2015"'. If no match is found, then no action is required, and you can continue with step 6 in this procedure.
      grep 'bug number="2015"' inventory.xml 
      echo $?  ( the command returns 0, if match found )   
    4. Take a backup of inventory.xml.
      cp inventory.xml inventory.xml.$(date +%Y%m%d-%H%M)
    5. Delete entry like <bug number="2015" ...> from inventory.xml.
      sed '/bug number="2015"/d' inventory.xml
  5. Run prechecks using the opatch command for every patch.
    su - oracle -c '$ORACLE_HOME/OPatch/opatch prereq CheckComponents -ph /tmp/patchesDir/29301631 -oh $ORACLE_HOME'
    su - oracle -c '$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOH -ph /tmp/patchesDir/29301631 -oh $ORACLE_HOME'

    Run both commands for all the sub patches also. If there are any conflicting bugs, then rollback the one-off patches.

  6. Apply the patch using the opatch auto command.
    $ORACLE_HOME/OPatch/opatch auto /tmp/patchesDir/29301631 -oh $ORACLE_HOME -ocmrf /opt/oracle/dcs/rdbaas/config/default-ocm.rsp
  7. Verify that the database is up and running after applying the patch.
    /u01/app/18.0.0.0/grid/bin/crsctl status resource ora.on6i.db -n patched_db
  8. Run catbundle.sql on all databases in this home.
    su oracle
      export ORACLE_HOME=ORACLE_HOME
      export ORACLE_HOME=ORACLE_SID       
            $ORACLE_HOME/bin/sqlplus "/ as sysdba" @$ORACLE_HOME/rdbms/admin/catbundle.sql
                @$ORACLE_HOME/rdbms/admin/catbundle.sql

Cleaning Up the Patch Repository

Use the Browser User Interface or Command Line Interface to delete obsolete or old patches from the repository.

About Cleaning Up the Patch Repository

After patching Oracle Database Appliance with the latest patches, old unused clone files may remain and consume space in the repository. Cleanup repository deletes the unused clone/patch files to reclaim space. You can clean up the repository using the Browser User Interface or ODACLI commands.

Cleaning Up the Patch Repository Using ODACLI command

# odacli cleanup-patchrepo [-cl clones] | [-v version [-comp component]]| [-l local|-n NodeID]

Specify the clone file names and the node from which you want to delete the old patches.

For details about the command options, see the chapter Oracle Database Appliance Command-Line Interface.

Cleaning Up the Patch Repository Using the Browser User Interface

Follow these steps to clean up the patch repository:
  1. In the Browser User Interface, click Patch Manager.
  2. Click the Cleanup Repository tab, and then click Refresh to list the available patches in the repository.
  3. Select the Components, such as old Oracle Grid Infrastructure or Oracle Database clone files that you want to clean up from the repository, and the Patch Version, for example, an older release of Oracle Database Appliance.
  4. Click Cleanup Clone Files to clean up clone files for selected components from the repository at /opt/oracle/oak/pkgrepos/orapkgs/clones/.
  5. Click Cleanup Repository to submit the cleanup repository job.
  6. Click Activity to monitor the progress. When the job completes successfully, the Patch Repository is updated.

Related Topics

About Upgrading to a Different Database Home

Understand how to upgrade to a different database home using either the Browser User Interface or odacli commands.

Prerequisites for Upgrading Database Homes

You can upgrade database homes of the following releases:
  • Oracle Database 18c to 19c
  • Oracle Database 12.2 to 19c
  • Oracle Database 12.1 to 19c
  • Oracle Database 11.2.0.4 to 19c
  • Oracle Database 12.2 to 18c
  • Oracle Database 12.1 to 18c
  • Oracle Database 11.2.0.4 to 18c
  • Oracle Database 12.1 to 12.2
  • Oracle Database 11.2.0.4 to 12.2
  • Oracle Database 11.2.0.4 to 12.1

Before upgrading to a different database home, upload the Oracle RDBMS Clone Files for the database version, to the repository, and then create the database home. See Patching Oracle Database Appliance Using the CLI for the procedure to update the repository with the latest Oracle RDBMS Clone Files.

About Upgrading Database Homes Using the Browser User Interface

Follow these steps to upgrade to a different database home using the Browser User Interface.

  1. Log into the Browser User Interface with the oda-admin user name and password.
    https://Node0–host-ip-address:7093/mgmt/index.html
  2. Click the Database tab, and then select a database, click Actions, and then click Upgrade.
  3. Verify that the database upgrade job completes successfully.

About Upgrading Database Homes Using the Command-Line Interface

Run the following command to upgrade to a different database home using the CLI:

odacli upgrade-database -i Comma separated list of database ids -from source dbhome id -to destination dbhome id [-j] [-h]

For more information about the options for the upgrade-database command, see the chapter Oracle Database Appliance Command-Line Interface.