Skip Headers
Oracle® Enterprise Manager Administrator's Guide for Software and Server Provisioning and Patching
11g Release 1 (11.1.0.1.0)

E16599-06
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Index
Index
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
PDF · Mobi · ePub

23 Patching Oracle RAC

This chapter explains how you can patch Oracle RAC. In particular, this chapter covers the following:

Getting Started

This section helps you get started with this chapter by providing an overview of the steps involved in patching Oracle RAC. Consider this section to be a documentation map to understand the sequence of actions you must perform to successfully patch Oracle RAC. Click the reference links provided against the steps to reach the relevant sections that provide more information.

Table 23-1 Getting Started with Patching Oracle RAC

Step Description

Step 1

Understanding the Deployment Procedure

Understand the Deployment Procedure that is offered by Enterprise Manager Grid Control for patching Oracle RAC. Know how the Deployment Procedure functions, what use cases it covers, what core components are patched, and so on.

To learn about the Deployment Procedure, see Deployment Procedures.

To learn about the core components that are patched, see Core Components Patched.

Step 2

Knowing About the Supported Releases

Know what releases of Oracle RAC can be patched by the Deployment Procedure.

To learn about the releases supported by the Deployment Procedure, see Supported Releases.

Step 3

Selecting the Patching Mode

Identify whether the host that runs Enterprise Manager Grid Control has an Internet connection to connect to My Oracle Support. If it has, then choose to patch in online mode. Otherwise, choose to patch in offline mode.

Step 4

Patching RAC Nodes Individually or Collectively

Decide whether you want to patch the nodes one by one, using the Rolling methodology, or all at a time, in parallel, using the All Nodes methodology.

Online Mode:

Offline Mode:

Step 5

Meeting the Prerequisites

Before you run any Deployment Procedure, you must meet the prerequisites, such as setting up of the environment for online or offline patching, applying mandatory patches, setting up of Oracle Software Library, and so on.

Online Mode:

Offline Mode:

Step 6

Running the Deployment Procedure

Run the Deployment Procedure to successfully patch Oracle RAC.

Online Mode:

Offline Mode:


Deployment Procedures

Enterprise Manager Grid Control offers the following Deployment Procedures for patching Oracle RAC:

While the former one allows you to patch the nodes one by one, the latter one allows you to patch all the nodes at a time, in parallel. For more information about these patching methodologies and circumstances under which you must use the Rolling methodology or All Nodes methodology, see Understanding Patching Methodologies.

Core Components Patched

When you patch Oracle RAC, essentially, the Deployment Procedures patch the following core components:

Supported Releases

Using these Deployment Procedures, you can patch the following releases of Oracle RAC:

IMPORTANT:

  • You cannot run this procedure on Oracle RAC installations that are on third-party clusterware (that is, other than Oracle Clusterware).

  • You can run this procedure on Oracle RAC installations that have shared Oracle homes, as long as they are not mixed-shared.

    For example, let us consider that the Oracle RAC installation is cluster1 and it has four nodes, mainly node1, node2, node3, and node4. You can run this procedure on this cluster if all the nodes are part of the same shared Oracle home, say /nfs_1/home/oracle/db10g. However, you cannot run this procedure if two of the nodes are from one shared Oracle home, say /nfs_1/home/oracle/db10g, and the other two are from another shared Oracle home, say /nfs_2/home/oracle/db10g.

  • This deployment procedure supports patching of shared Oracle home RAC installations only for Oracle Real Application Clusters (RAC) 10g Release 2 (10.2.0.1) or higher.

Patching Oracle RAC in Online Mode

This section describes how you can patch Oracle RAC in online mode, that is, by connecting to My Oracle Support. This option is best suited when you have an Internet connection on the host where Enterprise Manager Grid Control is running.

In online mode, you can either patch the nodes one by one, or patch all the nodes at a time, in parallel. This section provides instructions for both these methodologies.

In particular, this section provides instructions for the following:

Patching Oracle RAC Nodes Individually

This section describes how you can patch the nodes of Oracle RAC one by one, using the Rolling methodology.

This methodology is best suited when you are applying one-off patches that support this methodology, and when you want to maintain high availability of your targets, so when one node is being patched, the other nodes are available for service. For example, if you are patching a clusterware that has five nodes, then the first node is shut down, patched, and restarted, and then the process is rolled over to the next node, and so on, until all the nodes are patched successfully.

This section covers the following:

IMPORTANT:

  • You cannot use this methodology to patch shared Oracle homes. If you want to patch a shared Oracle home, then use Deployment Procedure that support the All Nodes methodology. For more information, see Patching Oracle RAC Nodes Collectively.

  • Although most one-off patches support this methodology, there are some one-off patches that can be applied only using the All Nodes methodology. You can identify the supported methodology while downloading the one-off patches from My Oracle Support.

  • If the Deployment Procedure abruptly stops or terminates at any step while it is being run, you may rerun the same procedure from where it stopped or from the beginning, all over again.

Prerequisites

Before running the Deployment Procedure, meet the following prerequisites:

Prerequisites for Designers

  • Ensure that you meet the prerequisites described in the following sections:

  • Ensure that operating system users such as oracle and crsuser are available on all the nodes of the cluster. These users must be a part of the relevant operating system groups such as dba or oinstall.

  • Ensure that you run the patchability report to understand whether the Oracle Database you want to patch is suitable for this patching operation. In the report, if the Oracle Database you want to patch appears as a target that cannot be patched, then fix the target configuration issues to make it a target that can be patched. For more information about the patchability report, and to understand how you can diagnose the issues with an Oracle Database target and take corrective actions, see Appendix H, "About Patching Automation Reports".

  • Ensure that you run the Deployment Procedure in the Analyze mode.

Prerequisites for Operators

  • Ensure that you use an operating system user that has the privileges to run the Deployment Procedure, and that can switch to root user and run all commands on the target hosts. For example, commands such as mkdir, ls, and so on.

    If you do not have the privileges to do so, that is, if you are using a locked account, then request your administrator (a designer) to either customize the Deployment Procedure to run it as another user or ignore the steps that require special privileges.

    For example, user account A might have the root privileges, but you might use user account B to run the Deployment Procedure. In this case, you can switch from user account B to A by customizing the Deployment Procedure.

    For information about customization, see Chapter 31, "Customizing Steps to Run as Another User".

  • Ensure that you are the owner of the Oracle home of the targets being patched. If you are not the owner or if you do not have direct access to those targets, then customize the Deployment Procedure to run as another user.

    For information about customization, see Chapter 31, "Customizing Steps to Run as Another User".

Patching Procedure

To patch Oracle Real Application Clusters, follow these steps:

  1. In Grid Control, click the Deployments tab.

  2. On the Deployments page, in the Patching section, click Patching through Deployment Procedures.

  3. On the Deployment Procedure Manager page, in the Procedures subtab, from the table, select Patch Oracle RAC Database - Rolling. Then click Schedule Deployment.

    Enterprise Manager Grid Control displays the Software Updates page of the Deployment Procedure.

  4. On the Software Updates page, do the following:

    1. In the Select Stage Location section, retain the default selection, that is, Enter Stage Location that has the value %emd_root%/EMStage.

      This default value refers to a location on the target host where the Oracle RAC to be patched is running. The software updates are downloaded from My Oracle Support to Oracle Software Library, and then staged in this location on the target hosts.

      Staging Location

      You can modify the location if you want to, and click Save. If you save multiple custom locations, then the Select Stage Location option gets enabled and allows you to select any of the saved custom locations the next time you visit the same page.

      Instead of %emd_root%, you can also specify %oracle_home% to indicate the current home location being patched.

    2. In the RAC Database Updates section, click Add to search and select software updates from My Oracle Support. Alternatively, if you have a text file that contains details about the patches, then click Upload From File to select and upload that text file.

      On clicking Add, Enterprise Manager Grid Control displays the Search and Select Software Updates page. On this page, select Search My Oracle Support, specify the details of the patches you want to search, and click Go.

      Enterprise Manager Grid Control connects to My Oracle Support, searches the patches, and displays details about them on the Search and Select Software Updates page. Select the ones you want to apply using the Deployment Procedure, and click Select.

      Note:

      Selection of multiple patches is supported.

      On clicking Upload From File, Enterprise Manager Grid Control displays the Upload File page. On this page, select Upload from Local Machine or Upload from Software Library depending on where the text file is available, click Browse to search and select that file, and then click Upload.

    3. (Optional) In the Apply SQL Script section, retain the default selection so that the default SQL script that is packaged with the patch is used for modifying the schemas.

      Apply SQL Script

      If you want to run any custom script, then select Enter the script to apply SQL and specify the full path to the location where the custom script is available. For example, %oracle_home%/rdbms/admin/catupgrd.sql .

      Enterprise Manager Grid Control automatically runs certain scripts such as Catcpu.sql, utlrp.sql for CPUs; Catpsu.sql, utlrp.sql for PSUs; and catupgrade.slq for Patch Sets. However, some scripts mentioned in the ReadMe of a patch might have to be run manually. To run those scripts, you can select Enter the script to apply SQL and specify the full path to the location where the custom scripts are available.

      The variables you can use for the directory location are %emd_root% to indicate a location on the target host, %oracle_home% to indicate the current home location being patched, or %perlbin% to indicate the location of perl binary used by the Management Agent.

      You have an option of running both the scripts by selecting both these options or one of the scripts by selecting only one of these options. If you select both these options, then both the scripts are run, that is, the default script runs first and then the custom script runs.

    4. (Optional) In the Upgrade OPatch section, retain the default selection so that the OPatch software on the target host is upgraded before the patches are applied on the database targets.

      Upgrade OPatch

      Enterprise Manager Grid Control uses OPatch software to apply the one-off patches. It is assumed that this software is already available on the target hosts managed by Oracle Management Agents (Management Agent), but Oracle recommends you to retain the selection so that the existing software is upgraded to the latest release.

    5. (Optional) In the Black Out Associated Targets section, retain the default selection so that all targets associated with the clusterware to be patched are blacked out while the patching is in progress.

      Black Out Associated Targets
    6. (Optional) In the Clean Up Backup Files section, select the option if you want to clean up all patching backup files after the patches are applied. The backup files are cleaned up by running OPatch utility cleanup command. Select this option depending on the space available on the target Oracle home.

    7. (Optional) In the Advanced OPatch Options section, specify any opatch-related options you want to pass while running this Deployment Procedure. Typically, you can use this section when you have patch conflicts or when inventory location is not default for the target Oracle home being patched. For example, after running the Deployment Procedure in Analyze Mode, if you identified any patch conflicts, then you can pass options to rectify the patch conflicts.

      Advanced OPatch Options

      The following are the options you can specify:

      -skip_duplicate, skips duplicate patches

      -skip_conflict, skips conflicting patches

      -skip_missing_component, skips patches whose components are missing

      -invPtrLoc, the location of oraInst.loc file

      Note:

      If you want to specify multiple options, then separate the list with a white space. For example, -skip_duplicate -skip_conflict
    8. Click Next.

  5. On the Target List page, in the RAC Database Targets to be Patched section, click Add to add the Oracle RAC database targets you want to patch. If you have multiple Oracle RAC databases in a cluster and if you are not sure of the selection, then search for all RAC databases in that cluster by selecting Cluster from the Target Type list. Then click Next.

    Select RAC Target

    Note:

    Selection of multiple targets to be patched in a single operation is supported. Typically, this procedure orchestrates patching in parallel across multiple hosts and does a sequential operation across the Oracle homes within a host. Within a given Oracle home, the procedure patches all of the Databases simultaneously.
  6. On the Library Step Properties page (appears only if you had customized the Deployment Procedure and marked some steps to be prompted at runtime), specify values for the library steps.

  7. (Optional) On the Oracle Configuration Manager page, specify an e-mail address to configure Oracle Configuration Manager. If you had already set up My Oracle Support credentials for the Enterprise Manager user account being used, then the fields are pre- filled with those details. If you have a proxy server, then provide details about the proxy server, and click Next.

    If you do not want to configure Oracle Configuration Manager, then delete the details appearing on this screen, and click Next.

  8. On the Credentials page, in the Home Credentials section, specify the Oracle home credentials required to patch the Oracle homes, and in the Host Credentials section, specify the operating system credentials to log in to the hosts where the Oracle homes are present. Then click Next.

    Home Credentials

    Note:

    When there are Oracle home active steps, the Home Credentials section appears. When there are host-level active steps, the Host Credentials section appears. When Oracle home active steps and host-level active steps are present, both the sections appear.

    For both sections, choose to use the preferred credentials so that the credentials stored in the Management Repository can be used.

    Of course, you can always override the preferred credentials with a new set of credentials. If you choose to do so, you can specify either a common set of credentials to be used across Oracle homes and hosts, or a unique set of credentials for each Oracle home and host.

    After specifying a new set of credentials to override the preferred credentials, click Save OH Credential if you want to store the new credentials in the Management Repository.

  9. On the Schedule page, schedule the Deployment Procedure to run either immediately or later.

    For Instance Name, specify a unique name for this Deployment Procedure instance so that it can be tracked later and reused with the same settings.

  10. On the Review page, review the details and do one of the following:

    • If you want to check for prerequisites, then click Analyze.

    • If you have already checked for prerequisites or if you do not want to check for prerequisites, then click Deploy to patch the selected databases.

Patching Oracle RAC Nodes Collectively

The prerequisites and the deployment instructions for patching all the nodes of Oracle RAC at a time, in parallel, using the All Nodes methodology, are the same as the ones described in Patching Oracle RAC Nodes Individually. However, in Step (3), on the Deployment Procedure Manager page, in the Procedures tab, from the table, select Patch Oracle RAC Database - All Nodes. Then click Schedule Deployment.

Although the instructions remain the same, when you submit the Deployment Procedure, Enterprise Manager Grid Control internally patches all the nodes of Oracle RAC collectively, that is, all the nodes are shut down and the patch is applied on all of them at the same time.

This methodology is best suited when you are applying a patch set or those one-off patches that support this methodology, or when you want to patch a shared Oracle RAC home. For example, if you are patching a shared Oracle home clusterware that has five nodes, the patch and the scripts within it are run only once on the shared Oracle home, and the other nodes are ignored or skipped. This saves time as the patching operation is performed only once, that is, only on the shared Oracle home, not on all the nodes.

Note:

  • Patching of shared Oracle homes is supported only for Oracle Database 10g Release 2 (10.2.0.1) or higher.

  • If the Deployment Procedure abruptly stops or terminates at any step while it is being run, you may rerun the same procedure from where it stopped or from the beginning, all over again.

Note:

To view an online demonstration of this feature, access the following URL:

http://www.oracle.com/technology/obe/demos/admin/demos.html

Patching Oracle RAC in Offline Mode

This section describes how you can patch Oracle RAC in offline mode, that is, without connecting to My Oracle Support. This option is best suited when you do not have an Internet connection on the host where Enterprise Manager Grid Control is running.

In offline mode, you can either patch the nodes one by one, or patch all the nodes at a time, in parallel. This section provides instructions for both these methodologies.

In particular, this section provides instructions for the following:

Patching Oracle RAC Nodes Individually

This section describes how you can patch the nodes of Oracle RAC one by one, using the Rolling methodology.

This methodology is best suited when you are applying one-off patches that support this methodology, and when you want to maintain high availability of your targets, so when one node is being patched, the other nodes are available for service. For example, if you are patching a clusterware that has five nodes, then the first node is shut down, patched, and restarted, and then the process is rolled over to the next node, and so on, until all the nodes are patched successfully.

Prerequisites

Before running the Deployment Procedure, meet the following prerequisites:

Prerequisites for Designers

  • Ensure that you meet the prerequisites described in the following sections:

  • Ensure that operating system users such as oracle and crsuser are available on all the nodes of the cluster. These users must be a part of the relevant operating system groups such as dba and oinstall.

  • Ensure that you run the patchability report to understand whether the Oracle Database you want to patch is suitable for this patching operation. In the report, if the Oracle Database you want to patch appears as a target that cannot be patched, then fix the target configuration issues to make it a target that can be patched. For more information about the patchability report, and to understand how you can diagnose the issues with an Oracle Database target and take corrective actions, see Appendix H, "About Patching Automation Reports".

  • Ensure that you run the Deployment Procedure in the Analyze mode.

Prerequisites for Operators

  • Ensure that you use an operating system user that has the privileges to run the Deployment Procedure, and that can switch to root user and run all commands on the target hosts. For example, commands such as mkdir, ls, and so on.

    If you do not have the privileges to do so, that is, if you are using a locked account, then request your administrator (a designer) to either customize the Deployment Procedure to run it as another user or ignore the steps that require special privileges.

    For example, user account A might have the root privileges, but you might use user account B to run the Deployment Procedure. In this case, you can switch from user account B to A by customizing the Deployment Procedure.

    For information about customization, see Chapter 31, "Customizing Steps to Run as Another User".

  • Ensure that you are the owner of the Oracle home of the targets being patched. If you are not the owner or if you do not have direct access to those targets, then customize the Deployment Procedure to run as another user.

    For information about customization, see Chapter 31, "Customizing Steps to Run as Another User".

Patching Procedure

To patch an Oracle RAC database, follow these steps:

  1. In Grid Control, click the Deployments tab.

  2. On the Deployments page, in the Patching section, click Patching through Deployment Procedures.

  3. On the Deployment Procedure Manager page, in the Procedures subtab, from the table, select Patch Oracle RAC Database - Rolling. Then click Schedule Deployment.

    Enterprise Manager Grid Control displays the Software Updates page of the Deployment Procedure.

  4. On the Software Updates page, do the following:

    1. In the Select Stage Location section, retain the default selection, that is, Enter Stage Location that has the value %emd_root%/EMStage.

      This default value refers to a location on the target host where the Oracle RAC to be patched is running. The software updates are downloaded from My Oracle Support to Oracle Software Library, and then staged in this location on the target hosts.

      Staging Location

      You can modify the location if you want to, and click Save. If you save multiple custom locations, then the Select Stage Location option gets enabled and allows you to select any of the saved custom locations the next time you visit the same page.

      Instead of %emd_root%, you can also specify %oracle_home% to indicate the current home location being patched.

    2. In the RAC Database Updates section, click Add to search and select software updates from My Oracle Support. Alternatively, if you have a text file that contains details about the patches, then click Upload From File to select and upload that text file.

      On clicking Add, Enterprise Manager Grid Control displays the Search and Select Software Updates page. On this page, select Search Software Library, specify the details of the patches you want to search, and click Go.

      Enterprise Manager Grid Control searches the patches in Oracle Software Library, and displays details about them on the Search and Select Software Updates page. Select the ones you want to apply using the Deployment Procedure, and click Select.

      Note:

      Selection of multiple patches is supported.

      On clicking Upload From File, Enterprise Manager Grid Control displays the Upload File page. On this page, select Upload from Local Machine or Upload from Software Library depending on where the text file is available, click Browse to search and select that file, and then click Upload.

    3. (Optional) In the Apply SQL Script section, retain the default selection so that the default SQL script that is packaged with the patch is used for modifying the schemas.

      Apply SQL Script

      If you want to run any custom script, then select Enter the script to apply SQL and specify the full path to the location where the custom script is available. For example, %oracle_home%/rdbms/admin/catupgrd.sql .

      Enterprise Manager Grid Control automatically runs certain scripts such as Catcpu.sql, utlrp.sql for CPUs; Catpsu.sql, utlrp.sql for PSUs; and catupgrade.slq for Patch Sets. However, some scripts mentioned in the ReadMe of a patch might have to be run manually. To run those scripts, you can select Enter the script to apply SQL and specify the full path to the location where the custom scripts are available.

      The variables you can use for the directory location are %emd_root% to indicate a location on the target host, %oracle_home% to indicate the current home location being patched, or %perlbin% to indicate the location of perl binary used by the Management Agent.

      You have an option of running both the scripts by selecting both these options or one of the scripts by selecting only one of these options. If you select both these options, then both the scripts are run, that is, the default script runs first and then the custom script runs.

    4. (Optional) In the Black Out Associated Targets section, retain the default selection so that all targets associated with the Oracle RAC database to be patched are blacked out while the patching is in progress.

      Black Out Associated Targets
    5. (Optional) In the Clean Up Backup Files section, select the option if you want to clean up all patching backup files after the patches are applied. The backup files are cleaned up by running OPatch utility cleanup command. Select this option depending on the space available on the target Oracle home.

    6. (Optional) In the Advanced OPatch Options section, specify any opatch-related options you want to pass while running this Deployment Procedure. Typically, you can use this section when you have patch conflicts or when inventory location is not default for the target Oracle home being patched. For example, after running the Deployment Procedure in Analyze Mode, if you identified any patch conflicts, then you can pass options to rectify the patch conflicts.

      Advanced OPatch Options

      The following are the options you can specify:

      -skip_duplicate, skips duplicate patches

      -skip_conflict, skips conflicting patches

      -skip_missing_component, skips patches whose components are missing

      -invPtrLoc, the location of oraInst.loc file

      Note:

      If you want to specify multiple options, then separate the list with a white space. For example, -skip_duplicate -skip_conflict
    7. Click Next.

  5. On the Target List page, in the RAC Database Targets to be Patched section, click Add to add the Oracle RAC database targets you want to patch. If you have multiple Oracle RAC databases in a cluster and if you are not sure of the selection, then search for all RAC databases in that cluster by selecting Cluster from the Target Type list. Then click Next.

    Select RAC Target

    Note:

    Selection of multiple targets to be patched in a single operation is supported. Typically, this procedure orchestrates patching in parallel across multiple hosts and does a sequential operation across the Oracle homes within a host. Within a given Oracle home, the procedure patches all of the Databases simultaneously.
  6. On the Library Step Properties page (appears only if you had customized the Deployment Procedure and marked some steps to be prompted at runtime), specify values for the library steps.

  7. (Optional) On the Oracle Configuration Manager page, specify an e-mail address to configure Oracle Configuration Manager. If you had already set up My Oracle Support credentials for the Enterprise Manager user account being used, then the fields are pre- filled with those details. If you have a proxy server, then provide details about the proxy server, and click Next.

    If you do not want to configure Oracle Configuration Manager, then delete the details appearing on this screen, and click Next.

  8. On the Credentials page, in the Home Credentials section, specify the Oracle home credentials required to patch the Oracle homes, and in the Host Credentials section, specify the operating system credentials to log in to the hosts where the Oracle homes are present. Then click Next.

    Home Credentials

    Note:

    When there are Oracle home active steps, the Home Credentials section appears. When there are host-level active steps, the Host Credentials section appears. When Oracle home active steps and host-level active steps are present, both the sections appear.

    For both sections, choose to use the preferred credentials so that the credentials stored in the Management Repository can be used.

    Of course, you can always override the preferred credentials with a new set of credentials. If you choose to do so, you can specify either a common set of credentials to be used across Oracle homes and hosts, or a unique set of credentials for each Oracle home and host.

    After specifying a new set of credentials to override the preferred credentials, click Save OH Credential if you want to store the new credentials in the Management Repository.

  9. On the Schedule page, schedule the Deployment Procedure to run either immediately or later.

    For Instance Name, specify a unique name for this Deployment Procedure instance so that it can be tracked later and reused with the same settings.

  10. On the Review page, review the details and do one of the following:

    • If you want to check for prerequisites, then click Analyze.

    • If you have already checked for prerequisites or if you do not want to check for prerequisites, then click Deploy to patch the selected databases.

Patching Oracle RAC Nodes Collectively

The prerequisites and the deployment instructions for patching all the nodes of Oracle RAC at a time, in parallel, using the All Nodes methodology, are the same as the ones described in Patching Oracle RAC Nodes Individually. However, in Step (3), on the Deployment Procedure Manager page, in the Procedures tab, from the table, select Patch Oracle RAC Database - All Nodes. Then click Schedule Deployment.

Although the instructions remain the same, when you submit the Deployment Procedure, Enterprise Manager Grid Control internally patches all the nodes of Oracle RAC collectively, that is, all the nodes are shut down and the patch is applied on all of them at the same time.

This methodology is best suited when you are applying a patch set or those one-off patches that support this methodology, or when you want to patch a shared Oracle RAC home. For example, if you are patching a shared Oracle home clusterware that has five nodes, the patch and the scripts within it are run only once on the shared Oracle home, and the other nodes are ignored or skipped. This saves time as the patching operation is performed only once, that is, only on the shared Oracle home, not on all the nodes.

Note:

  • Patching of shared Oracle homes is supported only for Oracle Database 10g Release 2 (10.2.0.1) or higher.

  • If the Deployment Procedure abruptly stops or terminates at any step while it is being run, you may rerun the same procedure from where it stopped or from the beginning, all over again.