B How to Perform Oracle Clusterware Rolling Upgrades

This appendix describes how to perform Oracle Clusterware rolling upgrades. Because you must stop database processes on the nodes you intend to upgrade when you perform an Oracle Clusterware upgrade, it includes information about how to stop processes in Oracle Real Application Clusters (Oracle RAC) databases.

The instructions in this section specify a single node, and assume that you are upgrading one node at a time. To upgrade a subset of nodes together, you can specify a list of nodes (the subset), where the example commands specify a single node. For example, instead of -n node, specify -n node1,node2,node3.

This appendix contains the following topics:

Note:

You can use the procedures in this chapter to perform rolling upgrades of Oracle Clusterware from any Oracle Clusterware 10g or Oracle Clusterware 11g installation to the latest patchset update. For example, you can use these procedures to prepare to upgrade from Oracle Clusterware 10.2.0.1 to 10.2.0.3.

B.1 Back Up the Oracle Software Before Upgrades

Before you make any changes to the Oracle software, whether you intend to upgrade or patch part of the database or clusterware, or all of your cluster installation, Oracle recommends that you create a backup of the Oracle software.

B.2 Restrictions for Clusterware Upgrades to Oracle Clusterware 11g

To upgrade existing Oracle Clusterware and Cluster Ready Services installations to Oracle Clusterware 11g, you must first upgrade the existing installations to a minimum patch level. The minimum patch level is listed in the following table:

Table B-1 Minimum Oracle Clusterware Patch Levels Required for Rolling Upgrades to 11g

Oracle Clusterware Release Minimum Patch Level Required

10g Release 2

10.2.0.3, or 10.2.0.2 with CRS bundle # 2 (Patch 526865)

10g Release 1

10.1.0.3


To upgrade your Oracle Clusterware installation to the minimum patch level using a rolling upgrade, follow the directions in the Patch Readme file.

Note:

You can use the procedures in this chapter to prepare to perform rolling upgrades of Oracle Clusterware from any Oracle Clusterware 10g release 10.2 or Oracle Clusterware 11g installation to the latest patchset update. For example, you can use these procedures to prepare to upgrade from Oracle Clusterware 10.2.0.1 to 10.2.0.3.

See Also:

Oracle Database Upgrade Guide for additional information about upgrades, and check the following site on the Oracle Technology Network for relevant information about rolling upgrades:

http://www.oracle.com/technology/deploy/availability

My Oracle Support also has an Upgrade Companion for each release that provide additional upgrade information. It is available at the following URL:

https://metalink.oracle.com/

B.3 Verify System Readiness for Patchset and Release Upgrades

If you are completing a patchset update of your database or clusterware, then after you download the patch software, and before you start to patch or upgrade your database, review the Patch Set Release Notes that accompany the patch to determine if your system meets the system requirements for the operating system and the hardware platform.

Use the following Cluster Verification Utility (CVU) command to assist you with system checks in preparation for starting a database patch or upgrade, where node is the node, or comma-delimited subset of nodes, that you want to check, and inventory_group is the name of the Oracle Inventory group:

cluvfy stage -pre crsinst -n node -orainv inventory_group

For example, to perform a system check on nodes node1, node2 and node3, and where the Oracle Inventory group is oinstall, enter the following command:

$ cluvfy stage -pre crsinst -n node1,node2,node3 -orainv oinstall

Note:

Before you start an upgrade, Oracle recommends that you download the latest Cluster Verification Utility version from Oracle Technology Network at the following URL:

http://otn.oracle.com

B.4 Installing a Patch Set On a Subset of Nodes

To patch Oracle Clusterware, review the instructions in the Patch Set README document for additional instructions specific to the patchset.

Before you shut down any processes that are monitored by Enterprise Manager Grid Control, set a blackout in Grid Control for the processes that you intend to shut down. This is necessary so that the availability records for these processes indicate that the shutdown was planned downtime, rather than an unplanned system outage.

To patch a subset of nodes, complete the following steps:

Note:

You must perform these steps in the order listed.
  1. Change directory to the Oracle Clusterware home. As root, run the preupdate.sh script on the local node, and on all other nodes in the subset that you intend to upgrade. Use the following command syntax, where clusterware_home is the path to the existing Oracle Clusterware home, and installation_owner is the Oracle Clusterware installation owner:

    ./preupdate.sh -crshome clusterware_home -crsuser installation_owner

    For example:

    # cd $ORACLE_HOME/install
    # ./preupdate.sh -crshome /opt/crs -crsuser oracle
    

    The script output should be similar to the following:

    Shutting down Oracle Cluster Ready Services (CRS):
    Stopping resources. This could take several minutes.
    Successfully stopped CRS resources.
    Stopping CSSD.
    Shutting down CSS daemon.
    Shutdown request successfully issued.
    Shutdown has begun. The daemons should exit soon.
    Checking to see if Oracle CRS stack is down...
    Oracle CRS stack is down now.
    
  2. Confirm that you are logged in as the Oracle Clusterware installation owner, and start Oracle Universal Installer to install the software.

    For example:

    $ whoami
    crs
    $ cd /cdrom/clusterware/
    ./runInstaller
    

    Provide information as prompted by the Installer.

    Note:

    You cannot change the owner of the Oracle Clusterware home.
  3. During an Oracle Clusterware installation, if Oracle Universal Installer detects an existing Oracle Clusterware 10g release 1 or release 2 installation, then you are given the option to perform a rolling upgrade by installing the patch on a subset of cluster member nodes.

    You can patch the entire cluster, and then run the root.sh patch script in a rolling fashion to make the patch update active.

  4. After you select the nodes you want to upgrade, the Installer installs the patch software in the existing Oracle Clusterware home on the local and the remote subset of nodes you have selected.

    OUI prompts you to run the appropriate root script for the patchset. The script starts the Oracle Clusterware stack on the upgraded subset of nodes. However, it lists it as an inactive version.

  5. After you upgrade the initial node or subset of nodes, repeat steps 1 through 4 for each remaining node or subset of nodes until all nodes in the cluster have been patched, and the new version is the active version.

    When all member nodes of the cluster are running the new Oracle Clusterware release, then the new clusterware becomes the active version. Otherwise, the older Oracle Clusterware release is still used.

    To list the version of Oracle Clusterware that is installed on a node, enter the following command, where CRShome is the Oracle Clusterware home, and nodename is the name of the node:

    # CRShome/bin/crsctl query crs softwareversion [nodename] 
    

    To list the Oracle Clusterware software version that is running on a node, enter the following command, where CRShome is the Oracle Clusterware home:

    # CRShome/bin/crsctl query crs activeversion 
    

    If you intend to install or upgrade Oracle RAC, then you must first complete the upgrade to Oracle Clusterware 11g release 1 (11.1) on all cluster member nodes before you install the Oracle Database 11g release 1 (11.1) version of Oracle RAC.

  6. Check with Oracle Support to confirm you have installed any recommended patch sets, bundle patches or critical patches.

    To check for the latest recommended patches for Oracle Clusterware and Oracle Real Application Clusters, log on to the following site:

    https://metalink2.oracle.com
    

    Click Patches & Updates, and click Oracle Database from the Recommended Patches list. Provide information as prompted.

B.5 Installing an Upgrade On a Subset of Nodes

To upgrade an Oracle Clusterware release, you must shut down all Oracle Database instances on the subset of nodes you want to upgrade before modifying the Oracle software. Review the instructions in the release upgrade README document for additional instructions specific to the upgrade.

Before you shut down any processes that are monitored by Enterprise Manager Grid Control, set a blackout in Grid Control for the processes that you intend to shut down. This is necessary so that the availability records for these processes indicate that the shutdown was planned downtime, rather than an unplanned system outage.

To shut down Oracle processes and upgrade a subset of nodes, complete the following steps:

Note:

You must perform these steps in the order listed.
  1. Shut down any processes that may be accessing a database on each node you intend to upgrade. For example, shut down Oracle Enterprise Manager Database Control.

  2. Shut down all Oracle RAC instances on the nodes you intend to upgrade. To shut down an Oracle RAC instance for a database, enter the following command db_name is the name of the database and inst_name is the database instance:

    $ Oracle_home/bin/srvctl stop instance -d db_name -i inst_name
    
  3. Shut down all ASM instances on all nodes you intend to upgrade. To shut down an ASM instance, enter the following command, where ASM_home is the ASM home location, and node is the name of the node where the ASM instance is running:

    $ ASM_home/bin/srvctl stop asm -n node
    

    Note:

    If you shut down ASM instances, then you must first shut down all database instances on the nodes you intend to upgrade that use ASM, even if these databases run from different Oracle homes.
  4. Stop all listeners on the node. To shut down listeners on the node, enter the following command, where nodename is the name of the node, and the listener is running from the ASM home:

    $ ASM_home/bin/srvctl stop listener -n node_name
    
  5. Stop all node applications on all nodes. To stop node applications running on a node, enter the following command, where node is the name of the node where the applications are running

    $ Oracle_home/bin/srvctl stop nodeapps -n node
    
  6. Change directory to the Oracle Clusterware home. As root, run the preupdate.sh script on the local node, and on all other nodes in the subset that you intend to upgrade. Use the following command syntax, where clusterware_home is the path to the existing Oracle Clusterware or Cluster Ready Services home, and installation_owner is the Oracle Clusterware installation owner:

    ./preupdate.sh -crshome clusterware_home -crsuser installation_owner

    For example:

    # cd $ORACLE_HOME/install
    # ./preupdate.sh -crshome /opt/crs -crsuser oracle
    

    The script output should be similar to the following:

    Shutting down Oracle Cluster Ready Services (CRS):
    Stopping resources. This could take several minutes.
    Successfully stopped CRS resources.
    Stopping CSSD.
    Shutting down CSS daemon.
    Shutdown request successfully issued.
    Shutdown has begun. The daemons should exit soon.
    Checking to see if Oracle CRS stack is down...
    Oracle CRS stack is down now.
    
  7. Confirm that you are logged in as the Oracle Clusterware installation owner, and start Oracle Universal Installer to install the software.

    For example:

    $ whoami
    crs
    $ cd /cdrom/clusterware/
    ./runInstaller
    

    Provide information as prompted by the Installer.

    Note:

    You cannot change the owner of the Oracle Clusterware home.
  8. During an Oracle Clusterware installation, if Oracle Universal Installer detects an existing Oracle Clusterware 10g release 1 or release 2 installation, then you are given the option to perform a rolling upgrade by installing Oracle Clusterware 11g release 1 on a subset of cluster member nodes.

    By default, all the cluster member nodes are checked for upgrade. To perform a rolling upgrade of a subset of nodes, uncheck the cluster member nodes you do not want to upgrade on the Specify Hardware Cluster Installation Mode installation screen.

  9. After you select the nodes you want to upgrade, the Installer installs the Oracle Clusterware 11g release 1 software in the existing Oracle Clusterware home on the local and the remote subset of nodes you have selected.

    OUI prompts you to run the appropriate root script for the release or patchset. The script starts the Oracle Clusterware 11g release 1 stack on the upgraded subset of nodes. However, it lists it as an inactive version.

  10. After you upgrade the initial node or subset of nodes, repeat steps 1 through 10 for each remaining node or subset of nodes until all nodes in the cluster have been upgraded and the new version is the active version.

    When all member nodes of the cluster are running the new Oracle Clusterware release, then the new clusterware becomes the active version. Otherwise, the older Oracle Clusterware release is still used.

    To list the version of Oracle Clusterware that is installed on a node, enter the following command, where CRShome is the Oracle Clusterware home, and nodename is the name of the node:

    # CRShome/bin/crsctl query crs softwareversion [nodename] 
    

    To list the Oracle Clusterware software version that is running on a node, enter the following command, where CRShome is the Oracle Clusterware home:

    # CRShome/bin/crsctl query crs activeversion 
    

    If you intend to install or upgrade Oracle RAC, then you must first complete the upgrade to Oracle Clusterware 11g release 1 (11.1) on all cluster member nodes before you install the Oracle Database 11g release 1 (11.1) version of Oracle RAC.

  11. Check with Oracle Support, and apply any recommended patch sets, bundle patches or critical patches.

    To check for the latest recommended patches for Oracle Clusterware and Oracle Real Application Clusters, log on to the following site:

    https://metalink2.oracle.com
    

    Click Patches & Updates, and click Oracle Database from the Recommended Patches list. Provide information as prompted.