D How to Upgrade to Oracle Grid Infrastructure 11g Release 2

This appendix describes how to perform Oracle Clusterware and Oracle Automatic Storage Management (Oracle ASM) upgrades.

Oracle Clusterware upgrades can be rolling upgrades, in which a subset of nodes are brought down and upgraded while other nodes remain active. Oracle ASM 11g Release 2 (11.2) upgrades can be rolling upgrades. If you upgrade a subset of nodes, then a software-only installation is performed on the existing cluster nodes that you do not select for upgrade.

This appendix contains the following topics:

D.1 About Oracle ASM and Oracle Grid Infrastructure Installation and Upgrade

In past releases, Oracle ASM was installed as part of the Oracle Database installation. With Oracle Database 11g Release 2 (11.2), Oracle ASM is installed when you install the Oracle Grid Infrastructure software. Oracle ASM shares an Oracle home with Oracle Clusterware when installed in a cluster such as with Oracle RAC or with Oracle Restart on a standalone server. The directory in which Oracle Clusterware and Oracle ASM is installed is called the Grid home.

If you have an existing Oracle ASM instance, you can either upgrade it at the time that you install Oracle Grid Infrastructure, or you can upgrade it after the installation, using Oracle ASM Configuration Assistant (ASMCA). However, a number of Oracle ASM features are disabled until you upgrade Oracle ASM, and Oracle Clusterware management of Oracle ASM does not function correctly until Oracle ASM is upgraded, because Oracle Clusterware only manages Oracle ASM when it is running in the Oracle Grid Infrastructure home. For this reason, Oracle recommends that if you do not upgrade Oracle ASM at the same time as you upgrade Oracle Clusterware, then you should upgrade Oracle ASM immediately afterward.

You can perform out-of-place upgrades to an Oracle ASM instance using ASMCA. In addition to running ASMCA using the graphic user interface (GUI), you can run ASMCA in non-interactive (silent) mode.

In earlier releases, you could use Database Upgrade Assistant (DBUA) to upgrade either an Oracle Database, or Oracle ASM. With Oracle Database 11g Release 2 (11.2), you can use DBUA to upgrade only an Oracle Database instance. To upgrade Oracle ASM separately, you use ASMCA.

See Also:

Oracle Database Upgrade Guide and Oracle Automatic Storage Management Administrator's Guide for additional information about upgrading existing Oracle ASM installations

D.2 Restrictions for Oracle Clusterware and Oracle ASM Upgrades

Be aware of the following restrictions and changes for upgrades to Oracle Grid Infrastructure installations, which consists of Oracle Clusterware and Oracle ASM.

D.2.1 Upgrade Your Existing Installation to the Minimum Required Version

  • To upgrade existing Oracle Clusterware installations to Oracle Grid Infrastructure 11g, your current release must be at least 10.1.0.3, 10.2.0.3, 11.1.0.6, or 11.2.

  • To upgrade existing Oracle ASM installations to Oracle Grid Infrastructure 11g Release 2 (11.2) using the rolling upgrade method, your current release must be at least 11.1.0.6.

    See Also:

    Oracle Upgrade Companion" Note 785351.1 on My Oracle Support:

    https://support.oracle.com

D.2.2 Apply Required Patches

To upgrade Oracle Grid Infrastructure release 11.2.0.1 to Oracle Grid Infrastructure release 11.2.0.2, you must first do at least one of the following:

  • Patch the Oracle Grid Infrastructure release 11.2.0.1 home with the patch for bugs 9413827 (9978481) and 9706490 (10047003).

  • Install the latest bundle for the cumulative patches for Oracle Grid Infrastructure (Patch 11 bundle or higher).

D.2.3 Other Requirements and Restrictions

  • Adding nodes to a cluster during a rolling upgrade is not supported.

  • Oracle Clusterware and Oracle ASM upgrades are always out-of-place upgrades. With Oracle Grid Infrastructure 11g Release 2 (11.2), you cannot perform an in-place upgrade of Oracle Clusterware and Oracle ASM to existing homes.

  • With Oracle Clusterware 11g Release 1 and later releases, the same user that owned the Oracle Clusterware 10g software must perform the Oracle Clusterware 11g upgrade. Before the Oracle Database 11g release, either all Oracle software installations were owned by a single user, typically oracle, or Oracle Database software was owned by oracle, and Oracle Clusterware software was owned by a separate user, typically crs.

D.2.4 Changes to the Upgrade Process

  • If the existing Oracle Clusterware home is a shared home, you can use a non-shared home for the Oracle Grid Infrastructure for a Cluster home for Oracle Clusterware and Oracle ASM 11g Release 2 (11.2).

  • Oracle ASM and Oracle Clusterware both run in the Oracle Grid Infrastructure home.

  • During a major version upgrade to Oracle Clusterware 11g Release 2 (11.2), the software in the Grid home for Oracle Grid Infrastructure 11g Release 2 (11.2) is not fully functional until the upgrade is completed. Running the Server Control Utility (SRVCTL), crsctl, and other commands from the 11g Release 2 (11.2) Grid home is not supported until the upgrade is complete across all nodes.

    To manage pre-existing databases using earlier versions (release 10.x or 11.1) of Oracle Database software during the Oracle Grid Infrastructure upgrade, use SRVCTL in the existing database homes.

D.3 Understanding Out-of-Place and Rolling Upgrades

Oracle Clusterware upgrades can be rolling upgrades, in which a subset of nodes are brought down and upgraded while other nodes remain active. Oracle ASM 11g Release 2 (11.2) upgrades can be rolling upgrades. If you upgrade a subset of nodes, then a software-only installation is performed on the existing cluster nodes that you do not select for upgrade. Rolling upgrades avoid downtime and ensure continuous availability while the software is upgraded to a new version.

Note:

In contrast with releases prior to Oracle Clusterware 11g Release 2, Oracle Universal Installer (OUI) always performs rolling upgrades, even if you select all nodes for the upgrade.

During an out-of-place upgrade, the installer installs the newer version in a separate Oracle Clusterware home. Both versions of Oracle Clusterware are on each cluster member node, but only one version is active. By contrast, an in-place upgrade overwrites the software in the current Oracle Clusterware home.

To perform an out-of-place upgrade, you must create new Oracle Grid Infrastructure homes on each node. Then you can perform an out-of-place rolling upgrade, so that some nodes are running Oracle Clusterware from the original Oracle Clusterware home, and other nodes are running Oracle Clusterware from the new Oracle Grid Infrastructure home.

If you have an existing Oracle Clusterware installation, then you upgrade your existing cluster by performing an out-of-place upgrade. An in-place upgrade of Oracle Clusterware 11g Release 2 is not supported.

See Also:

"Performing a Rolling Upgrade of Oracle Clusterware" for instructions on completing rolling upgrades

D.4 Preparing to Upgrade an Existing Oracle Clusterware Installation

If you have an existing Oracle Clusterware installation, then you upgrade your existing cluster by performing an out-of-place upgrade. You cannot perform an in-place upgrade. The following sections describe the tasks you should complete before starting an upgrade:

D.4.1 Verify System Readiness for Patches and Upgrades

If you are completing a patch update of Oracle Clusterware or Oracle ASM, then after you download the patch software and before you start to patch or upgrade your software installation, review the Patch Set Release Notes that accompany the patch to determine if your system meets the system requirements for the operating system and the hardware.

Use the Cluster Verification Utility (CVU) to assist you with system checks in preparation for patching or upgrading. You can run CVU before starting the upgrade, however, the installer runs the appropriate CVU checks automatically, and prompts you to fix problems before proceeding with the upgrade.

D.4.2 Gather the Necessary System Information

Ensure that you have the information you will need when installation Oracle Grid Infrastructure into a new home directory, including the following:

  • The Oracle home location for the current Oracle Clusterware installation.

    With Oracle Clusterware 11g Release 2 (11.2), you can perform upgrades on a shared Oracle Clusterware home.

  • An Oracle Grid Infrastructure home location that is different from your existing Oracle Clusterware home or Grid home location

  • A single client access name (SCAN) address

  • Local Administrator user access, or access as the user who performed the previous Oracle Clusterware installation

D.4.3 Upgrade to the Minimum Required Oracle Clusterware Version

If your current Oracle Clusterware installation is not currently running at least the minimum supported version for that release (10.1.0.3, 10.2.0.3, 11.1.0.6, or 11.2), then a prerequisite check failure is reported. You must upgrade your current Oracle Clusterware installation to at least the minimum supported version for that release before upgrading. For example, if you plan to upgrade your Oracle Clusterware 10g Release 1 installation to Oracle Clusterware 11g Release 2 (11.2) and your current Oracle Clusterware installation is version 10.1.0.2, then you must upgrade your current Oracle Clusterware installation to version 10.1.0.3 or higher before starting the upgrade

D.4.4 Unset Environment Variables Used By Oracle Software

If you have had an existing installation on your system, and you are using the same user account to install this installation, then unset the following environment variables, and any other environment variable set for the Oracle installation user that is connected with Oracle software homes.

  • ORACLE_HOME

  • ORACLE_SID

  • ORA_NLS10

  • TNS_ADMIN

  • ORA_CRS_HOME

If you have set ORA_CRS_HOME as an environment variable, following instructions from Oracle Support, then unset it before starting an installation or upgrade. You should never use ORA_CRS_HOME as an environment variable except under explicit direction from Oracle Support.

To remove the environment variable settings for all sessions, from the Start menu, right click My Computer and select Properties. In the System Properties window, select Advanced, then click the Environment Variables button.

Also, check to ensure that the user profile for the installation owner does not set any of these environment variables.

D.5 Using CVU to Verify System Readiness for Patches and Upgrades

If you are completing a patch update of Oracle Clusterware or Oracle ASM, then after you download the patch software and before you start to patch or upgrade your software installation, review the Patch Set Release Notes that accompany the patch to determine if your system meets the system requirements for the operating system and the hardware.

Navigate to the staging area for the upgrade, where the runcluvfy command is located, and run the command runcluvfy stage -pre crsinst -upgrade to check the readiness of your Oracle Clusterware installation for upgrades. Running runcluvfy with the -pre crsinst -upgrade flags performs system checks to confirm if the cluster is in a correct state for upgrading from an existing clusterware installation.

The command uses the following syntax, where variable content is indicated by italics:

runcluvfy stage -pre crsinst -upgrade [-n node_list] [-rolling] -src_crshome
src_Gridhome -dest_crshome dest_Gridhome -dest_version dest_version
[-fixup[-fixupdir fixupdirpath]] [-verbose]

The options used in this command are:

Option Description
-n nodelist The -n flag indicates cluster member nodes, and nodelist is the comma-delimited list of non-domain qualified node names on which you want to run a preupgrade verification. If you do not add the -n flag to the verification command, then all the nodes in the cluster are verified.
-rolling Use this flag to verify readiness for rolling upgrades.
-src_crshome src_Gridhome Use this flag to indicate the location of the source Oracle Clusterware or Grid home that you are upgrading, where src_Gridhome is the path to the home that you want to upgrade.
-dest_crshome dest_Gridhome Use this flag to indicate the location of the upgrade Grid home, where dest_Gridhome is the path to the Grid home.
-dest_version dest_version Use the dest_version flag to indicate the release number of the upgrade, including any Patch set. The release number must include the five digits designating the release to the level of the platform-specific patch. For example: 11.2.0.2.0.
-fixup [-fixupdir fixupdirpath] (Linux and UNIX) Use the -fixup flag to indicate that you want to generate instructions for any required steps you need to complete to ensure that your cluster is ready for an upgrade. The default location is the CVU work directory. If you want to place the fixup instructions in a different directory, then add the flag -fixupdir, and provide the path to the fixup directory
-verbose Use the -verbose flag to produce detailed output of individual checks

Use the Cluster Verification Utility (CVU) to assist you with system checks in preparation for patching or upgrading. You can run CVU before starting the upgrade, however, the installer runs the appropriate CVU checks automatically, and prompts you to fix problems before proceeding with the upgrade.

Example D-1 Using CVU to Verify System Upgrade Readiness for Grid Infrastructure

You can verify that the permissions required for installing Oracle Clusterware have been configured on the nodes node1 and node2 by running the following command:

C:\stage> runcluvfy.bat stage -pre crsinst -upgrade -n node1,node2 -rolling 
-src_crshome C:\app\11.2.0\grid -dest_crshome C:\app\11.2.0.3\grid
 -dest_version 11.2.0.3.0 -fixup -fixupdirpath C:\user\grid\fixup -verbose

D.6 Back Up the Oracle Software Before Upgrades

Before you make any changes to the Oracle software, Oracle recommends that you create a backup of the Oracle software and databases.

D.7 Performing a Rolling Upgrade of Oracle Clusterware

Use the following procedure to upgrade Oracle Clusterware from an earlier release to Oracle Clusterware 11g Release 2. You can also choose to upgrade Oracle ASM during the upgrade of Oracle Clusterware.

  1. If there are non-clustered, or single-instance, Oracle databases that use Oracle ASM running on any of the nodes in the cluster, they must be shut down before you start the upgrade. Listeners associated with those databases do not have to be shut down.

    Note:

    Oracle recommends that you leave Oracle Real Application Clusters (Oracle RAC) instances running during the Oracle Clusterware upgrade. During the upgrade process, the database instances on the node being upgraded are stopped and started automatically during the upgrade process.
  2. Stop the Enterprise Manager Database Control on each node. Run the following command to stop Database Control:

    C:\> emctl stop dbconsole
    
  3. Start the installer, and select the option to upgrade an existing Oracle Clusterware and Oracle ASM installation.

  4. On the node selection page, select all nodes.

    Note:

    In contrast with releases prior to Oracle Clusterware 11g Release 2, all upgrades are rolling upgrades, even if you select all nodes for the upgrade. If you are upgrading from Oracle Clusterware 11g release 11.2.0.1 to release 11.2.0.2, then all nodes are selected by default. You cannot select or de-select the nodes.
  5. Select installation options as prompted.

  6. If prompted to do so, run the scripts as directed.

  7. When the Oracle Clusterware upgrade is complete, if an earlier version of Oracle ASM is installed, then the installer starts ASMCA to upgrade Oracle ASM to release 11.2. You can upgrade Oracle ASM at this time, or upgrade it later.

    Oracle recommends that you upgrade Oracle ASM at the same time that you upgrade Oracle Clusterware. Until Oracle ASM is upgraded, Oracle databases that use Oracle ASM cannot be created. Until Oracle ASM is upgraded, the Oracle ASM management tools in the Oracle Grid Infrastructure 11g Release 2 (11.2) home (for example, SRVCTL) will not work.

    Note:

    After the upgrade, if you set the Oracle Cluster Registry (OCR) backup location manually to the earlier release Oracle Clusterware home (CRS home), then you must change the OCR backup location to the Oracle Grid Infrastructure home (Grid home). If you did not set the OCR backup location manually, then this issue does not concern you.

    Because upgrades of Oracle Clusterware are out-of-place upgrades, the CRS home for the earlier release Oracle Clusterware cannot be the location of the OCR backups for Oracle Clusterware 11g Release 2. Backups in the old Oracle Clusterware home could be deleted.

D.8 Performing a Rolling Upgrade of Oracle ASM

After you have completed the Oracle Clusterware 11g Release 2 (11.2) upgrade, if you did not choose to upgrade Oracle ASM when you upgraded Oracle Clusterware, then you can do it separately using ASMCA to perform rolling upgrades.

You can use ASMCA to complete the upgrade of Oracle ASM separately, but you should perform the upgrade as soon as possible after you upgrade Oracle Clusterware, because Oracle ASM management tools such as SRVCTL will not work until Oracle ASM is upgraded.

Note:

ASMCA performs a rolling upgrade only if the earlier version of Oracle ASM is either 11.1.0.6 or 11.1.0.7. Otherwise, ASMCA performs a normal upgrade, during which ASMCA shuts down all Oracle ASM instances on all nodes of the cluster, and then starts an Oracle ASM instance on each node from the new Oracle Grid Infrastructure home.

D.8.1 About Upgrading Oracle ASM Separately

Note the following if you intend to perform rolling upgrades of Oracle ASM:

  • The active version of Oracle Clusterware must be 11g Release 2 (11.2). To determine the active version, enter the following command from the Grid home:

    C:\..\bin> crsctl query crs activeversion
    
  • You can upgrade a single instance Oracle ASM installation to a clustered Oracle ASM installation. However, you can only upgrade an existing single instance Oracle ASM installation if you run the installation from the node on which the Oracle ASM installation is installed. You cannot upgrade a single instance Oracle ASM installation on a remote node.

  • You must ensure that any rebalance operations on your existing Oracle ASM installation are completed before starting the upgrade.

  • During the upgrade, you alter the state of the Oracle ASM instances to an upgrade state. Because this upgrade state limits Oracle ASM operations, you should complete the upgrade soon after you begin. The following operations are allowed when an Oracle ASM instance is in the upgrade state:

    • Diskgroup mounts and dismounts

    • Opening, closing, resizing, or deleting database files

    • Recovering instances

    • Queries of fixed views and packages: Users are allowed to query fixed views and run anonymous PL/SQL blocks using fixed packages, such as DBMS_DISKGROUP)

D.8.2 Using ASMCA to Upgrade Oracle ASM

Complete the following procedure to upgrade Oracle ASM:

  1. On the node on which you plan to start the upgrade, set the environment variable ASMCA_ROLLING_UPGRADE to true to put the Oracle ASM instance in an upgrade state, for example:

    C:\> set ASMCA_ROLLING_UPGRADE=true
    
  2. From the Oracle Grid Infrastructure 11g Release 2 (11.2) home, start ASMCA, as shown in the following example, where C:\app\11.2.0\grid is the Grid home:

    C:\> cd app\11.2.0\grid\bin
    C:\..\bin> asmca.bat
    
  3. Select Upgrade.

    ASMCA upgrades Oracle ASM in succession for all nodes in the cluster.

  4. When the upgrade is complete for all the nodes, unset the environment variable ASMCA_ROLLING_UPGRADE:

    C:\> set ASMCA_ROLLING_UPGRADE=
    

See Also:

Oracle Database Upgrade Guide and Oracle Automatic Storage Management Administrator's Guide for additional information about preparing an upgrade plan for Oracle ASM, and for starting, completing, and stopping Oracle ASM upgrades

D.9 Updating Oracle Enterprise Manager Target Parameters

Because Oracle Clusterware 11g Release 2 (11.2) is an out-of-place upgrade, the upgraded software has a new location for the Grid home. The path for the Oracle Clusterware home in some Oracle Enterprise Manager parameter files must be changed. If you do not change the parameter, then you will encounter errors such as "cluster target broken" on Oracle Enterprise Manager Database Control or Grid Control.

Use the following procedure to resolve this issue:

  1. Log in to Database Control or Grid Control.

  2. Select the Cluster tab.

  3. Click Monitoring Configuration.

  4. Update the value for Oracle Home with the new path for the Grid home.

D.10 Checking Cluster Health Monitor Repository Size After Upgrading

If you are upgrading from a prior release using IPD/OS to Oracle Grid Infrastructure Release 2 (11.2.0.3), then you should review the Cluster Health Monitor repository size (the CHM repository). Oracle recommends that you review your CHM repository needs, and enlarge the repository size if you want to maintain a larger CHM repository.

Note:

Your previous IPD/OS repository is deleted when you install Oracle Grid Infrastructure.

By default, the CHM repository size for release 11.2.0.3 and later is a minimum of either 1GB or 3600 seconds (1 hour).

To enlarge the CHM repository, use the following command syntax, where RETENTION_TIME is the size of CHM repository in number of seconds:

oclumon manage -repos resize RETENTION_TIME

The value for RETENTION_TIME must be more than 3600 (one hour) and less than 259200 (three days). If you enlarge the CHM repository size, then you must ensure that there is local space available for the repository size you select on each node of the cluster. If there is not sufficient space available, then you can move the repository to shared storage.

For example, to set the repository size to four hours:

C:\> oclumon manage -repos resize 14400

D.11 Troubleshooting Upgrade Issues

This section describes some commonly encountered problems during the upgrade process.

OUI-35078: Error in updating env on nodes nodename
Cause: Processes from a prior release are not completely stopped before the upgrade. Typically, a dbconsole process is running
Action: Run the command emctl stop dbconsole. Check to ensure other Oracle processes from the earlier release software are stopped.

D.12 Downgrading Oracle Clusterware After an Upgrade

After a successful or a failed upgrade to Oracle Clusterware 11g Release 2 (11.2), you might want to restore Oracle Clusterware to the previous version.

To restore Oracle Clusterware 11g Release 2 (11.2) to an earlier version of Oracle Clusterware, use the following procedure. In the following procedure, the local node is the first node on which the rootupgrade script was run. The remote nodes are all other nodes that were upgraded.

  1. Use the downgrade procedure for the release to which you want to downgrade.

    Downgrading to Oracle Clusterware 11g Release 1 (11.1) or Oracle Clusterware 10g Release 2 (10.2):

    There is currently no supported downgrade procedure. Instead, you must reinstall Oracle Clusterware for the target release and then reapply all required patches.

    Downgrading to releases 11.2.0.1, 11.2.0.2, or 11.2.0.3:

    Use the command syntax rootcrs.pl -downgrade -oldcrshome oldGridHomePath -version oldGridversion, where oldGridhomepath is the path to the previous release Oracle Grid Infrastructure home, and oldGridversion is the release to which you want to downgrade. For example:

    rootcrs.pl -downgrade -oldcrshome C:\app\grid -version 11.2.0.1.0
    

    Note:

    Ensure that Oracle Clusterware version is specified in the correct format. For example, 11.1.0.1.0.

    If you want to stop a partial or failed 11g Release 2 (11.2) installation and restore the previous release of Oracle Clusterware, then use the -force flag with this command.

  2. After the rootcrs.pl -downgrade script has completed on all remote nodes, on the local node use the command Grid_home\perl\bin\perl\rootcrs.pl -downgrade -lastnode -oldcrshome oldGridhome -version oldGridversion [-force], where oldGridhome is the home of the earlier Oracle Clusterware installation, and oldGridversion is the release number of the earlier Oracle Clusterware installation.

    For example:

    C:\app\11.2.0.3\grid\perl\bin\perl> rootcrs.pl -downgrade -lastnode 
    -oldcrshome C:\app\grid -version 11.2.0.1.0
    

    This script downgrades the OCR. If you want to stop a partial or failed 11g Release 2 (11.2) installation and restore the previous release Oracle Clusterware, then use the -force flag with this command.

  3. Run the as prompted by the rootcrs.pl -downgrade command.

  4. Log in as the Grid Infrastructure installation owner and run the following commands, where C:\app\11.2.0\grid is the location of the new Grid home (11.2):

    setup.exe -nowait -waitforcompletion -ignoreSysPrereqs
     -updateNodeList -silent CRS=false ORACLE_HOME=C:\app\11.2.0\grid
    

    Append -cfs to the end of the above command is the Oracle Grid home is a shared home.

  5. If the newer version of Oracle Grid Infrastructure is 11.2.0.2 or later

    1. If you are downgrading from Oracle Grid Infrastructure Release 2 (11.2.0.4) to an earlier release of 11.2, after execution of the upgrade scripts is completed on all cluster nodes, run the following command from the 11.2.0.4 Grid home:

      acfsroot uninstall
      

      After the command completes, from the earlier release Grid home, run the following command as an Administrator user:

      acfsroot install
      
    2. On each node, start Oracle Clusterware from the earlier release Oracle Clusterware home using the command crsctl start crs. For example, where the earlier release home is C:\app\grid, use the following command on each node:

      C:\app\grid\bin> crsctl start crs
      
  6. Run the gridconfig.bat script as prompted by the rootcrs.pl -downgrade command from the earlier release Grid home in sequence on each member node of the cluster. After you complete this task, downgrade is completed.

    Running gridconfig.bat from the earlier release Oracle Clusterware installation home restarts the Oracle Clusterware stack, starts up all the resources previously registered with Oracle Clusterware in the older version, and configures the old initialization scripts to run the earlier release Oracle Clusterware stack.

  7. As the Grid infrastructure installation owner, run the following command, where C:\app\grid represents the home directory of the earlier Oracle Clusterware installation, for example:

    setup.exe -nowait -waitforcompletion -ignoreSysPrereqs
     -updateNodeList -silent CRS=true ORACLE_HOME=C:\app\grid
    

    Append -cfs to the end of the above command is the Oracle Clusterware home is a shared home.