Patching and Upgrading Oracle Grid Infrastructure

The Oracle Fleet Patching and Provisioning Server provides an efficient and secure platform for patching and upgrading Oracle Grid Infrastructure.

Patching Oracle Grid Infrastructure

Fleet Patching and Provisioning provides two methods to patch Oracle Grid Infrastructure software homes: rolling (in batches) and non-rolling.

Patching Oracle Grid Infrastructure software involves moving the Grid home to a patched version of the current Grid home. When the patching operation is initiated by a Fleet Patching and Provisioning Server or Client, the patched version must be a working copy of a gold image. The working copy to which you are moving the Grid home can be at a lower patch level than the current home. This facilitates rollback if any problems occur after moving to the higher-level patched home.

When Oracle FPP performs out-of-place patching, it evaluates the list of non-rolling patches on the source and the destination homes. Oracle FPP compares the two lists, and if the target home has additional non-rolling patches that are not in the source home, the Oracle FPP reports that the patching cannot be performed in a rolling mode. You can then take one of the following actions:

Re-provision the target home so that it does not include those additional non-rolling patches and proceed with patching in rolling mode.
Continue with the rolling patching operation with the currently provisioned home using the -forcerolling option.

You can also patch Oracle Grid Infrastructure using Fleet Patching and Provisioning Local Mode. In this case, the source and destination homes are not working copies of gold images, but are two installed homes that you deployed with some method other than using Fleet Patching and Provisioning.

For information about patching using batches, refer to Patching Oracle Grid Infrastructure and Oracle AI Database Using Batches.

Patching Oracle Grid Infrastructure Using the Rolling Method

The rolling method for patching Oracle Grid Infrastructure is the default method.

You use the rhpctl move gihome command (an atomic operation), which returns after the Oracle Grid Infrastructure stack on each node has been restarted on the new home. Nodes are restarted sequentially, so that only one node at a time will be offline, while all other nodes in the cluster remain online.

Move the Oracle Grid Infrastructure home to a working copy of the same release level, as follows:
```
$ rhpctl move gihome -sourcewc Grid_home_1 -destwc Grid_home_2
```
The preceding command moves the running Oracle Grid Infrastructure home from the current managed home (the sourcewc) to the patched home (destwc) on the specific client cluster. The patched home must be provisioned on the client.
If the move operation fails at some point before completing, then you can rerun the operation by running the command again and the operation will resume where it left off. This enables you to fix whatever problem caused the failure and resume processing from the point of failure. Or you can undo the partially completed operation and return the configuration to its initial state, as follows:
```
$ rhpctl move gihome -destwc destination_workingcopy_name -revert | -forcecomplete [authentication_option]
```
You cannot use the -revert parameter with an un-managed home.

Notes:

You cannot move the Grid home to a home that Fleet Patching and Provisioning does not manage. Therefore, rollback (to the original home) applies only to moves between two working copies. This restriction does not apply when using the independent automaton since it operates on unmanaged homes only.
You can delete the source working copy at any time after moving a Grid home. Once you delete the working copy, however, you cannot perform a rollback. Also, use the rhpctl delete workingcopy command (as opposed to rm, for example) to remove the source working copy to keep the Fleet Patching and Provisioning inventory correct.
If you use the -abort parameter to terminate the patching operation, then Fleet Patching and Provisioning does not clean up or undo any of the patching steps. The cluster, databases, or both may be in an inconsistent state because all nodes are not patched.
Use the -forcecomplete parameter to mark the move operation as complete after completing it manually.

Patching Oracle Grid Infrastructure Using the Non-Rolling Method

You can use the -nonrolling parameter with the rhpctl move gihome command, which restarts the Oracle Grid Infrastructure stack on all nodes in parallel.

As with the rolling method, this is an atomic command which returns after all nodes are online.

Note:

The non-rolling patching method is available only if there is just one cluster node. If there are multiple nodes, then there is an error suggesting to use the rolling patching method.

Use the following command to patch Oracle Grid Infrastructure in an non-rolling fashion:
```
$ rhpctl move gihome -sourcewc Grid_home_1 -destwc Grid_home_2 -nonrolling
```

Combined Oracle Grid Infrastructure and Oracle AI Database Patching

When you patch an Oracle Grid Infrastructure deployment, Oracle FPP enables you to simultaneously patch the Oracle AI Database homes on the cluster, so you can patch both types of software homes in a single maintenance operation.

Note:

You cannot patch both Oracle Grid Infrastructure and Oracle AI Database in combination with Oracle Fleet Patching and Provisioning (Oracle FPP) Local Mode.

The following optional parameters of the rhpctl move gihome command are relevant to the combined Oracle Grid Infrastructure and Oracle AI Database patching use case:

-auto: Automatically patch databases along with patching Oracle Grid Infrastructure
-dbhomes mapping_of_Oracle_homes: Mapping of source and destination working copies in the following format:
```
sourcewc1=destwc1,...,source_oracle_home_path=destwcN
```
-dblist db_name_list: Patch only the specified databases
-excludedblist db_name_list: Patch all databases except the specified databases
-nodatapatch: Indicates that datapatch is not be run for databases being moved

As an example, assume that an Oracle FPP Server with Oracle Grid Infrastructure 19c (19.20) has provisioned the following working copies on an Oracle Grid Infrastructure 19c (19.19) rhpclient-less target cluster which includes the node test_749:

WC_1919_fppc03: The active Grid home on the Oracle Grid Infrastructure 19c (19.19) cluster
WC_GI_1920_fppc03: A software-only Grid home on the Oracle Grid Infrastructure 19c (19.20) cluster
WC_DB_1919_fppc03: An Oracle RAC 19c (19.19) database home running database instances
WC_DB_1920_fppc03: An Oracle RAC 19c (19.20) database home with no database instances (this is the patched home)
WC_DB_1920_fppc03: An Oracle RAC 19c (19.20) database home running database instances
WC_DB_1920_220719_fppc03: An Oracle RAC 19c (19.20) database home with no database instances (this is the patched home)

Further assume that you want to simultaneously move

Oracle Grid Infrastructure from working copy WC_1919_fppc03 to working copy WC_GI_1920_fppc03
Oracle RAC Database db1 from working copy WC_DB_1919_fppc03 to working copy WC_DB_1920_fppc03
Oracle RAC Database db2 in working copy WC_DB_1920_fppc03 to working copy WC_DB_1920_220719_fppc03

The following single command accomplishes the moves:

$ rhpctl move gihome -destwc WC_GI_1920_fppc03 -sourcewc WC_1919_fppc03
-auto -dbhomes WC_DB_1919_fppc03=WC_DB_1920_fppc03,WC_DB_1920_fppc03=WC_DB_1920_220719_fppc03

Notes:
If you have an existing Oracle home that is not currently a working copy, then specify the Oracle home path instead of the working copy name for the source home. In the preceding example, if the Oracle home path for an existing 19.19 home is /u01/app/prod/19.19.0/dbhome1, then replace WC_DB_1919_fppc03=WC_DB_1920_fppc03 with /u01/app/prod/19.20.0/dbhome1=WC_DB_1920_fppc03.
If the move operation fails at some point before completing, then you can either resolve the cause of the failure and resume the operation by rerunning the command, or you can undo the partially completed operation by issuing the following command, which restores the configuration to its initial state:
$ rhpctl move gihome -destwc WC_GI_1920_fppc03 -revert {authentication_option}

In the preceding command example, the Oracle Grid Infrastructure 19c Grid home moves from working copy WC_1919_fppc03 to working copy WC_GI_1920_fppc03, databases running on working copy WC_DB_1919_fppc03 move to working copy WC_DB_1920_fppc03, and databases running on working copy WC_DB_1920_fppc03 move to working copy WC_DB_1920_220719_fppc03.

For each node in the client cluster, RHPCTL:

Runs any configured pre-operation user actions for moving the Oracle Grid Infrastructure ( move gihome ).
Runs any configured pre-operation user actions for moving the database working copies ( move database ).
Stops services running on the node, and applies drain and disconnect options.
Performs the relevant patching operations for Oracle Clusterware and Oracle AI Database.
Runs any configured post-operation user actions for moving the database working copies ( move database ).
Runs any configured post-operation user actions for moving the Oracle Grid Infrastructure working copy ( move gihome ).

Zero-Downtime Oracle Grid Infrastructure Patching

Use Fleet Patching and Provisioning to patch Oracle Grid Infrastructure without bringing down Oracle RAC database instances.

Current methods of patching the Oracle Grid Infrastructure require that you bring down all Oracle RAC database instances on the node where you are patching the Oracle Grid Infrastructure home. This issue is addressed in the Grid Infrastructure layer where by the database instances can continue to run during the Oracle Grid Infrastructure patching.

Note:

You can use zero-downtime patching only for out-of-place patching of Oracle Grid Infrastructure 19c Release Update (RU) 19.8 or later releases with Oracle RAC or Oracle RAC One Node databases of 19c or later releases. If your Oracle RAC or Oracle RAC One Node database release is older than 19c, then the database instances stop during zero-downtime patching.

To enable zero-downtime Oracle Grid Infrastructure patching, use the rhpctl move gihome command in a manner similar to the following:

rhpctl move gihome -tgip -sourcewc source_workingcopy_name -destwc destination_workingcopy_name

Patching System Software Binaries

When using Zero Downtime Patching, only the binaries in the Oracle Grid Infrastructure user space are patched. Additional Oracle Grid Infrastructure OS system software, kernel modules and system commands including ACFS, AFD, and OKA, are not updated. These commands continue to run the version previous to the patch version. After patching, the OPatch inventory displays the new patch number in the inventory; however, the running OS system software does not contain these changes. Only the OS system software that is available in the Grid Infrastructure home has been patched.

To determine the OS system software that is available in the Grid Infrastructure home, you can run the crsctl query driver activeversion -all command. To determine what OS system software is running on the system, use crsctl query driver softwareversion -all.

To update the Grid Infrastructure OS system software on a single node, you must completely stop the Grid Infrastructure software. To stop the Grid Infrastructure software, you must stop the Oracle RAC databases on the single node. After stopping the Oracle RAC databases, run root.sh -updateosfiles to update all the Grid Infrastructure OS system software on the single node.

Zero-downtime patching does not automatically update the operating system drivers. During the zero-downtime patching process, updated operating system drivers are copied in the Grid home, but these drivers are not installed into the operating system. The cluster continues to use the older version of the operating system drivers until the updated drivers are installed in the operating system.

If you are using Oracle ASM Filter Driver (Oracle ASMFD) or Oracle ACFS for database storage, then operating system drivers are updated in either of the following scenarios:

You update your operating system kernel and restart the cluster node.
You run the rootcrs.sh -updateosfiles command on each cluster node and restart the cluster nodes, if the operating system drivers fail to install.

You can check the active operating system driver version on a cluster node using the crsctl query driver activeversion [-all] command and available operating system driver version using the crsctl query driver softwareversion [-all] [-f] command.