Performing Rolling Upgrade of Oracle Grid Infrastructure

Review this information to perform rolling upgrade of Oracle Grid Infrastructure.

Upgrading Oracle Grid Infrastructure Using Setup Wizard

Complete this procedure to upgrade Oracle Grid Infrastructure (Oracle Clusterware and Oracle Automatic Storage Management) from an earlier release using the setup wizard.

At any time during the upgrade, if you have a question about what you are being asked to do, or what input you are required to provide during upgrade, click the Help button on the installer page.

Note:

You must migrate from Oracle ASMFD to Oracle ASMLib before starting the upgrade.
You must remove Grid Infrastructure Management Repository (GIMR) before starting the upgrade.

You should have your network information, storage information, and operating system users and groups available to you before you start upgrade, and you should be prepared to run root scripts.

As the grid user, download the Oracle Grid Infrastructure image files and extract the files to the Grid home.

For example:
```
mkdir -p /u01/app/23.0.0/grid
chown grid:oinstall /u01/app/23.0.0/grid
cd /u01/app/23.0.0/grid
unzip -q download_location/grid_home.zip
```
download_location/grid_home.zip is the path of the downloaded Oracle Grid Infrastructure image file.
Note:
- You must extract the image software into the directory where you want your Grid home to be located.
- Download and copy the Oracle Grid Infrastructure image files to the local node only. During upgrade, the software is copied and installed on all other nodes in the cluster.
If Oracle ASMFD was configured in the earlier release, then migrate from Oracle ASMFD to Oracle ASMLib.
Confirm if Grid Infrastructure Management Repository (GIMR) was configured in the earlier release.
```
srvctl config mgmtdb
```
Confirm if Oracle Fleet Patching and Provisioning (Oracle FPP) was configured in central server mode in the earlier release.
```
srvctl config rhpserver
```
Note:
- If Oracle FPP is configured on your cluster, then you are recommended to use the Oracle FPP Self-Upgrade feature for smooth migration of the metadata from GIMR to the new metadata repository. Refer to Oracle Fleet Patching and Provisioning Self Upgrade for more information about how to use the Oracle FPP Self-Upgrade feature.
- If you upgrade Oracle Grid Infrastructure using this procedure, then you must re-configure Oracle FPP metadata repository to use the Oracle FPP Server. Refer to Migrating Metadata from GIMR to Oracle FPP Metadata Repository for steps to re-configure Oracle FPP metadata repository.
If the GIMR was configured in the earlier release and Oracle FPP was configured in central mode, then export the Oracle FPP Metadata, export the CHA models, and remove GIMR from the existing Oracle Grid Infrastructure installation.
```
Grid_home/crs/install/reposScript.sh -export_dir=dir_to_export_CHA_model_and_Oracle_FPP_metadata
```
Note:
- Exporting CHA models is optional. If the export fails, then you can continue removing the GIMR.
- You can remove GIMR independent of the upgrade to Oracle Grid Infrastructure 26ai.
Start the Oracle Grid Infrastructure wizard by running the following command.
```
/u01/app/23.0.0/grid/gridSetup.sh
```
Select the Upgrade Oracle Grid Infrastructure: option to upgrade Oracle Grid Infrastructure (Oracle Clusterware and Oracle ASM).
On the Node Selection page, ensure that all cluster nodes are selected.

Note:
If any cluster node is unreachable, then select Skip upgrade on unreachable nodes. Otherwise, the upgrade will not complete until the unreachable nodes become reachable.
Select installation options as prompted. Oracle recommends that you configure root script automation, so that the rootupgrade.sh script can be run automatically during the upgrade.
Run the root scripts, either automatically or manually:
- Running root scripts automatically:
  If you have configured root script automation, then use the pause between batches to relocate services from the nodes running the previous release to the new release.
- Running root scripts manually
  If you have not configured root script automation, then when prompted, run the rootupgrade.sh script on each node in the cluster that you want to upgrade.
If you run the root scripts manually, then run the script on the local node first. The script shuts down the earlier release installation, replaces it with the new Oracle Clusterware release, and starts the new Oracle Clusterware installation. After the script completes successfully, you can run the script in parallel on all nodes except for one, which you select as the last node. When the script is run successfully on all the nodes except the last node, run the script on the last node.
Because the Oracle Grid Infrastructure home is in a different location than the former Oracle Clusterware and Oracle ASM homes, update any scripts or applications that use utilities, libraries, or other files that reside in the Oracle Clusterware and Oracle ASM homes.
Update the Oracle Enterprise Manager target parameters as described in the topic Updating Oracle Enterprise Manager Cloud Control Target Parameters.

Note:

After upgrading to Oracle Grid Infrastructure 26ai, remove the ADR_BASE=/u01/app/grid entry from $ORACLE_HOME/network/admin/sqlnet.ora file, if you use Oracle ASM for database storage.
At the end of the upgrade, if you set the Oracle Cluster Registry (OCR) backup location manually to the earlier release Oracle Clusterware home (CRS home), then you must change the OCR backup location to the new Oracle Grid Infrastructure home (Grid home). If you did not set the OCR backup location manually, then the backup location is changed for you during the upgrade.
Because upgrades of Oracle Clusterware are out-of-place upgrades, the previous release Oracle Clusterware home cannot be the location of the current release OCR backups. Backups in the old Oracle Clusterware home can be deleted.
If the cluster being upgraded has a single disk group that stores the OCR, OCR backup, Oracle ASM password, and the Oracle ASM password file backup, then Oracle recommends that you create a separate disk group or use another existing disk group and store the OCR backup and Oracle ASM password file backup in that disk group.

Upgrading Oracle Grid Infrastructure Using Command-Line Interface

Complete this procedure to upgrade Oracle Grid Infrastructure (Oracle Clusterware and Oracle Automatic Storage Management) from an earlier release using the command-line interface.

Note:

You must migrate from Oracle ASMFD to Oracle ASMLib before starting the upgrade.
You must remove Grid Infrastructure Management Repository (GIMR) before starting the upgrade.

You should have your network information, storage information, and operating system users and groups available to you before you start upgrade, and you should be prepared to run root scripts.

As the grid user, download the Oracle Grid Infrastructure image files and extract the files to the Grid home.

For example:
```
mkdir -p /u01/app/23.0.0/grid
chown grid:oinstall /u01/app/23.0.0/grid
cd /u01/app/23.0.0/grid
unzip -q download_location/grid_home.zip
```
download_location/grid_home.zip is the path of the downloaded Oracle Grid Infrastructure image file.
Note:
- You must extract the image software into the directory where you want your Grid home to be located.
- Download and copy the Oracle Grid Infrastructure image files to the local node only. During upgrade, the software is copied and installed on all other nodes in the cluster.
If Oracle ASMFD was configured in the earlier release, then migrate from Oracle ASMFD to Oracle ASMLib.
Confirm if Grid Infrastructure Management Repository (GIMR) was configured in the earlier release.
```
srvctl config mgmtdb
```
Confirm if Oracle Fleet Patching and Provisioning (Oracle FPP) was configured in central server mode in the earlier release.
```
srvctl config rhpserver
```
Note:
- If Oracle FPP is configured on your cluster, then you are recommended to use the Oracle FPP Self-Upgrade feature for smooth migration of the metadata from GIMR to the new metadata repository. Refer to Oracle Fleet Patching and Provisioning Self Upgrade for more information about how to use the Oracle FPP Self-Upgrade feature.
- If you upgrade Oracle Grid Infrastructure using this procedure, then you must re-configure Oracle FPP metadata repository to use the Oracle FPP Server. Refer to Migrating Metadata from GIMR to Oracle FPP Metadata Repository for steps to re-configure Oracle FPP metadata repository.
If the GIMR was configured in the earlier release and Oracle FPP was configured in central mode, then export the Oracle FPP Metadata to re-configure Oracle FPP after the upgrade and remove GIMR from the existing Oracle Grid Infrastructure installation.
```
Grid_home/crs/install/reposScript.sh -export_dir=dir_to_export_CHA_model_and_Oracle_FPP_metadata
```
Note:
- Exporting CHA models is optional. If the export fails, then you can continue removing the GIMR.
- You can remove GIMR even if you do not upgrade to Oracle Grid Infrastructure 26ai. However, it is mandatory to remove GIMR before the upgrade.

Start the Oracle Grid Infrastructure installation by running the following command.

cd /u01/app/23.0.0/grid
./gridSetup.sh -upgrade -ORACLE_BASE /u01/app/grid [-ignoreDownNodes] -executeConfigTools -executeRootScript -configMethod ROOT

Provide the root user password when prompted.
Because the Oracle Grid Infrastructure home is in a different location than the former Oracle Clusterware and Oracle ASM homes, update any scripts or applications that use utilities, libraries, or other files that reside in the Oracle Clusterware and Oracle ASM homes.
Update the Oracle Enterprise Manager target parameters as described in the topic Updating Oracle Enterprise Manager Cloud Control Target Parameters.

Note:

After upgrading to Oracle Grid Infrastructure 26ai, remove the ADR_BASE=/u01/app/grid entry from $ORACLE_HOME/network/admin/sqlnet.ora file, if you use Oracle ASM for database storage.
At the end of the upgrade, if you set the Oracle Cluster Registry (OCR) backup location manually to the earlier release Oracle Clusterware home (CRS home), then you must change the OCR backup location to the new Oracle Grid Infrastructure home (Grid home). If you did not set the OCR backup location manually, then the backup location is changed for you during the upgrade.
Because upgrades of Oracle Clusterware are out-of-place upgrades, the previous release Oracle Clusterware home cannot be the location of the current release OCR backups. Backups in the old Oracle Clusterware home can be deleted.
If the cluster being upgraded has a single disk group that stores the OCR, OCR backup, Oracle ASM password, and the Oracle ASM password file backup, then Oracle recommends that you create a separate disk group or use another existing disk group and store the OCR backup and Oracle ASM password file backup in that disk group.

Completing an Oracle Clusterware Upgrade when Nodes Become Unreachable

If some nodes become unreachable in the middle of an upgrade, then you cannot complete the upgrade, because the upgrade script (rootupgrade.sh) did not run on the unreachable nodes. Because the upgrade is incomplete, Oracle Clusterware remains in the previous release.

Confirm that the upgrade is incomplete by entering the following command:
```
$ crsctl query crs activeversion
```
The active release should be the source release.
To resolve this problem, run the rootupgrade.sh command with the -force flag using the following syntax:
```
Grid_home/rootupgrade -force
```
For example:
```
# /u01/app/23.0.0/grid/rootupgrade -force
```
This command forces the upgrade to complete. The force cluster upgrade has the following limitations:
- All active nodes must be upgraded to the newer release.
- All inactive nodes (accessible or inaccessible) may be either upgraded or not upgraded.
- For inaccessible nodes, after patch set upgrades, you can delete the node from the cluster. If the node becomes accessible later, and the patch version upgrade path is supported, then you can upgrade it to the new patch version.
Verify that the upgrade has completed by using the following command:
```
$ crsctl query crs activeversion -f
Oracle Clusterware active version on the cluster is [23.0.0.0.0]. 
The cluster upgrade state is [FORCED]. The cluster active patch level is [0].
```
The active release should be the upgraded release. The upgrade state FORCED indicates that Oracle Clusterware upgrade was completed forcibly on all reachable nodes, while one or more cluster nodes were unreachable. You can either upgrade or delete the unreachable cluster nodes to change the cluster upgrade state from FORCED to NORMAL.

Joining Inaccessible Nodes After Forcing an Upgrade

Use this procedure to join inaccessible nodes after a force cluster upgrade.

After you complete a force cluster upgrade, you can use the procedure described here to join inaccessible nodes to the cluster as an alternative to deleting the nodes, which was required in earlier releases. To use this option, you must already have Oracle Grid Infrastructure 26ai software installed on the nodes.

Log in as the root user on the node that you want to join to the cluster.
Change directory to the Oracle Grid Infrastructure home directory:
```
$ cd Grid_home
```
Run the following command, where upgraded_node is one of the cluster nodes that is upgraded successfully:
```
$ rootupgrade.sh -join -existingnode upgraded_node
```

Changing the First Node for Install and Upgrade

If the first node becomes inaccessible, you can force another node to be the first node for installation or upgrade.

During installation, if root.sh fails to complete on the first node, run the following command on another node using the -force option:

root.sh -force -first

For upgrade:

rootupgrade.sh -force -first