Oracle Fail Safe allows a cluster system to continue to provide service while the software is being upgraded to the next release. This process is called a rolling upgrade because each node is upgraded and restarted in turn, until all server nodes and all client nodes have been upgraded. You might want to perform a rolling upgrade for the following reasons:
To upgrade the Oracle Fail Safe software or install patches
To upgrade or patch a highly available Oracle single-instance database
To upgrade or patch other Oracle software
When you upgrade Oracle software, do not start the installation procedure while any Oracle Fail Safe Manager operations or MSCS Cluster Administrator operations are in progress on groups. You must wait for a quiet period in cluster operations before you proceed with the steps described in this appendix.
Note:
To ensure minimal downtime and to identify any potential issues with other software that might be running on the cluster, Oracle recommends that you test the operations described in this appendix on an identically configured test cluster before you upgrade the production cluster.This appendix includes the following topics:
During a rolling upgrade, you need to perform a planned failover to make the cluster resources running on one node move to another node. Each planned failover will disconnect users and cause a rollback of any interrupted, uncommitted work if a database is involved in the operation.
This brief outage typically is less than 1 minute and always is less than the downtime that would be experienced by users if Oracle Fail Safe was not installed. To prevent loss of work, notify users that you plan to shut down the system, and have all users commit their work and log off before you perform the failover. Advise users when they can reconnect and resume work after the database fails over to another cluster node.
While you are performing upgrades to a two-node cluster, remember that failback is disabled and your cluster is not protected against failures. In addition, each time that you stop and restart the cluster service on a node (with MSCS Cluster Administrator), all remaining cluster resources running on the node fail over to another node with a similar brief loss of service.
Oracle recommends the following procedure for upgrading software on your cluster when just the database software or both the database software and Oracle Fail Safe software must be upgraded.
Remove each database from its group.
Upgrade the Oracle Database software and the Oracle Fail Safe software if required. The order in which you perform these two upgrades is not important.
Add each database back to its group.
If you are only performing an Oracle Fail Safe software upgrade or applying a patch to the database software (but you are not going to upgrade the database software), then you do not need to remove any database from its group prior to performing the upgrade or applying the patch.
Patching typically involves updating the software from one release to another where only the fourth or fifth digit in the release number changes, for example, 9.2.0.1 to 9.2.0.2. An upgrade is required when the first, second, or third digit in the release number changes, for example 9.0.1 to 9.2.0 or 9.0.1 to 10.0.1.
This section describes how to perform a rolling upgrade when you want to upgrade the Oracle Fail Safe software on a cluster that is currently running Oracle Fail Safe (for example, to upgrade the cluster from Oracle Fail Safe release 3.3.2 to release 3.3.3). The tasks described in this section apply to upgrades from any Oracle Fail Safe release before the current Oracle Fail Safe release.
If you are performing a rolling upgrade of the Oracle Fail Safe software, then you must upgrade the Oracle Services for MSCS and Oracle Fail Safe Manager software clusterwide. You cannot run multiple releases of the Oracle Services for MSCS software on the same cluster. See the Oracle Fail Safe Release Notes for information about the compatibility of various Oracle Fail Safe Manager releases with various releases of Oracle Fail Safe Server or Oracle Services for MSCS.
Perform the steps described in Table A-1 on only one cluster node at a time.
Table A-1 Steps Required to Upgrade to Oracle Fail Safe
| Step | Task | Tool | Comments |
|---|---|---|---|
|
1 |
Change the group attributes to the Prevent Failback mode for each group. |
Oracle Fail Safe Manager |
Select a group from the Oracle Fail Safe Manager tree view, select the Failback tab, make a note of the current settings, and then change the setting to Prevent Failback. Repeat these steps for each group in the cluster. Changing the failback attributes prevents the group from failing back after its preferred node is restarted (later in Step 6). |
|
2 |
Move groups off the node on which you plan to perform the upgrade. |
Oracle Fail Safe Manager |
In the tree view, select a group name, then select Move to a Different Node. Repeat for each group on the node. By moving groups to another node, you can upgrade the current node while users continue to access the group on another node. When moving groups that contain databases with this method, Oracle Fail Safe will perform a checkpoint operation. |
|
3 |
Exit Oracle Fail Safe Manager. |
Oracle Fail Safe Manager |
From the File menu, select Exit. Note: Do not use Oracle Fail Safe Manager on other cluster nodes during the rest of the upgrade procedure until you run |
|
4 |
Deinstall the previous release of Oracle Fail Safe Manager and Oracle Services for MSCS. |
Oracle Universal Installer |
See Chapter 4 for instructions. |
|
5 |
Restart the Cluster Service service |
Windows Services |
The Cluster Service service must be restarted to eliminate all references to Fail Safe executables and message files. |
|
6 |
Install the current release of Oracle Fail Safe. |
Oracle Fail Safe installation CD-ROM |
You must install the same release of Oracle Fail Safe software on all server nodes in the cluster. Follow the installation instructions in Chapter 2. |
|
7 |
Restart the node. |
Microsoft Windows |
Note: You must wait until the first node is finished restarting before you begin installing Oracle Fail Safe on other nodes. |
|
8 |
Move groups that contain Oracle databases to the upgraded node. |
MSCS Administrator |
Move cluster groups that contain Oracle databases to the upgraded node. Note: To avoid problems that can arise due to version mismatches, do not use Oracle Fail Safe Manager to perform this task; use MSCS Administrator. |
|
9 |
Repeat steps 4 through 6 on the other cluster node or nodes. |
Various |
No comments. |
|
10 |
Install Oracle Fail Safe Manager on client nodes. |
Oracle Fail Safe installation CD-ROM |
See the Oracle Fail Safe Release Notes for information about which releases of Oracle Fail Safe Manager can be used with various releases of Oracle Services for MSCS. |
|
11 |
Run the |
Oracle Fail Safe Manager |
Right-click the cluster name in the tree view, then select Verify Cluster. This step verifies that there are no discrepancies in the software installation (such as with the release information) on each node in the cluster, and registers resource DLLs. |
|
12 |
Review the |
Oracle Fail Safe Manager |
Carefully review each error or warning and take corrective action if necessary. |
|
13 |
Restore the failback policy attributes on the groups. |
Oracle Fail Safe Manager |
Select a group from the Oracle Fail Safe Manager tree view, select the Failback tab, and restore the original settings (as noted in Step 1). Repeat for each group on the cluster. |
|
14 |
Move each group back to its preferred node. |
Oracle Fail Safe Manager |
In the tree view, right-click the group, then select Move to a Different Node. If you are working in a two-node cluster, then you will be asked to confirm the move to the other node. If you are working in a cluster that contains more than two nodes, then you will be asked to select the destination node. This step redistributes the workload across the nodes in the cluster. |
|
15 |
Run the |
Oracle Fail Safe Manager |
In the tree view, right-click a group, then select Verify Group to check all resources in all groups and confirm that they have been configured correctly. Repeat for each group in the cluster. See the Oracle Fail Safe Concepts and Administration Guide for a description of the validation tests performed during a |
This section describes how to patch or perform a rolling upgrade on a database that has been configured for high availability using Oracle Fail Safe. As mentioned previously, patching typically involves updating the database from one release to another where only the fourth or fifth digit in the release number changes, for example, 9.2.0.1 to 9.2.0.2. An upgrade is required when the first, second, or third digit in the release number changes, for example 9.0.1 to 9.2.0 or 9.0.1 to 10.0.1.
This section contains the following topics:
An upgrade requires installing a new version of the database. Typically, it is installed into its own Oracle home, separate from the earlier version of the database and from the home into which Oracle Fail Safe was installed.
To save time, you can install the new database version on each cluster node while the old version of the database is still running in an Oracle Fail Safe group.
Consider performing a database backup prior to any major upgrade.
Perform the steps in Table A-2 to perform the upgrade. Perform the steps described on one cluster node at a time.
Table A-2 Steps for Upgrading a Database Configured for High Availability
| Step | Task | Tool | Comments |
|---|---|---|---|
|
1 |
Check compatibility |
Not applicable |
Ensure that the release of the database to which you plan to upgrade is compatible with the version of Oracle Fail Safe you are running. See the section on software compatibility in the Oracle Fail Safe Release Notes. If the Oracle Database release and the Oracle Fail Safe software release are not compatible, then see Section A.3. |
|
2 |
Install the Oracle Database software on each cluster node. |
Oracle Universal Installer |
Use the same directory and Oracle home name on each cluster node. |
|
3 |
Remove each database from its group. |
Oracle Fail Safe Manager |
In the tree view, right-click the database and select Remove from Group. Repeat for each database that you plan to upgrade. |
|
4 |
Upgrade each database. |
Database Migration Assistant (or perform manually) |
Instructions for performing the upgrade are included in the database upgrade (or migration) manual for the release to which you are upgrading. |
|
5 |
Verify each standalone database |
Oracle Fail Safe Manager |
Right-click the database in the tree view and select Verify Standalone Database. Correct any reported problems. |
|
6 |
Add each database back to its original group. |
Oracle Fail Safe Manager |
Right-click the database, then select Add to Group and follow the steps through the Add Resource to Group Wizard. |
When you patch a database that has been configured for high availability, it is not necessary to remove the database from the group prior to applying the patch.
Table A-3 describes how to patch databases configured for high availability in a two-node cluster. These steps assume that there are two nodes, Node A and Node B, and that Node A is the first node that will be patched. Perform these steps on only one cluster node at a time.
Table A-3 Steps for Patching a Database Configured for High Availability
| Step | Task | Tool | Comments |
|---|---|---|---|
|
1 |
Change the failback attributes to Prevent Failback for each group in the cluster. |
Oracle Fail Safe Manager |
In the tree view, select a group name, then select the Failback tab. Make a note of the current settings, and then change the setting to Prevent Failback. Repeat these steps for each group in the cluster. This step prevents the group from failing back to its current node while the node is being restarted or when the cluster service is restarted. |
|
2 |
Move all groups on Node A to Node B. |
Oracle Fail Safe Manager and MSCS Cluster Administrator |
In the tree view, select a group from Node A, and then select Move to a Different Node. Repeat for each group on Node A. Use MSCS Cluster Administrator to move the Cluster Group if it is on Node A. By moving all groups to another node, you can work on the current node. Oracle Fail Safe will perform a checkpoint operation on any databases within a group that is moved. |
|
3 |
Exit Oracle Fail Safe Manager and Cluster Administrator. |
Oracle Fail Safe Manager and MSCS Cluster Administrator |
In the File menu, select Exit. (The steps are the same in both Oracle Fail Safe Manager and MSCS Cluster Administrator.) |
|
4 |
Stop the cluster service on Node A. |
Microsoft Control Panel |
Open the Microsoft Windows Services window and then stop the Cluster Services service. |
|
5 |
Stop all running Oracle applications and all remaining Oracle services. |
Microsoft Control Panel |
Open the Microsoft Windows Services window and then stop all running Oracle services, including the Oracle Fail Safe service and the Distributed Transaction Coordinator Service (if it is running). |
|
6 |
Install the database patch in the correct home on Node A. |
Oracle Universal Installer |
Follow the instructions that come with the patch. If you did not stop the Distributed Transaction Coordinator Service in Step 5, then it is likely that a write error for |
|
8 |
Adjust the initialization parameter file, if needed. |
Editor of choice for pfile, SQL ALTER SYSTEM statement for spfile. |
Changes to the initialization parameter file should be made before databases are failed back. See the readme file supplied with the database patch for information about required parameter file changes. If you are maintaining multiple copies of the initialization parameter file (because you are maintaining them on private disks), then ensure that you update the file that will be used when the database is moved back to the current node. Once the patch scripts are completed, you will change the initialization parameter file back to the original values and restart the database. |
|
9 |
Move groups back to the selected node. |
Oracle Fail Safe Manager |
Once the groups are moved back to Node A, they will be restarted and running under the patched version of the database management system. Therefore, run the patch scripts as soon as the groups are moved back. By having changed the initialization parameter file as described in Step 8, you ensure that the database starts with the parameters needed to complete the patch. |
|
10 |
See Table A-4 for instructions on migrating to a new database release. |
Various |
See Table A-4 for the steps. |
|
11 |
Restrict user access on each database to be upgraded. |
SQL*Plus |
As each database comes online on the selected node, restrict user access with the following SQL statement: ALTER SYSTEM ENABLE RESTRICTED SESSION; Do this as soon as possible after the database instance starts to prevent users from connecting to the database while the upgrade scripts run. You can also take the listener service for the group offline to prevent users from making Oracle Net connections. |
|
12 |
Run the upgrade scripts for the patch against each database being patched. |
SQL*Plus |
See the readme file supplied with the database patch. |
|
13 |
Repeat steps 4 through 7, replacing Node A in the instructions with Node B. |
Various |
To save time, you can run the upgrade scripts against the databases on Node A while the patch is being installed on Node B. |
|
14 |
Verify each group in the cluster. |
Oracle Fail Safe Manager |
In the tree view, right-click a group name, then select Verify Group. Repeat for each group in the cluster. |
|
15 |
End the restricted session on each database. |
SQL*Plus |
End the restricted session on each database with the following SQL statement: SQL> ALTER SYSTEM DISABLE RESTRICTED SESSION; |
|
16 |
Verify the cluster. |
Oracle Fail Safe Manager |
In the tree view, right-click the cluster name, then select Verify Cluster. |
|
17 |
Oracle Fail Safe Manager |
Select a group from the Oracle Fail Safe Manager tree view, select the Failback tab, and restore the original settings (as noted in step 1). Repeat for each group on the cluster. |
|
|
18 |
Move each group back to its preferred node. |
Oracle Fail Safe Manager |
In the tree view, right-click a group name, then select Move to a Different Node. Repeat for each node in the cluster. This step balances the workload across the nodes in the cluster. |
For information about database upgrades, refer to the Oracle Fail Safe Concepts and Administration Guide and to the upgrade instructions for Oracle Database Upgrade Assistant.
For Oracle release 9.2 and later, when any type of patch is applied, the database must be started with the MIGRATE option, as shown in Table A-4.
Table A-4 Steps for Starting Up a Database with the MIGRATE Option
| Step | Task | Tool | Comments |
|---|---|---|---|
|
1 |
Take the database offline. |
Oracle Fail Safe Manager |
In the tree view, right-click the database, then select Take Offline. |
|
2 |
Exit Oracle Fail Safe Manager. |
Oracle Fail Safe Manager |
From File menu, select Exit. |
|
3 |
Manually start the service for the instance |
Microsoft Windows Services Window |
Instance service names are of the form, ORACLESERVICESOFS1 |
|
4 |
Start up the database with the MIGRATE option. |
SQL*Plus |
For example, c:\> SET ORACLE_SID=OFS1 c:\> SQLPLUS "/as sysdba" SQL> STARTUP MIGRATE |
|
5 |
Run the upgrade scripts against each database being patched. |
SQL*Plus |
See the readme file supplied with the database patch. |
|
6 |
Shut down the database and stop the service. |
SQL*Plus and Microsoft Windows Services Window |
See the readme file supplied with the database patch. |
|
7 |
Place the database resource online. |
Oracle Fail Safe Manager |
In the tree view, right-click the database, then select Place Online. |
|
8 |
Resume the steps in table Table A-3, Step 12. |
Various |
You are being directed to skip step 11 in Table A-3 intentionally. When you start up a database with the |
This section describes upgrading non-database Oracle product software and installing patches that require you to stop the Oracle services being used by Oracle Fail Safe. See Section A.4 for the same information on Oracle Database software.
If your cluster is in an active/passive configuration, then you can eliminate a failover if you start the rolling upgrade on the passive cluster node. Once you have finished the upgrade on the passive cluster node, you can skip to Step 4 in Table A-5.
Perform the steps on only one cluster node at a time.
Table A-5 Steps Required to Upgrade Other Oracle Product Software