8 Scenario: Upgrading and Patching Database with Manually Configured Oracle Data Guard
This scenario describes upgrading and patching a database with manually configured Oracle Data Guard on Oracle Database Appliance on the DCS stack.
Note:
The following section does not apply for Oracle Data Guard set up with ODACLI commands. Use ODACLI commands to upgrade the databases.- Upgrading All Components
Upgrading an Oracle Database Appliance environment consists of upgrading DCS, server, storage, and database components. - Upgrading Oracle Database
The purpose of this section is to provide a high-level overview of the upgrade process in a primary-standby setup. - Patching Oracle Database
Patching databases on Oracle Database Appliance is an online operation. The following steps describe how to patch databases on a standby configuration. These steps apply to databases on bare metal systems and databases on DB systems.
Upgrading All Components
Upgrading an Oracle Database Appliance environment consists of upgrading DCS, server, storage, and database components.
When upgrading an Oracle Database Appliance environment where a standby system is already implemented, you can use the standby system to reduce the downtime required for completing the upgrade activities. The purpose of this section is to provide a high-level overview of the upgrade process in a primary-standby setup.
- Verify that the system is operating correctly by running pre-checks, validating hardware and system processes, and verifying system configuration using ORAChk.
- Take a backup of the operating system, Oracle Grid Infrastructure, Oracle homes, and databases in the primary environment. Refer to My Oracle Support Note 2466177.1 - ODA (Oracle Database Appliance): ODABR a System Backup/Restore Utility.
- Upgrade DCS and server components on the standby Oracle Database Appliance system.
- Switchover the primary database role and application connections to the standby system.
- Upgrade DCS and server components on the current standby, that is, the former primary system.
- For deployments with Oracle Database Appliance release earlier than 19.14, patch or upgrade the database. On bare metal deployments with Oracle Database Appliance release 19.14 and later, refer to the Oracle Database Appliance Deployment and User's Guide for the steps to patch the databases using ODACLI commands. ODACLI provides complete lifecycle management for Oracle Data Guard environments including database patching and upgrade if your Oracle Data Guard deployment was configured using ODACLI commands.
With this upgrade process, the downtime during the upgrade is minimized and system availability is affected only for the duration of upgrade or patching of the database component.
Upgrading Oracle Database
The purpose of this section is to provide a high-level overview of the upgrade process in a primary-standby setup.
Upgrading DCS, server, operating system, Oracle Grid
Infrastructure, and general firmware, storage with switchover and
switchback can help reduce downtime during upgrade. If you are only
upgrading the database component, then unless you are using a zero
downtime solution such as active-active Oracle GoldenGate solution,
some downtime is expected for the application. Following is the
process for database upgrade when a standby configuration exists:
- Verify that the system is operating correctly by running pre-checks, validating hardware and system processes, and verifying system configuration using ORAChk.
- Stop the standby
database.
[oracle@stbydb1]$ srvctl stop database -d boston - Create a new database home or use an existing one on the
standby with the version that you want to upgrade
the database to on the
primary.
[oracle@stbydb1]# odacli create-dbhome -v 19.14.0.0.220118 - Stop log shipping on the
primary.
[oracle@ proddb1] dgmgrl connect sys/welcome1@chicago DGMGRL> SHOW DATABASE 'boston' 'LogShipping'; LogShipping = 'ON' DGMGRL> edit database 'boston' SET PROPERTY 'LogShipping'='OFF'; Property "LogShipping" updated DGMGRL> SHOW DATABASE 'boston' 'LogShipping'; LogShipping = 'OFF' - Create a new database home or use an existing database
home on the primary with the version that you want
to upgrade the
database.
# odacli create-dbhome -v 19.14.0.0.220118 - Stop the application.
- Upgrade the primary database using the
odacli upgrade databasecommand.[root@proddb1]# odacli list-databases ID DB Name DB Type DB Version CDB Class Shape Storage Status DbHomeID ---------------------------------------- ---------- -------- ----------------------------------- e97cc2f3-bdd8-4775-b959-d5f79a6c59fc chicago Rac 18.11.0.0.200714 false Oltp Odb1 Asm Configured 88ce2c7-fa3d-4f93-802a-bfa50d180758 [root@proddb1]# odacli list-dbhomes ID Name DB Version Home Location Status ---------------------------------------- -------------------- ---------------------------------------- --------------------------------------------- ---------- 863c8cbe-1c5f-450e-866c-15c384580ad3 OraDB19000_home1 19.14.0.0.220118 /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_1 Configured 288ce2c7-fa3d-4f93-802a-bfa50d180758 OraDB18000_home1 18.11.0.0.200714 /u01/app/oracle/product/18.0.0.0/dbhome_1 Configured [root@proddb1]# odacli upgrade-database -i 713b68d3-8c43-4d10-973e-90a3fa88a84a -destDbHomeId 863c8cbe-1c5f-450e-866c-15c384580ad3 -sourceDbHomeId 288ce2c7-fa3d-4f93-802a-bfa50d180758 [root@proddb1]# odacli list-databases ID DB Name DB Type DB Version CDB Class Shape Storage Status DbHomeID ---------------------------------------- ---------- -------- -------------------- ---------- -------- -------- ---------- ------------ ---------------------------------- 713b68d3-8c43-4d10-973e-90a3fa88a84a chicago Rac 19.14.0.0.220118 false Oltp Odb1 Asm Configured 863c8cbe-1c5f-450e-866c-15c384580ad3 - Start the application.
- Copy the
tnsnames.orafile on the standby from the old Oracle home to the new on all nodes. - Copy the password file from the primary to
the
standby.
[oracle@proddb1]$ srvctl config database -d chicago |grep Password Password file: +DATA/CHICAGO/PASSWORD/pwdchicago.277.1023633847 [grid@proddb1 ~]$ asmcmd ASMCMD> pwcopy +DATA/CHICAGO/PASSWORD/pwdchicago.277.1023633847 /tmp/pwdboston copying +DATA/CHICAGO/PASSWORD/pwdchicago.277.1023633847 -> /tmp/pwdboston [oracle@proddb1]$ scp /tmp/pwdboston oracle@stbydb1: /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_1/dbs/orapwboston [grid@stbydb1 ~]$ asmcmd ASMCMD> pwcopy /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_1/dbs/orapwboston +DATA/BOSTON/PASSWORDFILE/pwdboston copying /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_1/dbs/orapwboston -> +DATA/BOSTON/PASSWORDFILE/pwdboston - Remove the Oracle database from Oracle
Clusterware on the
standby.
[oracle@ stbydb1]# srvctl remove database -db boston Remove the database boston? (y/[n]) y - Add the database back to the Clusterware on
the standby. The Oracle home must point to the new
version of the
home.
Example with single-instance Oracle Database: [oracle@stbydb1]$ srvctl add database -db boston -oraclehome /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_1 -dbtype SINGLE -instance boston1 -node stbydb1 -dbname chicago -diskgroup 'DATA,REDO,RECO' -role physical_standby -spfile '+DATA/BOSTON/PARAMETERFILE/spfileboston' -pwfile '+DATA/BOSTON/PASSWORDFILE/pwdboston' -startoption mount Example with Oracle RAC Database: [oracle@stbydb1]$ srvctl add database -db boston -oraclehome /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_1 -dbtype RAC -dbname chicago -diskgroup 'DATA,RECO,REDO' -role physical_standby -spfile '+DATA/BOSTON/PARAMETERFILE/spfileboston' -pwfile '+DATA/BOSTON/PASSWORDFILE/pwdboston' -startoption mount [oracle@stbydb1]$ srvctl add instance -database boston -instance boston1 -node stbydb1 [oracle@stbydb1]$ srvctl add instance -database boston -instance boston2 -node stbydb2 [oracle@stbydb1]$ srvctl start instance -db boston -instance boston1 -o mount [oracle@stbydb1]$ srvctl start instance -db boston -instance boston2 -o mount - Enable log shipping and validate Oracle
Data Guard
configuration.
[oracle@stbydb1]$ dgmgrl DGMGRL> connect sys/welcome1@chicago DGMGRL> edit database 'boston' SET PROPERTY 'LogShipping'='ON'; Property "LogShipping" updated DGMGRL> SHOW DATABASE 'boston' 'LogShipping'; LogShipping = 'ON' DGMGRL> show configuration verbose DGMGRL> show database verbose chicago DGMGRL> show database verbose boston DGMGRL> validate database chicago DGMGRL> validate database boston - Verify switchover and failover operations.
Switchover tests are as follows:
$ dgmgrl DGMGRL> connect sys/welcome1@boston DGMGRL> switchover to boston DGMGRL> connect sys/welcome1@chicago DGMGRL> switchover to chicago;Failover tests are as follows:
//Connect to standby before failover: $ dgmgrl DGMGRL> connect sys/welcome1@boston DGMGRL> failover to boston DGMGRL> reinstate database chicago //Connect to former primary before failover: DGMGRL> connect sys/welcome1@chicago DGMGRL> failover to chicago; DGMGRL> reinstate database boston //Health check: DGMGRL> show database verbose chicago DGMGRL> show database verbose boston DGMGRL> validate database chicago DGMGRL> validate database boston - Sync up the registry on the standby
system:
[root@ stbydb1~]# odacli list-databases ID DB Name DB Type DB Version CDB Class Shape Storage Status DbHomeID ---------------------------------------- ---------- -------- -------------------- ---------- --- e6450a56-5a7d-4dab-9ca9-25b004b66646 chicago Rac 18.11.0.0.200714 false Oltp Odb1 Asm Configured 755b4b5d-6211-4d94-81e8-cf611868fe39 Sync up registry entries: [root@ stbydb1~]# odacli update-registry -n db -u chicago Job details ---------------------------------------------------------------- ID: e4bbd7cc-6d0a-406d-8525-556f192b9f7a Description: Discover Components : db Status: Created Created: March 09, 2023 15:08:41 PM CET Message: [root@ stbydb1~]# odacli describe-job -i e4bbd7cc-6d0a-406d-8525-556f192b9f7a Job details ---------------------------------------------------------------- ID: e4bbd7cc-6d0a-406d-8525-556f192b9f7a Description: Discover Components : db Status: Success Created: March 9, 2023 3:08:41 PM CET Message: Task Name Node Name Start Time End Time Status ---------------------------------------- ------------------------- ----------------------------------- ---------------------------- Discover DBHome stdby1 March 9, 2023 3:08:41 PM CET March 9, 2023 3:08:43 PM CET Success Discover DB: chicago stdby1 March 9, 2023 3:08:43 PM CET March 9, 2023 3:08:55 PM CET SuccessConfirm the changes in the registry:[root@ stbydb1~]# odacli list-databases ID DB Name DB Type DB Version CDB Class Shape Storage Status bHomeID ---------------------------------------- ---------- -------- -------------------- ---- e6450a56-5a7d-4dab-9ca9-25b004b66646 chicago Rac 19.14.0.0.220118 false Oltp Odb1 Asm Configured 17f68bbf-b812-42e5-96ba-1433c30f75edThe total downtime requirement is the duration of the database upgrade. A switchover and switchback is not required for a database upgrade. Note that the update registry operation removes backup, dbdomain, CPU pools and associated network settings for all databases. Backup, CPU pools and associated network settings can be added again with the
odacli modify-databasecommand.
Patching Oracle Database
Patching databases on Oracle Database Appliance is an online operation. The following steps describe how to patch databases on a standby configuration. These steps apply to databases on bare metal systems and databases on DB systems.
Note that if the database uses, Oracle JVM, then you cannot patch the standby system first. Refer to My Oracle Support Note 2217053.1 - RAC Rolling Install Process for the "Oracle JavaVM Component Database PSU/RU" to confirm OJVM usage. In such a case, defer log shipping on the primary system and patch the primary system first.
Follow these steps:
- Verify that the system is operating correctly by running pre-checks, validating hardware and system processes, and verifying system configuration using ORAChk.
- Take a backup of the database.
- Stop log shipping on the
primary.
$ dgmgrl DGMGRL> connect sys/welcome1@chicago DGMGRL> edit database 'CHICAGO' SET STATE="LOG-TRANSPORT-OFF"; DGMGRL> SHOW DATABASE 'boston' 'LogShipping'; LogShipping = 'ON' DGMGRL> edit database 'boston' SET PROPERTY 'LogShipping'='OFF'; Property "LogShipping" updated DGMGRL> SHOW DATABASE 'boston' 'LogShipping'; LogShipping = 'OFF' - Stop the standby database and restart it in
read onlymode.[oracle@stbydb1]$ srvctl stop database -d boston [oracle@stbydb1]$ srvctl start database -db boston -o "read only" - Patch the standby database first. Identify the Oracle
home of the
database.
[root@ocboda10 ~]# odacli list-databases ID DB Name DB Type DB Version CDB Class Shape Storage Status DbHomeID ---------------------------------------- ---------- -------- -------------------- ---------- ----------------------------------- 667a0eec-910c-404b-9820-aedcddf668d7 chicago Rac 19.11.0.0.210420 false Oltp Odb1 Asm Configured 863c8cbe-1c5f-450e-866c-15c384580ad3 [oracle@stbydb1]# odacli list-dbhomes ID Name DB Version Home Location Status ---------------------------------------- -------------------- ---------------------------------------- --------------------------------- 863c8cbe-1c5f-450e-866c-15c384580ad3 OraDB19000_home1 19.11.0.0.210420 /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_1 ConfiguredRun pre-checks on the Oracle home.[oracle@stbydb1]# odacli update-dbhome -p -i 6d05e3f1-e948-4482-bcba-c560d9c8e5e5 -v 19.14.0.0 [oracle@stbydb1]# odacli describe-job -i b4ee24d9-2b82-4c80-b789-ced90013e4b3 Job details ---------------------------------------------------------------- ID: b4ee24d9-2b82-4c80-b789-ced90013e4b3 Description: DB Home Prechecks Status: Success Created: November 7, 2021 6:26:51 PM CETApply the patches.[oracle@stbydb1]# odacli update-dbhome -i 6d05e3f1-e948-4482-bcba-c560d9c8e5e5 -v 19.14.0.0 [oracle@stbydb1]# odacli describe-job -i "e3556125-7ce6-4560-9f22-3fdd9738f955" Job details ---------------------------------------------------------------- ID: e3556125-7ce6-4560-9f22-3fdd9738f955 Description: DB Home Patching: Home Id is e4e9fcbd-63d4-4c56-bb0c-b239a4e749f3 Status: Success Created: November 7, 2021 7:09:52 PM CETVerify the results.[oracle@stbydb1]# odacli list-dbhomes ID Name DB Version Home Location Status ---------------------------------------- -------------------- ----------------------------------------- e4e9fcbd-63d4-4c56-bb0c-b239a4e749f3 OraDB19000_home2 19.14.0.0.220118 /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_2 Configured 863c8cbe-1c5f-450e-866c-15c384580ad3 OraDB19000_home1 19.11.0.0.210420 /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_1 Configured [oracle@stbydb1]# odacli list-databases ID DB Name DB Type DB Version CDB Class Shape Storage Status DbHomeID ---------------------------------------- ---------- -------- -------------------- ---------- -------- -------- ---------- ------------ 667a0eec-910c-404b-9820-aedcddf668d7 chicago Rac 19.14.0.0.220118 false Oltp Odb1 Asm Configured e4e9fcbd-63d4-4c56-bb0c-b239a4e749f3 - Patch the primary database, similar to the steps for patching the standby database.
- Start log shipping on the primary and verify Oracle Data
Guard
configuration.
DGMGRL> connect sys/welcome1@chicago DGMGRL> edit database 'boston' SET PROPERTY 'LogShipping'='ON'; Property "LogShipping" updated DGMGRL> SHOW DATABASE 'boston' 'LogShipping'; LogShipping = 'ON' DGMGRL> show configuration verbose DGMGRL> show database verbose chicago DGMGRL> show database verbose boston DGMGRL> validate database chicago DGMGRL> validate database boston