6 Manual Multi-Node Patching of Grid Infrastructure and RAC DB Environment Using OPatchAuto
Patch orchestration is the automated execution of the patching steps, such as the execution of pre-patch checks, stopping services, applying the binary patches, and starting the services. Patch orchestration for Oracle Database 12c applies the patch to the GI/RAC configuration on that machine, including all of its databases. The OPatchAuto patch orchestration utility has been made available with version 12.1 of the OPatch utility.
This chapter covers the following topics:
Important:
This chapter applies to Oracle Database 12c only.6.1 Running OPatchAuto on a Single Node
Prerequisites: Node Availaibility During Patching (Rolling vs. Non-rolling)
In order to start a new patching session, the following conditions must be met.
Note:
The following conditions apply only for the first node, such as when the session is first started on the cluster.-
Local node must be up for both Rolling and Non-rolling modes.
-
At least one of the remote nodes must be up in order to start a Rolling mode session.
-
ALL the remote nodes must be down in order to start a Non-rolling session.
To ensure successful patching, Oracle recommends that a patching session always consist of the four following steps:
-
Run
opatch lsinventoryWhen to run: During cluster uptime.
Before you begin a new patching session you must ensure that all nodes of the cluster are at the same patch level.
$GRID_HOME/OPatch/opatch lsinventory
Example Output
Patch level status of Cluster nodes : Patching Level Nodes -------------- ----- 12345678 node123, node456
-
Run
opatchauto apply -analyzeWhen to run: During cluster uptime.
The apply -analyze command simulates an opatchauto apply session by running all prerequisite checks, when possible, without making changes to the system (either bits or configurations).
-
Run
opatchauto applyWhen to run: During cluster downtime.
The apply command applies all patches within a specified system patch to the GI or RAC home from which the
opatchautocommand is run. -
Run
opatch lsinventoryWhen to run: During cluster uptime.
After successfully running
opatchauto applyon each individual node, you must runopatch lsinventoryon that node to confirm that the patch level has been correctly updated.
6.2 Patching Session Output
The following patching session output examples illustrate successful OPatchAuto apply and rollback sessions. Note: This session applies to both single and multi-node environments.
Example 6-1 OPatchAuto Apply/Rollback Session in Analyze Mode
--------------------------------Summary-------------------------------- Analysis for applying patches has completed successfully: Host:myhostq CRS Home:/scratch/aime_ordb_myhostq/crso1/crshome_crso1 ==Following patches were SUCCESSFULLY analyzed to be applied: Patch: /tmp/patch_gipsu_12024/patch/22191349/21436941 Log: /scratch/aime_ordb_myhostq/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-03-08_23-21-21PM_1.log Patch: /tmp/patch_gipsu_12024/patch/22191349/21948341 Log: /scratch/aime_ordb_myhostq/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-03-08_23-21-21PM_1.log Patch: /tmp/patch_gipsu_12024/patch/22191349/21948344 Log: /scratch/aime_ordb_myhostq/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-03-08_23-21-21PM_1.log Patch: /tmp/patch_gipsu_12024/patch/22191349/21948354 Log: /scratch/aime_ordb_myhostq/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-03-08_23-21-21PM_1.log Host:myhostr CRS Home:/scratch/aime_ordb_myhostq/crso1/crshome_crso1 ==Following patches were SUCCESSFULLY analyzed to be applied: Patch: /scratch/aime_ordb_myhostq/crso1/crshome_crso1/OPatch/auto/dbtmp/22191349/21436941 Log: /scratch/aime_ordb_myhostq/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-03-08_23-24-56PM_1.log Patch: /scratch/aime_ordb_myhostq/crso1/crshome_crso1/OPatch/auto/dbtmp/22191349/21948341 Log: /scratch/aime_ordb_myhostq/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-03-08_23-24-56PM_1.log Patch: /scratch/aime_ordb_myhostq/crso1/crshome_crso1/OPatch/auto/dbtmp/22191349/21948344 Log: /scratch/aime_ordb_myhostq/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-03-08_23-24-56PM_1.log Patch: /scratch/aime_ordb_myhostq/crso1/crshome_crso1/OPatch/auto/dbtmp/22191349/21948354 Log: /scratch/aime_ordb_myhostq/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-03-08_23-24-56PM_1.log OPatchAuto successful.
Example 6-2 OPatchAuto Apply Session
--------------------------------Summary-------------------------------- Patching is completed successfully. Please find the summary as follows: Host:mymachineemq CRS Home:/scratch/aime_ordb_mymachineemq/crso1/crshome_crso1 Summary: ==Following patches were SUCCESSFULLY applied: Patch: /tmp/patch_gipsu_12024/patch/22191349/21436941 Log: /scratch/aime_ordb_mymachineemq/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-03-08_23-41-38PM_1.log Patch: /tmp/patch_gipsu_12024/patch/22191349/21948341 Log: /scratch/aime_ordb_mymachineemq/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-03-08_23-41-38PM_1.log Patch: /tmp/patch_gipsu_12024/patch/22191349/21948344 Log: /scratch/aime_ordb_mymachineemq/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-03-08_23-41-38PM_1.log Patch: /tmp/patch_gipsu_12024/patch/22191349/21948354 Log: /scratch/aime_ordb_mymachineemq/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-03-08_23-41-38PM_1.log Host:mymachineemr CRS Home:/scratch/aime_ordb_mymachineemq/crso1/crshome_crso1 Summary: ==Following patches were SUCCESSFULLY applied: Patch: /scratch/aime_ordb_mymachineemq/crso1/crshome_crso1/OPatch/auto/dbtmp/22191349/21436941 Log: /scratch/aime_ordb_mymachineemq/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-03-08_23-59-15PM_1.log Patch: /scratch/aime_ordb_mymachineemq/crso1/crshome_crso1/OPatch/auto/dbtmp/22191349/21948341 Log: /scratch/aime_ordb_mymachineemq/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-03-08_23-59-15PM_1.log Patch: /scratch/aime_ordb_mymachineemq/crso1/crshome_crso1/OPatch/auto/dbtmp/22191349/21948344 Log: /scratch/aime_ordb_mymachineemq/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-03-08_23-59-15PM_1.log Patch: /scratch/aime_ordb_mymachineemq/crso1/crshome_crso1/OPatch/auto/dbtmp/22191349/21948354 Log: /scratch/aime_ordb_mymachineemq/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-03-08_23-59-15PM_1.log OPatchAuto successful.
Example 6-3 OPatchAuto Rollback Session
--------------------------------Summary-------------------------------- Patching is completed successfully. Please find the summary as follows: Host:mymachineemm CRS Home:/scratch/aime_ordb_mymachineemm/crso1/crshome_crso1 Summary: ==Following patches were SUCCESSFULLY rolled back: Patch: /tmp/patch_gipsu_12019/patch/22191492/17077442 Log: /scratch/aime_ordb_mymachineemm/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-03-08_19-25-46PM_1.log Patch: /tmp/patch_gipsu_12019/patch/22191492/17303297 Log: /scratch/aime_ordb_mymachineemm/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-03-08_19-25-46PM_1.log Patch: /tmp/patch_gipsu_12019/patch/22191492/21951844 Log: /scratch/aime_ordb_mymachineemm/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-03-08_19-25-46PM_1.log OPatchAuto successful.
Example 6-4 OPatchAuto Apply/Rollback Failure
---------------------------Patching Failed--------------------------------- Command execution failed during patching in home: /scratch/aime/app/aime/product/11.2.0/dbhome_2, host: mymachineemg. Command failed: /bin/sh -c 'ORACLE_HOME=/scratch/aime/app/aime/product/11.2.0/dbhome_2 /scratch/aime/app/aime/product/11.2.0/dbhome_2/bin/srvctl stop home -o /scratch/aime/app/aime/product/11.2.0/dbhome_2 -n mymachineemg -f -t TRANSACTIONAL -s /scratch/aime/app/aime/product/11.2.0/dbhome_2/cfgtoollogs/opatchautodb/statfile/mymachineemg/OracleHome-eca39d53-5b51-4cdf-9c79-ce9d9312d86a_mymachineemg.stat' Command failure output: PRCH-1000 : Failed to stop resources running from Oracle home /scratch/aime/app/aime/product/11.2.0/dbhome_2 PRCH-1029 : One or more resources failed to stop: PRCH-1006 : Failed to stop Listener PRCR-1014 : Failed to stop resource ora.LISTENER2.lsnr PRCR-1065 : Failed to stop resource ora.LISTENER2.lsnr CRS-5016: Process "/scratch/aime/app/aime/product/11.2.0/dbhome_2/bin/lsnrctl" spawned by agent "/scratch/aime_ordb_mymachineemg/crso1/crshome_crso1/bin/oraagent.bin" for action "stop" failed: details at "(:CLSN00010:)" in "/scratch/aime_ordb_mymachineemg/crso1/crshome_crso1/log/mymachineemg/agent/crsd/oraagent_aime/oraagent_aime.log" CRS-2675: Stop of 'ora.LISTENER2.lsnr' on 'mymachineemg' failed After fixing the cause of failure Run opatchauto resume with session id "J5A3" ] OPATCHAUTO-68061: The orchestration engine failed. OPATCHAUTO-68061: The orchestration engine failed with return code 1 OPATCHAUTO-68061: Check the log for more details.
6.3 Sample Console Output
Example 6-5 opatchauto apply -analyze
System initialization log file is /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1/cfgtoollogs/opatchautodb/systemconfig2016-05-05_01-55-58PM.log. Session log file is /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1/cfgtoollogs/opatchauto/opatchauto2016-05-05_01-56-18PM.log WARNING: the option -ocmrf is deprecated and no longer needed. OPatch no longer checks for OCM configuration. It will be removed in a future release. The id for this session is MDAN [init:init] Executing OPatchAutoBinaryAction action on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 Executing OPatch prereq operations to verify patch applicability on CRS Home........ [init:init] OPatchAutoBinaryAction action completed on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 successfully [init:init] Executing GIRACPrereqAction action on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 Executing prereq operations before applying on CRS Home........ [init:init] GIRACPrereqAction action completed on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 successfully OPatchAuto successful. --------------------------------Summary-------------------------------- Analysis for applying patches has completed successfully: Host:mymachineelu CRS Home:/scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 ==Following patches were SUCCESSFULLY analyzed to be applied: Patch: /tmp/patch_gipsu_12019/patch/22654153/17077442 Log: /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_13-56-25PM_1.log Patch: /tmp/patch_gipsu_12019/patch/22654153/17303297 Log: /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_13-56-25PM_1.log Patch: /tmp/patch_gipsu_12019/patch/22654153/22291141 Log: /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_13-56-25PM_1.log
System initialization log file is /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1/cfgtoollogs/opatchautodb/systemconfig2016-05-05_02-22-02PM.log. Session log file is /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1/cfgtoollogs/opatchauto/opatchauto2016-05-05_02-22-19PM.log WARNING: the option -ocmrf is deprecated and no longer needed. OPatch no longer checks for OCM configuration. It will be removed in a future release. The id for this session is WLR9 [init:init] Executing OPatchAutoBinaryAction action on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 Executing OPatch prereq operations to verify patch applicability on CRS Home........ [init:init] OPatchAutoBinaryAction action completed on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 successfully [init:init] Executing GIRACPrereqAction action on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 Executing prereq operations before applying on CRS Home........ [init:init] GIRACPrereqAction action completed on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 successfully [shutdown:shutdown] Executing GIShutDownAction action on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 Performing prepatch operations on CRS Home........ Prepatch operation log file location: /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1/cfgtoollogs/crsconfig/crspatch_mymachineelu_2016-05-05_02-22-52PM.log [shutdown:shutdown] GIShutDownAction action completed on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 successfully [offline:binary-patching] Executing OPatchAutoBinaryAction action on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 Start applying binary patches on CRS Home........ [offline:binary-patching] OPatchAutoBinaryAction action completed on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 successfully [startup:startup] Executing GIStartupAction action on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 Performing postpatch operations on CRS Home........ Postpatch operation log file location: /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1/cfgtoollogs/crsconfig/crspatch_mymachineelu_2016-05-05_02-27-03PM.log [startup:startup] GIStartupAction action completed on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 successfully [finalize:finalize] Executing OracleHomeLSInventoryGrepAction action on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 Verifying patches applied on CRS Home. [finalize:finalize] OracleHomeLSInventoryGrepAction action completed on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 successfully OPatchAuto successful. --------------------------------Summary-------------------------------- Patching is completed successfully. Please find the summary as follows: Host:mymachineelu CRS Home:/scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 Summary: ==Following patches were SUCCESSFULLY applied: Patch: /tmp/patch_gipsu_12019/patch/22654153/17077442 Log: /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_14-23-38PM_1.log Patch: /tmp/patch_gipsu_12019/patch/22654153/17303297 Log: /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_14-23-38PM_1.log Patch: /tmp/patch_gipsu_12019/patch/22654153/22291141 Log: /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_14-23-38PM_1.log
Example 6-7 opatchauto rollback
System initialization log file is /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1/cfgtoollogs/opatchautodb/systemconfig2016-05-05_04-34-39PM.log. Session log file is /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1/cfgtoollogs/opatchauto/opatchauto2016-05-05_04-35-00PM.log The id for this session is K5BA [init:init] Executing OPatchAutoBinaryAction action on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 Executing OPatch prereq operations to verify patch applicability on CRS Home........ [init:init] OPatchAutoBinaryAction action completed on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 successfully [init:init] Executing GIRACPrereqAction action on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 Executing prereq operations before rolling back on CRS Home........ [init:init] GIRACPrereqAction action completed on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 successfully [shutdown:shutdown] Executing GIShutDownAction action on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 Performing prepatch operations on CRS Home........ Prepatch operation log file location: /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1/cfgtoollogs/crsconfig/crspatch_mymachineelu_2016-05-05_04-35-22PM.log [shutdown:shutdown] GIShutDownAction action completed on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 successfull [offline:binary-patching] Executing OPatchAutoBinaryAction action on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 Start rolling back binary patches on CRS Home........ [offline:binary-patching] OPatchAutoBinaryAction action completed on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 successfully [startup:startup] Executing GIStartupAction action on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 Performing postpatch operations on CRS Home........ Postpatch operation log file location: /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1/cfgtoollogs/crsconfig/crspatch_mymachineelu_2016-05-05_04-38-59PM.log [startup:startup] GIStartupAction action completed on home /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 successfully OPatchAuto successful. --------------------------------Summary-------------------------------- Patching is completed successfully. Please find the summary as follows: Host:mymachineelu CRS Home:/scratch/aime_ordb_mymachineelu/crso1/crshome_crso1 Summary: ==Following patches were SUCCESSFULLY rolled back: Patch: /tmp/patch_gipsu_12019/patch/22654153/17077442 Log: /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_16-36-04PM_1.log Patch: /tmp/patch_gipsu_12019/patch/22654153/17303297 Log: /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_16-36-04PM_1.log Patch: /tmp/patch_gipsu_12019/patch/22654153/22291141 Log: /scratch/aime_ordb_mymachineelu/crso1/crshome_crso1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_16-36-04PM_1.log
Example 6-8 opatchauto apply -analyze (Multi-node)
System initialization log file is /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1/cfgtoollogs/opatchautodb/systemconfig2016-05-05_10-19-07PM.log. Session log file is /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1/cfgtoollogs/opatchauto/opatchauto2016-05-05_10-19-19PM.log The id for this session is KMKT [init:init] Executing OPatchAutoBinaryAction action on home /scratch/aime_ordb_mymachineelg/obase/rachome_racr1 [init:init] Executing OPatchAutoBinaryAction action on home /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1 Executing OPatch prereq operations to verify patch applicability on RAC Home........ Executing OPatch prereq operations to verify patch applicability on CRS Home........ [init:init] OPatchAutoBinaryAction action completed on home /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1 successfully <mymachineelh> : [init:init] Executing OPatchAutoBinaryAction action on home /scratch/aime_ordb_mymachineelg/obase/rachome_racr1 <mymachineelh> : [init:init] Executing OPatchAutoBinaryAction action on home /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1 <mymachineelh> : Executing OPatch prereq operations to verify patch applicability on CRS Home........ <mymachineelh> : Executing OPatch prereq operations to verify patch applicability on RAC Home........ <mymachineelh> : [init:init] OPatchAutoBinaryAction action completed on home /scratch/aime_ordb_mymachineelg/obase/rachome_racr1 successfully [init:init] OPatchAutoBinaryAction action completed on home /scratch/aime_ordb_mymachineelg/obase/rachome_racr1 successfully <mymachineelh> : [init:init] OPatchAutoBinaryAction action completed on home /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1 successfully [init:init] Executing GIRACPrereqAction action on home /scratch/aime_ordb_mymachineelg/obase/rachome_racr1 [init:init] Executing GIRACPrereqAction action on home /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1 Executing prereq operations before applying on CRS Home........ Executing prereq operations before applying on RAC Home........ [init:init] GIRACPrereqAction action completed on home /scratch/aime_ordb_mymachineelg/obase/rachome_racr1 successfully [init:init] GIRACPrereqAction action completed on home /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1 successfully <mymachineelh> : [init:init] Executing GIRACPrereqAction action on home /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1 <mymachineelh> : [init:init] Executing GIRACPrereqAction action on home /scratch/aime_ordb_mymachineelg/obase/rachome_racr1 <mymachineelh> : Executing prereq operations before applying on RAC Home........ <mymachineelh> : Executing prereq operations before applying on CRS Home........ <mymachineelh> : [init:init] GIRACPrereqAction action completed on home /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1 successfully <mymachineelh> : [init:init] GIRACPrereqAction action completed on home /scratch/aime_ordb_mymachineelg/obase/rachome_racr1 successfully [init:init] Executing RACDBPrereqAction action on home /scratch/aime_ordb_mymachineelg/obase/rachome_racr1 Executing prereq operations before applying on RAC Home........ <mymachineelh> : [init:init] Executing RACDBPrereqAction action on home /scratch/aime_ordb_mymachineelg/obase/rachome_racr1 <mymachineelh> : Executing prereq operations before applying on RAC Home........ <mymachineelh> : Following step failed during analysis: /bin/sh -c 'ORACLE_HOME=/scratch/aime_ordb_mymachineelg/obase/rachome_racr1 ORACLE_SID=racr12 /scratch/aime_ordb_mymachineelg/obase/rachome_racr1/OPatch/datapatch -prereq' <mymachineelh> : [init:init] RACDBPrereqAction action completed on home /scratch/aime_ordb_mymachineelg/obase/rachome_racr1 successfully [init:init] RACDBPrereqAction action completed on home /scratch/aime_ordb_mymachineelg/obase/rachome_racr1 successfully [finalize:finalize] Executing DeletePatchAction action on home /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1 [finalize:finalize] DeletePatchAction action completed on home /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1 successfully [finalize:finalize] Executing DeletePatchAction action on home /scratch/aime_ordb_mymachineelg/obase/rachome_racr1 [finalize:finalize] DeletePatchAction action completed on home /scratch/aime_ordb_mymachineelg/obase/rachome_racr1 successfully <mymachineelh> : [finalize:finalize] Executing DeletePatchAction action on home /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1 <mymachineelh> : [finalize:finalize] DeletePatchAction action completed on home /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1 successfully <mymachineelh> : [finalize:finalize] Executing DeletePatchAction action on home /scratch/aime_ordb_mymachineelg/obase/rachome_racr1 <mymachineelh> : [finalize:finalize] DeletePatchAction action completed on home /scratch/aime_ordb_mymachineelg/obase/rachome_racr1 successfully OPatchAuto successful. --------------------------------Summary-------------------------------- Analysis for applying patches has completed successfully: Host:mymachineelg CRS Home:/scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1 ==Following patches were SUCCESSFULLY analyzed to be applied: Patch: /tmp/patch_gipsu_12024/patch/22646084/21436941 Log: /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_22-24-37PM_1.log Patch: /tmp/patch_gipsu_12024/patch/22646084/22291127 Log: /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_22-24-37PM_1.log Patch: /tmp/patch_gipsu_12024/patch/22646084/22502518 Log: /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_22-24-37PM_1.log Patch: /tmp/patch_gipsu_12024/patch/22646084/22502555 Log: /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_22-24-37PM_1.log Host:mymachineelg RAC Home:/scratch/aime_ordb_mymachineelg/obase/rachome_racr1 ==Following patches were SKIPPED: Patch: /tmp/patch_gipsu_12024/patch/22646084/21436941 Reason: This patch is not applicable to this specified target type - "rac_database" Patch: /tmp/patch_gipsu_12024/patch/22646084/22502518 Reason: This patch is not applicable to this specified target type - "rac_database" ==Following patches were SUCCESSFULLY analyzed to be applied: Patch: /tmp/patch_gipsu_12024/patch/22646084/22291127 Log: /scratch/aime_ordb_mymachineelg/obase/rachome_racr1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_22-24-39PM_1.log Patch: /tmp/patch_gipsu_12024/patch/22646084/22502555 Log: /scratch/aime_ordb_mymachineelg/obase/rachome_racr1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_22-24-39PM_1.log Host:mymachineelh RAC Home:/scratch/aime_ordb_mymachineelg/obase/rachome_racr1 ==Following patches were SKIPPED: Patch: /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1/OPatch/auto/dbtmp/22646084/21436941 Reason: This patch is not applicable to this specified target type - "rac_database" Patch: /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1/OPatch/auto/dbtmp/22646084/22502518 Reason: This patch is not applicable to this specified target type - "rac_database" ==Following patches were SUCCESSFULLY analyzed to be applied: Patch: /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1/OPatch/auto/dbtmp/22646084/22291127 Log: /scratch/aime_ordb_mymachineelg/obase/rachome_racr1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_22-25-55PM_1.log Patch: /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1/OPatch/auto/dbtmp/22646084/22502555 Log: /scratch/aime_ordb_mymachineelg/obase/rachome_racr1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_22-25-55PM_1.log Host:mymachineelh CRS Home:/scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1 ==Following patches were SUCCESSFULLY analyzed to be applied: Patch: /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1/OPatch/auto/dbtmp/22646084/21436941 Log: /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_22-28-24PM_1.log Patch: /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1/OPatch/auto/dbtmp/22646084/22291127 Log: /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_22-28-24PM_1.log Patch: /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1/OPatch/auto/dbtmp/22646084/22502518 Log: /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_22-28-24PM_1.log Patch: /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1/OPatch/auto/dbtmp/22646084/22502555 Log: /scratch/aime_ordb_mymachineelg/crsr1/crshome_crsr1/cfgtoollogs/opatchauto/core/opatch/opatch2016-05-05_22-28-24PM_1.log
6.4 OPatchAuto Apply
When you run OPatchAuto's apply command, numerous operations are performed to implement the complete patch application cycle. These operations vary depending on the environment to be patched. The following environment is representative of the vast majority of patching environments in which OPatchAuto is used. For example, a typical patching environment would be one GI Home managing two RAC Homes. When you run opatchauto apply, OPatchAuto will perform the operations shown in Figure 6-1, "Patching with OPatchAuto: Process Flow".
6.4.1 OPatchAuto: System Reboot Request
Depending on the patch or the home directory configuration, you may encounter a request to reboot the system. After a reboot during the patching process, you need to invoke the opatchauto utility again so that it seamlessly continues with rest of the patch application process.
Typically an error message, as shown in the following example, will be displayed when a problem arises.
Example 6-9 OPatchAuto Console Error
# OPatch/opatchauto apply /scratch/aime/sh/RDBMS_12.1.0.1.0_LINUX.X64_130418/patches/v2/nosql/gipsu/11111111 -ocmrf /tmp/ocm.rsp OPatch Automation Tool Copyright (c) 2013, Oracle Corporation. All rights reserved. ... CLSRSC-400: A system reboot is required to continue installing. ... Apply Summary: Following patch(es) are successfully installed: GI_HOME=/u01/GI12/app/12.1.0/grid:13852018, 22222222, 123456788 DB_HOME=/scratch/aime/DB12_2/app/aime/product/12.1.0/dbhome_1:13852018, 123456788 DB_HOME=/scratch/aime1/DB12N/app/aime1/product/12.1.0/dbhome_1:13852018, 123456788 opatchauto failed with error code 1.
When you receive an error like this, follow the reboot instructions specified in the console. The following example shows a system reboot request issued by the user.
6.5 Example: Manual OPatchAuto Patching Steps
The following example shows the commands OPatchAuto runs during a normal patching sessions. The environment being patched consists of a single GI Home (grid) and a single RAC Home (dbhome_1).
-
As the 'patchadmin' user on the host 'myhost', run the following command:
[patchadmin@myhost]$ /scratch/patchadmin/product/12.1.0/dbhome_1/OPatch/opatchauto apply /scratch/patchadmin/patch/19954978/ -oh /scratch/patchadmin/product/12.1.0/dbhome_1 -target_type oracle_database -binary -persistresult /scratch/patchadmin/product/12.1.0/dbhome_1/OPatch/auto/dbsessioninfo/sessionresult_analyze_myhost_orcl.ser -analyze -online
-
Run OPatch Inventory Check for Database Home. To check the consistency of inventory information, run the OPatch lsinventory command and look for "OPatch succeeded" message at the end.
As the 'patchadmin' user on the host 'myhost' run the following command:
[patchadmin@myhost]$ /scratch/patchadmin/product/12.1.0/dbhome_1/OPatch/opatch lsinventory -invPtrLoc /scratch/patchadmin/product/12.1.0/grid/oraInst.loc -oh /scratch/patchadmin/product/12.1.0/dbhome_1
-
SQL patch analyze.
As the 'patchadmin' user on the host 'myhost' run the following command:
[patchadmin@myhost]$ ORACLE_HOME=/scratch/patchadmin/product/12.1.0/dbhome_1 ORACLE_SID=orcl /scratch/patchadmin/product/12.1.0/dbhome_1/OPatch/datapatch -prereq
-
As the 'patchadmin' user on the host 'myhost' run the following command:
[patchadmin@myhost]$ /scratch/patchadmin/product/12.1.0/grid/OPatch/opatchauto apply /scratch/patchadmin/patch/19954978/ -oh /scratch/patchadmin/product/12.1.0/grid -target_type has -binary -persistresult /scratch/patchadmin/product/12.1.0/grid/OPatch/auto/dbsessioninfo/sessionresult_analyze_myhost_siha.ser -analyze -online
-
Run OPatch Inventory Check for SIHA Home.
To check the consistency of inventory information, run the OPatch lsinventory command and look for "OPatch succeeded" message at the end.
As the 'patchadmin' user on the host 'myhost' run the following command:
[patchadmin@myhost]$ /scratch/patchadmin/product/12.1.0/grid/OPatch/opatch lsinventory -invPtrLoc /scratch/patchadmin/product/12.1.0/grid/oraInst.loc -oh /scratch/patchadmin/product/12.1.0/grid
-
Save the PDB state before stopping database.
As the 'patchadmin' user on the host 'myhost' run the following commands:
[patchadmin@myhost]$ echo "DECLARE l_count NUMBER(4) :=0; BEGIN SELECT count(*) INTO l_count FROM CDB_PDBS; IF l_count >1 then EXECUTE IMMEDIATE 'alter pluggable database ALL SAVE STATE'; END IF; commit; END;" > /tmp/OraDB12Home1_patchadmin_orcl.sql [patchadmin@myhost]$ echo "/" >> /tmp/OraDB12Home1_patchadmin_orcl.sql [patchadmin@myhost]$ echo "EXIT;" >> /tmp/OraDB12Home1_patchadmin_orcl.sql [patchadmin@myhost]$ cd /scratch/patchadmin/product/12.1.0/dbhome_1 ; ORACLE_HOME=/scratch/patchadmin/product/12.1.0/dbhome_1 ORACLE_SID=orcl /scratch/patchadmin/product/12.1.0/dbhome_1/bin/sqlplus / as sysdba @/tmp/OraDB12Home1_patchadmin_orcl.sql
-
Stop the resources running from Database Home.
On myhost stop the resources running out of the Database Home
As the 'patchadmin' user on the host 'myhost' run the following commands:
[patchadmin@myhost]$ mkdir -p /scratch/patchadmin/product/12.1.0/dbhome_1/cfgtoollogs/opatchautodb/statfile/myhost [patchadmin@myhost]$ ORACLE_HOME=/scratch/patchadmin/product/12.1.0/dbhome_1 /scratch/patchadmin/product/12.1.0/dbhome_1/bin/srvctl stop home -o /scratch/patchadmin/product/12.1.0/dbhome_1 -f -s /scratch/patchadmin/product/12.1.0/dbhome_1/cfgtoollogs/opatchautodb/statfile/myhost/OracleHome-045366ab-6fd1-4ed2-8f3f-72870a959a3a_myhost.stat
-
Run rootcrs.pl -prepatch
As a 'root' user, run rootcrs.pl -prepatch
As the 'root' user on the host 'myhost' run the following command:
[root@myhost]# /scratch/patchadmin/product/12.1.0/grid/perl/bin/perl -I/scratch/patchadmin/product/12.1.0/grid/perl/lib -I/scratch/patchadmin/product/12.1.0/grid/OPatch/auto/dbtmp/bootstrap_myhost/patchwork/crs/install /scratch/patchadmin/product/12.1.0/grid/OPatch/auto/dbtmp/bootstrap_myhost/patchwork/crs/install/roothas.pl -prepatch
-
As the 'patchadmin' user on the host 'myhost' run the following command:
[patchadmin@myhost]$ /scratch/patchadmin/product/12.1.0/dbhome_1/OPatch/opatchauto apply /scratch/patchadmin/patch/19954978/ -oh /scratch/patchadmin/product/12.1.0/dbhome_1 -target_type oracle_database -binary -persistresult /scratch/patchadmin/product/12.1.0/dbhome_1/OPatch/auto/dbsessioninfo/sessionresult_myhostorcl.ser -analyzedresult /scratch/patchadmin/product/12.1.0/dbhome_1/OPatch/auto/dbsessioninfo/sessionresult_analyze_myhost_orcl.ser
-
As the 'patchadmin' user on the host 'myhost' run the following command:
[patchadmin@myhost]$ /scratch/patchadmin/product/12.1.0/grid/OPatch/opatchauto apply /scratch/patchadmin/patch/19954978/ -oh /scratch/patchadmin/product/12.1.0/grid -target_type has -binary -persistresult /scratch/patchadmin/product/12.1.0/grid/OPatch/auto/dbsessioninfo/sessionresult_myhostsiha.ser -analyzedresult /scratch/patchadmin/product/12.1.0/grid/OPatch/auto/dbsessioninfo/sessionresult_analyze_myhost_siha.ser
-
Run rootadd_rdbms.sh
As a 'root' user, run rootadd_rdbms.sh
As the 'root' user on the host 'myhost' run the following command:
[root@myhost]# /scratch/patchadmin/product/12.1.0/grid/rdbms/install/rootadd_rdbms.sh
-
Run rootcrs.pl -postpatch -rollback
As a 'root' user, run rootcrs.pl -postpatch -rollback
Important:
If the command fails with following error messages, reboot the host and re-try the same step.CLSRSC-400: A system reboot is required to continue installing.As the 'root' user on the host 'myhost' run the following command:
[root@myhost]# /scratch/patchadmin/product/12.1.0/grid/perl/bin/perl -I/scratch/patchadmin/product/12.1.0/grid/perl/lib -I/scratch/patchadmin/product/12.1.0/grid/OPatch/auto/dbtmp/bootstrap_myhost/patchwork/crs/install /scratch/patchadmin/product/12.1.0/grid/OPatch/auto/dbtmp/bootstrap_myhost/patchwork/crs/install/roothas.pl -postpatch
-
Start the resources running from Database Home.
On myhost start the resources running out of the Database Home
As the 'patchadmin' user on the host 'myhost' run the following commands:
[patchadmin@myhost]$ ORACLE_HOME=/scratch/patchadmin/product/12.1.0/dbhome_1 /scratch/patchadmin/product/12.1.0/dbhome_1/bin/srvctl start home -o /scratch/patchadmin/product/12.1.0/dbhome_1 -s /scratch/patchadmin/product/12.1.0/dbhome_1/cfgtoollogs/opatchautodb/statfile/myhost/OracleHome-045366ab-6fd1-4ed2-8f3f-72870a959a3a_myhost.stat [patchadmin@myhost]$ rm -rf /scratch/patchadmin/product/12.1.0/dbhome_1/cfgtoollogs/opatchautodb/statfile/myhost
-
Apply SQL
As the 'patchadmin' user on the host 'myhost' run the following command:
[patchadmin@myhost]$ ORACLE_HOME=/scratch/patchadmin/product/12.1.0/dbhome_1 ORACLE_SID=orcl /scratch/patchadmin/product/12.1.0/dbhome_1/OPatch/datapatch