Patching Procedures

Concepts

Applying Oracle E-Business Suite patches without a significant system downtime is referred to as online patching, and a new utility, adop, is used to apply patches.

Online patching is supported by the capability of storing multiple application editions in the database, and the provision of a dual application tier file system. At any given point in time, one of these file systems is designated as run (part of the running system) and the other as patch (either being patched or awaiting the start of the next patching cycle). Whichever is the current run file system appears to the user in exactly the same way as the single application tier file system did in Oracle E-Business Suite releases prior to 12.2.

The existence of the dual file system has implications for patches that change the system configuration. The adop utility is required for applying software patches to the patch file system, but is not required to perform configuration changes. Depending on the specific situation, configuration changes can be made to either the run file system or the patch file system: automatic synchronization subsequently takes place in both cases. The relevant principles are described further in the What's Next section of this chapter, under the heading Configuration Management and Online Patching.

There are also implications for general (non-patching) maintenance activities. For information on choosing the appropriate file system to run AD tools from, refer to: Choosing the Correct File System For Maintenance Tasks in Chapter 7 of this book.

A new environment variable, $FILE_EDITION, shows the current designation of a given dual file system member. Three other new environment variables designate the root directories of the run ($RUN_BASE), patch ($PATCH_BASE), and non-editioned ($NE_BASE) file systems.

For example:

When a patch is being applied, the Oracle E-Business Suite system is running in normal production mode (full functionality, with some documented exceptions) in the run edition of the file system and database. Full application functionality is retained as patch execution proceeds, until the cutover phase is reached (as described later in this section).

Note: For more information about how online patching works, refer to Chapter 4, Patching and Managment Tools, of Oracle E-Business Suite Concepts.

Overview

It is more appropriate to think in terms of a patching cycle than a single patching operation. The online patching cycle consists of a number of phases:

  1. Prepare

  2. Apply

  3. Finalize

  4. Cutover

  5. Cleanup

You specify the desired phase or phases as arguments to the adop utility. The actions taken in these phases are described in Oracle E-Business Suite Concepts. This and the preceding chapter of this book provide details of the available options.

Note: The adop utility sets its own environment. There is therefore no need to source the environment before running it.

An online patching consists of several phases, which are specified on the adop command line as follows:

adop phase=<phase_name>

Prepare phase - Used to start a new online patching cycle:

$ adop phase=prepare

Apply phase - Used to apply one or more patches to the patch edition of an Oracle E-Business Suite system:

$ adop phase=apply patches=123456,789101 workers=8

Finalize phase - Used to perform the final patching operations that can be executed while the application is still online:

$ adop phase=finalize

Cutover phase - Used to perform the transition to the patched environment:

$ adop phase=cutover

Cleanup phase - Used to remove old objects that are no longer needed:

$ adop phase=cleanup

The adop phases are described in more detail in The Online Patching Cycle section of this chapter.

Abort command

If necessary, an online patching cycle can be terminated, with the actions taken being discarded.

The command to perform this operation is:

$ adop phase=abort

This abort command is only available up to (but not including) the cutover phase. After cutover, the system is running on the new edition, and abort is no longer possible for that patching cycle.

Before You Start

Before you can use online patching, you must perform some initial one-off setup steps:

Each of these requirements is described below.

Check Inventory Setup

Summary

Using the Global Inventory

If you are using a UNIX platform, you should verify the existence and contents of the oraInst.loc file, which specifies the location of the oraInventory.loc file global inventory file.

  1. Check that oraInst.loc exists in the correct directory for your platform:

    Platforms and Corresponding oraInst.loc Locations
    Platform oraInst.loc Location
    Oracle Solaris SPARC (64-bit) /var/opt/oracle
    Linux x86-64 /etc
    IBM AIX on Power Systems (64-bit) /etc
    HP-UX-Itanium /var/opt/oracle

  2. Confirm that the contents of oraInst.loc look like this example:

    inventory_loc=/oracle/oraInventory

    where /oracle/oraInventory is the path to the directory where the central inventory is located. This location must be writable by the user account that will run Rapid Install.

    If the oraInst.loc file does not exist, create it in the correct directory with contents based on the example shown above.

    Warning: Incorrect permissions on oraInventory may cause issues not only with online patching (fs_clone phase), but also when installing a system with Rapid Install or cloning a system with Rapid Clone.

    Note: If your system has separate installation user accounts for the database and the applications, both users must be in the same install group (inst_group) in oraInst.loc, which will need to contain a line such as inst_group=oracle.

Important: If you opted to use a Local Inventory, your opatch lsinventory command must include the -invPtrLoc parameter and appropriate value, or the command will fail. The syntax is as follows:

$ opatch lsinventory -invPtrLoc <path to oraInst.loc file>

Using a Central Inventory

As well as supporting the traditional global inventory, the AD-TXK Delta.7 codelevel also introduces support for an EBS Installation Central Inventory on the application tier of UNIX platforms. Such an inventory is specific to a particular Oracle E-Business Suite instance, and identified by the <s_base>/oraInventory/oraInst.loc file.This feature is useful where there are multiple Oracle E-Business Suite installations on the same host. In particular, it allows safe simultaneous running of fs_clone on the different instances.

For an Oracle E-Business Suite instance to use the EBS Installation Central Inventory, all application tier Oracle Homes registered in the global inventory for the instance need to be migrated to the new inventory. You can do this can by performing the following steps on the primary application tier node.

  1. Source the run edition file system.

  2. Edit the context file and set the value of the context variable 's_ebs_central_inventory' to 'true'.

  3. Run AutoConfig.

  4. Execute the following command:

    $ perl <FND_TOP>/patch/115/bin/txkMigrateInventory.pl -contextfile=<CONTEXT_FILE>

  5. Ensure that all Application tier Oracle Homes of the current instance, registered with the global inventory have been migrated to the 'EBS Installation Central Inventory'.

Repeat all the above steps on any non-shared nodes and shared primary nodes (for example, in a hybrid setup). For all shared secondary nodes, perform Steps 1 to 3 (only) on each node.

Important: Once the inventory is migrated, any subsequently added nodes will be automatically configured to use the EBS Installation Central Inventory. Similarly, any new target instance cloned from this instance will automatically be configured to use the EBS Installation Central Inventory.

Set Up Secure Shell on Application Tier Nodes

In a multi-node environment, adop commands are invoked by a user on the primary node. Internally, adop uses Secure Shell (ssh) to automatically execute required patching actions on all secondary nodes. You must set up passwordless ssh connectivity from the primary node to all secondary nodes.

Note: Rapid Install and Rapid Clone set up the ssh key infrastructure.

Principles

The ssh-keygen command is used to generate a private/public key pair. The private key is for the node from where all the remote nodes will subsequently be accessible by an ssh login that requires no password. The public key must be copied to each remote node's <User Home Dir>/.ssh directory.

In essence, the sequence is as follows:

  1. The following command initiates creation of the key pair:

    $ ssh-keygen -t rsa

    Note: The <Enter> key should be pressed instead of a passphrase being entered.

  2. The private key is saved in <User Home Dir>/.ssh/id_rsa.

    Important: As this read-only file is used to decrypt all correspondence encrypted with the public key, its contents must not be shared with anyone.

  3. The public key is saved in <User Home Dir>/.ssh/id_rsa.pub.

  4. The contents of the public key are then copied to the <User Home Dir>/.ssh /authorized_keys file on the systems you subsequently wish to ssh to without being prompted for a password.

The following example demonstrates the steps:

  1. $ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/u01/user2/.ssh/id_rsa):<Enter> 
Enter passphrase:<Enter>
Enter same passphrase again:<Enter>
Your identification has been saved in /u01/user2/.ssh/id_rsa.
Your public key has been saved in /u01/user2/.ssh/id_rsa.pub.
The key fingerprint is: 16:d0:e2:dd:37:2f:8e:d5:59:3e:12:9d:2f:12:1e:5a
  2. $ scp -pr /u01/user2/.ssh/id_rsa.pub \
user2@system1:/u01/user2/.ssh/authorized_keys
user2@system1's password:<password>
id_rsa.pub 100% 398 0.4KB/s 00:00
  3. $ ssh user2@system1

    Note: If you receive this message, it can safely be ignored: Warning: untrusted X11 forwarding setup failed: xauth key data not generated Warning: No xauth data; using fake authentication data for X11 forwarding.

Once this has been done for the relevant operating system account on all nodes - that is, ssh can log in from the primary node to each secondary node without entering a password - so you are ready to run adop on multiple application tier nodes. It must be run on at least the primary (admin) node: from there, it will attempt to contact all the other application tier nodes that are part of the same Oracle E-Business Suite instance, and will run the required steps remotely on those nodes.

Important: If you change the password for the relevant operating system account on one or more nodes, you must regenerate the ssh credentials either using the $AD_TOP/patch/115/bin/txkRunSSHSetup.pl script, or your own native solution if you prefer.

The txkRunSSHSetup.pl script has a -help option that shows relevant usage options.

For example, a basic command to enable ssh would be:

$ perl $AD_TOP/patch/115/bin/txkRunSSHSetup.pl enablessh -contextfile=<CONTEXT_FILE> -hosts=h1,h2,h3$

To verify ssh operation:

$ perl $AD_TOP/patch/115/bin/txkRunSSHSetup.pl verifyssh -contextfile=<CONTEXT_FILE> -hosts=h1,h2,h3 \
-invalidnodefile=<filename to report ssh verification failures>

To disable ssh:

$ perl $AD_TOP/patch/115/bin/txkRunSSHSetup.pl disablessh \
-contextfile=<CONTEXT_FILE> -hosts=h1,h2,h3 \
-invalidnodefile=<filename to report ssh verification failures>

Create Customized Instructions for Patch Application Assistant

For patches that have manual steps, the patch readme file may instruct you to use Oracle Patch Application Assistant (PAA) to create customized instructions for your system. PAA consolidates and displays only the relevant manual steps for all the patches you want to apply, including steps that you have completed. It also automatically merges the contents of individual patch readme files for a merged patch.

Requirement: How do I know which manual steps associated with a patch apply to my system?

Sorting through the manual steps in a patch readme file to determine which ones apply to your system can be time-consuming. The Patch Application Assistant allows you to create a customized set of steps to that apply to your unique instance. Using the information on this list reduces the possibility of performing steps that are not necessary or that have been completed previously during the application of another patch.

When you download and unzip a patch, it delivers a static README.html file that advises you if the patch requires manual steps. If manual steps are required, you can generate a list of the steps by running a Perl script (admsi.pl) to initiate PAA. Once you have generated the list, use the PAA interface to see a full list of steps, or only those steps that apply to your system.

After successfully performing each manual step, you can record that it was completed. When applying patches in the future, this information is displayed in the PAA interface so that you can see which manual steps you have already performed.

To run PAA

  1. Download the patch that you want to apply and set (source) your environment. On UNIX systems, you must also set the environment variable DISPLAY to an active and authorized display.

    For instructions on setting your environment, see: Running AD Utilities in this book.

  2. Run the admsi.pl script to generate customized installation instructions.

    $ admsi.pl

    The Oracle Patch Application Assistant welcome page appears:

    You can select:

    • View instance-specific instructions for a new patch.

    • View generic instructions as shipped by Oracle for a new patch - to view all the generic manual steps for a particular patch, including the completed steps.

    • Look at all incomplete tasks from previous patches - to view all the manual steps that have not been completed from previous patches.

  3. Select View instance-specific instructions for a new patch. Enter the APPS password, and select the location where the patch is staged. Click Next.

    The Summary of Installation Instructions page appears:

    This page summarizes all the manual steps for the patch, grouped into the following categories: Preparation Tasks, Pre-Install Tasks, Apply the Patch, Post-Install Tasks, Finishing Tasks, and Additional Information. This page displays only those categories in which there are manual steps.

  4. Click the plus-sign icon in each category for more detailed information. For example, if you click the plus sign icon next to Best Practices, the Preparation Tasks screen appears with the tasks suggested for preparing your system for patching.

  5. After you have completed all the manual steps in a category, check the Completed box to record the completion status in the database, then click Next. If a patch that you apply in the future contains any of the same manual steps, it will be marked as completed to inform you that you do not have to perform that task again.

    After you have completed all manual steps in all categories, the system returns you to the Summary of Installation Instructions page.

    Note the column of Completed boxes that corresponds to each task in a category. Check marks appear in the boxes for which you have completed manual steps.

  6. Click Save to record tasks completed in the database. Click Cancel to exit PAA.

The Online Patching Cycle

This section describes the online patching cycle from beginning to end, illustrating the actions taken in the different phases and putting into context the more detailed description of online patching in the following sections. It is designed to be read in conjunction with the important background material provided in the "Patching and Management Tools" chapter of Oracle E-Business Suite Concepts.

The online patching cycle consists of a number of high level phases:

  1. prepare

  2. apply

  3. finalize

  4. cutover

  5. cleanup

A high level overview of an online patching cycle would, programmatically, look like this:

# Prepare for patching:
$ adop phase=prepare

# Apply patches:
$ adop phase=apply patches=<patch number>

# Apply any customizations to patch edition (optional):
$ . <EBS_ROOT>/EBSapps.env patch
$ sqlplus apps/apps @my_custom_script_01
$ sqlplus apps/apps @my_custom_script_02
...

# Finalize patch application:
$ adop phase=finalize

# Perform cutover:
$ adop phase=cutover
$ . <EBS_ROOT>/EBSapps.env run

# Perform user acceptance testing via application UI

# Perform cleanup:
$ adop phase=cleanup

Important Additional Points

Special Phases

Two additional phases are provided for specialized use. Neither can be run in conjunction with any other phase. Further details of these phases are described in later sections.

The Online Patching Cycle

the picture is described in the document text

adop will automatically set its environment as required, but it is the user's responsibility to set the environment correctly for any other commands that may be run. Set the run edition environment whenever executing commands that you intend to affect the run edition.

For example:

$ . <EBS_ROOT>/EBSapps.env run
$ adstrtal.sh

Set the patch edition environment whenever you intend to execute commands that affect the patch edition.

For example:

$ . <EBS_ROOT>/EBSapps.env patch
$ sqlplus apps/apps @my_custom_patch_script.sql

The adop tool executes non-interactively, executing the specified phase or phases in order. In a multi-node deployment, adop is only executed by the user on the primary node: internally, adop will use ssh remote execution to run required actions on all secondary nodes automatically. In addition, adop can be used to generate reports about patching operations in the environment.

adop is typically run as follows:

$ adop phase=<phase_name>

The phase parameter accepts the following phase names. These names can be specified individually, or (except where otherwise noted) with other phase names in a comma-separated list:

Online Patching Cycle Steps - Prepare Phase

This section describes the principles of adop operation in the prepare phase, followed by the steps you take to run this phase.

Note: The exact actions taken during the prepare phase are context-dependent: for example, the first time it is ever run on a system; when it is run after an apply phase has been aborted; and when it has been run after cutover.

Principal adop Actions

During the prepare phase, adop performs the following steps.

  1. Checks whether to perform a cleanup, which will be needed if the user failed to invoke cleanup after the cutover phase of a previous online patching cycle.

  2. Validates system configuration to ensure that the system is ready to start an online patching cycle.

  3. Checks to see if the database is prepared for online patching:

    • Checks if the database user is edition-enabled. If not, adop immediately exits with an error.

    • Checks to see if the patch service has been created. adop requires that a special database service exists for the purpose of connecting to the patch edition. This service is created automatically, but its continued existence is validated on each prepare.

    • Checks to see if logon trigger exists and is enabled. If the logon trigger is missing or the patch service has not been created, adop will automatically try to fix the issue so that it can proceed. If it cannot do so, it will exit with an error message.

    • Checks the integrity of the database data dictionary. If any corruption is found, adop will exit with an error. For information on how to resolve data dictionary corruptions, refer to My Oracle Knowledge Document 1531121.1, Using the Online Patching Readiness Report in Oracle E-Business Suite Release 12.2.

    • Checks that the E-Business Suite Technology Codelevel Checker (ETCC) has has been run, to verify that all required patches have been applied to the database.

  4. Checks system configuration on each application tier node. A number of critical settings are validated to ensure that each application tier node is correctly registered, configured, and ready for patching.

  5. Checks for the existence of the "Online Patching In Progress" (ADZDPATCH) concurrent program. This program prevents certain predefined concurrent programs from being started, and as such needs to be active while a patching cycle is in progress (that is, while a database patch edition exists).

    The flow of control is as follows.

    1. If the ADZDPATCH program has not yet been requested to run, a request is submitted.

    2. The status of ADZDPATCH is determined. If it is pending, it may be waiting for an incompatible program to finish. At that point, its status will change to running, and it will allow the prepare phase to proceed. A message to this effect is displayed to the user.

    3. The next stage depends on whether the concurrent managers are running:

      1. If the concurrent managers are all down, the prepare phase continues, with ADZDPATCH entering a status of pending (with the highest priority) until the managers are started.

      2. If the concurrent managers are partially up, but there is no manager defined that can run ADZDPATCH, then the prepare phase will exit with an error.

      3. If the concurrent managers are up, and there is one defined that can run ADZDPATCH, processing will loop until ADZDPATCH changes status from pending to running (that is to say, as noted in Step 2, no incompatible programs are found). The prepare phase then continues.

    Note: ADZDPATCH is cancelled when the cutover phase is complete.

  6. Invokes the TXK script $AD_TOP/patch/115/bin/txkADOPPreparePhaseSynchronize.pl to synchronize the patches which have been applied to the run APPL_TOP, but not the patch APPL_TOP. The script depends on the adop repository for patches that have been applied on the run APPL_TOP but not the patch APPL_TOP.

  7. Checks the database for the existence of a patch edition, and creates one if it does not find one.

  8. Calls the $AD_TOP/patch/115/bin/txkADOPPreparePhaseSanityCheck.pl script again to confirm that the database connection to the patch edition is working.

If any of these checks fail, adop will exit with an error.

Optional User Checks

Before you run the prepare phase to start a new patching cycle, you may wish to perform a couple of optional checks.

The first check is to validate your system for patching, by running the command:

$ adop -validate

Note: If you run this command while a patching cycle is in progress, will take place for the cutover phase.

If you run this command from the primary node, adop will perform the checks on all the available nodes available in the system. In contrast, if you run it from a secondary node, it will run only on that node.

The second check is to confirm there is adequate free space on your system to support a patching cycle:

You can do this by running the $AD_TOP/sql/ADZDSHOWTS.sql script:

Note: For instructions on how to increase the size of a tablespace, refer to the "Altering and Maintaining Tablespaces" section in the "Managing Tablespaces" chapter of the Oracle Database Administrator's Guide.

Required User Actions

You perform the following steps in the prepare phase.

  1. Set the environment by executing (sourcing) the run file system environment file:

     $ source <EBS install base>/EBSapps.env run

    For more information, see Setting the Environment in Running AD Utilities

    You can confirm that the environment is properly set by examining the relevant environment variables:

    $ echo $FILE_EDITION
run
$ echo $TWO_TASK
dbSID

    If you had sourced the incorrect environment file (i.e. from the patch file system), the environment variables would show as:

    $ echo $FILE_EDITION
patch
$ echo $TWO_TASK
dbSID_patch

  2. Download patches to be applied and place them in the $PATCH_TOP directory of your system. This directory is pre-created by the install in the non-editioned file system (fs_ne) and should not be changed.

    Important: On a multi-node system with non-shared file systems, you must copy the patch files to each separate $PATCH_TOP directory, so that the patch files are available from the same location on all nodes.

  3. Unzip the patch:

    $ unzip p99999999.zip

  4. Prepare the system for patching by running the following command to start a new patching cycle:

    $ adop phase=prepare

    File System Synchronization Options

    Sometimes, the patch file system application tier needs to be synchronized with the run file system application tier. There are three supported methods for doing this, the choice of which will depend on the circumstances and specific requirements. Each will be discussed in turn, with the first being the default and the third the most sophisticated.

    1. Default File Synchronization Method - Identify patches that were applied to the run file system APPL_TOP and then apply them to the patch file system APPL_TOP.

      The following steps are performed automatically:

      1. The patches that need to be applied to the patch APPL_TOP are identified and merged.

      2. The merged patches are applied by the adop utility.

      The adop utility identifies the patches to be applied, and applies them silently to the current patch APPL_TOP. As this procedure only requires the application of previously unapplied patches, it needs less time and disk space than fs_clone Synchronization (the second method described below).

      In some circumstances, this default synchronization method may encounter issues when applying a series of patches to the patch edition. For example, if the previous patching cycle included patches that failed to apply correctly, and was followed by application of subsequent patches that corrected the issue.

      The skipsyncerror parameter enables you to specify that you expect any synchronization errors in the prepare phase to be fixed automatically in the synchronization that takes place with subsequent patches. If the value of this parameter is specified as yes, the first patch to be synchronized will be done with the autoskip flag set.

      Note: You must check the log files and confirm that synchronization with subsequent patches resolved the issue.

      An example of using skipsyncerror is as follows.

      1. You run adop phase=prepare.

      2. The prepare phase fails with an error when trying to synchronize the run and patch file systems. That is, the attempt to synchronize a patch fails, but it is known that a subsequent patch will correct the problem.

      3. You examine the log files and conclude that the errors will be fixed automatically in the synchronization that takes place with application of subsequent patches.

      4. You run the command adop phase=prepare skipsyncerror=yes to restart the prepare phase. This time, application of the patch that failed in the previous prepare will be retried with the autoskip flag set.

      Alternatively, if you are not confident that the error will be fixed (for example, you cannot identify the cause from examination of the log files), you should run the following commands in the order shown.

      1. adop phase=abort

      2. adop phase=cleanup cleanup_mode=full

      3. adop phase=fs_clone force=yes

    2. fs_clone File Synchronization Method - Create a new patch file system by cloning the run file system using a special adop phase, fs_clone.

      This method is useful if the APPL_TOPs have become very unsynchronized (meaning that there would be a large number of delta patches to apply). It is a resource-intensive process, taking a backup of the entire current patch APPL_TOP and then cloning the run APPL_TOP to create a new patch APPL_TOP. As this method requires more time and disk space, it should only be used when the state of the patch file system is unknown.

      The fs_clone command is run as a special adop phase that must be invoked from the run file system, using the command:

      $ adop phase=fs_clone

      Note: You cannot specify fs_clone in conjunction with any other adop phase.

      From Release 12.2.10, file system synchronization is implemented via a new defaults file, $APPL_TOP_NE/ad/admin/adopdefaults.txt. This file holds two new parameters. The one that applies to the fs_clone phase is called fs_clone_sync_mode. It can have a value of either cp or delta. The default is cp. Specifying cp creates the complete patch file system from the run file system. Specifying delta uses the file system synchronization command specified in the $APPL_TOP_NE/ad/admin/delta_sync_drv.txt file. This parameter is also applicable to the prepare phase.

      This parameter is also available for use from the adop command line, and regular customer input files.

      Note: The patch file system requires at least 25 GB of free disk space to be available for adop operations, including fs_clone. If there is insufficient free space, the adop operation will fail.

      If desired, you can change the temporary file system location used by fs_clone, by setting the T2P_JAVA_OPTIONS environment variable to point to a temporary location of your choice:

      $ T2P_JAVA_OPTIONS="-Djava.io.tmpdir=/home/t2p/temp"
$ export T2P_JAVA_OPTIONS

      Note: You cannot change the fs_clone temporary location by changing the value of the $TMP environment variable.

      If an fs_clone operation fails, you can rerun it with the option force=yes to restart it from the beginning (with the same session ID), or force=no to restart it from the point where it failed.

      Note: Windows users should refer to their platform-specific release notes for restrictions that may apply when running fs_clone.

    3. Configurable File Synchronization Method - This fast synchronization method uses your choice of third-party utility (we recommend rsync for UNIX platfoms) to synchronize the file systems by copying files as applicable from the source directory to the destination directory. Optionally, you can ignore selected files and directories by specifying them in an exclusion file, $APPL_TOP_NE/ad/admin/delta_sync_exclude_list.txt. Configurable file synchronization uses a new adop parameter, prepare_sync_mode, which can have a value of delta or patch. Choosing delta uses the file system synchronization command specified in the $APPL_TOP_NE/ad/admin/delta_sync_drv.txt file. Choosing patch reapplies all the patches that were applied to the run file system. The default is patch.

      To use this method, specify the parameter/value pair prepare_sync_mode=<value> on the adop command line. For example:

      $ adop phase=prepare prepare_sync_mode=delta

      The delta_sync_drv.txt file includes examples for setting up synchronization using rsync on UNIX and RoboCopy on Windows.

    Synchronizing Customizations

    The default method of file system synchronization handles Oracle-supplied patches but will not synchronize any customizations that have been manually applied.

    Additional Information: The synchronization modes and their associated actions are as follows:

    • Patch synchronization - apply missing patches

    • Delta (file) synchronization - copy file changes

    • fs_clone synchronization - clone entire file system

    Examples of patching actions that are not synchronized by default include:

    • Compiling user-defined JSPs

    • Copying some third-party libraries

    • Copying and compiling user-defined concurrent programs

    • Copying and generating user-defined forms

    To include custom patching actions in the default file system synchronization, you must include the required commands in the Custom Synchronization Driver, $APPL_TOP_NE/ad/custom/adop_sync.drv. You will add your customizations to the following section of the file:

    #Begin Customization
...
#End Customization

    All the actions defined in this file will be performed by adop automatically during the prepare phase. Be aware that there are two categories of custom command in adop_sync.drv: those that are run one time only, and those that are run at each file system synchronization (during the adop prepare phase).

    The adop_sync.drv file is not currently reset to its template file at any point. Consequently, after cutover (and before the next prepare phase), you should review the contents of adop_sync.drv and ensure the requirements for your custom commands continue to be met.

    Note: This is only an outline of the steps you need to take to preserve customizations. For full details, refer to Oracle E-Business Suite Developer's Guide

    Prepare Phase in Multi-Node Environments

    In a multi-node environment, one application tier node will be designated as the primary node. This is the node where the Admin Server is located, and will usually also be the node that runs Oracle HTTP Server. All other application tier nodes are designated as secondary nodes.

    In a multi-node environment, you must enable ssh from the primary node to all secondary nodes to permit adop remote invocation. For ssh setup steps, refer to Set Up Secure Shell on Application Tier Nodes. In a multi-node environment, adop commands are always run from the primary node only. adop executes required patching actions on the secondary nodes automatically via remote invocation.

    If a node unexpectedly becomes inaccessible via ssh, it will be abandoned by adop, and the appropriate further actions taken. Consider a scenario where the adop phase=prepare command is run in a system with ten application tier nodes. The command is successful on nine nodes, but fails on the tenth. In such a case, adop will identify the services enabled on nodes 1-9. If they are sufficient for Oracle E-Business Suite to continue to run normally, adop will mark node 10 as abandoned and then proceed with its patching actions. If they are not sufficient, adop will proceed no further.

Online Patching Cycle Steps - Apply Phase

This section describes the principles of adop operation in the apply phase, followed by the steps you take to run this phase.

Principles

In the apply phase, adop applies the specified patches to the system. In an online patching cycle, patches are applied to the patch edition of the database and file system.

Steps

In this phase, you will apply the patches that you designated for inclusion in this patching cycle. You can apply as many patches as you want per patching cycle. By default, a list of patches is applied one at a time, in the specified order. If you specify the merge option "merge=yes", the listed patches will automatically be merged and the resulting merged patch will be applied.

The following example will illustrate the options.

$ adop phase=apply input_file=<inputfile.txt>

This uses the input_file that was mentioned earlier in this section.

An example input_filemight liook like this:

workers=<number of workers>
patches=<patch number 1>:<driver file 1>.drv, <patch number 2>:<driver file 2>.drv ...

Reports under the $APPL_TOP/admin/<SID>/out directory can help you identify and diagnose problems that may occur in the online patching cycle. These reports list the proposed changes to database objects, both new and modified.

The key files to examine are:

Note: For merged patches, the log file name will be derived from the timestamp when merging was performed.

Using the analytics parameter in apply

If you want to use the analytics parameter (see adop Parameters) with the apply phase, enter the command:

$ adop phase=apply analytics=yes

Specifying this option will cause adop to run the following scripts and generate the associated output files (reports):

Note: The analytics parameter should only be used when required, because of the extra processing needed.

Online Patching Cycle Steps - Finalize Phase

The finalize phase is used to perform any remaining processing that is needed to ensure the system is ready for the fastest possible cutover.

The key actions of the finalize phase are:

  1. Pre-compute DDL that needs to be run at cutover.

  2. Compile all invalid objects.

  3. Validate that the system is ready for cutover.

  4. If finalize_mode=full, compute statistics for key data dictionary tables for improved performance.

Run the finalize phase as follows:

     $ adop phase=finalize

Online Patching Cycle Steps - Cutover Phase

This section describes the principles of adop operation in the cutover phase, followed by the manual steps you can optionally execute to run this phase.

Important: No users should remain on the system during cutover, as there will be a short downtime period while the application tier services are restarted. Also, any third-party processes connected to the old run edition of the database should be shut down, or they will be terminated automatically. If desired, you can defer running cutover until a time which will cause minimal disruption to users.

Principles

The key actions performed in the cutover phase are:

  1. Shut down internal concurrent manager: The adop utility signals the internal concurrent manager to shut down, but will wait for any existing concurrent requests to finish before it proceeds with cutover actions. The system is still available to users during this waiting period.

    If you do not wish to wait indefinitely for concurrent requests to finish, specify the option cm_wait=<maximum_minutes_to_wait> with a number of minutes that reflects your operational needs.

    When deciding whether to use this option, Oracle recommends:

    • On production systems, do not specify cm_wait, but monitor progress of concurrent tasks and take manual action on them if needed.

    • On non-production systems, specify cm_wait to limit the waiting time before cutover proceeds.

  2. Shut down application tier services: All application tier services are brought down. During this period, the system is unavailable to users.

  3. Cutover database: Promote patch database edition to become the new run database edition, using adzdpmgr.pl script.

  4. Cutover file system: Promote patch file system to become the new run file system, switching the $FILE_EDITION values in the patch and run enviroments. The current patch APPL_TOP becomes the new run APPL_TOP, and the current run APPL_TOP becomes the new patch APPL_TOP.

  5. Terminate old database sessions: Terminate any database connections to the old run edition of the database.

  6. Start application tier services: Application tier services are restarted, on the new run edition. The system is now available again to users.

Note: The adop utility invokes the TXK script txkADOPCutOverPhaseCtrlScript.pl to perform tasks 1, 2, 3, 5, and 6. Task 4 is performed by AutoConfig.

  1. Before running the cutover command, ensure you are ready to commit to application of the selected patches. Once cutover is complete, it is not possible to revert to the previous edition.

    Note: Cutover will take longer if it has to wait for long-running concurrent processes to complete. In such a case, you can expect to see an informational message of the form:

    [STATEMENT] [END 2013/10/28 23:47:16] Waiting for ICM to go down

    If you do not want to wait for in-progress concurrent requests to finish normally, you can terminate the internal concurrent manager by executing the adcmctl.sh abort command from a different shell.

  2. In most cases (but see below for the important exception of analytics), you then proceed to execute cutover with the command:

    $ adop phase=cutover

    This will promote the patch edition to be the new run edition, as well as switching the patch and run labels on the file systems (and thereby, as noted above, changing the patch file system to be the new run file system and the run file system to be the new patch file system).

    Important: In the event of problems with the cutover phase, refer to My Oracle Support Knowledge Document 1584097.1, Oracle E-Business Suite Release 12.2: Backup and Recovery Guidelines For Online Patching Cutover.

    Deferring Application Tier Restart at Cutover

    In some cases, you may need to perform additional manual steps after cutover but before restarting the application tier services. If this is the case, you can supply an additional parameter to the cutover command that causes the application services to remain shut down:

    $ adop phase=cutover mtrestart=no

    With this parameter, cutover will complete without restarting the application tier services. You can perform any additional steps that require the services to be shut down, and then start the application tier services manually using the adstrtal.sh script.

    You must then also run the steps in the following section, Patching the Database Tier:

    Patching the Database Tier

    These steps are performed post-cutover.

    1. On the application tier, as the applmgr user:

      1. Change directory to the run file system $APPL_TOP and source your environment file.

      2. Run the following command:

        $ perl <AD_TOP>/bin/admkappsutil.pl

        This will create the appsutil.zip file in <INST_TOP>/admin/out.

    2. On the database tier, as the oracle user:

      Copy or ftp the appsutil.zip file to the RDBMS_ORACLE_HOME, then run the following commands:

      $ cd <RDBMS_ORACLE_HOME>
$ unzip -o appsutil.zip

    3. Run AutoConfig on the database tier.

    4. Run AutoConfig on the run file system of each application tier node.

    5. Start the application tier services.

    JAR Files and Cutover

    In an online patching cycle, the requisite JAR files are initially stored in the $APPL_TOP/admin/<SID>/out directory, and then uploaded into the database during the cutover phase. Therefore, the out directory must not be deleted at least until cutover is complete.

Online Patching Cycle Steps - Cleanup Phase

This section describes the principles of adop operation in the cleanup phase, followed by the steps performed in this phase.

Important: If you do not run the cleanup phase explicitly, it will be run automatically on the next prepare cycle, but this will cause a delay when starting your next online patching cycle.

Additional Information: With Oracle Database 19c Release Update 19.18 or later, cleanup of unused editioned objects in old database editions is handled by a background cleanup process in the database itself. A requested cleanup in adop will signal to the database that an old database edition is no longer in use, and over time the database will clean up unused objects in old editions using a background job. This will allow adop cleanup to complete much more quickly.

Principles

Various actions are performed during cleanup, including dropping (removing) obsolete objects such as:

Steps

  1. Cleanup is performed with the command:

    $ adop phase=cleanup

    The adop parameter cleanup_mode provides control of cleanup processing:

    • cleanup_mode=quick - Performs minimum cleanup, including removal of obsolete crossedition triggers and seed data.

    • cleanup_mode=standard - Does the same as quick mode, and also drops (removes) obsolete editioned code objects (covered objects). This is the default mode , so does not need to be specified.

    • cleanup_mode=full - Performs maximum cleanup, which drops all obsolete code and data from earlier editions.

    Choosing the Cleanup Mode

    Generally, you can use standard mode (the default). You might want to use the other modes in the following circumstances:

    • Use quick cleanup when you need to start the next patching cycle as soon as possible. For example, if you want to start a new patching cycle right away, but have not yet run cleanup from the previous patching cycle, you can use quick cleanup mode to complete the essential cleanup tasks as fast as possible.

    • Use full cleanup when you want to recover the maximum amount of space in the database. If you have run a large number of patching cycles, or applied a very large patch such as a rollup, significant space may be consumed by obsolete table columns and recovered by running a full cleanup. A full cleanup should only be performed when there is no immediate need to start a new patching cycle.

    Note: Prior to AD-TXK Delta 8, if a table is patched and the definition of an existing column changed, the original column is marked as unused on a subsequent full cleanup. From AD-TXK Delta 8, a lower-versioned column is not marked as unused. If an abort is carried out, a higher-versioned column is marked as unused, as such a newly-added column is not in use.

    Terminating Sessions Connected to Old Editions

    During a full cleanup operation, the adop utility can only drop old editions if those editions are not in use. An SQL script, ADZDKILLOLDSESSIONS.sql, provides a solution to let you terminate sessions that are connected to old editions. If you receive an error notification indicating that an edition could not be dropped because it is in use, you can run this script to terminate any sessions using the edition, and then proceed with the cleanup operation.

    Using the analytics Parameter in Cleanup

    If you want to use the analytics parameter (see adop Parameters) with the cleanup phase, enter the command:

    $ adop phase=cleanup analytics=yes

    Specifying this option will cause adop to run the following script and generate the associated output file (report):

    • ADZDCLEANUPRP.sql - This script is used to display the display the cleanup status. The output file location is: $NE_BASE/EBSapps/log/adop/<adop_sessionID>/<cleanup_directory>/<context_name>/adzdcleanuprp.out.

    Note: The analytics parameter should only be used when required, because of the extra processing needed.

Online Patching Cycle Steps - Abort Phase

If for some reason either the prepare or apply phase failed or gave problems, you can abort the patching cycle at either of these points by running a special phase with the command:

$ adop phase=abort

In the abort phase, adop does the following:

  1. Confirms that there is an in-progress online patching cycle, so the abort call is therefore valid.

  2. Checks for the existence of a patch edition and drops one if it exists.

  3. Cancels the ADZDPATCH concurrent program, if it is running.

  4. Deletes the rows inserted for the pending session ID from the ad_adop_sessions and ad_adop_session_patches tables.

Be aware of the following important points:

This section covers a variety of tasks that may apply either to individual online patching operations, or to your system setup as a whole. Diagnostic, troubleshooting, and reporting features are also described.

Manual Post-Patch Installation Tasks

Traditionally, some patches have associated post-patch installation tasks, including recompilation of invalid packages, regenerating JAR files, and running AutoConfig. In an online patching environment such as Release 12.2 such tasks will normally be performed automatically in the apply phase.

If a post-installation patch step mentions any tasks that need to be performed explicitly, where they are run from depends on the type of patching:

Dropping Old Editions With the actualize_all Phase

As each online patching cycle is completed, the database will accumulate an additional old database edition. If the number of these grows too large, system performance will start to be affected. When the number of old database editions reaches 25 or more, you should consider dropping all old database editions by running the adop actualize_all phase and then performing a full cleanup.

Important: This procedure will take a large amount of time (significantly longer than a normal patching cycle), and should only be performed when there is no immediate need to start a new patching cycle.

Before starting, you should ensure that the system has the recommended database patches and latest AD-TXK code level installed.

To proceed, run the following commands in the order shown:

$ adop phase=prepare
$ adop phase=actualize_all
$ adop phase=finalize finalize_mode=full
$ adop phase=cutover
$ adop phase=cleanup cleanup_mode=full

You have now completed removal of the old database editions.

With Oracle Database 19c Release Update 19.18 or later, cleanup of unused editioned objects in old database editions is handled by a background cleanup process in the database itself. A requested cleanup in adop will signal to the database that an old database edition is no longer in use, and over time the database will clean up unused objects in old editions using a background job. This will allow adop cleanup to complete much more quickly.

Context Variable Requirements Across Nodes and File Systems

The following context variables must have same value across all nodes, and also across the run and patch file systems:

The following context variables must have same value across the run and patch file systems of a given node:

Note: Following the application of one-off Patch 18942757:R12.TXK.C, the following MWA-related ports will have the same values on the run and patch file systems:

Configuration Management and Patching

The following guidelines apply to making configuration changes to Oracle E-Business Suite in the context of online patching. They particularly apply to the technology stack and application components that reside in the file system.

Note: For specific instructions on how to patch technology stack components, refer to My Oracle Support Knowledge Document 1355068.1, Oracle E-Business Suite 12.2 Patching Technology Components Guide.

The two basic scenarios are online and offline configuration changes. Each will be considered in turn.

Online configuration changes are performed within the context of an online patching cycle. This is the recommended strategy.

First, you prepare your system by running the adop phase=prepare command. You then make the desired configuration changes to the patch file system. They may include:

After making the configuration changes, you must run the command adop phase=cutover to promote them.

You must also run the command adop phase=fs_clone to propagate the configuration changes to the secondary file system.

Offline configuration changes are applied directly to the run file system, outside an online patching cycle.You can use the adop -status command to verify that no patching cycle is currently active. After making the desired configuration changes, you must explicitly run the adop phase=fs_clone command to propagate the changes to the patch file system.

Important: This offline scenario will require a period of downtime for users.

Support for Single File System Development Environments

A normal Release 12.2 online patching environment requires two application tier file systems, one for the run edition and another for the patch edition. This dual file system architecture is fundamental to patching of Oracle E-Business Suite Release 12.2, and is necessary both for production environments and test environments that are intended to be representative of production. This feature makes it possible to create a development environment with a single file system, where custom code can be built and tested. The code should then always be tested in a standard dual file system test environment before being applied to production.

You can set up a single file system development environment by installing Oracle E-Business Suite Release 12.2 in the normal way, and then deleting the $PATCH_BASE directory with the command:

$ rm -rf $PATCH_BASE

A limited set of adop phases and modes are available to support patching of a single file system development environment. These are:

Specification of any other phase or mode will cause adop to exit with an error.

The following important restrictions apply to using a single file system environment:

Restrictions on Applying Patches in hotpatch Mode

Applying patches in hotpatch mode is only supported for use with patches that have been designed and tested to be applied in this way. This is because hotpatch mode applies changes to the run edition while this edition is in active use, which may result in one or more of the following issues for patches not designed to be applied as hotpatches:

Also, when you use hotpatch mode to apply a patch that contains a downloadable resource (such as a Forms-related client JAR file), that resource will only become available after you restart the Oracle WebLogic Server Managed Servers. Until you perform the restart, you may receive an error on the client or server about the integrity of the resource file.

Therefore, you should not attempt to apply a patch in hotpatch mode unless explicitly directed to do so by the patch readme.

Emergency Application of Patches

In an emergency, you can deploy patches directly to the run file system and run edition, with the following important restrictions being strictly adhered to:

Warning: Most Oracle E-Business suite patches are not tested in downtime or hotpatch mode. It is therefore important that this type of deployment is only used in an emergency, and not incorporated into standard maintenance practices.

Considerations When Re-Applying Patches

As mentioned in the "adop Options" section of Chapter 2, if you try to apply a patch that has already been applied, and you do not specify options=forceapply, adop will display an error such as:

[WARNING] Skipping the application of patch 14125999_AR since it has been already applied
[WARNING] Hint: Patches can be applied again by specifying options=forceapply when invoking adop

There are two more scenarios that may occur in this type of situation:

Using the Database Recycle Bin With Online Patching

Section 6 of My Oracle Support Knowledge Document 396009.1, Database Initialization Parameters for Oracle E-Business Suite Release 12, states:

#########
#
# recyclebin parameter
#
# The database recyclebin must be turned off to allow
# the cleanup phase of the online patching cycle to be
# performed without having to connect as SYS.
#
# This feature may still be used at other times.
#
#########

recyclebin=off

You can use the database recycling bin by following these steps:

  1. Set the 'recyclebin' database initialization parameter to 'on'

  2. Run the command adop phase=prepare

  3. Purge the dba_recyclebin table

  4. Run the command adop phase=actualize_all

  5. Run the command adop phase=finalize,cutover

  6. Run the command adop phase=cleanup cleanup_mode=full

For more information about the database recycle bin, refer to the 'Using Flashback Drop and Managing the Recycle Bin' section in Oracle Database Administrator's Guide 11g Release 2 (11.2).

Requirements When Running Oracle HTTP Server on a Privileged Port

On a UNIX system, the TCP/IP port numbers below 1024 are special in that only processes with root privileges are allowed to listen on those ports. If you have configured Oracle HTTP Server to run on a privileged port, you must perform the following additional steps when running an online patching cycle. These steps are required for both SSL and non-SSL privileged ports.

For more information, see: Starting Oracle HTTP Server on a Privileged Port, Oracle Fusion Middleware Administrator's Guide for Oracle HTTP Server and Running Oracle HTTP Server on a Privileged Port, My Oracle Support Knowledge Document 1905593.1, Managing Configuration of Oracle HTTP Server and Web Application Services in Oracle E-Business Suite Release 12.2.

Integrating Your Custom Tasks Into the Online Patching Cycle

You may have business-specific tasks specific that need to be performed before, during or after a patching cycle. Support for such tasks is currently provided by callout points at the begining and end of the cutover phase of the online patching cycle. This support will be extended in future releases of Oracle E-Business Suite.

Monitoring, Reporting and Troubleshooting

This section is in three parts, that describe:

Monitoring and Reporting

You can run the Online Patching Monitoring utility (adopmon) to provide a continuously refreshed view of key adop actions, in a coceptually similae way to the tail -f UNIX command. The adopmon utility is useful both in following the overall progress of a patching cycle and identifying the various individual actions that are being taken.

The following example shows how adopmon is used, with some sample output from the finalize phase.

$ adopmon

Running script. Press Ctrl-C to quit.

Enter the APPS password: 

2015/08/04 11:59:02 testsystem          EVENT        Checking for existing adop sessions.
2015/08/04 11:59:02 testsystem          EVENT        Checking for pending hotpatch session.
2015/08/04 11:59:02 testsystem          EVENT        Checking for pending cleanup session.
2015/08/04 11:59:02 testsystem          EVENT        Continuing with existing session [Session ID: 66].
2015/08/04 11:59:02 testsystem          EVENT        ADOP (C.Delta.7)
2015/08/04 11:59:02 testsystem          EVENT        Session ID: 66
2015/08/04 11:59:02 testsystem          EVENT        Node: testsystem
2015/08/04 11:59:02 testsystem          EVENT        Phase: finalize
2015/08/04 11:59:02 testsystem          EVENT        Log: /u01/orauser/usertest/fs_ne/EBSapps/log/adop/66/adop_20150804
                                                     _115825.log
2015/08/04 11:59:03 testsystem          EVENT        Verifying existence of context files in database.
2015/08/04 11:59:04 testsystem          EVENT        Verifying data dictionary.
2015/08/04 11:59:35 testsystem          EVENT        Running finalize phase database actions.
2015/08/04 11:59:35 testsystem          EVENT        Log: @ADZDSHOWLOG.sql "2015/08/04 11:59:35"
2015/08/04 11:59:35 testsystem          EVENT        Finalize System
2015/08/04 12:00:01 testsystem          EVENT        Executing FINALIZE actions
2015/08/04 12:00:01 testsystem          EVENT        Compiling invalid objects.
2015/08/04 12:00:01 testsystem          EVENT        Compile Edition: V_20150804_1041
2015/08/04 12:01:55 testsystem          EVENT        Generating log report.
2015/08/04 12:01:55 testsystem          EVENT        Output: /u01/orauser/usertest/fs_ne/EBSapps/log/adop/66/finalize_2
                                                     0150804_115825/usertest_testsystem/adzdshowlog.out
2015/08/04 12:01:56 testsystem          EVENT        The finalize phase completed successfully.

A related tool, the Online Patching Diagnostic Reports utility, $AD_TOP/bin/adopreports, can be used to help diagnose issues or simply gather information about the status of your system.

The adopreports utility is invoked by entering the command:

$ adopreports <APPS username> <APPS password>

This displays the adopreports Main Menu:

Online Patching Diagnostic Reports Main Menu
--------------------------------------------

    1.  Run edition reports
    2.  Patch edition reports
    3.  Other generic reports
    4.  Exit

Choosing option 1 from the Main Menu displays the Run Edition Reports Sub Menu:

 Run Edition Reports Sub Menu
 ----------------------------

    1.  All
    2.  Count of uncovered objects per edition
    3.  List of uncovered objects per edition
    4.  Cleanup status - summary of covered objects per edition,etc.
    5.  Show covered objects count per edition.
    6.  Show list of covered objects per edition.
    7.  Back to main menu

Choosing option 2 from the Main Menu displays the Patch Edition Reports Sub Menu:

Patch Edition Reports Sub Menu
------------------------------

    1.  All
    2.  Patch status - new/changed objects
    3.  Objects patch in the current edition
    4.  Table manager status
    5.  Back to main menu

Choosing option 3 from the Main Menu displays the Other Generic Reports Sub Menu:

 Other Generic Reports Sub Menu
 ------------------------------

    1.  Editions summary
    2.  Editioned objects summary
    3.  Free space in important tablespaces
    4.  Status of critical AD_ZD objects
    5.  Actual objects in current edition
    6.  Objects dependencies
    7.  Objects dependency tree
    8.  Editioning views column mappings
    9.  Index details for a table
    10.  Inherited objects in the current edition
    11.  All log messages
    12.  Materialized view details
    13.  Database sessions by edition
    14.  Table details (Synonyms, EV, etc.)
    15.  Count and status of DDL execution by phase
    16.  Back to main menu

Analyzing Log Files

The Online Patching Log Analyzer Utility analyzes adop log directories for errors and warnings, and displays messages to help the user quickly identify any problems that may have occurred during an adop run. It thereby offers an alternative to reviewing log files manually.

The Log Analyzer utility can be run without options, to scan all log directories of the latest adop session for errors:

$ adopscanlog

The utility can also be run with various options, as shown in the following list.

Common Issues

This section describes known issues and common problems with online patch application.

Patching HRMS Legislative Data

Special instructions apply to installation of the HRMS Legislative Data patch. These include adop options that will:

Important: Before applying the HRMS Legislative Data patch, refer to the full instructions provided in My Oracle Support Knowledge Document 1469456.1, Datainstall and HR Global Application: 12.2 Specifics.

Patching NLS Systems

These patching procedures apply regardless of whether you are running American English (US) and one additional language, or American English (US) and several additional languages. If your system uses multiple languages, you can use AD Merge Patch to create merged patches in whichever of the following ways suits you best:

Before the introduction of online patching, the choice of which strategy to follow largely depended on the downtime that was acceptable: for example, the first option was straightforward but required the greatest downtime, and the third was the most complex but allowed users of a particular language to resume their work as soon as the relevant patch was applied. The second option often provided the best compromise between easy application and minimum downtime. Now, however, use of online patching means that downtime is greatly reduced as a factor when determining the strategy that most closely suits the organization's needs

When merging multiple language patches, AD Merge Patch converts the character set according to the NLS_LANG variable in the Oracle E-Business Suite environment file. If you changed your character set since the initial installation, you might need to update the NLS_LANG variable. If this variable is not set properly, run AutoConfig from Oracle Applications Manager to update the Oracle E-Business Suite context with the correct character set information, then run the appropriate AutoConfig command to recreate the Oracle E-Business Suite environment file. Reset the environment using the new environment file before merging patches.

For more information, see: AD Merge Patch.

Applying a Single Patch to an NLS Installation

Requirement: I need to apply a single patch to an Oracle E-Business Suite NLS installation.

If an Oracle E-Business Suite system contains languages other than American English (US), the recommended method is to apply the US patch first and then apply the translation patch for each installed language. If you have installed more than one additional language, you can merge all the translation patches and apply them as a single, merged NLS patch.

You can also merge US patches with the additional language patches. However, depending on your system topology, it may be necessary to keep the US and non-US patches separate.

To apply a single patch to an NLS installation

This procedure assumes that you will apply US and language patches separately.

  1. Use adop to start a new patching cycle (adop phase=prepare).

  2. Use adop to apply the patch driver of the US patch.

  3. Use adop to apply the patch drivers of each NLS patch. If you have merged the individual NLS patches for a system that runs multiple languages, apply the driver for the merged NLS patch.

  4. Use adop to complete the patching cycle (finalize, cutover, cleanup phases).

Applying Multiple Patches to an NLS Installation

Requirement: I need to apply several patches to an Oracle E-Business Suite NLS installation.

If an Oracle E-Business Suite system contains multiple languages other than American English (US) and you are applying multiple patches for each language, the recommended method is to merge all US patches into a single patch and all patches for every non-US language into a single patch. Then apply the merged US patch, followed by the merged language patch.

You can also merge US patches with the additional language patches or merge each language in separate language-specific patches. Depending on your system topology, it may be necessary to keep the US and non-US patches separate. This procedure assumes that you will apply US and language patches separately.

To apply multiple patches to an NLS installation

This example assumes the system has American English, French, and German installed.

  1. Use adop to start a new patching cycle (adop phase=prepare).

  2. Use AD Merge Patch to merge the US (American English) patches into a single patch.

  3. Use AD Merge Patch to merge the French and German patches into a single NLS patch.

  4. Use adop to apply all drivers of the merged US patch.

  5. Use adop to apply all drivers of the merged NLS patch.

  6. Use adop to complete the patching cycle (finalize, cutover, cleanup phases).

Keeping Your System Current

Each time you apply a patch, associated information is stored in the Oracle Applications Manager patch history database. The Patch Wizard and Applied Patches tools provide graphical user interfaces that you can use to query the database for a complete history of patches applied to your system, to search for the patches you have already applied, and to determine existing patches that should be applied to keep your system current. Patch Wizard determines which recommended patches you should apply to your system, and the impact of applying these patches.

Before running Patch Wizard, you must set up My Oracle Support credentials. You must also set up preferences and filters that govern the way you download patches. To see how to complete these one-time tasks, as well as learn about navigating the Patch Wizard pages and submitting requests, see: Accessing Patch Wizard.

Creating a List of Recommended Patches

Requirement: How do I determine if there are patches that I have not yet applied?

Patch Wizard creates a list of patches by comparing the patches in the patch history database against a list of recommended patches in a Patch Information Bundle file downloaded from My Oracle Support. It then determines which of the recommended patches you should apply to your system and reports the contents of the patch and the files that it will update when applied.

It does not report on all available patches, but only patches at the current codeline, such as high-priority patches, and those that update your system to a new codeline (pre-upgrade patches).

Note: The following steps describe how to access Patch Wizard using Oracle Applications Manager. Alternatively, a user can use the out-of-the-box Patch Wizard responsibility. This responsibility has the Patching and Utilities Standalone menu associated with it. A user with this responsibility will not need to navigate through Oracle Applications Manager to use Patch Wizard.

To see a list of patches recommended for your system using Oracle Applications Manager

  1. Access Oracle Applications Manager.

    Follow the instructions in Accessing Patch Wizard to access OAM. All procedures in this section begin with the Site Map.

  2. Access the Patch Wizard home page.

    From the Site Map (Maintenance tab), click Patch Wizard under the Patching and Utilities heading.

    Site Map Page

    the picture is described in the document text

    The Patch Wizard home page appears.

  3. Submit a request for recommended patches.

    From the Recommend Patches page, select a patch filter. Use the magnifying glass icon to see a list of available patch filters.

    Recommend Patches Page - Top

    the picture is described in the document text

    After you have entered the requested information, click OK. The results of your request are shown in the Results section of the Patch Wizard main page. You can also schedule the request for a future date.

  4. Track the status of your request.

    From the main page, you can track the status of your recommended patch request. Click the Job Status icon for the Recommend/Analyze Patches task.

    Job Status Page

    the picture is described in the document text

    The Job Status page displays summary information. If you click the Show/Hide icon corresponding to your request ID, the page displays more details. For more information about the fields and functions on this page, see: Accessing Patch Wizard.

Downloading Recommended Patches

Requirement: How do I use Patch Wizard to download patches?

Patch Wizard can download patches based on either the list created by the "recommend patches" request or any list of patches entered in the Download Patches page.

The Download Patches page prompts you for information about the patches to download, then downloads them directly from My Oracle Support. The Merge Options section of this page defines how patches should be merged after they are downloaded.

To download patches using Patch Wizard

  1. Access Oracle Applications Manager.

    Follow the instructions in Accessing Patch Wizard to access OAM. All procedures in this section begin with the Site Map.

  2. Access the Patch Wizard home page.

    From the Site Map (Maintenance tab), click Patch Wizard under the Patching and Utilities heading.

    Site Map Page

    the picture is described in the document text

    The Patch Wizard home page appears.

    Patch Wizard Home Page

    the picture is described in the document text

  3. Submit a request to download patches.

    Click the Tasks icon for Download Patches. The Download Patches page appears.

    Download Patches Page - Top

    the picture is described in the document text

    On this page, list the patches you want to download in the Patch List field.

    Another option is to click the Details icon for a recommended patch request in the Results section of the Patch Wizard home page.

    Patch Wizard Home Page - Recommended Patches Results

    the picture is described in the document text

    The Recommended Patches Results page for the recommended patch request appears.

    Recommended Patches Results Page

    the picture is described in the document text

    Note: Before proceeding, you can use the Export All button to download a list of all the patches in CSV (Comma Separated Values) file format that can be opened with Microsoft Excel, or use the View All button to view all the patches on one page.

    Select any number of recommended patches on this page and click the Download button. This populates the Patch List field in the Download Patches page with the selected patch numbers.

  4. Set download options.

    On the Download Patches page, set Merge options and indicate information about languages and platforms. If you choose to automatically merge patches while downloading, specify the merged patch name and the merging strategy. You can select the languages and platform of the patches to download. When you provide information in this section of the page, Patch Wizard only downloads patches that match the selected languages and platform. You can also schedule the download for a future date.

  5. Submit request.

    After you have entered the patch information, click OK.

  6. Track the status of your request.

    From the main page, you can track the status of your patch request. Click the Job Status icon for Download Patches.

    Job Status Page

    the picture is described in the document text

    The Job Status page displays. If you click the Show/Hide icon corresponding to you request ID, the page displays more details. For more information about the fields and functions on this page, see: Accessing Patch Wizard.

Determining Patch Impact on System Files

Requirement: Before I apply a patch, can I see which system files will be affected?

Patch Wizard provides a Patch Impact Summary page that shows the impact of a specific patch if applied to your system. It contains the following information: Patch Impact Analysis, Direct Impact Summary, and Indirect Impact Summary. By reviewing these results, you can see detailed information about files included in a patch, as well as the effect a specific patch will have on your existing system files. For example, you can see information about total files in the patch, the number and type of files that will be installed, and which existing files will be changed. See: Accessing Patch Wizard.

To view the information on the Patch Impact Summary page

  1. Access Oracle Applications Manager.

    Follow the instructions in Accessing Patch Wizard to access OAM. All procedures in this section begin with the Site Map.

  2. Access the Patch Wizard home page.

    From the Site Map (Maintenance tab), click Patch Wizard under the Patching and Utilities heading.

    Site Map Page

    the picture is described in the document text

  3. View recommended patches results.

    From the home page, click the Details icon for an item in the Results section.

    Recommended Patches Results Page

    the picture is described in the document text

    The Recommended Patches Results page for the recommended patch request appears.

    Recommended Patches Results Page

    the picture is described in the document text

  4. Access the Patch Impact Analysis page.

    Clicking the Impact icon in the Recommended Patches Results page opens the Patch Impact Analysis page for the selected patch.

    Patch Impact Analysis Page

    the picture is described in the document text

    Many of the line items on this page are links to detailed information about the impact of the patch on the system. For example, the File Types Installed value is a link to a page that lists the file types and the number of unchanged, changed, and new files in the file system as a result of applying the selected patch. For more information, see: Patch Impact Analysis.

Creating Patch Recommendations Without an Internet Connection

Requirement: How do I use the features of creating patch recommendations if I do not have access to an Internet connection?

You can run Patch Wizard without access to an Internet connection, if necessary, by downloading the Patch Information Bundle to a system which has Internet access. Once the download is complete, copy the Patch Information Bundle file to the Patch Wizard's staging directory. Then run Patch Wizard as you normally would based on the files you copied to the staging directory.

To create recommendations without using an Internet connection

  1. Download the Patch Information Bundle to a system which has Internet access.

  2. Set up a staging directory on a system that does not have Internet access. Patch Wizard must be able to read from and write to this staging directory.

  3. Copy the Patch Information Bundle zip file to the staging directory. The zip file must be copied to a system that can access the Patch Wizard staging directory. If the staging directory is on a local disk, the zip file must be copied to the system where you run Patch Wizard. If the staging directory is on a shared (network) disk, it can be copied to any system with access to the shared disk.

  4. Run Create Recommendations as you normally would from this point.

Analyzing Applied Patches

As you apply patches, the actions taken are recorded in the Oracle E-Business Suite patch history database. You can query this database using the Oracle Applications Manager (OAM) Applied Patches feature, which provides easy access to reports based on your search criteria.

Note: Patch information is not stored in the database if the patch is applied in pre-install mode or test mode. Also, if patch application does not run successfully to completion, the associated information is neither uploaded to the patch history database nor available in the Applied Patches feature.

You enter search criteria on a search patches page, either Simple Search or Advanced Search. A summary report is displayed at the bottom of the search page.

Several detailed reports are also available, including Timing Details, Files Copied, Bug Fixes, and Action Summary. Most of these detailed reports have a standard layout. The top portion displays the criteria that were used for the search, and the bottom portion displays the results of the search.

See also Applied Patches in the OAM Help system.

Determining If a Patch Was Applied

Requirement: Can I determine if a specific patch has been applied to my Oracle E-Business Suite system?

To determine which patches were applied, enter a patch ID in the Applied Patch Check area of the Software Updates page. You can perform a simple search by entering an ID or a series of IDs separated by commas.

To determine if a patch was applied

  1. Access Oracle Applications Manager.

    Follow the instructions in Accessing Patch Wizard to access OAM.

  2. Access the Software Updates page.

    From the Applications Dashboard, click the Software Updates tab. The Software Updates page appears.

    Software Updates Page

    the picture is described in the document text

  3. Enter a patch ID.

    In the Applied Patch Check area of the Software Updates page, enter a patch ID or a series of IDs separated by commas. Your queried ID appears in the corresponding column depending on whether it has been applied.

Searching for Patch Details

Requirement: What information is available on the Patch Details report? How do I create the report?

From any Patch Summary report, you can click the Details icon for a selected row to open the Patch Details report, which displays summary information carried over from the Results portion of either the Simple Search or Advanced Search page.

This report also contains more specific information about the patch, including:

From the Patch Details page, you can also access additional information about a patch, including timing details, files copied, bug fixes, and a summary of actions performed.

See: Applied Patches. See also Applied Patches in the OAM Help.

To review patch details

  1. Access Oracle Applications Manager.

    Follow the instructions in Accessing Patch Wizard to access OAM.

  2. Create a Patch Summary report.

    From the Site Map (Maintenance tab), click Applied Patches under the Patching and Utilities heading. From either the Simple Search or Advanced Search page, enter a patch number or a date range to create a Patch Summary report. Click Go.

  3. Select the patch.

    Click the Details icon in any selected row of the Patch Summary report. The Patch Details report appears.

    Patch Details Report

    the picture is described in the document text

    The report displays patch details such as driver files, start and end dates, and platform. It also provides access to other patch details related to the driver files, such as files copied and bug fixes. You can select a driver from the list, and click one of the additional detail buttons to see other reports.

  4. View additional details.

    As an example of the details that are available for a selected driver, click Files Copied.

    Files Copied Report

    the picture is described in the document text

    For each file, the Files Copied report shows the product short name, the directory where the file was copied, the name of the file, and the version number. To view other information associated with the driver file, click the Patch Details link at the top of the page to return to the previous page.

    As another example, click the Bug Fixes button.

    Bug Fixes Report

    the picture is described in the document text

    The Bug Fixes report lists all bug fixes included in the selected driver file. It contains the bug number, the associated product, and whether the bug fix was applied. If the fix was not applied, the Remarks column explains why.

  5. View the Action Summary report.

    You can create a report that summarizes the actions of a selected driver file. Click the Patch Details link at the top of the page to return to the Patch Details page. (You can also access the Action Summary report by clicking the bug fix number on the Bug Fixes report.)

    From the Patch Details page, select a driver and click the Action Summary button.

    Action Summary Report

    the picture is described in the document text

    The Action Summary report shows more information about the driver and its actions. For definitions of the column headings, see Action Summary.

    If the driver selected contains a database portion, the Patch Summary report shows the driver actions, such as sql and exec. If the driver performed actions on the database, the Details icon is active. Click it to see the Action Details report.

Searching for Translation Patches

Requirement: My Oracle E-Business Suite system operates in multiple languages. I want to make sure translation patches have been applied successfully.

If a patch has an associated translation patch, you apply the translation patch separately. The patch history database keeps a record of all translation patches you apply.

To search for translation patches

  1. Access the Oracle Applications Manager.

    Follow the instructions in Accessing Patch Wizard to access OAM.

  2. Access the Simple Search page.

    Site Map Page

    the picture is described in the document text

    From the Site Map (Maintenance tab), click Applied Patches under the Patching and Utilities heading. The Simple Search page appears.

    Enter the search criteria. For details about using the Simple Search page, see: Determining If a Patch Was Applied, or click the OAM Help button.

  3. Specify the patch.

    On the Simple Search page, enter the ID of the translation patch in the Patch field. Click Go.

  4. Review the Patch Summary report

    All applications of the patch are displayed. If multiple translations were applied, there will be multiple rows. The Language column shows the languages applied.

    Applied Patches Page - Simple Search

    the picture is described in the document text

Viewing Applied Patches in a Report Format

Requirement: Can I review applied patches information without the OAM screens?

There may be times when you want to view applied patch history without running the Oracle Applications Manager. For example, you may need to view large amounts of data, or you may just need a list of patches without the detail provided in the OAM Patch History reports. In these cases, you can run command line scripts that list all patches applied in each patching session, all files affected by a patch, or all patches applied within a certain date range. The scripts, and a description of the reports they produce, are listed in the following table.

Patch Report Scripts
Script Name Report Content Output Format
adphrept.sql Lists patches applied in individual patching sessions, and includes details. XML
adfhrept.sql Displays information about files changed by patches. XML
adpchlst.sql Lists patches applied in a given date range. Text

The XML reports produced by adphrept.sql and adfhrept.sql can either be processed as XML or viewed as HTML.

To run a report that provides a listing of applied patches, follow the appropriate instructions in this section.

To see a list of all completed patching sessions with patch details

Run the adphrept.sql script (located in <AD_TOP>/patch/115/sql). This script produces an XML report showing individual patching sessions. If a patch was applied more than once, this report lists each application of the patch. If a merged patch was applied, it lists the merged patch by patch name. It does not list the individual patches within the merged patch.

To run adphrept.sql, use the following parameters:

<query_depth> <bug_number or ALL> <bug_product or ALL> \
<start_date_from (mm/dd/rr or ALL)> <end_date_to (mm/dd/yyyy or ALL)> \
<patchname/ALL> <patchtype/ALL> <level/ALL> <language/ALL> \
<appltop/ALL> <limit to forms server? (Y/N)> \
<limit to web server? (Y/N)> \
<limit to node server? (Y/N)> \
<limit to admin server? (Y/N)> \
<only patches that change DB? (Y/N)> <report_name>.xml

For <query_depth>, specify 1 (details of patches only), 2 (details of patches and their bug fixes only), or 3 (details of patches, bug fixes, and bug actions).

At the command prompt, enter the report command and enter values for the parameters and prompts. For example, to see complete patch details for patching sessions that were run during January 2012, enter the following, using the mm/dd/yyyy date format:

UNIX:

$ cd $AD_TOP/patch/115/sql
$ sqlplus <APPS username>/<APPS password> \
@adphrept.sql 3 ALL ALL 01/01/2012 01/31/2012 \
ALL ALL ALL ALL ALL N N N N N jan09.xml

Windows:

C:>\ cd %AD_TOP%\patch\115\sql
C:>\ sqlplus <APPS username>/<APPS password> @adphrept.sql 3 ALL ALL 01/01/2009 01/31/2009 ALL ALL ALL ALL ALL N N N N N jan09.xml

The <AD_TOP>/html directory contains the adpchrep.xsl style sheet for displaying the XML output file in HTML format. To view the XML file as HTML, copy both the adpchrep.xsl style sheet and XML output report to a directory accessible by a browser. Open the directory in your browser and click the XML filename.

To display information about files changed by patches

Run the adfhrept.sql script (located in <AD_TOP>/patch/115/sql) to produce an XML report named adfilerep.xml. Use the following parameters:

<filename> <latest file version only? (Y/N) \
<start_date (mm/dd/rr or ALL)> <end_date (mm/dd/yyyy or ALL)> \
<patchtype/ALL> <language/ALL> \
<appltop/ALL> <limit to forms server? (Y/N)> \
<limit to web server? (Y/N)> \
<only patches that change DB? (Y/N)>

At the command prompt, enter the report command and enter values for the parameters and prompts. For example, to see the complete file version history for admorgb.pls considering only patches applied in January 2013, enter the following, using mm/dd/yyyy format:

UNIX:

$ cd $AD_TOP/patch/115/sql
$ sqlplus <APPS username>/<APPS password> \
@adfhrept.sql admorgb.pls N 01/01/2013 01/31/2013 \
ALL ALL ALL N N N N N

Windows:

C:>\ cd %AD_TOP%\patch\115\sql
C:>\ sqlplus <APPS username>/<APPS password> @adfhrept.sql admorgb.pls  N 01/01/2013 01/31/2013 ALL ALL ALL N N N N N

The <AD_TOP>/html directory contains the adfilerep.xsl style sheet for displaying the XML output file in HTML format. To view the XML file as HTML, copy both the adfilerep.xsl style sheet and XML output report to a directory accessible by a browser. Open the directory in your browser and click on the XML filename.

To see a list of all patches in a given date range

The adpchlst.sql report (located in <AD_TOP>/patch/115/sql) produces a list (adpchlst.lst) of all patches in a date range, without patch detail. It differs from adphrept.sql in two ways: it lists a patch only once regardless of how many times it was applied, and it lists individual patches included within a merged patch. For example, if you combine patches 123, 124, and 125 in a merged patch called merged1, the report lists patches 123, 124, and 125, but not merged1.

At the command prompt, enter the report command and enter the date parameters in mm/dd/yyyy format. For example, to see a list of patches applied in July 2014, enter the following:

UNIX:

$ cd $AD_TOP/patch/115/sql
$ sqlplus <APPS username>/<APPS password> \
@adpchlst.sql 07/01/2014 07/31/2014

Windows:

C:>\ cd %AD_TOP%\patch\115\sql
C:>\ sqlplus <APPS username>/<APPS password> @adpchlst.sql 07/01/2014 07/31/2014

Monitoring Patches in Progress

Requirement: Can I monitor the progress of a patch while it is being applied?

Depending on the size and complexity of a patch, it may take from several minutes to several hours to completely apply it to your system. It is useful to know what a patch is currently doing and how long individual steps are taking.

The Timing Reports feature of Oracle Applications Manager allows administrators to view live information about in-progress patches, and the time taken by each patch action. To use this feature:

  1. Run adop to start the patching session.

  2. In Oracle Applications Manager, navigate to Timing Reports (Navigation: Sitemap > Maintenance > Patching and Utilities > Timing Reports).

  3. Click on the refresh icon at any time to obtain the latest status.

It is also possible to monitor progress of a patching cycle by reviewing:

Analyzing Patches Without an Internet Connection

Requirement: How do I analyze specific patches if I do not have access to an Internet connection?

You can run Patch Wizard to analyze specific patches without access to an Internet connection, if necessary, by downloading the patches to a system which has Internet access. Once the download is complete, copy the patches to the Patch Wizard's staging directory. Then run Patch Wizard as you normally would based on the files you copied to the staging directory.

To analyze specific patches without using an Internet connection

  1. Download the patch zip file(s) to a system which has Internet access.

  2. Set up a staging directory on a system that does not have Internet access. Patch Wizard must be able to read from and write to this staging directory.

  3. Copy the patch zip file(s) to the <staging directory>/ad directory, if the downloaded patch is an AD product patch. Otherwise, copy the patch zip file(s) to <staging directory>/nonad directory. The zip file(s) must be copied to a system that can access the Patch Wizard staging directory. If the staging directory is on a local disk, the zip file(s) must be copied to the system where you run Patch Wizard. If the staging directory is on a shared (network) disk, it can be copied to any system with access to the shared disk.

  4. Run Analyze Specific Patches as you normally would from this point.