1 Introduction to Zero Downtime Patching

Zero Downtime Patching in Oracle WebLogic Server provides a way to create various workflows to apply updates across a domain without interrupting your applications to service requests. Learn how to apply an update across a domain, revert an update, and understand the workflow process.

• What Is Zero Downtime Patching?

• Identifying a Zero Downtime Patch

• Types of Patching Workflows

• The Patching Workflow Process

• Reverting an Update

• Rolling Out a Patched Oracle Home: Overview

• Rolling Out a New Java Version: Overview

• Rolling Out Updated Applications: Overview

• In-Memory Session Replication for ZDT Rollouts

What Is Zero Downtime Patching?

Zero Downtime Patching (ZDT Patching) automates the rollout of out-of-place patching or updates across a domain while allowing your applications to continue servicing requests. After defining your patching strategy, you can use either the WebLogic Scripting Tool (WLST) or the WebLogic Server Administration Console to orchestrate the rollout of updates across some or all the servers in your domain.

Although WebLogic Server has supported rolling upgrades since version 9.2, the process has always been manual. ZDT Patching automates this process by using workflows that you define. You can patch or update any number of nodes in a domain with little or no manual intervention. Changes are rolled out to one node at a time, allowing a load balancer such as Oracle Traffic Director to redirect incoming traffic to the remaining nodes until the node has been updated.

ZDT Patching also provides support for custom hooks. The ZDT custom hooks provide a flexible mechanism for modifying the patching workflow by executing additional scripts at specific points in the patching rollout. This feature allows application developers and administrators to include any operation that is specific to a particular type of rollout but that may not be applicable to the base patching workflow. See Preparing to Modify Rollouts Using Custom Hooks for information about using this feature.

Identifying a Zero Downtime Patch

You can identify a ZDT patch by the value of the patch uptime option in the patch metadata.

After you download a patch, open up patchdeploy.xml in the PATCH_HOME/etc/config directory, where PATCH_HOME is the location of the patch directory that contains the patch.

If the value of patch-uptime-option is FMW_ROLLING_ORACLE_HOME, as shown in the following example:

<patch-uptime-option>FMW_ROLLING_ORACLE_HOME<patch-uptime-option>

Or the value is FMW_ROLLING_SESSION:

<patch-uptime-option>FMW_ROLLING_SESSION<patch-uptime-option>

Then, the patch is suitable for ZDT patching.

If FMW_ROLLING_ORACLE_HOME or FMW_ROLLING_SESSION does not appear in the patch metadata, then you know that the patch is not suitable for ZDT patching. As a result, the patch is not compatible with a ZDT patch plan.

Types of Patching Workflows

You can create different types of workflows with ZDT Patching, for moving servers to a patched Oracle home, updating to a new Java version, deploying updated applications, and more.

You can create a workflow that performs any one of these tasks. You can also create a workflow that performs any combination of an Oracle home update, Java version update, and application update.

• Moving servers to a patched Oracle home:The workflow transitions the Administration Server or clusters or both to another Oracle home that has already been patched using the OPatch utility.

• Updating to a new Java version:The workflow updates the Administration Server or clusters or both to use a newly installed Java home.

• Deploying updated applications:The workflow deploys updated applications to the selected clusters.

• Performing a rolling restart of servers:The workflow sequentially restarts the Administration Server or servers in the selected clusters or both safely, including graceful shutdown of the servers and starting them up again.

• Performing a rolling restart of partitions: The workflow sequentially restarts the partitions in a cluster or in the specified resource group, including the graceful shutdown of each partition, one at a time, and starting them up again.

  Note:

  WebLogic Server Multitenant domain partitions, resource groups, resource group templates, virtual targets, and Resource Consumption Management are deprecated in WebLogic Server 12.2.1.4.0 and will be removed in the next release.

Prior to creating a patching workflow, you must complete the preliminary steps for each of these tasks with the exception of rolling restarts. See Preparing for Zero Downtime Patching.

The Patching Workflow Process

A ZDT Patching workflow constitutes a systematic set of steps that are executed in a particular order to roll out an update.

When you use a ZDT patching workflow to roll out an update, the rollout:

• Systematically works its way through each applicable node

• Identifies the servers on the node that are included in the rollout

• Gracefully shuts down those servers

• When switching to a patched Oracle home:

  • Backs up the existing Oracle home to a backup directory

  • Calls Node Manager to switch the contents of the current Oracle home to the contents of the specified Oracle home

• When updating to a new Java version:

  • Updates all scripts in the domain's Oracle home that contain a reference to Java home to point to the new Java home

  • Updates all scripts in the domain's home directory that contain a reference to Java home to point to the new Java home

• When updating to new application versions:

  • Locates the current directory for each application

  • Moves the current directory for each application to a backup location

  • Moves the directory for the new version of each application to the location of each original application

• Restarts each server once the update has completed on the node

The workflow executes the appropriate steps in order and monitors the success of each step. If a step fails, the workflow may attempt to retry it. If a step cannot be completed successfully, then the workflow reverts each previous step in order. Updates can be reverted either automatically or can be initiated manually, as described in Reverting an Update.

Reverting an Update

During the process of the patching workflow, ZDT Patching monitors the success and failure of each step and provides the capability to revert to the previous step. You can configure the revert process to execute automatically, or initiate the process manually.

ZDT Patching is able to revert an update at any point in the process, even after it has completed. Updates can be reverted:

• Automatically—When creating a workflow, you can opt to have the update revert automatically if there is a failure. The update will be rolled back from the point of failure, starting with the last successfully completed step.

• Manually—While a workflow is in progress, you can stop it and revert the process at any point. The update will then be rolled back, starting with the last successfully completed step.

  After a workflow has completed, you can create a workflow to reverse the update that was made. The revert process differs slightly depending on the update. If you are reverting to the previous Oracle home, then you are provided with an option to specify that the process is a rollback. For Java and applications, to revert you can point to the previous version of Java or the application.

For information about reverting an update, see Executing, Reverting, and Resuming Stopped Workflows.

Rolling Out a Patched Oracle Home: Overview

You can roll out a patched Oracle home across your domain by using WLST or the WebLogic Server Administration Console while ensuring that your application continues servicing requests.

Note:

OPatchAutoFMW (installed in OPatch/auto/fmw directory) is deprecated and is automatically removed when you update to OPatch 13.9.4.2.2 or later. You can continue to use OPatchAutoCore (installed in OPatch/auto/core directory) for auto updates during Fusion Middleware installation.

Before rolling out a patched Oracle home to all nodes in your domain, ensure that the following conditions are met:

• The domain is distributed across all nodes and stored in the same location on all nodes.

• The existing Oracle home is in the same location on all nodes.

• Node Manager is running on all nodes.

• All Managed Servers in all clusters that will be included in the rollout are running.

See ZDT Patching Restrictions, for additional requirements and restrictions. intro.html#GUID-806B30E5-52E7-49EC-A65F-3B240377630F__BHCBICFD shows the sequence of operations that are performed for an Oracle home rollout on each node, regardless of whether you use WLST or the WebLogic Server Administration Console to perform the rollout.

To roll out a patched Oracle home, perform the following tasks:

1. Create and distribute the patched Oracle home archive.

   Manually create and distribute the patched Oracle home archive:

   1. Use the copyBinary command to create an archive of your existing Oracle home.

      For details on this step and the next step, see Creating a Second Oracle Home.

   2. Use the pasteBinary command to create an Oracle home to be patched on a development or test system that has a domain topology similar to your production domain. This gives you an Oracle home that has the same patch level and products as you have on your production system.

   3. Use OPatch to apply the desired patch or patches to the Oracle home on your development or test system.

      See Applying Patches to the Second Oracle Home.

   4. Test and verify the patched Oracle home.

   5. When you are satisfied that the patched Oracle home is stable, use copyBinary to create an archive of the patched Oracle home.

      For details on this and the next step, see Creating an Archive and Distributing It to Each Node.

   6. Distribute this archive to all nodes in your production system.

      Note:

      There is no need to use pasteBinary to create the archive on each node. The rollout process will create the new Oracle home on each node from the archive.

2. Create a ZDT workflow to roll out the patched Oracle home to your Administration Server. You can do this in any of the following ways:

   • Use the WLST rolloutOracleHome command and specify the Administration Server as the rollout target.

     See Rolling Out a New Oracle Home.

   • In the WebLogic Server Administration Console, click the ZDT Control tab and navigate to the Servers tab. In the Servers tab, select the Administration Server, and then initiate and configure the workflow. You can also click the Domain tab and select the domain to initiate the workflow.

     See Creating a New Workflow for a Domain, Clusters, or Servers.

3. After the workflow completes successfully, create another ZDT workflow to roll out the patched Oracle home to the clusters in your domain. You can do this in any of the following ways:

   • Use the WLST rolloutOracleHome command and specify a comma-separated list of clusters as the rollout target.

   • In the WebLogic Server Administration Console, select the ZDT Control > Clusters tab, select the clusters to which you want to roll out the Oracle home, and then initiate and configure the workflow.

   Note:

   You can combine the last two steps into one workflow by specifying the domain as the target in the rolloutOracleHome command, or by initiating and configuring the workflow from the ZDT Control > Domain tab.

Figure 1-1 Oracle Home Rollout Operations

This figure shows the sequence of operations that are performed for an Oracle home rollout on each node, regardless of whether you use WLST or the WebLogic Server Administration Console to perform the rollout.

Description of "Figure 1-1 Oracle Home Rollout Operations "

Rolling Out a New Java Version: Overview

You can roll out a new Java version across your domain without affecting the continuity of servicing requests during the patching process. Use WLST or the WebLogic Server Administration Console to roll out updates to Java home.

Before rolling out a new Java version to all nodes in your domain, ensure that the following conditions are met:

• The domain is distributed across all nodes and is stored in the same location on all nodes.

• Oracle home must be in the same location on all nodes.

• Node Manager is running on all nodes.

• All Managed Servers in all clusters that will be included in the rollout is running.

See ZDT Patching Restrictions, for additional requirements and restrictions.

To roll out a new Java version:

1. Install the new Java version on all nodes. The full path to this Java home must be the same on all nodes.

   See Preparing to Upgrade to a New Java Version.

2. Create a ZDT workflow to roll out the new Java home to your Administration Server. You can do this in either of the following ways:

   • Use the WLST rolloutJavaHome command and specify the Administration Server as the rollout target.

     See Updating Your Java Version.

   • In the WebLogic Server Administration Console, select the ZDT Control > Servers tab, select the Administration Server, and then initiate and configure the workflow.

     See Creating a New Workflow for a Domain, Clusters, or Servers.

3. After the workflow completes successfully, create another ZDT workflow to roll out the new Java home to the clusters in your domain. To do this, you can either:

   • Use the WLST rolloutJavaHome command and specify a comma-separated list of clusters as the rollout target.

   • In the WebLogic Server Administration Console, select the ZDT Control > Clusters tab, select the clusters to which you want to roll out the new Java version, and then initiate and configure the workflow.

   Note:

   You can combine the last two steps into one workflow by either specifying the domain as the target in the rolloutJavaHome command or by initiating and configuring the workflow from the ZDT Control > Domain tab.

Rolling Out Updated Applications: Overview

ZDT provides the ability to update applications deployed to your domain without causing the application to suffer downtime. Use WLST or the WebLogic Server Administration Console to roll out application updates.

Note:

WebLogic Server Multitenant domain partitions, resource groups, resource group templates, virtual targets, and Resource Consumption Management are deprecated in WebLogic Server 12.2.1.4.0 and will be removed in the next release.

This section provides an overview of how to roll out new application versions to Managed Server nodes, partitions, or resource groups in your domain.

ZDT supports WebLogic Server Mutitenant to provide application rollout capabilities to both partitions and resource groups. This means that any application deployed to a resource group (partition scoped or global) or to a specific partition can now be updated without affecting other resource groups or partitions within that WebLogic Server instance. A WebLogic Server administrator can update applications defined in the resource group templates that are consumed by many partitions. They can also update applications deployed to a single partition. Partition administrators can update only those applications that are defined in their own partition. For more information about Multitenancy in WebLogic Server, see Oracle WebLogic Server Multitenant in Using Oracle WebLogic Server Multitenant (Deprecated).

Prior to doing the rollout, ensure that the following conditions are met:

• The domain that is being updated is distributed across all nodes and must be stored in the same location on all nodes.

• Oracle Home is in the same location on all nodes.

• Node Manager is running on all nodes.

• All Managed Servers in all clusters that will be included in the rollout is running.

• An instance of the partition that is being updated must be running on more than one cluster.

Note:

WebLogic Server does not support the rollout of applications deployed to the Administration Server. Applications deployed to the Administration Server cannot be updated without downtime because session replication can be applied only to clustered instances, whereas Administration Server is a standalone instance.

See ZDT Patching Restrictions, for additional requirements and restrictions. Figure 1–2, Figure, and Figure 1–4 illustrate the scenario for patching staged, no-stage, and external staged applications, respectively. The patched application source will be moved to the appropriate application source locations for each stage type during the rollout.

To roll out new application versions to your Managed Servers:

1. Place a copy of the updated application directory as follows:

   • (Stage mode) Place a copy of each updated application directory on the domain's Administration Server.

   • (No-stage mode and external stage mode) Place a copy of each updated application directory on each node that will be affected. The directory must be the same on each node.

   See The Effects of Staging Modes.

2. Create a JavaScript Object Notation (JSON) file that defines each application name, the partition name (if applicable), the resource group template name (if applicable), the path and file name for each updated application archive, and the path and file to which you want to back up the original application archive.

   See Creating an Application Update JSON File.

3. Create a ZDT workflow to roll out the new application versions. To do this, you can either:

   • Use the WLST rolloutApplications command and specify a comma-separated list of clusters as the rollout target.

   • In the Administration Console, select the ZDT Control > Clusters tab, select the Clusters to which you want to roll out the applications, and then initiate and configure the workflow.

Figure 1-2 Patching Staged Applications

Description of "Figure 1-2 Patching Staged Applications"

Figure 1-3 Patching No-Stage Applications

Description of "Figure 1-3 Patching No-Stage Applications"

Figure 1-4 Patching External Staged Applications

Description of "Figure 1-4 Patching External Staged Applications"

In-Memory Session Replication for ZDT Rollouts

During ZDT rollouts, the forceful shutdown of a server could lead to loss of in-memory sessions. To avoid any loss of session data, set the rollout command to allow time for the graceful shutdown of the server instance before shutting it down forcefully.

For web applications that use in-memory session replication, the in-memory sessions are never replicated or persisted to allow for failover. As a result web applications may lose session state due to the sudden failure of a server or front-end misdirection causing the request to land on a server without the session.

With regard to Zero Downtime (ZDT) rollouts, when you shut down any server that holds the in-memory session, the server waits for that session to complete before shutting down. Because the default value for session timeout is 1 hour, the server may be in the SUSPENDING state for 1 hour or even longer if sessions continue to be used or updated. If you do not wait for the session to complete its life cycle, then the state is lost because in-memory sessions are neither replicated nor persisted for web applications.

If you do not want to wait for an hour or longer, then Oracle recommends that you set the shutdownTimeout argument in the WLST rolloutcommand to the time (in seconds) that you want the server to wait before shutting down. For information about using the shutdownTimeout argument, see configuring_patching.html#GUID-138B2554-6D70-4C29-8F36-D0B138EAD749__CHDEADAF.