1 Introduction to Zero Downtime Patching

This chapter provides an overview of Zero Downtime Patching, including the types of workflows that you can create, how the patching workflow proceeds, and how patching is reverted.

This chapter includes the following sections:

• What Is Zero Downtime Patching?

• Types of Patching Workflows

• The Patching Workflow Process

• Reverting an Update

• Rolling Out a Patched Oracle Home: Overview

• Rolling Out a New Java Version: Overview

• Rolling Out Updated Applications: Overview

• In-Memory Session Replication for ZDT Rollouts

What Is Zero Downtime Patching?

Zero Downtime Patching (ZDT Patching) automates the rollout of out-of-place patching or updates across a domain while allowing your applications to continue servicing requests. After defining your patching strategy, you can use either the WebLogic Scripting Tool (WLST) or the WebLogic Server Administration Console to orchestrate the rollout of updates across some or all of the servers in your domain.

Although WebLogic Server has supported rolling upgrades since version 9.2, the process has always been manual. ZDT Patching automates this process by using workflows that you define. You can patch or update any number of nodes in a domain with little or no manual intervention. Changes are rolled out to one node at a time, allowing a load balancer such as Oracle Traffic Director to redirect incoming traffic to the remaining nodes until the node has been updated.

Types of Patching Workflows

ZDT Patching supports the following tasks. You can create a workflow that performs any one of these tasks. You can also create a workflow that performs any combination of an Oracle home update, Java version update, and application update.

• Moving servers to a patched Oracle home:The workflow transitions the Administration Server or clusters or both to another Oracle home that has already been patched using the OPatch utility.

• Updating to a new Java version:The workflow updates the Administration Server or clusters or both to use a newly installed Java home.

• Deploying updated applications:The workflow deploys updated applications to the selected clusters.

• Performing a rolling restart of servers:The workflow sequentially restarts the Administration Server or servers in the selected clusters or both safely, including graceful shutdown of the servers and starting them up again.

• Performing a rolling restart of partitions: The workflow sequentially restarts the partitions in a cluster or in the specified resource group, including the graceful shutdown of each partition, one at a time, and starting them up again.

Prior to creating a patching workflow, you must complete the preliminary steps for each of these tasks with the exception of rolling restarts. See Preparing for Zero Downtime Patching, for more information.

The Patching Workflow Process

When you use a ZDT patching workflow to roll out an update, the rollout:

• Systematically works its way through each applicable node

• Identifies the servers on the node that are included in the rollout

• Gracefully shuts down those servers

• When switching to a patched Oracle home:

  • Backs up the existing Oracle home to a backup directory

  • Calls Node Manager to switch the contents of the current Oracle home to the contents of the specified Oracle home

• When updating to a new Java version:

  • Updates all scripts in the domain's Oracle home that contain a reference to Java home to point to the new Java home

  • Updates all scripts in the domain's home directory that contain a reference to Java home to point to the new Java home

• When updating to new application versions:

  • Locates the current directory for each application

  • Moves the current directory for each application to a backup location

  • Moves the directory for the new version of each application to the location of each original application

• Restarts each server once the update has completed on the node

The workflow executes the appropriate steps in order and monitors the success of each step. If a step fails, the workflow may attempt to retry it. If a step cannot be completed successfully, then the workflow reverts each previous step in order. The revert process can be configured to execute automatically or can be initiated manually, as described in Reverting an Update.

Reverting an Update

ZDT patching is also able to revert an update at any point in the process, even after it has completed. Updates can be reverted:

• Automatically—When creating a workflow, you can opt to have the update revert automatically if there is a failure. The update will be rolled back from the point of failure, starting with the last successfully completed step.

• Manually—While a workflow is in progress, you can stop it and revert the process at any point. The update will then be rolled back, starting with the last successfully completed step.

  After a workflow has completed, you can create a workflow to reverse the update that was made. The revert process differs slightly depending on the update. If you are reverting to the previous Oracle home, then you are provided with an option to specify that the process is a rollback. For Java and applications, to revert you can point to the previous version of Java or the application.

For more information about reverting an update, see Executing, Reverting, and Resuming Stopped Workflows.

Rolling Out a Patched Oracle Home: Overview

This section provides an overview of how to roll out a patched Oracle home to all nodes in your domain. Prior to doing the rollout, ensure that the following conditions are met:

• The domain is distributed across all nodes and stored in the same location on all nodes.

• The existing Oracle home is in the same location on all nodes.

• Node Manager is running on all nodes.

• All Managed Servers in all clusters that will be included in the rollout are running.

See ZDT Patching Restrictions, for additional requirements and restrictions. Figure 1-1 shows the sequence of operations that are performed for an Oracle home rollout on each node, regardless of whether you use OPatchAuto, WLST, or the WebLogic Server Administration Console to perform the rollout.

To roll out a patched Oracle home, perform the following tasks:

1. Create and distribute the patched Oracle home archive in either of the following ways:

   • Use OPatchAuto

   1. Create the patched Oracle home archive.

      For details, see Creating a Patched Oracle Home Archive Using OPatchAuto.

   2. Distribute the archive to all nodes to which you want to roll out the patched Oracle home.

      For details, see Distributing the Patched Archive to Each Node Using OPatchAuto.

   • Manually create and distribute the patched Oracle home archive:

   1. Use the copyBinary command to create an archive of your existing Oracle home.

      For details on this step and the next step, see Creating a Second Oracle Home.

   2. Use the pasteBinary command to create an Oracle home to be patched on a development or test system that has a domain topology similar to your production domain. This gives you an Oracle home that has the same patch level and products as you have on your production system.

   3. Use OPatch to apply the desired patch or patches to the Oracle home on your development or test system.

      For details, see Applying Patches to the Second Oracle Home, and Patching with OPatch.

   4. Test and verify the patched Oracle home.

   5. When you are satisfied that the patched Oracle home is stable, use copyBinary to create an archive of the patched Oracle home.

      For details on this and the next step, see Creating an Archive and Distributing It to Each Node.

   6. Distribute this archive to all nodes in your production system.

      Note:

      There is no need to use pasteBinary to create the archive on each node. The rollout process will create the new Oracle home on each node from the archive.

2. Create a ZDT workflow to roll out the patched Oracle home to your Administration Server. You can do this in any of the following ways:

   • Use OPatchAuto to initiate the rollout and specify Administration Server as the target.

     For details, see Using OPatchAuto to Initiate a Rollout.

   • Use the WLST rolloutOracleHome command and specify the Administration Server as the rollout target.

     For details, see Rolling Out a New Oracle Home.

   • In the WebLogic Server Administration Console, click the ZDT Control tab and navigate to the Servers tab. In the Servers tab, select the Administration Server, and then initiate and configure the workflow. You can also click the Domain tab and select the domain to initiate the workflow.

     For details, see Creating a New Workflow for a Domain, Clusters, or Servers.

3. After the workflow completes successfully, create another ZDT workflow to roll out the patched Oracle home to the clusters in your domain. You can do this in any of the following ways:

   • Use OPatchAuto to initiate the rollout and specify a cluster or a comma-separated list of clusters as the rollout target.

     For details, see Using OPatchAuto to Initiate a Rollout.

   • Use the WLST rolloutOracleHome command and specify a comma-separated list of clusters as the rollout target.

   • In the WebLogic Server Administration Console, select the ZDT Control > Clusters tab, select the clusters to which you want to roll out the Oracle home, and then initiate and configure the workflow.

   Note:

   You can combine the last two steps into one workflow by either specifying the domain as the target in the opatchauto or rolloutOracleHome command, or by initiating and configuring the workflow from the ZDT Control > Domain tab.

Figure 1-1 Oracle Home Rollout Operations

This figure shows the sequence of operations that are performed for an Oracle home rollout on each node, regardless of whether you use OPatchAuto, WLST, or the WebLogic Server Administration Console to perform the rollout.

Description of "Figure 1-1 Oracle Home Rollout Operations "

Rolling Out a New Java Version: Overview

This section provides an overview of how to roll out a new Java version to all nodes in your domain. Prior to doing the rollout, ensure that the following conditions are met:

• The domain is distributed across all nodes and is stored in the same location on all nodes.

• Oracle home must be in the same location on all nodes.

• Node Manager is running on all nodes.

• All Managed Servers in all clusters that will be included in the rollout is running.

See ZDT Patching Restrictions, for additional requirements and restrictions.

To roll out a new Java version:

1. Install the new Java version on all nodes. The full path to this Java home must be the same on all nodes.

   For more details, see Preparing to Upgrade to a New Java Version.

2. Create a ZDT workflow to roll out the new Java home to your Administration Server. You can do this in either of the following ways:

   • Use the WLST rolloutJavaHome command and specify the Administration Server as the rollout target.

     For details, see Updating Your Java Version.

   • In the WebLogic Server Administration Console, select the ZDT Control > Servers tab, select the Administration Server, and then initiate and configure the workflow.

     For details, see Creating a New Workflow for a Domain, Clusters, or Servers.

3. After the workflow completes successfully, create another ZDT workflow to roll out the new Java home to the clusters in your domain. To do this, you can either:

   • Use the WLST rolloutJavaHome command and specify a comma-separated list of clusters as the rollout target.

   • In the WebLogic Server Administration Console, select the ZDT Control > Clusters tab, select the clusters to which you want to roll out the new Java version, and then initiate and configure the workflow.

   Note:

   You can combine the last two steps into one workflow by either specifying the domain as the target in the rolloutJavaHome command or by initiating and configuring the workflow from the ZDT Control > Domain tab.

Rolling Out Updated Applications: Overview

This section provides an overview of how to roll out new application versions to Managed Server nodes, partitions, or resource groups in your domain.

ZDT now supports WebLogic Server Mutitenant to provide application rollout capabilities to both partitions and resource groups. This means that any application deployed to a resource group (partition scoped or global) or to a specific partition can now be updated without affecting other resource groups or partitions within that WebLogic Server instance. A WebLogic Server administrator can update applications defined in the resource group templates that are consumed by many partitions. They can also update applications deployed to a single partition. Partition administrators can update only those applications that are defined in their own partition. For more information about Multitenancy in WebLogic Server, see Using WebLogic Server Multitenant.

Prior to doing the rollout, ensure that the following conditions are met:

• The domain that is being updated is distributed across all nodes and must be stored in the same location on all nodes.

• Oracle Home is in the same location on all nodes.

• Node Manager is running on all nodes.

• All Managed Servers in all clusters that will be included in the rollout is running.

• An instance of the partition that is being updated must be running on more than one cluster.

See ZDT Patching Restrictions, for additional requirements and restrictions. Figure 1-2, Figure, and Figure 1-4 illustrate the scenario for patching staged, no-stage, and external staged applications, respectively. The patched application source will be moved to the appropriate application source locations for each stage type during the rollout.

To roll out new application versions to your Managed Servers:

1. Place a copy of the updated application directory as follows:

   • (Stage mode) Place a copy of each updated application directory on the domain's Administration Server.

   • (No-stage mode and external stage mode) Place a copy of each updated application directory on each node that will be affected. The directory must be the same on each node.

   For details, see The Effects of Staging Modes.

2. Create a JavaScript Object Notation (JSON) file that defines each application name, the partition name (if applicable), the resource group template name (if applicable), the path and file name for each updated application archive, and the path and file to which you want to back up the original application archive.

   For details, see Creating an Application Update JSON File.

3. Create a ZDT workflow to roll out the new application versions. To do this, you can either:

   • Use the WLST rolloutApplications command and specify a comma-separated list of clusters as the rollout target.

   • In the Administration Console, select the ZDT Control > Clusters tab, select the Clusters to which you want to roll out the applications, and then initiate and configure the workflow.

Figure 1-2 Patching Staged Applications

Description of "Figure 1-2 Patching Staged Applications"

Figure 1-3 Patching No-Stage Applications

Description of "Figure 1-3 Patching No-Stage Applications"

Figure 1-4 Patching External Staged Applications

Description of "Figure 1-4 Patching External Staged Applications"

In-Memory Session Replication for ZDT Rollouts

For web applications that use in-memory session replication, the in-memory sessions are never replicated or persisted to allow for failover. As a result web applications may lose session state due to a the sudden failure of a server or front-end misdirection causing the request to land on a server without the session.

With regard to Zero Downtime (ZDT) rollouts, when you shut down any server that holds the in-memory session, the server waits for that session to complete before shutting down. Because the default value for session timeout is 1 hour, the server may be in the SUSPENDING state for 1 hour or even longer if sessions continue to be used or updated. If you do not wait for the session to complete its life cycle, then the state is lost because in-memory sessions are neither replicated nor persisted for web applications.

If you do not want to wait for an hour or longer, then Oracle recommends that you set the shutdownTimeout argument in the WLST rolloutcommand to the time (in seconds) that you want the server to wait before shutting down. For information about using the shutdownTimeout argument, see Table 3-1.