3 Configuring and Monitoring Workflows

Configure ZDT Patching workflows to rollout a patched Oracle home, upgrade to a new Java version, update the applications on your managed servers, or a combination of these tasks. Using WLST or WebLogic Server Administration Console to create and monitor workflows.

This chapter describes how to configure and monitor a patching workflow that moves Managed Servers to a patched Oracle home, updates the Java version on your Managed Servers, updates the applications on your Managed Servers, or any combination of these update tasks.

Note:

Before initiating the update process, you must have completed all appropriate preparation steps for the type of update you are doing, as described in Preparing for Zero Downtime Patching.

For Windows-based domains, before initiating a workflow to update an Oracle home, on each node, ensure that there are no locked directories or files in the Oracle home being updated, as this can prevent the Oracle home from being moved to the specified backup directory. A directory can be locked by something as simple as having a DOS command window open to that directory. A file can be locked by having it open in an application.

Strategies for Rolling Out a Patched Oracle Home

You must roll out the patched Oracle home first to the Administration Server before rolling it out to the targeted clusters. You can do this by either using two sequential workflows or by using a single workflow.

When you roll out a new Oracle home using either WLST or the Administration Console, you must ensure that the patched Oracle home is first rolled out to the Administration Server. There are two approaches you can take to do this:

  • Use one workflow to roll out the patched Oracle home to the Administration Server, and then use a second workflow to roll out the patched Oracle home to your clusters. Oracle recommends using this approach, but it is not required.

    In this scenario:

    • If using WLST, you would execute either the rolloutOracleHome or rolloutUpdate command, and specify the name of the Administration Server as the target. You would then execute rolloutOracleHome or rolloutUpdate again, and specify cluster targets.

    • If using the WebLogic Server Administration Console, you would create one workflow from the Servers tab and select your Administration Server as the target. After that workflow completes, you would create a second workflow from the Clusters tab and select the clusters to include.

  • Use only one workflow to roll out the patched Oracle home to the entire domain. The workflow will automatically roll out the patched Oracle home first before rolling it out to the target clusters.

    In this scenario:

    • If using WLST, you would execute either the rolloutOracleHome or rolloutUpdate command, and specify the domain name as the target.

    • If using the Administration Console, you would create one workflow from the Domain tab.

Starting the Administration Server

Before you initiate the rollout operation, it is important that you start the Administration Server using the Node Manager. If there is no Node Manager configured for the Administration Server, then you can start the Administration Server by using the startWebLogic script.

If the Administration Server will be included in a workflow, then you can start the Administration server using either the startWebLogic script or the Node Manager. The Administration Server will be automatically restarted during the rollout operation if the specified target for the rollout is a domain. However, when the rollout operation restarts the Administration Server, you might experience a brief downtime when you will not be able to connect to either WLST or Administration Console. As a workaround, you must wait and then reconnect when the Administration Server has reached the RUNNING state in order to receive updates on the progress of the rollout operation.

To start the Administration Server before you initiate the rollout operation, you can start the Administration server in one of the following ways:

  • Using the startWebLogic script

    If there is no Node Manager configured for the Administration Server, then you can start the Administration Server by using the startWebLogic script. To start the Administration Server using this script, see Starting an Administration Server with a Startup Script in Administering Server Startup and Shutdown for Oracle WebLogic Server.

  • Using the Node Manager

    If a Node Manager is configured for the Administration Server, then you must start the Administration Server using the Node Manager. To start the Administration Server using the Node Manager, perform the following steps:

  1. If the Administration Server is running and was started using the startWebLogic script in the domain home, use the stopWebLogic command to shut it down:

    UNIX

    cd domain_home/bin
    ./stopWebLogic.sh
    
    

    Windows

    cd domain_home\bin
    stopWebLogic.cmd
    
  2. Ensure that Node Manager is running on the host.
  3. Start WLST. See Invoking WLST in Understanding the WebLogic Scripting Tool.
  4. Use the nmConnect command to establish a Node Manager session. For example, use the following command to connect to the domain mydomain located in /domains/mydomain using SSL, where the NodeManager port is 5556:
    wls:/myserver/serverConfig> nmConnect('username', 'password, 'localhost',
    '5556', 'mydomain', '/domains/mydomain','ssl') 
    
  5. After successfully connecting, run the nmStart command. For example, use the following command if the Administration Server is called AdminServer and the domain is located in /domains/mydomain:
    nmStart('AdminServer', '/domains/mydomain')
    

For more information, see Starting the Administration Server Using Node Manager in Administering Node Manager for Oracle WebLogic Server.

Using OPatchAuto to Initiate, Revert, and Resume Rollouts

You can use the OPatchAuto tool to automate the rollout workflows for applying a patched Oracle home to the nodes in your domain. You can initiate, revert, and resume rollouts using OPatchAuto.

The following sections describe how to initiate and monitor rollout workflows for applying a patched Oracle home to the nodes in your domain. You can use this approach only if the workflow is applying a patched Oracle home. If you want to include a Java version update or application update in the workflow, then you must use WLST or the Administration Console.

Note:

When using OPatchAuto to initiate a workflow, you must use the Administration Console to monitor the progress of the workflow. See Monitoring and Managing Workflows.

Using OPatchAuto to Initiate a Rollout

Use the following commands to initiate a workflow using OPatchAuto:

cd ORACLE_HOME/OPatch/auto/core/bin
opatchauto.sh apply -plan wls-zdt -image-location path 
-wls-zdt-host adminserver:port -wls-zdt-target target 
-wls-zdt-remote-image path -wallet path -walletPassword password

The following table describes the parameters in the opatchauto applycommand:

Parameter Description

-plan

Indicates the type of operation to be performed by opatchauto apply. For rolling out a patched Oracle home for ZDT, always specify wls-zdt as the value for this parameter.

-image-location path

Specify the full path and file name of the image JAR file that contains the patched Oracle home to use for the rollout. For example:

-image-location /u01/images/rollout-OH-image.jar

-wls-zdt-host adminserver:port

Specify the Administration Server host name and port number for the domain you are rolling out the patched Oracle home to.

-wls-zdt-target target

For target, specify the target for the rollout. This can be:

  • A domain name

  • A cluster name

  • A comma-separated list of clusters

When rolling out the archive, you must specify the same target as you specified when you distributed the archive that you are using for the rollout.

-wls-zdt-backup path

The full path to the directory to use for backing up your existing Oracle home. For example:

-wls-zdt-backup /u01/images/rollout-OH-image.jar

-wls-zdt-remote-image path

The full path to the archive file that you want to create on each node to be included in the ZDT rollout. This does not have to be the same file name as the original archive. For example:

-wls-zdt-remote-image /u01/images/rollout-OH-backup

-wallet path

The full path to a wallet directory that was created using configWallet.sh or configWallet.cmd. For example:

-wallet $HOME/wallet

-walletPassword password

The password for the specified wallet, if needed. For example:

-walletPassword mypassword

Using OPatchAuto to Resume a Failed Rollout

If an Oracle home rollout failed and you want to resume it, use the following commands:

cd ORACLE_HOME/OPatch/auto/core/bin
opatchauto.sh resume -session workflow_id -walletPassword password

The following table describes the parameters in the opatchauto resumecommand:

Parameter Description

-session -workflow_id

Specify the workflow ID of the workflow to roll back.

-wallet path

The full path to a wallet directory that was created using configWallet.sh or configWallet.cmd. For example:

-wallet $HOME/wallet

-walletPassword password

The password for the specified wallet, if needed. For example:

-walletPassword mypassword

Using OPatchAuto to Revert a Rollout

You can use OPatchAuto to revert a rollout that has failed or stopped, as well as to revert a completed workflow. If the rollout has failed or stopped, OPatchAuto will revert it starting with the last successfully completed step. If the rollout has completed, then OPatchAuto will initiate a new rollout, using the backed up Oracle home as the Oracle home to roll out.

Use the following commands to revert a rollout:

cd ORACLE_HOME/OPatch/auto/core/bin
opatchauto.sh rollback -session workflow_id -walletPassword password

The following table describes the parameters in the opatchauto rollbackcommand:

Parameter Description

-session -workflow_id

Specify the workflow ID of the workflow to roll back.

-wallet path

The full path to a wallet directory that was created using configWallet.sh or configWallet.cmd. For example:

-wallet $HOME/wallet

-walletPassword password

The password for the specified wallet, if needed. For example:

-walletPassword mypassword

Using WLST to Initiate and Monitor Workflows

You can use the ZDT specific WLST commands for rolling out a patched Oracle home, a new Java version or a combination of both, or new application versions.

This section describes the WLST commands that you can use to initiate workflows to update your Managed Servers, partitions, or resource groups, and provides sample WLST scripts demonstrating various workflow (rollout) scenarios.

Note:

When using the WLST rolloutOracleHome or rolloutUpdate commands to initiate a rollout of a new Oracle home for a Windows-based domain, you cannot run WLST from any Oracle home that will be updated as part of the workflow. For more information, see ZDT Patching Restrictions.

Use the following WLST commands to perform automated rolling updates of your servers. You must execute these commands from the Administration Server for the target domain.

  • rolloutOracleHome — Rolls out a patched Oracle home to your Managed Servers or reverts your Managed Servers to a previous Oracle home. The patched Oracle home archive that you use in this command can be one that was created either using opatchauto or the copyBinary and pasteBinary commands.

  • rolloutJavaHome — Updates your Managed Servers to use a new Java version.

  • rolloutUpdate — Updates your Managed Servers to use a patched Oracle home and a new Java version. The patched Oracle home archive that you use in this command can be one that was created either using opatchauto or the copyBinary and pasteBinary commands.

  • rolloutApplications—Updates specified applications that are running on your Managed Servers, partitions, or resource groups.

Note:

When specifying paths for Windows in rollout commands, you must use backslashes instead of forward slashes. To avoid unnecessary errors, ensure that the backslashes are escaped. (For example, C:\\myhome\\files\\apps.json). For more information, see Syntax for WLST Commands in Understanding the WebLogic Scripting Tool.

When you execute one of these WLST commands, the command determines which servers need to be updated and in which order, and creates a patching workflow that will update them safely. This workflow includes:

  • Performing a graceful shutdown of Managed Servers, partitions, or resource groups one at a time. This does not include Managed Servers that are currently in ADMIN or STANDBY mode. This includes migration of singleton services if the migrationProperties option is included in the rollout command. The ADMIN and STANDBY modes are not supported for rolling out application updates to partitions or resource groups.

  • Replacing the Oracle home directory (if applicable)

  • Replacing the Java home directory (if applicable)

  • Replacing application directories (if applicable)

  • Restarting Node Manager on the node

  • Restarting the Managed Servers on the node

Table 3-1 describes the parameters available for the WLSTrolloutcommands.

Table 3-1 Arguments for WLST rollout Commands

Argument Description

target

Required for all rollout commands.

Specifies which Managed Servers, partitions, or resource groups will be included in the update. target can be one of:

partition_name — Specify a partition name or a comma-separated list of partition names in the rolloutApplications command if you want to roll out application updates to specific partitions. Note that the partitions that you specify as targets must be the same partitions that you specify in the application update JSON file.

domain_name — Specify a domain name as the target if you want the Administration Server and all Managed Servers in that domain to be updated. You must also specify the domain name as the target in the rolloutApplications command if you want to roll out application updates to resource groups. Additionally, you must also specify the resource group template name in the application update JSON file if you want to roll out application updates to resource groups.

clusters — Specify a cluster name or a comma-separated list of cluster names if you want to update all Managed Servers in the specified cluster or clusters, but not Managed Servers in other clusters.

servers — Specify a server name or a comma-separated list of server names if you only want to update those Managed Servers. Note that the servers you specify must still be part of a cluster; they cannot be unclustered servers.

Note: Typically, you should specify a server target only when updating the Administration Server. Oracle recommends that you not update individual Managed Servers in most cases as sessions may not be preserved and downtime for users may not be avoided. However, one situation in which you can safely specify Managed Server targets is if you have added one or more new Managed Servers and they are not at the same Java version as your other Managed Servers.

rolloutOracleHome

Applies only to and is required for the rolloutOracleHome command.

Specifies the location of the Oracle home archive (JAR file) or local Oracle home directory to roll out, thereby replacing the existing Oracle home.

backupOracleHome

Applies only to and is required for the rolloutOracleHome command.

Specifies the full path of the directory to which the existing Oracle home will be moved. This effectively renames the original Oracle home. For example, if your original Oracle home is /u01/Oracle_Homeand you specify /u01/Oracle_Home_backupfor this parameter, /u01/Oracle_Homewill be moved (renamed) to /u01/Oracle_Home_backup.

isRollback

Optional. Applies only to the rolloutOracleHome and rolloutUpdate commands.

javaHome

Applies to and is required for the rolloutJavaHome command. Optionally, this argument may be required by the rolloutUpdate command.

Specifies the location of the new Java home to use.

applicationProperties

Applies to and is required for the rolloutApplications command. Optionally, this argument may be required by the rolloutUpdate command.

Specifies the full path to the JSON file that defines one or more application names, application archive locations, and application backup locations.

options

The following options can be included in rollout commands.

  • isDryRun — If TRUE, the workflow operation will be evaluated but not executed. The default is FALSE.

  • autoRevertOnFailure — If TRUE, the workflow operation should automatically revert on failure. If FALSE, the workflow operation will stop on a failure and you can resume or revert it. The default is TRUE.

  • isSessionCompatible — This option is applicable to all rolloutcommands, as it affects rollout time regardless of whether the rollout impacts session handling.

    The default is FALSE, which means that the very last server to be updated on each cluster will wait for all existing sessions to complete. This ensures that a compatible server is available in the cluster to handle sessions that must be served by a Managed Server that is still running on the existing version.

    If set to TRUE, this indicates that the session state in servers is 100% compatible between the existing version and the new version. Therefore, the last Managed Server in the update sequence in a cluster will shut down without waiting for all existing sessions to complete.

    Oracle recommends that you set this to FALSE unless you are absolutely sure that the session state is identical. This may cause the rollout to take longer due to the wait for session completion.

    Note: Serialization and deserialization in WebLogic Server differs slightly from Java serialization and deserialization. Therefore, additional fields on classes may result in a session being incompatible with servers on the new version, requiring that they be served by a server on the existing version. For example, a User class that adds a field such as Information will cause that session to be incompatible between versions.

  • migrationProperties — The full path to a JSON file that defines singleton service migrations to be performed during the rollout. For more information about this file and service migration, see Preparing to Migrate Singleton Services.

  • shutdownTimeout — Time (in seconds) WLST waits for a server to shut down gracefully before shutting it down forcefully. The forceful shutdown of servers may cause undesirable consequences, such as loss of session data and loss of in-flight transactions. A value of less than 1 second is ignored.

    If isSessionCompatible is set to TRUE, then the shutdownTimeout option defaults to zero, which means that WLST waits forever for the server to shut down gracefully.

    If isSessionCompatible is set to FALSE, then the user must specify a value for the shutdownTimeout option. Oracle recommends that you specify a value that gives typical applications plenty of time to complete. Because different applications have different behaviors, this value must be decided by the user.

  • DelayBetweenNodes — Use this option to specify the number of seconds to wait between the shutdown of servers on one node and the shutdown of servers on the next node in the workflow. This delay allows for:

    • The servers on the first node to be restarted and join the cluster

    • The load balancer to evenly distribute traffic

    • Any slow (lazy) stateful session bean clients to continue making requests before shutdown of the servers on the next node begins

    If not specified, this value defaults to 60 seconds. If you are not concerned about the lazy stateful session bean clients, you can include this option and set it to a lower value.

  • coherenceServiceHATarget — Use this option to specify the High Availability (HA) Status of Coherence services on a managed Coherence server which must be met before the server is shutdown. The ZDT workflow checks and waits until all Coherence services attain the specified status. The rollout workflow can prevent cache data loss by waiting until the HA Status is met. The valid values are none, machine-safe, and node-safe. A value of machine-safe is generally preferred and ensures that a machine loss during the rollout process does not result in data loss. A value of node-safe ensures that loss to a single Coherence node does not result in data loss.

  • coherenceServiceHAWaitTimeout — Use this option to specify the amount of time to wait for the Coherence HA Status task in the workflow. If the HA Status is not met within the specified time, then the task times-out. The task completes and managed Coherence servers are shutdown as soon as the HA Status is met within the specified time. The default value is 60 seconds.

You can also use WLST to monitor the progress of a workflow. For more information, see Monitoring Workflow Progress.

Rolling Out a New Oracle Home

Use the rolloutOracleHome command if you only want to do one of the following tasks:

  • Update your Administration Server to use a patched Oracle home.

  • Update your entire domain (Administration Server and clustered Managed Servers) to use a patched Oracle home.

  • Update clustered Managed Servers to use a patched Oracle home.

  • Revert your Administration Server, clustered Managed Servers, or domain to use the previous unpatched Oracle home.

rolloutOracleHome has the following syntax:

rolloutOracleHome(target, rolloutOracleHome, backupOracleHome, [isRollback], [options])

This command supports the isDryRun, autoRevertOnFailure, and isSessionCompatible options. For information about these options, see Using WLST to Initiate and Monitor Workflows.

The following example shows how to roll out a new Oracle home to the domain mydomain. The JAR file for the patched Oracle home is located at /net/wls/wls_patched.jar. The original Oracle home will be moved (renamed) to /u01/Oracle_Home_backup. The process will not automatically revert if it fails.

connect('adminname', 'adminpassword', 't3://hostname:port')
domain='/domains/mydomain'
progress=rolloutOracleHome(domain, '/net/wls/wls_patched.jar', 
'/u01/Oracle_Home_backup', autoRevertOnFailure=FALSE)

Note:

Specifying a local Oracle home directory in the rolloutOracleHome command is not supported when you are rolling out a new Oracle home. For more information, see ZDT Patching Restrictions..

Updating Your Java Version

Use the rolloutJavaHome command if you only want to do one of the following tasks:

  • Update your Administration Server to use a new Java version.

  • Update your entire domain (Administration Server and Managed Servers) to use a new Java version.

  • Update your Managed Servers to use a new Java version.

  • Revert your Administration Server, Managed Servers, or domain to use the previous Java version.

rolloutJavaHome has the following syntax:

rolloutJavaHome(target, javaHome, [options])

This command supports the isDryRun and autoRevertOnFailure options. For information about these options, see Using WLST to Initiate and Monitor Workflows.

The following example shows how to roll out a new Java home to clusters Cluster1, Cluster2, Cluster3. The new Java home location is /u01/jdk1.8.0_50. The autoRevertOnFailure option is not included in this example; therefore, the workflow will automatically revert if the process fails.

connect('adminname', 'adminpassword', 't3://hostname:port')
clusters='Cluster1,Cluster2,Cluster3'
progress=rolloutJavaHome(clusters, '/u01/jdk1.8.0_50')

Updating Both Oracle Home and the Java Version

Use the rolloutUpdate command if you only want to do one of the following tasks:

  • Update your Administration Server to use both a patched Oracle home and a new Java version.

  • Update your entire domain (Administration Server and clustered Managed Servers) to use both a patched Oracle home and a new Java version.

  • Update your Managed Servers to use both a patched Oracle home and a new Java version.

  • Revert your Administration Server, Managed Servers, or domain to the previous Oracle home and previous Java version.

rolloutUpdate has the following syntax:

rolloutUpdate(target, rolloutOracleHome, backupOracleHome, [isRollback], [javaHome], [options])

This command supports the isDryRun, autoRevertOnFailure, and isSessionCompatible options. For information about these options, see Using WLST to Initiate and Monitor Workflows.

The following example shows how to roll out a new Oracle home and a new Java home to the Administration Server. The JAR file for the patched Oracle home is located at /net/wls/wls_patched.jar. The original Oracle home will be moved (renamed) to /u01/Oracle_Home_backup. The new Java home location is /u01/jdk1.8.0_50. The autoRevertOnFailure option is not included in this example; therefore, the workflow will automatically revert if the process fails.

connect('adminname', 'adminpassword', 't3://hostname:port')
server='AdminServer'
progress=rolloutUpdate(server, '/net/wls/wls_patched.jar', 
'/u01/Oracle_Home_backup', '/u01/jdk1.8.0_50')

Rolling Out Updated Applications

Use the rolloutApplications command if you want to do one of the following tasks:

  • Update your Managed Servers to use a new version of one or more applications.

  • Revert your Managed Servers to the previous version of one or more applications.

  • Update your partition to use a new version of one or more applications.

  • Update your resource group to use a new version of one or more applications.

rolloutApplications has the following syntax:

rolloutApplications(target, applicationProperties, [options])

This command supports the isDryRun, autoRevertOnFailure, and isSessionCompatible options. For information about these options, see Using WLST to Initiate and Monitor Workflows.

The following example shows how to roll out the applications defined in the JSON-formatted application properties file /u01/scratch/app_update.json to all clusters Cluster1, Cluster2, Cluster3 on a UNIX system.

connect('adminname', 'adminpassword', 't3://hostname:port')
clusters='Cluster1,Cluster2,Cluster3'
progress=rolloutApplications(clusters, '/u01/scratch/app_update.json')

Reverting to the Previous Oracle Home, Java Home, or Applications

After a successful rollout, if you want to roll back to the previous Oracle home, Java home, or application version, you must perform the following two steps to complete the rollback operation:

  • Use the rolloutUpdate command to roll back to the previous Oracle home and Java home. However, you must keep the following restrictions in mind before you execute the rolloutUpdate command to roll back:

    • You must specify the backed up Oracle home as the Oracle home directory to roll out. This directory should be the backup directory from the previous rollout.

    • Once you specify the backup Oracle home directory as the Oracle home directory to roll back to, you must not specify the new Java home in the command. The Java home will be automatically rolled back to the original Java home that was used in the previous Oracle home that you have specified to roll back to.

  • Use the rolloutApplications command to rollback to the previous application version by specifying the old application archive in the json file. For more information about using this command, see Rolling Out Updated Applications

    .

The following example shows how to roll back to the previous Oracle home, Java home and applications. In this example, myDomain is the name of the domain to roll back to, /pathto/unpatchedOracleHomeBackup/ is the location of the backup Oracle home directory from the previous rollout, /pathto/unpatchedOracleHomeBackup1/ is the path of the directory to which the existing Oracle home will be moved. To enable the roll back operation, the isRollback parameter must be set to true as shown in the example:

rolloutUpdate('myDomain', '/pathto/unpatchedOracleHomeBackup/', '/pathto/unpatchedOracleHomeBackup1/', 'true')

Initiating a Rolling Restart of Servers or Partitions

Use the rollingRestart command if you want to do one of the following tasks:

  • Initiate a rolling restart of all servers in a domain.

  • Initiate a rolling restart of all servers in a specific cluster or clusters.

  • Initiate a rolling restart of all partitions in a cluster on a server.

The rolling restart of a partition involves restarting the partitions in a cluster on each server in a sequential manner in such a way that it does not affect other partitions across the cluster or server. Both the WebLogic Server administrator and the partition administrator can perform the rolling restart of partitions on a server.

rollingRestart has the following syntax:

rolloutRestart(target, [options])

The following example shows how to perform a rolling restart of all servers in Cluster1 and Cluster2.

connect('adminname', 'adminpassword', 't3://hostname:port')
clusters='Cluster1,Cluster2'
progress=rollingRestart(clusters)

Monitoring Workflow Progress

Each rollout command returns a WorkFlowTaskRuntimeMBean that you can use to poll the current status of the workflow. To monitor the progress of a rollout, use a rolloutcommand in the following format:

progress=rollout_command

For example, use this command if you are rolling out a new Oracle home:

progress=rolloutOracleHome(DomainA, '/net/patched/wls1221p.jar', 
'/net/backups/wls1221', autoRevertOnFailure=FALSE)

You can then use the methods of the WorkflowTaskRuntimeMBean to return information about the workflow. For more information, see WorkflowTaskRuntimeMBean in the MBean Reference for Oracle WebLogic Server. Here are some examples:

progress.getWorkflowId()

Returns the ID of the workflow.

progress.getProgressString()
'Workflow wf0011 Running: 13/36'

Returns a human-readable message containing information about the current workflow progress. In this example, workflow wf0011is currently running and has completed 13 of the 36 workflow commands.

progress.getStatus()
STARTED

Returns the current status of the workflow, which can be STARTED, SUCCESS, RETRY, REVERTING, FAIL, REVERTED, REVERT_FAIL, CANCELED, or REVERT_CANCELED.

The following Python script segment demonstrates one way to use the progress object to monitor a workflow and output the progress of a rollout task. Sample output is shown after the script.

# Print the starting information
rolloutName = progress.getName()
startTime = progress.getStartTime()
print "Started rollout task \"" + rolloutName + "\" at " + str(startTime)
 
# Check the state every 2 minutes
domainRuntime()
cd('RolloutService/rollout-service/ActiveWorkflows')
cd(progress.getWorkflowId())
while(get('Running')==1):
  progressString = progress.getProgressString()
  print progressString
  time.sleep(120)
 
# Print the ending information
endTime = progress.getEndTime()
state = progress.getState()
print "rollout \"" + rolloutName + "\" finished with state 


Output
Started rollout task "Domain1Rollout" at 2014-07-22 07:29:06.528971
Running step 1 of 9
Running step 2 of 9
Running step 3 of 9
Running step 4 of 9
Running step 5 of 9
Running step 6 of 9
Running step 7 of 9
Running step 8 of 9
Running step 9 of 9
rollout "Domain1Rollout" finished with state "SUCCESS" at 
2014-07-22 07:47:15.538299

Executing, Reverting, and Resuming Stopped Workflows

A workflow can stop in either the executing or reverting direction for the following reasons:

  • The workflow failed while executing, with the autoRevertOnFailure option set to FALSE.

  • The workflow was manually canceled.

  • An unrecoverable error occurred during a revert operation.

When a workflow is stopped, you can resolve any errors manually. You can then set the workflow to continue to execute or revert by using the following methods on the RolloutServiceRuntimeMBean:

Method Description

executeWorkflow(WorkflowTaskRuntimeMBean)

Takes a progress object that is eligible to be resumed and resumes it in the execute direction. If the last successful operation on the workflow was an execute, then the execute will resume with the next execute step. If the last successful operation on the workflow was a revert, then the execute will resume by executing that revert step.

revertWorkflow(WorkflowTaskRuntimeMBean)

Takes a progress object that is eligible to be resumed and resumes it in the revert direction. If the last successful operation on the workflow was an execute, then the revert will resume with that step. If the last successful operation on the workflow was a revert, then the revert will resume by reverting the next step in the revert sequence.

canResume(WorkflowTaskRuntimeMBean)

Returns true if the workflow stopped before it was completed and is not currently running in either direction. A workflow in this state is eligible to be resumed in either the execute or revert direction.

Useful WLST Commands for Workflows

This section describes several WLST commands that you may find useful.

  • To get a list of completed workflows:

    wls:/domain_name/domainRuntime/RolloutService/rollout-service> completeWfs=
    cmo.getCompleteWorkflows()
    
    
  • To get a list of active workflows:

    wls:/domain_name/domainRuntime/RolloutService/rollout-service> activeWfs = 
    cmo.getActiveWorkflows()
    
    
  • To look up a workflow by ID and retrieve its status:

    wls:/domain_name/domainRuntime/RolloutService/rollout-service> 
     progress=cmo.getWorkflowTask('workflow_id')
    wls:/Domain1221/domainRuntime/RolloutService/rollout-service> progress.getStatus()
    
    
  • To cancel a running workflow:

    wls:/domain_name/domainRuntime/RolloutService/rollout-service>
     progress=cmo.getWorkflowTask('workflow_id')
    wls:/domain_name/domainRuntime/RolloutService/rollout-service> progress.cancel()
    
    
  • To delete a completed workflow:

    wls:/domain_name/domainRuntime/RolloutService/rollout-service> cmo.deleteWorkflow('workflow_id')
    

Sample WLST Script

This section contains a sample WLST script that illustrates how to perform a rolling restart of all servers in a cluster called cluster1 with single service migration. In this script, the following arguments are defined:

  • username — The WebLogic Server administrator user name.

  • password — The WebLogic Server administrator password.

  • adminURL — The host name and port number of the domain's Administration Server.

  • target — The target or targets for the operation. See Table 3-1.

  • options — The rollout option or options for the operation. See Table 3-1.

The following example shows a sample WLST script for a rollout operation.

import sys, socket
import os
import time
from java.util import Date
from java.text import SimpleDateFormat
 
argUsername = sys.argv[1]
argPassword = sys.argv[2]
argAdminURL = sys.argv[3]
argTarget = sys.argv[4]
argTarget = sys.argv[5]

try:
   connect(argUsername, argPassword, argAdminURL)
   progress = rollingRestart(argTarget, argTarget)
   lastProgressString = ""
 
   progressString=progress.getProgressString()
   # for testing progressString="12 / 12"
   steps=progressString.split('/')
 
 
   while not (steps[0].strip() == steps[1].strip()):
     if not (progressString == lastProgressString):
       print "Completed step " + steps[0].strip() + " of " + steps[1].strip()
       lastProgressString = progressString
 
     java.lang.Thread.sleep(1000)
 
     progressString=progress.getProgressString()
     steps=progressString.split('/')
     if(len(steps) == 1):
       print steps[0]
       break;
 
   if(len(steps) == 2):
     print "Completed step " + steps[0].strip() + " of " + steps[1].strip()
 
   t = Date()
   endTime=SimpleDateFormat("hh:mm:ss").format(t)
 
   print ""
   print "RolloutDirectory task finished at " + endTime
   print ""
 
   state = progress.getStatus()
   error = progress.getError()

   stateString = '%s' % state   
   if stateString != 'SUCCESS':
     #msg = 'State is %s and error is: %s' % (state,error)
     msg = "State is: " + state
     raise(msg)
   elif error is not None:
     msg = "Error not null for state: " + state
     print msg
     #raise("Error not null for state: %s and error is: %s" + (state,error))
     raise(error)  
except Exception, e:
  e.printStackTrace()
  dumpStack()
  raise("Rollout failed")
 
exit()

To execute this script, save it in a Python (.py) file and then enter commands similar to this. If you are running WLST on Windows, see ZDT Patching Restrictions, for important information about using WLST on Windows.

$ORACLE_HOME/oracle_common/common/bin/wlst.sh 
/u01/scripts/rollout/RollingRestart.py username password 
t3://hostname:port cluster1 "migrationProperties=/u01/json/mig.txt"

Using the WebLogic Server Administration Console to Create and Monitor Workflows

Use the Administration Console to create and monitor a patching workflow that rolls out a patched Oracle home, a new Java version, new application versions or a combination of these tasks.

This section contains the following topics:

Accessing ZDT Workflow Functions in the WebLogic Server Administration Console

To access the ZDT workflow functions in the WebLogic Server Administration Console:

  1. In the WebLogic Server Administration Console, click the domain name under Domain Structure.
  2. On the Settings for domain_name page, select the ZDT Control tab.

    This displays the four tabs (Domain, Clusters, Servers, and Workflow Progress) from which you can manage all workflow-related tasks.

Creating a New Workflow for a Domain, Clusters, or Servers

You can create a workflow to roll out an update to all servers in a domain, all servers in one or more clusters, or only selected servers. The workflow can be for rolling out a new Java version, rolling out a new patched Oracle home, rolling out one or more updated applications, or any combination of these. You can also create a patching workflow to roll back to a previous Oracle home, Java home, or application versions, or create a workflow to perform a rolling restart of servers.

Prior to following this procedure, access the ZDT Control tabs as described in Accessing ZDT Workflow Functions in the WebLogic Server Administration Console.

To create a new workflow:

  1. Select one of the following tabs:

    • Domain: Select this tab if you want to create a workflow for the Administration Server and all clustered servers in the domain.

    • Clusters: Select this tab if you want to create a workflow only for servers in specific clusters.

    • Servers: Select this tab if you want to create a workflow only for specific servers. Typically, you would select this option only in the following situations:

      • The Administration Server is the only server that will be included in the workflow.

      • A situation exists in which a Managed Server is out-of-sync with other Managed Servers that were already updated. For example, you may have added a new server to a cluster, but that server is using an older version of Java than the other Managed Servers in the cluster.

      Note:

      Oracle recommends that you not use the Servers tab to perform updates to individual Managed Servers unless it is absolutely necessary. When you update individual Managed Servers, there is no guarantee that sessions will be preserved and downtime will be avoided.

  2. If you selected the Clusters tab, then select the clusters to include in the workflow. All servers in the selected clusters will be included in the workflow.

    If you selected the Servers tab, then select the servers to include in the workflow.

  3. Click Patch to configure the workflow tasks.

  4. Select the type of rollout (or rollback) that you want to perform:

    • Java home: Select if you only want to change to another Java version.

    • Oracle home: Select if you only want to roll out a new Oracle home or roll back to a previous Oracle home.

    • Application: Select if you only want to roll out one or more updated applications or roll back to one or more previous application versions.

    • All Combinations: Select if you want to roll out or roll back any combination of Java home, Oracle home, and application updates.

    • Rolling Restart: Select if you want to perform a rolling restart of the selected targets.

  5. Click Next.

    The displayed fields and options depend on the type of rollout or rollback you are performing.

  6. If you are changing the Java home, in the Java Home field, enter the full path to the Java home to change to. For example:

    UNIX

    /jdks/jdk1.8.0_50
    

    Windows

    C:\jdks\jdk1.8.0_50
    
  7. If you are rolling out a new Oracle home or rolling back to a previous Oracle home:

    1. In Rollout Oracle Home, enter the full path to the JAR archive. Only if you are rolling back to a previous Oracle home, you can specify the path to the local directory that contains the Oracle home to roll back to. For more information about rolling back to a previous Oracle home, see Reverting to the Previous Oracle Home, Java Home, or Applications.

    2. In Backup Oracle Home, enter the full path to the directory in which you want to back up the current Oracle home. For example, if your original Oracle home is /u01/Oracle_Home and you specify /u01/Oracle_Home_backup for this field, then /u01/Oracle_Home will be moved (renamed) to /u01/Oracle_Home_backup.

    3. If you are rolling back to a previous Oracle home, select the Rollback check box.

  8. If you are rolling out one or more new application versions, in the Application Properties field, enter the full path to a JSON-formatted text file that contains the information needed to upgrade the applications. For more information about creating this file, see Creating an Application Update JSON File.

  9. If you only want to evaluate the patching workflow before executing it, then select the Dry Run check box.

  10. If you want to migrate singleton services, such as JTA or JMS, during the rollout, then in the Migration Properties field, enter the full path to a JSON-formatted file that contains the migration information. For more information about creating this file, see Preparing to Migrate Singleton Services.

  11. By default, the Auto Revert on Failure check box is already selected. This will cause the patching operation to automatically revert everything if there is a failure while the workflow is executing. If you clear this check box, then the patching operation will not automatically revert if there is a failure; the operation will stop and wait for you to resume it or revert it.

  12. The Session Compatibility option determines whether or not the very last server being updated on a cluster will wait for sessions to complete on that server.

    • If not selected, the last server in a cluster waits for sessions to complete.This ensures that a compatible server is available in the cluster to handle sessions that must be served by a Managed Server that is still running on the existing version.

    • If selected, this indicates that the session state in servers is 100% compatible between the existing version and the new version. Therefore, the last Managed Server in the update sequence in a cluster will shut down without waiting for all existing sessions to complete.

    Oracle recommends that you not select this option unless you are absolutely sure that the session state is identical. This may cause the rollout to take longer due to the wait for session completion. The default session timeout value is 1 hour.

    Note:

    Serialization and deserialization in WebLogic Server differs slightly from Java serialization and deserialization. Therefore, additional fields on classes may result in a session being incompatible with servers on the new version, requiring that they be served by a server on the existing version. For example, a User class that adds a field such as Information will cause that session to be incompatible between versions.

  13. Click Finish to initiate the patching workflow.

    The workflow will be added to the Workflow Progress table.

After the workflow has started, you can monitor and manage its progress from the Workflow Progress page as described in Monitoring and Managing Workflows.

Monitoring and Managing Workflows

This section describes how to monitor and manage the progress of all running or completed workflows.

Prior to following this procedure, if you have not already done so, access the ZDT patching tabs as described in Accessing ZDT Workflow Functions in the WebLogic Server Administration Console.

To monitor and manage workflows, select the Workflow Progress tab. This page contains two tables:

  • The Workflows in Progress table shows all workflows that are not yet completed (active); that is, they are in an executing, reverting, stopped, canceled, or failed state. Depending on its status, you can perform various actions on an active workflow:

    • You can Cancel any workflow that is in a STARTED, REVERTING, or RETRY state.

      To cancel one or more workflows, select the check box for each workflow that you want to cancel, and then click Cancel. You can then revert the workflow by clicking Revert or resume it by clicking Execute.

    • You can Execute any workflow that is in a CANCELED, REVERT_CANCELED, FAIL, or REVERT_FAIL state.

      To execute one or more stopped (canceled) workflows, select the check box for each workflow that you want to resume, and then click Execute. The workflow will continue executing, starting with the step after the last successfully completed step.

    • You can Revert any workflow that is in a CANCELED, REVERT_CANCELED, FAIL, or REVERT_FAIL state.

      To revert one or more stopped (canceled) workflows, select the check box for each workflow that you want to revert, and then click Revert. The workflow will revert, starting with the last successfully completed step.

    • You can Delete any workflow that is in a CANCELED, REVERT_CANCELED, FAIL or REVERT_FAIL state. You can delete only one workflow at a time.

      To delete a workflow, select the check box for each workflow that you want to delete, and then click Delete.

  • The Completed Workflows table shows all workflows that have completed. This table is sorted based on when the workflow completed, with the most recently completed workflow at the top of the table.

    To delete completed workflows, select one or more of them and click Delete.

From these tables, you can also view additional details about the status of a workflow. To do so, click the workflow ID in the Workflow ID column. For more information, see Viewing Workflow Details. For information about workflow statuses, see Workflow Statuses.

Viewing Workflow Details

This section describes how to view the details of an active or completed workflow, and also describes the information that is displayed for the workflow.

To view the details for a workflow, click the workflow ID (for example, wf00071) in the Workflow ID column of either the Workflows in Progress or Completed Workflows table on the Workflow Progress page. A page is displayed with the information described in the following table.

Field Description

Workflow ID

The workflow that was automatically assigned when you created it

Type

The type of workflow, which can be:

  • rolloutJavaHome: You are rolling out or rolling back to a different Java home version.

  • rolloutOracleHome: You are rolling out or rolling back to a different Oracle home.

  • rolloutApplications: You are rolling out one or more new application versions or rolling back to one or more previous application versions.

  • rolloutUpdate: You are rolling out or rolling back to any combination of Java home, Oracle home, or application version.

Target

The servers to which the workflow is targeted, which can be:

  • Domain: The workflow is targeted to all eligible servers in the domain, including the Administration Server.

  • Comma-separated list of cluster names: The workflow is targeted to all eligible servers in the listed clusters.

  • Comma-separated list of servers: The workflow is targeted only to those servers that are listed.

Status

The current status of the workflow. For more information, see Workflow Statuses.

Can Resume

Indicates whether or not the workflow can be resumed or reverted. If false, you will not be able to use the Execute or Revert functions on the workflow.

# of Completed Commands

The number of workflow commands that have currently been completed

# of Total Commands

The total number of commands in the workflow that need to be executed to complete the workflow

Progress String

A detailed message about the progress of the workflow, such as:

Workflow wf0008 finished successfully. 36 steps completed.

Next Execute Step

If the workflow is still active and is not reverting, then this field shows the next command that will be executed after the current command completes.

Next Revert Step

If the workflow is still active and is reverting, this field shows the next command that will be executed in the revert process after the current command completes.

Begin Time

The time at which the workflow was started

End Time

If the workflow has completed, then this field displays the time of completion.

Exception

If the workflow failed, then this field displays the exception that occurred when it failed.

Advanced

Click the arrow to expand the Advanced section, which shows all steps that have been executed for the workflow up to the current time. If the workflow has completed, then this section lists all commands that were completed by the workflow.

Viewing Server Status

From the Servers page, you can view the current status of all your servers before and after running a workflow and while workflows are in progress. When you click the Servers tab, you can view the workflow-related information about each server in your domain. For information about the columns in the Servers table and additional columns that you can add to the table, see View Server Patching Status in Administration Console Online Help.

When a workflow is running, you can monitor and refresh the information on this page to get up-to-date status for each server.

Viewing Cluster Status

From the Clusters page, you can view information about all clusters in your domain before and after running a workflow and while workflows are in progress. For information about the columns in the Clusters table and additional columns that you can add to the table, see View Cluster Patching Status in Administration Console Online Help.

When a workflow is running, you can monitor and refresh the information on this page to get up-to-date information for each cluster.

Workflow Statuses

An active workflow can have any of the following statuses:

Status Description

STARTED

The workflow has started and is currently running.

RESUME

A stopped workflow has been resumed.

REVERTING

A workflow that failed or was stopped is reverting.

FAIL

The workflow has failed to execute completely. This status appears only if the Auto Revert on Failure option was not configured for the workflow when it was started.

REVERTED

A workflow that was either automatically or manually reverted has successfully completed the revert process.

REVERT_FAIL

A workflow that was either automatically or manually reverted failed to revert successfully.

CANCELED

The workflow was canceled (paused).

REVERT_CANCELED

A workflow that was either automatically or manually reverted was canceled (paused).

Workflow Logging

A rollout consists of a series of steps. Each step logs a message to the Administration Server log when it starts and when it finishes. Messages are also logged if a step reverts, fails, or retries. The Administration Server log is located at

domain_home/servers/AdminServer/logs

Filtering the Log File

The workflow ID is included in every log message related to a given workflow. Use the workflow ID to filter the Administration Server log file for messages related to a given workflow. If you initiated the workflow from the WebLogic Server Administration Console, then you can get the workflow ID from the ZDT Control > Workflow Progress tab. From WLST, you can use the following command to get the workflow ID:

progress.getWorkflowId()

To filter the log file, enter the following command:

fgrep wf0001 domain_home/servers/AdminServer/logs/AdminServer.log

Log Message Format

The log messages for ZDT patching are formatted as follows:.

Message Type Message Format

A step begins executing.

Workflow workflowId is executing step name on target.

A step is complete.

Workflow workflowId successfully completed step name on target.

A step is being reverted.

Workflow workflowId is reverting step name on target.

A step has successfully reverted.

Workflow workflowId successfully completed revert of step name on target.

A step is being retried.

Workflow workflowId is retrying step name on target.

A step could not be completed successfully.

Workflow workflowId failed to complete step name on target.

A step could not be completed successfully due to an exception.

Workflow workflowId failed to complete step name on target due to error exception.

A step could not be reverted successfully.

Workflow workflowId failed to revert step name on target.

A step could not be reverted successfully due to an exception.

Workflow workflowId failed to revert step name on target due to error exception.