Using Edition-Based Redefinition (EBR) to Update Applications in a Production Environment

About Edition-Based Redefinition

With EBR, an application can store two editions of the database schema within the same database, at the same time. One to be used by the pre-upgrade application and the other by the post-upgrade application.

When an online update is complete, the pre-upgrade application references the earlier edition (Edition 1) of the database schema and the post-upgrade application references the new edition (Edition 2) of the database schema.

This co-existence of multiple editions of the database enables the ongoing user sessions to continue using the pre-upgrade application while the new sessions are directed to the post-upgrade application. After all the existing sessions are complete, the pre-upgrade application is retired and the post-upgrade application becomes fully operational; the entire upgrade process is completed with zero downtime.

For more details about EBR and to familiarize yourself with key terminologies associated with this feature, see Using Edition-Based Redefinition in the Database Development Guide.

Note:

You can also perform an application update by using EBR with the WebLogic Zero Downtime Patching (ZDT Patching) feature. However, you have to EBR-enable your application and synchronize the configuration update during the ZDT rollout of updates across a domain. For details, see the following sections:

Synchronizing Application Software and Domain Configuration Updates
Introduction to Zero Downtime Patching in Administering Zero Downtime Patching Workflows

Enabling Your Application for EBR

Before you begin EBR-enabling your environment, you must stop the WebLogic Server domain.

See Starting and Stopping Servers in Administering Server Startup and Shutdown for Oracle WebLogic Server.

You must complete the following key steps to use EBR:

Enabling users and schema objects to use editions
Enabling your applications to use editioning views

For more information about readying your application for EBR, see Using EBR to Upgrade an Application in the Database Development Guide.

Configuring WebLogic Data Sources to Use Editions

During an EBR update, WebLogic Server automatically manages the user sessions such that existing user sessions remain connected to the pre-upgrade application (referencing Edition 1 of the database schema) while the new user sessions are connected to the post-upgrade application (referencing Edition 2 of the database schema).

To facilitate the automatic management of user sessions, you have to configure the WebLogic data sources to determine the database edition to which it should connect. There are different ways to configure data sources. For more information, see the section "Configuring WebLogic Data Sources to Use Editions", under Using Edition-Based Redefinition in Administering JDBC Data Sources for Oracle WebLogic Server.

About Application Software and Domain Configuration Updates

An application update is typically a two-step process – software (code) update and WebLogic Server domain configuration data update. Depending on your environment, you may use a different update mechanism for each step.

A software update, for instance, is done by installing the new product binaries while the configuration data update flows in through the Administration Server. The configuration changes are first communicated to the Administration Server through tools such as WLST, Management REST API, JMX, and so on. The Administration Server then rolls out the changes to the Managed Servers within the domain.

When performing a rolling update of an application that uses EBR, if the software update requires a concurrent configuration update, the different update mechanisms may lead to a mismatch between the software version and the domain configuration version (old version of the software using the new configuration data or the new version of the software using the old configuration data) resulting in incompatibility issues.

Example 8-1 Illustrates the incompatibility between the software and the domain configuration versions

Let’s assume that you have an environment comprising:

One Administration Server - AS.
Two Managed Servers - MS1 and MS2.
The servers are using the software ‘S1’ and are connected to the database edition ‘E1’ (pre-upgrade state).
In the post-upgrade state, the servers are expected to use the new software ‘S2’ and connect to the new database edition ‘E2’.

Now, the Administration Server, the first server in the domain to get updated, receives a configuration change followed by an installation of the updated software S2. The two Managed Servers (MS1 and MS2) continue to be in the preupgrade state (that is, S1-E1 combination). The Administration Server then broadcasts the configuration change to MS1 and MS2. The servers receive the updated configuration data and connect to the new edition (E2) of the database while continuing to use the old software (S1).

At this point, the servers will have the old software configured with the new configuration data (S1-E2 combination). This incompatible state exists until you upgrade the application, typically by performing a rolling restart of the Managed Servers, to have the software updated to S2 in both MS1 and MS2. The servers have now attained a consistent state of using the updated versions of both software and configuration data (S2-E2 combination).

A state of incompatible versions can exist even if you were to consider updating the software first, followed by the configuration data. In this case, the Administration Server is updated with the new software S2, and then you perform a rolling restart of the Managed Servers to have the software updated to S2 in both MS1 and MS2. At this point, the servers use the new software configured with the old configuration data (S2-E1 combination). The Administration Server then receives a configuration update which it broadcasts to the Managed Servers. The servers now attain a consistent state of using the new software configured with the new configuration data (S2-E2 combination).

Irrespective of the sequence of the update (software first or configuration first), there will be an incompatibility issue. When updating applications using EBR, a mismatch between software version and configuration data version is not desirable.

A coordinated and synchronized rolling update can be used to prevent these interim periods of inconsistency between the software and the configuration data.

Synchronized Rolling Update

During a synchronized rolling update, the configuration data gets updated concurrently as the servers are updated with the new software version, ensuring that there is complete synchronization between the configuration and the software versions on the server.

Example 8-2 Synchronized Rolling Update

Let's assume that you have updated the software and the configuration on the Administration Server (S2-E2). The Managed Servers continue to be in the pre-upgrade state and use the old software (S1) that is communicating with the earlier edition (E1). While they still have the old software, they are not ready for the new configuration (that is, the new edition E2).

The Administration Server does not know whether the Managed Servers are ready, and so it assumes that when a configuration change arrives, it should pass on the change to the Managed Servers. However, when the Managed Servers are configured to support synchronization, they ignore the changes until they are ready. For example, if you perform a synchronized rolling restart of MS1 and reboot it with new software, the synchronization logic on MS1 will indicate that MS1 is now ready to accept the previously ignored changes. This means that MS1 has its software updated (from S1 to S2) and uses the new edition (E2) of the database schema to communicate with the updated software (S2).

At this point in time, there are two editions of the database running and the individual servers, MS1 and MS2, are each connected to a different edition - an individual server is configured for a single edition at a time. MS1 is running the new version of the software (S2) and is using the new edition of the database (E2). MS2 is running the old version of the software (S1) and is using the old edition of the database (E1). Although the Managed Servers have mismatched configuration with one another (MS1 – E2 and MS2 – E1), individually they have compatible versions of the software and the database schema (MS1 with S2-E2 and MS2 with S1-E1). This process is called a synchronized rolling update - the version of the software is kept synchronized with the version of the WebLogic configuration and, therefore, the database edition.

Likewise, when MS2 gets the new version of the software, it is ready and willingly accepts the new configuration. After the process is complete, all servers have the new version of the software (S2) and point to the new database edition (E2).

To perform a coordinated and synchronized rolling update, WebLogic Server provides built-in mechanisms using which you can ensure that the Managed Server accepts the domain configuration update only after the software update is complete. That is, the Managed Server processes a configuration change only after the updated software is installed on the server. These mechanisms prevent incompatible software and configuration data versions, and ensure that the configuration version and the software version always stay synchronized.

Synchronizing Application Software and Domain Configuration Updates

WebLogic Server helps you determine whether a Managed Server should accept the configuration change or not.

You can use one of the following two checkers depending on how you want to update the Managed Server:

Lockfile Checker
Version Checker

These checkers prevent the Managed Servers from accepting configuration changes that are not synchronized with the installed software. Depending on when these checkers get initiated, they are categorized into:

Configuration Independent (Example: Lockfile Checker): Any change in the environment will initiate the checker (not necessarily a configuration change). Depending on the nature of the change, the checker instructs the Managed Server to ignore or accept the change.
Configuration Based (Example: Version Checker): Any configuration update will initiate the checker. Depending on what the change is, the checker instructs the Managed Server to accept or reject the change.

The checkers get initiated when the Managed Server receives an indication from the Administration Server of a possible change, during the following events:

When the Administration Server accepts a configuration change and passes it over to the Managed Server.
When you restart the Managed Server and it detects updates from the Administration Server.
When the Managed Server receives periodic signals from the Administration Server indicating a change.

In addition to these checkers, WebLogic Server also enables you to write your own Java code plug-ins. See Writing and Configuring Your Own Java Plug-In.

Lockfile Checker

The configuration independent Lockfile Checker gets initiated when the Managed Server expects a configuration change coming from the Administration Server. Set the Lockfile on the machine where the Managed Server is running. For as long as the Lockfile exists, the Managed Server does not accept the configuration change from the Administration Server. After the software is updated on the Managed Server, remove the Lockfile. As soon as the Lockfile is removed, the configuration change is accepted and applied by the Managed Server.

Version Checker

The configuration based Version Checker, on the other hand, gets initiated when the Managed Server receives a configuration update. This checker enables the Managed Server to decide the version of the software for which it should accept the configuration data change. To achieve this, you need to include a file that contains the version of the software, as part of the software installation process.

When the Managed Server receives the configuration update from the Administration Server, it compares the version (a numerical value) that is part of the Version Checker with the software version included in the installation file and accepts the change only if the two versions match.

Configuring the Lockfile Checker

The Lockfile Checker gets called when a Managed Server expects a configuration change from the Administration Server. To make use of this checker, you have to configure it at the ‘Expected Change’ hook point.

Note:

The assumption is that your environment consists of an Administration Server and a Managed Server running on different machines. Therefore, the servers have their own copies of the config.xml file.

Use WLST to connect to the Administration Server:

connect('<username>', 'ok', 't3://localhost:7001')

Create a callout to refer to the Lockfile Checker, as shown below:

edit()
startEdit()
cd('/')
callout1=cmo.createCallout('LockFileChecker-1')
callout1.setHookPoint("HOOK_POINT_EXPECTED_CHANGE")
callout1.setClassName("weblogic.configuration.ConfigurationLockfileChecker")
callout1.setArgument("<Location of the Lock File in the Managed Server>")
activate()

Example 8-3 Illustrates the use of the Lockfile Checker

Use WLST to connect to the Administration Server, as described in step 1 above.
Create the callout on the Administration Server node and activate it to push the callout changes to the Managed Server.
Create a file named Lockfile on the machine where the Managed Server is running (in the location that you have specified as the argument of the callout when configuring the Lockfile Checker). Now, the Managed Server will not accept any configuration changes from the Administration Server.
Log in to the WebLogic Server Administration Console, expand Environment, and then click Servers.
Select the Managed Server whose configuration you want to change.
Navigate to the Configuration > Concurrency page and change the value of Max Concurrent New Threads from 50 to 100.
Save and activate the changes.
Check the config.xml file for the Managed Server or connect to the Managed Server by using WLST, and then observe the value of Max Concurrent New Threads.
The value is not updated on the Managed Server because of the presence of the Lockfile.
Now, remove the Lockfile from the Managed Server and check the config.xml file for the Managed Server or connect to the server by using WLST.
Observe the value of Max Concurrent New Threads. The value is now updated to 100.

Configuring the Version Checker

The Version Checker gets called when a Managed Server receives a configuration update from the Administration Server. To make use of this checker, you have to configure it at the ‘Change Received’ hook point.

Note:

The assumption is that your environment consists of an Administration Server and a Managed Server running on different machines. Therefore, the servers have their own copies of the config.xml file.

Use WLST to connect to the Administration Server:

connect('<username>', '<password>', 't3://localhost:7001')

Change the software version installed on the domain to say, 1.1:

edit()
startEdit()
cd('/')
cmo.setInstalledSoftwareVersion('1.1')
activate()

Create a callout to refer to the Version Checker, as shown below:

edit()
startEdit()
cd('/')
callout1=cmo.createCallout('VersionChecker-1')
callout1.setHookPoint("HOOK_POINT_CHANGE_RECEIVED")
callout1.setClassName("weblogic.configuration.ConfigurationVersionChecker")
callout1.setArgument("<Location of the Version File on the Managed Server>")
activate()

Example 8-4 Illustrates the use of the Version Checker

Use WLST to connect to the Administration Server, as described in step 1 above.
Create the callout on the Administration Server node and activate it to push the callout changes to the Managed Server.
Create the Version Checker file (in the location that you have specified as the argument of the callout at the time of configuring the Version Checker, for example /MyInstallation/Servers/MS1/VersionChecker) on the machine where the Managed Server is running. The file should contain the version number.
```
echo "1.0"  >  </MyInstallation/Servers/MS1/VersionChecker >
```
Log in to the WebLogic Server Administration Console, expand Environment, and then click Servers.
Select the Managed Server whose configuration you want to change.
Navigate to the Configuration > Web Services > Buffering page and change the value of retry-count to 999.
Save and activate the changes.
Check the config.xml file for the Managed Server or connect to the Managed Server by using WLST and observe the value of retry-count.
Notice that the value is not updated to 999. This occurs because the version of the installed software (1.1) as specified in the Version Checker file does not match with the expected version set in the callout (1.0).
Modify the version number in the Version Checker file to contain the same version as the version configured in the callout. This is the point where the installed software is to be updated before updating the version in the Version Checker file.
```
echo "1.1" >  <Location of the Version Checker>
```
Check the config.xml file for the Managed Server or connect to the server by using WLST.
Observe that the value of retry-count is now updated to 999 because the two versions matched.

Writing and Configuring Your Own Java Plug-In

In addition to the Lockfile Checker and the Version Checker, WebLogic Server also gives you the flexibility to write your own custom logic for deciding whether to accept or reject configurations. In fact, both Lockfile and Version checkers are simply Java plug-ins themselves.

There are two types of Java plug-ins: Expected Change and Change Received. The difference is that while the Change Received plug-in gives the ability to look at the configuration changes to make the accept/reject decision, the Expected Change plug-in does not. However, the former requires less computation on both WebLogic's part and the plug-in's part.

Implementing the plug-in requires overriding the init and callout methods from weblogic.callout.spi.WebLogicCallout.java .

To install the plug-in, do the following:

Add the Java plug-in classes to the Managed Server classpaths and restart the servers.
Create the callouts using WLST, as described for the out-of-the-box checkers. See Configuring the Lockfile Checker and Configuring the Version Checker.

ConfigurationLockfileChecker is an Expected Change plug-in. The code below does not have access to the WebLogic DomainMBean. It simply makes its decision based on the state of the machine on which it is running (that is, the Managed Server on which it is installed) and checks for the presence of the configured file.

package weblogic.configuration;

import java.io.File;
import java.util.Map;
import weblogic.callout.spi.WebLogicCallout;
/*
 This is provided out of box for the usage of orchestrating configuration updates and rollout.
 To use this, user will need to create a Callout under domain,  with the following parameter

     name:        callout-0
     hookpoint:   HOOK_POINT_EXPECTED_CHANGE
     classname:   weblogic.configuration.ConfigurationLockfileChecker
     argument:    /wls-installation/servers/Server-0/CONFIGURATION_LOCKFILE

     both hookpoint and classname should be as above,  argument is the path to the configuration lockfile.  You can name
     it any way you want.
     This class will be invoked whenever the following occurs:
      #1. managed server boots up,
      #2. managed server does periodic 'heartbeat' for getting any changes from the admin server
      #3. admin server commit a configuration transaction and send the bits to this managed server

      The existence of this lockfile is checked.  If the lockfile exists, the managed server configuration will be updated,
      otherwise, the change will be ignored until the next time, one of the above 3 situation occurs.

 * Copyright (c) 2019, Oracle and/or its affiliates. All rights reserved.
 */
public class ConfigurationLockfileChecker implements WebLogicCallout {

  File lockfile = null;

  @Override
  public String callout(String hookPoint, String location, Map<String, Object> values) {


    if (WebLogicCallout.HOOK_POINT_EXPECTED_CHANGE.equals(hookPoint)) {
      if (lockfile != null && lockfile.exists()) {
        return WebLogicCallout.RESPONSE_IGNORE;
      }
      return WebLogicCallout.RESPONSE_ACCEPT;
    }

    return WebLogicCallout.RESPONSE_CONTINUE;
  }

  public void init(String argument) {
    if (argument != null) {
      lockfile = new File(argument);
    }
  }
}

ConfigurationVersionChecker is a Change Received plug-in. The code below looks at both the local machine and the WebLogic DomainMBean to make its decision. It checks if the configured version (in the WebLogic DomainMBean) matches the version of the software installed on the Managed Server.

package weblogic.configuration;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.util.Map;
import weblogic.callout.spi.WebLogicCallout;
import weblogic.management.configuration.DomainMBean;

/**
 * Copyright (c) 2019, Oracle and/or its affiliates. All rights reserved.
 */

/*
     This is provided out of box for the usage of orchestrating configuration updates and rollout.
     To use this, user will need to create a Callout under the domain,  with the following parameter
       name:        callout-0
       hookpoint:   HOOK_POINT_CHANGE_RECEIVED
       classname:   weblogic.configuration.ConfigurationVersionChecker
       argument:    /wls-installation/servers/Server-0/CONFIGURATION_VERSION_CHECKER

     both hookpoint and classname should be as above,  argument is the path to the configuration version checker file.
     You can name it any way you want.

     You will also need to set the domain's <installed-software-version> for using this callout.

     This class will be invoked whenever the following occurs:
      #1. managed server boots up,
      #2. managed server does periodic 'heartbeat' for getting any changes from the admin server
      #3. admin server commit a configuration transaction and send the bits to this managed server

      If this configuration version checker exists, the content of this file,  which should be a version nmuber  eg, "1.0"
      will be used to compare with the domain's <installed-software-version>  property, eg "1.1".  If it matches, managed server's
      configuration will be updated,  otherwise, the change will be ignored until the next time, one of the above 3 situation
      occurs.
 */

public class ConfigurationVersionChecker implements WebLogicCallout {

  File versionFile = null;

  @Override
  public String callout(String hookPoint, String location, Map<String, Object> values) {
    if (WebLogicCallout.HOOK_POINT_CHANGE_RECEIVED.equals(hookPoint)) {
      String versionFromFile = getVersionFromFile();
      String pendingVersion = getVersionFromDomainMBean(values.get(WebLogicCallout.VALUES_PENDING));
      if (versionFromFile != null && pendingVersion != null){
        return versionFromFile.equals(pendingVersion) ? WebLogicCallout.RESPONSE_ACCEPT: WebLogicCallout.RESPONSE_IGNORE;
      }
    }
    return WebLogicCallout.RESPONSE_CONTINUE;
  }

  @Override
  public void init(String argument) {
    if (argument != null){
      versionFile = new File(argument);
    }
  }

  String getVersionFromFile() {
    if (versionFile != null && versionFile.exists()) {
      try (FileReader fileReader = new FileReader(versionFile)) {
        BufferedReader bufferedReader = new BufferedReader(fileReader);
        StringBuffer stringBuffer = new StringBuffer();
        String readLine;
        while( (readLine = bufferedReader.readLine()) != null) {
          stringBuffer.append(readLine);
        }
        return stringBuffer.toString();
      } catch (FileNotFoundException e) {
      } catch (IOException e) {
      }
    }
    return null;
  }

  String getVersionFromDomainMBean(Object domainMBean) {
    if (domainMBean != null && domainMBean instanceof DomainMBean) {
      return ((DomainMBean)domainMBean).getInstalledSoftwareVersion();
    }
    return null;
  }
}