1. Administering Oracle Java Cloud Service
  2. Upgrade the WebLogic Server Release for an Oracle Java Cloud Service Instance

10 Upgrade the WebLogic Server Release for an Oracle Java Cloud Service Instance

For an existing Oracle Java Cloud Service instance, you can upgrade the WebLogic Server release from 12.2.1.0, or 12.2.1.2 to WebLogic Server release 12.2.1.4.

The manual upgrade process leverages the basic procedures for the on-premises upgrade, with some additional procedures.

Topics:

About Upgrading the WebLogic Server Release for an Oracle Java Cloud Service Instance

You can leverage Fusion Middleware tools to manually upgrade the WebLogic Server release 12c or WebLogic Server release 11g to either WebLogic Server release 12.2.1.4 or 12.2.1.3 for an existing Oracle Java Cloud Service instance.

Upgrading differs from patching. When upgrading you replace an existing WebLogic Server release, whereas patching only applies the latest patch set update (PSU). Unlike in patching, which applies updates in a rolling fashion, requiring no down time, upgrade requires downtime. No servers can be running during the upgrade process.

The following restrictions apply:

  • You can't upgrade a service instance created before the Oracle Java Cloud Service release 17.4.1 (October 2017).
  • The service instance can't be provisioned with Oracle SOA Suite.
  • Only in-place binary installation is supported. The upgrade must use /u01/app/oracle/middleware as opposed to a new directory.
  • You can upgrade WebLogic Server 12c (12.2.1.2, or 12.2.1.0) or WebLogic Server 11g (11.1.1.7) to WebLogic Server release 12.2.1.4.
  • You can upgrade WebLogic Server 12c (12.2.1.3) to WebLogic Server release 12.2.1.4.
  • When upgrading to 12.2.1.3, you can't upgrade a service instance that includes an Oracle Traffic Director (OTD) load balancer. You must remove the OTD before performing the upgrade, then add the OTD back after the WebLogic Server nodes have been upgraded. The OTD added back is version 12.2.1.2.
  • When upgrading to 12.2.1.4, you can upgrade a service that includes an OTD load balancer. Note, however, the following additional restrictions:
    • The service instance must be running WebLogic Server release 12.2.1.3, 12.2.1.2 or 12.2.1.0. Both WebLogic Server and OTD versions are upgraded at the same time. You can't upgrade one component without upgrading the other component.
    • For a service instance that's based on WebLogic Server 11g, you must remove the OTD before performing the upgrade, then add the OTD back after the WebLogic Server nodes have been upgraded. The OTD added back is version 12.2.1.4.
  • These upgrade instructions are not applicable to Oracle Data Integrator (ODI) and Oracle WebCenter Portal (WCP) instances.
  • Prior to upgrading, ensure that your version of Oracle Java Cloud Service is on the latest patch.

This upgrade process leverages the following Fusion Middleware tools:

  • Reconfiguration Wizard: Helps you reconfigure the WebLogic domain.
  • Upgrade Assistant: Helps you perform a readiness check, upgrade the infrastructure schemas, and upgrade the domain component configurations.

During the manual upgrade process, you will be referring to the WebLogic Server documentation. See Introduction to Upgrading Oracle Fusion Middleware Infrastructure to 12c (12.2.1.4.0) or Introduction to Upgrading Oracle Fusion Middleware Infrastructure to 12c (12.2.1.3.0) in Upgrading to the Oracle Fusion Middleware Infrastructure.

Perform Prerequisite Tasks

Before upgrading your service instance, check whether there is enough space for temporary backups, back up the database, and back up the MIDDLEWARE_HOME, JDK_HOME, and domain directories.

To perform prerequisite tasks:
  1. Ensure there is enough free disk space on all of the nodes in your Oracle Java Cloud Service instance so that you can download the new 12.2.1.4 or 12.2.1.3 binaries, and back up the current binaries and configurations.

    Verify that there is at least 2GB of available space on the MIDDLEWARE_HOME and root volumes. Run the following command on all nodes as the oracle user. 

    df -kh /tmp /u01/app/oracle/middleware

    See Add Storage to a Node.

  2. Back up the infrastructure database associated with your service instance. You will need the backup in the unlikely case that the upgrade doesn't succeed and you must roll back the service instance to its original state.

    Note:

    Restoring the database that is used with multiple Java Cloud Service instances may risk data loss for other service instances. Oracle recommends that during the upgrade process you bring down all instances that share the database.
  3. Set the server group for the administration server using the WebLogic Scripting Tool (WLST) .
    1. Connect to the Administration Server node in your service instance using SSH.
    2. Switch to the oracle user.
       sudo su oracle
    3. Identify the name of the Administration Server.
      ls /u01/data/domains/<domain_name>/server
    4. Start a WLST prompt.
      /u01/app/oracle/middleware/oracle_common/common/bin/wlst.sh
    5. Run the following commands from the prompt:
      > readDomain('/u01/data/domains/<domain_name>')
> groups = ["WSMPM-MAN-SVR"]
> setServerGroups("<adminserver-name>", groups)
> updateDomain()
> closeDomain()
  4. Back up the MIDDLEWARE_HOME directory (/u01/app/oracle/middleware), JDK_HOME directory (/u01/jdk) directory, and the domain directory (/u01/data/domains/<your_domain_name>) on all your service's nodes.

    Note:

    Certain customizations to the environment will be lost during the upgrade. For example, the upgrade process might overwrite your changes to setDomainEnv.sh.
    For example:
    zip -r old_jdk.zip /u01/jdk 
zip -r old_middleware.zip /u01/app/oracle/middleware 
zip -r old_domain.zip /u01/data/domains/Example1_domain
  5. If your service instance is based on WebLogic Server 12.2.1.3, 12.2.1.2, or 12.2.1.0, and you have configured Oracle Traffic Director (OTD) for the service instance:
    • Back up /u01/data/otd-instance/otd_domain, the domain that is used to manage and monitor OTD.

      For example:

      zip -r old_domainotd.zip /u01/data/otd-instance/otd_domain
  6. If your service instance is based on WebLogic Server 11g, and you have configured Oracle Traffic Director (OTD) for the service instance, use the REST API to remove the OTD node.
    curl -i --user <user>:<password>
          -X PUT -H "Content-Type:application/vnd.com.oracle.oracloud.provisioning.Service+json" -d "{}" https://<rest_server_url>:/paas/api/v1.1/instancemgmt/<identity-domain>/services/jaas/instances/<servicename>/servicecomponent

    Note:

    In a multi-cluster service instance you have performed a scale-out operation and added a new cluster, the OTD routing information for the scaled-out server in a new cluster will be lost when you remove the OTD node. After upgrade, you must manually add the routing information back to OTD.

  7. Open a VNC session on the WebLogic Administration Server node so that you can run the Reconfiguration Wizard and the Upgrade Assistant. See Connect to a Node with VNC.

    To use the Reconfiguration Wizard and Upgrade Assistant (Fusion Middleware tools) during the upgrade process, you need a graphical user interface (GUI) environment. X11 forwarding may be used to forward the GUI to your local desktop. If you set up VNC, then X11 forwarding is not needed.

    If your service instance is based on WebLogic Server 12.2.1.3, 12.2.1.2, or 12.2.1.0, and you have configured Oracle Traffic Director (OTD) for the service instance, also set up VNC or X11 forwarding on the OTD VM.

Download the Upgrade Software

Use the REST API to download the WebLogic Server 12.2.1.4 or 12.2.1.3 binaries to the specified Oracle Java Cloud Service instance.

This procedure does not perform the actual process of upgrading the WebLogic Server software to the 12.2.1.4 or 12.2.1.3 version. The upgrading is done later by manually installing the binaries and using the WebLogic Upgrade Assistant. See Upgrade the Domain.
  1. Use the REST API to get the patch ID of the WebLogic Server 12.2.1.4 or 12.2.1.3 upgrade patch.
    curl -i -X GET -u <user>:<password> -H "X-ID-TENANT-NAME:<identity_domain>" https://<rest_server_url>/paas/api/v1.1/instancemgmt/<identity_domain>/services/jaas/instances/<servicename>/patches/available?filter=upgrades
    For example:
    curl -i -X GET -u username:password -H "X-ID-TENANT-NAME:ExampleIdentityDomain" https://rest_server_url/paas/api/v1.1/instancemgmt/ExampleIdentityDomain/services/jaas/instances/Example1/patches/available?filter=upgrades
    Example output:
    [{
   "availablePatchGuiMetadata":{
      "supportsPreCheck":true
    },
   "patchId":"wls_upg_12.2.1.3.190115_for_12cRelease212",
   "patchCategory":"MajorPatch",
   "patchSeverity":"Normal",
   "includesConfigUpgrade":false,
   "patchDescription":"WebLogic Server 12.2.1.3.0 with PSU Update 12.2.1.3.190115",
   "patchReleaseUrl":"https://support.oracle.com/epmos/faces/PatchDetail?patchID\u003d28710939",
   "serviceType":"JaaS",
   "serviceVersion":"12cRelease212",
   "releaseDate":"2019-01-14T17:40:00.000+0000",
   "entryDate":"2019-02-05T12:04:51.975+0000",
   "entryUserId":"smctl",
   "componentPatches":{

   },
   "patchType":"PSU",
   "requiresRestart":false,
   "isDeleted":false,
   "displayName":"12.2.1.3.190115",
   "releaseVersion":"12.2.1.3.190115",
   "patchCustomActions":[

   ],
   "restartStrategy":"RESTART_AFTER_PATCH",
   "isUpgrade":true,
   "extensionId":0
}]
    In the response, look for the patch that has "patchCategory":"MajorPatch" and copy the patchId value. In the example, the upgrade patch ID is wls_upg_12.2.1.3.190115_for_12cRelease212.
  2. Use the REST API to apply the upgrade patch to the service instance. No parameters are needed in the request payload.
    curl -i -X PUT -u <user>:<password> -H "X-ID-TENANT-NAME:<identity_domain>" https://<rest_server_url>/paas/api/v1.1/instancemgmt/<identity_domain>/services/jaas/instances/<servicename>/patches/<upgrade-patch>
    For example:
    curl -i -X PUT -u username:password -H "X-ID-TENANT-NAME:ExampleIdentityDomain" https://rest_server_url/paas/api/v1.1/instancemgmt/ExampleIdentityDomain/services/jaas/instances/Example1/patches/wls_upg_12.2.1.3.190115_for_12cRelease212
    Example output:
    {
   "status":"Completed",
     "details":{
     "message":"Patching  service with patch [wls_upg_12.2.1.3.190115_for_12cRelease212] is  submitted as an asynchronous job.",
     "jobId":"107376"
     }
   }

    Note:

    If your service instance is based on WebLogic Server 12.2.1.3, 12.2.1.2, or 12.2.1.0 and Oracle Traffic Director (OTD) is configured, this step needs to be run only once on the service instance. The REST API operation downloads both the WebLogic Server and OTD upgrade binaries and places them on the respective virtual machines.
  3. This process may take some time. Wait several minutes, and then check the job status:
    https://<rest_server_url>/paas/api/v1.1/activitylog/<identity_domain>/job/{jobId}

The upgrade patch downloads the WebLogic Server 12.2.1.4 or 12.2.1.3 binaries to each virtual machine on the specified Oracle Java Cloud Service instance, then updates Oracle Java Cloud Service to indicate that the specified upgrade patch has been applied to the service instance.

Note:

Applying the upgrade patch does not perform the actual process of upgrading the WebLogic Server software to the 12.2.1.4 or 12.2.1.3 version. This is done later by manually installing the binaries and using the WebLogic Upgrade Assistant.

Stop All WebLogic Server Processes

Before you upgrade the service instance, you must stop all Oracle WebLogic Server and Node Manager processes.

If you did not enable console access for this service instance, see Enable Console Access for a Service Instance.

  1. Access the Oracle Java Cloud Service console.
  2. Click Manage this instance menu icon for the service instance, and then select Open WebLogic Server Administration Console.
  3. Enter the WebLogic Server administrator user name and password.
  4. From Domain Structure, expand Environment.
  5. Click Servers.
  6. Click the Control tab.
  7. Click the check box to the left of each server.
  8. Click Shutdown, and then select Force Shutdown Now.
    When you shut down the Administration Server, a message warns you that the browser session will end.
  9. Access the Administration Server node as the oracle user.
  10. If you're upgrading from WebLogic Server release 12.2.1.0, 12.2.1.2, or 12.2.1.3, enter the following command to stop the Node Manager:
    /u01/data/domains/<domain>/bin/stopNodeManager.sh
  11. If you're upgrading from WebLogic Server 11g (11.1.1.7), use the following procedure to stop the Node Manager:
    1. Start WLST.
      /u01/app/oracle/middleware/oracle_common/common/bin/wlst.sh
    2. Connect to the Node Manager.
      nmConnect('<username>', '<password>', '<host_name>', '5556',  '<domain>', '/u01/data/domains/<domain>', 'ssl')
      For example:
      nmConnect('MyName', '<password>', 'example1-wls-1', '5556', 'Example1_domain', '/u01/data/domains/Example1_domain', 'ssl')
    3. Stop the Node Manager.
      stopNodeManager()
    4. Quit WLST.
      exit()
  12. Verify that no weblogic.Server or weblogic.NodeManager processes are running on this node.
    jps -l
  13. Repeat Steps 10 through 12 on the Managed Server nodes in this service instance.
  14. If your service instance is based on WebLogic Server 12.2.1.3, 12.2.1.2, or 12.2.1.0 and Oracle Traffic Director (OTD) is configured:
    1. Stop the OTD node.
      /u01/data/otd-instance/otd_domain/config/fmwconfig/components/OTD/instances/<lb-instance-name>/bin/stopserv
    2. Stop the NodeManager.
      /u01/data/otd-instance/otd_domain/bin/stopNodeManager.sh
    3. Stop the AdminServer.
      /u01/data/otd-instance/otd_domain/bin/stopWebLogic.sh

Install the Upgrade Software

Replace the current Oracle Fusion Middleware (FMW) and Java Development Kit (JDK) software installations in your service instance with the new versions of this software.

These product binaries include tools to upgrade your WebLogic Server domain and infrastructure database schema.
  1. Access the Administration Server node as the oracle user.
  2. Identify the location of the new Oracle Fusion Middleware binaries under the directory /u01/app/oracle/middleware/jcs/FMW.
    Example:
    /u01/app/oracle/middleware/jcs/FMW/12.2.1.3.190115/190115/fmiddleware.zip
  3. Identify the location of the new JDK binaries under the directory /u01/jdk/jcs/JDK.
    Example:
    /u01/jdk/jcs/JDK/8.0.201/190115/jdk.zip
  4. Move the new Oracle Fusion Middleware and JDK binaries to a temporary location.
    Example:
    mkdir /tmp/fmiddleware.zip
mkdir /tmp/jdk.zip
mv /u01/app/oracle/middleware/jcs/FMW/12.2.1.3.190115/190115/fmiddleware.zip/* /tmp/fmiddleware.zip
mv /u01/jdk/jcs/JDK/8.0.201/190115/jdk.zip/* /tmp/jdk.zip
  5. Delete the current Oracle Fusion Middleware and JDK installations, MIDDLEWARE_HOME and JAVA_HOME.
    rm -rf /u01/app/oracle/middleware/*
rm -rf /u01/jdk/*
  6. Move the new Oracle Fusion Middleware and JDK binaries from the temporary location to MIDDLEWARE_HOME and JAVA_HOME.
    mv /tmp/fmiddleware.zip/* /u01/app/oracle/middleware
mv /tmp/jdk.zip/* /u01/jdk
  7. Repeat Steps 4 to 6 on all Managed Server nodes in this instance.
  8. Edit the MIDDLEWARE_HOME oraInst.loc file on the Administration Server, which contains the location of the central inventory (oraInventory).
    cat /u01/app/oracle/middleware/oraInst.loc
#Oracle Installer Location File Location
#Mon Jan 14 12:43:52 PST 2019
inst_group=dba
inventory_loc=/u01/app/oracle/middleware/inventory
  9. If your service instance is based on WebLogic Server 12.2.1.3, 12.2.1.2, or 12.2.1.0, and you have configured Oracle Traffic Director (OTD) for the service instance:
    1. Make directories and move the upgrade binaries.

      For example:

      mkdir -p /tmp/wls_otd.zip
mkdir -p /tmp/jdk.zip

mv /u01/app/oracle/middleware/jcs/lb/12.2.1.4.190910/190910/wls_otd.zip/* /tmp/wls_otd.zip/
mv /u01/jdk/jcs/JDK/8.0.241/200107/jdk.zip/* /tmp/jdk.zip/
    2. Remove the existing installations and copy the new binaries.

      For example:

      rm -rf /u01/app/oracle/middleware/*
rm -rf /u01/jdk/*

mv /tmp/wls_otd.zip/* /u01/app/oracle/middleware
mv /tmp/jdk.zip/* /u01/jdk/

Perform a Readiness Check

Perform a readiness check to determine if your service instance is ready for upgrade.

  1. Start the Upgrade Assistant.
    export USER_MEM_ARGS=-Djava.security.egd=file:/dev/urandom
/u01/app/oracle/middleware/oracle_common/upgrade/bin/ua -readiness

    Setting USER_MEM_ARGS to use the /dev/urandom device reduces the time it takes to run the Oracle Fusion Middleware upgrade tools.

  2. Use the Upgrade Assistant to perform a readiness check, as described in Upgrading to the Oracle Fusion Middleware Infrastructure:

    If your service instance is based on WebLogic Server 12.2.1.3, 12.2.1.2, or 12.2.1.0, and you have configured Oracle Traffic Director (OTD) for the service instance, see also Starting the Upgrade Assistant for Oracle Traffic Director 12c Specific Upgrade.

  3. On the Readiness Check Type screen, select the domain-based readiness check.
    The domain-based readiness check enables the Upgrade Assistant to discover and select all upgrade-eligible schemas or component configurations in the domain specified in the Domain Directory field.
  4. On the End of Readiness screen in the Upgrade Assistant, review the results of the readiness check (Readiness Success or Readiness Failure).
    • If the readiness check is successful, click View Readiness Report to review the complete report. Oracle recommends that you review the Readiness Report before you perform the upgrade even when the readiness check is successful. Use the Find option to search for a particular word or phrase within the report. The report also indicates where the completed Readiness Check Report file is located.
    • If the readiness check encounters an issue or error, click View Log to review the log file, identify and correct the issues, and then restart the readiness check.

Upgrade the Infrastructure Database Schemas

After you perform a readiness check, use the Upgrade Assistant to upgrade the infrastructure database schemas. The procedures for upgrading from a previous WebLogic Server 12c release and a WebLogic Server 11g release differ, so they are presented in different topics.

Topics:

Upgrade the WebLogic Server 12c Infrastructure Database Schemas

Upgrade the previous WebLogic Server 12c database schemas by using the Upgrade Assistant (UA).

Note:

Use this procedure when upgrading from WebLogic Server release 12.2.1.0, 12.2.1.2, or 12.2.1.3.

The Upgrade Assistant creates missing or required schemas by using the default schema settings.

Ensure that you have backed up the database. See Perform Prerequisite Tasks.
  1. Start the Upgrade Assistant if you have not already done so. For example:
    export USER_MEM_ARGS=-Djava.security.egd=file:/dev/./urandom
/u01/app/oracle/middleware/oracle_common/upgrade/bin/ua
  2. Complete the steps as described in Upgrading to the Oracle Fusion Middleware Infrastructure:
  3. On the Selected Schemas screen, select All Schemas Used by a Domain, and then enter a domain directory name in the Domain Directory field.
    The All Schemas Used by a Domain selection allows the Upgrade Assistant to discover and select all components that have a schema available to upgrade in the domain specified in the Domain Directory field. This is also known as a domain-assisted schema upgrade. In addition, the Upgrade Assistant prepopulates connection information on the schema input screens.
  4. On the Upgrade Progress screen in the Upgrade Assistance, monitor the schema upgrade progress.
  5. Finish the schema upgrade process.
    • If the schema upgrade succeeds, click Close to complete the upgrade and close the wizard.
    • If the upgrade fails, click View Log to view and troubleshoot the errors. The logs are available in the following directory: 
      /u01/app/oracle/middleware/oracle_common/upgrade/logs
  6. Remedy the database connection failure if one occurs. See Problems with Database Connectivity When Upgrading the Infrastructure Schema Database.
  7. Verify the schema upgrade was successful by checking that the schemas in schema_version_registry have been properly updated. See Verifying the Schema Upgrade (upgrade to 12.2.1.4) or Verifying the Schema Upgrade (upgrade to 12.2.1.3) in Upgrading to the Oracle Fusion Middleware Infrastructure.
    One way to verify the schema upgrade is to use SQL*Plus commands to obtain data from the SCHEMA_VERSION_REGISTRY.
    1. Find the Oracle Java Cloud Service instance's schema prefix in the Upgrade Assistant log file at /u01/app/oracle/middleware/oracle_common/upgrade/logs.
    2. Connect to the database as a user having Oracle DBA privileges and run the following commands from SQL*Plus to get the current version numbers.
      sqlplus / as sysdba
SQL> connect <user_name>/<password>@<host_name>:<port>/<service_name> as sysdba
SQL> SELECT MRC_NAME,COMP_ID,OWNER,VERSION,STATUS,UPGRADED FROM SCHEMA_VERSION_REGISTRY WHERE MRC_NAME like 'SP1556690734';
      Example output for the SCHEMA_VERSION_REGISTRY:
      MRC_NAME COMP_ID OWNER VERSION STATUS UPGRADED
      SP1556690734 IAU SP1556690734_IAU 12.2.1.2.0 VALID Y
      SP1556690734 IAU_APPEND SP1556690734_IAU_APPEND 12.2.1.2.0 VALID N
      SP1556690734 IAU_VIEWER SP1556690734_IAU_VIEWER 12.2.1.2.0 VALID Y
      SP1556690734 MDS SP1556690734_MDS 12.2.1.3.0 VALID Y
      SP1556690734 OPSS SP1556690734_OPSS 12.2.1.0.0 VALID Y
      SP1556690734 STB SP1556690734_STB 12.2.1.3.0 VALID Y
      SP1556690734 UCSUMS SP1556690734_UMS 12.2.1.0.0 VALID N
      SP1556690734 WLS SP1556690734_WLS 12.2.1.0.0 VALID N

Upgrade the WebLogic Server 11g Infrastructure Database Schemas

Upgrade the previous WebLogic Server 11g database schemas by using the Upgrade Assistant (UA).

Note:

Use this procedure when upgrading from WebLogic Server release 11g (11.1.1.7).

The Upgrade Assistant creates required WebLogic Server 12c schemas by using the default schema settings.

One of the missing schemas is the Service Table schema (<prefix>_STB), which is new in WebLogic Server 12c and is required for domain-based upgrades. The Service Table schema stores basic schema configuration information (for example, schema prefixes and passwords) that can be accessed and used by other Oracle Fusion Middleware components when creating the domain. The WebLogic Services schema (<schema_prefix>_WLS) is also new in WebLogic Server 12c and is required for domain-based upgrades.

Before upgrading the infrastructure database schemas, ensure that you have backed up the database. See Perform Prerequisite Tasks.
  1. Start the Upgrade Assistant if you have not already done so. For example:
    export USER_MEM_ARGS=-Djava.security.egd=file:/dev/./urandom
/u01/app/oracle/middleware/oracle_common/upgrade/bin/ua
  2. Complete the steps as described in Upgrading to the Oracle Fusion Middleware Infrastructure:
  3. On the Selected Schemas screen, select All Schemas Used by a Domain, and then enter a domain directory name in the Domain Directory field.
    The All Schemas Used by a Domain selection allows the Upgrade Assistant to discover and select all components that have a schema available to upgrade in the domain specified in the Domain Directory field. This is also known as a domain-assisted schema upgrade. In addition, the Upgrade Assistant prepopulates connection information on the schema input screens.
  4. On the Create Schemas screen:
    1. Accept the default selection for Create missing schemas for specified domain.
    2. Enter and confirm the password for the Common Infrastructure Services (STB schema) and the Oracle WebLogic Services (WLS) components. Use the password you specified when you created the Oracle Java Cloud Service instance.
      • For a service instance on Oracle Cloud Infrastructure Classic, use the WebLogic administrator password.
      • For a service instance on Oracle Cloud Infrastructure, the schema password is encrypted, so you must first change this password. Use the procedure in Change the Schema Password Manually.
    3. Locate the <schema_prefix>_IAU schema and note the generated schema prefix. You will need this value if you need to change the password for the schema later.
  5. On the Upgrade Progress screen in the Upgrade Assistant, monitor the schema upgrade progress.
  6. Finish the schema upgrade process.
    • If the schema upgrade succeeds, click Close to complete the upgrade and close the wizard.
    • If the upgrade fails, click View Log to view and troubleshoot the errors. The logs are available in the following directory: 
      /u01/app/oracle/middleware/oracle_common/upgrade/logs
  7. Run the Upgrade Assistant a second time to manually upgrade the <schema_prefix>_IAU schema.
    This step is necessary because the Upgrade Assistant only upgrades schemas used by the domain. In WebLogic Server 11g, the <schema_prefix>_IAU schema is not used in the domain by default. Therefore, the schema is not upgraded by Upgrade Assistant and you must upgrade the schema manually.
    1. Start the Upgrade Assistant. For example:
      export USER_MEM_ARGS=-Djava.security.egd=file:/dev/./urandom
/u01/app/oracle/middleware/oracle_common/upgrade/bin/ua
    2. Select Individually Selected Schema, and then Oracle Audit Services.
    3. In the Database Connect String field, enter the same connect string as the one used by the other schemas.
    4. In the Schema User Name field, enter the user name with the same schema prefix used by the other schema.
    5. Enter the schema password.
      If you do not know the schema password, change the password by using the procedure in Change the Schema Password Manually. You will need the schema prefix you noted in Step 4.
  8. Remedy the database connection failure if one occurs. See Problems with Database Connectivity When Upgrading the Infrastructure Schema Database.
  9. Verify the schema upgrade was successful by checking that the schemas in SCHEMA_VERSION_REGISTRY have been properly updated. See Verifying the Schema Upgrade (upgrade to 12.2.1.4) or Verifying the Schema Upgrade (upgrade to 12.2.1.3) in Upgrading to the Oracle Fusion Middleware Infrastructure.
    One way to verify the schema upgrade is to use SQL*Plus commands to obtain data from the SCHEMA_VERSION_REGISTRY.
    1. Find the Oracle Java Cloud Service instance's schema prefix in the Upgrade Assistant log file at /u01/app/oracle/middleware/oracle_common/upgrade/logs.
    2. Connect to the database as a user having Oracle DBA privileges and run the following commands from SQL*Plus to get the current version numbers.
      sqlplus / as sysdba
SQL> connect <user_name>/<password>@<host_name>:<port>/<service_name> as sysdba
SQL> SELECT MRC_NAME,COMP_ID,OWNER,VERSION,STATUS,UPGRADED FROM SCHEMA_VERSION_REGISTRY WHERE MRC_NAME like 'SP1556656982';
      Example output for the SCHEMA_VERSION_REGISTRY:
      MRC_NAME COMP_ID OWNER VERSION STATUS UPGRADED
      SP1556656982 IAU SP1556656982_IAU 12.2.1.2.0 VALID Y
      SP1556656982 IAUDES SP1556656982_IAUDS 11.1.1.4.0 VALID N
      SP1556656982 IAU_APPEND SP1556656982_IAU_APPEND 12.2.1.2.0 VALID Y
      SP1556656982 IAU_VIEWER SP1556656982_IAU_VIEWER 12.2.1.2.0 VALID Y
      SP1556656982 MDS SP1556656982_MDS 12.2.1.3.0 VALID Y
      SP1556656982 OPSS SP1556656982_OPSS 12.2.1.0.0 VALID Y
      SP1556656982 STB SP1556656982_STB 12.2.1.3.0 VALID N
      SP1556656982 WLS SP1556656982_WLS 12.2.1.0.0 VALID N

Reconfigure the Domain

After you upgrade the infrastructure database schemas, use the Reconfiguration Wizard to reconfigure the domain. The procedures for reconfiguring the WebLogic Server 12c (12.2.1.0, 12.2.1.2, or 12.2.1.3) and WebLogic Server 11g (11.1.1.7) domains differ, so they are presented in different topics.

Topics:

Reconfigure the WebLogic Server 12c Domain

Reconfigure the domain by using the Reconfiguration Wizard.

Note:

Do not reconfigure the domain if you are upgrading an existing WebLogic Server release 12.2.1.3 to WebLogic Server release 12.2.1.4 for an existing Oracle Java Cloud Service instance. If you skip this step in the upgrade process, the domain config.xml, specifically, the domain-version is not updated, and you still see the earlier version. This is a known issue and you need not run the upgrade again. See Incorrect Version Numbers After a Reduced Downtime Upgrade . After the upgrade is complete, you see the correct version in the admin server logs and in the WebLogic Server Administration Console.

However, if your service instance is based on WebLogic Server 12.2.1.3, 12.2.1.2, or 12.2.1.0, and you have configured Oracle Traffic Director (OTD) for the service instance, you must reconfigure the domain on your OTD instances. See also Reconfiguring the Existing Oracle Traffic Director Domain in Graphical Mode.

Before you begin, ensure you have backed up the domain on all nodes. See Perform Prerequisite Tasks.
  1. Start the Reconfiguration Wizard as user oracle with the following logging options, with log_file as the absolute path of the log file you'd like to create for the domain reconfiguration session. This can be helpful if you need to troubleshoot the reconfiguration process.
    For example:
    /u01/app/oracle/middleware/oracle_common/common/bin/reconfig.sh -log_priority=all -log="/u01/reconfig0212.log"
  2. Perform the reconfiguration tasks as described in Upgrading to the Oracle Fusion Middleware Infrastructure.
  3. On the Advanced Configuration screen of the Reconfiguration Wizard, select Deployment and Services.
  4. Target the wsm-pm app to the Administration Server.
  5. Click Reconfig.
  6. Check the End of Configuration screen to learn whether the reconfiguration process completed successfully or failed.
    • If the reconfiguration is successful, Oracle WebLogic Server Reconfiguration Succeeded is displayed. The location of the domain that was reconfigured as well as the Administration Server URL (including the listen port) are displayed as well.
    • If the reconfiguration process did not complete successfully, an error message is displayed which indicates the reason. Take appropriate action to resolve the error.

Reconfigure the WebLogic Server 11g Domain

Reconfigure the WebLogic Server 11g domain by using the Reconfiguration Wizard.

Note:

Use this procedure if you are upgrading from WebLogic Server 11g (11.1.1.7).
Before you begin, ensure you have backed up the domain on all nodes. See Perform Prerequisite Tasks.
  1. Start the Reconfiguration Wizard as the oracle user. Specify the following logging options, including the absolute path of the log file you'd like to create for the domain reconfiguration session.
    For example:
    /u01/app/oracle/middleware/oracle_common/common/bin/reconfig.sh -log_priority=all -log="/u01/reconfig0212.log"
    You can use this log file to troubleshoot any reconfiguration problems.
  2. Perform the reconfiguration tasks shown in Upgrading to the Oracle Fusion Middleware Infrastructure.

    Follow the special instructions below for the following screens: Database Configuration Type, Component Datasources, Node Manager, and Advanced Configuration.

  3. On the Database Configuration Type screen, enter the RCU data.
    The Service Table schema (<prefix>_STB) is new in WebLogic Server 12c, so it's not present in WebLogic Server 11g. As a result, you must enter the RCU data manually.
    1. Select RCU Data to connect to the Server Table (_STB) schema. The Reconfiguration Wizard uses this connection to automatically configure the data sources required for components in your domain.
    2. Enter the database connection details using the RCU service table (_STB) schema credentials. Note that if your service instance is on Oracle Cloud Infrastructure, you changed this password when you upgraded the infrastructure database schema. See Upgrade the WebLogic Server 11g Infrastructure Database Schemas.
    3. Click Get RCU Configuration.
    4. Click Next.
  4. If your database has _OPSS or _IAU WebLogic Server 11g database schemas, enter the database connection details on the Component Datasources screen, and then click Next.
    • The _IAU or _OPSS schema displayed on the Component Datasources screen has no connection data, so the default connection data is used.
    • For each schema (<schema_prefix>_IAU, <schema_prefix>_IAU_APPEND, <schema_prefix>_IAU_VIEWER, and schema_prefix_OPSS) enter data in the Host Name, DBMS Service, Port, Schema Owner, and Schema Password fields for each of the IAU schemas. The Host Name, DBMS Service, Port, and Schema Password are the same as those used for the other schema.
  5. Migrate the per-host Node Manager configuration to a per-domain configuration for the reconfigured domain.

    The Node Manager screen is only displayed if the domain you are reconfiguring is currently using a per-host Node Manager. Because you are upgrading your service instance from WebLogic Server 11g, the Node Manager screen appears, and you must migrate the existing per-host Node Manager configuration.

    1. On the Node Manager screen, select Per Domain Default Location for the Node Manager Type.
      In this configuration, the Node Manager home is redefined to $domain_name/nodemanager.
    2. Select Migrate Existing Configuration for Node Manager Configuration, browse and select your Node Manager Home, and then select Apply Oracle Recommended Defaults.
      The existing per-host Node Manager configuration will be migrated to a per-domain configuration for the reconfigured domain.
    3. For Node Manager Credentials, specify the username and password that you want to use to start Node Manager in the reconfigured domain.
    4. Click Next.
  6. On the Advanced Configuration screen of the Reconfiguration Wizard, select Deployment and Services.
  7. Target the wsm-pm app to the Administration Server.
  8. Click Reconfig.
  9. Check the End of Configuration screen to learn whether the reconfiguration process completed successfully or failed.
    • If the reconfiguration is successful, Oracle WebLogic Server Reconfiguration Succeeded is displayed. The location of the domain that was reconfigured as well as the Administration Server URL (including the listen port) are displayed as well.
    • If the reconfiguration process did not complete successfully, an error message is displayed which indicates the reason. Take appropriate action to resolve the error.

Upgrade the Domain

Use the Upgrade Assistant to update the domain component configurations to match the updated domain configuration.

Before you begin, you must first run the Reconfiguration Wizard to reconfigure the WebLogic domain to WebLogic Server release 12.2.1.3.
  1. Start the Upgrade Assistant, for example:
    export USER_MEM_ARGS=-Djava.security.egd=file:/dev/./urandom
/u01/app/oracle/middleware/oracle_common/upgrade/bin/ua
  2. Use the Upgrade Assistant to upgrade the domain configurations, as described in Upgrading to the Oracle Fusion Middleware Infrastructure.

    If your service instance is based on WebLogic Server 12.2.1.3, 12.2.1.2, or 12.2.1.0, and you have configured Oracle Traffic Director (OTD) for the service instance, see also Upgrading Collocated Oracle Traffic Director in Graphical Mode.

  3. On the All Configurations screen, select All Configurations Used by a Domain and specify your domain location in the Domain Directory field. Enter the domain directory directly or click Browse to select a valid domain directory.
  4. On the Upgrade Summary page, review the summary of the options you have selected for the component configuration upgrade, and then click Upgrade to start the upgrade process.
  5. View the Upgrade Progress page to monitor the upgrade.
  6. View the results and finish the upgrade.
    • If the upgrade succeeds, the Upgrade Success page is displayed. Click Close to complete the upgrade and close the wizard.
    • If the upgrade fails, the Upgrade Failure screen is displayed. Click View Log to view and troubleshoot the errors. The logs are available at 
      ORACLE_HOME/oracle_common/upgrade/logs
  7. If you are upgrading the WebLogic Server release from 11g, connect to the Administration Server and set the CrashRecoveryEnabled property to true in the <domain_home>/nodemanager/nodemanager.properties file.
    CrashRecoveryEnabled=true
    Repeat this process for each Managed Server.
    The Node Manager CrashRecoveryEnabled configuration property is used to restart servers after a system crash. During domain reconfiguration, this property is reset to false, so it's not enabled by default.

Restart the Administration Server Node

After you complete the upgrade of your service instance, restart the Node Manager and WebLogic Server processes on the first node in your instance.

  1. Access the Administration Server node as the oracle user.
  2. Start the Node Manager process.
    /u01/data/domains/<domain>/bin/startNodeManager.sh >nm.out 2>&1 &
  3. Launch the WebLogic Scripting Tool (WLST).
    $MIDDLEWARE_HOME/oracle_common/common/bin/wlst.sh
  4. Connect to the Node Manager.
    nmConnect(username="<nm_user>", password="<nm_password>", domainName="<domain>", domainDir="/u01/data/domains/<domain>", nmType="ssl", host="<hostname>", port="5556", verbose="false")

    Example:

    nmConnect(username="weblogic", password="<nm_password>", domainName="MyInstan_domain", domainDir="/u01/data/domains/MyInstan_domain", nmType="ssl", host="myinstance-wls-1", port="5556", verbose="false")

    By default, the Node Manager credentials are the same as the WebLogic Server credentials that you specified when you created the instance.

  5. Start the Administration Server and the first Managed Server.
    nmStart('<server>')
nmServerStatus('<server>')

    Example:

    nmStart('MyInstan_adminserver')
nmServerStatus('MyInstan_adminserver')
nmStart('MyInstan_server_1')
nmServerStatus('MyInstan_server_1')
  6. Exit WLST.
    exit()
  7. If your service instance is based on WebLogic Server 12.2.1.3, 12.2.1.2, or 12.2.1.0, and you have configured Oracle Traffic Director (OTD) for the service instance:
    1. Make sure execute permissions are in place for the scripts in otd_domain.

      For example:

      chmod -R +x /u01/data/otd-instance/
    2. Start the AdminServer.
      nohup /u01/data/otd-instance/otd_domain/bin/startWebLogic.sh & 2>&1
    3. Start the NodeManager.
      nohup /u01/data/otd-instance/otd_domain/bin/startNodeManager.sh & 2>&1
    4. Start the OTD node.
      /u01/data/otd-instance/otd_domain/config/fmwconfig/components/OTD/instances/<lb-instance-name>/bin/startserv

Update and Restart the Managed Server Nodes

If your service instance has one or more Managed Server nodes, then update the domain on these nodes before restarting the Managed Servers.

  1. Inspect all your managed server nodes to ensure that the 12.2.1.4 or 12.2.1.3 binaries previously downloaded using the REST API are in the /u01/app/oracle/middleware directory and the /u01/jdk directory. See Install the Upgrade Software.
  2. Access the Administration Server node as the oracle user.
  3. Use the pack command to create a Managed Server template from your domain.
    /u01/app/oracle/middleware/oracle_common/common/bin/pack.sh -domain=/u01/data/domains/<domain> -template_name=managedServerTemplate -template=/tmp/managed_server_template.jar -managed=true -log=/tmp/pack.log
  4. Upload the template file from the Administration Server node to a Managed Server node in this service instance.
    scp /tmp/managed_server_template.jar <node_host>:/tmp

    Example:

    scp /tmp/managed_server_template.jar myinstance-wls-2:/tmp
  5. Connect to the Managed Server node.
    Example:
    ssh myinstance-wls-2
  6. Use the unpack command to apply the Managed Server template to the domain directory on this node.
    /u01/app/oracle/middleware/oracle_common/common/bin/unpack.sh -domain=/u01/data/domains/<domain> -template=/tmp/managed_server_template.jar -overwrite_domain=true -log=/tmp/unpack.log

    Ignore any warnings about "invalid or missing JDBC datasource connection parameters."

  7. Start the Node Manager process on this node.
    /u01/data/domains/<domain>/bin/startNodeManager.sh >nm.out 2>&1 &
  8. Disconnect from the Managed Server node.
  9. Repeat Steps 3 to 6 on all other Managed Server nodes in your service instance.
  10. Access the WebLogic Server Administration Console for your service instance.
  11. From Domain Structure, expand Environment.
  12. Click Servers.
  13. Click the Control tab.
  14. Click the check box to the left of each server that is not running.
  15. Click Start. When prompted for confirmation, click Yes.

Perform Post-Upgrade Tasks

Perform any needed post-upgrade tasks.

After upgrading the WebLogic Server release for your Oracle Java Cloud Service instance, perform only those tasks described in Upgrading to the Oracle Fusion Middleware Infrastructure that apply to your upgraded environment.

For example, see the topics Using the Upgrade Validation Checklist, Verifying the Domain-Specific-Component Configurations Upgrade, and Reapplying Custom Configuration Settings to setDomainEnv.

Roll Back an Upgrade

If the upgrade fails, you can roll back the upgrade.

Rolling back an upgrade returns the WebLogic Servers for your Oracle Java Cloud Service instance to their previous release number.
  1. Shutdown all WebLogic Server processes, including Node Manager, on each node. See Stop All WebLogic Server Processes.
  2. Use the REST API to get a list of applied patches.
    curl -i -X GET -u <user>:<password> -H "X-ID-TENANT-NAME:<identitydomain>" https://<rest_server_url>/paas/api/v1.1/instancemgmt/<identity-domain>/services/jaas/instances/<servicename>/patches/applied
    For example:
    curl -i -X GET -u username:password -H "X-ID-TENANT-NAME:ExampleIdentityDomain" https://<rest_server_url>/paas/api/v1.1/instancemgmt/ExampleIdentityDomain/services/jaas/instances/Example1/patches/applied
    This example output shows details about applied patches:
    [
   {
      "backupStatus": "Available", 
      "rollbackStatus":"Available",                 
      "additionalNote": "This note is the default note: Applying patch [wls_upg_12.2.1.3.190115_for_12cRelease212].",
      "appliedBy": "weblogic",
      "appliedDate":"2019-03-07T19:07:37.246+0000",
      "totalTime": "2 min, 11 sec",
      "patchId":"wls_upg_12.2.1.3.190115_for_12cRelease212",                
      "patchDescription": "WebLogic Server 12.2.1.3.0 with PSU Update 12.2.1.3.190115",
      "patchReleaseUrl":"https://support.oracle.com/epmos/faces/PatchDetail?patchId=28710939",
      "releaseDate": "2019-01-14T17:40:00.000+0000",                
      "lastStatus": "COMPLETED",
      "lastStatusMessage": "No errors",
      "componentPatches": {},                
      "patchType": "PSU",
      "patchCategory": "MajorPatch",
      "patchSeverity": "Normal",
      "jobId":"108717",
      "displayName": "12.2.1.3.190115",
      "inProgress": false,
      "operationType":"None",
      "id": 16,                
      "patchingResult": {
         "patchingId": 16,
         "versionBeforeThisPatch": "OTD 12.2.1.2.190115,WLS 12.2.1.2.190128,SERVICEVERSION 12cRelease212",                    
         "strategy": "Rolling",                    
         "releaseVersionBeforeThisPatch": "12.2.1.2.190128",                    
         "customRollbackId": "108717_1551985657246",                    
         "startDate": "2019-03-07T19:07:37.246+0000",                    
         "endDate": "2019-03-07T19:09:48.411+0000",                    
         "patchingStatus": "COMPLETED","resultMessage":"No errors",
         "additionalNote": "This note is the default note:Applying patch [wls_upg_12.2.1.3.190115_for_12cRelease212].",
         "appliedBy":"weblogic",                     
         "jobId": "108717",
         "completeLog": "",
         "progressMessages": [                        
            "7:07:37.153 PM Phase initialize started",                        
            "7:07:37.445 PM Phase initialize completed",
            "7:07:37.718 PM patching.action.tools.phase_started",
            "7:07:37.861 PM patching.action.tools.phase_completed",                        
            "7:07:38.067 PM Phase backup started",                        
            "7:07:38.214 PM Phase backup completed",                        
            "7:07:38.529 PM patching.action.patch-pre-action.phase_started",
            "7:07:38.683 PM patching.action.patch-pre-action.phase_completed",
            "7:07:38.949 PM Phase patch started",
            "7:07:31.043 PM PSM-PATCH-60099: Removing any retained old artifacts",                         
            "7:07:36.767 PM patching.progress.remove.left.over.binary.completed$jcs/FMW/12.2.1.3.190115/190115/fmiddleware.zip$pltinstance-wls-1",
            "7:07:36.767 PM Retrieving pre-patched binary artifact [jcs/FMW/12.2.1.3.190115/190115/fmiddleware.zip] on vm [pltinstance-wls-1]",
            "7:07:49.404 PM Retrieved pre-patched binary artifact [jcs/FMW/12.2.1.3.190115/190115/fmiddleware.zip] on vm [pltinstance-wls-1]",
            "7:07:49.404 PM Unpacking binary [jcs/FMW/12.2.1.3.190115/190115/fmiddleware.zip] on vm [pltinstance-wls-1]",
            "7:08:35.616 PM Unpacked binary [jcs/FMW/12.2.1.3.190115/190115/fmiddleware.zip] on vm [pltinstance-wls-1]",
            "7:08:35.616 PM PSM-PATCH-60099: Removing any retained old artifacts",            
            "7:08:36.283 PM patching.progress.remove.left.over.binary.completed$jcs/JDK/8.0.201/190115/jdk.zip$pltinstance-wls-1",
            "7:08:36.283 PM Retrieving pre-patched binary artifact [jcs/JDK/8.0.201/190115/jdk.zip] on vm [pltinstance-wls-1]",
            "7:08:38.250 PM Retrieved pre-patched binary artifact [jcs/JDK/8.0.201/190115/jdk.zip] on vm         [pltinstance-wls-1]",
            "7:08:38.250 PM Unpacking binary [jcs/JDK/8.0.201/190115/jdk.zip] on vm [pltinstance-wls-1]",
            "7:08:44.180 PM Unpacked binary [jcs/JDK/8.0.201/190115/jdk.zip] on vm [pltinstance-wls-1]",
            "7:08:44.180 PM Retrieving pre-patched binary artifact [jcs/lb/12.2.1.2.0/161019/wls_otd.zip] on vm         [pltinstance-wls-1]",
            "7:08:44.181 PM Retrieved pre-patched binary artifact [jcs/lb/12.2.1.2.0/161019/wls_otd.zip] on vm         [pltinstance-wls-1]",
            "7:08:44.181 PM Retrieving pre-patched binary artifact [jcs/JDK/8.0.201/190115/jdk.zip] on vm         [pltinstance-wls-1]",
            "7:09:46.509 PM Phase patch completed",
            "7:09:46.771 PM patching.action.patch-post-action.phase_started",  
            "7:09:46.937 PM patching.action.patch-post-action.phase_completed",
            "7:09:48.227 PM Phase finalize started",
            "7:09:48.352 PM Completed"
         ]                 
      },                
      "rollbackId":"16",                 
      "rollbackVersion":"12.2.1.2.190128",                 
      "currentPatchLevel":"12.2.1.3.190115",                 
      "isUpgrade": true,
      "appliedPatchGuiMetadata": {                    
         "supportsPreRollbackCheck": false                 
      }            
   }
]
  3. From the response, find the previously applied upgrade patch, patchId, and then find the value of rollbackId. In this example, the rollbackId is 16.
  4. Use the REST API to roll back the patch.
    curl -i -X PUT -u <user>:<password> -d {} -H "Content-Type:application/json" -H "X-ID-TENANT-NAME:<identitydomain>" https://<rest_server_url>/paas/api/v1.1/instancemgmt/<identitydomain>/services/jaas/instances/<servicename>/patches/<rollbackid>/rollback
    For example:
    curl -i -X PUT -u username:password -d {} -H "Content-Type:application/json" -H "X-ID-TENANT-NAME:ExampleIdentityDomain" https://<rest_server_url>/paas/api/v1.1/instancemgmt/ExampleIdentityDomain/services/jaas/instances/Example1/patches/16/rollback
    Example output:
    {
   "status":"Completed",
   "details":{
     "message":"PSM-PATCH-50038:  Rollback of service from patch [wls_upg_12.2.1.3.190115_for_12cRelease212] is submitted as an  asynchronous job.",
     "jobId":"206236"
   }
}
  5. Restore the backup of the database instance.
    • If you backed up an Oracle Database Cloud Service deployment, use the procedure shown in Restoring from a Specific Backup in Administering Oracle Database Cloud Service.
    • If you backed up an Oracle Cloud Infrastructure database, use the procedure shown in To restore a database from a specific backup from Object Storage in Recovering a Database from Object Storage in the Oracle Cloud Infrastructure documentation.
  6. Replace the files in MIDDLEWARE_HOME (/u01/app/oracle/middleware), JDK_HOME (/u01/jdk), and domain directory (/u01/data/domains) on each node with those you saved earlier. See Perform Prerequisite Tasks.
  7. Restart the Node Manager and WebLogic Server processes on the Administration Server node. See Restart the Administration Server Node.
  8. Restart the Node Manager and WebLogic Server processes on the Managed Server nodes. See Update and Restart the Managed Server Nodes.
The WebLogic Server installation and domain for your Oracle Java Cloud Service instance have been returned to their previous release number.