Upgrade a 12.2.1.4 or 14.1.1.0 Domain to 15.1.1.0

Use these steps to upgrade your WebLogic Server for OCI 12.2.1.4 or 14.1.1.0 stack to 15.1.1.0.

This document describes a reduced set of steps from the Upgrading Oracle WebLogic Server to 15.1.1.0.0 guide that are amended to support the recommended upgrade procedure for WLS for OCI stack instances. You should refer to official Oracle WebLogic upgrade guide when in doubt or encounter problems with the prescribed steps.

When you upgrade your 12.2.1.4 or 14.1.1.0 stack to 15.1.1.0, you create a cloned copy of the stack's domain. This process will not affect the original stack, but the new 15.1.1.0 stack instances will have different IPs.

Note:

If your 12.2.1.4 stack is using ADF do not upgrade to 15.1.1.0, upgrade to 14.1.2.0.

The following are the advantages of this documented process over the traditional in-place upgrade:

The stack has the latest scripts on disk volumes.
There is an easier roll back process. Since the original stack with 12.2.1.4 or 14.1.1.0 is still available no action needs to be taken to use the original working instance.
The WebLogic binaries are the latest. You do not have to locate the WebLogic installers and apply the required patch

Note:

This guide is intended for 12.2.1.4 or 14.1.1.0 upgrades to 15.1.1.0. However, if you are using this guide to assist you with a 14.1.2.0 to 15.1.1.0 upgrade please consult the Maintain FIPS 140-2 Compliance section of the Upgrading Oracle WebLogic Server to 15.1.1.0.0 guide to see if this applies to your use case.

To upgrade your OCI 12.2.1.4 or 14.1.1.0 stack to 15.1.1.0:

Identify Your Applications

Access your source 12.2.1.4 or 14.1.1.0 instance and list all your applications. Assuming that you are using the WebLogic cluster set up for you by WLS for OCI you can use the following commands to do this as the "oracle" user:

export WL_SERVER_HOME=$WL_HOME/server
export ADMIN_URL=t3://$(hostname -f):9071
export ADMIN_USER=$(python3 /opt/scripts/databag.py wls_admin_user)
"$JAVA_HOME/bin/java" -cp "$WL_SERVER_HOME/lib/weblogic.jar" weblogic.Deployer -adminurl "$ADMIN_URL" -username "$ADMIN_USER" -listapps

Find your source code and Maven build for each application identified.

Note:
If you deployed the "sample-app" as part of Stack creation you can ignore this application. It does not need to be upgraded.

Locate JAX-RPC in your applications

Support for JAX-RPC has been removed in WebLogic Server 15.1.1.0.0 and the UpgradeTo1511 recipe noted in the next section Upgrade Your Applications does not:

Convert JAX-RPC endpoint or client code to JAX-WS.
Convert EJB 2.x web-service EJBs to EJB 3.x.
Map WebLogic-specific JAX-RPC features to JAX-WS equivalents.
Update Ant jws, wsdlc, and clientgen tasks to type="JAXWS".
Regenerate JAX-WS client artifacts. Find and record any JAX-RPC usage with the following steps.

Locate JAX-RPC usage with the following command:

cd <directory with Maven POM file>
export SOURCE=src
grep -RInE --include='*.java' --include='*.xml' --include='*.properties' '(javax\.xml\.rpc|JAXRPC|jaxrpc|weblogic\.jws\.ServiceClient|weblogic\.jws\.(WLHttpTransport|WLJMSTransport)|weblogic\.wsee\.)' $SOURCE pom.xml build.xml 2>/dev/null

Based on results plan for action after the next section Upgrade Your Applications.
1. If there are no results skip the Convert JAX-RPC endpoint or client code to JAX-WS section.
2. If there are results start inspecting the application with its owner/developer; it needs a JAX-RPC-to-JAX-WS assessment before upgrading WLS to 15.1.1.0.0.
  Do not make the changes until after the completing the next section Upgrade Your Applications.
  Example of finding results:
```
src/foo-ws.xml:22:<webservice-type>JAXRPC</webservice-type>
src/foo.java:14:import javax.xml.rpc.Service;
```

Upgrade Your Applications

Note:

Refer to the common use cases in the Rewrite WebLogic recipesRewrite WebLogic recipes . These WLS recipes cover WebLogic, Java, Jakarta, JSF-related, descriptor, and Spring migration areas. Other dependencies still need to be discovered from the Maven POM and code, then addressed with another OpenRewrite recipe, a custom recipe, or manual changes.

Repeat the following for every application deployed.

Make sure you are using source control, and if not, make a copy of your entire application directory before proceeding.
Select the mandatory Java Recipe that matches your intended JDK for 15.1.1.0.0:
UpgradeToJava21 or UpgradeToJava17
Select the mandatory recipes:
JakartaEE9_1 and UpgradeTo1511

Build the repository and inspect the dependency tree with:

cd <directory with Maven POM file>
mvn clean install -DskipTests
mvn dependency:tree | grep -E 'org\.springframework|org\.springframework\.data|org\.hibernate|javax\.persistence|jakarta\.persistence'

Based on the results select the optional recipes:
1. If you see org.springframework:*:5.* or org.springframework.data:*, add this recipe:
```
com.oracle.weblogic.rewrite.spring.framework.UpgradeToSpringFramework_6_2
```
2. If you see org.hibernate:* or a JPA API such as javax.persistence:*, add this recipe:
```
com.oracle.weblogic.rewrite.hibernate.MigrateHibernateToJakartaEE9
```
3. If you see both, add both.

Confirm that you have the intended JDK for 15.1.1.0 in your path.

export JAVA_HOME=/path/to/jdk-21
export PATH="$JAVA_HOME/bin:$PATH"

Perform a run of the recipes:

Note:

When upgrading to WebLogic Server 15.1.1, the UpgradeTo1511 recipe must be run after the Java and Jakarta recipes.

Here is an example where "both" were found and JDK21 is the target Java version (consult the documentation at Rewrite WebLogic recipes for latest recipe names):

mvn -U org.openrewrite.maven:rewrite-maven-plugin:run -Drewrite.recipeArtifactCoordinates=org.openrewrite.recipe:rewrite-migrate-java:RELEASE,com.oracle.weblogic.rewrite:rewrite-weblogic:LATEST,org.openrewrite.recipe:rewrite-spring:RELEASE,org.openrewrite.recipe:rewrite-hibernate:RELEASE -Drewrite.activeRecipes=org.openrewrite.java.migrate.UpgradeToJava21,com.oracle.weblogic.rewrite.JakartaEE9_1,com.oracle.weblogic.rewrite.UpgradeTo1511,com.oracle.weblogic.rewrite.spring.framework.UpgradeToSpringFramework_6_2,com.oracle.weblogic.rewrite.hibernate.MigrateHibernateToJakartaEE9 -Drewrite.exportDatatables=true

Stop here and investigate if there are errors. Otherwise, continue to the next steps to perform follow up steps.

Search for ehcache usage since ehcache 2.0 was deprecated in Spring Framework 6.x
```
grep -RInE 'hibernate-ehcache|net\.sf\.ehcache|EhCacheCacheManager|EhCacheManagerFactoryBean' pom.xml <directory where code is at; typically "src">
```
1. If found replace it with a cache provider from the supported list of cache providers https://docs.spring.io/spring-boot/docs/3.0.8/reference/html/io.html#io.caching.provider
  
  Note:
  For an example of changing a cache provider review Step 4 in the Migrate Spring Framework PetClinic to WLS 15.1.1 tutorial.
2. Open up pom.xml and remove the entire dependency block containing the unsupported Hibernate ecache:
```
<artifactId>hibernate-ehcache</artifactId>
```
3. Optionally, since they aren't used anymore, remove any ehcache.xsd or ehcache.xml files.
Find remaining Java EE namespace imports to assess:
```
grep -RInE --include='*.java''^[[:space:]]*import[[:space:]]+javax\.' <directory where code is at; typically "src">
```
If anything is found inspect them individually. They may indicate incomplete migration work or a library that has not moved to Jakarta.
Verify that the intended JDK for 15.1.1.0 is the current one used.
```
java -version
```
If the Java version is not the intended JDK change your JAVA_HOME and PATH so that 'java -version' includes the expected JDK version.
Compile and package using the intended JDK
```
mvn clean package -DskipTests
```
If there are failures use the reported class, method, or dependency to decide whether a further third-party recipe exists at OpenRewrite recipes or whether a manual change is required.

Convert JAX-RPC endpoint or client code to JAX-WS

If you got any results from Locate JAX-RPC in your applications section for an application use this guidance to address the finding(s):

JAX-RPC client only	Recreate the client with JAX-WS tooling.
JAX-RPC service endpoint	Perform a manual JAX-WS conversion.
rpc/encoded WSDL or SOAP arrays	Treat as a design compatibility issue; JAX-WS does not support those constructs. Even a converted JAX-WS service needs design review if its WSDL uses rpc/encoded or SOAP arrays, which are unsupported by JAX-WS.

Compile and package using the intended JDK

cd <directory with Maven POM file>
mvn clean package -DskipTests

Backup the Database

Since ADF/JRF are not supported a database backup is not required. However, this is a best practice. Follow these guides depending on the database you are using for your applications:

Stop the WebLogic Server Processes on the Source Compute Instances

Log into the each source stack non-administration (no WLS admin server) compute instance as an opc user and run the following commands:
```
sudo su - oracle
/opt/scripts/restart_domain.sh -o stop
```
Run the following command to confirm if there are any running processes:
```
jps
```
If there are any processes running, then run kill -9 against each of the processes.
After stopping the processes on the non-administration compute instances perform steps 1 and 2 on the administration (has WLS admin server) compute instance.

Replace the Domain on the Target 15.1.1.0 Instance with Cloning

Create a new 15.1.1.0 WLS OCI stack. See Create a Stack.
Note:
- When creating the new 15.1.1.0 stack, use the same configuration as the source stack, but use a different resource prefix. If you are unsure of the exact set up of your source Stack review the Stack variables.
- 15.1.1.0 WLS for OCI stacks set "Secured Production Mode" by default while 12.2.1.4 and 14.1.1.0 stacks do not. Be sure to deselect this option when creating your new Stack instance.
- If you are using a reserved IP with your load balancer, then delete the load balancer on the source instance to free up the reserved IP address.
Clone each existing Compute instance in the domain using the WebLogic OCI cloning steps in Clone a non-JRF Instance. Clone only the data block volume, using the cloning command:
```
python3 /opt/scripts/cloning/create_clone.py -s <Original_instance_stack_OCID>
```
Note:
Cloning might fail at the StartServers step, as your applications may not be upgraded to handle 12.2.1.4 binaries. If this error occurs, ignore the error and continue with the next step. This should be:
```
python3 /opt/scripts/cloning/create_clone.py -p StartAppGateway
```
Note:
While StartServers will often fail if other steps in Cloning fail try to run them again using the command for the "retry the failed action" log entry retrieved from:
```
tail /u01/logs/provisioning.log
```
If the cloning action fails at DestroyDataVolume it is okay to proceed without completing this step. You will just have an unattached volume that you can remove later.

Upgrade the Domain

The servers are likely started through Cloning although the managed servers will likely be in ADMIN state due to failure of starting deployed applications that are still using javax. You will need to stop all the servers again before upgrading the domain.

Log into the each source stack non-administration (no WLS admin server) compute instance as an opc user and run the following commands:
```
sudo su - oracle
/opt/scripts/restart_domain.sh -o stop
```
Run the following command to confirm if there are any running processes:
```
jps
```
If there are any processes running, then run kill -9 against each of the processes.
After stopping the processes on the non-administration compute instances perform steps 1 and 2 on the administration (has WLS admin server) compute instance.

sudo su - oracle
/u01/app/oracle/middleware/oracle_common/common/bin/wlst.sh
readDomainForUpgrade('/u01/data/domains/<domain_name>')
updateDomain()
closeDomain()
exit()

Start Servers on Administration Compute Instance

While still logged into the administration (has WLS admin server) compute instance as the "oracle" user, run the following command to start the servers:

/opt/scripts/restart_domain.sh -o start

Deploy Upgraded Applications

Copy the applications that you modified in the Upgrade Your Applications and Convert JAX-RPC endpoint or client code to JAX-WS sections of this document to the Administration Compute instance by downloading them from OCI Object Storage or using scp through a bastion host.
Log into the Administration Compute instance and verify that servers are running:
```
sudo su - oracle
jps
```
You should see 2 "Server" processes running like the example below:
```
jps
182075 NetworkServerControl
486878 Jps
182009 NodeManager
182479 Server
182076 Server
```
If you don't see both Server processes investigate why they did not start by inspecting the logs. At least one managed server needs to be running.

Note:
Your servers on the Administration Compute instance will likely be in an ADMIN state due to failure to start applications that were not yet migrated. This is expected.

While still logged into the administration (has WLS admin server) compute instance as the "oracle" user deploy the applications that you modified in the Upgrade Your Applications and Convert JAX-RPC endpoint or client code to JAX-WS sections of this document. Example undeploy and deploy code using the PetClinic Tutorial application:

export WL_SERVER_HOME=$WL_HOME/server
export ADMIN_URL=t3://$(hostname -f):9071
export ADMIN_USER=$(python3 /opt/scripts/databag.py wls_admin_user)
export TARGET_CLUSTER=$(python3 /opt/scripts/databag.py wls_cluster_name)
export APP_NAME=petclinic
export WAR_FILE="/tmp/petclinic.war"
"$JAVA_HOME/bin/java" -cp "$WL_SERVER_HOME/lib/weblogic.jar" weblogic.Deployer -adminurl "$ADMIN_URL" -username "$ADMIN_USER" -name "$APP_NAME" -undeploy
"$JAVA_HOME/bin/java" -cp "$WL_SERVER_HOME/lib/weblogic.jar" weblogic.Deployer -adminurl "$ADMIN_URL" -username "$ADMIN_USER" -deploy -name "$APP_NAME" -source "$WAR_FILE" -targets "$TARGET_CLUSTER" -upload

Restart Servers

Beginning with the Administration Compute instance run the following on all Compute instances:
```
sudo su - oracle
/opt/scripts/restart_domain.sh -o restart
```
For each non Administration Compute instance verify that the domain was upgraded. It should be upon server start since the Admin Server was upgraded in the Upgrade the Domain step.

Check the state of each managed server to make sure it is in the RUNNING state. Example using Node Manager:

sudo su - oracle
/u01/app/oracle/middleware/oracle_common/common/bin/wlst.sh
nmConnect(username="weblogic", password="****", domainName="upgrade12to15_domain", domainDir="/u01/data/domains/upgrade12to15_domain", nmType="ssl", host="upgradedto15-wls-0", port="5556", verbose="false")
nmServerStatus("upgrade12to15_server_1")

If you still see the managed server in the ADMIN state, while still connected to node manager kill and start the server again and check the status. Example using Node Manager:

nmKill("upgrade12to15_server_1")
nmStart("upgrade12to15_server_1")
nmServerStatus("upgrade12to15_server_1")

If you still do not see the server in the RUNNING state examine the server logs.

Beginning with the Administration Compute instance run the following on all Compute instances:

Post Upgrade

If the upgrade was successful, complete following tasks if they apply to your instance:

If you use a Hosting Provider to manage DNS, then reset the CNAME records at your Hosting Provider to point to the new IP addresses of the load balancer and WebLogic VMs.
Destroy the source stack instances. See Destroy Stack ResourcesDestroy Stack Resources.

Note:
Do not delete the Identity Cloud Service Resources as these resources are required in the upgraded cloned instance.

Roll Back Upgrade

If the upgrade fails, you can roll back the upgrade. Since the source stack instances are still present you would just switch back to using those instances.

If you used a Reserved IP Address for the Load Balancer you will need to destroy the Load Balancer and create a new Load Balancer with the reserved IP address with backends pointing to the source stack instances.