Creating OSM Cloud Native Images

3 Creating OSM Cloud Native Images

OSM cloud native requires container images be made available to create and manage OSM cloud native instances. This chapter describes how to create those OSM cloud native images.

OSM cloud native requires two container images. The OSM DB Installer image is used to manage the OSM and Fusion MiddleWare schemas - create, delete, upgrade - as well as deploy and fast-undeploy OSM cartridges in the OSM schema. The other image is the OSM image itself. This image is the basis for all of the long running pods - the WebLogic admin server and all the Managed Servers that comprise an OSM cloud native instance. Each image is built on top of a Linux base image and adds Java, Fusion MiddleWare components and OSM product components on top.

OSM Cloud native images are created using the OSM cloud native builder toolkit and a dependency manifest file. The OSM cloud native Image Builder is intended to be run as part of a Continuous Integration process that generates images. It needs to run on Linux and have access to the local Docker daemon. The versions of these are as per the OSM statement of certification in the OSM documentation. The dependency manifest is a file that describes all the versions and patches required to build out the image.

See the following topics for further details:

Downloading the OSM Cloud Native Image Builder

You download the OSM cloud native image builder from My Oracle Support at: https://support.oracle.com

The OSM cloud native image builder is bundled with the following components:

An unpatched dependency manifest file.

This file does not include any artifacts that require contract-driven access to Oracle download sites (for example, for Fusion MiddleWare patches). Use this unpatched manifest file for evaluation purposes only.

For production use (and throughout the adoption lifecycle leading up to production), obtain the latest dependency manifest file. See OSM Compatibility Matrix for details about the latest recommended manifest file for your OSM release.
OSM cloud native builder kit. The kit contains:
- The OSM Domain WDT Model.
- The OSM DB Installer scripts and manifest files.
- The WDT Deployment tool and the WebLogic Image tool.
Staging directory structure.

Prerequisites for Creating OSM Images

The pre-requisites for building OSM cloud native images are:

Docker client and daemon on the build machine.
Installers for WebLogic Server and JDK. Download these from the Oracle Software Delivery Cloud:

https://edelivery.oracle.com
Oracle Instant Client. Download this from Oracle Software Downloads:

https://www.oracle.com/downloads/
Required patches. Download these from My Oracle Support:

https://support.oracle.com/
Java, installed with JAVA_HOME set in the environment.
Bash, to enable the `<tab>` command complete feature.

See OSM Compatibility Matrix for details about the required and supported versions of these pre-requisite software.

Configuring the OSM Cloud Native Images

The dependency manifest file describes the input that goes into the OSM images. It is consumed by the image build process. The default configuration in the latest manifest file (from the Knowledge Article 2170105.1 on My Oracle Support) provides all the necessary components and required patches for creating the OSM cloud native images easily.

You can also modify the manifest file to extend it to meet your requirements. This enables you to:

Specify any Linux image as the base, as long as its binary is compatible with Oracle Linux.
Upgrade the Oracle Enterprise Linux version to a newer version to uptake a quarterly CPU.
Upgrade the JDK version to a newer JDK version to uptake a quarterly CPU.
Upgrade the Fusion Middleware version to a newer version. For example, you upgrade the Fusion Middleware version to a newer version when you initiate the upgrade to pick up new PSU or when Oracle recommends a new update.
Change the set of patches applied on WebLogic Server, Coherence, Fusion Middleware, and OPatch to stay aligned with evolving OSM recommendations.
Change the OSM artifacts to newer artifacts to uptake a new OSM patch.
Choose a different userid and groupid for oracle:oracle user:group that the image specifies. The default is 1000:1000.

The breakdown of each section in the dependency manifest file is as follows:

Note:

The schemaVersion and date parameters are maintained by Oracle. Do not modify these parameters.

Version numbers provided here are only examples. The manifest file used specifies the actual versions currently recommended.

OSM Cloud Native Infrastructure Image

While not required by OSM cloud native to create or manage OSM instances, this infrastructure image is a necessary building block of the final OSM container image.

linux:
  vendor: Oracle
  version: 7.x
  image: oraclelinux:7-slim

The Linux parameter specifies the base Linux image to be used as the base docker image. The version is the two-digit version from /etc/redhat-release.

The vendor and version details are specified and used for:

Validation when an image is built.
Querying at run-time. To troubleshoot issues, Oracle support requires you to provide these details in the manifest file used to build the image.

userGroup:
  username: oracle
  userid: 1000
  groupname: oracle
  groupid: 1000

The userGroup parameter specifies the default userId and groupId for oracle

jdk:
  vendor: Oracle
  version: 8u251
  path: $CN_BUILDER_STAGING/java/jdk-8u251-linux-x64.tar.gz

The jdk parameter specifies the JDK vendor, version, and the staging path.

fmw:
    version: 12.2.1.4.0
    path: $CN_BUILDER_STAGING/fmw/install/fmw_12.2.1.4.0_infrastructure_Disk1_1of1.zip

The fmw parameter specifies the Fusion Middleware version and staging path.

oPatch:
  description: Weblogic Opatch
  patchId: 28186730_13.9.4.2.2
  path: $CN_BUILDER_STAGING/fmw/patch/p28186730_139422_Generic.zip

The oPatch parameter specifies the Oracle Patch tool and staging path.

fmwPatch:
  - description: p28334768,p27184424(on top of 12.2.1.4.0)
    patchId: 28334768_12.2.1.4.0
    path: $CN_BUILDER_STAGING/fmw/patch/p28334768_122140_Generic.zip
  - description: p28334768,p27184424(on top of 12.2.1.4.0)
    patchId: 27184424_12.2.1.4.0
    path: $CN_BUILDER_STAGING/fmw/patch/p27184424_122140_Generic.zip
  - description: SAF to dynamic cluster
    patchId: 30656708_12.2.1.4.0
    path: $CN_BUILDER_STAGING/fmw/patch/p30656708_122140_Generic.zip
  - description: user-group association bug fix
    patchId: 30319071_12.2.1.4.0
    path: $CN_BUILDER_STAGING/fmw/patch/p30319071_122140_Generic.zip
  - description: PSU Coherence
    patchId: 31030896_12.2.1.4.0
    path: $CN_BUILDER_STAGING/fmw/patch/p31030896_122140_Generic.zip
  - description: PSU WLS
    patchId: 31101341_12.2.1.4.0
    path: $CN_BUILDER_STAGING/fmw/patch/p31101341_122140_Generic.zip
  - description: PSU WLS
    patchId: 30970477_12.2.1.4.0
    path: $CN_BUILDER_STAGING/fmw/patch/p30970477_122140_Generic.zip
  - description: PSU WLS
    patchId: 30761841_12.2.1.4.0
    path: $CN_BUILDER_STAGING/fmw/patch/p30761841_122140_Generic.zip
  - description: Opatch
    patchId: 31101362_12.2.1.4.0
    path: $CN_BUILDER_STAGING/fmw/patch/p31101362_1394002_Generic.zip

The fmwPatch parameter specifies additional patches and their staging paths.

OSM Cloud Native Image

Note:
Do not modify these parameters. These parameters are maintained by Oracle.
```
osmCnImage:
  name: osm-cn-base
  tag: 7.4.1
  wdt:
    version: 1.8.0
    path: $CN_BUILDER_STAGING/cnsdk/tools/weblogic-deploy.zip
  modelfiles: $CN_BUILDER_STAGING/cnsdk/osm_model/osm-domain-config/osm-base-domain.yaml,$CN_BUILDER_STAGING/cnsdk/osm_model/osm-domain-config/properties/docker-build/domain.properties
  application: $CN_BUILDER_STAGING/cnsdk/osm_model/osm-full-wdt-app-archive.zip
  dockerExtension: $CN_BUILDER_STAGING/cnsdk/osm_model/additionalBuildCommands.txt
```
The osmCnImage section specifies details about the OSM artifacts required to build the OSM base image. These include the oms.ear, cartridge management WS ear file, default cartridge par file, job control cartridge par file, WDT and base model files.
OSM Cloud Native DB Installer Image
```
osmCnDbInstallerImage:
  name: osm-cn-db-installer
  tag: 7.4.1
```
The osmCnDbInstallerImage parameter specifies the DB Installer image name and version. This includes the transformed OSM DB model and Semele jar file.
```
oracleInstantClient:
   version: 19.8.0.0.0
   basic:
      path: $CN_BUILDER_STAGING/instant-client/oracle-instantclient-basic-19.8.0.0.0-1.x86_64.rpm
    sqlplus:
      path: $CN_BUILDER_STAGING/instant-client/oracle-instantclient-sqlplus-19.8.0.0.0-1.x86_64.rpm
    tools:
      path: $CN_BUILDER_STAGING/instant-client/oracle-instantclient-tools-19.8.0.0.0-1.x86_64.rpm
```
The oracleInstantClient parameter specifies details about the Oracle Instant Client required by the DB installer.

Creating OSM Cloud Native Images

To create the OSM image, the image builder does the following:

Starts with a base-level operating system image (for example, oraclelinux:7-slim).
Creates user and group (for example, oracle:oracle).
Updates the image with the necessary packages for installing Fusion Middleware.
Installs Java, Fusion Middleware and applies patches.
Installs the OSM application base on the WDT model.

To create the OSM DB Installer image, the image builder does the following:

Starts with a base-level operating system image (for example, oraclelinux:7-slim).
Creates a user and a group (for example, oracle:oracle)
Updates the image with the necessary packages for installing Fusion Middleware.
Installs Java, Fusion Middleware and applies the required patches.
Installs SQL Plus and SQL Loader and the supporting libraries.
Installs the OSM DB Installer.

You can specify any Linux image as the base, as long as its binary is compatible with Oracle Linux and conforms to the compatibility matrix. See OSM Compatibility Matrix for details about the supported software.

The following packages must be installed onto the given base image, or be already present:

gzip
tar
unzip

Creating the OSM and OSM DB Installer Images

To create the OSM and OSM DB Installer images:

Create the workspace directory:
```
mkdir workspace
```
Obtain and untar the OSM image builder file: osm-image-builder.tar.gz to the workspace directory:
```
tar -xf ./osm-image-builder.tar.gz --directory workspace
```

(Optional) Download and copy a newer version of Oracle Instant Client to the workspace/osm-image-builder/staging/instant-client directory and update the version and the file names.

Note:

Oracle Instant Client packages are included in the OSM Image Builder and can be used as-is without additional downloads.

Note:

Follow your organization standard for $http_proxy.

curl -x $http_proxy --output osm-image-builder/staging/instant-client/oracle-instantclient-basic-19.8.0.0.0-1.x86_64.rpm https://download.oracle.com/otn_software/linux/instantclient/19800/oracle-instantclient19.8-basic-19.8.0.0.0-1.x86_64.rpm
curl -x $http_proxy --output osm-image-builder/staging/instant-client/oracle-instantclient-sqlplus-19.8.0.0.0-1.x86_64.rpm https://download.oracle.com/otn_software/linux/instantclient/19800/oracle-instantclient19.8-sqlplus-19.8.0.0.0-1.x86_64.rpm
curl -x $http_proxy --output osm-image-builder/staging/instant-client/oracle-instantclient-tools-19.8.0.0.0-1.x86_64.rpm https://download.oracle.com/otn_software/linux/instantclient/19800/oracle-instantclient19.8-tools-19.8.0.0.0-1.x86_64.rpm

Download JDK to the workspace/osm-image-builder/staging/java directory. The JDK version to be downloaded is described in the dependency manifest file.
```
cp jdk-8u251-linux-x64.tar.gz ./workspace/osm-image-builder/staging/java/jdk-8u251-linux-x64.tar.gz
  
```
From Oracle Software Delivery Cloud: https://edelivery.oracle.com, download Fusion Middleware Infrastructure installer and copy it to the workspace/osm-image-builder/staging/fmw/install directory. The Fusion Middleware Infrastructure installer version to be download is described in the dependency manifest file under the fmw section.
```
cp fmw_12.2.1.4.0_infrastructure_Disk1_1of1.zip ./workspace/osm-image-builder/staging/fmw/install/fmw_12.2.1.4.0_infrastructure_Disk1_1of1.zip
```
Download all the listed patches to the workspace/osm-image-builder/staging/fmw/patch directory. The list of required patches is in the dependency manifest file in the oPatch and fmwPatch sections.

Note:
This step is not required if osm_cn_ci_manifest_unpatched.yaml is the manifest used.
You can download the patches using any of the following options:
- (Recommended) Manually search for and download each OPatch/FMW patches from Oracle Support to the current working directory and then copy to the staging directory.
```
cp pxxxxxx_xxxxx_Generic.zip ./workspace/osm-image-builder/staging/fmw/patch
```
- Provide your My Oracle Support account credentials when invoking the build-osm-images.sh script, and let the builder download the patches automatically:
  
  Note:
  Some patches may not be retrievable in this manner. If the image build process fails with errors about a missing patch, use the recommended option.
```
./workspace/osm-image-builder/bin/build-osm-images.sh -f $DMANIFEST -s $STAGING -c osm -u MOS_username -p MOS_password
```

Run build-osm-images.sh and pass the dependency manifest file, staging path, and the type of image to be created.

export DMANIFEST=./workspace/osm-image-builder/bin/osm_cn_ci_manifest_unpatched.yaml
export STAGING=$(pwd)/workspace/osm-image-builder/staging

To create OSM image, use "-c osm" as shown:

./workspace/osm-image-builder/bin/build-osm-images.sh -f $DMANIFEST -s $STAGING -c osm

To create OSM DB installer image, use "-c dbinstaller" as shown:

./workspace/osm-image-builder/bin/build-osm-images.sh -f $DMANIFEST -s $STAGING -c dbinstaller

These steps can be included into your CI pipeline as long as the required components are already downloaded to the staging area.

Additional Considerations When Using the Unpatched Manifest File

When an OSM image is created by the image builder with the osm_cn_ci_manifest_unpatched.yaml file, the resulting image does not contain the Fusion Middleware patches that are required for proper OSM cloud native functioning. It is intended to be used only for evaluation purposes. One workaround is to manually establish the association between OSM users and groups.

OSM users and groups are not associated after the start up of the admin server, which results in OSM EJB failing to deploy to the managed server. You should manually associate users and the group before starting up the managed server.

To associate OSM users with a group when using the unpatched manifest file:

Create a new instance with only the admin server running. In the instance specification, change the value for clusterSize manually. This change would ultimately be performed by an automated CI/CD pipeline.
```
vi $SPEC_PATH/project-instance.yaml

# Change the cluster size to 0 
clusterSize: 0
```
Create the OSM instance.
Run the config-security.sh script passing the domain namespace and domain UID.
```
$OSM_CNTK/scripts/config-security.sh project project-instance
```
Start the managed servers.
- In the instance specification, set clusterSize to the desired number of managed servers.
```
vi $SPEC_PATH/project-instance.yaml 
# Change the cluster size to the desired number 
clusterSize: 8
```
- Upgrade the OSM instance.

The associations are reset every time the Admin Server pod terminates or restarts. This can happen when the instance is deleted, or on an unexpected event (such as an hardware issue), or as a side-effect of an instance upgrade that involves a rolling restart. Regardless of the scenario that led to Admin Server pod being recreated, the associations must be set up afresh.

To recreate the user and group association:

Stop all the managed servers by setting the cluster size to 0 in the instance specification and upgrade the instance.
Run the config-security.sh script as described in step 2 in the above procedure.
Start the managed servers as described in step 3 in the above procedure.

Post-build Image Management

The OSM cloud native image builder creates images with names and tags based on the settings in the manifest file. By default, this results in the following images:

osm-cn-base:7.4.1.0.0
osm-cn-db-installer:7.4.1.0.0

Once images are built in a CI pipeline, the pipeline uniquely tags the images and pushes them to an internal Docker repository. An uptake process can then be triggered for the new images:

Sanity Test
Development Test (for explicit retesting of scenarios that triggered the rebuild, if any)
System Test
Integration Test
Pre-Production Test
Production