OpenTofu for Oracle Cloud Native Environment

This repository provides a modular approach to deploying OCNE and its components in OCI using OpenTofu. Each submodule focuses on a specific part of an OCNE deployment, allowing you to select the configuration to deploy.

This module creates the following resources:

OCNE API Server
Orchestrates OCNE agents running on control plane and worker nodes to install Kubernetes and other OCNE modules.
Control plane nodes
Compute instances for the Kubernetes cluster's control plane nodes.
Worker nodes
Compute instances for the Kubernetes cluster's worker nodes.
Mount target
A mount target and two file systems using File Storage Service (FSS) for persistent volumes (PVs): one for the domain home and the other for the database volume.
Bastion server
A secure gateway for accessing private machines in a network.

High-Level Deployment Options

The module supports the following deployment options:

  • Deploying the OCNE API Server on a dedicated compute instance.
  • Using a default network for the deployment.
  • Allowing the modules to create and configure a new network.
  • Using OpenSSL to generate and distribute certificates to each node.

Prerequisites

Before using the OpenTofu scripts, complete the following prerequisites:

  • Ensure that you have an existing tenancy with enough compute and networking resources for the desired cluster.
  • Ensure that an Identity and Access Management policy is in place within that tenancy to allow the OCI Container Engine for Kubernetes service to manage tenancy resources.
  • Ensure that you have a user defined within that tenancy.
  • Ensure that you have an API key for use with the OCI API. See Managing User Credentials.
  • Ensure that you have an SSH key pair for configuring SSH access to the cluster nodes.
  • Download, install, and configure the OCI CLI. See Installing the CLI for installation instructions.

Overview of terraform.tfvars.template file variables

Name Description
tenancy_id The OCID of your tenancy. See Where to Get the Tenancy's OCID and User's OCID.
compartment_id The OCID of the compartment.
user_id The OCID of the user that OpenTofu uses to create OCI resources. See Where to Get the Tenancy's OCID and User's OCID.
fingerprint The fingerprint of the key pair. See How to Get the Key's Fingerprint.
api_private_key_path The path to the private key the OCI user uses to authenticate with OCI APIs. See How to Generate an API Signing Key and How to Upload the Public Key.
region The OCI region where resources are created. See Regions and Availability Domains.
availability_domain_id The ID of the availability domain within region where the deployment is created.
prefix A unique prefix added to the names of all OCNE resources created as part of the deployment.
ssh_private_key_path The path to the SSH private key paired with the public key used to access compute resources created as part of this deployment. See Generating an SSH Key Pair for Oracle Compute Cloud Service Instances.
ssh_public_key_path The path to the SSH public key used to configure access to compute resources created as part of this deployment. See Generating an SSH Key Pair for Oracle Compute Cloud Service Instances.
control_plane_node_count The number of Kubernetes control plane nodes to deploy. For recommended node counts, see Kubernetes High Availability Requirements.
worker_node_count The number of Kubernetes worker nodes to deploy. For the recommended worker node count, see Kubernetes High Availability Requirements.
os_version The Oracle Linux version used as the base image for all compute resources in this deployment.
environment_name The name of the OCNE environment created by this module to host module instances. See Creating an Environment.
kubernetes_name The name of the OCNE Kubernetes module instance installed as part of this deployment. See Creating a Kubernetes Module.
ocne_version The OCNE version and release to deploy. See OCNE Release Notes. Set the value to <major.minor> to install the latest patch for that version, or to <major.minor.patch> to install a specific patch version.
config_file_path The path to the OCNE configuration file. See OCNE Configuration File.

Deploy the environment

Set up the deployment scripts as described in Set up the Code Repository to Deploy Oracle SOA Suite Domains.

Copy oci.props.template from $WORKDIR/ocne/samples/opentofu to oci.props and provide all required values.

Run the script using the following command:

$ $WORKDIR/ocne/samples/opentofu/ocne.create.sh oci.props

The script reads the values from oci.props and performs the following steps:

  1. Creates a new tfvars file based on the values in oci.props.
  2. Downloads and installs the required binaries for OpenTofu, yq, and jq.
  3. Applies the configuration, creates an OCNE environment using OpenTofu, and generates a kubeconfig file.

If the OCNE environment is created successfully, ocne.create.sh displays output such as the following:

api_private_key_path = "/home/user1/.oci/oci_api_key.pem"
apiserver_ip = "10.0.0.33"
availability_domain_id = "PJzM:PHX-AD-2"
bastion_private_key_path = "/home/user1/.ssh/id_rsa"
bastion_public_ip = "129.xxx.xxx.xx"
bastion_user = "opc"
compartment_id = "ocid1.compartment.oc1..aaaaaaaaq6xxxxxxx....4a"
config_file_path = ""
container_registry = "container-registry.oracle.com/olcne"
control_plane_node_count = 1
control_plane_nodes = tolist([
  "10.0.0.155",
])
environment_name = "myenvironment"
extra_cas = tolist([])
fingerprint = "a6:c8:xx:xx:xx:xx..:XX"
fmw1_export_path = "/fmw1"
fmw1_fs_ocid = "ocid1.filesystem.oc1.phx.aaaaaaaaaaj7vxxxxxx...aa"
fmw2_export_path = "/fmw2"
fmw2_fs_ocid = "ocid1.filesystem.oc1.phx.aaaaaaaaaaxxxxxx...aaaa"
freeform_tags = tomap({})
image_ocid = "ocid1.image.oc1.phx.aaaaaaaahgrsxxxxx...cq"
instance_shape = tomap({
  "memory" = "32"
  "ocpus" = "2"
  "shape" = "VM.Standard.E4.Flex"
})
key_ocid = ""
kube_apiserver_endpoint_ip = "10.0.0.207"
kube_apiserver_port = "6443"
kube_apiserver_virtual_ip = ""
kubernetes_name = "mycluster"
load_balancer_ip = "10.0.0.207"
load_balancer_policy = "LEAST_CONNECTIONS"
load_balancer_shape = tomap({
  "flex_max" = "50"
  "flex_min" = "10"
  "shape" = "flexible"
})
mount_target_ip = "10.0.0.128"
ocne_secret_name = "vk1-ocne_keys"
ocne_vault_client_token = ""
ocne_version = "1.9"
os_version = "8"
prefix = "vk1"
provision_mode = "OCNE"
provision_modes_map = {
  "provision_mode_infrastucture" = "Infrastructure"
  "provision_mode_ocne" = "OCNE"
}
proxy = ""
region = "us-phoenix-1"
secret_name = "vk1-vault_keys"
ssh_private_key_path = "/home/user1/.ssh/id_rsa"
standalone_api_server = true
subnet_id = "ocid1.subnet.oc1.phx.aaaaaaaas77xxxxxxx...wq"
tenancy_id = "ocid1.tenancy.oc1..aaaaaaaxxxxx......bmffq"
use_vault = false
user_id = "ocid1.user.oc1..aaaaaaxxxxxxx.....sya"
vault_ha_storage_bucket = ""
vault_instances = []
vault_namespace = ""
vault_ocid = ""
vault_pool_size = 1
vault_storage_bucket = ""
vault_uri = ""
vault_version = "1.3.4"
vcn_id = "ocid1.vcn.oc1.phx.amaaaaaxxxxxx....kq"
worker_node_count = 1
worker_nodes = [
  "10.0.0.76",
]
yum_repo_url = "http://yum.oracle.com/repo/OracleLinux/OL8/olcne16/x86_64"
kubeconfig file successfully created in /home/user1/ocne_env/opentofu.

To delete the OCNE environment, run ocne.delete.sh. The script reads oci.props from the current directory and deletes the cluster.

$ $WORKDIR/ocne/samples/opentofu/ocne.delete.sh oci.props

Install the Oracle Cloud Infrastructure Cloud Controller Manager module (OCI-CCM) in an OCNE environment configured using OpenTofu

The Oracle Cloud Infrastructure Cloud Controller Manager module provisions Oracle Cloud Infrastructure storage. See Installing the Oracle Cloud Infrastructure Cloud Controller Manager Module for detailed installation instructions.

After creating the OCNE environment using OpenTofu, obtain the VCN and subnet IDs from the console output. Alternatively, run tofu and obtain the IDs from its output. Sign in to the platform API server node (OCNE Operator node), then perform the following steps:

  1. Create an Oracle Cloud Infrastructure Cloud Controller Manager module and associate it with the Kubernetes module named mycluster using --oci-ccm-kubernetes-module. In this example, the Cloud Controller Manager module is named myoci:

    # The path to the node certificate(IP refers to the operator/platform API server node).
    export OLCNE_SM_CA_PATH=/home/opc/.olcne/certificates/10.0.0.137\:8091/ca.cert
    
    # The path to the Certificate Authority certificate. 
    export OLCNE_SM_CERT_PATH=/home/opc/.olcne/certificates/10.0.0.137\:8091/node.cert
    
    # The path to the key for the node's certificate. 
    export OLCNE_SM_KEY_PATH=/home/opc/.olcne/certificates/10.0.0.137\:8091/node.key
    
    olcnectl module create \
    --environment-name myenvironment \
    --module oci-ccm \
    --name myoci \
    --oci-ccm-kubernetes-module mycluster \
    --oci-region us-ashburn-1 \
    --oci-tenancy ocid1.tenancy.oc1..unique_ID \
    --oci-compartment ocid1.compartment.oc1..unique_ID \
    --oci-user ocid1.user.oc1..unique_ID \
    --oci-fingerprint b5:52:... \
    --oci-private-key-file /home/opc/.oci/oci_api_key.pem \
    --oci-vcn ocid1.vcn.oc1..unique_ID \
    --oci-lb-subnet1 ocid1.subnet.oc1..unique_ID 
    
  2. Install the module using olcnectl module install:

    olcnectl module install \
    --environment-name myenvironment \
    --name myoci
    
  3. Verify that the module is deployed using olcnectl module instances. For example:

    olcnectl module instances \
    --environment-name myenvironment
    
    INSTANCE        MODULE          STATE
    10.0.0.24:8090  node            installed
    10.0.0.199:8090 node            installed
    mycluster       kubernetes      installed
    myoci           oci-ccm         installed
    
    
    [opc@vk-api-server-001 ~]$ kubectl get nodes --kubeconfig=kubeconfig.myenvironment.mycluster
    NAME                   STATUS   ROLES           AGE   VERSION
    vk-control-plane-001   Ready    control-plane   17m   v1.29.3+3.el8
    vk-worker-001          Ready    <none>          17m   v1.29.3+3.el8
    

Using Object Storage for State Files

To use Object Storage for state files, create an AWS S3 Compatible API key in OCI. You can create the key using either the OCI Console or the CLI. See Using Object Storage for State Files.

To get started, rename state_backend.tf.example to state_backend.tf and provide values for the appropriate variables. For S3 backend variable definitions, see OpenTofu S3 Backend Documentation.

Deploy SOA on an OCNE cluster using Helmfile

See Oracle SOA Suite Helm Charts README for detailed instructions.

OCI NFS volume static provisioning

For SOA, reference the file system directly in the domain.storage section of values.yaml. For example:

   storage:
     capacity: 10Gi
     reclaimPolicy: Retain
     type: nfs 
     path: /fmw1   #Export Path
     nfs:
      server: 10.0.10.156 #Mount Target IP Address

For the database, the following example defines a persistent volume (PV) using a file system (FS) and a mount target IP address:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: fmw2-pv-db
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  csi:
    driver: fss.csi.oraclecloud.com
    volumeHandle: "<OCID of the file system>:<Mount Target IP Address>:/<Export Path>"

Update the oracledb.persistence section of values.yaml to use the created PV as a volume. For example:

   persistence:
     storageClass: ""
     size: 10Gi
     accessMode: "ReadWriteOnce"
     volumeName: "fmw2-pv-db"

Note:

  • An example volumeHandle for the preceding configuration:
    volumeHandle:
              "ocid1.filesystem.oc1.phx.aaaaanoxxxxx....aaaa:10.0.10.156:/fmw2"
  • Obtain the volumeHandle details from the console output after the cluster is created successfully.
  • When a mount target is provisioned in OCI, its Reported Size (GiB) values are very large. These values appear on the mount target page in the OCI Console. Applications can fail to install if a space requirements check reports too much available disk space. In the OCI Console, click the pencil icon beside the mount target's Reported Size parameter to specify, in gigabytes (GiB), the maximum capacity reported by file systems exported through this mount target. This setting does not limit the actual amount of data you can store. See Changing a File System's Reported Size.