OpenTofu for Oracle Cloud Native Environment
This repository provides a modular approach to deploying OCNE and its components in OCI using OpenTofu. Each submodule focuses on a specific part of an OCNE deployment, allowing you to select the configuration to deploy.
This module creates the following resources:
- OCNE API Server
- Orchestrates OCNE agents running on control plane and worker nodes to install Kubernetes and other OCNE modules.
- Control plane nodes
- Compute instances for the Kubernetes cluster's control plane nodes.
- Worker nodes
- Compute instances for the Kubernetes cluster's worker nodes.
- Mount target
- A mount target and two file systems using File Storage Service (FSS) for persistent volumes (PVs): one for the domain home and the other for the database volume.
- Bastion server
- A secure gateway for accessing private machines in a network.
High-Level Deployment Options
The module supports the following deployment options:
- Deploying the OCNE API Server on a dedicated compute instance.
- Using a default network for the deployment.
- Allowing the modules to create and configure a new network.
- Using OpenSSL to generate and distribute certificates to each node.
Prerequisites
Before using the OpenTofu scripts, complete the following prerequisites:
- Ensure that you have an existing tenancy with enough compute and networking resources for the desired cluster.
- Ensure that an Identity and Access Management policy is in place within that tenancy to allow the OCI Container Engine for Kubernetes service to manage tenancy resources.
- Ensure that you have a user defined within that tenancy.
- Ensure that you have an API key for use with the OCI API. See Managing User Credentials.
- Ensure that you have an SSH key pair for configuring SSH access to the cluster nodes.
- Download, install, and configure the OCI CLI. See Installing the CLI for installation instructions.
Overview of terraform.tfvars.template file variables
| Name | Description |
|---|---|
tenancy_id |
The OCID of your tenancy. See Where to Get the Tenancy's OCID and User's OCID. |
compartment_id |
The OCID of the compartment. |
user_id |
The OCID of the user that OpenTofu uses to create OCI resources. See Where to Get the Tenancy's OCID and User's OCID. |
fingerprint |
The fingerprint of the key pair. See How to Get the Key's Fingerprint. |
api_private_key_path |
The path to the private key the OCI user uses to authenticate with OCI APIs. See How to Generate an API Signing Key and How to Upload the Public Key. |
region |
The OCI region where resources are created. See Regions and Availability Domains. |
availability_domain_id |
The ID of the availability domain within region
where the deployment is created.
|
prefix |
A unique prefix added to the names of all OCNE resources created as part of the deployment. |
ssh_private_key_path |
The path to the SSH private key paired with the public key used to access compute resources created as part of this deployment. See Generating an SSH Key Pair for Oracle Compute Cloud Service Instances. |
ssh_public_key_path |
The path to the SSH public key used to configure access to compute resources created as part of this deployment. See Generating an SSH Key Pair for Oracle Compute Cloud Service Instances. |
control_plane_node_count |
The number of Kubernetes control plane nodes to deploy. For recommended node counts, see Kubernetes High Availability Requirements. |
worker_node_count |
The number of Kubernetes worker nodes to deploy. For the recommended worker node count, see Kubernetes High Availability Requirements. |
os_version |
The Oracle Linux version used as the base image for all compute resources in this deployment. |
environment_name |
The name of the OCNE environment created by this module to host module instances. See Creating an Environment. |
kubernetes_name |
The name of the OCNE Kubernetes module instance installed as part of this deployment. See Creating a Kubernetes Module. |
ocne_version |
The OCNE version and release to deploy.
See OCNE Release Notes.
Set the value to <major.minor> to install the latest
patch for that version, or to <major.minor.patch>
to install a specific patch version.
|
config_file_path |
The path to the OCNE configuration file. See OCNE Configuration File. |
Deploy the environment
Set up the deployment scripts as described in Set up the Code Repository to Deploy Oracle SOA Suite Domains.
Copy oci.props.template from
$WORKDIR/ocne/samples/opentofu to oci.props
and provide all required values.
Run the script using the following command:
$ $WORKDIR/ocne/samples/opentofu/ocne.create.sh oci.props
The script reads the values from oci.props and performs the following steps:
- Creates a new tfvars file based on the values in
oci.props. - Downloads and installs the required binaries for OpenTofu, yq, and jq.
- Applies the configuration, creates an OCNE environment using OpenTofu, and generates a kubeconfig file.
If the OCNE environment is created successfully, ocne.create.sh
displays output such as the following:
api_private_key_path = "/home/user1/.oci/oci_api_key.pem"
apiserver_ip = "10.0.0.33"
availability_domain_id = "PJzM:PHX-AD-2"
bastion_private_key_path = "/home/user1/.ssh/id_rsa"
bastion_public_ip = "129.xxx.xxx.xx"
bastion_user = "opc"
compartment_id = "ocid1.compartment.oc1..aaaaaaaaq6xxxxxxx....4a"
config_file_path = ""
container_registry = "container-registry.oracle.com/olcne"
control_plane_node_count = 1
control_plane_nodes = tolist([
"10.0.0.155",
])
environment_name = "myenvironment"
extra_cas = tolist([])
fingerprint = "a6:c8:xx:xx:xx:xx..:XX"
fmw1_export_path = "/fmw1"
fmw1_fs_ocid = "ocid1.filesystem.oc1.phx.aaaaaaaaaaj7vxxxxxx...aa"
fmw2_export_path = "/fmw2"
fmw2_fs_ocid = "ocid1.filesystem.oc1.phx.aaaaaaaaaaxxxxxx...aaaa"
freeform_tags = tomap({})
image_ocid = "ocid1.image.oc1.phx.aaaaaaaahgrsxxxxx...cq"
instance_shape = tomap({
"memory" = "32"
"ocpus" = "2"
"shape" = "VM.Standard.E4.Flex"
})
key_ocid = ""
kube_apiserver_endpoint_ip = "10.0.0.207"
kube_apiserver_port = "6443"
kube_apiserver_virtual_ip = ""
kubernetes_name = "mycluster"
load_balancer_ip = "10.0.0.207"
load_balancer_policy = "LEAST_CONNECTIONS"
load_balancer_shape = tomap({
"flex_max" = "50"
"flex_min" = "10"
"shape" = "flexible"
})
mount_target_ip = "10.0.0.128"
ocne_secret_name = "vk1-ocne_keys"
ocne_vault_client_token = ""
ocne_version = "1.9"
os_version = "8"
prefix = "vk1"
provision_mode = "OCNE"
provision_modes_map = {
"provision_mode_infrastucture" = "Infrastructure"
"provision_mode_ocne" = "OCNE"
}
proxy = ""
region = "us-phoenix-1"
secret_name = "vk1-vault_keys"
ssh_private_key_path = "/home/user1/.ssh/id_rsa"
standalone_api_server = true
subnet_id = "ocid1.subnet.oc1.phx.aaaaaaaas77xxxxxxx...wq"
tenancy_id = "ocid1.tenancy.oc1..aaaaaaaxxxxx......bmffq"
use_vault = false
user_id = "ocid1.user.oc1..aaaaaaxxxxxxx.....sya"
vault_ha_storage_bucket = ""
vault_instances = []
vault_namespace = ""
vault_ocid = ""
vault_pool_size = 1
vault_storage_bucket = ""
vault_uri = ""
vault_version = "1.3.4"
vcn_id = "ocid1.vcn.oc1.phx.amaaaaaxxxxxx....kq"
worker_node_count = 1
worker_nodes = [
"10.0.0.76",
]
yum_repo_url = "http://yum.oracle.com/repo/OracleLinux/OL8/olcne16/x86_64"
kubeconfig file successfully created in /home/user1/ocne_env/opentofu.
To delete the OCNE environment, run ocne.delete.sh.
The script reads oci.props from the current directory
and deletes the cluster.
$ $WORKDIR/ocne/samples/opentofu/ocne.delete.sh oci.props
Install the Oracle Cloud Infrastructure Cloud Controller Manager module (OCI-CCM) in an OCNE environment configured using OpenTofu
The Oracle Cloud Infrastructure Cloud Controller Manager module provisions Oracle Cloud Infrastructure storage. See Installing the Oracle Cloud Infrastructure Cloud Controller Manager Module for detailed installation instructions.
After creating the OCNE environment using OpenTofu, obtain the VCN and subnet IDs
from the console output. Alternatively, run tofu and obtain
the IDs from its output. Sign in to the platform API server node (OCNE Operator node),
then perform the following steps:
-
Create an Oracle Cloud Infrastructure Cloud Controller Manager module and associate it with the Kubernetes module named
myclusterusing--oci-ccm-kubernetes-module. In this example, the Cloud Controller Manager module is namedmyoci:# The path to the node certificate(IP refers to the operator/platform API server node). export OLCNE_SM_CA_PATH=/home/opc/.olcne/certificates/10.0.0.137\:8091/ca.cert # The path to the Certificate Authority certificate. export OLCNE_SM_CERT_PATH=/home/opc/.olcne/certificates/10.0.0.137\:8091/node.cert # The path to the key for the node's certificate. export OLCNE_SM_KEY_PATH=/home/opc/.olcne/certificates/10.0.0.137\:8091/node.key olcnectl module create \ --environment-name myenvironment \ --module oci-ccm \ --name myoci \ --oci-ccm-kubernetes-module mycluster \ --oci-region us-ashburn-1 \ --oci-tenancy ocid1.tenancy.oc1..unique_ID \ --oci-compartment ocid1.compartment.oc1..unique_ID \ --oci-user ocid1.user.oc1..unique_ID \ --oci-fingerprint b5:52:... \ --oci-private-key-file /home/opc/.oci/oci_api_key.pem \ --oci-vcn ocid1.vcn.oc1..unique_ID \ --oci-lb-subnet1 ocid1.subnet.oc1..unique_ID -
Install the module using
olcnectl module install:olcnectl module install \ --environment-name myenvironment \ --name myoci -
Verify that the module is deployed using
olcnectl module instances. For example:olcnectl module instances \ --environment-name myenvironment INSTANCE MODULE STATE 10.0.0.24:8090 node installed 10.0.0.199:8090 node installed mycluster kubernetes installed myoci oci-ccm installed [opc@vk-api-server-001 ~]$ kubectl get nodes --kubeconfig=kubeconfig.myenvironment.mycluster NAME STATUS ROLES AGE VERSION vk-control-plane-001 Ready control-plane 17m v1.29.3+3.el8 vk-worker-001 Ready <none> 17m v1.29.3+3.el8
Using Object Storage for State Files
To use Object Storage for state files, create an AWS S3 Compatible API key in OCI. You can create the key using either the OCI Console or the CLI. See Using Object Storage for State Files.
To get started, rename state_backend.tf.example to
state_backend.tf and provide values for the appropriate variables.
For S3 backend variable definitions, see OpenTofu S3 Backend Documentation.
Deploy SOA on an OCNE cluster using Helmfile
See Oracle SOA Suite Helm Charts README for detailed instructions.
OCI NFS volume static provisioning
For SOA, reference the file system directly in the domain.storage
section of values.yaml.
For example:
storage:
capacity: 10Gi
reclaimPolicy: Retain
type: nfs
path: /fmw1 #Export Path
nfs:
server: 10.0.10.156 #Mount Target IP Address
For the database, the following example defines a persistent volume (PV) using a file system (FS) and a mount target IP address:
apiVersion: v1
kind: PersistentVolume
metadata:
name: fmw2-pv-db
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
csi:
driver: fss.csi.oraclecloud.com
volumeHandle: "<OCID of the file system>:<Mount Target IP Address>:/<Export Path>"
Update the oracledb.persistence section of
values.yaml to use the created PV as a volume. For example:
persistence:
storageClass: ""
size: 10Gi
accessMode: "ReadWriteOnce"
volumeName: "fmw2-pv-db"
Note:
- An example
volumeHandlefor the preceding configuration:volumeHandle: "ocid1.filesystem.oc1.phx.aaaaanoxxxxx....aaaa:10.0.10.156:/fmw2" - Obtain the
volumeHandledetails from the console output after the cluster is created successfully. - When a mount target is provisioned in OCI, its Reported Size (GiB) values are very large. These values appear on the mount target page in the OCI Console. Applications can fail to install if a space requirements check reports too much available disk space. In the OCI Console, click the pencil icon beside the mount target's Reported Size parameter to specify, in gigabytes (GiB), the maximum capacity reported by file systems exported through this mount target. This setting does not limit the actual amount of data you can store. See Changing a File System's Reported Size.