Note:

Build a Container Engine for Kubernetes (OKE) cluster with OCI provider for Pulumi

Introduction

Oracle Cloud Infrastructure (OCI) Container Engine for Kubernetes (OKE) is a fully managed, scalable, and highly available service that you can use to deploy your containerized applications to the cloud. You specify the compute resources that your applications require, and OKE provisions them on OCI in an existing tenancy. OKE uses Kubernetes to automate the deployment, scaling, and management of containerized applications across clusters of hosts.

Pulumi is a modern infrastructure as code platform that allows you to use familiar programming languages and tools to build, deploy, and manage cloud infrastructure. The Oracle Cloud Infrastructure (OCI) provider for Pulumi can be used to provision any of the resources available in OCI. The OCI provider must be configured with credentials to deploy and update resources in OCI.

This tutorial provides an overview about how to manage infrastructure as code using the OCI provider for Pulumi. You’ll learn how to provision an OKE cluster using Python as the development language.

Objective

Prerequisites

Task 1: Create a Pulumi Stack

Every Pulumi program is deployed to a stack. A stack is an isolated, independently configurable instance of a Pulumi program. Stacks are commonly used to denote different phases of development (such as development, staging, and production) or feature branches (such as feature-x-dev).

Create a home directory

Create a home directory for your stack where we will store a state file and your Stack source code.

$> mkdir pulumi-oke-py && cd pulumi-oke-py   (on Linux or MacOS )

Create a State Repository

Pulumi stores the state of your Stack in Pulumi Service or in a local state file to track changes on your infrastructure. In this tutorial, we will store it in a local state file.

Option 1. Create a local state file directory

This option will create a local directory to store states. For this tutorial, Option 1 will be configured.

$> mkdir oci-stack-statefile (on Linux or MacOS )
$> pulumi login file://oci-stack-statefile

Option 2. Store state in Pulumi free service

Pulumi offers a service to store the Stack state. There are multiple layers from free to organizational.

$> pulumi login

Note: For this tutorial we will only be saving the state file locally as described in Option 1. Option 2 is only for additional information.

Create a new Python Stack

  1. Confirm which service and user you are using by running the whoami command.

    $> pulumi whoami -v

User: <user>
Organizations:
Backend URL: file://oci-stack-statefile

  2. When creating a Stack, Pulumi needs to understand what type of template to use for your project.

    Note:

    • As we are going to code in Python, you must select a Python template by passing the name of the template “python” after keywords “pulumi new”.
    • Pulumi CLI will prompt you for a name of your project (defaults to the name of the parent folder) and a specific environment (defaults to dev) to help you categorize and classify your Stacks.
    $>pulumi new python --force

This command will walk you through creating a new Pulumi project.

Enter a value or leave blank to accept the (default), and press <ENTER>.
Press ^C at any time to quit.

project name: (pulumi-oke-py)
project description: (A minimal Python Pulumi program)
Created project 'pulumi-oke-py'

stack name: (dev)
Created stack 'dev'
Enter your passphrase to protect config/secrets:
Re-enter your passphrase to confirm:

Failed to resolve python version command: fork/exec $PATH/pulumi-oke-py/venv/bin/python: no such file or directory

Installing dependencies...

Creating virtual environment...
Finished creating virtual environment
Updating pip, setuptools, and wheel in virtual environment...
Collecting pip
Using cached https://files.pythonhosted.org/packages/96/2f/caec18213f6a67852f6997fb0673ae08d2e93d1b81573edb93ba4ef06970/pip-22.1.2-py3-none-any.whl
Collecting setuptools
Using cached https://files.pythonhosted.org/packages/e9/1c/ec080fde54ab30a738c92f794eab7f5d2f354f2b619ee95b2efe353e0766/setuptools-62.3.2-py3-none-any.whl
Collecting wheel
Using cached https://files.pythonhosted.org/packages/27/d6/003e593296a85fd6ed616ed962795b2f87709c3eee2bca4f6d0fe55c6d00/wheel-0.37.1-py2.py3-none-any.whl
Installing collected packages: pip, setuptools, wheel
.....
Collecting grpcio~=1.33
Using cached grpcio-1.46.3-cp37-cp37m-macosx_10_10_x86_64.whl (4.3 MB)
Installing collected packages: six, semver, pyyaml, protobuf, dill, grpcio, pulumi
Successfully installed dill-0.3.5.1 grpcio-1.46.3 protobuf-3.20.1 pulumi-3.33.2 pyyaml-5.4.1 semver-2.13.0 six-1.16.0
Finished installing dependencies

Your new project is ready to go! ✨

To perform an initial deployment, run 'pulumi up'

    Note: When storing a state file locally within the same path as your Stack, the option –force is required as the directory is not empty (it contains the pulumi-statefile folder).

  3. You will now run the initial deployment. Keep the passphrase you entered in the previous step handy.

       $> pulumi up
   Enter your passphrase to unlock config/secrets
      (set PULUMI_CONFIG_PASSPHRASE or PULUMI_CONFIG_PASSPHRASE_FILE to remember):
   Previewing update (dev):
      Type                 Name             Plan
   +   pulumi:pulumi:Stack  pulumi-oke-py-dev  create

   Resources:
      + 1 to create

   Do you want to perform this update? yes
   Updating (dev):
      Type                 Name             Status
   +   pulumi:pulumi:Stack  pulumi-oke-py-dev  created

   Resources:
      + 1 created

   Duration: 1s

  4. Confirm the environment is created using the following commands.

    Note: Right after the template is properly created and stored locally, Pulumi will also create a Python virtual (venv) environment with site-packages used by your Python program. Virtual environments are not in scope for this tutorial.

    -rw-r--r--. 1 <user> oci 45 May 23 20:07 __main__.py
-rw-r--r--. 1 <user> oci 117 May 23 20:07 Pulumi.yaml
-rw-r--r--. 1 <user> oci 21 May 23 20:07 requirements.txt
-rw-r--r--. 1 <user> oci 21 May 23 20:07 Pulumi.dev.yaml
drwxr-xr-x. 5 <user> oci 100 May 23 20:08 venv
drwxr-xr-x. 5 <user> oci 100 May 23 20:06 oci-stack-statefile

    • __main__py : The entry point for your Python application and this is what Pulumi will scan for.

    • Pulumi.yaml : Stack descriptor.

    • Pulumi.dev.yaml : Stack environment configuration.

    • requirements.txt: Standard file defined to list the pip python needed packages.

    • venv : Python virtual environment.

Task 2: Configure the Python virtual environment

Pulumi has registered the OCI provider and it is available through pip. pip is a Python package manager to install existing libraries. Let’s install it, but only in the virtual environment.

  1. Activate your Python virtual environment using the following command.

    $>  source venv/bin/activate

  2. Install pulumi_oci package using pip.

    (venv) j2joi@cloudshell:pulumi-oke-py (us-ashburn-1)$  pip install pulumi_oci

  3. Configure the OCI Provider for Pulumi.

    Note: Pulumi stores configuration details within it’s own configuration service. You must configure the OCI provider for Pulumi to authenticate against your tenancy. Collect the user profile, API key, user ocid, tenancy ocid, region and store it as secrets inside the pulumi env configuration. Note that you will be passing them as secrets, so it will be encrypted using the passphrase you provided when creating the environment.

    $> pulumi config set oci:tenancyOcid "ocid1.tenancy.oc1..<unique_ID>" --secret  
$> pulumi config set oci:userOcid "ocid1.user.oc1..<unique_ID>" --secret
$> pulumi config set oci:fingerprint "<key_fingerprint>" --secret
$> pulumi config set oci:region "<target region in OCI>"
$> cat "~/.oci/oci_api_key.pem" | pulumi config set oci:privateKey --secret
$> pulumi config set compartment_ocid "ocid1.compartment.oc1..aaaaaaaaqiuXXXXXXXXXX"

    Tip:

    • Export PULUMI_CONFIG_PASSPHRASE with your passphrase value. It will save time instead of providing it every time a secret is added.
    • OCI config values can be read directly from environment variables in the form of TF_VAR_${var_name}.

  4. Confirm that your secrets are stored in the configuration store.

    $> (venv) j2joi@cloudshell:pulumi-oke-py (us-ashburn-1)$  pulumi config

KEY VALUE
oci:fingerprint [secret]
oci:privateKey [secret]
oci:region us-ashburn-1
oci:tenancyOcid [secret]
oci:userOcid [secret]

    Note: Pulumi uses two main var types, input [T] and Output[T] of type T. Input[T] is a wrapper to raw values of basic Python data types ( String, boolean, Dict, Sequence ) and Output holds the future value once a resource is created.

    Keep this in mind if you are attempting to manipulate the value of an Output by passing it to any Python standard library, that is, attempting to split a resource Output [String] name, like name.split(), as it will end up in error because name variable of type Output does not support split.

  5. Create the first OKE cluster. Before you start coding, the following code snippet requires an existing Virtual Cloud Network created. Subnets for OKE should have 3 subnets required for Kubernetes Endpoint, Worker nodes, and Load Balancer Service. For more information on the subnet requirements for OKE, such as policy, routing, see the OCI documentation.

    Let’s review the OCI Cluster Python variable definition:

    Type: pulumi_oci.containerengine.Cluster

Arguments:

   - compartment_id : This is the Oracle Cloud Identifier OCID linked to the target compartment.
   - kubernetes_version : Update with the Kubernetes main stream versions that are supported in OCI.
   - name: This is the Kubernetes cluster displayed name.
   - options: Network CIDRs for different services and a list of subnets used for loadbalancer services, wrapped in a ClusterOptionsKubernetesNetworkConfigArgs object.
   - endpoint_config - Define if the cluster Kubernetes endpoint will have a public IP address and which subnet it will be attached to.
   - vcn_id - Virtual cloud network the Kubernetes cluster will be attached to.

  6. Once your OCI VCN is ready and you have the VCN, Subnets and compartment OCID details, proceed to edit the __main__.yaml file with the following code snippet. Update values with corresponding OCIDs.

     "A Python Pulumi program"
 import pulumi
 import pulumi_oci as oci

 config = pulumi.Config()

 target_compartment_ocid = "ocid1.compartment.oc1.iad.XXXXXXX"

 subnets_lb_ocid= ["ocid1.subnet.oc1.iad.aaaaaaaan6fXXXXXXXXX"]

 subnet_workernodes_ocid = "ocid1.subnet.oc1.iad.aaaaaaaakYYYYYYY"

 subnet_endpoint_ocid = "ocid1.subnet.oc1.iad.aaaaaaaaxmfZZZZZZZZZ"

 vcn_ocid="ocid1.vcn.oc1.iad.amaaaaaadoggtjaasym4AAAAAAAAAA"

 kubernetes_version="v1.23.4"

 oke_cluster_opts_args = oci.containerengine.ClusterOptionsArgs(
 kubernetes_network_config=oci.containerengine.ClusterOptionsKubernetesNetworkConfigArgs(
 pods_cidr="10.244.0.0/16",
 services_cidr="10.96.0.0/16"),
 service_lb_subnet_ids = subnets_lb_ocid
 )

 oke_cluster = oci.containerengine.Cluster("oke_cluster",
 compartment_id = target_compartment_ocid,
 kubernetes_version = kubernetes_version,
 name = "firstPulumiPy_OKE",
 options = oke_cluster_opts_args,
 endpoint_config = oci.containerengine.ClusterEndpointConfigArgs(
 is_public_ip_enabled = True,
 subnet_id = subnet_endpoint_ocid),
 vcn_id = vcn_ocid,
 )
 pulumi.export("oke_cluster_ocid", oke_cluster)

    Tip: You can request resource ocids from Pulumi config store to reuse your code for other environments instead of assigning ocid values directly in your Python code.

    Note: Indentation is essential to Python. Make sure your code follows Python indentation.

  7. Review infrastructure changes: Pulumi has the option to preview what resources the current stack is going to create/modify. Run the “pulumi preview” option to validate your current python stack.

    (venv) j2joi@cloudshell:pulumi-oke-py (us-ashburn-1)$ pulumi preview
Previewing update (dev)

     Type                            Name               Plan       Info
 +   pulumi:pulumi:Stack             pulumi-oke-py-dev  create     1 message
 +   └─ oci:ContainerEngine:Cluster  oke_cluster        create

  8. Apply changes to your infrastructure: If no error message is displayed, you can create an OKE cluster. Run “pulumi up” to trigger the stack to apply changes.

    (venv) j2joi@cloudshell:pulumi-oke-py (us-ashburn-1)$ pulumi up
Updating (dev)

     Type                            Name               Status      Info
 +   pulumi:pulumi:Stack             pulumi-oke-py-dev  created     12 messages
 +   └─ oci:ContainerEngine:Cluster  oke_cluster        created

  9. Add an OKE Node Pool resource: Pulumi Stack created your first OKE cluster “oke_cluster”. Now, lets add worker nodes assigned to a Node Pool variable in the same main.py file. Changes will be incrementally added to your set of resources in OCI because Pulumi tracks the created resources and will compare any differences between the resource inside the cloud provider and the state stored in the selected backend to validate if updates are required to existing or new resources.

    Worker node Compute Instance details
number_worker_nodes= 3
default_node_memory_size=16
default_node_num_ocpus =1

# Get the Availability Domain names for this region
ad_list=oci.identity.get_availability_domains(target_compartment_ocid)
ad=ad_list.availability_domains

# Get the list of supported images based on the compartment and filter it by most recent image
list_of_supported_image =oci.core.get_images(compartment_id=target_compartment_ocid,
                                      operating_system="Oracle Linux",
                                      operating_system_version= "7.9",
                                      shape= "VM.Standard.E3.Flex",
                                      sort_by="TIMECREATED",
                                      sort_order="DESC"
                                      )
#Obtain the first image ocid from  list_of_supported_image var
os_image_ocid= list_of_supported_image.images[0].id

# Pin for this example all worker nodes to a first Availability Domain
place_nodes_subnet=[oci.containerengine.NodePoolNodeConfigDetailsPlacementConfigArgs(
                availability_domain=ad[0].name,
                subnet_id= subnet_workernodes_ocid
            ),]

#Declare all Pool properties
node_pool_args = oci.containerengine.NodePoolArgs(cluster_id=oke_cluster.id,
                                                      compartment_id= target_compartment_ocid,
                                                      kubernetes_version=kubernetes_version,
                                                      node_shape= "VM.Standard.E3.Flex",
                                                      name="E3Flex",
                                                      node_config_details=oci.containerengine.NodePoolNodeConfigDetailsArgs(
                                                          placement_configs= place_nodes_subnet ,
                                                          size= number_worker_nodes
                                                      ),
                                                      node_shape_config=oci.containerengine.NodePoolNodeShapeConfigArgs(
                                                          memory_in_gbs= default_node_memory_size,
                                                          ocpus= default_node_num_ocpus,
                                                      ),
                                                      node_source_details=oci.containerengine.NodePoolNodeSourceDetailsArgs(
                                                          image_id= os_image_ocid,
                                                          source_type="IMAGE",
                                                          boot_volume_size_in_gbs=60,
                                                      ),
                                                      )
#Assign node pool properties as args to node_pool
node_pool = oci.containerengine.NodePool("nodepool", args=node_pool_args)

pulumi.export("oke_cluster_ocid", oke_cluster)
pulumi.export("oke_node_pool", node_pool)

  10. Update your infrastructure again by running the “pulumi up” command.

    $> pulumi up

Additional Information

Pulumi Marketplace already has some other existing providers that you can use, like Kubernetes, Helm, and so on. One advantage of creating resources in Oracle Cloud, such as Kubernetes, is that it is compatible with most Kubernetes standards and the integration is simple. The following example shows how to create your own Kubernetes provider using Pulumi’s provider and deploy applications in your Python code.

import pulumi_kubernetes

kubeconfig = oci.containerengine.get_cluster_kube_config_output(cluster_id=oke_cluster.id).apply(
   lambda kube_config: kube_config.content)

k8sProvider = pulumi_kubernetes.Provider("okeK8s",
                                        kubeconfig=kubeconfig
                                        )


pulumi.export("kubeconfig", kubeconfig)

Task 3: Remove Resources

You have completed your first Python Stack with OCI provider for Pulumi. To delete all the resources, run the pulumi destroy command.

$> pulumi destroy

pulumi destroy
Previewing destroy (dev):
     Type                            Name             Plan
 -   pulumi:pulumi:Stack             local-state-dev  delete
 -   └─ oci:ContainerEngine:Cluster  oke_cluster      delete

Acknowledgments

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.