Note:

Use Golang microservices with OCI Resource Manager to deploy Airflow on OKE

Introduction

This is part one of a six-part tutorial series that shows you how to deploy a temporary set of resources on an OKE cluster using Golang microservices representing the usage of OCI SDK, OCI-CLI, Resource Manager, OCI Devops and Helm to deploy and destroy Apache Airflow.

Objective

This tutorial series demonstrates how to implement the following process flow using a Golang microservice to create and destroy resources on an OKE cluster. In this tutorial we will cover how to install and configure an OKE cluster.

Process Flow

Architecture

Architecture

Prerequisites

Task 1: Install and configure an OKE cluster using the wizard

  1. Create a new compartment for this lab: Log in to your Oracle Cloud account. Click on the top left navigation menu, select Identity & Security, and then click Compartments.

    T1_1

  2. Create a new OKE cluster inside the compartment we just created oci-resource-manager-lab.

    Note: In this section we will use shape “Ampere” which is ARM64 architecture. This is an important note because the microservices we’ll deploy must be in the same architecture, otherwise it will not work!

    T1_2

    T1_2

  3. Create VM instance to work as a jump-box for your bastion: In order to access the OKE endpoint we will need to setup a bastion session. For that, you need first to provide a jump box VM instance. You can follow the standard documentation for creating new instances: Launch Instances on OCI.

    Note: You can also use the console Cloud shell which allows you to access the OKE endpoint. For the purpose of this tutorial, we will go with Bastion on a jump box. For more information, see CloudShell

    1. Go to Compute and click to create a new compute instance on the free-tier.

      T1_3

    2. Select Ampere VM Shape (VM.Standard.A1.Flex) which is on the free-tier and place it on the same subnet as the OKE nodes (“oke-nodesubnet-quick-….”).

      T1_3

    3. Paste your laptop public keys in order to access the VM from your machine. If you don’t have a ssh key pair, follow these instructions: Generate ssh-key-pair.

      T1_3

    4. Click Show advanced options and go to the Oracle Cloud Agent tab, and select the Bastion checkbox.

      T1_3

  4. Create your bastion and point it to the jump-box VM. Make sure you select the same subnet as your jump-box VM.

    T1_4

    1. Now you need to create a bastion SESSION which lives for certain amount of time, so after it’s expired, you will need to create a new session. Make sure you paste your ssh public key.

      T1_4

      Note: For Oracle Linux image, the default username is “opc”.

      T1_4

    2. Get the SSH command snipped and then just setup your ssh key location.

      ssh -i <privateKey> -o ProxyCommand="ssh -i <privateKey> -W %h:%p -p 22 ocid1.bastionsession.oc1.sa-saopaulo-1.amaaaaaa3jinyjqa4kfosxytglr6q27365j67yjdh5hxrfpqn4y72nq2p6ma@host.bastion.sa-saopaulo-1.oci.oraclecloud.com" -p 22 opc@10.0.10.127

    3. Replace the “privateKey” for your local ssh key location, usually it is ~/.ssh/id_rsa.

      ssh -i ~/.ssh/id_rsa -o ProxyCommand="ssh -i ~/.ssh/id_rsa -W %h:%p -p 22 ocid1.bastionsession.oc1.sa-saopaulo-1.amaaaaaa3jinyjqa4kfosxytglr6q27365j67yjdh5hxrfpqn4y72nq2p6ma@host.bastion.sa-saopaulo-1.oci.oraclecloud.com" -p 22 opc@10.0.10.127

      T1_4

  5. Install and set up OCI CLI on your jump-box. You must set up the OCI CLI on your jump-box in order to use OCI resources. For full instructions and details, see Install and configure the OCI Command Line Interface (CLI).

    1. Go to your jump-box console and run the following commands.

       sudo dnf -y install oraclelinux-developer-release-el8 -y
 sudo dnf install python36-oci-cli -y

      T1_5

      Note: Make a note of the following details as you will need it for the later steps:

      • User OCID: OCID of the user you want to have access to OCI resources

      • Tenancy OCID: OCID for your tenancy

      • Your region: for example, sa-saopaulo-1

    2. Run the OCIsetup config command.

      oci setup config

    3. Complete the required with the values notes in the above step and click Y to create a RSA Key pair and go with default enter for the rest of fields.

      T1_5

      • The ssh-key files has been created, oci_api_key_public.pem and oci_api_key.pem.

      • Now you must upload the generated ssh key into your user API inside Oracle Cloud. For details, see Upload Key. This is needed in order to be able to use OCI-CLI and communicate with your Oracle Cloud tenant. For this tutorial, we created a specific user called devmaster, but you can choose any of yours.

      • Get the content of your oci_api_key_public.pem file.

        T1_5

      • Click Add API Key and paste the contents of the oci_api_key_public.pem file.

        T1_5

        T1_5

        T1_5

      • The OCI config file should be already created on your jump-box in Step 2. Check the ~/.oci/config file and make sure the key_file path is correct and then test your connection.

      • In your jump-box console, run the following command.

        oci iam compartment list

      • You should get the list of current compartments in your tenancy, which means you have access to your OCI tentant.

  6. Setup your jump-box to access your OKE cluster using the command line.

    1. Install the kubectl command on your jump-box instance.

      Note: We are using “arm64” as the architecture, this is needed since we’re using Ampere shape on our jump-box.

    2. In your jump-box console, run the following command.

       cd /home/opc   
 curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/arm64/kubectl
 chmod +x ./kubectl
 sudo mv ./kubectl /usr/local/bin

  7. Go to your OKE cluster on the console page and click Access Cluster to get the commands you need to run on your jump-box.

    T1_6

    T1_6

  8. Run the fix permissions and test the connection.

    chmod 600 /home/opc/.kube/config
kubectl get nodes

    T1_6

Now you’re connected to your Kubernetes cluster from the jump-box!

Next Step

To proceed to the next tutorial in this learning path, click here.

Acknowledgments

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.