Set Up a CI/CD Pipeline for Cloud Deployments

Rapid delivery of software is important for running your applications in the cloud efficiently. Jenkins is a popular product for automating the Ccontinuous Iintegration (CI) and Continuous Deployment (CD) pipelines for workloads in Oracle Cloud.

Architecture

In this reference architecture, Jenkins is hosted on Oracle Cloud Infrastructure to centralize build automation and scale the deployment. The Continuous Integration process creates an application and deploys it by using Oracle Cloud Infrastructure Registry (OCIR) and Container Engine for Kubernetes (OKE). GitHub is used to manage source code.

GitHub provides web hook integration, so Jenkins starts running automated builds and tests after each code check-in. A sample web application is deployed as part of CI/CD pipeline, which end users can access from the Container Engine for Kubernetes cluster. To simplify the process, Terraform is used for infrastructure automation.

In this reference architecture, Terraform leverages a Terraform module capable of deploying Jenkins in several different configurations from a single Jenkins VM through to a controller (server) and agent (worker) node configuration. This means that the Jenkins module can be incorporated into other architecture deployments for non-Kubernetes use cases (for example APEX application development, WebLogic on VMs and so on).

The agent configuration means that the environment can be reconfigured to provide different numbers of compute instances reflecting the workload requirements. This includes the possibility of replacing agent VMs to avoid problems of accumulating old dependencies, transient dependency conflicts, and builds working with undeclared dependencies.

The VM Nodes running Jenkins are configured using a predefined VM image that has been prebuilt with all the necessary software to support Jenkins.

The following diagram illustrates this reference architecture.

Description of the illustration cicd-oci.png

cicd-oci-oracle.zip

This architecture has the following components:

• Region

  An Oracle Cloud Infrastructure region is a localized geographic area that contains one or more data centers, called availability domains. Regions are independent of other regions, and vast distances can separate them (across countries or even continents).

• Availability domains

  Availability domains are standalone, independent data centers within a region. The physical resources in each availability domain are isolated from the resources in the other availability domains, which provides fault tolerance. Availability domains don’t share infrastructure such as power or cooling, or the internal availability domain network. So, a failure at one availability domain is unlikely to affect the other availability domains in the region.

• Virtual cloud network (VCN) and subnets

  Jenkins runs on a virtual machine (VM) Compute instance that’s deployed in a VCN that you can segment into subnets. Jenkins is hosted on regional public subnet A, and Container Engine for Kubernetes is deployed in regional public subnet B.

• Compute instance

  Jenkins is deployed to a Compute instance VM. The Container Engine for Kubernetes cluster also runs its nodes on Compute instances.

• Container Engine for Kubernetes

  Container Engine for Kubernetes is a fully managed, scalable, and highly available service that you can use to deploy your containerized applications to the cloud. You specify the Compute resources that your applications require, and Container Engine for Kubernetes provisions them on Oracle Cloud Infrastructure in an existing tenancy.

• Registry

  Registry is an Oracle-managed registry that enables you to simplify the development of production workflow. Registry makes it easy for you as a developer to store, share, and manage development artifacts like Docker images.

• Jenkins

  Jenkins is an open source automation server that lets developers reliably build, test, and deploy software. Jenkins supports the master/agent mode, where the workload of building projects is delegated to multiple agent nodes by the master. A single Jenkins installation can host multiple projects or provide different environments for builds and tests.

Recommendations

Your requirements might differ from the architecture described here. Use the following recommendations as a starting point.

• Compute shapes

  This architecture uses an Oracle Linux OS image with the VM.Standard2.1 shape to host both the Jenkins server and the Container Engine for Kubernetes cluster nodes. If your application needs more memory or cores, you can choose a different shape.

• VCN

  When you create a VCN, determine the number of CIDR blocks required and the size of each block based on the number of resources that you plan to attach to subnets in the VCN. Use CIDR blocks that are within the standard private IP address space.

  After you create a VCN, you can change, add, and remove its CIDR blocks.

  This architecture uses a public VCN to host Container Engine for Kubernetes. You can also use a private VCN. In that case, use a NAT gateway to give the cluster access over the public internet.

• Jenkins

  This architecture deploys Jenkins on a Compute instance. A Jenkins master node is used to build the CI/CD pipeline. If you have multiple pipelines to build and run in parallel, you can use a Jenkins agent node to develop more pipelines.

• Container Engine for Kubernetes

  This architecture deploys the Container Engine for Kubernetes cluster. The worker nodes are deployed on a VM.Standard2.1 Oracle Linux OS. This architecture uses three worker nodes in the cluster, but you can create up to 1000 nodes on each cluster.

• Registry

  This architecture deploys Registry as a private Docker registry for internal use. Docker images are pushed to and pulled from the Registry. You can also use Registry as a public Docker registry, enabling any user with internet access and knowledge of the appropriate URL to pull images from public repositories in Oracle Cloud.

Considerations

• Scalability

  You can scale out your application by updating the number of worker nodes in the Container Engine for Kubernetes cluster, depending on the load. You can also scale in by reducing the number of worker nodes in the cluster. When you create a service on the cluster, you can create a load balancer to distribute traffic among the nodes assigned to that service. For Jenkins, you can use the Jenkins master node to create more agents for multiple pipelines.

• Application availability

  Fault domains provide the best resilience within a single availability domain. You can deploy Compute instances that perform the same tasks in multiple availability domains. This design removes a single point of failure by introducing redundancy.

• Manageability

  This architecture uses a sample web application that’s hosted on GitHub for source control. Registry is used in the build pipeline to store the Docker build image for the application.

• Security

  Use policies to restrict who can access the Oracle Cloud Infrastructure resources that your company has and how.

  Container Engine for Kubernetes is integrated with Oracle Cloud Infrastructure Identity and Access Management (IAM), which provides easy authentication with native Oracle Cloud Infrastructure identity functionality.

Deploy

The code required to deploy this reference architecture is available in GitHub. You can pull the code into Oracle Cloud Infrastructure Resource Manager with a single click, create the stack, and deploy it. Alternatively, download the code from GitHub to your computer, customize the code, and deploy the architecture by using the Terraform CLI.

• Deploy by using Oracle Cloud Infrastructure Resource Manager:
  1. Click

     If you aren't already signed in, enter the tenancy and user credentials.

  2. Review and accept the terms and conditions.
  3. Select the region where you want to deploy the stack.
  4. Follow the on-screen prompts and instructions to create the stack.
  5. After creating the stack, click Terraform Actions, and select Plan.
  6. Wait for the job to be completed, and review the plan.

     To make any changes, return to the Stack Details page, click Edit Stack, and make the required changes. Then, run the Plan action again.

  7. If no further changes are necessary, return to the Stack Details page, click Terraform Actions, and select Apply.
• Deploy by using the Terraform CLI:
  1. Go to GitHub.
  2. Download or clone the code to your local computer.
  3. Follow the instructions in README.md.

Explore More

Learn more about deploying containerized applications on Kubernetes in the cloud.

• Overview of Container Engine for Kubernetes
• Overview of Registry
• Best practices framework for Oracle Cloud Infrastructure
• terraform-oci-arch-ci-cd on Github
• Jenkins Plugins for OCI
• Oracle Cloud Infrastructure Documentation Images
• oracle-quickstart / oci-jenkins on Github (module for deploying just Jenkins components)
• Maven Dependency Management
• Jenkins Pipeline Best Practices

Change Log

This log lists significant changes:

January 6, 2021	Added instructions for deploying the architecture by using Oracle Cloud Infrastructure Resource Manager.
August 5, 2022	Added new content to Architecture topic. Added new links to Explore More topic.