Deploy Jenkins in Master/Agent Mode

Faster software development has become a competitive advantage for companies. The automation of software development processes facilitates speed and consistency, which led to the rise of continuous integration (CI) and continuous delivery and deployment (CD) pipelines. Jenkins is a popular product among Oracle Cloud Infrastructure customers that can automate all phases of CI and CD. You can host Jenkins on Oracle Cloud Infrastructure to centralize your build automation and scale your deployment as the needs of your software projects grow.

Architecture

This reference architecture shows how to deploy Jenkins in master/agent mode by using the Jenkins Oracle Cloud Infrastructure Compute plugin. When installed on a Jenkins master instance, the plugin lets you create agent instances on demand within Oracle Cloud Infrastructure and remove instances or free resources automatically after the build job finishes.

This architecture contains one master instance and two agent instances as a starting point of a deployment. You can adjust the number of agent instances or the size of the master instance as needed. The Jenkins master instance should be installed with the Oracle Cloud Infrastructure plugin code.

This architecture uses a region with one availability domain and a regional subnet. The same reference architecture can be used in a region with multiple availability domains. We recommend using a regional subnet for your deployment regardless of the number of availability domains.

This architecture also contains a load balancer and a NAT gateway to provide secure access to the internet.

The following diagram illustrates this reference architecture.

Description of jenkins-oci.eps follows
Description of the illustration jenkins-oci.eps

The architecture has the following components:

  • Jenkins master instance

    This virtual Compute instance acts as the master node. It monitors the state of the agent instances (offline or online) and receives task result responses from the agents.

  • Jenkins agent instances

    These virtual Compute instances act as agent nodes. The master node creates them as needed, and they perform any jobs as directed by the master node.

  • Region

    An Oracle Cloud Infrastructure region is a localized geographic area that contains one or more data centers, called availability domains. Regions are independent of other regions, and vast distances can separate them (across countries or even continents).

  • Load balancer

    The load balancer distributes incoming traffic to the Jenkins master node and provides internet access to users accessing the master node. We recommend using a 100-Mbps load balancer because it will mainly be used to connect to Jenkins master and the traffic flow back to the user will not be heavy.

  • NAT gateway

    A Network Address Translation (NAT) gateway provides NAT service. You don’t have to choose the size of the gateway

  • Virtual cloud network (VCN) and subnets

    Every Compute instance is deployed into a VCN that can be segmented into subnets.

  • Security lists

    For each subnet, you can create security rules that specify the source, destination, and type of traffic that must be allowed in and out of the subnet.

  • Availability domains

    Availability domains are standalone, independent data centers within a region. The physical resources in each availability domain are isolated from the resources in the other availability domains, which provides fault tolerance. Availability domains don’t share infrastructure such as power or cooling, or the internal availability domain network. So, a failure at one availability domain is unlikely to affect the other availability domains in the region.

  • Fault domains

    A fault domain is a grouping of hardware and infrastructure within an availability domain. Each availability domain has three fault domains with independent power and hardware. When you distribute resources across multiple fault domains, your applications can tolerate physical server failure, system maintenance, and power failures inside a fault domain.

Recommendations

Your requirements might differ from the architecture described here. Use the following recommendations as a starting point.

  • Compute Shapes (Jenkins master)

    This architecture uses two core virtual machine (VM) shapes for the Jenkins master node. A master node is responsible for distributing tasks, collecting agent node results, and monitoring agent nodes for availability.

  • Compute Shapes (Jenkins agent)

    This architecture uses four core VM shapes for Jenkins agent nodes. Ensure that the CPU for agents is higher than it is for the master node.

  • Compute Shapes (bastion host)

    A bastion host is used to access any nodes in the private subnet. We recommend using the VM.Standard.E2.1 or the VM.Standard.E2.2 shape.

  • VCN

    When you create the VCN, determine how many IP addresses your cloud resources in each subnet require. Using the Classless Inter-Domain Routing (CIDR) notation, specify a subnet mask and a network address range that's large enough for the required IP addresses.

    After you create a VCN, you can't change its address range.

    When you design the subnets, consider your traffic flow and security requirements. Attach all the resources within a specific tier or role to the same subnet, which can serve as a security boundary.

    Use a regional subnet.

  • Security lists

    Use security lists to define inbound and outboard traffic that applies to the entire subnet. This architecture allows ICMP internally for the entire private subnet.

Considerations

  • Performance

    To get the best performance, ensure that the Jenkins master node has enough cores and memory. Depending on the build or other tasks run by agent nodes, create agent nodes with enough cores and memory.

  • Availability

    The Jenkins master node monitors agent nodes for availability and spawns a new node as needed. In a region with multiple availability domains, you can create a deployment template (on the Jenkins master) for agent nodes in different availability domains.

  • Costs

    Size the CPUs on both the master and agent nodes according to your expected work load deployment. You can change node shapes in the Console, so start with a smaller CPU count (preferably two) and scale as needed. Specify the shape of your agent node with the Instance template in the master node.

  • Monitoring and alerts

    Set up monitoring and alerts on CPU and memory usage for your master and agent nodes so that you can scale the VM shape up or down as needed.

Deploy

The Terraform code for this reference architecture is available in GitHub. You can pull the code into Oracle Cloud Infrastructure Resource Manager with a single click, create the stack, and deploy it. Alternatively, you can download the code from GitHub to your computer, customize the code, and deploy the architecture by using the Terraform CLI.

  • Deploy by using Oracle Cloud Infrastructure Resource Manager:
    1. Click Deploy to Oracle Cloud

      If you aren't already signed in, enter the tenancy and user credentials.

    2. Review and accept the terms and conditions.
    3. Select the region where you want to deploy the stack.
    4. Follow the on-screen prompts and instructions to create the stack.
    5. After creating the stack, click Terraform Actions, and select Plan.
    6. Wait for the job to be completed, and review the plan.

      To make any changes, return to the Stack Details page, click Edit Stack, and make the required changes. Then, run the Plan action again.

    7. If no further changes are necessary, return to the Stack Details page, click Terraform Actions, and select Apply.
  • Deploy by using the Terraform CLI:
    1. Go to GitHub.
    2. Download or clone the code to your local computer.
    3. Follow the instructions in the README.

Change Log

This log lists only the significant changes: