Creating Load Balancers to Distribute Traffic Between Cluster Nodes

Consider the following configuration file, nginx_lb.yaml. It defines a deployment (kind: Deployment) for the nginx app, followed by a service definition with a type of LoadBalancer (type: LoadBalancer) that balances http traffic on port 80 for the nginx app.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-nginx
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: my-nginx-svc
  labels:
    app: nginx
spec:
  type: LoadBalancer
  ports:
  - port: 80
  selector:
    app: nginx

The first part of the configuration file defines an Nginx deployment, requesting that it be hosted on 3 pods running the nginx:1.7.9 image, and accept traffic to the containers on port 80.

The second part of the configuration file defines the Nginx service, which uses type LoadBalancer to balance Nginx traffic on port 80 amongst the available pods.

To create the deployment and service defined in nginx_lb.yaml while connected to your Kubernetes cluster, enter the command:

kubectl apply -f nginx_lb.yaml

This command outputs the following upon successful creation of the deployment and the load balancer:

deployment "my-nginx" created
service "my-nginx-svc" created

The load balancer may take a few minutes to go from a pending state to being fully operational. You can view the current state of your cluster by entering:

kubectl get all

The output from the above command shows the current state:

NAME                                  READY     STATUS    RESTARTS   AGE
po/my-nginx-431080787-0m4m8           1/1       Running   0          3m
po/my-nginx-431080787-hqqcr           1/1       Running   0          3m
po/my-nginx-431080787-n8125           1/1       Running   0          3m

NAME               CLUSTER-IP      EXTERNAL-IP      PORT(S)        AGE
svc/kubernetes     203.0.113.1     <NONE>           443/TCP        3d
svc/my-nginx-svc   203.0.113.7     192.0.2.22       80:30269/TCP   3m

NAME                      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/my-nginx           3         3         3            3           3m

NAME                            DESIRED   CURRENT   READY     AGE
rs/my-nginx-431080787           3         3         3         3m

The output shows that the my-nginx deployment is running on 3 pods (the po/my-nginx entries), that the load balancer is running (svc/my-nginx-svc) and has an external IP (192.0.2.22) that clients can use to connect to the app that's deployed on the pods.

You can create a load balancer with SSL termination, allowing https traffic to an app to be distributed among the nodes in a cluster. This example provides a walkthrough of the configuration and creation of a load balancer with SSL support.

Consider the following configuration file, nginx-demo-svc-ssl.yaml, which defines an Nginx deployment and exposes it via a load balancer that serves http on port 80, and https on port 443. This sample creates an Oracle Cloud Infrastructure load balancer, by defining a service with a type of LoadBalancer (type: LoadBalancer).

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
---
kind: Service
apiVersion: v1
metadata:
  name: nginx-service
  annotations:
    service.beta.kubernetes.io/oci-load-balancer-ssl-ports: "443"
    service.beta.kubernetes.io/oci-load-balancer-tls-secret: ssl-certificate-secret
spec:
  selector:
    app: nginx
  type: LoadBalancer
  ports:
  - name: http
    port: 80
    targetPort: 80
  - name: https
    port: 443
    targetPort: 80

The Load Balancer's annotations are of particular importance. The ports on which to support https traffic are defined by the value of service.beta.kubernetes.io/oci-load-balancer-ssl-ports. You can declare multiple SSL ports by using a comma-separated list for the annotation's value. For example, you could set the annotation's value to "443, 3000" to support SSL on ports 443 and 3000.

The required TLS secret, ssl-certificate-secret, needs to be created in Kubernetes. This example creates and uses a self-signed certificate. However, in a production environment, the most common scenario is to use a public certificate that's been signed by a certificate authority.

The following command creates a self-signed certificate, tls.crt, with its corresponding key, tls.key:

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=nginxsvc/O=nginxsvc"

Now that you created the certificate, you need to store both it and its key as a secret in Kubernetes. The name of the secret must match the name from the service.beta.kubernetes.io/oci-load-balancer-tls-secret annotation of the load balancer's definition. Use the following command to create a TLS secret in Kubernetes, whose key and certificate values are set by --key and --cert, respectively.

kubectl create secret tls ssl-certificate-secret --key tls.key --cert tls.crt

You must create the Kubernetes secret before you can create the service, since the service references the secret in its definition. Create the service using the following command:

kubectl create -f manifests/demo/nginx-demo-svc-ssl.yaml

Watch the service and wait for a public IP address (EXTERNAL-IP) to be assigned to the Nginx service (nginx-service) by entering:

kubectl get svc --watch

The output from the above command shows the load balancer IP to use to connect to the service.

NAME            CLUSTER-IP     EXTERNAL-IP      PORT(S)        AGE
nginx-service   192.0.2.1      198.51.100.1     80:30274/TCP   5m

The load balancer is now running, which means the service can now be accessed as follows:

• using http, by entering:

  curl http://198.51.100.1

• using https, by entering:

  curl --insecure https://198.51.100.1

  The "--insecure" flag is used to access the service using https due to the use of self-signed certificates in this example. Do not use this flag in a production environment where the public certificate was signed by a certificate authority.

Note: When a cluster is deleted, a load balancer that's dynamically created when a service is created will not be removed. Before deleting a cluster, delete the service, which in turn will result in the cloud provider removing the load balancer. The syntax for this command is:

kubectl delete svc SERVICE_NAME

For example, to delete the service from the previous example, enter:

kubectl delete svc nginx-service

You can create Oracle Cloud Infrastructure load balancers to control access to services running on a cluster:

• When you create a cluster in the 'Custom Create' workflow you select an existing VCN that contains the network resources to be used by the new cluster. If you want to use load balancers to control traffic into the VCN, you select existing public or private subnets in that VCN to host the load balancers.
• When you create a cluster in the 'Quick Create' workflow, the VCN that's automatically created contains a public regional subnet to host a load balancer. If you want to host load balancers in private subnets, you can add private subnets to the VCN later.

Alternatively, you can create an internal load balancer service (often referred to simply as an 'internal load balancer') in a cluster to enable other programs running in the same VCN as the cluster to access services in the cluster. An internal load balancer is an Oracle Cloud Infrastructure private load balancer. A private load balancer has a private IP address assigned by the Load Balancing service, which serves as the entry point for incoming traffic. For more information about Oracle Cloud Infrastructure private load balancers, see Load Balancer Types.

You can host internal load balancers in public subnets and private subnets.

To create an internal load balancer hosted on a public subnet, add the following annotation in the metadata section of the manifest file:

service.beta.kubernetes.io/oci-load-balancer-internal: "true"

To create an internal load balancer hosted on a private subnet, add both following annotations in the metadata section of the manifest file:

service.beta.kubernetes.io/oci-load-balancer-internal: "true"

service.beta.kubernetes.io/oci-load-balancer-subnet1: "ocid1.subnet.oc1..aaaaaa....vdfw"

where ocid1.subnet.oc1..aaaaaa....vdfw is the OCID of the private subnet.

For example:

apiVersion: v1
kind: Service
metadata:
  name: my-nginx-svc
  labels:
    app: nginx
  annotations:
    service.beta.kubernetes.io/oci-load-balancer-internal: "true"
    service.beta.kubernetes.io/oci-load-balancer-subnet1: "ocid1.subnet.oc1..aaaaaa....vdfw"
spec:
  type: LoadBalancer
  ports:
  - port: 8100
  selector:
    app: nginx

The shape of an Oracle Cloud Infrastructure load balancer specifies its maximum total bandwidth (that is, ingress plus egress). By default, load balancers are created with a shape of 100Mbps. Other shapes are available, including 400Mbps and 8000Mbps.

To specify an alternative shape for a load balancer, add the following annotation in the metadata section of the manifest file:

service.beta.kubernetes.io/oci-load-balancer-shape: <value>

where value is the bandwidth of the shape (for example, 100Mbps, 400Mbps, 8000Mbps).

For example:

apiVersion: v1
kind: Service
metadata:
  name: my-nginx-svc
  labels:
    app: nginx
  annotations:
    service.beta.kubernetes.io/oci-load-balancer-shape: 400Mbps
spec:
  type: LoadBalancer
  ports:
  - port: 80
  selector:
    app: nginx

Note: Sufficient load balancer quota must be available in the region for the shape you specify. Enter the following kubectl command to confirm that load balancer creation did not fail due to lack of quota:

kubectl describe service <service-name>

The shape of an Oracle Cloud Infrastructure load balancer specifies its maximum total bandwidth (that is, ingress plus egress). As described in Specifying Alternative Load Balancer Shapes, you can specify different load balancer shapes.

In addition, you can also specify a flexible shape for an Oracle Cloud Infrastructure load balancer, by defining a minimum and a maximum bandwidth for the load balancer.

To specify a flexible shape for a load balancer, add the following annotations in the metadata section of the manifest file:

service.beta.kubernetes.io/oci-load-balancer-shape: "flexible"
service.beta.kubernetes.io/oci-load-balancer-shape-flex-min: <min-value>
service.beta.kubernetes.io/oci-load-balancer-shape-flex-max: <max-value>

where:

• <min-value> is the minimum bandwidth for the load balancer, in Mbps (for example, 10)
• <max-value> is the maximum bandwidth for the load balancer, in Mbps (for example, 100)

Note that you do not include a unit of measurement when specifying bandwidth values for flexible load balancer shapes (unlike for pre-defined shapes). For example, specify the minimum bandwidth as 10 rather than as 10Mbps.

For example:

apiVersion: v1
kind: Service
metadata:
  name: my-nginx-svc
  labels:
    app: nginx
  annotations:
    service.beta.kubernetes.io/oci-load-balancer-shape: "flexible"
    service.beta.kubernetes.io/oci-load-balancer-shape-flex-min: 10
    service.beta.kubernetes.io/oci-load-balancer-shape-flex-max: 100
spec:
  type: LoadBalancer
  ports:
  - port: 80
  selector:
    app: nginx

You can specify the maximum idle time (in seconds) allowed between two successive receive or two successive send operations between the client and backend servers.

To explicitly specify a maximum idle time, add the following annotation in the metadata section of the manifest file:

service.beta.kubernetes.io/oci-load-balancer-connection-idle-timeout: <value>

where value is the number of seconds.

For example:

apiVersion: v1
kind: Service
metadata:
  name: my-nginx-svc
  labels:
    app: nginx
  annotations:
    service.beta.kubernetes.io/oci-load-balancer-connection-idle-timeout: 100
spec:
  type: LoadBalancer
  ports:
  - port: 80
  selector:
    app: nginx

Note that if you don't explicitly specify a maximum idle time, a default value is used. The default value depends on the type of listener:

• for TCP listeners, the default maximum idle time is 300 seconds
• for HTTP listeners, the default maximum idle time is 60 seconds

When a Kubernetes service of type LoadBalancer is deployed on a cluster, Container Engine for Kubernetes creates an Oracle Cloud Infrastructure public load balancer to accept traffic into the cluster. By default, the Oracle Cloud Infrastructure public load balancer is assigned an ephemeral public IP address. However, an ephemeral public IP address is temporary, and only lasts for the lifetime of the public load balancer.

If you want the Oracle Cloud Infrastructure public load balancer that Container Engine for Kubernetes creates to have the same public IP address deployment after deployment, you can assign the load balancer a reserved public IP address from the pool of reserved public IP addresses. For more information about creating and viewing reserved public IP addresses, see Public IP Addresses.

To assign a reserved public IP address to the Oracle Cloud Infrastructure public load balancer that Container Engine for Kubernetes creates, add the LoadBalancerIP property in the spec section of the manifest file that defines the LoadBalancer service, and specify the reserved public IP address.

apiVersion: v1
kind: Service
metadata:
  name: my-nginx-svc
  labels:
    app: nginx
spec:
  loadBalancerIP: 144.25.97.173
  type: LoadBalancer
  ports:
  - port: 80
  selector:
    app: nginx

Note the following:

• If you set the loadBalancerIP property of the LoadBalancer service, you cannot later directly change the IP address of the Oracle Cloud Infrastructure public load balancer that Container Engine for Kubernetes creates. If you do want to change the IP address of the public load balancer, delete the LoadBalancer service, specify a different reserved public IP address in the manifest file, and deploy the LoadBalancer service again.
• If you don't set the loadBalancerIP property of the LoadBalancer service, you cannot later directly change the IP address of the Oracle Cloud Infrastructure public load balancer that Container Engine for Kubernetes creates from an ephemeral IP address to a reserved public IP address. If you do want to change the ephemeral IP address to a reserved public IP address, delete the LoadBalancer service, set the loadBalancerIP property to a reserved public IP address in the manifest file, and deploy the LoadBalancer service again.
• You can delete the LoadBalancer service and release the reserved public IP address for other uses (for example, to assign it to another LoadBalancer service).
• You cannot specify a reserved public IP address for a LoadBalancer service if the reserved public IP address is already assigned to another resource (such as a compute instance, or another LoadBalancer service).
• You cannot add the loadBalancerIP property to the manifest file for an internal LoadBalancer service (that is, a manifest file that includes the service.beta.kubernetes.io/oci-load-balancer-internal: "true" annotation).
• By default, the reserved public IP address that you specify as the loadBalancerIP property of the LoadBalancer service in the manifest file is expected to be a resource in the same compartment as the cluster. If you want to specify a reserved public IP address in a different compartment, add the following policy to the tenancy:

  ALLOW any-user to read public-ips in tenancy where request.principal.type = 'cluster'
  ALLOW any-user to manage floating-ips in tenancy where request.principal.type = 'cluster'

Oracle Cloud Infrastructure network security groups (NSGs) enable you to control traffic into and out of resources, and between resources. The security rules defined for an NSG ensure that all the resources in that NSG have the same security posture. For more information, see Network Security Groups.

You can use an existing NSG to manage access to the Oracle Cloud Infrastructure load balancer that Container Engine for Kubernetes creates when a Kubernetes service of type LoadBalancer is deployed on a cluster.

To add the Oracle Cloud Infrastructure load balancer created by Container Engine for Kubernetes to an NSG, add the following annotation in the metadata section of the manifest file:

oci.oraclecloud.com/oci-network-security-groups: "<nsg-ocid>"

where <nsg-ocid> is the OCID of an existing NSG.

For example:

apiVersion: v1
kind: Service
metadata:
  name: my-nginx-svc
  labels:
    app: nginx
  annotations:
   oci.oraclecloud.com/oci-network-security-groups: "ocid1.nsg.oc1..aaaaaa....vdfw"
spec:
  type: LoadBalancer
  ports:
  - port: 80
  selector:
    app: nginx

Note the following:

• The NSG you specify must be in the same VCN as the Oracle Cloud Infrastructure load balancer.
• You can specify up to five NSGs, in a comma-separated list, in the format:

  oci.oraclecloud.com/oci-network-security-groups: "<nsg1-ocid>,<nsg2-ocid>,<nsg3-ocid>,<nsg4-ocid>,<nsg5-ocid>"

• To remove a load balancer from an NSG, or change the NSG that the load balancer is in, update the annotation and re-apply the manifest.

You can use the security list management feature in Kubernetes to manage security list rules. This feature is useful if you are new to Kubernetes, or for basic deployments.

To specify how the Kubernetes security list management feature manages security lists, add the following annotation in the metadata section of the manifest file:

service.beta.kubernetes.io/oci-load-balancer-security-list-management-mode: <value>

where <value> is one of:

• "All": All required security list rules for load balancer services are managed.
• "Frontend": Only security list rules for ingress to load balancer services are managed. You have to set up a rule that allows inbound traffic to the appropriate ports for node port ranges, the kube-proxy health port, and the health check port ranges.
• "None": No security list management is enabled. You have to set up a rule that allows inbound traffic to the appropriate ports for node port ranges, the kube-proxy health port, and the health check port ranges. Additionally, you have to set up rules to allow inbound traffic to load balancers.

For example:

apiVersion: v1
kind: Service
metadata:
  name: my-nginx-svc
  labels:
    app: nginx
  annotations:
    service.beta.kubernetes.io/oci-load-balancer-security-list-management-mode: "Frontend"
spec:
  type: LoadBalancer
  ports:
  - port: 80
  selector:
    app: nginx

Note that if you specify an invalid value for oci-load-balancer-security-list-management-mode, the value "All" is used instead.

You can define the type of traffic accepted by the load balancer listener by specifying the protocol on which the listener accepts connection requests.

To explicitly specify the load balancer listener protocol, add the following annotation in the metadata section of the manifest file:

service.beta.kubernetes.io/oci-load-balancer-backend-protocol: <value>

where <value> is the protocol that defines the type of traffic accepted by the listener. For example, "HTTP". To get a list of valid protocols, use the ListProtocols operation.

For example:

apiVersion: v1
kind: Service
metadata:
  name: my-nginx-svc
  labels:
    app: nginx
  annotations:
    service.beta.kubernetes.io/oci-load-balancer-backend-protocol: "HTTP"
spec:
  type: LoadBalancer
  ports:
  - port: 80
  selector:
    app: nginx

Note that if you don't explicitly specify a protocol, "TCP" is used as the default value.

An Oracle Cloud Infrastructure load balancer applies a health check policy to continuously monitor backend servers. A health check is a test to confirm backend server availability, and can be a request or a connection attempt. If a server fails the health check, the load balancer takes the server temporarily out of rotation. If the server subsequently passes the health check, the load balancer returns it to the rotation.

Health check policies include a number of parameters, which have default values. When you create a load balancer, you can override health check parameter default values by including annotations in the metadata section of the manifest file. Having created a load balancer, you can later add, modify, and delete those annotations. If you delete an annotation that specified a value for a health check parameter, the load balancer uses the parameter's default value instead.

To specify how many unsuccessful health check requests to attempt before a backend server is considered unhealthy, add the following annotation in the metadata section of the manifest file:

service.beta.kubernetes.io/oci-load-balancer-health-check-retries: <value>

where <value> is the number of unsuccessful health check requests.

To specify the interval between health check requests, add the following annotation in the metadata section of the manifest file:

service.beta.kubernetes.io/oci-load-balancer-health-check-interval: <value>

where <value> is a numeric value in milliseconds. The minimum is 1000.

To specify the maximum time to wait for a response to a health check request, add the following annotation in the metadata section of the manifest file:

service.beta.kubernetes.io/oci-load-balancer-health-check-timeout: <value>

where <value> is a numeric value in milliseconds. A health check is successful only if the load balancer receives a response within this timeout period.

For example:

apiVersion: v1
kind: Service
metadata:
  name: my-nginx-svc
  labels:
    app: nginx
  annotations:
    service.beta.kubernetes.io/oci-load-balancer-health-check-retries: 5
    service.beta.kubernetes.io/oci-load-balancer-health-check-interval: 15000
    service.beta.kubernetes.io/oci-load-balancer-health-check-timeout: 4000
spec:
  type: LoadBalancer
  ports:
  - port: 80
  selector:
    app: nginx

Note that if you don't explicitly specify health check parameter values by including annotations in the metadata section of the manifest file, the following defaults are used:

For more information about Oracle Cloud Infrastructure load balancer health check policies, see Health Check Management.

You can exclude particular worker nodes from the list of backend servers in an Oracle Cloud Infrastructure load balancer backend set. For more information, see node.kubernetes.io/exclude-from-external-load-balancers.