Configuring the OCI Native Ingress Controller

Find out how to configure and customize the OCI native ingress controller to load balance and route incoming traffic to service pods running on worker nodes in a Kubernetes cluster.

When you have installed the OCI native ingress controller and created the necessary Kubernetes ingress-related resources to use it, you can configure the OCI native ingress controller by:

Specifying Route Rules for the OCI Native Ingress Controller

To specify how the OCI load balancer created by the OCI native ingress controller routes incoming requests, you specify route rules in the Ingress manifest.

Route requests based on host

You can configure the OCI native ingress controller to route an incoming request based on the domain name in the request's Host header (the host to which the request was originally sent).

To route a request to a particular backend service and port based on the host, create a route rule in the Ingress manifest. If the host matches the route rule, the OCI native ingress controller routes the request to the associated backend service and port.

For example, you might define the following rule to route requests originally sent to http://foo.bar.com to a backend service named ServiceA on port 80. All incoming traffic originally sent to http://foo.bar.com is routed to ServiceA on port 80.

kind: Ingress
...
spec:
  rules:
  - host: "foo.bar.com"
    http:
      paths:
        - pathType: Prefix
          path: /
          backend:
            serviceName: ServiceA
            servicePort: 80

Route requests to different backend services based on path

You can configure the OCI native ingress controller to route incoming requests to different backend services, based on elements in the path to which the requests were originally sent.

To route a request to a particular backend service and port based on the path, create a route rule in the Ingress manifest. If the path matches the route rule, the OCI native ingress controller routes the request to the associated backend service and port. You can specify multiple paths in the same rule, to route requests to different backends.

For example, you might define the following rule to route requests based on the path to which the request was originally sent:

  • If the path starts with /app1, the OCI native ingress controller routes the request to a backend service named ServiceA on port 80.
  • If the path starts with /app2, the OCI native ingress controller routes the request to a backend service named ServiceB on port 443.

Since the rule does not specify a host, the rule applies to all incoming traffic.

kind: Ingress
...
spec:
  rules:
    - http:
      paths:
        - pathType: Prefix
          path: /app1
          backend:
            serviceName: ServiceA
            servicePort: 80
        - pathType: Prefix
          path: /app2
          backend:
            serviceName: ServiceB
            servicePort: 443

Route requests based on host and path

You can configure the OCI native ingress controller to route an incoming request based on both the domain name in the request's Host header (the host to which the request was originally sent) and elements in the path to which the original request was sent.

To route a request to a particular backend service and port based on the host and path, create a route rule in the Ingress manifest. If the host and path match the route rule, the OCI native ingress controller routes the request to the associated backend service and port.

For example, you might define the following rule to route requests originally sent to http://foo.bar.com/app1 to a backend service named foo on port 80:

kind: Ingress
...
spec:
  rules:
  - host: "foo.bar.com"
    http:
      paths:
        - pathType: Prefix
          path: /app1
          backend:
            serviceName: foo
            servicePort: 80

Route requests to a default backend

You can configure the OCI native ingress controller to route incoming requests to a default backend. You might configure a default backend to handle requests that do not match any route rules.

For example, you might define the following defaultBackend to route requests that do not match other rules in the Ingress manifest to a backend service named ServiceC on port 8080.

Note that if you do not specify any other rules in an Ingress manifest, you must specify a defaultBackend.

kind: Ingress
...
spec:
  rules:
    - http:
      paths:
        - pathType: Prefix
          path: /app1
          backend:
            serviceName: ServiceA
            servicePort: 80
        - pathType: Prefix
          path: /app2
          backend:
            serviceName: ServiceB
            servicePort: 443
  defaultBackend:
    service:
      name: ServiceC
      port:
        number: 8080

Customizing OCI Native Ingress Controller Behavior Using Annotations

You can add annotations to the IngressClass or the Ingress resource manifests to customize the behavior of the OCI native ingress controller.

Customizing general behavior using annotations

You can add annotations to the IngressClass or the Ingress resource manifests to customize general behavior of the OCI native ingress controller.

Annotation Description Add annotation to this resource manifest Example
oci-native-ingress.oraclecloud.com/id OCID of an existing OCI load balancer to use, rather than creating a new one.

Note that if you do specify an existing load balancer, its properties are updated as necessary to align with values in the IngressClassParameters and Ingress resource manifests.

IngressClass oci-native-ingress.oraclecloud.com/id: ocid1.loadbalancer.oc1.iad.aaaaaaaan___u7a
oci-native-ingress.oraclecloud.com/protocol Protocol to use for listener on the load balancer. (Note that if you specify HTTP2 as the protocol, a TLS-configured listener is required.) Ingress oci-native-ingress.oraclecloud.com/protocol: "HTTP2"
oci-native-ingress.oraclecloud.com/policy Policy to be used by the load balancer backend set for traffic distribution. Ingress oci-native-ingress.oraclecloud.com/policy: "ROUND_ROBIN"
oci-native-ingress.oraclecloud.com/waf-policy-ocid OCID of an existing web application firewall (WAF) policy. See Web Application Firewall Policies. Ingress oci-native-ingress.oraclecloud.com/waf-policy-ocid: ocid1.webappfirewallpolicy.oc1.iad.ama___aqq

Customizing health check behavior using annotations

You can add annotations to the Ingress resource manifest to customize the health checks performed by the load balancer created by the OCI native ingress controller. For more information about load balancer health checks, see Health Checks for Load Balancers.

Annotation Description Add annotation to this resource manifest Example
oci-native-ingress.oraclecloud.com/healthcheck-protocol Protocol to use for the load balancer backend set health checks. Ingress oci-native-ingress.oraclecloud.com/healthcheck-protocol: "HTTP"
oci-native-ingress.oraclecloud.com/healthcheck-port Port to use for the load balancer backend set health checks. Ingress oci-native-ingress.oraclecloud.com/healthcheck-port: "80"
oci-native-ingress.oraclecloud.com/healthcheck-path Path to use for the load balancer backend set health checks. Ingress oci-native-ingress.oraclecloud.com/healthcheck-path: "/test"
oci-native-ingress.oraclecloud.com/healthcheck-interval-milliseconds Interval between the load balancer backend set health checks. Ingress oci-native-ingress.oraclecloud.com/healthcheck-interval-milliseconds: "1000"
oci-native-ingress.oraclecloud.com/healthcheck-timeout-milliseconds Period of time after which the load balancer backend set health check is considered to have failed. Ingress oci-native-ingress.oraclecloud.com/healthcheck-timeout-milliseconds: "750"
oci-native-ingress.oraclecloud.com/healthcheck-retries Number of retries after which the load balancer backend set health check is considered to have failed. Ingress oci-native-ingress.oraclecloud.com/healthcheck-retries: "5"
oci-native-ingress.oraclecloud.com/healthcheck-return-code Status code the load balancer backend set must return in response to a health check to be considered healthy. Ingress oci-native-ingress.oraclecloud.com/healthcheck-return-code: "200"
oci-native-ingress.oraclecloud.com/healthcheck-response-regex Regular expression for parsing the response body from the load balancer backend set. You can specify any regex value (such as * or / ), or an empty value. Ingress oci-native-ingress.oraclecloud.com/healthcheck-response-regex: "*"
oci-native-ingress.oraclecloud.com/healthcheck-force-plaintext Whether to send a health check to the load balancer backend without SSL (HTTP only). If not specified, false is the default. Ingress oci-native-ingress.oraclecloud.com/healthcheck-force-plaintext: "true"

Setting Up a Pod Readiness Gate

Pod readiness gates are additional conditions included in a pod manifest to indicate that a pod is ready to receive traffic. Pod readiness gates enable you to implement complex custom readiness checks, and can help to achieve zero downtime during rolling deployments. For more information, see pod readiness details in the Kubernetes documentation.

When using the OCI native ingress controller with a cluster that has VCN-native pod networking as the network type, you can specify that the OCI native ingress controller is to inject a pod readiness gate into the pod spec of every pod created in a particular namespace. Note that you cannot use the OCI native ingress controller to inject pod readiness gates into pod specs if the cluster has Flannel overlay as the network type.

Specify that the OCI native ingress controller is to inject a pod readiness gate into the pod spec of every pod created in a particular namespace by entering:

kubectl label ns <namespace> podreadiness.ingress.oraclecloud.com/pod-readiness-gate-inject=enabled

The OCI native ingress controller injects a condition into the pod spec of every pod created in the namespace. For example:


kind: Pod
...
    spec:
      readinessGates:
      - conditionType: backend-health.lb.ingress.k8s.oci/ServiceA_80
You can verify the status of pod readiness gates by entering:
kubectl get pods -o wide -w
Example output:
NAME                                             READY   STATUS    RESTARTS   AGE   IP            NODE          NOMINATED NODE   READINESS GATES
testecho-7cdcfff87f-b6xt4                        1/1     Running   0          35s   10.0.10.242   10.0.10.135   <none>           0/1
testecho-7cdcfff87f-b6xt4                        1/1     Running   0          72s   10.0.10.242   10.0.10.135   <none>           1/1

Adding Support for HTTPS/TLS Requests

You can use the OCI native ingress controller to support secure HTTPS communication. Using the OCI native ingress controller, you can set up OCI load balancer listeners and backend sets to handle traffic encrypted using TLS (formerly SSL).

When using the OCI native ingress controller to support HTTPS communication, you have two options:

When handling HTTPS traffic, the OCI load balancer created by the OCI native ingress controller implements end-to-end TLS. The load balancer uses certificates to accept a TLS encrypted request from a client, and then uses routing rules to forward the request to the appropriate backend set. The backend set creates a new TLS connection with the service pods running on the cluster (using the CA bundle as the trust authority for the new connection).

Note the following:

  • If you delete an IngressClass resource, the OCI native ingress controller deletes the load balancer it created (or deletes the existing load balancer specified by the oci-native-ingress.oraclecloud.com/id annotation). However, note that the ingress controller does not delete resources created in the OCI Certificates service. You are responsible for deleting any such Certificates service resources. In particular, if you specified a Kubernetes secret in the Ingress resource manifest, note that you are responsible for deleting any Certificates service resources that the OCI native ingress controller has created for you.
  • Certificates obtained from the OCI Certificates service of type Imported cannot be automatically rotated. If you want certificates rotated automatically, manually obtain a certificate in the OCI Certificates service yourself by specifying that you want the certificate to be issued by a Certificates service internal CA (the certificates are of type Issued by internal CA). You can configure certificates of type Issued by internal CA to be automatically rotated.
  • If you specify a Kubernetes secret in the Ingress resource manifest, note that certificates the OCI native ingress controller obtains from the Certificates service are of type Imported and therefore not rotated automatically. To manually rotate such Certificates service certificates, first create a new Kubernetes secret using a new TLS public and private key pair and a new certificate. Then, update the Ingress resource with the name of the new secret. The OCI native ingress controller obtains a new certificate and a CA bundle from the Certificates service and associates them with the listener and the backend set (replacing the previous certificate and CA bundle).
  • A listener can only be associated with one certificate. Therefore, do not create multiple ingress resources that each have rules specifying the same backend service/port combination, but where the ingress resources use different certificates.

Option 1: OCI Native Ingress Controller obtains certificate from the Certificates service using a Kubernetes secret

To configure the OCI native ingress controller to obtain a certificate from the OCI Certificates service:

  1. Obtain a TLS public and private key pair, and a certificate.

    In production environments, you obtain a TLS certificate from your chosen Certificate Authority by submitting a certificate signing request. During the certificate request process, a public key and corresponding private key are generated.

    In development and test environments, you can create a self-signed certificate and generate a private key yourself using a tool such as OpenSSL. For example (using OpenSSL 3.0 or later):

    1. Generate a public and private key pair by entering the following commands:

      openssl genrsa -out rootCA.key 4096
      openssl req -x509 -addext basicConstraints=critical,CA:TRUE -new -nodes -key rootCA.key -sha256 -days 1024 -out rootCA.crt -subj /CN=RootCA
    2. Generate a certificate by entering the following commands:

      openssl genrsa -out server.key 4096
      openssl req -new -sha256 -key server.key -subj /C=US/ST=CA/O=MyOrg,Inc./CN=my.example.com -out server.csr
      openssl x509 -req -in server.csr -CA rootCA.crt -CAkey rootCA.key -CAcreateserial -out server.crt -days 500 -sha256

    In this example:

    • rootCA.key contains the key pair for the root CA.
    • rootCA.crt contains the root CA certificate.
    • server.key contains the key pair for generating the server certificate.
    • server.csr contains the certificate signing request for the server certificate.
    • server.crt contains the generated server certificate.
  2. Create a Kubernetes secret resource in either of the following ways:
    • Using the kubectl create secret generic command, by entering:

      kubectl create secret generic <k8s-secret-name> --type=kubernetes.io/tls --from-file=ca.crt=<path-and-filename>.crt --from-file=tls.crt=<path-and-filename>.crt --from-file=tls.key=<path-and-filename>.key

      where:

      • <k8s-secret-name> is your choice of name for the Kubernetes secret
      • --from-file=ca.crt=<path-and-filename>.crt specifies the path to the file containing the root CA certificate. For example, --from-file=ca.crt=rootCA.crt
      • --from-file=tls.crt=<path-and-filename>.crt specifies the path to the file containing the generated server certificate. For example, --from-file=tls.crt=server.crt
      • --from-file=tls.key=<path-and-filename>.key specifies the path to the file containing the generated private key. For example, --from-file=tls.key=server.key

      For example:

      kubectl create secret generic example-tls-secret --type=kubernetes.io/tls --from-file=ca.crt=rootCA.crt --from-file=tls.crt=server.crt --from-file=tls.key=server.key
    • Using a Secret resource manifest file:
      1. Define the Kubernetes secret in a .yaml file, in the format:

        apiVersion: v1
        kind: Secret
        metadata:
          name: <k8s-secret-name>
        type: kubernetes.io/tls
        data:
          ca.crt: <base64-encoded-certificate-chain>
          tls.crt: <base64-encoded-server-certificate>
          tls.key: <base64-encoded-private-key>

        where:

        • name: <k8s-secret-name> is your choice of name for the Kubernetes secret resource.
        • ca.crt: <base64-encoded-certificate-chain> is the contents of the file (or files) containing intermediate certificates that form a certificate chain from the leaf certificate back to the Certificate Authority.
        • tls.crt: <base64-encoded-server-certificate> is the contents of the file containing the generated server certificate.
        • tls.key: <base64-encoded-private-key> is the contents of the file containing the generated private key.

        For example:

        apiVersion: v1
        kind: Secret
        metadata:
          name: example-tls-secret
        type: kubernetes.io/tls
        data:
          ca.crt : MIIFGERTegcDFRTuSDGfghREdE______Jre
          tls.crt: MIIC2DCCAcCgAwIBAgIBATANBg______kqh
          tls.key: MIIEpgIBAAKCAQEA7yn3bRHQ5F______HMQ
      2. Create the secret resource by entering kubectl create -f <filename>.yaml

  3. Add the Kubernetes secret to an Ingress resource manifest:
    1. Define a new ingress resource in a .yaml file. See Create Ingress resource.
    2. In the spec: section of the manifest, add a tls: element that specifies both the host that is to receive HTTPS traffic and the name of the Kubernetes secret, in the format:
      kind: Ingress
      ...
      spec:
        tls:
        - hosts:
            - <host-name>
          secretName: <k8s-secret-name>

      For example:

      kind: Ingress
      ...
      spec:
        tls:
        - hosts:
            - my.example.com
          secretName: example-tls-secret
    3. In the rules: section of the manifest, add a rule for the host that is to receive HTTPS traffic, and specify 443 as the port on which to listen for HTTPS traffic.

      For example:

      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        name: acme-tls-secret-ingress
      spec:
        tls:
        - hosts:
            - my.example.com
          secretName: example-tls-secret
        rules:
        - host: "my.example.com"
          http:
            paths:
            - pathType: Prefix
              path: "/TLSPath"
              backend:
                service:
                  name: tls-test
                  port:
                    number: 443
    4. Create the resource by entering kubectl create -f <filename>.yaml

When you create the ingress resource, the OCI native ingress controller uses the Kubernetes secret to obtain a certificate of type Imported, and a CA bundle (Certificate Authority bundle), from the OCI Certificates service. The OCI native ingress controller associates the certificate with the listener, and the CA bundle with the backend set.

When the listener listening on port 443 receives an HTTPS request to the specified host, the listener uses the certificate for TLS termination. The listener then uses the routing rule to forward the request to the backend set. The backend set creates a new TLS connection with the service pods running on the cluster (using the CA bundle as the trust authority for the new connection).

Option 2: You obtain certificate from the Certificates service

To configure the OCI native ingress controller to use a certificate that you have obtained from the OCI Certificates service:

  1. Create a certificate in the OCI Certificates service in one of the following ways:
    • by importing a certificate issued by a third-party CA (the certificate will be of type Imported)
    • by issuing the certificate internally from a Certificates service CA (the certificate will be of type Issued by internal CA)

    Do not create a certificate to manage externally (of type Issued by internal CA, managed externally). For more information, see Creating a Certificate.

  2. Make a note of the certificate's OCID.
  3. Add the certificate's OCID to an Ingress resource manifest:
    1. Define a new ingress resource in a .yaml file. See Create Ingress resource.
    2. In the metadata: section, add an annotations: element that specifies the OCID of the certificate you created in the OCI Certificates service, in the format:
      kind: Ingress
      metadata:
        name: <i-name>
        annotations:
          oci-native-ingress.oraclecloud.com/certificate-ocid: <certificate-ocid>
      spec:
      ...

      where:

      • name: <i-name> is your choice of name for the ingress resource.
      • oci-native-ingress.oraclecloud.com/certificate-ocid: <certificate-ocid> is the OCID of the certificate you created in the OCI Certificates service

      For example:

      kind: Ingress
      metadata:
        name: acme-tls-certificate-ingress
        annotations:
          oci-native-ingress.oraclecloud.com/certificate-ocid: ocid1.certificate.oc1.iad.amaaaaaa______gabc
      spec:
      ...
    3. In the rules: section of the Ingress resource manifest, add a rule for the host that is to receive HTTPS traffic, and specify 443 as the port on which to listen for HTTPS traffic.

      For example:

      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        name: acme-tls-certificate-ingress
        annotations:
          oci-native-ingress.oraclecloud.com/certificate-ocid: ocid1.certificate.oc1.iad.amaaaaaa______gabc
      spec:
        rules:
        - host: "my.example.com"
          http:
            paths:
            - pathType: Prefix
              path: "/TLSPath"
              backend:
                service:
                  name: tls-test
                  port:
                    number: 443
    4. Create the resource by entering kubectl create -f <filename>.yaml

What happens when you create the ingress resource depends on how you created the certificate in the OCI Certificates service:

  • If you created the certificate by importing a certificate issued by a third-party CA, the certificate is of type Imported. The OCI native ingress controller associates the certificate with the listener, creates a CA bundle from the certificate chain, and associates the CA bundle with the backend set. Note that you cannot configure certificates of type Imported to be automatically rotated.
  • If you created the certificate by issuing the certificate internally from a Certificates service CA, the certificate is of type Issued by internal CA. The OCI native ingress controller associates the certificate with the listener, obtains the OCID of the CA, and associates that OCID with the backend set. Note that you can configure certificates of type Issued by internal CA to be automatically rotated.

When the listener listening on port 443 receives an HTTPS request to the specified host, the listener uses the certificate for TLS termination. The listener then uses the routing rule to forward the request to the backend set. The backend set creates a new TLS connection with the service pods running on the cluster (using the CA bundle, or the CA identified by its OCID, as the trust authority for the new connection).