Deploy a Kubernetes Cluster with Automatic DNS Management for Multiple Environments
Managing the cloud DNS strategy for your internal and external environments (development, test, and production for example) can become complicated and time consuming without an automated solution. Ideally, both your users and customers should be able to navigate to a domain, and your DNS solution should direct to the correct environment's IP address. This ideal scenario is user friendly and can prevent human error.
Kubernetes ExternalDNS offers capabilities to automate a DNS solution to make it easy for users to navigate to the domain they want, including advanced features such as global load balancing, automatic failover, DNS-based traffic management, and built-in security features such as Domain Name System Security Extensions (DNSSEC) to help protect against tampering and spoofing attacks.
This reference architecture uses a single kubernetes cluster deployment with Oracle Cloud Infrastructure DNS Zones and Kubernetes ExternalDNS to configure a DNS solution designed for multiple environments.
Architecture
This architecture uses an Oracle Cloud Infrastructure Container Engine for Kubernetes (OCI OKE) cluster deployed in an OCI DNS zone to hold the "A records" for your exposed services through load balancers. In this architecture, the OKE cluster has three worker nodes, and three different namespaces for deployment segregation. Use this architecture for a multi-environment OKE deployment that uses a single cluster instead of multiple.
The following diagram illustrates this reference architecture.
multi-tenant-oke-dns-architecture.zip
Each environment (development, test, and production) has its own set of kubernetes components such as namespaces, pods, and services. Kubernetes ExternalDNS allows the automatic creation and management of DNS records for kubernetes services exposed externally. This makes it easier for external clients to access the services inside the cluster. The kubernetes components route to Kubernetes ExternalDNS, which then creates "A records" pointing to each service load balancer in OCI.
The following diagram illustrates this software architecture.
The architecture has the following components:
- Region
An Oracle Cloud Infrastructure region is a localized geographic area that contains one or more data centers, called availability domains. Regions are independent of other regions, and vast distances can separate them (across countries or even continents).
- Availability domains
Availability domains are standalone, independent data centers within a region. The physical resources in each availability domain are isolated from the resources in the other availability domains, which provides fault tolerance. Availability domains don’t share infrastructure such as power or cooling, or the internal availability domain network. So, a failure at one availability domain is unlikely to affect the other availability domains in the region.
- Fault domains
A fault domain is a grouping of hardware and infrastructure within an availability domain. Each availability domain has three fault domains with independent power and hardware. When you distribute resources across multiple fault domains, your applications can tolerate physical server failure, system maintenance, and power failures inside a fault domain.
- Virtual cloud network (VCN) and subnets
A VCN is a customizable, software-defined network that you set up in an Oracle Cloud Infrastructure region. Like traditional data center networks, VCNs give you complete control over your network environment. A VCN can have multiple non-overlapping CIDR blocks that you can change after you create the VCN. You can segment a VCN into subnets, which can be scoped to a region or to an availability domain. Each subnet consists of a contiguous range of addresses that don't overlap with the other subnets in the VCN. You can change the size of a subnet after creation. A subnet can be public or private.
- Load balancer
The Oracle Cloud Infrastructure Load Balancing service provides automated traffic distribution from a single entry point to multiple servers in the back end.
- Security list
For each subnet, you can create security rules that specify the source, destination, and type of traffic that must be allowed in and out of the subnet.
- Container Engine for Kubernetes
Oracle Cloud Infrastructure Container Engine for Kubernetes is a fully managed, scalable, and highly available service that you can use to deploy your containerized applications to the cloud. You specify the compute resources that your applications require, and Container Engine for Kubernetes provisions them on Oracle Cloud Infrastructure in an existing tenancy. Container Engine for Kubernetes uses Kubernetes to automate the deployment, scaling, and management of containerized applications across clusters of hosts.
- DNS
Oracle Cloud Infrastructure Domain Name System (DNS) service is a highly scalable, global anycast domain name system (DNS) network that offers enhanced DNS performance, resiliency, and scalability, so that end users connect to customers’ application as quickly as possible, from wherever they are.
- Public/Private DNS Zone
The Oracle Cloud Infrastructure DNS service lets you manage zones. A zone is a portion of the DNS namespace. A Start of Authority record (SOA) defines a zone. A zone contains all labels underneath itself in the tree, unless otherwise specified.
Recommendations
- Virtual cloud network (VCN)
When you create a VCN, determine the number of CIDR blocks required and the size of each block based on the number of resources that you plan to attach to subnets in the VCN. Use CIDR blocks that are within the standard private IP address space.
When you design the subnets, consider your traffic flow and security requirements. Attach all the resources within a specific tier or role to the same subnet, which can serve as a security boundary.
- Load balancer bandwidth
While creating the load balancer, you can either select a predefined shape that provides a fixed bandwidth, or specify a custom (flexible) shape where you set a bandwidth range and let the service scale the bandwidth automatically based on traffic patterns. With either approach, you can change the shape at any time after creating the load balancer.
Considerations
When deploying multiple environments inside a single Oracle Cloud Infrastructure Container Engine for Kubernetes cluster, consider the following.
- Security
Consider using Kubernetes Network Policies to segregate the namespaces connectivity and rely on role-based access control to control access to the namespaces' management.