Creating Flannel Overlay Network Resources

The Flannel Overlay network type encapsulates communication between pods in the Flannel Overlay network. The Flannel Overlay network is a simple private overlay virtual network that satisfies the requirements of the OKE networking model by attaching IP addresses to containers. The pods in the private overlay network are only accessible from other pods in the same cluster.

The resource definitions in the following sections in this topic create a working example set of network resources for workload clusters when you are using Flannel Overlay networking. Use this configuration as a guide when you create these resources. You can change the values of properties such as CIDR blocks and IP addresses. You should not change the values of properties such as the network protocol, the stateful setting, or the private/public setting.

See Workload Cluster Network Ports for Flannel Overlay Networking for specific ports that must be open for specific purposes.

Create the following network resources. To use Terraform, see Example Terraform Scripts for Flannel Overlay Network Resources.

Note:

Create all of these network resources in the same compartment on the appliance.

• VCN. See Creating a Flannel Overlay VCN.

• Internet Gateway

• NAT Gateway

• Dynamic Routing Gateway

• Local Peering Gateway

• Route rules

• Security lists

• The following four subnets:

  • Worker. See Creating a Flannel Overlay Worker Subnet.

  • Worker load balancer. See Creating a Flannel Overlay Worker Load Balancer Subnet.

  • Control plane. See Creating a Flannel Overlay Control Plane Subnet.

  • Control plane load balancer. See Creating a Flannel Overlay Control Plane Load Balancer Subnet.

Workload Cluster Network CIDR Ranges for Flannel Overlay Networking

Throughout this documentation, variables are used to represent CIDR ranges for instances in different subnets. The following table lists the CIDR variables and example values for use with Flannel Overlay networking.

Note:

These are examples only. The CIDR ranges you use depend on the number of clusters you have, the number of nodes in each cluster, and the type of networking you are using.

For Flannel Overlay networking, IP addresses are managed by the underlying Container service. Pods are not assigned IP addresses from the IP address pool that is defined in the pod subnet CIDR. This is the reason you do not need a pod subnet when you are using Flannel Overlay networking.

The primary difference between IP address requirements of Flannel Overlay networking and VCN-Native Pod Networking is that VCN-Native Pod Networking requires more IP addresses to be available. The table in Workload Cluster Network CIDR Ranges for VCN-Native Pod Networking shows larger CIDR ranges than the following table for Flannel Overlay CIDR ranges. The CIDR ranges used with Flannel Overlay networking can be much smaller than the CIDR ranges used with VCN-Native Pod Networking.

Table 4-1 Example CIDR Values to Use with Flannel Overlay Networking

Variable Name	Description	Example Value
vcn_cidr	VCN CIDR range	172.31.252.0/23
worker_cidr	Worker subnet CIDR	172.31.253.0/24
workerlb_cidr	Worker load balancer subnet CIDR	172.31.252.0/25
kmi_cidr	OKE control plane subnet CIDR	172.31.252.224/28
kmilb_cidr	OKE control plane load balancer subnet CIDR	172.31.252.240/28
kube_client_cidr	CIDR for clients that are allowed to contact the Kubernetes API server	10.0.0.0/8
public_ip_cidr	Public IP CIDR configured in the Private Cloud Appliance Service Enclave	10.0.0.0/8
kube_internal_cidr	CIDR used by the Kubernetes infrastructure to allocate IP addresses for various internal services and components	253.255.0.0/16

The IP Subnet Calculator on Calculator.net is one tool for finding all available networks for a given IP address and prefix length.

Workload Cluster Network Ports for Flannel Overlay Networking

The following table lists ports that are used by workload clusters when you use Flannel Overlay networking. These ports must be available to configure workload cluster networking. You might need to open additional ports for other purposes.

All protocols are TCP. All port states are Stateful. Port 6443 is the port used for Kubernetes API and is also known as kubernetes_api_port in this guide.

See also the tables in Port Matrix in the Oracle Private Cloud Appliance Security Guide.

If you are using a separate administration network, see OKE Cluster Management with Administration Network.

Table 4-2 Ports that Must Be Available for Use by Workload Clusters for Flannel Overlay Networking

Source IP Address	Destination IP Address	Port	Description
bastion host: vcn_cidr	Worker nodes subnet: worker_cidr	22	Outbound connections from the bastion host to the worker CIDR.
bastion host: vcn_cidr	Control plane subnet: kmi_cidr	22	Outbound connections from the bastion host to the control plane nodes.
Worker nodes subnet: worker_cidr	yum repository	80	Outbound connections from the worker CIDR to external applications.
Worker nodes subnet: worker_cidr	Secure yum repository	443	Secure outbound traffic from the worker CIDR to external applications.
Worker nodes subnet: worker_cidr	Container registry	5000	Outbound connections from the worker CIDR to the container registry.
Worker nodes subnet: worker_cidr	Control plane subnet: kmi_cidr	6443	Outbound connections from the worker CIDR to the Kubernetes API. This is necessary to allow nodes to join through either a public IP address on one of the nodes or the load balancer public IP address. Port 6443 is called the kubernetes_api_port.
Worker nodes subnet: worker_cidr	Control plane load balancer	6443	Inbound connections from the worker CIDR to the Kubernetes API.
CIDR for clients: kube_client_cidr	Control plane load balancer	6443	Inbound connections from clients to the Kubernetes API server.
Worker nodes subnet: worker_cidr	Control plane subnet: kmi_cidr	6443	Private outbound connections from the worker CIDR to kubeapi on the control plane subnet.
kube_client_cidr	Worker nodes subnet: worker_cidr	30000-32767	Inbound traffic for applications from Kubernetes clients.

Example Terraform Scripts for Flannel Overlay Network Resources

The following Terraform scripts create the network resources that are required by OKE when you are using Flannel Overlay networking. Subsequent sections in this topic show other ways to define these same network resources.

Most of the values shown in these scripts, such as resource display names and CIDRs, are examples. Some ports must be specified as shown (see Workload Cluster Network Ports for Flannel Overlay Networking), and the OKE control plane subnet must be named control-plane. See Workload Cluster Network CIDR Ranges for Flannel Overlay Networking for comments about CIDR values.

• variables.tf

• terraform.tfvars

• provider.tf

• main.tf

• oke_vcn.tf

• oke_worker_seclist.tf

• oke_worker_subnet.tf

• oke_kmi_seclist.tf

• oke_kmi_subnet.tf

variables.tf

This file creates several variables that are used to configure OKE network resources when you are using Flannel Overlay networking. Many of these variables are not assigned values in this file. One port and five CIDRs are assigned values. The kubernetes_api_port, port 6443, is the port used to access the Kubernetes API. See also Workload Cluster Network Ports for Flannel Overlay Networking. The five CIDRs that are defined in this file are for the OKE VCN, worker subnet, worker load balancer subnet, control plane subnet, and control plane load balancer subnet.

variable "oci_config_file_profile" {
  type    = string
  default = "DEFAULT"
}

variable "tenancy_ocid" {
  description = "tenancy OCID"
  type        = string
  nullable    = false
}

variable "compartment_id" {
  description = "compartment OCID"
  type        = string
  nullable    = false
}

variable "vcn_name" {
  description = "VCN name"
  nullable    = false
}

variable "kube_client_cidr" {
  description = "CIDR of Kubernetes API clients"
  type        = string
  nullable    = false
}

variable "public_ip_cidr" {
  description = "Public IP CIDR configured in the Service Enclave"
  type        = string
  nullable    = false
}

variable "kubernetes_api_port" {
  description = "Port used for Kubernetes API"
  type        = string
  default     = "6443"
}

variable "vcn_cidr" {
  default = "172.31.252.0/23"
}

# Subnet for KMIs where kube-apiserver and other control
# plane applications run, max 9 nodes
variable "kmi_cidr" {
  description = " Kubernetes control plane subnet CIDR"
  default     = "172.31.252.224/28"
}

# Subnet for KMI load balancer 
variable "kmilb_cidr" {
  description = "Kubernetes control plane LB subnet CIDR"
  default     = "172.31.252.240/28"
}

# Subnet for worker nodes, max 128 nodes
variable "worker_cidr" {
  description = "Kubernetes worker subnet CIDR"
  default     = "172.31.253.0/24"
}

# Subnet for worker load balancer (for use by CCM)
variable "workerlb_cidr" {
  description = "Kubernetes worker LB subnet CIDR"
  default     = "172.31.252.0/25"
}

# Flag to Enable private endpoint
variable "enable_private_endpoint" {
  description = "Flag to create private control plane endpoint/service-lb"
  type = bool
  default = false
  nullable = false
}

terraform.tfvars

This file assigns values to some of the variables that were created in variables.tf.

# name of the profile to use from $HOME/.oci/config
oci_config_file_profile = "DEFAULT"

# tenancy ocid from the above profile
tenancy_ocid = "tenancy_OCID"

# compartment in which to build the OKE cluster
compartment_id = "compartment_OCID"

# display-name for the OKE VCN
vcn_name = "oketest"

provider.tf

This file is required in order to use the OCI provider. The file initializes the OCI module using the OCI profile configuration file.

provider "oci" {
  config_file_profile = var.oci_config_file_profile
  tenancy_ocid        = var.tenancy_ocid
}

main.tf

This file specifies the provider to use (oracle/oci), defines several security list rules, and initializes required local variables.

The version of the OCI provider that you use must be at least v4.50.0 but no greater than v6.36.0.

terraform {
  required_providers {
    oci = {
      source  = "oracle/oci"
      version = ">= 4.50.0, <= 6.36.0"
      # If necessary, you can pin a specific version here
      # version = "4.71.0"
    }
  }
  required_version = ">= 1.1"
}

locals {
  kube_internal_cidr = "253.255.0.0/16"
  worker_lb_ingress_rules = [
    {
      source   = var.kube_client_cidr
      port_min = 80
      port_max = 80
    },
    {
      source   = var.kube_client_cidr
      port_min = 443
      port_max = 443
    }
  ]
  worker_ingress_rules = [
     {
      source   = var.kube_client_cidr
      port_min = 30000
      port_max = 32767
    },
    {
      source   = var.vcn_cidr
      port_min = 22
      port_max = 22
    },
    {
      source   = var.workerlb_cidr
      port_min = 30000
      port_max = 32767
    },
    {
      source   = var.workerlb_cidr
      port_min = 10256
      port_max = 10256
    },
    {
      source   = var.kmi_cidr
      port_min = 22
      port_max = 65535
    },
  ]

  kmi_lb_ingress_rules = [
    {
      source   = local.kube_internal_cidr
      port_min = var.kubernetes_api_port
      port_max = var.kubernetes_api_port
    },
    {
      source   = var.kube_client_cidr
      port_min = var.kubernetes_api_port
      port_max = var.kubernetes_api_port
    },
    {
      source   = var.vcn_cidr
      port_min = var.kubernetes_api_port
      port_max = var.kubernetes_api_port
    },
  ]
  kmi_ingress_rules = [
    {
      source   = var.kube_client_cidr
      port_min = var.kubernetes_api_port
      port_max = var.kubernetes_api_port
    },
    {
      source   = var.kmilb_cidr
      port_min = var.kubernetes_api_port
      port_max = var.kubernetes_api_port
    },
    {
      source   = var.worker_cidr
      port_min = 1024
      port_max = 65535
    },
    {
      source   = var.kmi_cidr
      port_min = 1024
      port_max = 65535
    },
  ]
  worker_ingress_udp_rules = [
    {
      source   = var.worker_cidr
      port_min = 8285
      port_max = 8472
    },
    {
      source   = var.kmi_cidr
      port_min = 8285
      port_max = 8472
    },
  ]
  kmi_ingress_udp_rules = [
    {
      source   = var.worker_cidr
      port_min = 8285
      port_max = 8472
    },
    {
      source   = var.kmi_cidr
      port_min = 8285
      port_max = 8472
    },
  ]
}

oke_vcn.tf

This file defines a VCN, NAT gateway, internet gateway, private route table, and public route table. The private route table is the default route table for the VCN.

resource "oci_core_vcn" "oke_vcn" {
  cidr_block     = var.vcn_cidr
  dns_label      = var.vcn_name
  compartment_id = var.compartment_id
  display_name   = "${var.vcn_name}-vcn"
}

resource "oci_core_nat_gateway" "vcn_ngs" {
  compartment_id = var.compartment_id
  vcn_id         = oci_core_vcn.oke_vcn.id
  count          = var.enable_private_endpoint ? 0:1

  display_name = "VCN nat g6s"
}

resource "oci_core_internet_gateway" "vcn_igs" {
  compartment_id = var.compartment_id
  vcn_id         = oci_core_vcn.oke_vcn.id
  count          = var.enable_private_endpoint ? 0:1

  display_name = "VCN i6t g6s"
  enabled      = true
}

resource "oci_core_default_route_table" "default_private" {
  manage_default_resource_id = oci_core_vcn.oke_vcn.default_route_table_id
  display_name               = "Default - private"
  count                      = var.enable_private_endpoint ? 1:0
}

resource "oci_core_default_route_table" "private" {
  count          = var.enable_private_endpoint ? 0:1
  manage_default_resource_id = oci_core_vcn.oke_vcn.default_route_table_id
  display_name               = "Default - private"

  route_rules {
    destination       = "0.0.0.0/0"
    destination_type  = "CIDR_BLOCK"
    network_entity_id = oci_core_nat_gateway.vcn_ngs[0].id
  }
}

resource "oci_core_route_table" "public" {
  count          = var.enable_private_endpoint ? 0:1
  compartment_id = var.compartment_id
  vcn_id         = oci_core_vcn.oke_vcn.id

  display_name = "public"
  route_rules {
    destination       = "0.0.0.0/0"
    destination_type  = "CIDR_BLOCK"
    network_entity_id = oci_core_internet_gateway.vcn_igs[0].id
  }
}

oke_worker_seclist.tf

This file defines the security lists for both the worker subnet and the worker load balancer subnet. The rules for these security lists were defined in other Terraform files in this set.

resource "oci_core_security_list" "workerlb" {
  display_name   = "${var.vcn_name}-workerlb"
  compartment_id = var.compartment_id
  vcn_id         = oci_core_vcn.oke_vcn.id

  dynamic "ingress_security_rules" {
    iterator = port
    for_each = local.worker_lb_ingress_rules

    content {
      source      = port.value.source
      source_type = "CIDR_BLOCK"
      protocol    = "6"
      tcp_options {
        min = port.value.port_min
        max = port.value.port_max
      }
    }
  }
}

resource "oci_core_security_list" "worker" {
  display_name   = "${var.vcn_name}-worker"
  compartment_id = var.compartment_id
  vcn_id         = oci_core_vcn.oke_vcn.id

  dynamic "ingress_security_rules" {
    iterator = port
    for_each = local.worker_ingress_rules

    content {
      source      = port.value.source
      source_type = "CIDR_BLOCK"
      protocol    = "6"
      tcp_options {
        min = port.value.port_min
        max = port.value.port_max
      }
    }
  }

  dynamic "ingress_security_rules" {
    iterator = port
    for_each = local.worker_ingress_udp_rules

    content {
      source      = port.value.source
      source_type = "CIDR_BLOCK"
      protocol    = "17"
      udp_options {
        min = port.value.port_min
        max = port.value.port_max
      }
    }
  }
}

oke_worker_subnet.tf

This file defines the worker and worker load balancer subnets. The worker load balancer subnet is named service-lb.

resource "oci_core_subnet" "worker" {
  cidr_block     = var.worker_cidr
  compartment_id = var.compartment_id
  vcn_id         = oci_core_vcn.oke_vcn.id

  display_name   = "worker"
  dns_label      = "worker"
  prohibit_public_ip_on_vnic = true

  security_list_ids = [
    oci_core_default_security_list.oke_vcn.id,
    oci_core_security_list.worker.id
  ]
}

resource "oci_core_subnet" "worker_lb" {
  cidr_block     = var.workerlb_cidr
  compartment_id = var.compartment_id
  vcn_id         = oci_core_vcn.oke_vcn.id

  display_name               = "service-lb"
  dns_label                  = "servicelb"
  prohibit_public_ip_on_vnic = var.enable_private_endpoint
  route_table_id             = var.enable_private_endpoint==false ? oci_core_route_table.public[0].id : oci_core_vcn.oke_vcn.default_route_table_id

  security_list_ids = [
    oci_core_default_security_list.oke_vcn.id,
    oci_core_security_list.workerlb.id
  ]
}

oke_kmi_seclist.tf

This file defines the security lists for the control plane and control plane load balancer subnets. This file also defines updates to make to the default security list for the VCN.

resource "oci_core_default_security_list" "oke_vcn" {
  manage_default_resource_id = oci_core_vcn.oke_vcn.default_security_list_id

  egress_security_rules {
    destination      = "0.0.0.0/0"
    destination_type = "CIDR_BLOCK"
    protocol         = "all"
  }

  dynamic "ingress_security_rules" {
    iterator = icmp_type
    for_each = [3, 8, 11]

    content {
      # ping from VCN; unreachable/TTL from anywhere
      source      = (icmp_type.value == "8" ? var.vcn_cidr : "0.0.0.0/0")
      source_type = "CIDR_BLOCK"
      protocol    = "1"
      icmp_options {
        type = icmp_type.value
      }
    }
  }
}

resource "oci_core_security_list" "kmilb" {
  compartment_id = var.compartment_id
  vcn_id         = oci_core_vcn.oke_vcn.id

  display_name = "${var.vcn_name}-kmilb"

  dynamic "ingress_security_rules" {
    iterator = port
    for_each = local.kmi_lb_ingress_rules

    content {
      source      = port.value.source
      source_type = "CIDR_BLOCK"
      protocol    = "6"
      tcp_options {
        min = port.value.port_min
        max = port.value.port_max
      }
    }
  }

  dynamic "ingress_security_rules" {
    for_each = var.enable_private_endpoint ? [] : [0]

    content {
      source      = var.public_ip_cidr
      source_type = "CIDR_BLOCK"
      protocol    = "6"
      tcp_options {
        min = var.kubernetes_api_port
        max = var.kubernetes_api_port
      }
    }
  }
}

resource "oci_core_security_list" "kmi" {
  compartment_id = var.compartment_id
  vcn_id         = oci_core_vcn.oke_vcn.id

  display_name = "${var.vcn_name}-kmi"

  dynamic "ingress_security_rules" {
    iterator = port
    for_each = local.kmi_ingress_rules

    content {
      source      = port.value.source
      source_type = "CIDR_BLOCK"
      protocol    = "6"
      tcp_options {
        min = port.value.port_min
        max = port.value.port_max
      }
    }
  }

  dynamic "ingress_security_rules" {
    iterator = port
    for_each = local.kmi_ingress_udp_rules

    content {
      source      = port.value.source
      source_type = "CIDR_BLOCK"
      protocol    = "17"
      udp_options {
        min = port.value.port_min
        max = port.value.port_max
      }
    }
  }
}

oke_kmi_subnet.tf

This file defines the control plane and control plane load balancer subnets.

Important:

The name of the kmi subnet must be exactly control-plane.

resource "oci_core_subnet" "kmi" {
  cidr_block                 = var.kmi_cidr
  compartment_id             = var.compartment_id
  display_name               = "control-plane"
  dns_label                  = "kmi"
  vcn_id                     = oci_core_vcn.oke_vcn.id
  prohibit_public_ip_on_vnic = true
  security_list_ids = [
    oci_core_default_security_list.oke_vcn.id,
    oci_core_security_list.kmi.id
  ]
}

resource "oci_core_subnet" "kmi_lb" {
  cidr_block                 = var.kmilb_cidr
  compartment_id             = var.compartment_id
  dns_label                  = "kmilb"
  vcn_id                     = oci_core_vcn.oke_vcn.id
  display_name               = "control-plane-endpoint"
  prohibit_public_ip_on_vnic = var.enable_private_endpoint
  route_table_id             = var.enable_private_endpoint==false ? oci_core_route_table.public[0].id : oci_core_default_route_table.default_private[0].id
  security_list_ids = [
    oci_core_default_security_list.oke_vcn.id,
    oci_core_security_list.kmilb.id
  ]
}

Creating a Flannel Overlay VCN

Create the following resources in the order listed:

1. VCN

2. Route rules

   • Public clusters:

     • Internet gateway and a route table with a route rule that references that internet gateway.

     • NAT gateway and a route table with a route rule that references that NAT gateway.

   • Private clusters:

     • Route table with no route rules.

     • (Optional) Dynamic Routing Gateway (DRG) and a route table with a route rule that references that DRG. See Private Clusters.

     • (Optional) Local Peering Gateway (LPG) and a route table with a route rule that references that LPG. See Private Clusters.

3. Security list. Modify the VCN default security list.

Resource names and CIDR blocks are example values.

VCN

To create the VCN, use the instructions in Creating a VCN in the Oracle Private Cloud Appliance User Guide. For Terraform input, see Example Terraform Scripts for Flannel Overlay Network Resources.

For this example, use the following input to create the VCN. The VCN covers one contiguous CIDR block. The CIDR block cannot be changed after the VCN is created.

Compute Web UI property	OCI CLI property
Name: oketest-vcn CIDR Block: vcn_cidr DNS Label: oketest This label must be unique across all VCNs in the tenancy.	--display-name: oketest-vcn --cidr-blocks: '["vcn_cidr"]' --dns-label: oketest This label must be unique across all VCNs in the tenancy.

Note the OCID of the new VCN. In the examples in this guide, this VCN OCID is ocid1.vcn.oke_vcn_id.

Next Steps

• Public internet access. For traffic on a public subnet that connects to the internet using public IP addresses, create an internet gateway and a route rule that references that internet gateway.

• Private internet access. For traffic on a private subnet that needs to connect to the internet without exposing private IP addresses, create a NAT gateway and a route rule that references that NAT gateway.

• VCN-only access. To restrict communication to only other resources on the same VCN, use the default route table, which has no route rules.

• Instances in another VCN. To enable communication between the cluster and an instance running on a different VCN, create a Local Peering Gateway (LPG) and a route rule that references that LPG.

• On-premises IP address space. To enable communication between the cluster and the on-premises network IP address space, create a Dynamic Routing Gateway (DRG), attach the OKE VCN to that DRG, and create a route rule that references that DRG.

VCN Private Route Table

Edit the default route table that was created when you created the VCN. Change the name of the route table to vcn_private. This route table does not have any route rules. Do not add any route rules.

NAT Private Route Table

Create a NAT gateway and a route table with a route rule that references the NAT gateway.

NAT Gateway

To create the NAT gateway, use the instructions in Enabling Public Connections through a NAT Gateway in the Oracle Private Cloud Appliance User Guide. For Terraform input, see Example Terraform Scripts for Flannel Overlay Network Resources.

Note the name and OCID of the NAT gateway for assignment to the private route rule.

Private Route Rule

To create a route table, use the instructions in "Creating a Route Table" in Working with Route Tables in the Oracle Private Cloud Appliance User Guide. For Terraform input, see Example Terraform Scripts for Flannel Overlay Network Resources.

For this example, use the following input to create the route table with a private route rule that references the NAT gateway that was created in the preceding step.

Compute Web UI property	OCI CLI property
Name: nat_private Route rule Target Type: NAT Gateway NAT Gateway: Name of the NAT gateway that was created in the preceding step CIDR Block: 0.0.0.0/0 Description: NAT private route rule	--display-name: nat_private --route-rules networkEntityId: OCID of the NAT gateway that was created in the preceding step destinationType: CIDR_BLOCK destination: 0.0.0.0/0 description: NAT private route rule

Note the name and OCID of this route table for assignment to private subnets.

Local Peering Gateway

Create a Local Peering gateway (LPG) and a route table with a route rule that references the LPG.

Local Peering Gateway

To create the LPG, use the instructions in "Connecting VCNs through a Local Peering Gateway" in the Networking chapter of the Oracle Private Cloud Appliance User Guide.

Note the name and OCID of the LPG for assignment to the private route rule.

Private Route Rule

To create a route table, use the instructions in "Creating a Route Table" in Working with Route Tables in the Oracle Private Cloud Appliance User Guide.

For this example, use the following input to create the route table with a private route rule that references the LPG that was created in the preceding step.

Compute Web UI property	OCI CLI property
Name: lpg_rt Route rule Target Type: Local Peering Gateway Local Peering Gateway: Name of the LPG that was created in the preceding step CIDR Block: CIDR_for_the_second_VCN Description: LPG private route rule	--display-name: lpg_rt --route-rules networkEntityId: OCID of the LPG that was created in the preceding step destinationType: CIDR_BLOCK destination: CIDR_for_the_second_VCN description: LPG private route rule

Note the name and OCID of this route table for assignment to the "control-plane-endpoint" subnet (Creating a Flannel Overlay Control Plane Load Balancer Subnet).

Add the same route rule on the second VCN (the peered VCN), specifying the OKE VCN CIDR as the destination.

Dynamic Routing Gateway

Create a Dynamic Routing gateway (DRG) and a route table with a route rule that references the DRG.

Dynamic Routing Gateway

To create the DRG and attach the OKE VCN to that DRG, use the instructions in "Connecting to the On-Premises Network through a Dynamic Routing Gateway" in the Networking chapter of the Oracle Private Cloud Appliance User Guide. Create the DRG in the OKE VCN compartment, and then attach the OKE VCN to that DRG.

Note the name and OCID of the DRG for assignment to the private route rule.

Private Route Rule

To create a route table, use the instructions in "Creating a Route Table" in Working with Route Tables in the Oracle Private Cloud Appliance User Guide.

For this example, use the following input to create the route table with a private route rule that references the DRG that was created in the preceding step.

Compute Web UI property	OCI CLI property
Name: drg_rt Route rule Target Type: Dynamic Routing Gateway Dynamic Routing: Name of the DRG that was created in the preceding step CIDR Block: 0.0.0.0/0 Description: DRG private route rule	--display-name: drg_rt --route-rules networkEntityId: OCID of the DRG that was created in the preceding step destinationType: CIDR_BLOCK destination: 0.0.0.0/0 description: DRG private route rule

Note the name and OCID of this route table for assignment to the "control-plane-endpoint" subnet (Creating a Flannel Overlay Control Plane Load Balancer Subnet).

Public Route Table

Create an Internet gateway and a route table with a route rule that references the Internet gateway.

Internet Gateway

To create the internet gateway, use the instructions in Providing Public Access through an Internet Gateway in the Oracle Private Cloud Appliance User Guide. For Terraform input, see Example Terraform Scripts for Flannel Overlay Network Resources.

Note the name and OCID of the internet gateway for assignment to the public route rule.

Public Route Rule

To create a route table, use the instructions in "Creating a Route Table" in Working with Route Tables in the Oracle Private Cloud Appliance User Guide. For Terraform input, see Example Terraform Scripts for Flannel Overlay Network Resources.

For this example, use the following input to create the route table with a public route rule that references the internet gateway that was created in the preceding step.

Compute Web UI property	OCI CLI property
Name: public Route rule Target Type: Internet Gateway Internet Gateway: Name of the internet gateway that was created in the preceding step CIDR Block: 0.0.0.0/0 Description: OKE public route rule	--vcn-id: ocid1.vcn.oke_vcn_id --display-name: public --route-rules networkEntityId: OCID of the internet gateway that was created in the preceding step destinationType: CIDR_BLOCK destination: 0.0.0.0/0 description: OKE public route rule

Note the name and OCID of this route table for assignment to public subnets.

VCN Default Security List

Modify the default security list, using the input shown in the following table. Delete all of the default rules and create the rules shown in the following table.

To modify a security list, use the instructions in "Updating a Security List" in Controlling Traffic with Security Lists in the Oracle Private Cloud Appliance User Guide. For Terraform input, see Example Terraform Scripts for Flannel Overlay Network Resources.

Compute Web UI property	OCI CLI property
Name: Default	--security-list-id: ocid1.securitylist.default_securitylist_id
One egress security rule: Stateless: uncheck the box Egress CIDR: 0.0.0.0/0 IP Protocol: All protocols Description: "Allow all outgoing traffic."	One egress security rule: --egress-security-rules isStateless: false destination: 0.0.0.0/0 destinationType: CIDR_BLOCK protocol: all description: "Allow all outgoing traffic."
Three ingress security rules:	Three ingress security rules: --ingress-security-rules
Ingress Rule 1 Stateless: uncheck the box Ingress CIDR: vcn_cidr IP Protocol: ICMP Parameter Type: 8: Echo Description: "Allow ping from VCN."	Ingress Rule 1 isStateless: false source: vcn_cidr sourceType: CIDR_BLOCK protocol: 1 icmpOptions type: 8 description: "Allow ping from VCN."
Ingress Rule 2 Stateless: uncheck the box Ingress CIDR: 0.0.0.0/0 IP Protocol: ICMP Parameter Type: 3: Destination Unreachable Description: "Blocks incoming requests from any source."	Ingress Rule 2 isStateless: false source: 0.0.0.0/0 sourceType: CIDR_BLOCK protocol: 1 icmpOptions type: 3 description: "Blocks incoming requests from any source."
Ingress Rule 3 Stateless: uncheck the box Ingress CIDR: 0.0.0.0/0 IP Protocol: ICMP Parameter Type: 11: Time Exceeded Description: "Time exceeded."	Ingress Rule 3 isStateless: false source: 0.0.0.0/0 sourceType: CIDR_BLOCK protocol: 1 icmpOptions type: 11 description: "Time exceeded."

Note the name and OCID of this default security list for assignment to subnets.

Creating a Flannel Overlay Worker Subnet

Create the following resources in the order listed:

1. Worker security list

2. Worker subnet

Create a Worker Security List

To create a security list, use the instructions in "Creating a Security List" in Controlling Traffic with Security Lists in the Oracle Private Cloud Appliance User Guide. For Terraform input, see Example Terraform Scripts for Flannel Overlay Network Resources.

This security list defines traffic that is allowed to contact worker nodes directly.

For this example, use the following input for the worker subnet security list.

Compute Web UI property	OCI CLI property
Name: worker-seclist	--vcn-id: ocid1.vcn.oke_vcn_id --display-name: worker-seclist
Seven ingress security rules:	Seven ingress security rules: --ingress-security-rules
Ingress Rule 1 Stateless: uncheck the box Ingress CIDR: vcn_cidr IP Protocol: TCP Destination Port Range: 22 Description: "Allow intra-VCN ssh."	Ingress Rule 1 isStateless: false source: vcn_cidr sourceType: CIDR_BLOCK protocol: 6 tcpOptions destinationPortRange max: 22 min: 22 description: "Allow intra-VCN ssh."
Ingress Rule 2 Stateless: uncheck the box Ingress CIDR: kube_client_cidr IP Protocol: TCP Destination Port Range: 30000-32767 Description: "Allow clients to contact the node port range."	Ingress Rule 2 isStateless: false source: kube_client_cidr sourceType: CIDR_BLOCK protocol: 6 tcpOptions destinationPortRange max: 32767 min: 30000 description: "Allow clients to contact the node port range."
Ingress Rule 3 Stateless: uncheck the box Ingress CIDR: workerlb_cidr IP Protocol: TCP Destination Port Range: 30000-32767 Description: "Allow the worker load balancer to contact the worker nodes."	Ingress Rule 3 isStateless: false source: workerlb_cidr sourceType: CIDR_BLOCK protocol: 6 tcpOptions destinationPortRange max: 32767 min: 30000 description: "Allow the worker load balancer to contact the worker nodes."
Ingress Rule 4 Stateless: uncheck the box Ingress CIDR: workerlb_cidr IP Protocol: TCP Destination Port Range: 10256 Description: "Allow the worker load balancer to contact the worker nodes."	Ingress Rule 4 isStateless: false source: workerlb_cidr sourceType: CIDR_BLOCK protocol: 6 tcpOptions destinationPortRange max: 10256 min: 10256 description: "Allow the worker load balancer to contact the worker nodes."
Ingress Rule 5 Stateless: uncheck the box Ingress CIDR: kmi_cidr IP Protocol: TCP Destination Port Range: 22-65535 Description: "Allow the control plane to contact the worker nodes."	Ingress Rule 5 isStateless: false source: kmi_cidr sourceType: CIDR_BLOCK protocol: 6 tcpOptions destinationPortRange max: 65535 min: 22 description: "Allow the control plane to contact the worker nodes."
Ingress Rule 6 Stateless: uncheck the box Ingress CIDR: worker_cidr IP Protocol: UDP Destination Port Range: 8285-8472 Description: "Allow flannel traffic."	Ingress Rule 6 isStateless: false source: worker_cidr sourceType: CIDR_BLOCK protocol: 17 udpOptions destinationPortRange max: 8472 min: 8285 description: "Allow flannel traffic."
Ingress Rule 7 Stateless: uncheck the box Ingress CIDR: kmi_cidr IP Protocol: UDP Destination Port Range: 8285-8472 Description: "Allow flannel traffic."	Ingress Rule 7 isStateless: false source: kmi_cidr sourceType: CIDR_BLOCK protocol: 17 udpOptions destinationPortRange max: 8472 min: 8285 description: "Allow flannel traffic."

Create the Worker Subnet

To create a subnet, use the instructions in Creating a Subnet in the Oracle Private Cloud Appliance User Guide. For Terraform input, see Example Terraform Scripts for Flannel Overlay Network Resources.

For this example, use the following input to create the worker subnet. Use the OCID of the VCN that was created in Creating a Flannel Overlay VCN. Create the worker subnet in the same compartment where you created the VCN.

Create either a NAT private worker subnet or a VCN private worker subnet. Create a NAT private worker subnet to communicate outside the VCN.

Table 4-3 Create a NAT Private Worker Subnet

Compute Web UI property	OCI CLI property
Name: worker CIDR Block: worker_cidr Route Table: Select "nat_private" from the list Private Subnet: check the box DNS Hostnames: Use DNS Hostnames in this Subnet: check the box DNS Label: worker Security Lists: Select "worker-seclist" and "Default Security List for oketest-vcn" from the list	--vcn-id: ocid1.vcn.oke_vcn_id --display-name: worker --cidr-block: worker_cidr --dns-label: worker --prohibit-public-ip-on-vnic: true --route-table-id: OCID of the "nat_private" route table --security-list-ids: OCIDs of the "worker-seclist" security list and the "Default Security List for oketest-vcn" security list

The difference in the following private subnet is the VCN private route table is used instead of the NAT private route table.

Table 4-4 Create a VCN Private Worker Subnet

Compute Web UI property	OCI CLI property
Name: worker CIDR Block: worker_cidr Route Table: Select "vcn_private" from the list Private Subnet: check the box DNS Hostnames: Use DNS Hostnames in this Subnet: check the box DNS Label: worker Security Lists: Select "worker-seclist" and "Default Security List for oketest-vcn" from the list	--vcn-id: ocid1.vcn.oke_vcn_id --display-name: worker --cidr-block: worker_cidr --dns-label: worker --prohibit-public-ip-on-vnic: true --route-table-id: OCID of the "vcn_private" route table --security-list-ids: OCIDs of the "worker-seclist" security list and the "Default Security List for oketest-vcn" security list

Creating a Flannel Overlay Worker Load Balancer Subnet

Create the following resources in the order listed:

1. Worker load balancer security list

2. Worker load balancer subnet

Create a Worker Load Balancer Security List

To create a security list, use the instructions in "Creating a Security List" in Controlling Traffic with Security Lists in the Oracle Private Cloud Appliance User Guide. For Terraform input, see Example Terraform Scripts for Flannel Overlay Network Resources.

This security list defines traffic, such as applications, that is allowed to contact the worker load balancer.

For this example, use the following input for the worker load balancer subnet security list. These sources and destinations are examples; adjust these for your applications.

Note:

When you create an external load balancer for your containerized applications (see Exposing Containerized Applications), remember to add that load balancer service front-end port to this security list.

Compute Web UI property	OCI CLI property
Name: workerlb-seclist	--vcn-id: ocid1.vcn.oke_vcn_id --display-name: workerlb-seclist
Two ingress security rules:	Two ingress security rules: --ingress-security-rules
Ingress Rule 1 Stateless: uncheck the box Ingress CIDR: kube_client_cidr IP Protocol: TCP Destination Port Range: 80 Description: "Allow inbound traffic for applications."	Ingress Rule 1 isStateless: false source: kube_client_cidr sourceType: CIDR_BLOCK protocol: 6 tcpOptions destinationPortRange max: 80 min: 80 description: "Allow inbound traffic for applications."
Ingress Rule 2 Stateless: uncheck the box Ingress CIDR: kube_client_cidr IP Protocol: TCP Destination Port Range: 443 Description: "Allow inbound traffic for applications."	Ingress Rule 2 isStateless: false source: kube_client_cidr sourceType: CIDR_BLOCK protocol: 6 tcpOptions destinationPortRange max: 443 min: 443 description: "Allow inbound traffic for applications."

Create the Worker Load Balancer Subnet

To create a subnet, use the instructions in Creating a Subnet in the Oracle Private Cloud Appliance User Guide. For Terraform input, see Example Terraform Scripts for Flannel Overlay Network Resources.

For this example, use the following input to create the worker load balancer subnet. Use the OCID of the VCN that was created in Creating a Flannel Overlay VCN. Create the worker load balancer subnet in the same compartment where you created the VCN.

Create either a private or a public worker load balancer subnet. Create a public worker load balancer subnet to use with a public cluster. Create a private worker load balancer subnet to expose applications in a private cluster.

Table 4-5 Create a Public Worker Load Balancer Subnet

Compute Web UI property	OCI CLI property
Name: service-lb CIDR Block: workerlb_cidr Route Table: Select "public" from the list Public Subnet: check the box DNS Hostnames: Use DNS Hostnames in this Subnet: check the box DNS Label: servicelb Security Lists: Select "workerlb-seclist" and "Default Security List for oketest-vcn" from the list	--vcn-id: ocid1.vcn.oke_vcn_id --display-name: service-lb --cidr-block: workerlb_cidr --dns-label: servicelb --prohibit-public-ip-on-vnic: false --route-table-id: OCID of the "public" route table --security-list-ids: OCIDs of the "workerlb-seclist" security list and the "Default Security List for oketest-vcn" security list

The difference in the following private subnet is the VCN private route table is used instead of the public route table.

Table 4-6 Create a Private Worker Load Balancer Subnet

Compute Web UI property	OCI CLI property
Name: service-lb CIDR Block: workerlb_cidr Route Table: Select "vcn_private" from the list Private Subnet: check the box DNS Hostnames: Use DNS Hostnames in this Subnet: check the box DNS Label: servicelb Security Lists: Select "workerlb-seclist" and "Default Security List for oketest-vcn" from the list	--vcn-id: ocid1.vcn.oke_vcn_id --display-name: service-lb --cidr-block: workerlb_cidr --dns-label: servicelb --prohibit-public-ip-on-vnic: true --route-table-id: OCID of the "vcn_private" route table --security-list-ids: OCIDs of the "workerlb-seclist" security list and the "Default Security List for oketest-vcn" security list

Creating a Flannel Overlay Control Plane Subnet

Create the following resources in the order listed:

1. Control plane security list

2. Control plane subnet

Create a Control Plane Security List

To create a security list, use the instructions in "Creating a Security List" in Controlling Traffic with Security Lists in the Oracle Private Cloud Appliance User Guide. For Terraform input, see Example Terraform Scripts for Flannel Overlay Network Resources.

For this example, use the following input for the control plane subnet security list. The kubernetes_api_port is the port used to access the Kubernetes API: port 6443. See also Workload Cluster Network Ports for VCN-Native Pod Networking. See also Workload Cluster Network Ports for Flannel Overlay Networking.

Compute Web UI property	OCI CLI property
Name: kmi-seclist	--vcn-id: ocid1.vcn.oke_vcn_id --display-name: kmi-seclist
Six ingress security rules:	Six ingress security rules: --ingress-security-rules
Ingress Rule 1 Stateless: uncheck the box Ingress CIDR: kube_client_cidr IP Protocol: TCP Destination Port Range: kubernetes_api_port Description: "Allow inbound connections to the Kubernetes API server."	Ingress Rule 1 isStateless: false source: kube_client_cidr sourceType: CIDR_BLOCK protocol: 6 tcpOptions destinationPortRange max: kubernetes_api_port min: kubernetes_api_port description: "Allow inbound connections to the Kubernetes API server."
Ingress Rule 2 Stateless: uncheck the box Ingress CIDR: kmilb_cidr IP Protocol: TCP Destination Port Range: kubernetes_api_port Description: "Allow inbound connections from the control plane load balancer."	Ingress Rule 2 isStateless: false source: kmilb_cidr sourceType: CIDR_BLOCK protocol: 6 tcpOptions destinationPortRange max: kubernetes_api_port min: kubernetes_api_port description: "Allow inbound connections from the control plane load balancer."
Ingress Rule 3 Stateless: uncheck the box Ingress CIDR: worker_cidr IP Protocol: TCP Destination Port Range: 1024-65535 Description: "Allow inbound connections from worker nodes to the control plane."	Ingress Rule 3 isStateless: false source: worker_cidr sourceType: CIDR_BLOCK protocol: 6 tcpOptions destinationPortRange max: 65535 min: 1024 description: "Allow inbound connections from worker nodes to the control plane."
Ingress Rule 4 Stateless: uncheck the box Ingress CIDR: kmi_cidr IP Protocol: TCP Destination Port Range: 1024-65535 Description: "Allow inbound connections within the control plane."	Ingress Rule 4 isStateless: false source: kmi_cidr sourceType: CIDR_BLOCK protocol: 6 tcpOptions destinationPortRange max: 65535 min: 1024 description: "Allow inbound connections within the control plane."
Ingress Rule 5 Stateless: uncheck the box Ingress CIDR: worker_cidr IP Protocol: UDP Destination Port Range: 8285-8472 Description: "Allow flannel traffic."	Ingress Rule 5 isStateless: false source: worker_cidr sourceType: CIDR_BLOCK protocol: 17 udpOptions destinationPortRange max: 8472 min: 8285 description: "Allow flannel traffic."
Ingress Rule 6 Stateless: uncheck the box Ingress CIDR: kmi_cidr IP Protocol: UDP Destination Port Range: 8285-8472 Description: "Allow flannel traffic."	Ingress Rule 6 isStateless: false source: kmi_cidr sourceType: CIDR_BLOCK protocol: 17 udpOptions destinationPortRange max: 8472 min: 8285 description: "Allow flannel traffic."

Create the Control Plane Subnet

To create a subnet, use the instructions in Creating a Subnet in the Oracle Private Cloud Appliance User Guide. For Terraform input, see Example Terraform Scripts for Flannel Overlay Network Resources.

Use the following input to create the control plane subnet. Use the OCID of the VCN that was created in Creating a Flannel Overlay VCN. Create the control plane subnet in the same compartment where you created the VCN.

Create either a NAT private control plane subnet or a VCN private control plane subnet. Create a NAT private control plane subnet to communicate outside the VCN.

Important:

The name of this subnet must be exactly "control-plane".

Table 4-7 Create a NAT Private Control Plane Subnet

Compute Web UI property	OCI CLI property
Name: control-plane CIDR Block: kmi_cidr Route Table: Select "nat_private" from the list Private Subnet: check the box DNS Hostnames: Use DNS Hostnames in this Subnet: check the box DNS Label: kmi Security Lists: Select "kmi-seclist" and "Default Security List for oketest-vcn" from the list	--vcn-id: ocid1.vcn.oke_vcn_id --display-name: control-plane --cidr-block: kmi_cidr --dns-label: kmi --prohibit-public-ip-on-vnic: true --route-table-id: OCID of the "nat_private" route table --security-list-ids: OCIDs of the "kmi-seclist" security list and the "Default Security List for oketest-vcn" security list

The difference in the following private subnet is the VCN private route table is used instead of the NAT private route table.

Table 4-8 Create a VCN Private Control Plane Subnet

Compute Web UI property	OCI CLI property
Name: control-plane CIDR Block: kmi_cidr Route Table: Select "vcn_private" from the list Private Subnet: check the box DNS Hostnames: Use DNS Hostnames in this Subnet: check the box DNS Label: kmi Security Lists: Select "kmi-seclist" and "Default Security List for oketest-vcn" from the list	--vcn-id: ocid1.vcn.oke_vcn_id --display-name: control-plane --cidr-block: kmi_cidr --dns-label: kmi --prohibit-public-ip-on-vnic: true --route-table-id: OCID of the "vcn_private" route table --security-list-ids: OCIDs of the "kmi-seclist" security list and the "Default Security List for oketest-vcn" security list

Creating a Flannel Overlay Control Plane Load Balancer Subnet

Create the following resources in the order listed:

1. Control plane load balancer security list

2. Control plane load balancer subnet

Create a Control Plane Load Balancer Security List

To create a security list, use the instructions in "Creating a Security List" in Controlling Traffic with Security Lists in the Oracle Private Cloud Appliance User Guide. For Terraform input, see Example Terraform Scripts for Flannel Overlay Network Resources.

The control plane load balancer accepts traffic on port 6443, which is also called kubernetes_api_port in this guide. Adjust this security list to only accept connections from where you expect the network to run. Port 6443 must accept connections from the cluster control plane instances and worker instances.

For this example, use the following input for the control plane load balancer subnet security list.

Compute Web UI property	OCI CLI property
Name: kmilb-seclist	--vcn-id: ocid1.vcn.oke_vcn_id --display-name: kmilb-seclist
Four ingress security rules:	Four ingress security rules: --ingress-security-rules
Ingress Rule 1: Stateless: uncheck the box Ingress CIDR: kube_internal_cidr This value is required. Do not change this CIDR value. IP Protocol: TCP Destination Port Range: kubernetes_api_port Description: "Allow inbound connections to the control plane load balancer."	Ingress Rule 1: isStateless: false source: kube_internal_cidr This value is required. Do not change this CIDR value. sourceType: CIDR_BLOCK protocol: 6 tcpOptions destinationPortRange max: kubernetes_api_port min: kubernetes_api_port description: "Allow inbound connections to the control plane load balancer."
Ingress Rule 2: Stateless: uncheck the box Ingress CIDR: kube_client_cidr IP Protocol: TCP Destination Port Range: kubernetes_api_port Description: "Allow inbound connections to the control plane load balancer."	Ingress Rule 2: isStateless: false source: kube_client_cidr sourceType: CIDR_BLOCK protocol: 6 tcpOptions destinationPortRange max: kubernetes_api_port min: kubernetes_api_port description: "Allow inbound connections to the control plane load balancer."
Ingress Rule 3: Stateless: uncheck the box Ingress CIDR: vcn_cidr IP Protocol: TCP Destination Port Range: kubernetes_api_port Description: "Allow inbound connections to the control plane load balancer."	Ingress Rule 3: isStateless: false source: vcn_cidr sourceType: CIDR_BLOCK protocol: 6 tcpOptions destinationPortRange max: kubernetes_api_port min: kubernetes_api_port description: "Allow inbound connections to the control plane load balancer."
Ingress Rule 4: Public Endpoint Stateless: uncheck the box Ingress CIDR: public_ip_cidr IP Protocol: TCP Destination Port Range: kubernetes_api_port Description: "Used to access the control plane endpoint from the public CIDR. The public IP CIDR is configured in the Service Enclave. If you do not know what your public IP CIDR is, ask your Service Enclave administrator."	Ingress Rule 4: Public Endpoint isStateless: false source: public_ip_cidr sourceType: CIDR_BLOCK protocol: 6 tcpOptions destinationPortRange max: kubernetes_api_port min: kubernetes_api_port description: "Used to access the control plane endpoint from the public CIDR. The public IP CIDR is configured in the Service Enclave. If you do not know what your public IP CIDR is, ask your Service Enclave administrator."

Create the Control Plane Load Balancer Subnet

To create a subnet, use the instructions in Creating a Subnet in the Oracle Private Cloud Appliance User Guide. For Terraform input, see Example Terraform Scripts for Flannel Overlay Network Resources.

For this example, use the following input to create the control plane load balancer subnet. Use the OCID of the VCN that was created in Creating a Flannel Overlay VCN. Create the control plane load balancer subnet in the same compartment where you created the VCN.

Create either a private or a public control plane load balancer subnet. Create a public control plane load balancer subnet to use with a public cluster. Create a private control plane load balancer subnet to use with a private cluster.

See Private Clusters for information about using Local Peering Gateways to connect a private cluster to other instances on the Private Cloud Appliance and using Dynamic Routing Gateways to connect a private cluster to the on-premises IP address space. To create a private control plane load balancer subnet, specify one of the following route tables (see Creating a Flannel Overlay VCN):

• vcn_private

• lpg_rt

• drg_rt

Table 4-9 Create a Public Control Plane Load Balancer Subnet

Compute Web UI property	OCI CLI property
Name: control-plane-endpoint CIDR Block: kmilb_cidr Route Table: Select "public" from the list Public Subnet: check the box DNS Hostnames: Use DNS Hostnames in this Subnet: check the box DNS Label: kmilb Security Lists: Select "kmilb-seclist" and "Default Security List for oketest-vcn" from the list	--vcn-id: ocid1.vcn.oke_vcn_id --display-name: control-plane-endpoint --cidr-block: kmilb_cidr --dns-label: kmilb --prohibit-public-ip-on-vnic: false --route-table-id: OCID of the "public" route table --security-list-ids: OCIDs of the "kmilb-seclist" security list and the "Default Security List for oketest-vcn" security list

The difference in the following private subnet is the VCN private route table is used instead of the public route table. Depending on your needs, you could specify the LPG route table or the DRG route table instead.

Table 4-10 Create a Private Control Plane Load Balancer Subnet

Compute Web UI property	OCI CLI property
Name: control-plane-endpoint CIDR Block: kmilb_cidr Route Table: Select "vcn_private" from the list Private Subnet: check the box DNS Hostnames: Use DNS Hostnames in this Subnet: check the box DNS Label: kmilb Security Lists: Select "kmilb-seclist" and "Default Security List for oketest-vcn" from the list	--vcn-id: ocid1.vcn.oke_vcn_id --display-name: control-plane-endpoint --cidr-block: kmilb_cidr --dns-label: kmilb --prohibit-public-ip-on-vnic: true --route-table-id: OCID of the "vcn_private" route table --security-list-ids: OCIDs of the "kmilb-seclist" security list and the "Default Security List for oketest-vcn" security list