To deploy a multi-master Kubernetes cluster:

1. A number of additional ports are required to be open on master nodes in a multi-master deployment. For information on opening the required ports for a multi-master deployment, see Section 2.3.4.2, “Multi-Master Firewall Rules”.

2. A multi-master deployment needs a load balancer to provide high availability of master nodes. A load balancer can be deployed automatically when you perform a multi-master deployment, or you can use your own load balancer implementation.

   As part of using the load balancer deployed by the Platform CLI, NGINX and keepalived are installed on the master nodes to enable the container-based deployment of the load balancer. NGINX improves the resource availability and efficiency of your multi-master Kubernetes cluster. keepalived can be used to monitor services or systems and to failover automatically to a standby master node if problems occur. As part of deploying this load balancer for a multi-master deployment, the olcne-nginx and keepalived services are enabled and started on the master nodes.

   For information on preparing the master nodes to use the load balancer deployed by the Platform CLI, see Section 3.4.3, “Setting up a Load Balancer”.

3. Perform the same steps as a single master deployment, as discussed in Chapter 4, Creating and Managing a Kubernetes Cluster, using different options for the olcne module create command to specify either a virtual IP address for the load balancer deployed by the Platform CLI, or the hostname and port for your own load balancer instance.

   To use the load balancer deployed by the Platform CLI, use the --virtual-ip option to set the virtual IP address to be used for the primary master node, for example, --virtual-ip 192.0.2.100.

   Alternatively, you can set the hostname and port of your own load balancer implementation using the --load-balancer option, for example --load-balancer lb.example.com:6443.

   You do not need to include the --apiserver-advertise-address option for a multi-master deployment.

   The following example creates a multi-master deployment using the load balancer deployed by the Platform CLI. The virtual IP address of 192.0.2.100 is used for the primary master node.

   $ olcnectl --api-server 127.0.0.1:8091 module create --environment-name myenvironment \
     --module kubernetes --name mycluster \
     --container-registry container-registry.oracle.com/olcne \
     --virtual-ip 192.0.2.100 \
     --master-nodes master1.example.com:8090,master2.example.com:8090,master3.example.com:8090 \
     --worker-nodes worker1.example.com:8090,worker2.example.com:8090,worker3.example.com:8090,worker4.example.com:8090 \
     --restrict-service-externalip-ca-cert=/etc/olcne/configs/certificates/restrict_external_ip/production/ca.cert \
     --restrict-service-externalip-tls-cert=/etc/olcne/configs/certificates/restrict_external_ip/production/node.cert \
     --restrict-service-externalip-tls-key=/etc/olcne/configs/certificates/restrict_external_ip/production/node.key

   The following example creates a multi-master deployment using your own load balancer, available on the host lb.example.com and running on port 6443.

   $ olcnectl --api-server 127.0.0.1:8091 module create --environment-name myenvironment \
     --module kubernetes --name mycluster \
     --container-registry container-registry.oracle.com/olcne \
     --load-balancer lb.example.com:6443 \
     --master-nodes master1.example.com:8090,master2.example.com:8090,master3.example.com:8090 \
     --worker-nodes worker1.example.com:8090,worker2.example.com:8090,worker3.example.com:8090,worker4.example.com:8090 \
     --restrict-service-externalip-ca-cert=/etc/olcne/configs/certificates/restrict_external_ip/production/ca.cert \
     --restrict-service-externalip-tls-cert=/etc/olcne/configs/certificates/restrict_external_ip/production/node.cert \
     --restrict-service-externalip-tls-key=/etc/olcne/configs/certificates/restrict_external_ip/production/node.key

4. Continue with the module validation and install as described in in Chapter 4, Creating and Managing a Kubernetes Cluster.