Upgrading K8s container engine from Docker to Containerd

This section explains the procedure to upgrade K8s container engine from docker to container.

Note: This step is only for execution from 1.3.2 to 1.4.0 where kube version is same but there is a change to container engine for cluster, this step should be removed for future upgrade procedure.

Get k8s dependencies for 1.4.0 k8s upgrade for containerd on Bastion Host

Example-
ANSIBLE_NOCOLOR=1 OCCNE_VERSION= K8S_IMAGE=winterfell:5000/occne/k8s_install:1.4.0 CENTRAL_REPO=winterfell K8S_ARGS="" K8S_SKIP_TEST=1 K8S_SKIP_DEPLOY=1  /var/occne/cluster/<cluster-name>/artifacts/pipeline.sh

Create upgrade_container.yml in /var/occne/cluster/<cluster_name> directory with contents below:

- hosts: k8s-cluster
  tasks:
  - name: Switch Docker container runtime to containerd
    shell: "{{ item }}"
    with_items:
      - "sudo cp /etc/cni/net.d/calico.conflist.template 10-containerd-net.conflist"
      - "systemctl daemon-reload"
      - "systemctl enable containerd"
      - "systemctl restart containerd"
      - "systemctl stop docker"
      - "systemctl daemon-reload"
      - "systemctl restart kubelet"
      - "sudo yum remove -y docker-ce"
    ignore_errors: yes

Run k8s install in bash mode to update container engine from docker to container d

Bare Metal Clusters

docker run -it --rm --cap-add=NET_ADMIN --network host -v /var/occne/cluster/<cluster-name>:/host -v /var/occne:/var/occne:rw -e ANSIBLE_NOCOLOR=1 -e 'OCCNEARGS=       ' winterfell:5000/occne/k8s_install:1.4.0 bash

VCNE Clusters

// Get Values from Cloud Config
Example-
docker run -it --rm --cap-add=NET_ADMIN --network host -v /var/occne/cluster/<cluster-name>:/host -v /var/occne:/var/occne:rw -e OCCNEINV=/host/terraform/hosts -e 'OCCNEARGS=--extra-vars={"occne_vcne":"1","occne_cluster_name":"ankit-upgrade-3","occne_repo_host":"ankit-upgrade-3-bastion-1","occne_repo_host_address":"192.168.200.9"} --extra-vars={"openstack_username":"ankit.misra","openstack_password":"{Cloud-Password}","openstack_auth_url":"http://thundercloud.us.oracle.com:5000/v3","openstack_region":"RegionOne","openstack_tenant_id":"811ef89b5f154ab0847be2f7e41117c0","openstack_domain_name":"LDAP","openstack_lbaas_subnet_id":"2787146b-56fe-4c58-bd87-086856de24a9","openstack_lbaas_floating_network_id":"e4351e3e-81e3-4a83-bdc1-dde1296690e3","openstack_lbaas_use_octavia":"true","openstack_lbaas_method":"ROUND_ROBIN","openstack_lbaas_enabled":true}      ' winterfell:5000/occne/k8s_install:<image_tag> bash

Below steps are common once in bash docker mode for both vcne and bare metal:

sed -i /kubespray/roles/bootstrap-os/tasks/bootstrap-oracle.yml -re '2, 16d'
sed -i /kubespray/roles/kubernetes-apps/ingress_controller/cert_manager/tasks/main.yml -re '3, 58d'
//  The command runs the playbook to add configuration files for containerd
 
/copyHosts.sh ${OCCNEINV} && ansible-playbook -i /kubespray/inventory/occne/hosts \
    --become \
    --become-user=root \
    --private-key /host/.ssh/occne_id_rsa \
    /kubespray/cluster.yml ${OCCNEARGS}
 
 
// Once done run the upgrade_container in bash mode below.
// Around a 2 -3 minute timeout for some services may occur depending on how quickly the next command is executed.
 
 
/copyHosts.sh ${OCCNEINV} && ansible-playbook -i /kubespray/inventory/occne/hosts \
    --become \
    --become-user=root \
    --private-key /host/.ssh/occne_id_rsa \
    /host/upgrade_container.yml
 
// Note : There will be a prompt during running above task on vcne that calico.conflist.template does not exist, this is because flannel is used rather then calico. Prompt will be skipped for vcne

Wait for all pods to become ready with 1/1 and status as running. This can be done by executing kubectl get pods. Run next steps after confirming all pods are ready , running.

Test to check all containers are managed by containerd:

// Login into any node of the cluster to see all the containers are managed by crictl
sudo /usr/local/bin/crictl ps