Performing Helm Test

10 Performing Helm Test

Helm Test is a feature which validates CNCC Successful Installation along with readiness (Readiness probe URL configured will be checked for success) of all the pods. The pods to be checked will be based on the namespace and label selector configured for the helm test configurations.

Note:

Helm3 is mandatory for the Helm Test feature to work.

To perform Helm test, perform the following steps:

Note:

The configurations mentioned in the step below must be done before running the helm install command.

Configure the helm test configurations which are under global sections in occncc_custom_values_<version>.yaml file. Refer the following configurations :

CNCC Helm Test

global:  
  # Helm test related configurations
  test:
    nfName: cncc
    image:
      name: occncc/nf_test
      tag: 22.4.1
      imagePullPolicy: IfNotPresent
    config:
      logLevel: WARN
      timeout: 240

Run the following command to run Helm test on the installation done:

 helm test <helm_release_name> -n <k8s namespace>

Example:

 
E.g. [root@master cncc]# helm test cncc -n cncc
Pod cncc-test pending
Pod cncc-test pending
Pod cncc-test pending
Pod cncc-test pending
Pod cncc-test running
Pod cncc-test succeeded
NAME: cncc
LAST DEPLOYED: Tue Jun  7 06:47:14 2022
NAMESPACE: cncc
STATUS: deployed
REVISION: 1
TEST SUITE:     cncc-test
Last Started:   Wed Jun  8 06:01:10 2022
Last Completed: Wed Jun  8 06:01:44 2022
Phase:          Succeeded
NOTES:
# Copyright 2020 (C), Oracle and/or its affiliates. All rights reserved. Thank you for installing cncc. Your release is named cncc , Release Revision: 1.
To learn more about the release, try: 
 
  $ helm status cncc
  $ helm get cncc

Wait for the helm test to complete. Check for the output to see if the test job is successful.

Helm Test Kubernetes Resources Logging

The Helm test logging enhancement for Kubernetes resources lists the following details of Kubernetes resources:

Versions of Kubernetes resource available in OCCNE.
Preferred version for that Kubernetes resource on the OCCNE.
For each micro service, it list the version of Kubernetes resource used.

This information can be used in the following cases:

In case of CNE upgrade for customer, helm test shall list the Kubernetes resource versions used by NFs. The operator can use this information to run a pre-upgrade compatibility test to see if the Kubernetes resource versions are available on target CNE.
After CNE upgrade, there might be certain resources for which a newer version is available on CNE which is also supported by Console charts. If the output of helm test indicates failure of compliance for certain resource, then upgrade the Console to use the latest version of Kubernetes resource for which failure was indicated.
List all available versions on CNE. Console can use this detail as an input for apiVersion to be used in latest NF charts to which upgrade will be performed.

Note that this feature is tested and compatible with CNE version 1.10 and above.

To use this feature, set global.test.complianceEnable flag as true.

Separate helm test service account can be created and set at global.helmTestserviceAccountName, see Helm Test Service Account Configuration section.

Note:

For helm test execution preference goes to global.helmTestserviceAccountName first, if this is not available then global.serviceAccountName will be referred. If both of these are missing then default service account will be created and used.

 # Custom service account for Helm test execution
 helmTestserviceAccountName: ""
 
 # Helm test related configurations
 test:
   nfName: cncc
   image:
     name: occncc/nf_test
     tag: 22.4.1
     imagePullPolicy: IfNotPresent
   config:
     logLevel: WARN
     timeout: 240 #Beyond this duration helm test will be considered failure
   resources:
   - horizontalpodautoscalers/v1
   - deployments/v1
   - configmaps/v1
   - prometheusrules/v1
   - serviceaccounts/v1
   - poddisruptionbudgets/v1
   - roles/v1
   - statefulsets/v1
   - persistentvolumeclaims/v1
   - services/v1
   - rolebindings/v1
   complianceEnable: true

Run the following command to check the helm test logs:

helm test <releaseName> --logs -n <namespace>

Example:

helm test cncc --logs -n cncc

The output lists:

The versions of the Kubernetes resources used by Console, which helps in running pre upgrade compatibility test before CNE upgrade.
Compliance check for each Kubernetes resource, if compliance check is false we need to upgrade the Console charts as latest version is supported both by charts and available in new CNE.
Available Kubernetes resource versions on CNE.

Log Example:

{
  "horizontalpodautoscalers" : {
    "availableVersionOnCne" : [ "v1", "v2beta1", "v2beta2" ],
    "avaliableOnDeployment" : [ ],
    "compliant" : true,
    "prefferedVersionOnCne" : "v1",
    "maxNFVersion" : "v1"
  },
  "deployments" : {
    "availableVersionOnCne" : [ "v1" ],
    "avaliableOnDeployment" : [ ],
    "compliant" : true,
    "prefferedVersionOnCne" : "v1",
    "maxNFVersion" : "v1"
  },
  "configmaps" : {
    "availableVersionOnCne" : [ "v1" ],
    "avaliableOnDeployment" : [ ],
    "compliant" : true,
    "prefferedVersionOnCne" : "v1",
    "maxNFVersion" : "v1"
  },
  "serviceaccounts" : {
    "availableVersionOnCne" : [ "v1" ],
    "avaliableOnDeployment" : [ ],
    "compliant" : true,
    "prefferedVersionOnCne" : "v1",
    "maxNFVersion" : "v1"
  },
  "poddisruptionbudgets" : {
    "availableVersionOnCne" : [ "v1beta1" ],
    "avaliableOnDeployment" : [ ],
    "compliant" : true,
    "prefferedVersionOnCne" : "v1beta1",
    "maxNFVersion" : "v1"
  },
  "roles" : {
    "availableVersionOnCne" : [ "v1", "v1beta1" ],
    "avaliableOnDeployment" : [ ],
    "compliant" : true,
    "prefferedVersionOnCne" : "v1",
    "maxNFVersion" : "v1"
  },
  "statefulsets" : {
    "availableVersionOnCne" : [ "v1" ],
    "avaliableOnDeployment" : [ ],
    "compliant" : true,
    "prefferedVersionOnCne" : "v1",
    "maxNFVersion" : "v1"
  },
  "persistentvolumeclaims" : {
    "availableVersionOnCne" : [ "v1" ],
    "avaliableOnDeployment" : [ ],
    "compliant" : true,
    "prefferedVersionOnCne" : "v1",
    "maxNFVersion" : "v1"
  },
  "services" : {
    "availableVersionOnCne" : [ "v1" ],
    "avaliableOnDeployment" : [ ],
    "compliant" : true,
    "prefferedVersionOnCne" : "v1",
    "maxNFVersion" : "v1"
  },
  "rolebindings" : {
    "availableVersionOnCne" : [ "v1", "v1beta1" ],
    "avaliableOnDeployment" : [ ],
    "compliant" : true,
    "prefferedVersionOnCne" : "v1",
    "maxNFVersion" : "v1"
  }
}

Helm Test Cleanup

After running the helm test, the pod moves to completed state, to remove the pod manually, run the following command:

kubectl delete pod release_name -test -n <namespace_name>

Example:

kubectl delete pod cncc-test -n cncc