8 OCCM KPIs

This section describes the KPIs available for OCCM.

8.1 Certificate Expiry Time

Table 8-1 Certificate Expiry Time

Field Details
Description

Certificate Expiry Time to list NF, Certificate Name, Expiry Date

Expression

OCCM dashboard in grafana will show Certificate Expiry Time panel with columns. Table visualization listing Expires, NF, Certificate Name, Expiry Date. Expires column uses color coding to indicate near expiry status.

all:occm_cert_expiry{namespace="$namespace"} * 1000 != 0

Expires column:((occm_cert_expiry{namespace="$namespace"} != 0)-time())*1000

OCCM KPI Dashboard

Figure 8-1 Certificate Expiry Time


Certificate Expiry Time

Color coding description:-

Red (Critical):- Certificate expiring within 0 <= 7 days Or Certificate expired <= 0 days

Light Red(Major):- Certificate expiring within > 7 <= 30 days

Orange (Minor):- Certificate expiring within > 30 <= 90

Yellow :- Certificate expiring within > 90 <= 180

Green :- Certificates not Expiring sooner

8.2 Certificate Readiness Status

Table 8-2 Certificate Readiness Status

Field Details
Description Certificate Readiness Status to indicate if number of Ready and Failed Certificates
Expression

OCCM dashboard in grafana will show Certificate Readiness Status panel gauge visualization to indicate if number of Ready and Failed Certificates

Creating:count(occm_cert_status{namespace="$namespace"} == 1) (Color:Orange)

Ready:count(occm_cert_status{namespace="$namespace"} == 2) (Color:Green)

Failed:count(occm_cert_status{namespace="$namespace"} == 3) (Color:Red)

Waiting:count(occm_cert_status{namespace="$namespace"} == 8) (Color:Light Orange)

Expired:count(occm_cert_status{namespace="$namespace"} == 7) (Color:Red)

OCCM KPI Dashboard

Figure 8-2 Certificate Readiness Status


Certificate Readiness Status

Creating: Orange

Ready: Green

Failed: Red

Waiting: Light Orange

Expired : Red

8.3 CMP Request

Table 8-3 CMP Request

Field Details
Description Total CMP requests initiated from OCCM towards CA per NF
Expression

OCCM dashboard in grafana will show CMP Request panel which is total CMP requests per NF.

all:sum(rate(occm_cmp_requests_total{namespace="$namespace"}[2m]))

SCP:sum(rate(occm_cmp_requests_total{namespace="$namespace", nfType=~"SCP|scp"}[2m]))

NRF:sum(rate(occm_cmp_requests_total{namespace="$namespace", nfType=~"NRF|nrf"}[2m]))

8.4 CMP Responses

Table 8-4 CMP Responses

Field Details
Description Total CMP responses received from CA per NF by OCCM
Expression

OCCM dashboard in grafana will show CMP Response panel which is total CMP responses per NF.

all:sum(rate(occm_cmp_responses_total{namespace="$namespace"}[2m]))

SCP:sum(rate(occm_cmp_responses_total{namespace="$namespace", nfType=~"SCP|scp"}[2m]))

NRF:sum(rate(occm_cmp_responses_total{namespace="$namespace", nfType=~"NRF|nrf"}[2m]))

8.5 Configuration Requests

Table 8-5 Configuration Requests

Field Details
Description Total Issuer and Certificate configuration requests
Expression

OCCM dashboard in grafana will show Config Requests panel. Total Issuer and Certificate configuration requests.

all:sum(rate(occm_config_http_requests_total{namespace="$namespace"}[2m]))

SCP certs:sum(rate(occm_config_http_requests_total{namespace="$namespace", uri=~".*/certs.*", nfType=~"SCP|scp"}[2m]))

NRF certs:sum(rate(occm_config_http_requests_total{namespace="$namespace", uri=~".*/certs.*", nfType=~"NRF|nrf"}[2m]))

issuers:sum(rate(occm_config_http_requests_total{namespace="$namespace", uri=~".*/issuers.*"}[2m]))

8.6 Configuration Responses

Table 8-6 Configuration Responses

Field Details
Description Total Issuer and Certificate configuration responses
Expression

OCCM dashboard in grafana will show Config Responses panel. Total Issuer and Certificate configuration responses.

all:sum(rate(occm_config_http_responses_total{namespace="$namespace"}[2m]))

SCP certs:sum(rate(occm_config_http_responses_total{namespace="$namespace", uri=~".*/certs.*", nfType=~"SCP|scp"}[2m]))

NRF certs:sum(rate(occm_config_http_responses_total{namespace="$namespace", uri=~".*/certs.*", nfType=~"NRF|nrf"}[2m]))

issuers:sum(rate(occm_config_http_responses_total{namespace="$namespace", uri=~".*/issuers.*"}[2m]))

8.7 CPU Usage

Table 8-7 CPU Usage

Field Details
Description CPU usage of OCCM pod
Expression

Time series indicates CPU usage of OCCM pod.

sum(rate(container_cpu_usage_seconds_total{image!="",namespace="$namespace", pod=~"occm-.*."}[2m])) by(pod)

8.8 Memory Usage

Table 8-8 Memory Usage

Field Details
Description Memory usage of OCCM pod
Expression

Time series indicates Memory usage of OCCM pod.

(avg_over_time(container_memory_usage_bytes{container=~"occm", namespace="$namespace"}[2m]))

8.9 OpenSSL CLI Duration (occm_cmp_cli_durations)

Table 8-9 OpenSSL CLI Duration (occm_cmp_cli_durations)

Field Details
Description CMP cli time taken in between request and response from CA
Expression

Used to show the duration of openssl cmp calls

occm_cmp_cli_durations_bucket{namespace="occm-ns", uuid="fdsfds-9880-fsd99"}

8.10 Number of requests sent to the CA

Table 8-10 Number of requests sent to the CA

Field Details
Description Metric will peg when request cmd prepared and send to CA for generate certificate.
Expression count(occm_cmp_requests_total{namespace="$namespace"}

8.11 Number of responses received from CA

Table 8-11 Number of responses received from CA

Field Details
Description Metric will peg when response received from CA for generate certificate.
Expression count(occm_cmp_responses_total{namespace="occm-ns"})

8.12 Number of responses based on response code from CA

Table 8-12 Number of responses based on response code from CA

Field Details
Description Metric will peg when response received from CA for generate certificate.
Expression

count(occm_cmp_responses_total{namespace="occm-ns", statusCode="OK", status = "SUCCESS"}) or

count(occm_cmp_responses_total{namespace="occm-ns", statusCode="ERR_CMP_COMMAND_FAILED", status = "FAILED"})

8.13 Type of request sent to CA

Table 8-13 Type of request sent to CA

Field Details
Description Metric will peg when request cmd prepared and send to CA for generate certificate.
Expression

count(occm_cmp_requests_total{namespace="occm-ns", requestType="ir"}) or

count(occm_cmp_requests_total{namespace="occm-ns", requestType="kur"})

8.14 Number of certificates issued by CA

Table 8-14 Number of certificates issued by CA

Field Details
Description Metric will peg when response received from CA for generate certificate.
Expression count(occm_cmp_responses_total{namespace="occm-ns", status = "SUCCESS", statusCode = "OK"})

8.15 Number of CSRs denied by CA or TLS handshake failures or HTTPs connection failures during CA connection

Table 8-15 Number of CSRs denied by CA or TLS handshake failures or HTTPs connection failures during CA connection

Field Details
Description Metric will peg when response received from CA for generate certificate.
Expression count(occm_cmp_responses_total{namespace="occm-ns", status = "FAILED"})

or

count(occm_cmp_responses_total{namespace="occm-ns", statusCode="ERR_CMP_COMMAND_FAILED", status="FAILED"})

8.16 Error while writing the key, certificate, or chain in the Kubernetes secrets

Table 8-16 Error while writing the key, certificate, or chain in the Kubernetes secrets

Field Details
Description Metric will peg when cert renew or create worker complete its process
Expression occm_cert_request_status_total{namespace="occm-ns", errorReason= "ERR_SECRET_FAILED"}

8.17 Unable to access or read from Kubernetes secrets

Table 8-17 Unable to access or read from Kubernetes secrets

Field Details
Description Metric will peg when cert renew or create worker complete its process
Expression occm_cert_request_status_total{namespace="occm-ns", errorReason= "ERR_SECRET_EXIST"}

8.18 Check Renewed Certificate

Table 8-18 Check Renewed Certificate

Field Details
Description Metric will peg when cert renew or create worker complete its process
Expression occm_cert_request_status_total{namespace="occm-ns", operationType="RENEW"}

8.19 Certificate Error and Warnings

Table 8-19 Certificate Error and Warnings

Field Details
Description List of Certificates having Error and Warnings for duration of 5 mins
Expression rate(occm_cert_request_status_total{namespace="occm-ns", errorReason!="OK"}[5m])

OCCM KPI Dashboard

Figure 8-3 Certificate Error and Warnings


Certificate Error and Warnings

Displayed Columns

  1. Cert Name - Certificate Name
  2. UUID - Certificate UUID
  3. Operation - Certificate Operation Type (CREATE or RENEW)
  4. Reason - Error code indicating Certificate Error or Warning Reason
  5. Issuer - Issuer Name linked to the Certificate