8 OCCM KPIs
This section describes the KPIs available for OCCM.
8.1 CMP Identity (OCCM) Certificate Expiry Time
Table 8-1 CMP Identity (OCCM) Certificate Expiry Time
| Field | Details |
|---|---|
| Description |
CMP Identity (OCCM) certificate expiry time to list Certificate Name and Expiry Date. |
| Expression |
OCCM dashboard in grafana will show CMP Identity (OCCM) Certificate Expiry Time panel with columns. Table visualization listing Expires, NF, Certificate Name, Expiry Date. Expires column uses color coding to indicate near expiry status. all:occm_cmp_identity_cert_expiration_seconds{namespace="$namespace"} * 1000 != 0 Expires column:((occm_cmp_identity_cert_expiration_seconds{namespace="$namespace"} != 0)-time())*1000 |
OCCM KPI Dashboard
Figure 8-1 CMP Identity (OCCM) Certificate Expiry Time
Color coding description:-
Red (Critical):- Certificate expiring within 0 <= 7 days Or Certificate expired <= 0 days
Light Red(Major):- Certificate expiring within > 7 <= 30 days
Orange (Minor):- Certificate expiring within > 30 <= 90
Yellow :- Certificate expiring within > 90 <= 180
Green :- Certificates not expiring sooner
8.2 End Entity (NF) Certificate Expiry Time
Table 8-2 End Entity (NF) Certificate Expiry Time
| Field | Details |
|---|---|
| Description |
End Entity (NF) certificate expiry time to list Certificate Name and Expiry Date. |
| Expression |
OCCM dashboard in grafana will show End Entity (NF) Certificate Expiry Time panel with columns. Table visualization listing Expires, NF, Certificate Name, Expiry Date. Expires column uses color coding to indicate near expiry status. all:occm_end_entity_cert_expiration_seconds{namespace="$namespace"} * 1000 != 0 Expires column: ((occm_end_entity_cert_expiration_seconds{namespace="$namespace"} != 0)-time())*1000 |
OCCM KPI Dashboard
Figure 8-2 End Entity (NF) Certificate Expiry Time
Color coding description:-
Red (Critical):- Certificate expiring within 0 <= 7 days Or Certificate expired <= 0 days
Light Red(Major):- Certificate expiring within > 7 <= 30 days
Orange (Minor):- Certificate expiring within > 30 <= 90
Yellow :- Certificate expiring within > 90 <= 180
Green :- Certificates not expiring sooner
8.3 CMP Identity (OCCM) Certificate Readiness Status
Table 8-3 CMP Identity (OCCM) Certificate Readiness Status
| Field | Details |
|---|---|
| Description | CMP Identity (OCCM) Certificate Readiness Status to indicate if number of Ready and Failed Certificates. |
| Expression |
OCCM dashboard in grafana shows the CMP Identity (OCCM) Certificate Readiness Status panel panel gauge visualization to indicate if number of Ready and Failed Certificates Creating:count(occm_cmp_identity_cert_status{namespace="$namespace"} == 1) (Color:Orange) Ready:count(occm_cmp_identity_cert_status{namespace="$namespace"} == 2) (Color:Green) Failed:count(occm_cmp_identity_cert_status{namespace="$namespace"} == 3) (Color:Red) Waiting:count(occm_cmp_identity_cert_status{namespace="$namespace"} == 8) (Color:Light Orange) Expired:count(occm_cmp_identity_cert_status{namespace="$namespace"} == 7) (Color:Red) During bulk certificate migration, the intermediate status displayed on the gauge includes duplicate certificate and might show an increased number. However, once the process is completed the eventual state will be consistent and will show the correct count of all the certificates. |
OCCM KPI Dashboard
Figure 8-3 CMP Identity (OCCM) Certificate Readiness Status
Creating: Orange
Ready: Green
Failed: Red
Waiting: Light Orange
Expired : Red
8.4 End Entity (NF) Certificate Readiness Status
Table 8-4 End Entity (NF) Certificate Readiness Status
| Field | Details |
|---|---|
| Description | End Entity (NF) Certificate Readiness Status to indicate if number of Ready and Failed Certificates. |
| Expression |
OCCM dashboard in grafana shows the End Entity (NF) Certificate Readiness Status panel gauge visualization to indicate if number of Ready and Failed Certificates Creating:count(occm_end_entity_cert_status{namespace="$namespace"} == 1) (Color:Orange) Ready:count(occm_end_entity_cert_status{namespace="$namespace"} == 2) (Color:Green) Failed:count(occm_end_entity_cert_status{namespace="$namespace"} == 3) (Color:Red) Waiting:count(occm_end_entity_cert_status{namespace="$namespace"} == 8) (Color:Light Orange) Expired:count(occm_end_entity_cert_status{namespace="$namespace"} == 7) (Color:Red) During bulk certificate migration, the intermediate status displayed on the gauge includes duplicate certificate and might show an increased number. However, once the process is completed the eventual state will be consistent and will show the correct count of all the certificates. |
OCCM KPI Dashboard
Figure 8-4 End Entity (NF) Certificate Readiness Status
Creating: Orange
Ready: Green
Failed: Red
Waiting: Light Orange
Expired : Red
8.5 CMP Request
Table 8-5 CMP Request
| Field | Details |
|---|---|
| Description | Total CMP requests initiated from OCCM towards CA per NF |
| Expression |
OCCM dashboard in grafana will show CMP Request panel which is total CMP requests per NF. all:sum(rate(occm_cmp_requests_total{namespace="$namespace"}[2m])) SCP:sum(rate(occm_cmp_requests_total{namespace="$namespace", nfType=~"SCP|scp"}[2m])) NRF:sum(rate(occm_cmp_requests_total{namespace="$namespace", nfType=~"NRF|nrf"}[2m])) |
8.6 CMP Responses
Table 8-6 CMP Responses
| Field | Details |
|---|---|
| Description | Total CMP responses received from CA per NF by OCCM |
| Expression |
OCCM dashboard in grafana will show CMP Response panel which is total CMP responses per NF. all:sum(rate(occm_cmp_responses_total{namespace="$namespace"}[2m])) SCP:sum(rate(occm_cmp_responses_total{namespace="$namespace", nfType=~"SCP|scp"}[2m])) NRF:sum(rate(occm_cmp_responses_total{namespace="$namespace", nfType=~"NRF|nrf"}[2m])) |
8.7 Configuration Requests
Table 8-7 Configuration Requests
| Field | Details |
|---|---|
| Description | Total Issuer, Certificate Configuration, and Bulk Certificate Migration requests. |
| Expression |
OCCM dashboard in grafana will show Config Requests panel. Total Issuer, Certificate configuration, and Bulk Certificate Migration requests. all:sum(rate(occm_config_http_requests_total{namespace="$namespace"}[2m])) SCP certs:sum(rate(occm_config_http_requests_total{namespace="$namespace", uri=~".*/certs.*", nfType=~"SCP|scp"}[2m])) NRF certs:sum(rate(occm_config_http_requests_total{namespace="$namespace", uri=~".*/certs.*", nfType=~"NRF|nrf"}[2m])) issuers:sum(rate(occm_config_http_requests_total{namespace="$namespace", uri=~".*/issuers.*"}[2m])) Bulk cert migrations:sum(rate(occm_config_http_requests_total{namespace="$namespace", uri=~".*/certs/bulk-migrate.*"}[2m])) |
8.8 Configuration Responses
Table 8-8 Configuration Responses
| Field | Details |
|---|---|
| Description | Total Issuer, Certificate Configuration, and Bulk Certificate Migration responses. |
| Expression |
OCCM dashboard in grafana will show Config Responses panel. Total Issuer, Certificate configuration, and Bulk Certificate Migration responses. all:sum(rate(occm_config_http_responses_total{namespace="$namespace"}[2m])) SCP certs:sum(rate(occm_config_http_responses_total{namespace="$namespace", uri=~".*/certs.*", nfType=~"SCP|scp"}[2m])) NRF certs:sum(rate(occm_config_http_responses_total{namespace="$namespace", uri=~".*/certs.*", nfType=~"NRF|nrf"}[2m])) issuers:sum(rate(occm_config_http_responses_total{namespace="$namespace", uri=~".*/issuers.*"}[2m])) Bulk cert migrations:sum(rate(occm_config_http_responses_total{{namespace="$namespace", uri=~".*/certs/bulk-migrate.*"}[2m])) |
8.9 CPU Usage
Table 8-9 CPU Usage
| Field | Details |
|---|---|
| Description | CPU usage of OCCM pod |
| Expression |
Time series indicates CPU usage of OCCM pod. sum(rate(container_cpu_usage_seconds_total{image!="",namespace="$namespace", pod=~"occm-.*."}[2m])) by(pod) |
8.10 Memory Usage
Table 8-10 Memory Usage
| Field | Details |
|---|---|
| Description | Memory usage of OCCM pod |
| Expression |
Time series indicates Memory usage of OCCM pod. (avg_over_time(container_memory_usage_bytes{container=~"occm", namespace="$namespace"}[2m])) |
8.11 OpenSSL CLI Duration (occm_cmp_cli_durations_seconds)
Table 8-11 OpenSSL CLI Duration (occm_cmp_cli_durations_seconds)
| Field | Details |
|---|---|
| Description | The time taken by CMP CLI between request and response from CA. |
| Expression |
Used to show the duration of openssl cmp calls occm_cmp_cli_durations_seconds{namespace="occm-ns", uuid="fdsfds-9880-fsd99"} |
8.12 Number of requests sent to the CA
Table 8-12 Number of requests sent to the CA
| Field | Details |
|---|---|
| Description | Metric will peg when request cmd prepared and send to CA for generate certificate. |
| Expression | count(occm_cmp_requests_total{namespace="$namespace"} |
8.13 Number of responses received from CA
Table 8-13 Number of responses received from CA
| Field | Details |
|---|---|
| Description | Metric will peg when response received from CA for generate certificate. |
| Expression | count(occm_cmp_responses_total{namespace="occm-ns"}) |
8.14 Number of responses based on response code from CA
Table 8-14 Number of responses based on response code from CA
| Field | Details |
|---|---|
| Description | Metric will peg when response received from CA for generate certificate. |
| Expression |
count(occm_cmp_responses_total{namespace="occm-ns", statusCode="OK", status = "SUCCESS"}) or count(occm_cmp_responses_total{namespace="occm-ns", statusCode="ERR_CMP_COMMAND_FAILED", status = "FAILED"}) |
8.15 Type of request sent to CA
Table 8-15 Type of request sent to CA
| Field | Details |
|---|---|
| Description | Metric will peg when request cmd prepared and send to CA for generate certificate. |
| Expression |
count(occm_cmp_requests_total{namespace="occm-ns", requestType="ir"}) or count(occm_cmp_requests_total{namespace="occm-ns", requestType="kur"}) |
8.16 Number of certificates issued by CA
Table 8-16 Number of certificates issued by CA
| Field | Details |
|---|---|
| Description | Metric will peg when response received from CA for generate certificate. |
| Expression | count(occm_cmp_responses_total{namespace="occm-ns", status = "SUCCESS", statusCode = "OK"}) |
8.17 Number of CSRs denied by CA or TLS handshake failures or HTTPs connection failures during CA connection
Table 8-17 Number of CSRs denied by CA or TLS handshake failures or HTTPs connection failures during CA connection
| Field | Details |
|---|---|
| Description | Metric will peg when response received from CA for generate certificate. |
| Expression | count(occm_cmp_responses_total{namespace="occm-ns",
status =
"FAILED"})
or count(occm_cmp_responses_total{namespace="occm-ns", statusCode="ERR_CMP_COMMAND_FAILED", status="FAILED"}) |
8.18 Error while writing the key, certificate, or chain in the Kubernetes secrets
Table 8-18 Error while writing the key, certificate, or chain in the Kubernetes secrets
| Field | Details |
|---|---|
| Description | Metric will peg when cert renew or create worker complete its process |
| Expression | occm_cert_request_status_total{namespace="occm-ns", errorReason= "ERR_SECRET_FAILED"} |
8.19 Unable to access or read from Kubernetes secrets
Table 8-19 Unable to access or read from Kubernetes secrets
| Field | Details |
|---|---|
| Description | Metric will peg when cert renew or create worker complete its process |
| Expression | occm_cert_request_status_total{namespace="occm-ns", errorReason= "ERR_SECRET_EXIST"} |
8.20 Check Renewed Certificate
Table 8-20 Check Renewed Certificate
| Field | Details |
|---|---|
| Description | Metric will peg when cert renew or create worker complete its process |
| Expression | occm_cert_request_status_total{namespace="occm-ns", operationType="RENEW"} |
8.21 Certificate Error and Warnings
Table 8-21 Certificate Error and Warnings
| Field | Details |
|---|---|
| Description | List of certificates having Error and Warnings for duration of 5 mins. |
| Expression | rate(occm_cert_request_status_total{namespace="occm-ns", errorReason!~"OK.*"}[5m]) > 0 |
OCCM KPI Dashboard
Figure 8-5 Certificate Error and Warnings
Displayed Columns
- Cert Name - Certificate Name
- UUID - Certificate UUID
- Operation - Certificate Operation Type (CREATE or RENEW)
- Reason - Error code indicating Certificate Error or Warning Reason
- Issuer - Issuer Name linked to the Certificate
8.22 Bulk Certificate Migration Error
Table 8-22 Bulk Certificate Migration Error
| Field | Details |
|---|---|
| Description | List of certificates that are failed during bulk migration of certificate. |
| Expression | occm_cert_request_status_total{namespace="$namespace", errorReason="ERR_BULK_CERT_MIGRATION"} |
OCCM KPI Dashboard
Figure 8-6 Bulk Certificate Migration Error
Displayed Columns
- Cert Name - Name of the certificate whose migration failed.
- UUID - UUID of the certificate whose migration failed.
- Issuer - Name of the destination issuer to which the certificate is to be migrated.
- Bulk Migration UUID - The UUID of the bulk certificate migration.
Note:
Filtering and sorting are available on all the fields. To get the certificates that failed the migration, you must filter the rows based on the bulk migration UUID.