6 CNC Console Alerts

This section provides information about CNC Console Alerts.

Note:

For OCI:

The only section applicable for OCI is CNC Console Alerts on OCI.

Note:

Alert file is present in the Scripts directory. occncc_csar_<version>.zip can be downloaded from MOS. Unzip the file to get occncc_alertrules_<version>.yaml file.
  • Review the occncc_alerting_rules_promha_<version>.yaml file and edit the value of the parameters in the occncc_alerting_rules_promha_<version>.yaml file (if needed to be changed from default values) before configuring the alerts.
  • kubernetes_namespace is configured as kubernetes namespace in which CNCC is deployed. Default value is cncc. Update the occncc_alertrules_<version>.yaml file to reflect the correct CNCC kubernetes namespace.

Two sample Alert files are provided, one for supporting CNE 1.8 or lower and second one supporting CNE Prometheus HA.

  • CNC Console Alert Rules file: occncc_alertrules_<version>.yaml file.
  • CNC Console Alert Rules file supporting CNE Prometheus HA: occncc_alerting_rules_promha_<version>.yaml file.

6.1 CNC Console IAM Alerts

This section provides information about CNC Console IAM Alerts.

Parent topic: CNC Console Alerts

6.1.1 CnccIamTotalIngressTrafficRateAboveMinorThreshold

Table 6-1 CnccIamTotalIngressTrafficRateAboveMinorThreshold

Field Details
Trigger Condition The total CNCC IAM Ingress Message rate has crossed the configured minor threshold of 700 TPS.

Default value of this alert trigger point in occncc_alertrules_<version>.yamlis when CNCC IAM Ingress Rate crosses 70 % of 1000 (Maximum ingress request rate)

Severity Minor
Alert details provided

Description: CNCC IAM Ingress traffic Rate is above the configured minor threshold i.e. 700 requests per second (current value is: {{ $value }})

For CNE with Prometheus HA Operator:

summary: 'namespace: {{$labels.namespace}}, podname: {{$labels.pod}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Traffic Rate is above 70 Percent of Max requests per second(1000)'

For CNE without Prometheus Operator :

summary: 'namespace: {{$labels.kubernetes_namespace}}, podname: {{$labels.kubernetes_pod_name}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Traffic Rate is above 70 Percent of Max requests per second(1000)'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.7001
Metric Used oc_ingressgateway_http_requests_total
Resolution

The alert is cleared either when the total Ingress Traffic rate falls below the Minor threshold or when the total traffic rate crosses the Major threshold, in which case the CnccIamTotalIngressTrafficRateAboveMajorThreshold alert is raised.

Note: The threshold is configurable in the occncc_alertrules_<version>.yaml file.

Steps:

  1. Reassess why the CNCC IAM is receiving additional traffic.
  2. If this is unexpected, contact My Oracle Support.

Parent topic: CNC Console IAM Alerts

6.1.2 CnccIamTotalIngressTrafficRateAboveMajorThreshold

Table 6-2 CnccIamTotalIngressTrafficRateAboveMajorThreshold

Field Details
Trigger Condition

The total CNCC IAM Ingress Message rate has crossed the configured major threshold of 800 TPS.

Default value of this alert trigger point ino ccncc_alertrules_<version>.yaml is when CNCC IAM Ingress Rate crosses 80 % of 1000 (Maximum ingress request rate)

Severity Major
Alert details provided

Description: 'CNCC IAM Ingress traffic Rate is above the configured major threshold i.e. 800 requests per second (current value is: {{ $value }})'

For CNE with Prometheus HA Operator:

summary: 'namespace: {{$labels.namespace}}, podname: {{$labels.pod}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Traffic Rate is above 80 Percent of Max requests per second(1000)'

For CNE without Prometheus Operator :

summary: 'namespace: {{$labels.kubernetes_namespace}}, podname: {{$labels.kubernetes_pod_name}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Traffic Rate is above 80 Percent of Max requests per second(1000)'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.7001
Metric Used oc_ingressgateway_http_requests_total
Resolution

The alert is cleared when the total Ingress Traffic rate falls below the Major threshold or when the total traffic rate crosses the Critical threshold, in which case the CnccIamTotalIngressTrafficRateAboveCriticalThreshold alert is raised.

Note: The threshold is configurable in the occncc_alertrules_<version>.yaml file.

Steps:

  1. Reassess why the CNCC IAM is receiving additional traffic.
  2. If this is unexpected, contact My Oracle Support.

Parent topic: CNC Console IAM Alerts

6.1.3 CnccIamTotalIngressTrafficRateAboveCriticalThreshold

Table 6-3 CnccIamTotalIngressTrafficRateAboveCriticalThreshold

Field Details
Trigger Condition

The total CNCC IAM Ingress Message rate has crossed the configured critical threshold of 900TPS. Default value of this alert trigger point in occncc_alertrules_<version>.yaml is when CNCC IAM Ingress Rate crosses 90 % of 1000 (Maximum ingress request rate)

Severity Critical
Alert details provided

Description:CNCC IAM Ingress traffic Rate is above the configured critical threshold, that is, 900 requests per second (current value is: {{ $value }})

For CNE with Prometheus HA Operator:

summary: 'namespace: {{$labels.namespace}}, podname: {{$labels.pod}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Traffic Rate is above 90 Percent of Max requests per second(1000)'

For CNE without Prometheus Operator:

summary: 'namespace: {{$labels.kubernetes_namespace}}, podname: {{$labels.kubernetes_pod_name}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Traffic Rate is above 90 Percent of Max requests per second(1000)'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.7001
Metric Used oc_ingressgateway_http_requests_total
Resolution

The alert is cleared when the Ingress Traffic rate falls below the Critical threshold.

Note: The threshold is configurable in the occncc_alertrules_<version>.yaml file.

Steps:

  1. Reassess why the CNCC IAM is receiving additional traffic.
  2. If this is unexpected, contact My Oracle Support.

Parent topic: CNC Console IAM Alerts

6.1.4 CnccIamMemoryUsageCrossedMinorThreshold

Table 6-4 CnccIamMemoryUsageCrossedMinorThreshold

Field Details
Trigger Condition A pod has reached the configured minor threshold( 70%) of its memory resource limits.
Severity Minor
Alert details provided

Description: 'CNCC IAM Memory Usage for pod {{ $labels.pod }} has crossed the configured minor threshold (70%) (value={{ $value }}) of its limit.'

Summary: 'namespace: {{$labels.namespace}}, podname: {{$labels.pod}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }} : Memory Usage of pod exceeded 70% of its limit.'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.7002
Metric Used

container_memory_usage_bytes,

kube_pod_container_resource_limits

Note:This is a Kubernetes metric used for instance availability monitoring. If the metric is not available, use a similar metric as exposed by the monitoring system.

Resolution

The alert gets cleared when the memory utilization falls below the Minor Threshold or crosses the major threshold, in which case CnccIamMemoryUsageCrossedMajorThreshold alert is raised.

Note: The threshold is configurable in the occncc_alertrules_<version>.yaml file.

If guidance is required, contact My Oracle Support.

Parent topic: CNC Console IAM Alerts

6.1.5 CnccIamMemoryUsageCrossedMajorThreshold

Table 6-5 CnccIamMemoryUsageCrossedMajorThreshold

Field Details
Trigger Condition A pod has reached the configured major threshold( 80%) of its memory resource limits.
Severity Major
Alert details provided

Description: 'CNCC IAM Memory Usage for pod {{ $labels.pod }} has crossed the configured major threshold (80%) (value = {{ $value }}) of its limit.'

Summary: 'namespace: {{$labels.namespace}}, podname: {{$labels.pod}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }} : Memory Usage of pod exceeded 80% of its limit.'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.7002
Metric Used

container_memory_usage_bytes,

kube_pod_container_resource_limits

Note: This is a Kubernetes metric used for instance availability monitoring.If the metric is not available, use a similar metric as exposed by the monitoring system.

Resolution The alert gets cleared when the memory utilization falls below the Major Threshold or crosses the critical threshold, in which case CnccIamMemoryUsageCrossedCriticalThreshold alert shall be raised.

Note: The threshold is configurable in the occncc_alertrules_<version>.yaml file.

If guidance is required, contact My Oracle Support.

Parent topic: CNC Console IAM Alerts

6.1.6 CnccIamMemoryUsageCrossedCriticalThreshold

Table 6-6 CnccIamMemoryUsageCrossedCriticalThreshold

Field Details
Trigger Condition A pod has reached the configured critical threshold ( 90% ) of its memory resource limits
Severity Critical
Alert details provided

Description: 'CNCC IAM Memory Usage for pod {{ $labels.pod }} has crossed the configured critical threshold (90%) (value = {{ $value }}) of its limit.'

Summary: 'namespace: {{$labels.namespace}}, podname: {{$labels.pod}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }} : Memory Usage of pod exceeded 90% of its limit.'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.7002
Metric Used

container_memory_usage_bytes,

kube_pod_container_resource_limits

Note: This is a Kubernetes metric used for instance availability monitoring. If the metric is not available, use a similar metric as exposed by the monitoring system.

Resolution The alert gets cleared when the memory utilization falls below the Critical Threshold.

Note : The threshold is configurable in the occncc_alertrules_<version>.yaml file.

If guidance is required, contact My Oracle Support.

Parent topic: CNC Console IAM Alerts

6.1.7 CnccIamTransactionErrorRateAbove0.1Percent

Table 6-7 CnccIamTransactionErrorRateAbove0.1Percent

Field Details
Trigger Condition The number of failed transactions is above 0.1 percent of the total transactions.
Severity Warning
Alert details provided Description: 'CNCC IAM transaction Error rate is above 0.1 Percent of Total Transactions (current value is {{ $value }})'

Summary: 'CNCC IAM transaction Error Rate detected above 0.1 Percent of Total Transactions'
OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.7003
Metric Used oc_ingressgateway_http_responses_total
Resolution The alert is cleared when the number of failed transactions is below 0.1% of the total transactions or when the number of failed transactions crosses the 1% threshold in which case the CnccIamTransactionErrorRateAbove1Percent is raised.

Steps:
  1. Check the Service specific metrics to understand the specific service request errors.
  2. If guidance is required, contact My Oracle Support.

Parent topic: CNC Console IAM Alerts

6.1.8 CnccIamTransactionErrorRateAbove1Percent

Table 6-8 CnccIamTransactionErrorRateAbove1Percent

Field Details
Trigger Condition The number of failed transactions is above 1 percent of the total transactions.
Severity Warning
Alert details provided Description: 'CNCC IAM transaction Error rate is above 1 Percent of Total Transactions (current value is {{ $value }})'

Summary: 'CNCC IAM transaction Error Rate detected above 1 Percent of Total Transactions'
OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.7003
Metric Used oc_ingressgateway_http_responses_total
Resolution The alert is cleared when the number of failed transactions is below 1% of the total transactions or when the number of failed transactions crosses the 10% threshold in which case the CnccIamTransactionErrorRateAbove10Percent is raised.

Steps:

  1. Check the Service specific metrics to understand the specific service request errors.
  2. If guidance is required, contact My Oracle Support.

Parent topic: CNC Console IAM Alerts

6.1.9 CnccIamTransactionErrorRateAbove10Percent

Table 6-9 CnccIamTransactionErrorRateAbove10Percent

Field Details
Trigger Condition The number of failed transactions is above 10 percent of the total transactions.
Severity Minor
Alert details provided

Description: CNCC IAM transaction Error rate is above 10 Percent of Total Transactions (current value is {{ $value }})'

Summary: 'CNCC IAM transaction Error rate is above 10 Percent of Total Transactions (current value is {{ $value }})'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.7003
Metric Used oc_ingressgateway_http_responses_total
Resolution The alert is cleared when the number of failed transactions is below 10% of the total transactions or when the number of failed transactions crosses the 25% threshold in which case the CnccIamTransactionErrorRateAbove25Percent is raised.

Steps:

  1. Check the Service specific metrics to understand the specific service request errors.
  2. If guidance is required, contact My Oracle Support.

Parent topic: CNC Console IAM Alerts

6.1.10 CnccIamTransactionErrorRateAbove25Percent

Table 6-10 CnccIamTransactionErrorRateAbove25Percent

Field Details
Trigger Condition The number of failed transactions is above 25 percent of the total transactions.
Severity Major
Alert details provided

Description: 'CNCC IAM transaction Error Rate detected above 25 Percent of Total Transactions (current value is {{ $value }})'

Summary: 'CNCC IAM transaction Error Rate detected above 25 Percent of Total Transactions'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.7003
Metric Used oc_ingressgateway_http_responses_total
Resolution

TThe alert is cleared when the number of failed transactions are below 25% of the total transactions or when the number of failed transactions cross the 50% threshold in which case the CnccIamTransactionErrorRateAbove50Percent is raised.

Steps:

  1. Check the Service specific metrics to understand the specific service request errors.
  2. If guidance is required, contact My Oracle Support.

Parent topic: CNC Console IAM Alerts

6.1.11 CnccIamTransactionErrorRateAbove50Percent

Table 6-11 CnccIamTransactionErrorRateAbove50Percent

Field Details
Trigger Condition The number of failed transactions is above 50 percent of the total transactions.
Severity Critical
Alert details provided

Description: The number of failed transactions is above 50 percent of the total transactions.

Summary: 'CNCC IAM transaction Error Rate detected above 50 Percent of Total Transactions'.

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.7003
Metric Used oc_ingressgateway_http_responses_total
Resolution

The alert is cleared when the number of failed transactions is below 50 percent of the total transactions.

The threshold is configurable in the occncc_alertrules_<version>.yaml file.

Steps:

  1. Check the Service specific metrics to understand the specific service request errors.
  2. If guidance is required, contact My Oracle Support.

Parent topic: CNC Console IAM Alerts

6.1.12 CnccIamIngressGatewayServiceDown

Table 6-12 CnccIamIngressGatewayServiceDown

Field Details
Trigger Condition

The pods of the CNCC IAM Ingress Gateway microservice is available.
Severity Critical
Alert details provided

Description:'CNCC IAM Ingress-Gateway service InstanceIdentifier=~".*cncc-iam_ingressgateway" is down'

For CNE with Prometheus HA Operator:

summary: 'namespace: {{$labels.namespace}}, podname: {{$labels.pod}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }} : Ingress-gateway service down'

For CNE without Prometheus Operator:

summary: 'namespace: {{$labels.kubernetes_namespace}}, podname: {{$labels.kubernetes_pod_name}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }} : Ingress-gateway service down'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.7004
Metric Used

'up'

Note: This is a Prometheus metric used for instance availability monitoring. If this metric is not available, use a similar metric as exposed by the monitoring system.

Resolution

The alert is cleared when the cncc-iam_ingressgateway service is available.

Steps:

  1. Check the orchestration logs of cncc-iam_ingressgateway service and check for liveness or readiness probe failures.
  2. Refer to the application logs on Kibana and filter based on cncc-iam_ingressgateway service names. Check for ERROR WARNING logs related to thread exceptions.
  3. Depending on the failure reason, take the resolution steps.
  4. In case the issue persists, contact My Oracle Support.

Parent topic: CNC Console IAM Alerts

6.1.13 CnccIamFailedLogin

Table 6-13 CnccIamFailedLogin

Field Details
Trigger Condition

The count of failed login attempts in CNCC-IAM by a user goes above '3'
Severity Warning
Alert details provided

Description:'{{ $value }} failed Login attempts have been detected in CNCC IAM for user {{$labels.UserName}}, the configured threshold value is 3 failed login attempts for every 5 min'

For CNE with Prometheus HA Operator:

summary: 'namespace: {{$labels.namespace}}, podname: {{$labels.pod}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: failed login attempts are more than the configured threshold value'

For CNE without Prometheus Operator:

summary: 'namespace: {{$labels.kubernetes_namespace}}, podname: {{$labels.kubernetes_pod_name}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: failed login attempts are more than the configured threshold value'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.7005
Metric Used oc_ingressgateway_http_responses_total
Resolution

The alert gets cleared when the total failed login attempts for a particular user goes below the threshold value (default value is '3') in the last 5 min (default value is 5 m).

Note: The threshold and time is configurable in the alerts.yaml file.

If guidance is required, contact My Oracle Support.

Parent topic: CNC Console IAM Alerts

6.1.14 AdminUserCreation

Table 6-14 AdminUserCreation

Field Details
Trigger Condition

If a new admin account is created in the last 5 min

Severity Warning
Alert details provided

For CNE with Prometheus HA Operator:

Description: '{{ $value }} admin users have been created by {{$labels.UserName}} '

summary: 'namespace: {{$labels.namespace}}

summary: {{$labels.pod}}, user: {{$labels.UserName}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Admin users have been created '

For CNE without Prometheus Operator:

Description: '{{ $value }} admin users have been created by {{$labels.UserName}} '

summary: 'namespace: {{$labels.kubernetes_namespace}}

summary: {{$labels.kubernetes_pod_name}}, user: {{$labels.UserName}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Admin users have been created '

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.7006
Metric Used oc_ingressgateway_http_requests_total
Resolution

The alert gets cleared when the total failed login attempts for a particular user go below the threshold value (default value is '3') in the last 5 min (default value is 5 m)

Note: The threshold and time is configurable in the occncc_alertrules_<version>.yaml file.

Login to admin GUI and review the user created.

If guidance is required, contact My Oracle Support.

Parent topic: CNC Console IAM Alerts

6.1.15 CnccIamAccessTokenFailure

Table 6-15 CnccIamAccessTokenFailure

Field Details
Trigger Condition

If the count of failed token for CNCC-IAM goes above configured value of '3'
Severity Warning
Alert details provided

Description: 'CNCC Iam Access Token Failure count is above the configured value i.e. 3 for every 5 min. Failed access token request count per second is (current value is: {{ $value }})'

For CNE with Prometheus HA Operator:

summary: 'namespace: {{$labels.namespace}}, podname: {{$labels.pod}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Access Token Failure count is above the configured threshold value'

For CNE without Prometheus Operator:

summary: 'namespace: {{$labels.kubernetes_namespace}}, podname: {{$labels.pod}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Access Token Failure count is above the configured threshold value'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.7007
Metric Used oc_ingressgateway_http_responses_total
Resolution

The alert gets cleared when the total failed tokens for a particular user go below the threshold value (default value is '3') in the last 5 min (default value is 5 m)

Note: The threshold and time is configurable in the occncc_alertrules_<version>.yaml file.

If guidance is required, contact My Oracle Support.

Parent topic: CNC Console IAM Alerts

6.2 CNC Console Core Alerts

This section provides the information about CNC Console Core Alerts.

Parent topic: CNC Console Alerts

6.2.1 CnccCoreTotalIngressTrafficRateAboveMinorThreshold

Table 6-16 CnccCoreTotalIngressTrafficRateAboveMinorThreshold

Field Details
Trigger Condition

The total CNCC Core Ingress Message rate has crossed the configured minor threshold of 700 TPS.

Default value of this alert trigger point in cncc_alert_rules.yaml is when CNCC Core Ingress Rate crosses 70 % of 1000 (Maximum ingress request rate)
Severity Minor
Alert details provided

Description: 'CNCC Core Ingress traffic Rate is above the configured minor threshold i.e. 700 requests per second (current value is: {{ $value }})'

For CNE with Prometheus HA Operator:

summary: 'namespace: {{$labels.namespace}}, podname: {{$labels.pod}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Traffic Rate is above 70 Percent of Max requests per second(1000)'

For CNE without Prometheus Operator:

summary: 'namespace: {{$labels.kubernetes_namespace}}, podname: {{$labels.kubernetes_pod_name}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Traffic Rate is above 70 Percent of Max requests per second(1000)'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.8001
Metric Used oc_ingressgateway_http_requests_total
Resolution

The alert is cleared either when the total Ingress traffic rate falls below the minor threshold or when the total traffic rate crosses the major threshold, in which case the CnccCoreTotalIngressTrafficRateAboveMajorThreshold alert is raised.

Note: The threshold is configurable in the occncc_alertrules_<version>.yaml file.

Steps:

  1. Reassess why the CNCC Core is receiving additional traffic.
  2. If this is unexpected, contact My Oracle Support.

Parent topic: CNC Console Core Alerts

6.2.2 CnccCoreTotalIngressTrafficRateAboveMajorThreshold

Table 6-17 CnccCoreTotalIngressTrafficRateAboveMajorThreshold

Field Details
Trigger Condition The total CNCC Core Ingress Message rate has crossed the configured major threshold of 800 TPS.

Default value of this alert trigger point in cncc_alert_rules.yaml is when CNCC Core Ingress Rate crosses 80 % of 1000 (Maximum ingress request rate)
Severity Major
Alert details provided

Description: 'CNCC Core Ingress traffic Rate is above the configured major threshold i.e. 800 requests per second (current value is: {{ $value }})'

For CNE with Prometheus HA Operator:

summary: 'namespace: {{$labels.namespace}}, podname: {{$labels.pod}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Traffic Rate is above 80 Percent of Max requests per second(1000)'

For CNE without Prometheus Operator :

summary: 'namespace: {{$labels.kubernetes_namespace}}, podname: {{$labels.kubernetes_pod_name}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Traffic Rate is above 80 Percent of Max requests per second(1000)'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.8001
Metric Used oc_ingressgateway_http_requests_total
Resolution The alert is cleared when the total Ingress Traffic ratefalls below the Major threshold or when the total traffic rate cross the Critical threshold, in which case the CnccCoreTotalIngressTrafficRate Above CriticalThreshold.

Note: The threshold is configurable in the alerts.yaml file.

Steps:

  1. Reassess why the CNCC Core is receiving additional traffic.
  2. If this is unexpected, contact My Oracle Support.

Parent topic: CNC Console Core Alerts

6.2.3 CnccCoreTotalIngressTrafficRateAboveCriticalThreshold

Table 6-18 CnccCoreTotalIngressTrafficRateAboveCriticalThreshold

Field Details
Trigger Condition

The total CNCC Core Ingress Message rate has crossed the configured critical threshold of 900TPS.

Default value of this alert trigger point in cncc_alert_rules.yaml is when CNCC Core Ingress Rate crosses 90 % of 1000 (Maximum ingress request rate)
Severity Critical
Alert details provided

Description: 'CNCC Core Ingress traffic Rate is above the configured critical threshold i.e. 900 requests per second (current value is: {{ $value }})'

For CNE with Prometheus HA Operator:

summary: 'namespace: {{$labels.namespace}}, podname: {{$labels.pod}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Traffic Rate is above 90 Percent of Max requests per second(1000)'

For CNE without Prometheus Operator :

summary: 'namespace: {{$labels.kubernetes_namespace}}, podname: {{$labels.kubernetes_pod_name}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Traffic Rate is above 90 Percent of Max requests per second(1000)'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.8001
Metric Used oc_ingressgateway_http_requests_total
Resolution

The alert is cleared when the Ingress Traffic rate falls below the Critical threshold.

Note: The threshold is configurable in the occncc_alertrules_<version>.yaml file.

Steps:

  1. Reassess why the CNCC IAM is receiving additional traffic.
  2. If this is unexpected, contact My Oracle Support.

Parent topic: CNC Console Core Alerts

6.2.4 CnccCoreMemoryUsageCrossedMinorThreshold

Table 6-19 CnccCoreMemoryUsageCrossedMinorThreshold

Field Details
Trigger Condition

A pod has reached the configured minor threshold( 70%) of its memory resource limits.
Severity Minor
Alert details provided

Description: 'CNCC Core Memory Usage for pod {{ $labels.pod }} has crossed the configured minor threshold (70%) (value={{ $value }}) of its limit.'

Summary: 'namespace: {{$labels.namespace}}, podname: {{$labels.pod}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }} : Memory Usage of pod exceeded 70% of its limit.'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.8002
Metric Used

container_memory_usage_bytes

kube_pod_container_resource_limits

Note : This is a kubernetes metric used for instance availability monitoring. If the metric is not available, use the similar metric as exposed by the monitoring system.
Resolution

The alert gets cleared when the memory utilization falls below the Minor Threshold or crosses the major threshold, in which case CnccCoreMemoryUsageCrossedMajorThreshold alert is raised.

Note: The threshold is configurable in the occncc_alertrules_<version>.yaml file.

If guidance is required, contact My Oracle Support.

Parent topic: CNC Console Core Alerts

6.2.5 CnccCoreMemoryUsageCrossedMajorThreshold

Table 6-20 CnccCoreMemoryUsageCrossedMajorThreshold

Field Details
Trigger Condition

A pod has reached the configured major threshold( 80%) of its memory resource limits.
Severity Major
Alert details provided

Description: 'CNCC Core Memory Usage for pod {{ $labels.pod }} has crossed the configured major threshold (80%) (value = {{ $value }}) of its limit.'

Summary: 'namespace: {{$labels.namespace}}, podname: {{$labels.pod}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }} : Memory Usage of pod exceeded 80% of its limit.'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.8002
Metric Used

container_memory_usage_bytes

kube_pod_container_resource_limits

Note : This is a kubernetes metric used for instance availability monitoring. If the metric is not available, use the similar metric as exposed by the monitoring system.
Resolution

The alert gets cleared when the memory utilization falls below the Major Threshold or crosses the critical threshold, in which case CnccCoreMemoryUsageCrossedCriticalThreshold alert is raised

Note: The threshold is configurable in the occncc_alertrules_<version>.yaml file.

If guidance is required, contact My Oracle Support.

Parent topic: CNC Console Core Alerts

6.2.6 CnccCoreMemoryUsageCrossedCriticalThreshold

Table 6-21 CnccCoreMemoryUsageCrossedCriticalThreshold

Field Details
Trigger Condition

A pod has reached the configured critical threshold ( 90% ) of its memory resource limits
Severity Critical
Alert details provided

Description: 'CNCC Core Memory Usage for pod {{ $labels.pod }} has crossed the configured critical threshold (90%) (value = {{ $value }}) of its limit.'

Summary: 'namespace: {{$labels.namespace}}, podname: {{$labels.pod}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }} : Memory Usage of pod exceeded 90% of its limit.'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.8002
Metric Used

container_memory_usage_bytes

kube_pod_container_resource_limits

Note : This is a Kubernetes metric used for instance availability monitoring. If the metric is not available, use the similar metric as exposed by the monitoring system.
Resolution

The alert gets cleared when the memory utilization falls below the Critical Threshold.

Note: The threshold is configurable in the occncc_alertrules_<version>.yaml file.

If guidance is required, contact My Oracle Support.

Parent topic: CNC Console Core Alerts

6.2.7 CnccCoreTransactionErrorRateAbove0.1Percent

Table 6-22 CnccCoreTransactionErrorRateAbove0.1Percent

Field Details
Trigger Condition The number of failed transactions is above 0.1 percent of the total transactions..
Severity Warning
Alert details provided

Description:'CNCC Core transaction Error rate is above 0.1 Percent of Total Transactions (current value is {{ $value }})'

Summary:'CNCC Core transaction Error rate is above 0.1 Percent of Total Transactions (current value is {{ $value }})'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.8003
Metric Used oc_ingressgateway_http_responses_total
Resolution

The alert is cleared when the number of failed transactions are below 0.1% of the total transactions or when the number of failed transactions cross the 1% threshold in which case the CnccCoreTransactionErrorRateAbove1Percent is raised.

The threshold is configurable in the alerts.yaml file.

Steps:

  1. Check the Service specific metrics to understand the specific service request errors.
  2. If guidance is required, contact My Oracle Support.

Parent topic: CNC Console Core Alerts

6.2.8 CnccCoreTransactionErrorRateAbove1Percent

Table 6-23 CnccCoreTransactionErrorRateAbove1Percent

Field Details
Trigger Condition The number of failed transactions is above 1 percent of the total transactions.
Severity Warning
Alert details provided

Description: 'CNCC Core transaction Error rate is above 1 Percent of Total Transactions (current value is {{ $value }})'

Summary:'CNCC Core transaction Error Rate detected above 1 Percent of Total Transactions'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.8003
Metric Used oc_ingressgateway_http_responses_total
Resolution

The alert is cleared when the number of failed transactions are below 1% of the total transactions or when the number of failed transactions crosses the 10% threshold in which case the CnccCoreTransactionErrorRateAbove10Percent is raised.

Steps:

  1. Check the Service specific metrics to understand the specific service request errors.
  2. If guidance is required,contact My Oracle Support.

Parent topic: CNC Console Core Alerts

6.2.9 CnccCoreTransactionErrorRateAbove10Percent

Table 6-24 CnccCoreTransactionErrorRateAbove10Percent

Field Details
Trigger Condition The number of failed transactions is above 10 percent of the total transactions.
Severity Minor
Alert details provided

Description: 'CNCC Core transaction Error rate is above 10 Percent of Total Transactions (current value is {{ $value }})'

summary: 'CNCC Core ransaction Error Rate detected above 10 Percent of Total Transactions'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.8003
Metric Used oc_ingressgateway_http_responses_total
Resolution

The alert is cleared when the number of failed transactions are below 10% of the total transactions or when the number of failed transactions crosses the 25% threshold in which case the CnccCoreTransactionErrorRateAbove25Percent is raised.

Steps:

  1. Check the Service specific metrics to understand the specific service request errors.
  2. If guidance is required, contact My Oracle Support.

Parent topic: CNC Console Core Alerts

6.2.10 CnccCoreTransactionErrorRateAbove25Percent

Table 6-25 CnccCoreTransactionErrorRateAbove25Percent

Field Details
Trigger Condition The number of failed transactions is above 25 percent of the total transactions.
Severity Major
Alert details provided

Description: 'CNCC Core transaction Error Rate detected above 25 Percent of Total Transactions (current value is {{ $value }})'

Summary: 'CNCC Core transaction Error Rate detected above 25 Percent of Total Transactions'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.8003
Metric Used oc_ingressgateway_http_responses_total
Resolution

The alert is cleared when the number of failed transactions are below 25% of the total transactions or when the number of failed transactions crosses the 50% threshold in which case the CnccCoreTransactionErrorRateAbove50Percent is raised.

Steps:

  1. Check the Service specific metrics to understand the specific service request errors.
  2. If guidance is required, contact My Oracle Support.

Parent topic: CNC Console Core Alerts

6.2.11 CnccCoreTransactionErrorRateAbove50Percent

Table 6-26 CnccCoreTransactionErrorRateAbove50Percent

Field Details
Trigger Condition The number of failed transactions is above 50 percent of the total transactions.
Severity Critical
Alert details provided

Description: 'CNCC Core transaction Error Rate detected above 50 Percent of Total Transactions (current value is {{ $value }})'

Summary: 'CNCC Core transaction Error Rate detected above 50 Percent of Total Transactions'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.8003
Metric Used oc_ingressgateway_http_responses_total
Resolution

The alert is cleared when the number of failed transactions are below 50 percent of the total transactions

Steps:

  1. Check the Service specific metrics to understand the specific service request errors.
  2. If guidance is required, contact My Oracle Support.

Parent topic: CNC Console Core Alerts

6.2.12 CnccCoreIngressGatewayServiceDown

Table 6-27 CnccCoreIngressGatewayServiceDown

Field Details
Trigger Condition Cncc Core Ingress Gateway service is down
Severity Critical
Alert details provided

Description: 'CNCC Core Ingress-Gateway service InstanceIdentifier=~".*core_ingressgateway" is down'

For CNE with Prometheus HA Operator:

summary: 'namespace: {{$labels.namespace}}, podname: {{$labels.pod}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }} : Ingress-gateway service down'

For CNE without Prometheus Operator:

summary: 'namespace: {{$labels.kubernetes_namespace}}, podname: {{$labels.kubernetes_pod_name}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }} : Ingress-gateway service down'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.8004
Metric Used

'up'

Note: This is a Prometheus metric used for instance availability monitoring. If this metric is not available, use a similar metric as exposed by the monitoring system.
Resolution

The alert is cleared when the cncc-core_ingressgateway service is available.

Steps:

  1. Check the orchestration logs of cncc-core_ingressgateway service and check for liveness or readiness probe failures.
  2. Refer the application logs on Kibana and filter based on cncc-core_ingressgateway service names. Check for ERROR WARNING logs related to thread exceptions.
  3. Depending on the failure reason, take the resolution steps.
  4. In case the issue persists, contact My Oracle Support.

Parent topic: CNC Console Core Alerts

6.2.13 CnccCoreFailedLogin

Table 6-28 CnccCoreFailedLogin

Field Details
Trigger Condition

The count of failed login attempts in CNCC-Core by a user goes above '3'
Severity Warning
Alert details provided

Description:'{{ $value }} failed Login attempts have been detected in CNCC Core for user {{$labels.UserName}}, the configured threshold value is 3 failed login attempts for every 5 min'

For CNE with Prometheus HA Operator:

summary: 'namespace: {{$labels.namespace}}, podname: {{$labels.pod}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: failed login attempts are more than the configured threshold value'

For CNE without Prometheus Operator :

summary: 'namespace: {{$labels.kubernetes_namespace}}, podname: {{$labels.kubernetes_pod_name}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: failed login attempts are more than the configured threshold value'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.8005
Metric Used oc_ingressgateway_http_responses_total
Resolution

The alert gets cleared when the total failed login attempts for a particular user go below the threshold value (default value is '3') in the last 5 min (default value is 5 m)

Note: The threshold and time is configurable in the alerts.yaml file.

If guidance is required, contact My Oracle Support.

Parent topic: CNC Console Core Alerts

6.2.14 CnccCoreUnauthorizedAccess

Table 6-29 CnccCoreUnauthorizedAccess

Field Details
Trigger Condition

The count of failed login attempts in CNCC-Core by a user goes above '3'
Severity Warning
Alert details provided

Description:'{{ $value }} Unauthorized Accesses have been detected in CNCC-Core for {{$labels.ResourceType}} for {{$labels.Method}} request. The configured threshold value is 3 for every 5 min'

For CNE with Prometheus HA Operator:

summary: 'namespace: {{$labels.namespace}}, podname: {{$labels.pod}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Unauthorized Access for CNCC-Core are more than threshold value'

For CNE without Prometheus Operator :

summary: 'namespace: {{$labels.kubernetes_namespace}}, podname: {{$labels.kubernetes_pod_name}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Unauthorized Access for CNCC-Core are more than threshold value'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.8006
Metric Used oc_ingressgateway_http_responses_total
Resolution

The alert gets cleared when the total failed login attempts for a particular user go below the threshold value (default value is '3') in the last 5 min (default value is 5 m)

Note: The threshold and time is configurable in the alerts.yaml

If guidance is required, contact My Oracle Support.

Parent topic: CNC Console Core Alerts

6.2.15 CnccCoreAccessTokenFailure

Table 6-30 CnccCoreAccessTokenFailure

Field Details
Trigger Condition

If the count of failed token for CNCC-Core goes above configured value of '3'
Severity Warning
Alert details provided

Description: 'CNCC Core Access Token Failure count is above the configured value i.e. 3 for every 5 min. Failed access token request count per second is (current value is: {{ $value }})'

For CNE with Prometheus HA Operator:

summary: 'namespace: {{$labels.namespace}}, podname: {{$labels.kubernetes_pod_name}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Access Token Failure count is above the configured threshold value'

For CNE without Prometheus Operator :

summary: 'namespace: {{$labels.kubernetes_namespace}}, podname: {{$labels.kubernetes_pod_name}}, timestamp: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}: Access Token Failure count is above the configured threshold value'

OID used for SNMP Traps 1.3.6.1.4.1.323.5.3.51.1.2.8007
Metric Used oc_ingressgateway_http_responses_total
Resolution

The alert gets cleared when the total failed tokens for a particular user go below the threshold value (default value is '3') in the last 5 min (default value is 5 m)

Note: The threshold and time is configurable in the alerts.yaml file.

If guidance is required, Contact My Oracle Support.

Parent topic: CNC Console Core Alerts

6.3 CNC Console Alerts on OCI

This section provides information about CNC Console Alerts on OCI:

Parent topic: CNC Console Alerts

6.3.1 CnccCoreTotalIngressTrafficRateAboveMinorThreshold

Table 6-31 CnccCoreTotalIngressTrafficRateAboveMinorThreshold

Field Details
Trigger Condition

The total CNCC Core Ingress Message rate has crossed the configured minor threshold of 700 TPS.

Default value of this alert trigger point in cncc_alert_rules.yaml is when CNCC Core Ingress Rate crosses 70 % of 1000 (Maximum ingress request rate)
Severity minor
Alert details provided CNCC Core Ingress traffic Rate is above the configured minor threshold i.e. 700 requests per second
Metric Used oc_ingressgateway_http_requests_total
Resolution The alert is cleared either when the total Ingress Traffic rate falls below the Minor threshold or when the total traffic rate cross the Major threshold, inwhich case the CnccCoreTotalIngressTrafficRateAboveMajorThresholdalert shall be raised.Note: The threshold is configurable in the alerts.yamlSteps:Reassess why the CNCC Core is receiving additionaltraffic.If this is unexpected, contact My Oracle Support.

Parent topic: CNC Console Alerts on OCI

6.3.2 CnccCoreTotalIngressTrafficRateAboveMajorThreshold

Table 6-32 CnccCoreTotalIngressTrafficRateAboveMajorThreshold

Field Details
Trigger Condition

The total CNCC Core Ingress Message rate has crossed the configured major threshold of 800 TPS.

Default value of this alert trigger point in cncc_alert_rules.yaml is when CNCC Core Ingress Rate crosses 80 % of 1000 (Maximum ingress request rate)
Severity major
Alert details provided CNCC Core Ingress traffic Rate is above the configured major threshold i.e. 800 requests per second
Metric Used oc_ingressgateway_http_requests_total
Resolution The alert is cleared when the total Ingress Traffic ratefalls below the Major threshold or when the total traffic rate cross the Critical threshold, in which casethe CnccCoreTotalIngressTrafficRateAboveCriticalThresholdNote: The threshold is configurable in the alerts.yaml alert shall be raised.Steps:Reassess why the CNCC Core is receiving additionaltraffic.If this is unexpected, contact My Oracle Support.

Parent topic: CNC Console Alerts on OCI

6.3.3 CnccCoreTotalIngressTrafficRateAboveCriticalThreshold

Table 6-33 CnccCoreTotalIngressTrafficRateAboveCriticalThreshold

Field Details
Trigger Condition

The total CNCC Core Ingress Message rate has crossed the configured critical threshold of 900TPS.

Default value of this alert trigger point in cncc_alert_rules.yaml is when CNCC Core Ingress Rate crosses 90 % of 1000 (Maximum ingress request rate)
Severity critical
Alert details provided CNCC Core Ingress traffic Rate is above the configured critical threshold i.e. 900 requests per second
Metric Used oc_ingressgateway_http_requests_total
Resolution The alert is cleared when the Ingress Traffic rate falls below the Critical threshold.Note: The threshold is configurable in the alerts.yamlSteps:Reassess why the CNCC Core is receiving additional traffic.If this is unexpected, contact My Oracle Support.

Parent topic: CNC Console Alerts on OCI

6.3.4 CnccCoreMemoryUsageCrossedMinorThreshold

Table 6-34 CnccCoreMemoryUsageCrossedMinorThreshold

Field Details
Trigger Condition A pod has reached the configured minor threshold( 70%) of its memory resource limits.
Severity minor
Alert details provided CNCC Core Memory Usage for pod has crossed the configured minor threshold (70%) of its limit.
Metric Used

container_memory_usage_bytes

Note : This is a kubernetes metric used for instance availability monitoring. If the metric is not available, use the similar metric as exposed by the monitoring system.
Resolution

The alert gets cleared when the memory utilization falls below the Minor Threshold or crosses the major threshold, in which case CnccCoreMemoryUsageCrossedMajorThreshold alert shall be raised

Note : The threshold is configurable in the alerts.yaml

If guidance required, Contact My Oracle Support.

Parent topic: CNC Console Alerts on OCI

6.3.5 CnccCoreMemoryUsageCrossedMajorThreshold

Table 6-35 CnccCoreMemoryUsageCrossedMajorThreshold

Field Details
Trigger Condition A pod has reached the configured major threshold( 80%) of its memory resource limits.
Severity major
Alert details provided CNCC Core Memory Usage for pod has crossed the configured major threshold (80%) of its limit.
Metric Used

container_memory_usage_bytes

Note : This is a kubernetes metric used for instance availability monitoring. If the metric is not available, use the similar metric as exposed by the monitoring system.
Resolution

The alert gets cleared when the memory utilization falls below the Major Threshold or crosses the critical threshold, in which case CnccCoreMemoryUsageCrossedCriticalThreshold alert shall be raised

Note : The threshold is configurable in the alerts.yaml

If guidance required, Contact My Oracle Support.

Parent topic: CNC Console Alerts on OCI

6.3.6 CnccCoreMemoryUsageCrossedCriticalThreshold

Table 6-36 CnccCoreMemoryUsageCrossedCriticalThreshold

Field Details
Trigger Condition A pod has reached the configured critical threshold ( 90% ) of its memory resource limits
Severity critical
Alert details provided CNCC Core Memory Usage for pod has crossed the configured critical threshold (90%) of its limit.
Metric Used

container_memory_usage_bytes

Note : This is a kubernetes metric used for instance availability monitoring. If the metric is not available, use the similar metric as exposed by the monitoring system.
Resolution

The alert gets cleared when the memory utilization falls below the Critical Threshold.

Note : The threshold is configurable in the alerts.yaml

If guidance required, Contact My Oracle Support.

Parent topic: CNC Console Alerts on OCI

6.3.7 CnccCoreTransactionErrorRateAbovePointOnePercent

Table 6-37 CnccCoreTransactionErrorRateAbovePointOnePercent

Field Details
Trigger Condition The number of failed transactions is above 0.1 percent of the total transactions.
Severity warning
Alert details provided CNCC Core transaction Error rate is above 0.1 Percent of Total Transactions
Metric Used oc_ingressgateway_http_responses_total
Resolution The alert is cleared when the number of failure transactions are below 0.1% of the total transactions or when the number of failure transactions cross the 1% threshold in which case the CnccCoreTransactionErrorRateAbove1Percent shall beraised.Steps:1. Check the Service specific metrics to understand the specific service request errors.2. If guidance required, contact My Oracle Support.

Parent topic: CNC Console Alerts on OCI

6.3.8 CnccCoreTransactionErrorRateAboveOnePercent

Table 6-38 CnccCoreTransactionErrorRateAboveOnePercent

Field Details
Trigger Condition The number of failed transactions is above 1 percent of the total transactions.
Severity warning
Alert details provided CNCC Core transaction Error rate is above 1 Percent of Total Transactions
Metric Used oc_ingressgateway_http_responses_total
Resolution The alert is cleared when the number of failure transactions are below 1% of the total transactions or when the number of failure transactions cross the 10% threshold in which case the CnccCoreTransactionErrorRateAbove10Percent shall beraised.Steps:1. Check the Service specific metrics to understand the specific service request errors.2. If guidance required, contact My Oracle Support.

Parent topic: CNC Console Alerts on OCI

6.3.9 CnccCoreTransactionErrorRateAboveTenPercent

Table 6-39 CnccCoreTransactionErrorRateAboveTenPercent

Field Details
Trigger Condition The number of failed transactions is above 10 percent of the total transactions.
Severity minor
Alert details provided CNCC Core transaction Error rate is above 10 Percent of Total Transactions
Metric Used oc_ingressgateway_http_responses_total
Resolution The alert is cleared when the number of failure transactions are below 10% of the total transactions or when the number of failure transactions cross the 25% threshold in which case the CnccCoreTransactionErrorRateAbove25Percent shall beraised.Steps:1. Check the Service specific metrics to understand the specific service request errors.2. If guidance required, contact My Oracle Support.

Parent topic: CNC Console Alerts on OCI

6.3.10 CnccCoreTransactionErrorRateAboveTwentyFivePercent

Table 6-40 CnccCoreTransactionErrorRateAboveTwentyFivePercent

Field Details
Trigger Condition The number of failed transactions is above 25 percent of the total transactions.
Severity major
Alert details provided CNCC Core transaction Error Rate detected above 25 Percent of Total Transactions
Metric Used oc_ingressgateway_http_responses_total
Resolution The alert is cleared when the number of failure transactions are below 25% of the total transactions or when the number of failure transactions cross the 50% threshold in which case the CnccCoreTransactionErrorRateAbove50Percent shall beraised.Steps:1. Check the Service specific metrics to understand the specific service request errors.2. If guidance required, contact My Oracle Support.

Parent topic: CNC Console Alerts on OCI

6.3.11 CnccCoreTransactionErrorRateAboveFiftyPercent

Table 6-41 CnccCoreTransactionErrorRateAboveFiftyPercent

Field Details
Trigger Condition The number of failed transactions is above 50 percent of the total transactions.
Severity critical
Alert details provided CNCC Core transaction Error Rate detected above 50 Percent of Total Transactions
Metric Used oc_ingressgateway_http_responses_total
Resolution The alert is cleared when the number of failure transactions are below 50 percent of the total transactions.Steps:1. Check the Service specific metrics to understand the specific service request errors.2. If guidance required, contact My Oracle Support.

Parent topic: CNC Console Alerts on OCI

6.3.12 CnccCoreUnauthorizedAccess

Table 6-42 CnccCoreUnauthorizedAccess

Field Details
Trigger Condition If the count of unauthorized access goes above the configured value of '3'
Severity warning
Alert details provided Unauthorized Accesses have been detected in CNCC-Core for request. The configured threshold value is 3 for every 5 min
Metric Used oc_ingressgateway_http_responses_total
Resolution

The alert gets cleared when the total unauthorized accesses for a particular user go below the threshold value (default value is '3') in the last 5 min (default value is 5 m)

Note : The threshold and time is configurable in the alerts.yaml

If guidance required, Contact My Oracle Support.

Parent topic: CNC Console Alerts on OCI