7.2 Configuring Alerts

This section describes the measurement based alert rules configuration for NEF and CAPIF. The Alert Manager uses the Prometheus measurements values as reported by microservices in conditions under alert rules to trigger alerts.

The NEF package contains the sample alert files with the NEF Custom Templates. The NEF Custom Templates.zip file can be downloaded from MOS. Unzip the folder to access the following files:

• NefAlertrules-24.2.2.yaml
• CapifAlertrules-24.2.2.yaml

Note:

• If required, edit the threshold values of various alerts in the alert files before configuring the alerts. For more information on the alerts that can be updated, see Alert Details.
• The Alert Manager and Prometheus tools must run in Oracle CNE namespace, for example, occne-infra.

Table 7-89 Alert Details

Alert Name	Details	Default Value	Notes
OcnefTotalIngressTrafficRateAboveMinorThreshold	Traffic Rate is above 80 Percent of Max requests per second	Greater than/equal to 800 and Less than 900	Maximum Ingress rate considered is 1000 requests per second. So, here in default value 800 is 80% of 1000 and 900 is 90% of 1000. For example, if value need to be updated then depending upon maximum ingress request rate, set [ 90% of Max Ingress Request Rate] and [ 80% of Max Ingress Request Rate] for this alert
OcnefTotalIngressTrafficRateAboveMajorThreshold	Traffic Rate is above 90 Percent of Max requests per second	Greater than/equal to 900 and Less than 950	Maximum Ingress rate considered is 1000 requests per second. So, here in default value 900 is 90% of 1000 and 950 is 95% of 1000. For example, if value need to be updated then depending upon maximum ingress request rate, set [ 90% of Max Ingress Request Rate] and [ 95% of Max Ingress Request Rate] for this alert
OcnefTotalIngressTrafficRateAboveCriticalThreshold	Traffic Rate is above 95 Percent of Max requests per second	Greater than/equal to 950	Maximum Ingress rate considered is 1000 requests per second. So, here in default value 950 is 95% of 1000. For example, if value need to be updated then depending upon maximum ingress request rate, set [ 95% of Max Ingress Request Rate] for this alert

Update NEF alerts for OCCNE 1.8.x and previous versions

The following procedure describes how to update the NEF alerts for OCCNE version 1.8.x and previous versions:

_NAME_ :- Helm Release of Prometheus

_Namespace_ :- Kubernetes NameSpace in which Prometheus is installed

1. Find the config map to configure alerts in Prometheus server using the following command:

   kubectl get configmap -n <Namespace>

   where, <Namespace> is the prometheus server namespace used in the helm install command. For example, assuming Prometheus server is under occne-infra namespace, run the following command to find the config map:

   kubectl get configmaps -n occne-infra | grep prometheus-server 

2. Take backup of current configuration map of Prometheus:

   kubectl get configmaps _NAME_-server -o yaml -n _Namespace_ > /tmp/tempConfig.yaml

3. Check if alertsnef is present in the tempConfig.yaml file by running the following command:

   cat /tmp/tempConfig.yaml | grep alertsnef

   Depending on the outcome of the previous step, perform anyone of the following steps:

   1. If alertsnef is present, delete the alertsnef entry from the tempConfig.yaml file, by running the following commands:

      sed -i '/etc\/config\/alertsnef/d' /tmp/tempConfig.yaml

      sed -i '/rule_files:/a\  \- /etc/config/alertsnef' /tmp/tempConfig.yaml

      Note:

      This command should be run only once.
   2. If alertsnef is not present, add the alertsnef entry in the tempConfig.yaml file by running the following command:

      sed -i '/rule_files:/a\  \- /etc/config/alertsnef' /tmp/tempConfig.yaml

4. Update configuration map with updated file name of NEF alert file:

   kubectl replace configmap _NAME_-server -f /tmp/tempConfig.yaml

5. Add NEF alert rules in configuration map under file name of NEF alert file:

   kubectl patch configmap _NAME_-server -n _Namespace_--type merge --patch
   "$(cat ~/NefAlertrules-24.2.2.yaml)"

Update CAPIF/NEF alerts for OCCNE 1.9.x and later

This section describes the measurement based Alert rules configuration for CAPIF/NEF in Prometheus. Use the NefAlertrules-24.2.2.yaml or CapifAlertrules-24.2.2.yaml file updated in Alert configuration section.

1. Run the following command to apply the prometheusrules CRD:

   $ kubectl apply -f <alert.yaml file> --namespace <namespace>

   Example for NEF:

   $ kubectl apply -f NefAlertrules-24.2.2.yaml --namespace ocnef
     prometheusrule.monitoring.coreos.com/nef-alerting-rules created

   Example for CAPIF:

   $ kubectl apply -f CapifAlertrules-24.2.2.yaml --namespace ocnef
     prometheusrule.monitoring.coreos.com/capif-alerting-rules created

2. Run the following command to check CAPIF/NEF alert file is added to prometheusrules:

   $ kubectl get prometheusrules --namespace <namespace>

   Example for NEF:

   $ kubectl get prometheusrules --namespace nef
   

   Sample output:

   NAME                          AGE
   nef-alerting-rules          1m

   Example for CAPIF:

   $ kubectl get prometheusrules --namespace capif
   

   Sample output:

   NAME                          AGE
   capif-alerting-rules          1m

3. Log in to Prometheus GUI and verify the alerts section.

Note:

The Prometheus server takes an updated configuration map that is automatically reloaded after approximately 60 seconds. Refresh the Prometheus GUI to confirm that the CAPIF/NEF Alerts have been reloaded.