1. Cloud Native Policy Control Function User's Guide
  2. Policy Control Function Alerts

6 Policy Control Function Alerts

This section includes information about alerts for PCF.

Table 6-1 Common Alerts

Alert Name Description Severity
PCF_SERVICES_DOWN Alert if any PCF service down for 5mins for given namespace in AlertRules file Critical
IngressErrorRateAbove10PercentPerPod Alert if ingress error rate on each pod above 10% Critical

Table 6-2 SM Service Alerts

Alert Name Description Severity
SMTrafficRateAboveThreshold Alert if Ingress traffic on SM service reaches 90% of max MPS in 2mins Major
SMIngressErrorRateAbove10Percent Alert if Ingress transaction error rate exceeds 10% of all SM transactions in last 24 hours Critical
SMEgressErrorRateAbove1Percent Alert if Egress transaction error rate exceeds 1% of all SM transactions in last 24 hours Minor

Table 6-3 Diameter Connector Alerts

Alert Name Description Severity
DiamTrafficRateAboveThreshold Alert if Diameter Connector traffic reaches 90% of max MPS Major
DiamIngressErrorRateAbove10Percent Alert if error rate exceeds 10% of all Diameter transactions in last 24 hours Critical
DiamEgressErrorRateAbove1Percent Alert if Egress transaction error rate exceeds 1% of all Diameter transactions Minor

Table 6-4 User Service - UDR Alerts

Alert Name Description Severity
PcfUdrIngressTrafficRateAboveThreshold Alert if Ingress traffic from UDR reaches 90% of max MPS Major
PcfUdrEgressErrorRateAbove10Percent Alert if error rate exceeds 10% of all UDR transactions Critical

Table 6-5 User Service - CHF Alerts

Alert Name Description Severity
PcfChfIngressTrafficRateAboveThreshold Alert if Ingress traffic from CHF reaches 90% of max MPS Major
PcfChfEgressErrorRateAbove10Percent Alert if error rate exceeds 10% of all CHF transactions Critical

Table 6-6 PolicyDS Service Alerts

Alert Name Description Severity
PolicyDsIngressTrafficRateAboveThreshold Alert if Ingress traffic reaches 90% of max MPS Major
PolicyDsIngressErrorRateAbove10Percent Alert if Ingress error rate exceeds 10% of all PolicyDS transactions Critical
PolicyDsEgressErrorRateAbove1Percent Alert if Egress error rate exceeds 10% of all PolicyDS transactions Minor

PCF Alert Configuration

This section describes the Measurement based Alert rules configuration for PCF. The Alert Manager uses the Prometheus measurements values as reported by microservices in conditions under alert rules to trigger alerts.

PCF Alert Configuration

Note:

  • The alertmanager and prometheus tools should run in Oracle CNE namespace, for example, occne-infra.
  • Alert file is packaged with PCF Custom Templates. The PCF Templates.zip file can be downloaded from OHC. Unzip the PCF Templates.zip file to get PcfAlertRules.yaml file.
  • Edit the value of the following parameters in thePcfAlertRules.yaml file before following the procedure for configuring the alerts:
    • [ 90% of Max MPS].
      For Example, if the value of Max MPS is 10000, set [ 90% of Max MPS] as 9000 in yaml file as follows: 
      sum(rate(ocpm_ingress_request_total{servicename_3gpp="npcf-smpolicycontrol"}[2m])) >=9000
    • kubernetes_namespace.

      For Example,

      If PCF is deployed at more than one site, set kubernetes_namespace in yaml file as follows: 
      expr: up{kubernetes_namespace=~"pcf|ocpcf"} == 0

      If PCF is deployed at only one site, set kubernetes_namespace in yaml file as follows: 

      expr: up{kubernetes_namespace="pcf"}==0
To Configure PCF alerts in Prometheus:
  1. Find the config map to configure alerts in prometheus server by executing the following command: 
    kubectl get configmap -n <Namespace>
    where, <Namespace> is the prometheus server namespace used in helm install command.
    For Example, assuming prometheus server is under occne-infra namespace, execute the following command to find the config map:
    kubectl get configmaps -n occne-infra  | grep prometheus-server
    0utput: occne-prometheus-server 4 46d
  2. Take Backup of current config map of prometheus server by executing the following command: 
    kubectl get configmaps <Name> -o yaml -n <Namespace> > /tmp/t_mapConfig.yaml
    where, <Name> is the prometheus config map name used in helm install command.
  3. Check if alertspcf is present in the t_mapConfig.yaml file by executing the following command: 
    cat /tmp/t_mapConfig.yaml  | grep alertspcf
  4. If alertspcf is present, delete the alertspcf entry from the t_mapConfig.yaml file, by executing the following command: 
    sed -i '/etc\/config\/alertspcf/d' /tmp/t_mapConfig.yaml

    Note:

    This command should be executed only once.
  5. If alertspcf is not present, add the alertspcf entry in the t_mapConfig.yaml file by executing the following command: 
    sed -i '/rule_files:/a\    \- /etc/config/alertspcf'  /tmp/t_mapConfig.yaml

    Note:

    This command should be executed only once.
  6. Reload the config map with the modifed file by executing the following command: 
    kubectl replace configmap <Name> -f /tmp/t_mapConfig.yaml
  7. Add PcfAlertRules.yaml file into prometheus config map by executing the following command : 
    kubectl patch configmap <Name> -n <Namespace> --type merge --patch
"$(cat <PATH>/PcfAlertRules.yaml)"
    where, <PATH> is the location of the PcfAlertRules.yaml file.
  8. Restart prometheus-server pod.
  9. Verify the alerts in prometheus GUI. Below screenshot displays the PCF alerts:PCF Alerts