Configuring Service Communication Proxy Alert in Prometheus
SCP Helm Chart Release Name: _NAME_
Prometheus NameSpace: _Namespace _
To configure Service Communication Proxy Alert in Prometheus follow the procedure mentioned in Table 4-2:
Table 4-2 Configuring Service Communication Proxy Alert in Prometheus
| Step No. | Procedure | Description |
|---|---|---|
| 1. | Check the name of the config map | To check the name of the config map used by
Prometheus use below command:
$kubectl get configmap -n <_Namespace_> Example: $kubectl get configmap -n prometheus-alert2 NAME DATA AGE lisa-prometheus-alert2-alertmanager 1 146d lisa-prometheus-alert2-server 4 146d |
| 2. | Take backup of current config map | Select the map name appended with " -server ".
In above example its " lisa-prometheus-alert2-server " and take its backup
using below command:
# Take Backup of current config map of Prometheus. This command will save the configmap in the provided file. In below command /tmp/tempConfig.yaml is the file where the configmap will get stored. $ kubectl get configmaps <_NAME_>-server -o yaml -n <_Namespace_> /tmp/tempConfig.yaml Example: $ kubectl get configmaps lisa-prometheus-alert2-server -o yaml -n prometheus-alert2 > /tmp/tempConfig.yaml |
| 3. | Check and delete "alertsscp" rule | Check and delete "alertsscp" rule if its already
configured in the prometheus config map. If configured this step will delete
the " alertsscp " rule. This is optional step if doing for the first time.
$ sed -i '/etc\/config\/alertsscp/d' /tmp/tempConfig.yaml |
| 4. | Add the "alertsscp" rule | Add the "alertsscp" rule in the configmap dump
file under ' rule_files ' tag.
$ sed -i '/rule_files:/a\ \- /etc/config/alertsscp' /tmp/tempConfig.yaml |
| 5. | Update the configmap | Update the configmap using below command. Ensure
to use same configmap name which was used to take backup of prometheus
configmap in step 2.
# Update Config map with updated file name of SCP alert file $ kubectl replace configmap <_NAME_>-server -f /tmp/tempConfig.yaml Example: $ kubectl replace configmap lisa-prometheus-alert2-server -f /tmp/tempConfig.yaml |
| 6. | Add scpAlertrules in configmap | Patch the configmap with new "alertsscp" rule
using below command. Kindly note the patch file provided is the custom template
file provided with SCP (i.e SCPAlertrules.yaml).
# Add scpAlertrules in Config map under file name of SCP alert file $ kubectl patch configmap _NAME_-server -n _Namespace_ --type merge --patch "$(cat ~/SCPAlertrules.yaml)" Example $ kubectl patch configmap lisa-prometheus-alert2-server -n prometheus-alert2 --type merge --patch "$(cat ~/SCPAlertrules.yaml)" |
Note:
Prometheus takes nearly 20 seconds to apply the updated Config map.
