Managing Pods in BRM Cloud Native

Setting up Autoscaling of BRM Pods

You can use the Kubernetes Horizontal pod Autoscaler to automatically scale up or scale down the number of BRM pod replicas in your deployment based on a pod's CPU or memory utilization. For more information, see "Horizontal Pod Autoscaler" in the Kubernetes Tasks documentation.

In BRM cloud native deployments, the Horizontal Pod Autoscaler monitors and scales these BRM pods:

cm
dm-oracle
dm-aq
dm-eai
dm-ifw-sync
batch-controller
rel-daemon
realtime-pipe
brm-rest-services-manager

To set up autoscaling for BRM pods:

Open your override-values.yaml file for oc-cn-helm-chart.
Enable the Horizontal Pod Autoscaler by setting the ocbrm.isHPAEnabled key to true.
Enable the modification of resource limits by setting the ocbrm.isResourceLimitEnabled key to true.
Specify how often, in seconds, the Horizontal Pod Autoscaler checks a BRM pod's memory usage and scales the number of replicas. To do so, set the ocbrm.refreshInterval key to the number of seconds between each check. For example, set it to 60 for a one-minute interval.
For each BRM pod, set these keys to the appropriate values for your system:
- ocbrm.BRMPod.resourceLimits.limitsCpu: Set this to the maximum number of CPU cores that the pod can utilize.
  
  If the pod's CPU utilization exceeds this value, Kubernetes terminates the pod.
- ocbrm.BRMPod.resourceLimits.requestCpu: Set this to the minimum number of CPU cores that must be available in a Kubernetes node to deploy a pod. The default is 2 for the cm pod, 3.5 for the dm_oracle pod, 2 for the rel_daemon pod, and 1 for all other pods.
  
  If the minimum CPU amount is not available, the pod is set to Pending.
  
  Note:
  
  Because the CM contains two containers, its node must have twice the minimum CPU available (that is, 2 * requestCpu) to deploy the cm pod.
- ocbrm.BRMPod.resourceLimits.limitsMemory: Set this to the maximum amount of memory that a pod can utilize.
  
  If a pod's memory utilization exceeds this value, Kubernetes terminates the pod.
- ocbrm.BRMPod.resourceLimits.requestMemory: Set this to the minimum amount of memory required for a Kubernetes node to deploy a pod.
  
  If the minimum amount is not available, the pod is set to Pending due to insufficient memory.
- ocbrm.BRMPod.hpaValues.minReplica: Set this to the minimum number of pod replicas that can be deployed in a cluster.
  
  If a pod's utilization metrics drop below targetCPU or targetMemory, the Horizontal Pod Autoscaler scales down the number of pod replicas to this minimum count. If the number of pod replicas are already at the minimum, no changes are made.
- ocbrm.BRMPod.hpaValues.maxReplica: Set this to the maximum number of pod replicas to deploy when scale up is triggered.
  
  If a pod's metrics utilization goes above targetCPU or targetMemory, the Horizontal Pod Autoscaler scales up the number of pods to this maximum count.
- ocbrm.BRMPod.hpaValues.targetCpu: Set this to the percentage of requestCpu at which to scale up or scale down a pod.
  
  If a pod's CPU utilization exceeds targetCpu, the Horizontal Pod Autoscaler increases the pod replica count to maxReplica. If a pod's CPU utilization drops below targetCpu, the Horizontal Pod Autoscaler decreases the pod replica count to minReplica.
- ocbrm.BRMPod.hpaValues.targetMemory: Set this to the percentage of requestMemory at which to scale up or scale down a pod.
  
  If a pod's memory utilization exceeds targetMemory, the Horizontal Pod Autoscaler increases the pod replica count to maxReplica. If memory utilization drops below targetMemory, the Horizontal Pod Autoscaler decreases the pod replica count to minReplica.
Save and close your override-values.yaml file.
Run the helm upgrade command to update your Helm release:
```
helm upgrade BrmReleaseName oc-cn-helm-chart --values OverrideValuesFile -n BrmNameSpace
```
where:
- BrmReleaseName is the release name for oc-cn-helm-chart and is used to track this installation instance.
- OverrideValuesFile is the file name and path to your override-values.yaml file.
- BrmNameSpace is the name space in which to create BRM Kubernetes objects for the BRM Helm chart.

Automatically Rolling Deployments by Using Annotations

Whenever a ConfigMap entry or a Secret file is modified, you must restart its associated pod. This updates the container's configuration, but the application is notified about the configuration updates only if the pod's deployment specification has changed. Thus, a container could be using the new configuration, while the application keeps running with its old configuration.

You can configure a pod to automatically notify an application when a container's configuration has changed. To do so, configure a pod to automatically update its deployment specification whenever a ConfigMap or Secret file changes by using the sha256sum function. Add an annotations section similar to this one to the pod's deployment specification:

kind: Deployment
spec:
  template:
    metadata:
      annotations:
        checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}

For more information, see "Automatically Roll Deployments" in Helm Chart Development Tips and Tricks.

Restarting BRM Pods

You may occasionally need to restart a BRM pod, such as when an error occurs that you cannot fix or a pod is stuck in a terminating status. You restart a BRM pod by deleting it with kubectl.

To restart a BRM pod:

Retrieve the names of the BRM pods by entering this command:

kubectl -n NameSpace get pods

where NameSpace is the namespace in which Kubernetes objects for the BRM Helm chart reside.

The following provides sample output:

NAME                             READY   STATUS      RESTARTS   AGE
cm-6f79d95887-lp7qs              1/1     Running     0          6d17h
dm-oracle-5496bf8d94-vjgn7       1/1     Running     0          6d17h
dm-kafka-d5ccf6dbd-l968b         1/1     Running     0          6d17h

Delete a pod by entering this command:
```
kubectl delete pod PodName -n NameSpace
```
where PodName is the name of the pod

For example, to delete and restart the cm pod, you would enter:
```
kubectl delete pod cm-6f79d95887-lp7qs -n NameSpace
```