Reference Procedures

Preserving MetalLB Configuration During Upgrade

This purpose of this procedure is to verify the MetalLB configuration changes that were made in a non-standard manner, such as by editing the MetalLB ConfigMap using kubectl edit. This procedure should be performed prior to upgrade.

Note:

All commands in this procedure are executed from the Bastion Host.

Verifying current MetalLB configuration

This section describes the procedure to determine whether or not the MetalLB config file needs to be updated.

Following the procedure to verify the existing MetalLB configuration:

Ensure that the MetalLB ConfigMap exists in the occne-infra namespace:
```
$ kubectl get configmap -n occne-infra | grep occne-metallb
```
Expected output:
```
NAME            DATA
occne-metallb    1
```

Retrieve the current configuration stored in the MetalLB ConfigMap using the following command:

$ kubectl get configmap occne-metallb -n occne-infra -o yaml

Sample configuration:

apiVersion: v1
data:
  config: |
    address-pools:
    - addresses:
      - 10.75.182.88/29
      auto-assign: false
      name: oam
      protocol: bgp
    peers:
    - my-asn: 64512
      peer-address: 172.16.7.3
      peer-asn: 64501
    - my-asn: 64512
      peer-address: 172.16.7.2
      peer-asn: 64501
kind: ConfigMap
metadata:
  creationTimestamp: "2020-08-14T22:32:27Z"
  labels:
    app: metallb
    chart: metallb-0.12.0
    heritage: Helm
    release: occne-metallb
  managedFields:
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:data:
        .: {}
        f:config: {}
      f:metadata:
        f:labels:
          .: {}
          f:app: {}
          f:chart: {}
          f:heritage: {}
          f:release: {}
    manager: Go-http-client
    operation: Update
    time: "2020-08-14T22:32:27Z"
  name: occne-metallb
  namespace: occne-infra
  resourceVersion: "2442"
  selfLink: /api/v1/namespaces/occne-infra/configmaps/occne-metallb
  uid: 130e0464-856d-48d5-9a6a-626adc0234ee

Note:

The output is similar as in sample, but as per your configuration the IP addresses and ASN numbers might be different.

Display the MetalLB config file using the below command:

$ cat /var/occne/cluster/<cluster_name>/mb_configmap.yaml

Sample file:

configInline:
    peers:
    - peer-address: 172.16.7.3
      peer-asn: 64501
      my-asn: 64512
    - peer-address: 172.16.7.2
      peer-asn: 64501
      my-asn: 64512
    address-pools:
    - name: oam
      protocol: bgp
      auto-assign: false
      addresses:
      - '10.75.182.88/29'

Compare the current configuration against the configuration stored in the MetalLB config file. Compare the "peers" and "address-pools" sections from the ConfigMap output and the config file. Check the differences in these sections.

Note:
The ConfigMap output may show the contents of the "peers" and "address-pools" sections in a different order than the config file. This might be as per your configuration, and does not constitute a "difference".
If there no differences in these sections, then this procedure is complete. If the command output shows differences, such as in the sample output shown above, update the file as mentioned in Updating MetalLB configuration file.

Updating MetalLB configuration file

This section describes the procedure to update the current MetalLB configuration to the MetalLB config file, so that this configuration is preserved during upgrade.

Following the procedure to update the MetalLB configuration file:

Take backup of the current configuration file:

$ cp /var/occne/cluster/<cluster_name>/mb_configmap.yaml /var/occne/cluster/<cluster_name>/mb_configmap_backup.yaml

Copy the contents of the ConfigMap to the new config file.
Using a text editor such as vi, overwrite the "peers" and "address-pools" sections in /var/occne/cluster/<cluster_name>/mb_configmap.yaml with the same sections from the ConfigMap output derived from Verifying current MetalLB configuration. Save your changes and exit when the file is updated completely.

Installing Network Functions

This section describes the procedure to install Network Functions (NF) on the OCCNE.

Note:

It is assumed that the NF has docker images located on a docker registry that is reachable by the cluster's bastion, and associated helm charts located at a URL also accessible by the bastion.

Run the following commands from the cluster bastion.

Follow the procedure to install the NF:

Place docker images into the bastion-host docker registry:
1. Create a file docker_images.txt listing the required docker images and tags:
```
dockerRepo:5000/serviceNameImage1:1.0.1
dockerRepo:5000/serviceNameImage2:1.2.2
```
2. Load these images into the cluster-local docker registry by running (the parameters are the central-docker-registry-repo:port, and the local registry:port):
```
$ /var/occne/cluster/<cluster>/artifacts/retrieve_docker.sh <central-repo>:<central-repo-docker-port> ${HOSTNAME}:5000 < docker_images.txt
```
Place the Helm chart into the bastion-host helm chart repository:
1. Create a file helm_charts.txt listing the Helm chart and version:
```
helmRepoName/chart_name 2.7.6
```
2. Add the source helm repository to the cluster-local helm configuration (this need only be done once for each repo):
```
$ helm repo add helmRepoName https://someUrl.oracle.com
```
3. Load the chart into the cluster-local helm chart repository by running (the parameters are the source repo URL, the target directory, and the cluster directory (where helm binary exists)):
```
$ /var/occne/cluster/<cluster>/artifacts/retrieve_helm.sh http://<central-repo>/occne/charts /var/www/html/occne /var/occne/cluster/${OCCNE_CLUSTER}/artifacts/ < helm_charts.txt
```
Install the NF:
1. Create a values.yaml file on the Bastion Host that contains the values needed by the Helm chart and run the helm install command as follows:
```
$ helm install --name <release-name> --namespace <service-namespace> -f values.yaml <chart_name>
```
2. After successful install, follow section Storing NF Alerts File to configure the NF alerts.
Updating the previously installed NF:
```
$ helm upgrade -f values.yaml <release-name> <chart_name>
```
After successful upgrade, follow section Storing NF Alerts File to configure the NF alerts.
Removing the previously installed NF:
```
$ helm del <release-name> --purge
```

Storing NF Alerts File

After the NF install/upgrade, the alert file must be copied to a pre-defined location in bastion host before upgrading from OCCNE 1.5 to OCCNE 1.6.

Note:

A YAML file containing the alerting rules, hereby referenced as NFAlertRule.yaml will be added . This must be in a valid YAML format, and must have the extension .yaml.

Steps to follow on Bastion host:

Verify the environment variable OCCNE_CLUSTER is set correctly. Check if below directory exists, if not then create it using below command:
```
echo $OCCNE_CLUSTER
mkdir /var/occne/cluster/$OCCNE_CLUSTER/artifacts/alerts
chmod 755 /var/occne/cluster/$OCCNE_CLUSTER/artifacts/alerts
```
Set the permissions on the NFAlertRule.yaml file as below:
```
chmod 644 NFAlertRule.yaml
```
Copy the NFAlertRule.yaml file to the newly created alerts directory as above using below command:
```
cp NFAlertRule.yaml /var/occne/cluster/$OCCNE_CLUSTER/artifacts/alerts
```

Here is an example for occne_alerts.yaml with the expected permissions:

Directory:
$ ls -lrt -d  /var/occne/cluster/$OCCNE_CLUSTER/artifacts/alerts
drwxr-xr-x. 2 cloud-user cloud-user 4096 Sep  2 14:46 /var/occne/cluster/occne-cluster-name/artifacts/alerts
 
File:
$ ls -lrt  /var/occne/cluster/$OCCNE_CLUSTER/artifacts/alerts
total 16
-rw-r--r--. 1 cloud-user cloud-user 12991 Sep  2 14:46 occne_alerts.yaml

Configuring ZIPKIN Support in Jaeger Collector

This section describes how to configure Jaeger to support ZIPKIN after fresh installation of OCCNE or upgrade from older versions of OCCNE.

Note:

This procedure must be applied after upgrading to OCCNE 1.4, 1.5, or 1.6. It must be re-applied after each upgrade, because upgrade overwrites these changes.

Follow the below procedure to configure Jaeger:

From bastion host, edit deployment of jaeger-collector as:

$ kubectl edit deployment occne-tracer-jaeger-collector --namespace occne-infra

Add environment variable COLLECTOR_ZIPKIN_HTTP_PORT with value "9411" above SPAN_STORAGE_TYPE in jaeger-collector deployment. Sample file:
```
spec:
  containers:
  - env:
    - name: COLLECTOR_ZIPKIN_HTTP_PORT
      value: "9411"
    - name: SPAN_STORAGE_TYPE
      value: elasticsearch
```
Save the file to enable ZIPKIN support in Jaeger Collector.