4 Upgrading NRF

This section provides information on how to upgrade an existing Oracle Communications Cloud Native Core, Network Repository Function (NRF) deployment to the latest release using CLI procedures as outlined in the following table:

Note:

  • In a georedundant deployment, perform the steps explained in this section on all the georedundant sites.
  • Before upgrading NRF, ensure that the required cnDBTier resources are available. For more information about the cnDBTier resource changes, see cnDBTier Resource Requirement.
  • For NRF georedundant deployments, the difference between the NRF release versions for all the georedundant sites cannot be more than 2 releases.

    For example, in a three-site NRF deployment, all the 3 sites are at the same release version as N. During site upgrade, site 1 and site 2 are upgraded to N+2 version, and site 3 is not upgraded yet. In this state, before upgrading site 3 to N+2, upgrading site 1 and/or site 2 from N+2 version to N+3 or N+4 version is not supported as the difference between the NRF release versions for all the georedundant sites cannot be more than 2 releases.

    For more information about the cnDBTier georedundant deployments, see Oracle Communications Cloud Native Core, cnDBTier Installation, Upgrade, and Fault Recovery Guide.

    For more information about the CNC Console georedundant deployments, see Oracle Communications Cloud Native Configuration Console Installation, Upgrade, and Fault Recovery Guide.

  • While performing a cnDBTier upgrade to this release or beyond, cnDBTier recommends the value of global.additionalndbconfigurations.ndb.HeartbeatIntervalDbDb parameter should be 1250. This value should be incremented in two steps.
    If the value of global.additionalndbconfigurations.ndb.HeartbeatIntervalDbDb parameter is 500, follow the below steps to increase the value to 1250.
    • Modify the value of the global.additionalndbconfigurations.ndb.HeartbeatIntervalDbDb parameter to 900 in the ocnrf_dbtier_25.2.201_custom_values_25.2.201.yaml file and perform cnDBTier upgrade.
    • Once upgrade is completed successfully, modify the value of the global.additionalndbconfigurations.ndb.HeartbeatIntervalDbDb parameter to 1250 in ocnrf_dbtier_25.2.201_custom_values_25.2.201.yaml and perform cnDBTier upgrade.
  • While performing a cnDBTier upgrade to this release or beyond, verify if the ServiceMonitor is created or not. If ServiceMonitor is not present, then create it to enable the Prometheus Operator to fetch metrics from the /prometheus REST endpoint for db-monitor-svc. Use the following commands to create the ServiceMonitor.
    
# Create Service Monitor (Prom Operator CRD) with cnDBTier configuration.
$ kubectl apply -f namespace/occndbtier_service_monitor_${OCCNE_VERSION}.yaml -n
${OCCNE_INFRA_NAMESPACE}Note: ${OCCNE_INFRA_NAMESPACE} is the namespace where prometheus operator has been installed.
  • In case, NRF should accept the 3GPP compliant nfSetId value, then the value of global.deprecatedList parameter should be configured appropriately. For more information about global.deprecatedList parameter, see Global Parameters.

Table 4-1 NRF Upgrade Sequence

Upgrade Sequence Applicable for CLI
Preupgrade Tasks Yes
Upgrade Tasks Yes

4.1 Supported Upgrade Paths

The following table lists the supported upgrade paths for NRF.

Table 4-2 Supported Upgrade Paths

Source Release Target Release
25.1.2xx 25.2.201

Note:

  • While performing an upgrade from 25.1.2xx to 25.2.2xx, an additional step may be required. For more details, see Upgrading NRF.
  • For more information about the upgrade order of NFs, see Oracle Communications Cloud Native Core Solution Installation, Upgrade, and Fault Recovery Guide.

4.2 Upgrade Strategy

NRF supports in-service upgrade. The supported upgrade strategy is RollingUpdate. The rolling update strategy is a gradual process that allows you to update your Kubernetes system with only a minor effect on performance and no downtime. The advantage of the rolling update strategy is that the update is applied Pod-by-Pod so the greater system can remain active.

Note:

It is recommended to perform in-service upgrade during maintenance window where the recommended traffic rate is 25% of the configured traffic or below. It is also expected the traffic failure to stay below 5% during the upgrade and fully recover post upgrade.

Following engineering configuration parameters are used to define upgrade strategy:
  • upgradeStrategy parameter indicates the update strategy used in NRF.
  • maxUnavailable parameter determines the maximum number of pods that can be unavailable during upgrade.

Table 4-3 Predefined Upgrade Strategy Value

Microservice Upgrade Value (maxUnavailabe)
<helm-release-name>-nfregistration 25%
<helm-release-name>-nfsubscription 25%
<helm-release-name>-nfdiscovery 25%
<helm-release-name>-nrfauditor 25%
<helm-release-name>-nrfconfiguration Not Applicable

Note: maxSurge attribute is used for this microservice.

<helm-release-name>-appinfo 50%
<helm-release-name>-nfaccesstoken 25%
<helm-release-name>-nrfartisan Not Applicable

Note: maxSurge attribute is used for this microservice.

<helm-release-name>-alternate_route 25%
<helm-release-name>-egressgateway 25%
<helm-release-name>-ingressgateway 25%
<helm-release-name>-performance 50%

Note:

<helm-release-name> is the Helm release name. For example, if Helm release name is "ocnrf", then nrfartisan microservice name will be "ocnrf-nrfartisan".

Note:

When NRF is deployed with OCCM, follow the specific upgrade sequence as mentioned in the Oracle Communications Cloud Native Core Solution Installation, Upgrade, and Fault Recovery Guide.

4.3 Preupgrade Tasks

This section provides information about preupgrade tasks to be performed before upgrading NRF:

  1. Keep current custom_values.yaml file from the source release as backup, that is for upgrading to 25.2.201.
  2. Update the new custom_values.yaml defined for target NRF release.
  3. Enable enableNrfArtisanService in the Global Parameters, if DNS NAPTR must be implemented and it is not enabled in previous release.
  4. Enable performanceServiceEnable in the Global Parameters, if Perf-Info must be implemented and it is not enabled in previous release.
  5. Enable leaderElectionDbName in the Global Parameters if leaderElectionDB database must be implemented and it is not enabled in previous release.
  6. Enable alternateRouteServiceEnable in the Global Parameters, if alternate route service must be implemented and it is not enabled in previous release.
  7. Set the value of enablePodSecurityContext parameter to true in the Ingress Gateway Microservice and Egress Gateway Microservice sections, if HTTPS is already enabled at Ingress Gateway (that is, ingressgateway.enableIncomingHttps is set to true) and Egress Gateway microservice (that is, egressgateway.enableOutgoingHttps is set to true).
  8. Before starting upgrade, take a manual backup of NRF REST based configuration. This helps if preupgrade data has to be restored. See the Backup of NRF Configuration Data for taking the manual backup for REST based configuration.
  9. Before NRF upgrade starts, validate that there are no database backup operation is in progress or no scheduled backups are planned during the upgrade window.

    Check the cnDBTier backup status using http://<base-uri>/ocdbtier/ backup/status API, see the "Support for cnDBTier APIs in CNC Console" section in Oracle Communications Cloud Native Core, Network Repository Function User Guide.

  10. Before upgrading, perform sanity check using Helm test. See the Performing Helm Test section for the Helm test procedure.
  11. Install or upgrade the network policies, if applicable. For more information, see Configuring Network Policies.

    For Rest API configuration details, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide.

4.4 Upgrade Tasks

This section provides information about the sequence of tasks to be performed for upgrading an existing NRF deployment.

Helm Upgrade

Upgrading an existing deployment replaces the running containers and pods with new containers and pods. If there is no change in the pod configuration, it is not replaced. Unless there is a change in the service configuration of a microservice, the service endpoints remain unchanged. For example, ClusterIP.

Upgrade Procedure

Caution:

  • Ensure that there is no database backup is in progress or there are no scheduled backups planned during the NRF upgrade. For more information, see step 9 in Preupgrade Tasks section.
  • Do not perform any configuration changes during the upgrade.
  • Do not exit from the helm upgrade command manually. After running the helm upgrade command, it takes sometime (depending upon the number of PODs to upgrade) to upgrade all of the services. Do not press "ctrl+c" to come out from the helm upgrade command. It may lead to anomalous behavior.
  • When upgrading from versions earlier than 25.1.205 to 25.2.2xx, perform a Helm upgrade on the source release to set the value of ingressgateway.configMapReloadEnabled parameter to false. If this step is not completed, temporary traffic disruption may occur during the In Service Software Upgrade (ISSU) upgrade. The system automatically resumes to normal service once the upgrade completes. This step is not required when upgrading from version 25.1.205 or when performing an upgrade with site isolation.
  1. Untar the latest NRF package and if required, re-tag and push the images to registry. For more information, see Downloading the NRF package and Pushing the Images to Customer Docker Registry.
  2. Modify the ocnrf-custom-values-25.2.201.yaml file parameters as per site requirement.
  3. Run the following command to create and grant leaderElectionDb database, if does not exists:
    1. Create the leaderElectionDb database:
      CREATE DATABASE IF NOT EXISTS leaderElectionDB CHARACTER SET utf8;
    2. Grant the NRF privileged user permission to leaderElectionDb database:
      GRANT SELECT, INSERT, CREATE, ALTER, DROP, LOCK TABLES, CREATE TEMPORARY TABLES, DELETE, UPDATE, EXECUTE ON leaderElectionDB.* TO 'nrfPrivilegedUsr'@'%';

FLUSH PRIVILEGES;
  4. Run the following commands to update the secret with leaderElectionDb database details before upgrade:
    1. Run the following command:
      echo -n <leaderElectionDB_name> | base 64
      For example:
      echo -n "leaderElectionDB" | base64
bGVhZGVyRWxlY3Rpb25EQg==

      Note:

      Note down the output of this command.
    2. Run the following command to update the secret for NRF privileged user:
      kubectl patch secret -n <namespace> <secret-name> -p="{\"data\":{\"<leaderElectionDB_literal_key_name>\": \"<leaderElectionDB_literal_value\"}}" -v=1
      For example:
      kubectl patch secret -n ocnrf privilegeduser-secret -p="{\"data\":{\"leaderElectionDbName\": \"bGVhZGVyRWxlY3Rpb25EQg==\"}}" -v=1

      Note:

      The leaderElectionDbName parameter must be as defined in the Global Parameters section.
  5. Run the following commands to grant INDEX privilege to NRF privileged user:
    GRANT INDEX ON <NRF Application Database>.* TO '<NRF Privileged User Name>'@'%';
FLUSH PRIVILEGES;
    For example:
    GRANT INDEX ON nrfApplicationDB.* TO 'nrfPrivilegedUsr'@'%';
FLUSH PRIVILEGES;
  6. Run the following command to verify the INDEX grant:
    SHOW GRANTS FOR '<NRF Privileged User Name>'@'%';
    For example:
    SHOW GRANTS FOR 'nrfPrivilegedUsr'@'%';
    Sample output:
    SHOW GRANTS FOR 'nrfPrivilegedUsr'@'%';
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Grants for nrfPrivilegedUsr@%                                                                                                                                     |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| GRANT USAGE ON *.* TO `nrfPrivilegedUsr`@`%`                                                                                                                      |
| GRANT NDB_STORED_USER ON *.* TO `nrfPrivilegedUsr`@`%` WITH GRANT OPTION                                                                                          |
| GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE ON `commonConfigurationDB`.* TO `nrfPrivilegedUsr`@`%`   || GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE ON `nrfApplicationDB`.* TO `nrfPrivilegedUsr`@`%` |
| GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE ON `nrfNetworkDB`.* TO `nrfPrivilegedUsr`@`%`            |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  7. Create secrets for DNS NAPTR alternate route service as described in Creating Secrets for DNS NAPTR - Alternate route service section.

    Note:

    Skip this step, if DNS NAPTR feature is not required.
  8. Run the following command to upgrade an existing NRF deployment:

    Note:

    If you are upgrading an existing NRF deployment with georedundancy feature enabled, ensure that you configure dbMonitorSvcHost, dbMonitorSvcPort, and siteNameToNrfInstanceIdMapping parameters before running helm upgrade to NRF 25.2.201. For more information about the parameters, see Global Parameters.
    1. Using local Helm chart:
      $ helm upgrade <release_name> <helm_chart> -f <ocnrf_customized_values.yaml> --namespace <namespace-name> --timeout 12m
      For example:
      $ helm upgrade ocnrf ocnrf-25.2.201.tgz -f ocnrf-custom-values-25.2.201.yaml --namespace ocnrf --timeout 12m
    2. Using chart from Helm repo:
      $ helm upgrade <release_name> <helm_repo/helm_chart> --version <chart_version> -f <ocnrf_customized_values.yaml> --namespace <namespace-name>
      For example:
      $ helm upgrade ocnrf ocnrf-helm-repo/ocnrf --version 25.2.201 -f ocnrf-custom-values-25.2.201.yaml --namespace ocnrf
  9. Run the following command to check the status of the upgrade:
    $ helm status <release_name> -namespace <namespace-name>

    For example: $ helm status ocnrf -namespace ocnrf

    Sample output of Helm status:

    
$ helm status ocnrf -n ocnrf
NAME: ocnrf
LAST DEPLOYED: Mon Dec 16 11:49:46 2024
NAMESPACE: ocnrf
STATUS: deployed
REVISION: 2
NOTES:
# Copyright 2020 (C), Oracle and/or its affiliates. All rights reserved.
Thank you for installing ocnrf.
Your release is named ocnrf, Release Revision: 2.
  10. Run the following command to check the history of the upgrade: 
    $ helm history <release_name> -namespace <namespace-name>

    For example: $ helm history ocnrf -n ocnrf

    Sample output of a successful upgrade:
    
REVISION	UPDATED                 	STATUS    	CHART               	APP VERSION  	DESCRIPTION
1       	Mon Dec 16 10:32:35 2024	superseded	ocnrf-24.2.2        	24.2.2.0.0   	Install complete
2       	Mon Dec 16 11:49:46 2024	deployed  	ocnrf-25.1.1000     	25.1.1000.0.0	Upgrade complete
  11. Perform sanity check using Helm test. See the Performing Helm Test section for the Helm test procedure.
  12. If the upgrade fails, see Upgrade or Rollback Failure in Oracle Communications Cloud Native Core, Network Repository Function Troubleshooting Guide.

4.5 Postupgrade Tasks

Note:

To automate the lifecycle management of the certificates through OCCM, you can migrate certificates and keys from NRF to OCCM. For more information, see "Introducing OCCM in an Existing NF Deployment" in Oracle Communications Cloud Native Core, Certificate Management User Guide.

You can remove Kubernetes secrets if the current version of NRF does not use that secret by checking the ocnrf_custom_values_25.2.201.yaml file. Before deleting, please make sure that there is no plan to rollback to the NRF version which uses these secrets. Otherwise Rollback will fail.

4.5.1 Alert Configuration

This section describes how to modify or update NRF alerts as needed, based on requirements, after performing the upgrade.

For more details, see the following sections: