Rolling Back UDR

5.1 Supported Rollback Paths

The following table lists the supported rollback paths for UDR:

Table 5-1 Supported Rollback Path

Source UDR Release	Target UDR Release
23.4.2	23.2.x, 23.3.x, and 23.4.x

To rollback to the previous UDR version, run the following command to check if the pods have successfully started (shows running state).

kubectl get pods -n <namespace_name>

Where, <namespace_name> is the UDR namespace name. For example, ocudr.

If the pod logs or describe pods command results are not sufficient to handle the issue, then rollback using Helm and proceed with Rollback DB Schema instructions.

5.2 Rollback Tasks

Run the following command to check if the pods are successfully started:

kubectl get pods -n <namespace_name>

Note:

You have to rollback if there are issues that cannot be recovered by checking the logs. You must perform a helm rollback first and then proceed with rollback database schema instructions.

Use the backup of the 23.3.x version's <ocudr-values.yaml> file to roll back from UDR 23.4.2 to a previous version:

Run the following command to obtain the revision number:
```
helm history <helm release name>
```
Example: helm history ocudr
Run the following rollback command.

Note:
For multisite deployments, run the rollback command for all sites.
```
helm rollback <helm release name> <revision_no>
```
Where, <helm release_name> is the release name used by the Helm command.

<revision_no> is the release number to which UDR needs to be rolled back.

Example: helm rollback ocudr 1

Run the following command when you upgrade from 23.2.x or 23.3.x to 23.4.2 and then rollback to 23.2.x or 23.3.x:

Login to mysql, "kubectl exec <dataNodes> -n <namespace> -it bash"
Login to mysql password: mysql -h 127.0.0.1 -uroot -pNextGenCne
    delete from ReleaseConfig where SiteId=<siteId> and CfgKey=ocudr;

To rollback multiple site deployment, see Rollback of Multiple Site Deployments.

The below steps are applicable only when there is a helm rollback failure:

If the helm rollback failure is intermittent, check the below error scenarios.
1. If the helm rollback fails with <helm-release-name>-nudr-config-server-pre-rollback/<helm-release-name>-nudr-config-server-post-rollback hook running into Error state, check the logs of the failed hook.
2. Perform Step 3, if the logs show any of the following errors:
  1. "message":"Unknown column 'tb.snapshot_id' in 'where clause'"
  2. "message":"Rolling back to the savepoint and exiting...","thrown":{"commonElementCount":0,"localizedMessage":"Table 'udrconfigdb.snapshot_mapping' doesn't exist","message":"Table udrconfigdb.snapshot_mapping' doesn't exist"
3. Update the database tables using the following workaround SQL statements:
  1. Run the following command to log in to ndbappsql node pods.
```
"kubectl exec <sqlnode> -n <ns> -it bash"
```
  2. Copy the below command in codeblock to a commands.sql file. The commands.sql file can be created using vim editor that is present inside mysql pod.
    SQL Commands
```
SET default_storage_engine=NDBCLUSTER;
use udrconfigdb;
drop table if exists snapshot_mapping;
CREATE TABLE IF NOT EXISTS  snapshot_mapping
            ( snapshot_id bigint(20) NOT NULL AUTO_INCREMENT, 
              `name` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
              `created_time` datetime DEFAULT CURRENT_TIMESTAMP,
             `restored_time` datetime DEFAULT NULL,
              PRIMARY KEY (`snapshot_id`), 
              UNIQUE KEY `name` (`name`) )  
              AUTO_INCREMENT=21 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
alter table topic_info_backup add column snapshot_id bigint;
alter table topic_info_backup add foreign key (snapshot_id) references snapshot_mapping(snapshot_id);
alter table configuration_item_backup add column snapshot_id bigint;
alter table configuration_item_backup add foreign key (snapshot_id) references snapshot_mapping(snapshot_id);
use udrdb;
delete from ReleaseConfig where SiteId="5a7bd676-ceeb-44bb-95e0-f6a55a328b03" and CfgKey="nrf-client-nfmanagement";
```
    Note:
    - The second SQL command use udrconfigdb mentioned above must refer to the configuration database name of UDR or SLF
    - The second last SQL command use udrdb mentioned above must refer to the subscriber database name of UDR or SLF.
    - The last SQL command SiteId mentioned above must be same as the one configured in global.nfInstanceId in the ocudr 23.2.0 custom-values.yaml file.
  3. Run the below command to load the SQL commands to the database.
```
mysql -h127.0.0.1 -u<username> -p<password> < commands.sql
```
After performing the above workaround, retry the rollback again using the helm upgrade command.

Rollback of Multiple Site Deployments

You must follow a specific order during rollback of multiple site deployment. You must rollback all cnDBTier sites and then rollback UDR across all sites. For example, in a three site setup you must follow the below order for NF and cnDBTier rollback:

Rollback site 1 cnDBTier
Rollback site 2 cnDBTier
Rollback site 3 cnDBTier
Rollback Site 1 UDR, Provisioning Gateway, CNC Console, and install the rollback version of the tools (subscriber bulk import and subscriber export tool) deployments if present.
Rollback Site 2 UDR, Provisioning Gateway, CNC Console, and install the rollback version of the tools (subscriber bulk import and subscriber export tool) deployments if present.
Rollback Site 3 UDR, Provisioning Gateway, CNC Console, and install the rollback version of the tools (subscriber bulk import and subscriber export tool) deployments if present.

The above rollback order is required because of the SQL rollback limitation applicable for cnDBTier:

If there are schema changes on any of the tables when running on the higher n+1 release version of cnDBTier, then when you rollback the NF to "n" version of cnDBtier, the schema changes which happened on the higher n+1 version will not be handled and the tables with these changes are dropped and are not re-created on the lower version. For more information, see the Prerequisites section in the Oracle Communications Cloud Native Core, cnDBTier Installation, Upgrade, and Fault Recovery Guide.
Rollback of cnDBTier must be performed carefully as per the above specified order because UDR, Provisioning Gateway, and Tools (subscriber bulk import and subscriber export tool) upgrade and rollback procedure includes database schema change.
There must not be database schema change in the current cnDBTier version to rollback. If there is any database schema change after cnDBtier is upgraded, then these schema changes must be reverted. You must contact My Oracle Support for assistance.

Note:

You must disable the conflict resolution feature during rollback. For more information, see "Conflict Resolution" section in Oracle Communications Cloud Native Core, Unified Data Repository User Guide. You must perform the schema changes before upgrading UDR components and cnDBTier. Schema changes performed during upgrade can be retained since the changes do not impact when you rollback the UDR version.
During the upgrade procedure if there is an issue on UDR, Provisioning Gateway, or Tools in any sites, then UDR, Provisioning Gateway, or Tools must be rolled back immediately. Do not rollback cnDBTier. cnDBTier must be rolled back only if there is a cnDBTier upgrade issues.
Verify the cnDBTier after the upgrade is complete. If there are issues, rollback the cnDBTier immediately. Do not rollback UDR, Provisioning Gateway, or Tools, they can remain on the higher version and are compatible with n-1 cnDBTier release.
To avoid failures during cnDBTier rollback due to schema changes (for example, helm upgrade, rollback performed on UDR, Provisioning Gateway, and subscriber bulk import and subscriber export tool deployments), you can drop the modified schema tables manually before rollback and re-create the modified schema tables after the rollback is complete. For more information, see the Prerequisites section in the Oracle Communications Cloud Native Core, cnDBTier Installation, Upgrade, and Fault Recovery Guide and contact My Oracle Support for assistance.

If the rollback fails, see the Debugging Upgrade or Rollback Failure section in the Oracle Communications Cloud Native Core, Unified Data Repository Troubleshooting Guide.

5.3 Postrollback Tasks

After performing rollback, restore preupgrade data obtained earlier through manual backup.

GET API for different resources can be used to see current values in UDR, then accordingly if any update required, individual service APIs defined for different resources can be used to reconfigure the data backup taken before upgrade. See Oracle Communications Cloud Native Core, Unified Data Repository REST Specification Guide for API details.

5 Rolling Back UDR

5.1 Supported Rollback Paths

5.2 Rollback Tasks

5.3 Postrollback Tasks