Disaster Recovery Scenarios

Scenario 1: Complete Site Failure

This section describes how to perform disaster recovery when either one, many, or all of the sites have software failure.

The following are site failure scenarios:

Scenario 1A: Single or Multiple Site Failure

This scenario applies when one or more sites, and not all sites, have failed and there is a requirement to perform disaster recovery. It is assumed that the user has DBTier and Oracle Communications CNC Console installed on multiple sites with automatic data replication and backup enabled.

To recover the failed sites:

Run the Cloud Native Environment (CNE) installation procedure to install a new cluster. For more information, see Oracle Communications Cloud Native Environment (OCCNE) Installation Guide.
For DBTier disaster recovery:
1. Take on-demand backup from the mate site that has health replication with the failed site or sites. For more information about on-demand backup, see the Create On-demand Database Backup chapter in the Oracle Communications Cloud Native Core DBTier Disaster Recovery Guide.
2. Use the backup data from the mate site to restore the database. For more information about database restore, see the Restore Georeplication Failure chapter in the Oracle Communications Cloud Native Core DBTier Disaster Recovery Guide.
Install CNC Console helm chart. For more information about installing CNC Console, see the Installing CNC Console chapter in the Oracle Communications Cloud Native Core Console Installation and Upgrade Guide.

Scenario 1B: All Sites Failure

This scenario applies when all sites have failed and there is a requirement to perform disaster recovery. It is assumed that the user has DBTier and Oracle Communications CNC Console installed on multiple sites with automatic data replication and backup enabled.

To recover all the failed sites:

Run the Cloud Native Environment (CNE) installation procedure to install a new cluster. For more information, see Oracle Communications Cloud Native Environment (OCCNE) Installation Guide.
Use an on-demand backup file to restore the database from previous data backup. For more information about database restore, see the Restore Georeplication Failure chapter in the Oracle Communications Cloud Native Core DBTier Disaster Recovery Guide.
Note:
- The auto-data backup file is one that is built from scheduled automatic backup.
- The Restore Georeplication Failure chapter contains a procedure for two sites where one of the clusters has a fatal error. You can perform the same procedure for all the sites in a multiple site setup.
Install CNC Console helm chart. For more information about installing CNC Console, see the Installing CNC Console chapter in the Oracle Communications Cloud Native Core Console Installation and Upgrade Guide.

Scenario 2: cnDBTier Corruption

This section describes how to recover a database when the data replication has failed due to database corruption and cnDBTier has failed in single, multiple sites, or all sites.

When the database gets corrupted, the database on all the other sites can also get corrupted due to data replication. It depends on the replication status after the corruption has occurred. If the data replication fails due to database corruption, then DBTier fails in either single or multiple sites (not all sites). If the data replication is successful, then database corruption replicates to all the cnDBTier sites and cnDBTier fails in all sites.

The following are cnDBTier failure scenarios:

If corrupted database is replicated to mated sites, follow:

When cnDBTier fails in Single or Multiple (but not all) Sites

If corrupted database causes replication failure and hence local to a site, follow:

When cnDBTier fails in all Sites

Note:
This impacts all the NFs using the corrupted cnDBTier. All the NFs sharing cnDBTier needs to do a disaster recovery as cnDBTier is corrupted.

Scenario 2A: When DBTier fails in Single or Multiple (but not all) Sites

This section describes how to recover database when the data replication has failed due to database corruption and DBTier has failed in either single or multiple sites (not all sites).

To recover database:

Uninstall CNC Console helm chart. For information about uninstalling CNC Console, see the Uninstalling CNC Console chapter in the Oracle Communications Cloud Native Core Console Installation and Upgrade Guide.
For DBTier disaster recovery:
1. Create on-demand backup from mated site that has health replication with failed site. For more information about DBTier backup, see the Create On-demand Database Backup chapter in the Oracle Communications Cloud Native Core DBTier Disaster Recovery Guide.
2. Use the backup data from mate site for restoration. For more information about DBTier restore, see the Restore Georeplication Failure chapter in Oracle Communications Cloud Native Core DBTier Disaster Recovery Guide.
  
  Note:
  The Restore Georeplication Failure chapter in Oracle Communications Cloud Native Core DBTier Disaster Recovery Guide. has a procedure for two sites where one of the cluster has fatal error. You can perform that procedure for all the sites in a multiple site setup.
Install CNC Console helm chart. For more information about installing CNC Console, see the Installing CNC Console chapter in the Oracle Communication Cloud Native Core Console Installation and Upgrade Guide.

Scenario 2B: When DBTier failed in all Sites

This section describes how to recover database when successful data replication corrupts all the DBTier sites.

To recover database:

Uninstall CNC Console helm charts. For more information about uninstalling CNC Console, see the Uninstalling CNC Console chapter in the Oracle Communications Cloud Native Core Console Installation and Upgrade Guide.
For DBTier disaster recovery:
1. Use an on-demand backup file to restore the database from the previous data backup. For more information about DBTier restore, see the Restore Georeplication Failure chapter in the Oracle Communications Cloud Native Core DBTier Disaster Recovery Guide.
  
  Note:
  The Restore Georeplication Failure chapter has a procedure for two sites where one of the cluster has fatal error. You can perform that procedure for all the sites in a multiple site setup.
Install CNC Console helm charts. For more information about installing CNC Console, see the Installing CNC Console chapter in the Oracle Communication Cloud Native Core Console Installation and Upgrade Guide.

Scenario 3: Console Configuration Database Corruption

This section describes how to recover when the CNC Console configuration database is corrupted.

For recovery and restore procedure, see Disaster Recovery Procedures - DB Backup and Restore section.

Scenario 4: Deployment Failure

This section describes how to recover CNC Console when its deployment fails.

For recovery and restore procedure, see Restoring CNC Console.

Scenario 5: NF Instance Failure

Perform this procedure to recover from NF instance failure.

Refer to the procedures in the specific NF Disaster Recovery Guide to find the necessary action required to be taken.
Check whether the NF endpoints are the same.
1. If the NF endpoints are the same, no change is required in CNC Console side.
2. If the NF endpoints are different, update the NF IP and Port in the occncc_custom_values_<version>.yaml file and perform a helm upgrade operation to incorporate the new NF URL.