Adding Nodes to the Historical Database Cluster

The first time you set up a Historical database server in your Oracle Communications Unified Assurance environment, after installing Unified Assurance on the server, you create a database cluster when you run AnalyticsWizard. Initially, this is a standalone cluster with a single node.

You can add new nodes and set up a redundant cluster right away or later, by installing Unified Assurance on additional servers and running AnalyticsWizard on them.

Before running AnalyticsWizard to add a new node:

Confirm that the Historical database cluster is in green or yellow status. See Checking Cluster Status.
If you migrated your Historical database from Elasticsearch to OpenSearch, confirm that you have migrated all indexed documents and finalized the migration to stop Elasticsearch and Kibana and remove all related artifacts. See Finalizing the Migration.
Use an override to set the path.repo configuration parameter to the backup directory for OpenSearch. This is required for creating snapshots of the database for backing up and restoring. See Backing Up an OpenSearch Database in Unified Assurance System Administrator's Guide and Overriding Default Vendor Configurations for more information about overrides.

AnalyticsWizard automatically detects if a server is primary or redundant and if a Historical database cluster already exists in the environment. If the database exists, AnalyticsWizard prompts you to add the server to the cluster.

After running AnalyticsWizard:

Update the number of replicas for event, flow, journal, and log indexes. See Increasing Replicas for Indexes.
If you also have a microservice cluster, for example, if you are using Flow Collector for Flow Analytics, update the microservice cluster configuration. See Updating Microservice Cluster Configurations.

For information about installing Unified Assurance on a new server and running AnalyticsWizard, see the following in Unified Assurance Installation Guide:

For AnalyticsWizard usage details, see AnalyticsWizard.

Checking Cluster Status

You can use the OpenSearch administration console to query the Historical database for the cluster status.

From the main navigation menu, select Analytics, then Events, then Administration, and then Console.
Enter the following request in the console and click the arrow to run the request:
```
GET _cluster/health
```
The response should contain "status": "green" or "status": "yellow". Either status means all primary shards are allocated to nodes.

In the OpenSearch documentation, see Cluster health for information about the fields returned in the response and Cluster APIs for general information about the OpenSearch cluster APIs.

Increasing Replicas for Indexes

When you add a new node to the cluster, the default number of replicas for indexes is 0. To support high availability, you must update the number of replicas for each index in the index templates.

From the main Unified Assurance navigation menu, select Analytics, then Events, then Administration, and then Management.
Click Index Management.
From the list on the left, under Index Management, click Templates.
For each of the default templates, repeat the following:
1. Click the template name. For example, template-eventanalytics.
2. Click the Configuration tab.
3. Under Template Definition, set Number of replicas to a number greater than 0, but no more than the number of nodes in the cluster.
  
  Note:
  
  By default, Number of primary shards is set to 1. The number of replicas applies to each shard, so if you are using more shards, you will need more nodes in the cluster. For example, if you set shards to 2 and replicas to 2, there will be two replicas for each shard, and you must have at least three nodes in the cluster.
4. Click Save.
The number of replicas will be applied to all new indexes based on the templates.
(Optional) Apply the new number of replicas to existing indexes:
1. From the OpenSearch menu, under Management, select Dev Tools.
  
  Note:
  
  You can alternatively submit REST requests using your preferred API client. See Update Settings API in the OpenSearch API documentation for information about this endpoint.
2. Enter the following request, replacing <index> with the appropriate index name (eventanalytics or eventjournals) and <number> with the appropriate number of replicas:
```
PUT /<index>-*/_settings
{
  "index": {
    "number_of_replicas": <number>
  }
}
```
3. Click the green triangle on the first line of the request to submit it.
Note:

Because the number of replicas is a dynamic setting, you can change it as shown without closing the index. However, the number of shards is a static setting, which you cannot change after an index has been created. See Static index-level index settings in the OpenSearch documentation for more information about setting and updating static settings.

Updating Microservice Cluster Configurations

During an initial installation, if you are using Flow Analytics, you typically run AnalyticsWizard on all servers in the Historical database cluster before setting up a microservice cluster for the Flow Collector microservice. In this situation, the microservice cluster is created with the correct information about the Historical database nodes.

However, in some situations, such as during database migration or when adding new nodes to an existing Historical database cluster later, you run AnalyticsWizard after the microservice cluster is already set up. In these situations, you must update the microservice cluster configuration with the new Historical database information.

To do this, run the following command as the root user on one of the servers where the Cluster.Master role is installed:

$A1BASEDIR/bin/cluster/clusterctl update-config

Title and Copyright Information

Implementation Guide

G36451-01