Changing Default OpenSearch Settings for Observability Analytics

Learn about optional changes you can make to the default OpenSearch settings for Observability Analytics in Oracle Communications Unified Assurance.

Note:

If your installation uses OpenSearch database redundancy, you must make all changes on both presentation servers. A change made on one server is not automatically copied to the other server.

Changing OpenSearch Index Lifecycle Policies

By default, the eventanalytics and eventjournals indexes store data for 30 days in the Historical database. If, for example, you want to save hard drive space on a single server installation, you can change the retention period by changing the index lifecycle policy.

To change the default retention period, do the following on each presentation server:

  1. From the main navigation menu, select Analytics, then Events, then Administration, and then Management.

  2. Click Index Management.

  3. In the list of state management policies, click ism-<index>, where <index> is the name of the index you want to change, such as eventanalytics or eventjournals.

  4. Click Edit.

  5. Select JSON editor and click Continue.

    Note:

    Although it is quick to locate and update the appropriate field in the JSON editor, if you are not comfortable with JSON, you can use the visual editor instead.

  6. In the JSON document for the policy, locate the entry for the warm state. The default object for the warm state is:

    {
            "name": "warm",
            "actions": [
                {
                    "retry": {
                        "count": 3,
                        "backoff": "exponential",
                        "delay": "1m"
                    },
                    "force_merge": {
                        "max_num_segments": 1
                    }
                }
            ],
            "transitions": [
                {
                    "state_name": "delete",
                    "conditions": {
                        "min_index_age": "30d"
                    }
                }
            ]
        }

  7. Under transitions, for the transition to the delete state, change the value of the min_index_age property.

  8. Click Update.

    The new retention period will be applied to all new indexes only. Existing indexes retain the previous setting.

  9. (Optional) Apply the new retention period to existing indexes:

    1. From the OpenSearch menu, under Management, select Dev Tools.

      Note:

      You can alternatively submit REST requests using your preferred API client. See Update managed index policy in the OpenSearch ISM API documentation for information about this endpoint.

    2. Enter the following request, replacing <index> with the appropriate index name (eventanalytics or eventjournals):

      POST _plugins/_ism/change_policy/<index>-*
{
"policy_id": "ism-<index>"
}

    3. Click the green triangle on the first line of the request to submit it.

Caution:

When increasing the number of days to keep data for, you must account for any additional required hard drive space. You can estimate the amount of space required by looking at the existing indices, averaging the storage sizes, then multiplying that value by the number of days.

Discarding Incoming Records

By default, all records sent by the MySQL Replication Data Importer are inserted into the OpenSearch database. You can optionally discard some incoming records based on your requirements by adding a drop processor to the OpenSearch ingest pipeline. You add the processor by submitting a PUT request in the Observability Analytics console.

To add the drop processor, do the following on each presentation server:

  1. From the main navigation menu, select Analytics, then Events, then Administration, and then Console.

  2. Enter the following request:

    GET /_ingest/pipeline/eventanalytics

  3. Click the green triangle on the first line of the request to submit it.

    The console displays the current definition of the ingest pipeline. By default, this is:

    {
  "eventanalytics" : {
    "description" : "Parse Assure1 events",
    "processors" : [
      {
        "set" : {
          "field" : "Entity",
          "value" : "{{Node}}"
        }
      },
      {
        "set" : {
          "if" : "ctx.SubNode != ''",
          "field" : "Entity",
          "value" : "{{Node}}:{{SubNode}}"
        }
      }
    ]
  }
}

  4. Enter a PUT request to the same endpoint, adding a new drop processor to the existing processors from the GET request.

    For example, to add a processor that drops all records with severity 1 (unknown):

     _ingest/pipeline/eventanalytics 
    {
       "description": "Parse Assure1 events",
       "processors": [
          {
            "set" : {
              "value" : "{{Node}}",
              "field" : "Entity"
            }
          },
          {
            "set" : {
              "if" : "ctx.SubNode != ''",
              "value" : "{{Node}}:{{SubNode}}",
              "field" : "Entity"
            }
          },
          {
             "drop": {
                "if": "ctx.Severity == 1"
             }
          }
       ]
    }

  5. Click the green triangle on the first line of the request to submit it.

    The pipeline is updated with the new processor.

For more information about these requests, see Get pipeline and Create pipeline in the OpenSearch documentation.