Updating Unified Assurance

Learn about how to update Oracle Communications Unified Assurance packages to the latest version.

About Updating Unified Assurance

The Unified Assurance package system allows for near seamless installation of new packages and updates to installed packages. Installing updates is first done on the primary presentation server, then other servers can be updated as well.

The image shows the update process for four individual servers running Unified Assurance: a presentation server, a database server, a processing server, and a collection server. For simplicity, the example shows a non-redundant setup. You perform an update in the following order:

Unified Assurance Update

Description of illustration update-path.png

Pre-Update Tasks

To prepare to update Unified Assurance:

  1. If you are updating from version 6.0.4 or earlier, on all servers in the environment:

    1. Install RPM packages required by the Unified Assurance installation RPM by running the following command:

      dnf install selinux-policy-targeted policycoreutils policycoreutils-python-utils

      Although these packages are required to support SELinux, you must install them regardless of whether you are enabling SELinux or not.

    2. If you are planning to enable SELinux in an environment where it was previously disabled, perform all prerequisite steps described in Configuring SELinux.

  2. If you are updating from version 6.0.5 or earlier and have the Flow Collector microservice deployed, you must stop all data coming from all deployed Flow Collector instances to the Historical database. Scale down the microservice by running the following command for all instances of Flow Collector, as the assure1 user:

    a1k scale --replicas=0 deploy <flow-collector-release-name> -n <namespace>

    where:

    • <flow-collector-release-name> is the name of the microservice instance. This is typically flow-collector, but may be slightly different if you have multiple instances of the microservice deployed to the same cluster.

    • <namespace> is the namespace where the microservice is deployed. This is typically a1-zone1-pri, but may be slightly different if you deployed the microservice to a different zoned namespace or a redundant cluster.

    See Managing Microservices and Flow Collector in Unified Assurance Implementation Guide for information about microservices.

Updating Unified Assurance

After you have completed the pre-update tasks, update Unified Assurance:

  1. On the primary presentation server, download the latest available Unified Assurance patch, and extract the contents of the collection file to the installation directory where you initially installed Unified Assurance. This is typically /opt/install, and is referred to as <installation_directory> in the documentation.

    See Downloading the Latest Unified Assurance Software for more information about where to get the software and how to extract the collection file.

  2. In the Unified Assurance UI, from the Configuration menu, select Broker Control, then Servers.

  3. In the grid, select the primary presentation server.

  4. From the Version menu, select the version you want to install.

  5. Click Update Server(s) and wait for the Status column to show Complete.

    Updated packages are copied from the directory where they were extracted into the internal package location on the primary presentation server. After the package files are copied, packages that have updates on the primary presentation server are installed.

  6. Select all other servers by holding Ctrl or Shift and clicking the servers.

  7. Click Update Server(s).

    Packages that have updates available are downloaded from the primary presentation server to the other servers. After the packages are downloaded, packages that have updates on the server are installed.

    When all servers show Complete, the package update process is complete.

Proceed to Post-Update Tasks.

Post-Update Tasks

After updating Unified Assurance, complete post-update tasks, depending on the version you upgraded from and to:

Reauthenticating Before Accessing OpenSearch

After updating to version 6.1.1.0.0 from any previous version, before accessing any OpenSearch UIs (the Logs UI, and UIs accessed under Events and Flow in the Analytics menu), users with an active session must log out of Unified Assurance, and then log back in.

Adding Libraries for Oracle and PostgreSQL Databases

If you are updating to version 6.1.1.0.0 from any previous version, and you want to use an Oracle or PostgreSQL database with Unified Assurance, you must add required libraries:

  1. On all servers:

    1. Run the following commands as the root user:

      • For an Oracle database:

        dnf install libnsl

      • For a PostgreSQL database:

        dnf install libpq

    2. For Oracle databases only, copy over the library files:

      1. Switch to the assure1 user and create the following directory:

        su - assure1
mkdir -p $A1BASEDIR/lib/oracle/lib

      2. Copy the following files to the new directory from their location on your Oracle Database installation:

        • libclntshcore.so.19.1

        • libclntsh.so.19.1

        • libnnz19.so

        • libociicus.so

      3. Run the following command to confirm the directory structure:

        ls -al $A1BASEDIR/lib/oracle/lib/

        The output should look similar to this:

        drwxr-xr-x 2 assure1 assure1      100 Sep 22 16:07 .
drwxr-xr-x 3 assure1 assure1       17 Sep 22 16:07 ..
-rwxr-xr-x 1 assure1 assure1  8040416 Feb 17  2023 libclntshcore.so.19.1
-rwxr-xr-x 1 assure1 assure1 79927312 Feb 17  2023 libclntsh.so.19.1
-rw-r--r-- 1 assure1 assure1  6587832 Feb 17  2023 libnnz19.so
-rwxr-xr-x 1 assure1 assure1  6599152 Jun 17 08:48 libociicus.so

  2. (Optional) If you want to use the Databases UI to create and manage the databases, on each presentation server (primary, secondary, and external), edit the php.ini file:

    1. Switch to the assure1 user, if you did not already:

      su - assure1

    2. Open the php.ini file located in the $A1BASEDIR/etc directory.

    3. Locate the following lines and uncomment them by removing the leading ;:

      • For an Oracle database, change the following:

        ;extension=pdo_oci

        To:

        extension=pdo_oci

      • For a PostgreSQL database, change the following:

        ;extension=pdo_pgsql

        To:

        extension=pdo_pgsql

    4. Save and close the file.

  3. On the internal presentation server:

    1. Switch to the root user, if you did not already, and restart the Unified Assurance web service:

      systemctl restart assure1-web

    2. Stop and restart the DatabaseWatchdog service:

      1. Get the ID for the DatabaseWatchdog service:

        $A1BASEDIR/bin/BrokerControl --batch listservices | grep DatabaseWatchdog

      2. Stop and then start DatabaseWatchdog, using the returned ID:

        $A1BASEDIR/bin/BrokerControl --batch stopservice <ID>
$A1BASEDIR/bin/BrokerControl --batch startservice <ID>

      Tip:

      You can alternatively use the Services UI to restart the service.

Migrating Historical Data

If you are updating from version 6.0.5 or earlier to 6.1 or later, you must migrate your Historical data to OpenSearch, which replaced Elasticsearch as the Historical database in version 6.1.

Migration includes:

See About Migration in Unified Assurance Migration Guide for more information.

Redeploying Microservices

After updating from and to any version, you must update and redeploy all deployed microservices on all microservice cluster servers.

This topic shows you how to perform the tasks using the command line. You can also undeploy and redeploy microservices by using the UI. See Managing Microservices for more information.

To redeploy microservices:

  1. As the assure1 user, update the Helm repository:

    a1helm repo update

  2. As the root user, upgrade the cluster:

    $A1BASEDIR/bin/cluster/clusterctl upgrade

  3. Undeploy and redeploy each of the microservices listed above for your update path.

    To undeploy and redeploy the microservices:

    1. (Optional) Review the existing Helm chart and make note of any specific configuration settings:

      a1helm show values assure1/<helm-chart-name>

      where <helm-chart-name> is the name of the Helm chart for the microservice. For example, event-sink.

      Tip:

      You can run a1k get pods -A or a1k get deployments -A to see what pods and microservices are running in all namespaces.

    2. Undeploy the microservice:

      a1helm delete <microservice-release-name> -n <namespace>

      where:

      • <microservice-release-name> is the release name for the microservice deployment. For example, event-sink.

        In most cases, this is the same name as the microservice and Helm chart, but you may have used different values if you deployed multiple instances of the microservice in the same cluster.

      • <namespace> is the namespace where you deployed the microservice.

        The default zoned namespace is a1-zone1-pri, but you may have used other zones or secondary namespaces. Supporting microservices are deployed to non-zoned namespaces, such as a1-monitoring for Prometheus, a1-cache for Coherence, and a1-messaging for Pulsar.

      For example, to undeploy the Pulsar microservice:

      a1helm delete pulsar -n a1-messaging

      Any Pulsar backlog is also deleted.

      Tip:

      You can also use the UI to undeploy a microservice. See Undeploying a Microservice by Using the UI in Unified Assurance Implementation Guide.

    3. Redeploy the microservice, remembering to specify any configuration settings that you had previously used.

      See Deploying a Microservice in Unified Assurance Implementation Guide for information about deploying a microservice using the command line or the UI. The documentation for each microservice also provides specific commands and configuration details.

  4. Upgrade any other microservices that you did not have to undeploy:

    a1helm upgrade <microservice-release-name> -n <namespace>

    Note:

    You can only use a1helm upgrade to upgrade microservices that do not have configuration changes. You cannot use it to update configuration parameters in the Helm charts of running microservices.

Detaching and Rejoining Microservice Clusters

Unified Assurance version 6.1 includes a new version of Submariner.

To account for this change, if you are updating from version 6.0.5 or earlier to 6.1 or later, after you have updated all servers where microservices are deployed, upgraded the cluster, and redeployed microservices, you must detach and rejoin the paired servers.

  1. On each pair of servers in the cluster, run the following command as the root user from one of the pair:

    $A1BASEDIR/bin/cluster/clusterctl detach

    The command automatically identifies which cluster pair to detach based on the cluster association of the server host where the command was run.

  2. Rejoin the cluster by running the following command from any server in the cluster:

     $A1BASEDIR/bin/cluster/clusterctl join --primaryCluster <PrimaryHostFQDN> --secondaryCluster <SecondaryHostFQDN>

Enabling SELinux After Updating

Unified Assurance version 6.0.5 and later is supported on systems where SELinux is enabled and set to enforcing.

To enable SELinux after updating, do the following on all servers in the environment:

  1. Update the servers as described in Updating Unified Assurance, which includes performing the prerequisite steps described in Configuring SELinux and Installing Required Package Dependencies.

  2. Open the /etc/selinux/config file.

    Note:

    This is the core Linux etc directory, not the Unified Assurance $A1BASEDIR/etc directory.

  3. Change the SELINUX= line to SELINUX=enforcing.

  4. Save and close the file.

  5. Reboot the server.

  6. Run the ConfigSELinuxDocker application:

    $A1BASEDIR/bin/ConfigSELinuxDocker enable

    ConfigSELinuxDocker confirms that the required dependencies are met and sets up SELinux for the Unified Assurance instance of Docker.

Post Update Tasks for Vision

Unified Assurance version 6.1 and later supports zone IDs in Vision entities.

Because the entity name in Vision must be unique, if you had devices with the same name in different zones, you previously could only import the first device as an entity. Now, with the addition of zone IDs, you can import multiple devices with the same name and different zone IDs.

The update process automatically adds the default zone ID (1) to all existing entities.

If you are updating from version 6.0.5 or earlier to version 6.1 or later, run the Create Vision Entities job to import devices with duplicate names in different zones that you were previously unable to import:

  1. From the main Unified Assurance navigation menu, select Configuration, then Broker Control, and then Jobs.

  2. Select the Create Vision Entities job and click Run.

    Tip:

    You can find the job quickly by using the filter bar to filter by the name column.

Tasks when Updating from Version 6.0.3 or Earlier 6.0.X Versions

After updating from version 6.0.3 or earlier 6.0.X versions to 6.1 or later, perform these tasks:

  1. The Device picker combo box has been updated. Check the configuration of all Event List and Device Availability Pie dashboard panels to ensure that the correct device is selected.

  2. If you are using custom foreach loops in your rules files, you may receive an error similar to the following:

    experimental keys on scalar is now forbidden

    To resolve this issue, check your rules files for proper keys.

    For example, the following lines are invalid:

    foreach my $id ($hash_ref) {

    my @variants = keys $hash{$word}{variants};

    To fix them, dereference hash_ref and hash:

    foreach my $id (%{$hash_ref}) {

    my @variants = keys %{ $hash{$word}{variants} };

  3. A UI has been added to help add, remove, and make certain changes to fields in the Event.Events table, and a new column RootCauseKey has been added to the initial table layout. To apply the changes to the table, complete the following tasks in Adding Custom Fields:

    1. Task 1: Stop Event Services and Microservices

    2. Task 3: Apply the Changes to the Database

    3. Task 4: Start the MySQL Replication Data Importer

    4. Task 6: Start the Event Services and Microservices

    5. Task 7: Recreate the OpenSearch Dashboards Index

    6. Task 8: Verifying Functionality