Unified Assurance Event Custom Action Policy Engine (CAPE)

Overview

The Unified Assurance Event Custom Action Policy Engine (CAPE) is a policy-based, multi-threaded, multi-headed, and flexible correlation engine. It can be configured to run a variety of custom tasks on the Event stream before other actions are taken. CAPE allows for post-collection processing for enrichment and correlation outside of the database environment. Mix-and-match connectors (Generic Connector, Ticketing Connectors, Probable Root Cause Analysis) and agents (time of day correlation, time of day, existence or absence of events) and use any internal or external datasets to create custom correlation models. Utilize CAPE for northbound integration of Unified Assurance with an external ticketing system, enrichment/correlation of data, custom notifications as well as auto-remediation.

CAPE is comprised of two pieces:

Custom Action Policy Engine (CAPE) Setup

  1. Add CAPE Nodes or modify existing CAPE Nodes which are executed when a CAPE Policy detects an event that matches the criteria:

    Configuration -> Events -> CAPE -> Nodes

  2. Add CAPE Policies or modify existing CAPE Policies that will poll the Events database for the specified events. Select the primary Custom Action Node created in the First Node Executed field.

    Configuration -> Events -> CAPE -> Policies

  3. Enable the default Service, unless a specific configuration option is needed:

    Configuration -> Broker Control -> Services

Default Service

Field Value
Package Name coreProcessing-app
Service Name Event Custom Action Policy Engine (CAPE)
Service Program bin/core/processing/CustomPolicyEngined
Service Arguments
Service Description Daemon executes custom policies which execute custom nodes
Failover Type Standalone (Supported: Standalone, Primary/Backup, Cluster)
Status Disabled
Privileged (Checked)

Default Configuration

Name Value Possible Values Notes
CheckTime 900 Integer How often (in seconds) the application checks for new and removes old policies.
LogFile logs/EventCustomPolicyEngine.log Text, 255 characters Relative path to Log File.
LogLevel ERROR OFF, FATAL, ERROR, WARN, INFO, DEBUG Logging level used by application.
ShardID 1 Integer Events Database shard to use. 0 to read from all shards
Threads 3 Integer The number of process threads created.
DeviceZoneID Integer Optional - Runs policies within Policy (Device) Zone. 0 for all zones (default).

Node Rules

CAPE Nodes are 100% Perl syntax. Refer to the following articles to assist in node rules creation:

Tokens

CAPE exposes the following tokens for node processing.

Token Description
$AppConfig Hash reference to the application configuration name-value pairs that were configured. (i.e. use $AppConfig->{'Host'} to retrieve the set value for 'Host'.)
$CustomHash Custom key, value cache available across all nodes. Contents commonly defined in the first node of a policy and then used throughout the nodes. NOTE: This variable is a shared object and any additional sub hashes or arrays must be shared before use or it will cause the error: "Invalid value for shared scalar". Instantiate the sub hash/array using '&share({})' e.g. 
$CustomHash->{SubObject} = &share({});
$EventData Resulting data from Policy. Use $EventData->{'FieldName'} to access the "FieldName" data. When processing multiple events together, this is a hash of events keyed by the order it was returned from the query. Each row contains the reserved data field ShardID which is the ShardID it was read from.
$EventDBH Events Database Handle connected to configured shard. Usable by nodes to make updates, read additional information, etc. If ShardID is 0, this will be ShardID 1 only.
$StorageHash Internal cache used as the StorageHash option when calling functions such as FindDeviceID(). NOTE: The structure of this cache is subject to change! Not recommended for custom global storage or manual manipulation; use $CustomHash.

Example CAPE Setup

This example will demonstrate how threshold violation events can be checked to see if the device is reachable or not.

Dependencies

Steps

  1. Via the UI, go to the Nodes UI:

    Configuration -> Events -> CAPE -> Nodes

  2. Click on the Add button to bring up the form.

  3. Give the Node a useful name and description.

  4. Use the code below in the Node Text section:

    # This section grabs the values selected in the 'SELECT' statement used in the CAPE Policy configuration. Only things referenced in the SQL SELECT statement are available within the CAPE Node.
my $EventID   = $EventData->{'EventID'};
my $IPAddress = $EventData->{'IPAddress'};
my $Severity  = $EventData->{'Severity'};

# Due to the SQL code used, every event that gets pulled back are DeviceUpDown events, which get created due to the Threshold Engine for the Device Down threshold. The CAPE Policy created will now attempt to ping the device and store the result of the attempt into the $ping variable.
$Log->Message("DEBUG", "Running Ping check command");
my $ping = `ping -c10 $IPAddress`;
$Log->Message("DEBUG", "Result of Ping test ----> [ $ping ]");

# If the result of the ping contains the text 'ttl', the ping was successful, so the event severity will be set to '0' (cleared) and a log message is saved to the CAPE log.  If the ping failed, the event severity will be set to '5' (critical).
if ($ping =~ m/ttl/i){
   $Log->Message("DEBUG", "Result of Ping Test successful, suppressing event");
   $Severity = 0;
}
else {
   $Log->Message("DEBUG", "Result of Ping Test failed, escalating event");
   $Severity = 5;
}

# An Alarm Journal entry is created for the EventID in question with the output of the ping from the step above.
$Log->Message("DEBUG", "Inserting into Journal [$EventID:$ping]");
my ($ErrorFlag, $Message) = AddJournal({
    DBH       => \$EventDBH,
    EventID   => $EventID,
    TimeStamp => time(),
    Username  => '1',
    Entry     => $ping,
    ShardID   => $AppConfig->{'ShardID'}
});

# The next step is to update the Alarm with new information. The severity is updated (if ping was successful), and also update Custom5 so that this event is not caught be the CAPE Policy SQL code again.  This prevents CAPE re-processing events it has previously acted upon.
$Log->Message("DEBUG", "Updating alarm entry [$EventID:$Severity]");
my ($ErrorFlag, $Message) = UpdateEvent({
    DBH     => \$EventDBH,
    EventID => $EventID,
    ShardID => $AppConfig->{'ShardID'},
    Values  => {
        Severity => $Severity
    }
});

  5. Via the UI, go to the Policies UI:

    Configuration -> Events -> CAPE -> Policies

  6. Click on the Add button to bring up the form.

  7. Give the Policy a useful name and description.

  8. Set the Poll Interval to 5 minutes; this will cause the SQL to be run every 5 minutes against the event list.

  9. Select the First Node Executed to be the corresponding Node created in the previous steps.

  10. Use the query below Event Select Statement:

    SELECT EventID, Severity, IPAddress
  FROM Events 
 WHERE Severity   = 5 
   AND AlarmGroup = "DeviceUpDown"

  11. Enable the default Service, unless a specific configuration option is needed:

    Configuration -> Broker Control -> Services

Administration Details

The following list shows you the technical details you’ll need for advanced administration of the application: