Event Canal

Overview

The Unified Assurance Event Canal microservice provides the foundation for streaming analytics of events. The Event Canal is responsible for taking real-time events from the Event database and enqueueing them to a Pulsar topic. Multiple microservices can read from this same topic to create analytics pipelines for enrichment, suppression, north-bound integrations, or machine learning to name a few.

Prerequisites

  1. A microservices cluster must be setup. Refer to Microservice Cluster Setup.

  2. Apache Pulsar must be installed. Refer to Apache Pulsar microservice.

Setup

su - assure1
export NAMESPACE=a1-zone1-pri
export WEBFQDN=<Primary Presentation Web FQDN> 
a1helm install event-canal assure1/event-canal -n $NAMESPACE --set global.imageRegistry=$WEBFQDN

Default Configuration

Name Value Possible Values Notes
LOG_LEVEL INFO FATAL, ERROR, WARN, INFO, DEBUG Logging level used by application.
STREAM_INPUT mysql:///Event Text, 255 characters Not configurable at this time.
STREAM_OUTPUT persistent://assure1/event/canal Text, 255 characters Apache Pulsar topic path. Topic at end of path may be any text value.
REDUNDANCY_POLL_PERIOD 5 Integer How often the secondary service checks to see if the primary is online, in seconds.
REDUNDANCY_FAILOVER_THRESHOLD 4 Integer The number of failed polls before the secondary becomes the active application.
REDUNDANCY_FALLBACK_THRESHOLD 1 Integer The number of successful polls before the secondary goes back to sleep.

Configurations can be changed by passing the values to the a1helm install prefixed with the configData parent key.

Example of setting the log level to DEBUG

a1helm install ... --set configData.LOG_LEVEL=DEBUG

Example of enabling redundancy

Redundancy will not be enabled by default. To enable this functionality, pass the following to the installation command:

a1helm install ... --set redundancy.enabled=true

Schema

Event data is normalized into a common JSON schema described below. Any custom fields added to the Event.Events table will automatically be included in the JSON, but this example only shows the default fields provided out of the box.

Some fields are converted from their native format into ones appropriate for consumers of the output topic.

{
    "@timestamp": "2006-01-02T15:04:05.999999999Z",
    "_domain":    "fault",
    "_type":      "event",
    "_version":   "5.0.0",
    "event": {
        "EventID":        0,
        "EventKey":       "",
        "EventCategory":  3,
        "EventType":      "",
        "Ack":            0,
        "Action":         "",
        "Actor":          "",
        "Count":          1,
        "Customer":       "",
        "Department":     "",
        "Details":        {},
        "DeviceType":     "",
        "Duration":       0.000,
        "EscalationFlag": 0,
        "ExpireTime":     0,
        "FirstReported":  "2006-01-02T15:04:05.999999999Z",
        "GeoPath":        {"type": "LineString", "coordinates": [[0, 0], [0, 0]]},
        "GeoLocation":    {"type": "Point", "coordinates": [0, 0]},
        "IPAddress":      "",
        "LastChanged":    "2006-01-02T15:04:05.999999999Z",
        "LastReported":   "2006-01-02T15:04:05.999999999Z",
        "Location":       "",
        "Method":         "",
        "Node":           "",
        "OrigSeverity":   1,
        "OwnerName":      "",
        "RootCauseFlag":  0,
        "RootCauseID":    0,
        "Score":          0,
        "Service":        "",
        "ServiceImpact":  0,
        "Severity":       "Unknown",
        "SubDeviceType":  "",
        "SubMethod":      "",
        "SubNode":        "",
        "Summary":        "",
        "TicketFlag":     0,
        "TicketID":       "",
        "ZoneID":         0
    }
}

Microservice self-metrics

The Unified Assurance Event Canal microservice exposes the following self-metrics to Prometheus.

Metric Name Type Description
total_events_processed Counter Number of events processed
processing_time_per_event Gauge Processing time per event from receiving it to sending it to acknowledging it