Event Canal
Overview
The Unified Assurance Event Canal microservice provides the foundation for streaming analytics of events. The Event Canal is responsible for taking real-time events from the Event database and enqueueing them to a Pulsar topic. Multiple microservices can read from this same topic to create analytics pipelines for enrichment, suppression, north-bound integrations, or machine learning to name a few.
Prerequisites
-
A microservices cluster must be setup. Refer to Microservice Cluster Setup.
-
Apache Pulsar must be installed. Refer to Apache Pulsar microservice.
Setup
su - assure1
export NAMESPACE=a1-zone1-pri
export WEBFQDN=<Primary Presentation Web FQDN>
a1helm install event-canal assure1/event-canal -n $NAMESPACE --set global.imageRegistry=$WEBFQDN
Default Configuration
Name | Value | Possible Values | Notes |
---|---|---|---|
LOG_LEVEL | INFO | FATAL, ERROR, WARN, INFO, DEBUG | Logging level used by application. |
STREAM_INPUT | mysql:///Event | Text, 255 characters | Not configurable at this time. |
STREAM_OUTPUT | persistent://assure1/event/canal | Text, 255 characters | Apache Pulsar topic path. Topic at end of path may be any text value. |
REDUNDANCY_POLL_PERIOD | 5 | Integer | How often the secondary service checks to see if the primary is online, in seconds. |
REDUNDANCY_FAILOVER_THRESHOLD | 4 | Integer | The number of failed polls before the secondary becomes the active application. |
REDUNDANCY_FALLBACK_THRESHOLD | 1 | Integer | The number of successful polls before the secondary goes back to sleep. |
Configurations can be changed by passing the values to the a1helm install
prefixed with the configData parent key.
Example of setting the log level to DEBUG
a1helm install ... --set configData.LOG_LEVEL=DEBUG
Example of enabling redundancy
Redundancy will not be enabled by default. To enable this functionality, pass the following to the install command:
a1helm install ... --set redundancy.enabled=true
Schema
Event data is normalized into a common JSON schema described below. Any custom fields added to the Event.Events table will automatically be included in the JSON, but this example only shows the default fields provided out of the box.
Some fields are converted from their native format into ones appropriate for consumers of the output topic.
{
"@timestamp": "2006-01-02T15:04:05.999999999Z",
"_domain": "fault",
"_type": "event",
"_version": "5.0.0",
"event": {
"EventID": 0,
"EventKey": "",
"EventCategory": 3,
"EventType": "",
"Ack": 0,
"Action": "",
"Actor": "",
"Count": 1,
"Customer": "",
"Department": "",
"Details": {},
"DeviceType": "",
"Duration": 0.000,
"EscalationFlag": 0,
"ExpireTime": 0,
"FirstReported": "2006-01-02T15:04:05.999999999Z",
"GeoPath": {"type": "LineString", "coordinates": [[0, 0], [0, 0]]},
"GeoLocation": {"type": "Point", "coordinates": [0, 0]},
"IPAddress": "",
"LastChanged": "2006-01-02T15:04:05.999999999Z",
"LastReported": "2006-01-02T15:04:05.999999999Z",
"Location": "",
"Method": "",
"Node": "",
"OrigSeverity": 1,
"OwnerName": "",
"RootCauseFlag": 0,
"RootCauseID": 0,
"Score": 0,
"Service": "",
"ServiceImpact": 0,
"Severity": "Unknown",
"SubDeviceType": "",
"SubMethod": "",
"SubNode": "",
"Summary": "",
"TicketFlag": 0,
"TicketID": "",
"ZoneID": 0
}
}