FCOM Processor
Overview
The Unified Assurance FCOM Processor microservice is part of the microservice event pipeline. The FCOM Processor is responsible for taking collected fault data from an input topic, running it through fault common object models (FCOM) to normalize into an event structure, and sending the output to an output topic. The default output topic is the Event Sink microservice, but this can be configured to any topic for additional enrichment or suppression.
Prerequisites
-
A microservices cluster must be setup. Refer to Microservice Cluster Setup.
-
Apache Pulsar must be installed. Refer to Apache Pulsar microservice.
Setup
su - assure1
export NAMESPACE=a1-zone1-pri
export WEBFQDN=<Primary Presentation Web FQDN>
a1helm install fcom-processor assure1/fcom-processor -n $NAMESPACE --set global.imageRegistry=$WEBFQDN
Default Configuration
Name | Value | Possible Values | Notes |
---|---|---|---|
LOG_LEVEL | INFO | FATAL, ERROR, WARN, INFO, DEBUG | Logging level used by application. |
STREAM_INPUT | persistent://assure1/event/collection | Text, 255 characters | Apache Pulsar topic path. Topic at end of path may be any text value. |
STREAM_OUTPUT | persistent://assure1/event/sink | Text, 255 characters | Apache Pulsar topic path. Topic at end of path may be any text value. |
FCOM_FILES_LOCATION | core/default/processing/event/fcom | Text | SVN Path where Overrides, Lookups, FCOMs, and Grok files are located. |
backlogSize | 1000 | Integer | The number of items that need to be in the backlog before the auto-scaling starts additional processes. |
Configurations can be changed by passing the values to the a1helm install
prefixed with the configData parent key.
Example of setting the log level to DEBUG
a1helm install ... --set configData.LOG_LEVEL=DEBUG
Example of changing the auto-scaling value
Auto-scaling is enabled by default. To change the thresholds used for this functionality, pass the following to the install command:
a1helm install ... --set autoscaling.thresholds.backlogSize=2000
FCOM Overrides
What is an FCOM Override?
An FCOM Override is an authoritative, language-neutral format written in JSON that enables the ability to override or build upon the basic functionality across the execution steps of the FCOM Processor.
How do FCOM Overrides Work?
After an FCOM Override is written and uploaded to the FCOM_FILES_LOCATION path in SVN, the fcom-processor needs to be restarted to load the new rules. Once messages start coming in, the defined overrides run if they match the scope or objectName defined in any override.
There are four different stages during processing where an override can run. In order, these stages are:
-
pre global - This would run on all messages that are received before they are converted to an Event.
-
pre object specific - This would only run on a message that has an objectName that is explicitly defined in the override before it is converted to an Event
-
post object specific - This would only run on a message that has an objectName that is explicitly defined in the override after it is converted to an Event
-
post global - This would run on all messages that are received after they are converted to an Event.
JSON Path
There are a handful of JSON Paths which can be used to access locations outside of the message you're currently referencing. These are:
-
$.lookups
- This path can be used reference a lookup file value. The full breakdown of the path is
$.lookups.{lookupfile name}.{key}
.
- This path can be used reference a lookup file value. The full breakdown of the path is
-
$.foreach
- This path stores key/val values only during the lifecycle of an iteration when using a foreach processor. The full key for the key value would be
$.foreach.{keyVal}
while the val value would be$.foreach.{valField}
. Look at the foreach processor definition for more information.
- This path stores key/val values only during the lifecycle of an iteration when using a foreach processor. The full key for the key value would be
-
$.localmem
- This is a local memory location. It's cleared on a per event basis, so you can set a local memory value like
$.localmem.test
in a pre-processor and then reference it in a post processor.
- This is a local memory location. It's cleared on a per event basis, so you can set a local memory value like
-
$.globalmem
- This is a global memory location which is accessible from all pods within a deployment. It requires redis to be running in the cluster. The values are only cleared if the redis installation is restarted or another processor removes it. You can set a global memory value using like:
$.globalmem.test
.
- This is a global memory location which is accessible from all pods within a deployment. It requires redis to be running in the cluster. The values are only cleared if the redis installation is restarted or another processor removes it. You can set a global memory value using like:
-
$.error
- This path holds an error message based off the current state. For example, if a processor throws an error during runtime and you're attempting to set a field/print the error you can reference the error by doing
$.error.message
in a processor.
- This path holds an error message based off the current state. For example, if a processor throws an error during runtime and you're attempting to set a field/print the error you can reference the error by doing
Override Format
Name | Required | Possible Values | Notes |
---|---|---|---|
name | yes | Text | Name of the override file |
description | no | Text | Description of the override. |
domain | yes | fault | The domain this message belongs to. This should always be fault for fcom overrides |
method | yes | trap, syslog | Whether or not this FCOM applies to trap or syslog messages |
scope | yes | pre, post | Whether or not the override occurs before or after message conversion |
version | yes | v1, v2 | The override file version. |
@objectName | yes | GLOBAL or Text | Whether or not the override will run on all messages ( GLOBAL ) or a specific message uniquely identified by an object name |
_type | yes | override | The type of file |
processors | yes | Array of Processors | An array of processors executed sequentially on the message. Please see the processor documentation below. |
Example:
{
"name": "Override Example",
"description": "This is an example of a global FCOM override",
"domain": "fault",
"method": "trap",
"scope": "post",
"version": "v2",
"@objectName": "GLOBAL",
"_type": "override",
"processors": [
{
"set": {
"source": "Hello, this is an example of overriding the event Summary field",
"targetField": "$.event.Details"
}
},
{
"set": {
"source": "Switch is down with value: %+v",
"args": [ "$.event.Details.trap.variables.0" ]
"targetField": "$.event.EventType"
}
}
]
}
Processors
append
Appends the value of source to the specific array and stores in the targetField
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | Text, JSON Path, Number, Boolean, Object | The source value to append into the array. |
array | yes | JSON Path or Array | The array which the processor will append the value to |
targetField | yes | JSON Path | The path where value of the array after executing will be stored |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"append": {
"source": "Example Value",
"array": [],
"targetField": "$.event.NewArray"
}
}
The value of $.event.NewArray
will be [ "Example Value" ]
after execution.
appendToOutputStream
Immediately sends the source value to an external output stream. Currently in-cluster pulsar topics are supported.
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | Text, JSON Path, Number, Boolean, Object | The source value to send to the output stream |
output | yes | pulsar://{hostname}/{namespace}/{topic} | The URI of the output stream. Only in-cluster pulsar:// is supported at the moment. |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"appendToOutputStream": {
"source": "$.trap",
"output": "pulsar+ssl:///assure1/event/sink"
}
}
The value the JSON Path $.trap
references will be sent to the assure1 event/sink topic within the cluster.
break
Immediately breaks from the enclosing foreach loop.
Example
{
"break": {
}
}
convert
Does a conversion of the value of source into the targetField based on the type of conversion.
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | Text, JSON Path, Number | The source value to convert |
type | yes | inttostring, stringtoint, oidtoip, oidtomac | The type of conversion |
targetField | yes | JSON Path | The location where the result will be stored |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"convert": {
"source": "$.event.Count",
"type": "inttostring",
"targetField": "$.event.CountString",
"ignoreFailure": true
}
}
The value of $.event.CountString
will be the value of count in a string format, if it throws an error while converting the value of $event.Count
it will continue on to the next processor in the override.
copy
Copy the value of source to a target location
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | Text, JSON Path | The value to copy |
targetField | yes | JSON Path | The location where the result will be stored |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"copy": {
"source": "$.event.Count",
"targetField: "$.event.CopiedCount"
}
}
The value of $.event.CopiedCOunt
will be the same as "$.event.Count"
date
Converts a source date and stores it in a target field
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | Text, JSON Path | If the value is empty it will use the current timestamp in UTC |
offset | no | Text | The duration string to apply to the source time. Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". |
timezone | no | JSON Path | The timezone to convert the source to |
targetField | yes | JSON Path | The location where the result will be stored |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"date": {
"source": "",
"offset": "-2h45m",
"timezone": "America/New_York",
"targetField: "$.event.CurrentTimeInEST"
}
}
The value of $.event.CurrentTimeInEST
will be the current time in EST subtracted by 2h 45m.
discard
Immediately discards a message
Example:
{
"discard": {
}
}
This would immediately discard the message - which means that successive overrides won't be run nor will it be sent to the output topic, and the fcom-processor will continue on to the next message.
foreach
Loops over an array or an object ( key-val map )
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | Text, JSON Path, Array, Object | The array/object/string or JSON Path reference to loop over. |
keyField | yes | Text | The variable used to reference the value of the key during each iteration. |
valField | yes | Text | The variable used to reference the value of the value during each iteration. |
then | yes | Array of Processors | A list of processors to execute during each iteration |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"foreach": {
"source": "$.event.Details.trap.variables",
"keyField": "i",
"valField": "c",
"then": [
{
"log": {
"source": "The index is %v and the value is %v",
"args": [
"$.foreach.i",
"$.foreach.c.value"
],
"type": "info"
}
},
{
"if": {
"conditions": {
"and": [
{
"property": "$.foreach.i",
"operator": ">",
"value": 3
}
]
},
"then": [
{
"break": {}
}
],
"else": []
}
}
]
}
}
This foreach loops logs The index is %v and the value is %v
to the screen until it reaches the fourth index of the array. It then breaks out of the loop, stopping execution.
grok
Executes a pre-defined grok pattern on a string and stores the result in a target object.
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | Text, JSON Path | The string or JSON Path reference to a string which the grok pattern will be run on |
pattern | yes | Text | The grok pattern to parse the source string with |
targetField | yes | JSON Path | The location where the results will be stored |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"grok": {
"source": "$.syslog.datagram",
"pattern": "%LINK-5-CHANGED: Interface %{INTERFACE:interface}, changed state to %{STATUS:status}",
"targetField": "$.syslog.variables"
}
}
The result of $.syslog.variables
will be:
"variables": {
"interface": "someinterfacename",
"status": "somestatusstring"
}
if
This processor executes an array of processors if the list of specified conditions are true or executes a different set of processors if it's false set of actions if it is false.
Name | Required | Possible Values | Notes |
---|---|---|---|
conditions | yes | Object | An object of and/or conditions which are evaluated |
then | yes | Array of Processors | Processors executed if the conditions evaluate to true |
else | yes | Array of Processors | Processors executed if the conditions evaluate to false |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example 1
{
"if": {
"conditions": {
"and": [
{
"property": "$.event.Count",
"operator": "==",
"value": 3
}
]
},
"then": [
{
"log": {
"source": "The condition is true"
}
}
],
"else": []
}
}
This will result in a debug log message being printed to the screen if the event count is equal to 3
Example 2
{
"if": {
"conditions": {
"or": [
{
"property": "$.event.Count",
"operator": "==",
"value": 3
},
{
"property": "$.event.Count",
"operator": ">=",
"value": 1000
}
]
},
"then": [
{
"log": {
"source": "The condition is true"
}
}
],
"else": []
}
}
This will result in a debug log message being printed to the screen if the event count is equal to 3 or greater than or equal to 1000
interpolate
Interpolates the JSON Path variables contained in the source with the values which exist in the message currently being processed.
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | Text, JSON Path* | The string or JSON Path reference which will be interpolated. Variables must be in a JSON Path format and cannot contain the @ character |
targetField | yes | JSON Path | The new JSON Path where the interpolated string will be stored |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"interpolate": {
"source": "The $.event.EventType event expires in $.event.ExpireTime seconds",
"targetField": "$.event.Summary"
}
}
Unknown
and an ExpireTime value of 3600
. The Summary field will have a value of The Unknown event expires in 3600 seconds
after the processor is run.
length
Calculates the length of a string or a JSON Path referencing a map/array/string and stores it in the target field.
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | Text, JSON Path | A string or JSON Path of the array/object/string |
targetField | yes | JSON Path | The location where the results will be stored |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"length": {
"source": "$.trap.oid",
"targetField": "$.localmem.oidlength"
}
}
The result of $.localmem.oidlength
( which is a per message local memory location ) will be the string length of the oid.
log
Outputs a log message to stdout
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | Text, JSON Path, Object, Number, Boolean | The value or JSON Path reference which needs to be logged |
args | yes | Array | A list of variadic arguments |
type | no | info, debug, error | The type of log output |
Example
{
"log": {
"source": "There are %v devices in the device catalog",
"args": [ 100 ],
"type": "info"
}
}
This example would output an info log message of "There are 100 devices in the devi catalog" to the screen.
lookup
Enables external querying against MariaDB, Neo4j, or the Assure1 API.
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | gdb, db, api, cache | The lookup source |
properties | yes | Object | The lookup properties. See the tables below |
targetField | yes | JSON Path | The location where the results will be stored |
cache | no | Object | Caching properties and storage format |
fallback | no | Object | A fallback when a cache lookup fails to find any data. You cannot use fallbacks for the cache graph module |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
cache
Name | Required | Possible Values | Notes |
---|---|---|---|
enabled | no | true, false | Whether or not to enable caching. The default is true |
object | yes | Text | The base key name in the cache. If keys are not specified, the entire result is stored in 1 record with the object name as the key |
keys | no | Array | An array of strings or JSON Paths which will be concatenated to the object string by a : . Specifying keys enables caching on a per record basis |
ttl | no | Number | Expiration time in seconds of the cache record |
fallback
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | db, api | The lookup source |
properties | yes | Object | The lookup properties. See the properties tables below. |
object | yes | Text | The base key name in the cache. If keys are not specified, the entire result is stored in 1 record with the object name as the key |
keys | no | Array | An array of strings or JSON Paths which will be concatenated to the object string by a : . Specifying keys enables caching on a per record basis. |
ttl | no | Number | Expiration time in seconds of the cache record. The default is 300 seconds. |
DBLookupProperties ( db )
Name | Required | Possible Values | Notes |
---|---|---|---|
database | no | Text | Database name. By default, this is Assure1 |
hostname | no | Text | The server name. By default, it uses the Assure1 DB fqdn if the database name is also empty. If the database name is Event, then it uses the Events database fqdn |
query | yes | Text | The SQL Query |
variables | no | Object | A key-value binding map where the key is a parameter in the query field and the value is what to replace it with. |
Example 1
{
"lookup": {
"source": "db",
"properties": {
"database": "Event",
"variables": {
"v1": 10
},
"cache": {
"enabled": true,
"object": "Event",
"keys": [ "$.EventKey" ],
"ttl": 1000
},
"query": "SELECT * From Events LIMIT :v1"
},
"targetField": "$.localmem.dbresults"
}
}
Returns 10 events from the Event database and stores it in the local memory location $.localmem.dbresults
. It also populates the redis cache with 10 json records, each having a key name of Event:{Eventkey}
with a 1000 second expiration.
GDBLookupProperties ( gdb )
Name | Required | Possible Values | Notes |
---|---|---|---|
database | no | Text | By default, the graph database is used |
hostname | no | Text | The server name. By default, it uses the primary graph database fqdn |
query | yes | Text | The Neo4j Cypher Query |
variables | no | Object | A key-value binding map where the key is a parameter in the query field and the value is what to replace it with. |
Example 2
{
"lookup": {
"source": "gdb",
"properties": {
"database": "graph",
"variables": {
"v1": 1
},
"query": "MATCH(v) WHERE v.ZoneID > $v1 RETURN v"
},
"cache": {
"enabled": true,
"object": "Vertex",
"keys": [ "$.v.Props._id" ]
},
"targetField": "$.localmem.gdbresults"
}
}
Returns all the vertices which have a zone id that is greater than 1 and stores it in the local memory location $.localmem.gdbresults
. It stores each record within the result in the cache with a key name of Vertex:{vertex uuid}
with a 300 second expiration.
APILookupProperties ( api )
Name | Required | Possible Values | Notes |
---|---|---|---|
endpoint | yes | Text | The URL path |
hostname | no | Text | The URL hostname. By default, it uses the primary presentation server fqdn |
scheme | no | http or https | By default, this is set to https |
query | yes | Text | The URL GET arguments |
variables | no | Object | A key-value binding map where the key is a parameter in the query field and the value is what to replace it with. |
Example 3
{
"lookup": {
"source": "api",
"properties": {
"variables": {
},
"endpoint": "/api/device/devices?page=1&start=0"
},
"cache": {
"enabled": true,
"object": "Device",
"keys": [ "$.DeviceID" ]
},
"targetField": "$.localmem.apiresults"
}
}
Returns all devices in the first page of the device catalog and stores it in the local memory location $.localmem.apiresults
. It stores each record within the result within the cache with a key name of 'Device:{DeviceID}' and a 300 second expiration.
CacheLookupProperties ( cache )
Name | Required | Possible Values | Notes |
---|---|---|---|
key | yes | Text | The key of the value you're attempting to access |
hostname | no | Text | The redis server fqdn |
module | no | default, graph, json, ftsearch | The type of lookup within the cache. Documentation for each module can be found under their respective name at https://redis.io/docs/stack/ |
limit | yes | Number | The number of records to return |
query | yes | Text | The query or search args |
variables | no | Object | A key-value binding map where the key is a parameter in the query field and the value is what to replace it with. |
Example 4
{
"lookup": {
"source": "cache",
"properties": {
"module": "graph",
"key": "graph",
"query": "MATCH(n) RETURN n"
},
"targetField": "$.localmem.cacheresults"
}
}
Uses a cypher query to search the topology in the redis 'graph' key and returns the results into the local memory location $.localmem.cacheresults
. Please visit https://redis.io/commands/graph.query/ for an explanation on the graph module query syntax.
Example 5
{
"lookup": {
"source": "cache",
"properties": {
"module": "ftsearch",
"key": "jsonidx:device",
"query": "@DeviceID:[129, 129]"
},
"fallback": {
"source": "db",
"properties": {
"query": "SELECT *, INET_NTOA(IPAddress) AS IPAddress, INET6_NTOA(IPv6Address) AS IPv6Address, ST_AsGeoJson(GeoLocation) AS GeoLocation FROM Devices WHERE DeviceID = 129"
},
"keys": [
"$.DeviceID"
],
"object": "Device",
"ttl": 10000
}
"targetField": "$.localmem.cacheresults"
}
}
Does a fulltext search for any records in the device json repository which have a DeviceID of 129 and stores the results into the local memory location $.localmem.cacheresults
. If a device isn't found in the cache, it falls back to a direct db lookup which returns a result if it exists and populates it in the cache as Device:129 with a 10000 second expiration. Please visit https://redis.io/docs/stack/search/reference/query_syntax/ for an explanation on the fulltext search module query syntax.
math
Calculates the length of a string or a JSON Path referencing a map/array/string and stores it in the target field.
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | JSON Path, Number | A number or JSON Path reference to a number |
operation | yes | +, -, / , * | Operation between the source and value fields |
value | yes | JSON Path, Number | A number or JSON Path reference to a number |
targetField | yes | JSON Path | The location where the results will be stored |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"math": {
"source": "$.event.Count",
"value": 2,
"operation": "*",
"targetField": "$.localmem.CountTimesTwo"
}
}
The result of $.localmem.CountTimesTwo
( which is a per message local memory location ) will be the value of the event.Count times 2.
regex
Executes a regex pattern on a source value and stores the results and whether or not there was a match in the target field.
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | JSON Path, Text | A string or a JSON Path reference to a string |
pattern | yes | Text | Regex pattern which will be executed on the source. Lookarounds are not supported |
targetField | yes | JSON Path | The location where the results will be stored |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"regex": {
"source": "Hello",
"pattern": ".",
"targetField": "$.event.RegexLocation",
}
}
The value of $.event.RegexLocation
in this example will be:
"RegexLocation": {
"matched": true,
"results": [ "H", "e", "l", "l", "o" ]
}
remove
Deletes the source key and value referenced by the JSON Path
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | JSON Path | The location of what needs to be deleted |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"remove": {
"source": "$.trap.timeTicks"
}
}
rename
Renames the source field into the target field
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | JSON Path | The JSON Path which needs to be renamed |
targetField | yes | JSON Path | The new JSON Path which the source will be renamed to |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"rename": {
"source": "$.event.Details",
"targetField": "$.event.DetailsOld"
}
}
replace
Replaces all instances of the specified pattern in the source field and then stores the new result in the targetField. If a regex pattern is needed, it must be explicitly enabled by setting the regex field to true. If the source is an array, then it matches and replaces against each value in the array.
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | JSON Path, Array, Text | A string, array, or JSON Path reference |
pattern | yes | Text | The pattern to match against |
regex | no | true or false | Whether to enable regex based pattern matching. By default, this is false. |
replacement | no | Text | The string to replace the matches with. |
targetField | yes | JSON Path | The location of the new result |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"replace": {
"source": "This is a test",
"pattern": "a test",
"replacement": "not a test",
"targetField": "$.localmem.example"
}
}
The value of $.localmem.example
after this example runs would be This is not a test
set
Takes the source value, applies the variadic args ( if any are specified ), and stores the result in a target field.
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | JSON Path, Array, Text, Boolean, Number | The value which will be set in the target field |
args | no | Array | A list of variadic arguments |
targetField | yes | JSON Path | The location of the new result |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"set": {
"source": "$.event.%s",
"args": [ "Details" ],
"targetField": "$.event.Details2"
}
}
The value of $.event.Details2
after this example runs would be identical to the value of $.event.Details
.
setOutputStream
Overrides the output stream the message is sent to at the end of processing. This is on a per-message basis, and the original output stream is reset at the beginning of execution process.
Name | Required | Possible Values | Notes |
---|---|---|---|
output | yes | pulsar://{hostname}/{namespace}/{topic} | The URI of the output stream. Only in-cluster pulsar:// is supported at the moment. |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"setOutputStream": {
"output": "pulsar+ssl:///assure1/event/sink"
}
}
sort
Sorts a source array and stores the sorted result in a target field.
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | JSON Path | A JSON Path referencing a map or array |
targetField | yes | JSON Path | The location where the sorted array will be stored |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"sort": {
"source": "$.trap.variables.0",
"targetField": "$.trap.sortedVariables"
}
}
split
Splits a string by a delimiter and stores the result as a string array in the target field
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | JSON Path, Text | A string or JSON Path referencing a string |
delimiter | yes | Text | Delimiter to split the string by |
targetField | yes | JSON Path | The location where the array of split strings will be stored |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"split": {
"source": "1,2,3,4",
"delimiter": ",",
"targetField": "$.localmem.splitarr"
}
}
The value of $.localmem.splitarr
will be:
{
"splitarr": ["1", "2", "3", "4"]
}
strcase
Applies a case type on the value of source and stores it in the target field
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | JSON Path, Text | A string or JSON Path referencing a string |
type | no | upper, lower, ucfirst, lcfirst | By default strcase uses an upper type |
targetField | yes | JSON Path | The location where the new string will be stored |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"strcase": {
"source": "HELLO, WORLD",
"type": "lower"
"targetField": "$.localmem.lowercase"
}
}
The value of $.localmem.lowercase
after this example runs will be hello, world
substr
Stores a substring of a source string in a target field.
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | JSON Path, Text | A string or JSON Path referencing a string |
start | no | Number | The start index of the substring |
end | no | Number | The end index of the substring |
targetField | yes | JSON Path | The location where the new string will be stored |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"substr": {
"source": "Hello",
"start": 1,
"targetField": "$.localmem.substr"
}
}
The value of $.localmem.substr
after this example runs will be ello
switch
Take a source value and operator and checks the condition against the value of each case.
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | JSON Path, Text , Number | A string/number or JSON Path referencing a string/number |
operator | yes | ==, !=, >, <, >=, <= , =~ | The comparison operator |
case | no | Array of SwitchCases | An array of cases which the source and operator are evaluated against sequentially |
default | no | Array of Processors | The array of processors which are executed when there aren't any matching cases |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
SwitchCase
Name | Required | Possible Values | Notes |
---|---|---|---|
Match | yes | JSON Path, Text , Number | A string/number or JSON Path referencing a string/number |
operator | yes | ==, !=, >, <, >=, <= , =~ | The comparison operator. By default, the operator specified at the root is used but it can be overridden on a per case basis |
Then | no | Array of Processors | An array of processors to execute if the case is a match |
Example:
{
"switch": {
"source": "$.localmem.val1",
"operator": "!=",
"case": [
{
"match": 2,
"then": [
{
"discard": {
}
}
]
},
{
"match": 5,
"operator": "==",
"then": [
{
"discard": {
}
}
]
}
],
"default": [
{
"log": {
"type": "info",
"source": "Do nothing since none of the cases were met"
}
}
]
}
}
trim
Trims a cutset from the beginning and end of a string
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | JSON Path, Text | A string or JSON Path referencing a string |
cutset | no | Text | A string of values to trim from the beginning/ending of the source string |
targetField | yes | JSON Path | The location where the new string will be stored |
onFailure | no | Array of Processors | An array of processors to execute if an error occurs |
ignoreFailure | no | true or false | Ignore any processor runtime errors and continue executing successive processors. The default is false |
Example
{
"trim": {
"source": "Hello",
"cutset": "H",
"targetField": "$.localmem.trim"
}
}
The value of $.localmem.trim
after this example runs will be ello
FCOM Lookups
The FCOM Processor has the ability to read a lookup file and store it in memory for reference within an override. This is very useful if a set of constant values need to be used.
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | Text | A unique lookup file name |
_type | yes | lookup | The type of file |
lookup | yes | Object | JSON Object definition containing the key-values |
Example
{
"name": "alertTypeMap",
"_type": "lookup",
"lookup": {
"1": "Fault",
"2": "Outage",
"3": "Overload",
"4": "Reboot",
"10": "Failover",
"12": "Restore"
}
}
Usage in an override
{
"set": {
"source": "$.lookups.alertTypeMap.%s",
"args": [ $.trap.variables.0.value ],
"targetField": "$.trap.varMapping"
}
}
If the value of $.trap.variables.0.value
was "10" then the value of $.trap.varMapping
would equal Failover
FCOM Grok Definitions
The FCOM Processor has the ability to load a set of custom grok patterns for use in overrides. There is a list of vendor defined commonly used patterns in the vendor _grok folder within the FCOM_FILES_LOCATION path.
Name | Required | Possible Values | Notes |
---|---|---|---|
source | yes | Text | A unique grok file name |
_type | yes | grok | The type of file |
grok | yes | Object | JSON Object definition containing the pattern names and their regex |
Example
{
"name": "Custom Grok Definition 1",
"_type": "grok",
"grok": {
"VALUE": ".*",
"COMMONMAC": "(?:(?:[A-Fa-f0-9]{2}:){5}[A-Fa-f0-9]{2})",
"DATA": ".*?"
}
}
Using custom grok definitions in a FCOM override
{
"grok": {
"source": "$.syslog.datagram",
"pattern": "%CISCO-LINK-5: The error message is: %{VALUE:message}",
"targetField": "$.syslog.variables"
}
}
The contents of $.syslog.variables
would be an object containing a message field with a value of whatever string is after The error message is: