4 Work with Patterns

Patterns are a stage within a pipeline. When working from a pattern, you need to specify a few key fields to discover an interesting result. You can create pattern stages within the pipeline. Patterns are not stand-alone artifacts, they need to be embedded within a pipeline.

Topics:

• About Stream Analytics Patterns

• Create a Pipeline for a Pattern

About Stream Analytics Patterns

The visual representation of the event stream varies from one pattern type to another based on the key fields you choose.

Click Patterns on the Home page to see all the available patterns. Use the filters at left to view different categories of pattern. You can see full descriptions and learn more about each pattern by clicking the user assistant icon. Click again to hide the extra information.

A pattern provides you with the results displayed in a live output stream based on common business scenarios.

Note:

While entering data in the fields for a specific pattern, ensure that the data you enter corresponds to the datatype of the field. If there is a mismatch between the entered data and the datatype, the pattern will not deploy and throw an error.

You can include or exclude patterns based on their categories using the View All link in the left panel under Show Me. When you click View All, a check mark appears next to it and all the patterns are displayed on the page.

When you want to display/view only a few/selective patterns, deselect View All and select the individual patterns. Only the selected patterns are shown in the catalog.

Description of the illustration patterns_showtime.png

The following table lists the categories of patterns:

Category	Pattern
Enrichment	Reverse Geo Code: Near By Left Outer Join
Outlier	Fluctuation
Inclusion	Union Left Outer Join
Missing Event	'A' Not Followed by 'B' Detect Missing Event
Spatial	Proximity: Stream with Geo Fence Geo Fence Spatial: Speed Interaction: Single Stream Reverse Geo Code: Near By Geo Code Spatial: Point to Polygon Interaction: Two Stream Proximity: Two Stream Direction Reverse Geo Code: Near By Place Proximity: Single Stream Geo Filter
Filter	Eliminate Duplicates Fluctuation
State	'A' Not Followed by 'B' Inverse W Detect Missing Event W 'A' Followed by 'B'
Finance	Inverse W W
Trend	'A' Not Followed by 'B Top N Change Detector Up Trend Detect Missing Event Down Trend 'A' Followed by 'B' Detect Duplicates Bottom N
Shape Detector	Inverse W W
Statistical	Correlation Quantile

About the Spatial: Speed Pattern

Use this pattern to get the output average speed over the selected window range of a moving object.

For example, to analyze the average speed of a car.

Make sure that you have set the proxy details in System Settings.

Description of the illustration create_spatial_speed_pattern.png

Provide suitable values for the following parameters:

• Latitude: the latitude of the location. Select a suitable value.

• Longitude: the longitude of the location. Select a suitable value.

• Object Key: select a suitable value for the object key.

• Coordinate System: the default value is 8307 and this is the only value supported.

• Window Range: a time range over which the speed is being calculated for an event. For example, if window range=5 seconds and object key is phone no., then all the events with same phone no. received over last 5 seconds are used to calculate the average speed of that event.

The outgoing shape contains speed as an added field along with the incoming fields. This is a numeric field, but the speed is measured in meters per second.

About the Geo Code Pattern

When analyzing data, you may encounter situations where you need to obtain the geographical code of a moving object based.

Use this pattern to get geographic coordinates (like latitude and longitude) for an address.

Make sure that you have set the proxy details in System Settings.

Description of the illustration geo_code.png

Provide suitable values for the following parameters:

• Name — select a suitable value that can be uses as the place name. This is a mandatory parameter.

• Street — select a suitable value that can be used as the street name. This is a mandatory parameter.

• City — select a suitable value that can be used as the city name. This is a mandatory parameter.

• Region — select a suitable value that can be used for the region. This is a mandatory parameter.

• Country — select a suitable value for the country. This is a mandatory parameter.

• Postal Code — select a suitable value for the postal code. This is a mandatory parameter.

The outgoing shape contains latitude and longitude as additional fields along with incoming fields.

About the Interaction: Single Stream Pattern

Use this pattern to get interaction of an object with every other object in a stream.

For example, you can see if a set of sailing ships are too close to each other.

Description of the illustration interaction_single_stream_pattern.png

Provide suitable values for the following parameters:

• Geometry: the field of type SDO_GEOMETRY data type. Only field of SDO_GEOMETRY data type should be chosen for such fields.

• Object Key: field used to uniquely identify object on the geo fence and is used for partitioning of data where supported.

• Coordinate System: the default value is 8307 and this is the only value supported.

The outgoing shape contains two more fields along with the incoming shape: isInteract and distance. isInteract is trueif two shapes interact with each other, i.e., any or some portion of the two objects overlap. distance between them is 0, if no overlapping is observed; isInteract is false and distance is shown between those two objects as a positive number.

About the Interaction: Two Stream Pattern

Two shapes are said to interact with each other if any part of the shape overlaps. If two shapes interact, the distance between them is zero.

Use this pattern to get interaction of an object in one stream with objects in another stream.

Description of the illustration interaction_two_stream.png

Provide suitable values for the following parameters:

• Geometry: select a suitable value for geometry. This is a mandatory value.

• Object Key: select a suitable value for the object key. This is a mandatory value.

• Event Stream 2: select the second event stream. This is a mandatory value.

• Geometry: select a suitable value for geometry within the second stream. This is a mandatory value.

• Object Key: select a suitable value for the object key within the second stream. This is a mandatory value.

• Coordinate System: the default value is 8307 and this is the only value supported.

The outgoing shape contains two additional fields along with the incoming shape: isInteract and distance. isInteract is trueif two shapes interact with each other, i.e., any or some portion of the two objects overlap. distance between them is 0, if no overlapping is observed; isInteract is false and distance is shown between those two objects as a positive number.

About the Spatial: Point to Polygon Pattern

Use this pattern to get an object shape based on geographical coordinates, fixed length and breadth of and object.

For example, if you know the length and breadth of a group of a fleet of ships, you can get the shape of a ship using the position coordinates, where the coordinates keep changing as the ship moves.

Description of the illustration point_to_polygon.png

Provide suitable values for the following parameters:

• Latitude: select a suitable value for the latitude. This is a mandatory parameter.

• Longitude: select a suitable value for the longitude. This is a mandatory parameter.

• Object Key: select a suitable value for the object key. This is a mandatory parameter.

• Length: select a suitable field to be used as the reference length of the polygon.

• Width: select a suitable field to be used as the reference width of the polygon.

• Coordinate System: the default value is 8307 and this is the only value supported.

• Buffer: enter a positive value to be used as the geometry buffer.

The outgoing shape contains Updatedshape as an additional attribute which is the updated point into geometry of type SDO_GEOMETRY along with the incoming shape.

About the Proximity: Single Stream Pattern

Use this pattern to get proximity of each object with every other object in a stream.

For example, if there is stream of flying airplanes and the distance buffer is 1000 meters. You can raise an alert as the two planes come into a proximity of 1000 meters or less.

Description of the illustration proximity_single_stream.png

Provide suitable values for the following parameters:

• Latitude: select a suitable value for the latitude. This is a mandatory parameter.

• Longitude: select a suitable value for the longitude. This is a mandatory parameter.

• Object Key: select a suitable value for the object key. This is a mandatory parameter.

• Coordinate System: the default value is 8307 and this is the only value supported.

• Distance Buffer: enter a proximity value for the distance buffer. This is the distance that two points can be apart and still be considered the same. Select an appropriate unit for the distance.

The outgoing shape displays distance as another column, which is the distance between two object under consideration along with the incoming shape.

About the Proximity: Two Stream Pattern

Use use this pattern to get the proximity between objects of two streams.

The distance buffer acts as a filter in this pattern stage. For example, if there is a driver and passenger stream, you can get the proximity of each passenger with every other driver using a filter criteria of ‘within a distance of 1 km’.

Description of the illustration proximity_two_streams.png

Provide suitable values for the following parameters:

• Latitude: select a suitable value for the latitude. This is a mandatory parameter.

• Longitude: select a suitable value for the longitude. This is a mandatory parameter.

• Object Key: select a suitable value for the object key. This is a mandatory value.

• Event Stream 2: select the second event stream. This is a mandatory value.

• Latitude: select a suitable value for the latitude in the second stream. This is a mandatory parameter.

• Longitude: select a suitable value for the longitude in the second stream. This is a mandatory parameter.

• Object Key: select a suitable value for the object key in the second stream. This is a mandatory parameter.

• Coordinate System: the default value is 8307 and this is the only value supported.

• Distance Buffer: enter a proximity value for the distance buffer. This field acts as a filter criteria of two objects and the objects that do not fall in this distance (distance between them is more than chosen distance buffer) are filtered from result set.

Note:

When a pipeline with this pattern has a database reference with cache enabled, the pattern does not display any output in the live output stream.

The outgoing shape displays distance as another column, which is the distance between two object under consideration along with the incoming shape.

About the Proximity: Stream with Geo Fence Pattern

Use this pattern to get proximity of an object with a virtual boundary or geo fence.

For example, if you have certain stores in the city of California, you can send promotional messages as soon as the customer comes into a proximity of 1000 meters from any of the stores.

Description of the illustration proximity_stream_geofence.png

Provide suitable values for the following parameters:

• Geo Fence: select a geo fence that you like to analyze.

• Latitude: select a suitable value for the latitude. This is a mandatory parameter.

• Longitude: select a suitable value for the longitude. This is a mandatory parameter.

• Object Key: select a suitable value for the object key. This is a mandatory parameter.

• Coordinate System: the default value is 8307 and this is the only value supported.

• Distance Buffer: enter a proximity value for the distance buffer. This field acts as a filter criteria for events and the events that do not fall in this distance (distance between them is more than chosen distance buffer) are filtered from result set.

The outgoing shape displays distance as another column, which is the distance between the object and geo fence under consideration along with the incoming shape.

About the Direction Pattern

Use this pattern to get the direction of a moving object.

For example, you can evaluate the direction of a moving truck.

Description of the illustration direction.png

Provide suitable values for the following parameters:

• Latitude: select a suitable value for the latitude. This is a mandatory parameter.

• Longitude: select a suitable value for the longitude. This is a mandatory parameter.

• Object Key: select a suitable value for the object key. This is a mandatory parameter.

• Coordinate System: the default value is 8307 and this is the only value supported.

The outgoing shape displays direction as one of the columns, which is of type String along with the incoming shape.

About the Geo Fence Pattern

Use this pattern when you want to track object relation with a virtual boundary called goe fence.

Relations can be Enter, Exit, Stay, or Near with respect to a geo fence. For example, you can trigger an alert when an object enters the geo fence. You can also analyze a stream containing geo-location data. It helps in determining how events are related to a polygon in a geo fence.

The geo-location can be:

• Near to Geo Fence

• Exiting Geo Fence

• Staying within Geo Fence for a specified duration

• Entering Geo Fence

Description of the illustration geo_fence_pattern.png

Provide suitable values for the following parameters:

• Geo Fence

• Latitude

• Longitude

• Object Key

• Tracking Events

  • Near

  • Enter

  • Exit

  • Stay

• Coordinate system

• Distance Buffer — this parameter is enabled only if you select Near option in Tracking Events. This field acts as a buffer for filtering results. Only those events or objects which are within the specified distance from the geo fence are displayed in events table with status as Near.

• Stay Duration — this parameter is enabled only if you select Stay in Tracking Events. You can specify the stay duration and this duration acts as a filter for objects inside the geo fence. If an object stays for a duration more than the specified duration, only then the events are considered, else events are filtered out.

The outgoing shape displays Status and PlaceName as two extra columns in the output along with the incoming shape, where Status is one of Enter, Exit, Stay, or Near based on how the object behaves with geo fence. PlaceName is the name of geo fence with which status is being evaluated.

About the Geo Fence Filter: Inside Pattern

Use this pattern to when you want to track objects inside a virtual boundary.

For example, if users move from one geographical location to another, you can send promotional messages to the users when they are inside a specified geo fence.

Description of the illustration geo_filter_pattern.png

Provide suitable values for the following parameters:

• Geo Fence — select one of the existing geo fences to analyze. This is a mandatory field.

• Latitude — select a suitable value for the latitude. This is a mandatory parameter.

• Longitude — select a suitable value for the longitude. This is a mandatory parameter.

• Object Key — select a suitable value for the object key. This field acts as a partitioning criteria and also used to uniquely identify objects. This is a mandatory parameter.

• Coordinate system — the default value is 8307 and this is the only value supported.

The outgoing shape displays Status and PlaceName as two extra columns in the output along with the incoming shape, where Status is Inside if the object is inside geo fence (else the event is not considered) and PlaceName is the name of geo fence with which status is being evaluated.

About the Reverse Geo Code: Near By Pattern

Use this to obtain nearest place for the specified geographical coordinates.

Make sure that you have set the proxy details in System Settings.

Description of the illustration reverse_code_near_by.png

Provide suitable values for the following parameters:

• Latitude — select a suitable value for the latitude. This is a mandatory parameter.

• Longitude — select a suitable value for the longitude. This is a mandatory parameter.

• Object Key — select a suitable value for the object key. This is a mandatory parameter.

• Coordinate system — the default value is 8307 and this is the only value supported.

The outgoing shape displays PlaceName as an additional column along with the incoming shape. This column is the nearest place for specified longitude and latitude.

About the Reverse Geo Code: Near By Place Pattern

Use this pattern to obtain the near by location with granular information like city, country, street etc. for the specified latitude and longitude.

Make sure that you have set the proxy details in System Settings.

Description of the illustration reverse_code_near_by_place.png

Provide suitable values for the following parameters:

• Latitude — select a suitable value for the latitude. This is a mandatory parameter.

• Longitude — select a suitable value for the longitude. This is a mandatory parameter.

• Object Key — select a suitable value for the object key. This is a mandatory parameter.

• Coordinate system — the default value is 8307 and this is the only value supported.

The outgoing shape displays PlaceName as an additional column along with the incoming shape. This column is the nearest place for specified longitude and latitude.

About the Correlation Pattern

Use this pattern if you need to identify correlation between two numeric parameters.

Description of the illustration correlation_pattern.png

Provide suitable values for the following parameters:

• Partition Criteria: the field by which you want to partition.

• Observable Parameter 1: the first value used to identify the correlation.

• Observable Parameter 2: the second value used to identify the correlation.

• Window: a rolling time period, the duration within which the correlation is identified.

• Slide: the frequency at which you want to refresh the data.

The outgoing shape is same as the incoming shape.

About the Quantile Pattern

Use this pattern if you need to calculate the value of quantile function.

Description of the illustration quantile_pattern.png

Provide suitable values for the following parameters:

• Partition Criteria: the field based on which you want to partition.

• Observable Parameter: the observable to calculate the quantile.

• Phi-quantile: the value is used to calculate the quantile of the selected event stream.

• Window: a rolling time period, within which the events will be collected and quantile is calculated.

• Slide: how frequent newly updated output will be pushed downstream and into the browser.

The outgoing shape is the same as the incoming shape.

About the Standard Deviation Pattern

Use this pattern to calculate the standard deviation of the selected values with the expected values.

Description of the illustration std_deviation_pattern.png

Provide suitable values for the following parameters:

• Partition Criteria: the field based on which you want to partition.

• Observable Parameter : the value used to identify the standard deviation.

• Window: a rolling time period, the duration within which the standard deviation is identified.

• Slide: the frequency at which you want to refresh the data.

The outgoing shape is the same as the incoming shape.

About the Median Pattern

Use this pattern to calculate the median of an event stream with respect to a specific parameter.

Description of the illustration median_pattern.png

Provide suitable values for the following parameters:

• Partition Criteria: the field based on which you want to partition.

• Observable Parameter: the observable to calculate the median.

• Window: a rolling time period, within which the events will be collected and median is calculated.

• Slide: how frequent newly updated output will be pushed downstream and into the browser.

The outgoing shape is same as the incoming shape.

About the Detect Duplicates Pattern

The Detect Duplicates pattern detects duplicate events in your stream according to the criteria you specify and within a specified time window. Events may be partially or fully equivalent to be considered duplicates.

Use this pattern to understand how many duplicate events your stream has. For example, when you suspect that your aggregates are offset, you may want to check your stream for duplicate events.

Description of the illustration detect_duplicates.png

Provide suitable values for the following parameters:

• Duplicate Criteria: a list of fields, whose values will be compared between events to look for identical values. If all the configured fields have identical values, the incoming event will be considered a duplicate and an outgoing event will be fired.

• Window: a time period, within which the duplicates will be searched for. For example, if you set the window to 10 seconds, a duplicate event that arrives 9 seconds after the first one will be trigger an outgoing event, while a duplicate event that arrives 11 seconds after the first one will not do so.

Outgoing Shape

The outgoing shape is the same as the incoming shape with one extra field: Number_of_Duplicates. This extra field will carry the number of duplicate events that have been discovered. All the other fields will have values of the last duplicate event.

About the Change Detector Pattern

The Change Detector pattern looks for changes in the values of your event fields and report the changes once they occur within a specified range window. For example, and events arrives with value value1 for field field1. If any of the following incoming events within a specified range window contains a value different from value1, an alert is triggered. You can designate more than one field to look for changes.

Use it when you need to be aware of changes in a normally stable value. For example, a sensor reading that is supposed to be the same for certain periods of time and changes in readings may indicate issues.

The default configuration of this pattern stage is to alert on change of any selected fields.

Description of the illustration change_detector_pattern.png

Provide suitable values for the following parameters:

• Partition Criteria: a field to partition your stream by. For example, your stream contains events issues by a number of sensors. All sensors send the same but individual data. You would want to compare readings of a sensor to previous readings of the same sensor and not just a previous event in your stream, which is very likely to be from a different sensor. Select a field that would uniquely identify your sensors, such as sensor Id. This field is optional. For example, if your stream contains readings from just one sensor, you do not need to partition your data.

• Window range: a time period, within which the values of designated fields are compared for changes. For example, if you set the window range to 10 seconds, an event with changes in observed fields will trigger an alert if it arrives within 10 seconds after the initial event. The clock starts at the initial event.

• Change Criteria: a list of fields, whose values will be compared between events to look for changes. If the fields contain no changes, no alerts will be generated.

• Alert on group changes: this parameter is responsible for the default group changes support. If it is OFF, then alert on at least one field changes. If it is ON, then sends alert on every field change.

Outgoing Shape

The outgoing shape is based on the incoming shape, the difference being that all the fields except the one in the partition criteria parameter will be duplicated to carry both the initial event values and the change event values. Let's look at an example. Your incoming event contains the following fields:

• sensor_id

• temperature

• pressure

• location

Normally, you would use sensor_id to partition your data and say you want to look for changes in temperature. So, select sensor_id in the partition criteria parameter and temperature in the change criteria parameter. Use a range window that fits your use case. In this scenario, you will have the following outgoing shape:

• sensor_id

• temperature

• orig_temperature

• pressure

• orig_pressure

• location

• orig_location

The orig_ fields carry values from the initial event. In this scenario, temperature and orig_temperature values are different, while pressure and orig_pressure, location, and orig_location may have identical values.

About the W Pattern

The W pattern, also known as a double bottom chart pattern, is used in the technical analysis of financial trading markets.

Use this pattern to detect when an event data field value rises and falls in “W” fashion over a specified time window. For example, use this pattern when monitoring a market data feed stock price movement to determine a buy/sell/hold evaluation.

Description of the illustration w_pattern.png

Provide suitable values for the following parameters:

• Partition Criteria: a field to partition your stream by. For example, a ticker symbol.

• Window: a time period, within which the values of the designated field are analyzed for the W shape.

• Tracking value: a field, whose values are analyzed for the W shape.

Outgoing Shape

The outgoing shape is based on the incoming shape with an addition of five new fields. The new fields are:

• firstW

• firstValleyW

• headW

• secondValleyW

• lastW

The new fields correspond to the tracking value terminal points of the W shape discovered in the feed. The original fields correspond to the last event in the W pattern.

Rule

A rule is a set of conditions applied to the incoming stream and a set of actions performed on the stream when conditions are true. Each event is analyzed independently of other events.

For example, assume that your stream is a stream from pressure sensors and has the following fields:

• sensor_id

• pressure

• status

If you want to assign a status value based on the pressure, you can define the following rules:

• if the pressure is less than or equal to 50, the status must be set to GREEN

• if the pressure is between 50 and 100, the status must be set to YELLOW

• if the pressure is greater than 100, the status must be set to RED.

To achieve this, you need to create these rules in a rule stage. The YELLOW rule for example, looks as shown below:

Description of the illustration yellow_rule.png

The rules get applied to the events sequentially and actions are triggered if the conditions are met. If you look at the data in the previous screen, the pressure value is 120 in the last row and hence the RED rule conditions resolve to true.

You must be careful while defining the rules. Logical loops or contradictory rules lead to the application never returning any outgoing events. For example, the following rules force the application into running forever without any outgoing events:

• Rule 1: if n > 0, set n to -1

• Rule 2: if n <=0, set n to 1

About the ‘A’ Followed by ‘B’ Pattern

The 'A' Followed by 'B' pattern looks for particular events following one another and will output an event when the specified sequence of events occurs.

Use it when you need to be aware of a certain succession of events happening in you flow. For example, if an order status BOOKED is followed by an order status SHIPPED (skipping status PAID), you need to raise an alert.

Description of the illustration a_followed_by_b_pattern.png

Provide suitable values for the following parameters:

• Partition Criteria: (Optional) a field to partition your stream by. In the order example above, it may be order_id.

• State A: field: an initial state field, whose value will be used in the comparison of two events. In our example, it will be order_status.

• State A: value: the initial field state value. In our example, BOOKED.

• State B: field: a consecutive state field, whose value will be used in the comparison of two events. In our example, it will be order_status again.

• State B: value: the consecutive field state value. In our example, SHIPPED.

• Duration: the time period, within which to look for state changes.

Outgoing Shape

The outgoing shape is based on the incoming shape. A new abInterval field is added to carry the value of the time interval between the states in nanosecond. Also, all but the partition criteria fields are duplicated to carry values from both a and b states. For example, if you have the following incoming shape:

• order_id

• order_status

• order_revenue

You will get the following outgoing shape:

• order_id

• abInterval

• order_status (this is the value by which you partition your stream)

• aState_order_status (this is the value of order_status in state A, in our example 'BOOKED')

• order_revenue (this is the value of order_revenue in state B)

• aState_order_revenue (this is the value of order_revenue in state A)

About the Top N Pattern

The Top N pattern will output N events with highest values from a collection of events arriving within a specified time window sorted not in the default order of arrival but the way you specify.

Use it to get the highest values of fields in your stream within a specified time window. For example, use it to get N highest values of pressure sensor readings.

Description of the illustration topn_pattern.png

Provide suitable values for the following parameters:

• Window Range: a rolling time period, within which the events will be collected and ordered per your ordering criteria.

• Window Slide: how frequent newly updated output will be pushed downstream and into the browser.

• Order by Criteria: a list of fields to use to order the collection of events.

• Number of Events: a number of top value events to output.

The outgoing shape is the same as the incoming shape.

About the Bottom N Pattern

The Bottom N pattern will output N events with lowest values from a collection of events arriving within a specified time window sorted not in the default order of arrival but the way you specify.

Use it to get the lowest values of fields in your stream within a specified time window. For example, use it to get N lowest values of pressure sensor readings.

Description of the illustration bottomn_pattern.png

Provide suitable values for the following parameters:

• Window Range: a rolling time period, within which the events will be collected and ordered per your ordering criteria.

• Window Slide: how frequent newly updated output will be pushed downstream and into the browser.

• Order by Criteria: a list of fields to use to order the collection of events.

• Number of Events: a number of bottom value events to output.

The outgoing shape is the same as the incoming shape.

About the Up Trend Pattern

The Up Trend pattern detects a situation when a numeric value goes invariably up over a period of time.

Use the pattern if you need to detect situations of a constant increase in one of your numeric values. For example, detect a constant increase in pressure from one of your sensors.

Description of the illustration up_trend_pattern.png

Provide suitable values for the following parameters:

• Partition Criteria: a field by which to partition your stream. For example, your stream contains events issues by a number of sensors. All sensors send the same but individual data. You would want to compare readings of a sensor to previous readings of the same sensor and not just a previous event in your stream, which is very likely to be from a different sensor. Select a field that would uniquely identify your sensors, such as sensor id. This field is optional. For example, if your stream contains readings from just one sensor, you do not need to partition your data.

• Duration: a time period, within which the values of the designated field are analyzed for the upward trend.

• Tracking value: a field, whose values are analyzed for the upward trend.

Outgoing Shape

The outgoing shape is based on the incoming shape with an addition of two new fields. For example, if your incoming event contains the following fields:

• sensor_id

• temperature

• pressure

• location

Normally, you would use sensor_id to partition your data and say you want to look for the upward trend in temperature. So, select sensor_id in the partition criteria parameter and temperature in the tracking value parameter. Use a duration that fits your use case. In this scenario, you will have the following outgoing shape:

• sensor_id

• startValue (this is the value of temperature that starts the trend)

• endValue (this is the value of temperature that ends the trend)

• temperature (the value of the last event)

• pressure (the value of the last event)

• location (the value of the last event)

About the ‘A’ Not Followed by ‘B’ Pattern

The 'A' Not Followed by 'B' pattern will look for a missing second event in a particular combination of events and will output the first event when the expected second event does not arrive within the specified time period.

Use it when you need to be aware of a specific event not following its predecessor in your flow. For example, if an order status BOOKED is not followed by an order status PAID within a certain time period, you may need to raise an alert.

Description of the illustration a_not_followed_by_b_pattern.png

Provide suitable values for the following parameters:

• Partition Criteria: (Optional) a field to partition your stream by. In the order example above, it may be order_id.

• State A: field: an initial state field, whose value will be used in the comparison of two events. In our example, it will be order_status.

• State A: value: the initial field state value. In our example, BOOKED.

• State B: field: a consecutive state field, whose value will be used in the comparison of two events. In our example, it will be order_status again.

• State B: value: the consecutive field state value. In our example, SHIPPED.

• Duration: the time period, within which to look for state changes.

Outgoing Shape

The outgoing shape is the same as incoming shape. If the second (state B) event does not arrive within the specified time window, the first (state A) event is pushed to the output.

About the Down Trend Pattern

The Down Trend pattern detects a situation when a numeric value goes invariably down over a period of time.

Use this pattern if you need to detect situations of a constant reduction in one of your numeric values. For example, detect a constant drop in pressure from one of your sensors.

Description of the illustration down_trend_pattern.png

Provide suitable values for the following parameters:

• Partition Criteria: a field to partition your stream by. For example, your stream contains events issues by a number of sensors. All sensors send the same but individual data. You would want to compare readings of a sensor to previous readings of the same sensor and not just a previous event in your stream, which is very likely to be from a different sensor. Select a field that would uniquely identify your sensors, such as sensor id. This field is optional. For example, if your stream contains readings from just one sensor, you do not need to partition your data.

• Duration: a time period, within which the values of the designated field are analyzed for the downward trend.

• Tracking value: a field, whose values are analyzed for downward trend.

Outgoing Shape

The outgoing shape is based on the incoming shape with an addition of two new fields. Let's look at an example. Your incoming event contains the following fields:

• sensor_id

• temperature

• pressure

• location

Normally, you would use sensor_id to partition your data and say you want to look for the downward trend in temperature. So, select sensor_id in the partition criteria parameter and temperature in the tracking value parameter. Use a duration that fits your use case. In this scenario, you will have the following outgoing shape:

• sensor_id

• startValue (this is the value of temperature that starts the trend)

• endValue (this is the value of temperature that ends the trend)

• temperature (the value of the last event)

• pressure (the value of the last event)

• location (the value of the last event)

The pattern is visually represented based on the data you have entered/selected.

About the Union Pattern

The Union pattern merges two streams with identical shapes into one.

Use this pattern if you have two streams with identical shapes that you want to merge into one, for example when you have two similar sensors sending data into two different streams, and you want to process the streams simultaneously, in one pipeline.

Description of the illustration union_pattern.png

Provide suitable values for the following parameters:

• Second event stream: the stream you want to merge with your primary stream. Make sure you select a stream with an identical shape.

The outgoing shape is the same as the incoming shape.

About the Fluctuation Pattern

Use this pattern to detect when an event data field value changes in a specific upward or downward fashion within a specific time window. For example, use this pattern to identify the variable changes in an Oil Pressure value are maintained within acceptable ranges.

Description of the illustration fluctuation_pattern.png

Provide suitable values for the following parameters:

• Partition Criteria: the field based on which you want to partition.

• Tracking Value: the value is used to track the event data and create a pattern in the live output stream.

• Window: a rolling time period, the frequency at which you want to refresh the data.

• Deviation Threshold %: value indicates the percentage of deviation you want to be included in the pattern. This is the interval in which the pipeline looks for a matching pattern.

The outgoing shape is same as the incoming shape.

About the Inverse W Pattern

The Inverse W pattern, also known as a double top chart pattern, is used in the technical analysis of financial trading markets.

Use this pattern when you want to see the financial data in a graphical form.

Description of the illustration inverse_w_pattern.png

Provide suitable values for the following parameters:

• Partition Criteria: a field to partition your stream by. For example, a ticker symbol.

• Window: a time period, within which the values of the designated field are analyzed for the inverse W shape.

• Tracking value: a field, whose values are analyzed for the inverse W shape.

Outgoing Shape

The outgoing shape is based on the incoming shape with an addition of five new fields. The new fields are:

• firstW

• firstPeakW

• headInverseW

• secondpeakW

• lastW

The new fields correspond to the tracking value terminal points of the inverse W shape discovered in the feed. The original fields correspond to the last event in the inverse W pattern.

About the Eliminate Duplicates Pattern

The Eliminate Duplicates pattern looks for duplicate events in your stream within a specified time window and removes all but the first occurrence. A duplicate event is an event that has one or more field values identical to values of the same field(s) in another event. It is up to you to specify what fields are analyzed for duplicate values. You can configure the pattern to compare just one field or the whole event.

Use it to get rid of noise in your stream. If you know that your stream contains duplicates that might offset your aggregates, such as counts, use the Eliminate Duplicates pattern to cleanse your data.

Description of the illustration eliminate_duplicates.png

Provide suitable values for the following parameters:

• Duplicate Criteria: a list of fields, whose values will be compared between events to look for identical values. If all the configured fields have identical values, the second, third, and subsequent events will be dropped.

• Window: a time period, within which the duplicates will be discarded. For example, if you set the window to 10 seconds, a duplicate event that arrives 9 seconds after the first one will be discarded, while a duplicate event that arrives 11 seconds after the first one will be accepted and let through.

The outgoing shape is the same as the incoming shape.

About the Detect Missing Heartbeat Pattern

The Detect Missing Heartbeat pattern discovers simple situations when an expected event is missing.

Use this pattern if you need to detect missing events in your feed. For example, you have a feed when multiple sensors send their readings every 5 seconds. Use this pattern to detect sensors that have stopped sending their readings, which may indicate that the sensor is broken or there is no connection to the sensor.

Description of the illustration detect_missing_heartbeat_pattern.png

Provide suitable values for the following parameters:

• Partition Criteria:  a field to partition your stream by. For example, your stream contains events issues by a number of sensors. All sensors send the same but individual data. You would want to compare readings of a sensor to previous readings of the same sensor and not just a previous event in your stream, which is very likely to be from a different sensor. Select a field that would uniquely identify your sensors, such as sensor id. This field is optional. For example, if your stream contains readings from just one sensor, you do not need to partition your data.

• Heartbeat interval: a time period, within which missing events are detected. If there is no event from a sensor within the heartbeat interval after the last event, an alert is triggered.

Outgoing Shape

The outgoing shape is the same as incoming shape. If there are no missing heartbeats, no events are output. If there is a missing heartbeat, the previous event, which was used to calculate the heartbeat interval is output.

About the Left Outer Join Pattern

The Left Outer join pattern joins your flow with another stream or a reference using the left outer join semantics.

Use this pattern to join a stream or a reference using the left outer join semantics. The result of this pattern always contains the data of the left table even if the join-condition does not find any matching data in the right table.

Description of the illustration create_left_outer_join_pattern.png

Provide suitable values for the following parameters:

• Enriching Reference/Stream:  the stream or reference you want to join to your flow.

• Correlation Criteria: fields on which the stream / reference will be joined.

• Window Range of the Primary Stream: a rolling time window used to make a collection of events in your primary flow to be joined with the enriching stream / reference.

• Window Slide of the Primary Stream: how often the data will be pushed downstream and to the UI.

• Window Range of the Enriching Stream: a rolling time window used to make a collection of events in your enriching stream to be joined with the primary flow. Disabled, if a reference is used.

• Window Slide for the Enriching Stream: how often the data will be pushed downstream and to the UI. Disabled, if a reference is used.

The outgoing shape is a sum of two incoming shapes.

Create a Pipeline for a Pattern

Instead of creating a pattern stage from within a pipeline, you can also create a pipeline for a pattern directly.

To create a pipeline for a pattern:

1. Click Patterns in the left tree on the Home page.
   The Patterns page appears.
2. Scroll through the list of available patterns and select the required pattern.
3. Click Use this pattern within the selected pattern tile.

   
   Description of the illustration use_this_pattern.png

   The Create pipeline using <Pattern> dialog box appears.

   
   Description of the illustration create_app_using_pattern.png

4. Fill in the details for the metadata in the pipeline section.
5. Enter details for the Pattern Stage.
6. Click Save.
   The pipeline editor opens where you can specify the parameters required for the pattern. The pipeline also appears in the Catalog.