5.2.2.1 Creating a Derived Dataset

You create a derived dataset to combine or modify the contents of other datasets. A derived dataset takes one or more source datasets as input and applies a series of transformations to create a set of data records for use in behavior detection. Transformations (or transforms) describe the combination, alteration, and modification of inputs to produce the resulting dataset. Some transformations restrict or filter the existing elements in the source sets, others construct new elements by adding or modifying element attributes.

Before using the transforms, you must create at least one dataset. You build the new dataset by adding transforms to the grid, linking the transforms, and defining transform properties or variables. Only the Join transform requires establishment of more than one source dataset. Therefore, you can use all transforms within a single dataset. You can also use some transforms, such as the Filter Elements transform, more than once.

Note:

If a dataset used in augmentation contain thresholds, views used in augmentation then use the values of the thresholds from the scenario’s base threshold set. The values of the thresholds in the base set may be different from the values used in the detection job.

The following sections provide the basic instructions on creating a derived dataset:

• To Add a DataSet Source
• Joining DataSets
• Filtering a DataSet
• Creating a Derived Attribute Within a DataSet
• Saving a DataSet
• Modifying Dataset Properties
• Modifying Attribute Properties

The above procedures assume that you are using all transforms, as these sections present instructions for using each transform separately.

Note:

The Dataset Editor displays an exception on the console when a thread is terminated during processing.