Relation Operators

Relation operators enable you to perform addition and combination, subtraction, intersection, or unique identification operations on relations.

The following table lists the input and output for each operator.

Distinct

The Distinct operator retains records input from a relation that are unique. If duplicate records exist, they will not be sent out from the Distinct operator. All attributes of records must match to be filtered with the Distinct operators.

You typically use the Distinct operator in conjunction with a relation output from another operator. The Distinct operator does not forward duplicate records from input to it. Typical operators whose input the Distinct operator uses include:

• Attribute-Based Window operator

• Partitioned Window operator

• Time Based Window operator

• Tuple-Based Window operator

• Relation Map operator

• Relation Aggregator operator

When you use the Distinct operator, any records provided must be a complete duplicate, meaning that all attributes have to match for the Distinct operator to fill out all records.

The Distinct operator works with other operators. For example, you can use the Distinct operator with a Tuple-Based Window operator to limit the number of tuples that are affected by the operator. Or you can use the Distinct operator with the Time-Based Window operator to keep records for only 30 seconds, but to filter out records if there are no duplicates.

To Create a Distinct Operator

1. Drag a Distinct operator from the Palette to the Design view.

2. Connect an input from an operator that has a relation result.

3. Double-click the Distinct operator.

   The property editor opens with the name of the Distinct operator and the output schema name populated. The property editor displays the schema of the data that is input to the Distinct operator.

4. (Optional) Click the Documentation tab and supply notes on this component.

5. Click OK.

Intersect

The Intersect operator enables definition of relation intersection. In effect, the operator behaves like the SQL INTERSECT command and acts as an AND operator (that is, values are selected only if they appear in all inputs provided to the operator).

All input schemas to the Intersect operator must be identical in format: column name and type must match for all attributes. You cannot select a subset of columns (attributes) from the input schemas, as you can with the SQL INTERSECT command.

To Create an Intersect Operator

1. Drag an Intersect operator from the Palette to the Design view.

2. Connect two input relations to the Intersect operator.

3. Double-click the Intersect operator.

   The property editor opens.

4. Examine the Details and Attributes sections.

5. (Optional) Click the Documentation tab and supply notes on this component.

6. Click OK.

Minus

The Minus operator subtracts one stream from another stream. The order of subtraction is determined by the order in which you connect the input relations. You cannot subtract unlike operator attributes. The input schemas must be identical, with identical names and types for all attributes.

Specify the operator to subtract from, then choose the expression in the Expression drop-down list, where you can change the order for the subtraction operation. As with other operators, you can specify multiple input streams to a minus operation, and provide filters before the Minus operator.

To Create a Minus Operator

1. Drag a Minus operator from the Palette to the Design view.

2. Connect two or more input relations to the Minus operator.

3. Double-click the Minus operator.

   The Expression field provides the order of the minus expression. This is initially determined by the order in which the inputs were connected. You can determine this by opening the properties pane for the Minus operator. The input ID list provides the order of the operators that the Minus operator subtracts. Next, open the properties pane for the relations feeding into the Minus operator and examine their ID.

4. To change the order in the Expression dialog box, choose the Order by drop-down list and a new primary operator. The latter becomes the first operator on the Expression field. You can connect additional inputs to the Minus operator at a later time. You can also remove inputs. When you remove inputs, the Expression is automatically updated.

Union

The Union operator combines elements from input relations. The Union operator captures one of every record in every relation and provides one of every record as output, with duplicates removed.

All input relations must have the same schema. Relations that are output from the Union operator will have the same schema.

Because the Union operator does not pass through duplicates, if two different files input to it indicate size of 1, the behavior is similar to having two single streams input to a merged file.

That is, for each event a new row is added, and one subtracted. Note that if the input files are identical (the rows are exactly the same), and the Tuple Size is also 1, then the previous output is deleted, but nothing is added.

To Create a Union Operator

1. Drag a Union operator from the Palette to the Design view.

2. Connect the input of the new Union component to two or more operators that have relation output.

3. Double-click the Union operator.

   The property editor opens. Notice that the component has inherited the schema of its input relations.

4. As needed, configure the following property: Name:

5. (Optional) Click the Documentation tab and supply notes on this component.

6. Click OK.

Union All

The Union All operator combines elements from input relations, and captures one of every record in every relation and provides one of every record as output, including duplicates.

All input relations must have the same schema. Relations that are output from the Union All operator have the same schema.

You typically use the Union All operator to join data from two separate streams, such as when your application requires that you match data from one stream against that of another, without excluding duplicate records from the output.

To Create a Union All Operator

1. Drag a Union All operator from the Palette to the Design view.

2. Connect the input of the new Union All component to two or more operators that have relation output.

3. Double-click the Union All operator.

   The property editor opens. Notice that the component has inherited the schema of its input relations.

4. As needed, configure the following property: Name:

5. (Optional) Click the Documentation tab and supply notes on this component.

6. Click OK.