In simple cases, a manipulator produces correct output and can be
annotated with
"supportsIncrementals=true"
if it meets all of the
following criteria:
Does the manipulator output only the records it receives? That is, does it always output records with a
recordId
that was input to it?Does the manipulator output all the records that it receives?
Does the manipulator generate output for one record that depends only on the input of the same record and not on any other records?
Does the manipulator produce the same output over time in response to a given input record?
If the answer is yes to these four questions, then a manipulator can
be annotated with
"supportsIncrementals=true"
. If the answer is no to
any one of these questions, then further analysis is required to determine
whether correct output can be produced, and further work is required to produce
correct output. This work typically involves maintaining state.
If the answer is yes to all but the second question, it may be
possible to support incremental input with only a little extra work to address
the records that the manipulator does not output. To be correct, a manipulator
should emit a delete record for any record that it had previously output but is
now not being output. (Recall that a record marked for deletion has an
Endeca.Action
property set to
DELETE
.)
This would require the manipulator to track what records it had previously output. To make such implementation easier, a manipulator can replace input records that it should not output with a delete record instead.
For example, suppose a manipulator filters records to include only those records that contain a color property that is set to red. Record A has the property color set to red, so the manipulator includes record A. In a subsequent incremental acquisition, record A has changed so that now the color property is set to blue. Record A no longer meets the manipulator's filtering criteria. Instead of dropping the record, the manipulator emits a delete record for record A.