About deleted records

Any client of the Record Store (including Integrator ETL, the IAS Server, the Web Crawler, and the IAS API) can modify and delete records that are written to the Record Store. Clients either update or insert a record, delete a record, or delete all records. This topic describes the Endeca.Action property that the Record Store examines to determine whether to update or insert or delete records.

Deleting all records for a full crawl

A record that has only the Endeca.Action property set to DELETE (i.e., has no other properties) functions as a logical “Delete All” record. When the Record Store encounters such a record, the Record Store removes all records from a Record Store instance. This is useful when running a full crawl and you want to remove a generation of records before writing a new generation.

For example: 
<RECORDS>
  <RECORD>
    <PROP NAME="Endeca.Action">
      <PVAL>DELETE</PVAL>
    </PROP>
  </RECORD>
  ...
</RECORDS>

Deleting records for an incremental crawl

If a record has an Endeca.Action property set to DELETE, the record is removed from a Record Store instance. This property setting is useful in an incremental crawl where files may have been modified or deleted since the last crawl.

If an incremental crawl does not find a file that is listed in the crawl history, the IAS Server treats that file as deleted. For each deleted file, a record is created that contains the location of the deleted file and an Endeca.Action property with a value of DELETE.

For renamed files, the file with the old name is treated as a deleted file while the file with the new name is treated as a new (added) file.

This example shows the record for a TestPlan.doc file that was deleted: 
<RECORDS>
  <RECORD>
    <PROP NAME="Endeca.Action">
      <PVAL>DELETE</PVAL>
    </PROP>
    <PROP NAME="Endeca.FileSystem.Path">
      <PVAL>c:\endeca_test_docs\TestPlan.doc</PVAL>
    </PROP>
    <PROP NAME="Endeca.SourceType">
      <PVAL>FILESYSTEM</PVAL>
    </PROP>
    <PROP NAME="Endeca.SourceId">
      <PVAL>FileSystemSource</PVAL>
    </PROP>
  </RECORD>
  ...
</RECORDS>

Reading records marked with the DELETE property value

Any client of the Record Store, for example a Record Store Reader in an Integrator graph, can read from a Record Store instance and process records that are marked with the Endeca.Action property set to DELETE.