Overview of a differential crawling pipeline

Conceptually, a differential crawl is similar to a full crawl, with the exception that a differential crawl will only download those documents that have been modified since the previous crawl.

The differential crawling cannot be run from Developer Studio or Endeca Workbench. It can be run only via a control script. For a script example, see the Endeca Control System Guide.

Important: The Endeca Crawler is deprecated, and will be removed in a future version of the Endeca Information Access Platform. Therefore, if you are beginning a new project, it is recommended that you use the Endeca Web Crawler, which is a component of the Endeca Content Acquisition System.

This section assumes that you have read “Implementing the Endeca Crawler," and that you are familiar with creating a full crawl pipeline.