Differential spider

A differential crawl spider is configured in the same way as a full crawl spider, with the exception of the Differential Crawl URL field.

This field must be filled in, as in this example:

The URL specifies the location of the state file that contains the results of the previous pipeline run. With that one exception, the CrawlRefs spider is configured identically to the full crawl spider.