The crawler should return an indexing file. If using DocFetch, INDEX is used during crawl (index) time. If not using DocFetch, INDEX is used during both crawl and click-through.
DocumentFormat Class | Plumtree.Remote.Crawler Namespace