The default.xml file

The default.xml file is the main configuration file for the Endeca Web Crawler.

The default.xml configuration file contains properties for all the crawls. These properties should have values that can be used for most crawl scenarios. If necessary, you can override these default values with those in the site.xml file.

Do not change the name or location of the default.xml configuration file because the Web Crawler is hard-coded to look for that name and path. If you rename the file, the Web Crawler throws an exception at start-up and exit.

The default.xml file provides configuration values for these sets of properties:

HTTP properties
Authentication properties
Proxy properties
Fetcher properties
URL normalization properties
MIME type properties
Plugin properties
Parser properties
Parser filter properties
URL filter properties
Crawl scoping properties
Document Conversion properties
Output file properties

Each set of properties is covered in its own topic page.