Dimension Servers

Dimension servers work in conjunction with dimension adapters, and serve as a centralized source of dimension information for all other pipeline components.

Record Caches

A record cache component stores a temporary copy of record data that has been read in by a record adapter. With two exceptions, all sources feeding a join (record assembler) must be record caches.

Spiders

The spider component is the core of the Content Acquisition System, which adds the capability to crawl document hierarchies on a file system or over HTTP or HTTPS.