About utility components

Utility components help you perform basic tasks such as logging and record caching.

Component Description
Dimension Servers

Dimension servers work in conjunction with dimension adapters, and serve as a centralized source of dimension information for all other pipeline components.

Record Caches

A record cache component stores a temporary copy of record data that has been read in by a record adapter. With two exceptions, all sources feeding a join (record assembler) must be record caches.

Spiders

The spider component is the core of the Content Acquisition System, which adds the capability to crawl document hierarchies on a file system or over HTTP or HTTPS.