Additional pipeline components

Record assemblers

Join data from one or more secondary data sources to the current record.

"Adding a record assembler" in this guide and in the Endeca Developer Studio Help.

Record caches

Store a temporary copy of record data that has been read in by a record adapter. Record caches are generally used in conjunction with record assemblers and are set up to contain data from secondary data sources.

"Adding a record cache" in this guide and in the Endeca Developer Studio Help.

Java manipulators

A Java manipulator is your own code in Java that you can use to perform data manipulation on properties and records. Java manipulators provide you with the most generic way of changing records in the Forge pipeline.

A Java manipulator contains a class that is based on the Java API Adapter interface in the Content Adapter Development Kit (CADK).

For information on how to write your own Java manipulator and for a sample code, see the Endeca Content Adapter Development Kit (CADK) Guide.

Perl manipulators

Allow you to write custom Perl code that changes the data associated with an Endeca record. Perl manipulators are useful for such tasks as manually adding or removing source properties, changing the value of a source property, retrieving records based on a particular key, and so on.

See "Using Perl Manipulators to Change Source Properties" in the Developer Studio Help.

For details on Perl code syntax, see the Endeca Forge API Guide for Perl.

Spiders

Crawl document hierarchies on a file system or over HTTP. From a root URL, a spider spools URLs of documents to crawl.

"Creating a spider" in this guide.

Record manipulators

Provide support, such as URL extraction, for a content acquisition system, such as a crawler implementation.

See the Endeca Crawler section in this guide, and "Record Manipulators and Expressions" in the Developer Studio Help.

Update adapters

Provide support for partial (rapid) updates.

See the Endeca Partial Updates Guide.