com.endeca.cas.extension
Interface IncrementalDataSourceRuntime

public interface IncrementalDataSourceRuntime

Interface that can be implemented by a DataSourceRuntime that handles its own incremental acquisitions (in contrast with CAS handling the incremental acquisition).

Once an InrementalDataSourceRuntime is created, CAS only calls checkFullAcquisitionRequired if an incremental is requested to determine if runIncrementalAcquisition should be called. CAS calls runIncrementalAcquisition if calling checkFullAcquisitionRequired on every data source and manipulator in the crawl returns false.

CAS expects an incremental data source to coalesce multiple changes so that only one record is output for a given record id. For performance reasons, CAS does not ensure the output records from the data source are unique.

See Also:

DataSourceRuntime

Method Summary

boolean	checkFullAcquisitionRequired() Indicates whether a full acquisition is required based on the state maintained by the IncrementalDataSourceRuntime.
void	runIncrementalAcquisition() Runs an incremental acquisition of the data source.

Method Detail

checkFullAcquisitionRequired

boolean checkFullAcquisitionRequired()

Indicates whether a full acquisition is required based on the state maintained by the IncrementalDataSourceRuntime.

If CAS determines a full acquisition is required, this method will not be called. CAS does not call this method if the data source is first created and has never been run or when a configuration change requires full acquisition.

Returns:

true if this runtime determines a full data acquisition is required, false otherwise.

See Also:

PipelineComponentConfiguration.isFullAcquisitionRequired(PipelineComponentConfiguration)

runIncrementalAcquisition

void runIncrementalAcquisition()
                               throws FatalExecutionException

Runs an incremental acquisition of the data source. The data source runtime should output no more records after this method returns.

This method will only be called by CAS when the checkFullAcquisitionRequired() method on this instance returns false. However, if CAS determines a full acquisition is required, checkFullAcquisitionRequired will not be called and consequently this method will not be called either.

CAS does not ensure records will be processed in the ordered they were output, so it is important for the implementation to only output one record for a given record id.

Throws:

FatalExecutionException - if acquisition failed and all processing of records already output should stop.