Life cycle of a data source

The following actions indicate key points in the life cycle of a data source. The events take place after a data source extension has been implemented, packaged, and installed into the Integrator Acquisition System.

Action Corresponding life cycle event
The Endeca IAS Service starts. IAS Server instantiates any DataSource that is installed and correctly annotated.
The following items perform the same action.
  • In the IAS Server API - Calling either IasCrawler.createCrawl() or IasCrawler.updateCrawl().
  • In the ias-cmd utility - Running either createCrawls or updateCrawls.
IAS Server instantiates the PipelineComponentConfiguration specified by the DataSource.

IAS Server performs data type and constraint validation and also calls PipelineComponentConfiguration.validate() to execute any additional custom validation code you wrote in the method's implementation.

The following items perform the same action.
  • In the IAS Server API - Calling IasCrawler.startCrawl().
  • In the ias-cmd utility - Running startCrawl.
IAS Server re-creates the PipelineComponentConfiguration specified by the DataSource and re-validates the PipelineComponentConfiguration by calling validate(). (IAS Server injects configuration property values via reflection before calling validate().)

IAS Server calls createDataSourceRuntime() on the DataSource instance.

The resulting DataSourceRuntime exists for the duration of the acquisition.

If an incremental acquisition is requested, IAS Server calls checkFullAcquisitionRequired() on the new DataSourceRuntime. The check returns a Boolean value to indicate whether a full acquisition is required or not.

If any data source or manipulator in the IAS pipeline returns true, IAS Server calls DataSourceRuntime.runFullAcquisition().

If all extensions return false, IAS Server calls IncrementalDataSourceRuntime.runIncrementalAcquisition().

After all records have been processed by all extensions, IAS Server calls PipelineComponentRuntime.endAcquisition().

IAS Server passes in an AcquisitionEndState value to endAcquisition() to indicate whether the extension succeeded, failed, or requires a full acquisition to recover from a failure.

The following items perform the same action.
  • In the IAS Server API - Calling IasCrawler.deleteCrawl().
  • In the ias-cmd utility - Running deleteCrawl.
IAS Server calls PipelineComponent.deleteInstance() when a data source is removed from an application as part of updating or deleting an acquisition, and deleteInstance() is also called if the data source type of the acquisition has changed. (The data source type is represented by a ModuleId setting in the IAS Server API).