com.endeca.cas.extension

Class PipelineComponentRuntime

java.lang.Object

com.endeca.cas.extension.PipelineComponentRuntime

Direct Known Subclasses:

DataSourceRuntime, ManipulatorRuntime

public abstract class PipelineComponentRuntime
extends Object

The runtime representation of a pipeline component. A new PipelineComponentRuntime is created for every acquisition.

Subclasses must be thread-safe. The CAS Server will invoke methods on this class from multiple-threads and potentially concurrently. Access to any mutable state must be properly synchronized.

See Also:

DataSourceRuntime, ManipulatorRuntime

Constructor Summary

Constructors

Constructor and Description
PipelineComponentRuntime(PipelineComponentRuntimeContext context)

Method Summary

Modifier and Type	Method and Description
void	endAcquisition(AcquisitionEndState currentAcquisitionEndState) Performs any necessary cleanup of the PipelineComponentRuntime.
protected PipelineComponentRuntimeContext	getContext() Returns the PipelineComponentRuntimeContext of this runtime.
void	stop() Called to request the PipelineComponentRuntime stop processing records.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PipelineComponentRuntime

public PipelineComponentRuntime(PipelineComponentRuntimeContext context)

Method Detail

stop

public void stop()

Called to request the PipelineComponentRuntime stop processing records. Several things happen when the CAS Server stops a pipeline:

• Calls to {link OutputChannel.output(com.endeca.itl.record.Record) throw a PipelineStoppedException.
• PipelineComponentRuntimeContext.isStopped() returns true.
• PipelineComponentRuntime.stop() is called on all data source and manipulator runtimes in the crawl.
• The queue of pending pipeline work is drained.

Using these mechanisms, runtimes should make their best effort to terminate processing in a timely manner after a stop has been requested. Here are some implementation recommendations to ensure proper stop behavior:

• If the runtime does not go long periods of time without outputting records, just rely on {link OutputChannel.output(com.endeca.itl.record.Record) throwing a PipelineStoppedException to terminate runtime processing.
• If the runtime does go long periods of time without outputting records, periodically call PipelineComponentRuntimeContext.isStopped() and terminate processing if a stop has been requested.
• If the runtime does go long periods of time without outputting records and cannot periodically call PipelineComponentRuntimeContext.isStopped() to check for stop requests, override stop() to terminate processing. For example, if a runtime is making blocking network requests, stop() might be overridden to cancel these network requests.

The CAS Server will still invoke endAcquisition(AcquisitionEndState) after the acquisition has finished stopping.

Subclasses that override this method must do so in a thread-safe manner. It can be invoked concurrently to any of the following:

• DataSourceRuntime.runFullAcquisition()
• ManipulatorRuntime.processRecord(com.endeca.itl.record.Record)
• ManipulatorRuntime.onInputClose()
• BinaryContentFileProvider.getBinaryContentFile(com.endeca.itl.record.Record)
• BinaryContentInputStreamProvider.getBinaryContentInputStream(com.endeca.itl.record.Record)

See Also:

PipelineComponentRuntimeContext.isStopped(), PipelineStoppedException

getContext

protected final PipelineComponentRuntimeContext getContext()

Returns the PipelineComponentRuntimeContext of this runtime.

endAcquisition

public void endAcquisition(AcquisitionEndState currentAcquisitionEndState)
                    throws FullAcquisitionRecoveryRequiredException

Performs any necessary cleanup of the PipelineComponentRuntime. This method is guaranteed to be called any time after the runtime is constructed exactly once, normally after all pipeline processing has stopped for any reason.

The argument currentAcquisitionEndState is FAILURE when the crawl was stopped short of completion (aborted or due to a failure) and may change to FULL_ACQUISITION_RECOVERY_REQUIRED if an endAcquisition() call on another pipeline component threw an exception.

When the argument currentAcquisitionEndState is AcquisitionEndState.FULL_ACQUISITION_RECOVERY_REQUIRED, it means the next data acquisition will be in full mode, any previously saved crawl state will be cleared at the next acquisition if the pipeline data source runtime does not implement the IncrementalDataSourceRuntime interface. If the pipeline data source runtime implements IncrementalDataSourceRuntime, it is expected to perform the fastest cleanup to optimize for responsiveness and correctly start from a clean state the next time runFullAcquisition is called on the data source.

If CAS catches a Throwable during this call, current acquisition end state will transition to AcquisitionEndState.FULL_ACQUISITION_RECOVERY_REQUIRED.

The output channel will not be available in the scope of this call.

Parameters:

currentAcquisitionEndState -

Throws:

FullAcquisitionRecoveryRequiredException