Creating a pipeline component configuration class for a data source

A data source extension requires a PipelineComponentConfiguration class to describe the extension's configuration, to validate the data source's configuration, and to determine whether a configuration change requires a full acquisition for the data source.

1. In the Java project that contains your DataSource implementation, create a subclass of PipelineComponentConfiguration.
   For example:

   public class CsvDataSourceConfig extends 
      PipelineComponentConfiguration<CsvDataSourceConfig>{
   
   }

2. Add each field that you want available as a configuration property in the data source.
   For example:

   private String mInputFile;
   	
   private String mKeyColumn;

3. Add an annotation to each field. The annotation type must match the field's data type. See package com.endeca.ias.extension.annotation in the IAS Extension API Reference (Javadoc) to determine which annotations have required attributes and to determine which attributes are appropriate for the field. Also see, About IAS annotations.
   For example:

   @StringProperty(isRequired=true, name="inputFile", 
      displayName="Input File", description="Path to the input csv file e.g. c:\\incoming\\data.csv")
   private String mInputFile;
   	
   @StringProperty(isRequired=true, name="keyColumn", 
      displayName="Key Column", description="Name of the column with the record key")
   private String mKeyColumn;

4. If you want to order configuration properties within a single group, add a @ConfigurationGroupOrder annotation to the PipelineComponentConfiguration class and then add a nested @ConfigurationGroup annotation.
   For example, here is one group of fields that display in order — filePath, headerRow, and separator:

   @ConfigurationGroupOrder({@ConfigurationGroup(groupName="Basic", 
      propertyOrder={"filePath","headerRow","separator"})})
   public class CsvDataSourceConfig implements 
      PipelineComponentConfiguration<CsvDataSourceConfig>{
   
   }

   If you omit propertyOrder, the properties are sorted alphabetically and display alphabetically.

5. If you want to order multiple groups, add additional @ConfigurationGroup annotations within @ConfigurationGroupOrder for each group of fields that you want ordered.
6. Optionally, override the default implementation of isFullAcquisitionRequired(). The default implementation determines whether a configuration change should force full acquisition the next time an acquisition is run by comparing the old PipelineComponentConfiguration and the new PipelineComponentConfiguration using the equals() method. The default implementation of the equals() method uses reflection to compare all non-transient fields for equality.

   You can write code that checks a specific property to determine if a full acquisition is required (rather than the entire PipelineComponentConfiguration). If you want to force a full acquisition, write code that always returns true.

7. Optionally, override the default implementation of validate(). IAS Server performs data type and constraint validation (constraints may include minValue and maxValue for integer properties). Any code you write in validate() performs additional custom validation.
   For example:

   public List<ValidationFailure> validate(
   			CsvFileDataSourceConfiguration configuration) {
   		
   		List<ValidationFailure> validationFailures = 
           new LinkedList<ValidationFailure>();
   		File checkFile = new File(configuration.getFilePath());
   		
   		if (!checkFile.exists()) {
   			validationFailures.add(new ValidationFailure("File " + 
            checkFile.getAbsolutePath() + " does not exist"));
   		}
   		
   		return validationFailures;
   }

   If validation fails, the PipelineComponentConfiguration.validate() method returns a collection ValidationFailure objects.

8. If it is necessary for unit testing or for the implementation of the data source runtime, you may also need to write getter and setter methods for each field that you added.