oracle.spatial.hadoop.vector.mapred.input

Class CompositeInputFormat<K,V>

java.lang.Object

<any>

oracle.spatial.hadoop.vector.mapred.input.CompositeInputFormat<K,V>

Type Parameters:

K - The type of the record's key. The same used by internalInputFormat

V - The type of the record's value. The same used by internalInputFormat.

All Implemented Interfaces:

AbstractCompositeInputFormat<K,V>

Direct Known Subclasses:

FileSplitInputFormat, SpatialFilterInputFormat

public abstract class CompositeInputFormat<K,V>
extends <any>
implements AbstractCompositeInputFormat<K,V>

Abstract InputFormat which uses another InputFormat (internalInputFormat) to read data and allows adding additional functionality such as filtering the data. The internal input format must be a subclass of FileInputFormat or CombineFileInputFormat.

Author:

hsaucedo

Constructor Summary

Constructors

Constructor and Description
CompositeInputFormat()

Method Summary

Modifier and Type	Method and Description
void	configure(JobConf conf)
abstract <any>	getDelegateRecordReader(FileSplit split, JobConf conf, Reporter reporter) Creates a record reader instance.
static InputSplit	getFittingInputSplit(JobConf conf, <any> inputFormat, InputSplit split) Returns an instance of an InputSplit subclass appropriate for the given InputFormat.
<any>	getInternalInputFormat(JobConf jobConf) Gets an instance of the class set as the internal input format
static <K,V> java.lang.Class<? extends <any>>	getInternalInputFormatClass(JobConf jobConf) Gets the internal input format type which is used to actually read the data.
static <K,V> RecordInfoProvider<K,V>	getRecordInfoProvider(RecordInfoProvider<K,V> provider, JobConf conf) Gets an instance of the specified RecordInfoProvider implementation
static <K,V> java.lang.Class<? extends RecordInfoProvider<K,V>>	getRecordInfoProviderClass(JobConf jobConf) Gets the class of the RecordInfoProvider implementation
<any>	getRecordReader(InputSplit split, JobConf conf, Reporter reporter)
InputSplit[]	getSplits(JobConf conf, int splitCount)
static <K,V> void	setInternalInputFormatClass(JobConf jobConf, java.lang.Class<? extends <any>> iInputFormat) Sets the internal input format type which is used to actually read the data.
static <K,V> void	setRecordInfoProviderClass(JobConf jobConf, java.lang.Class<? extends RecordInfoProvider<K,V>> provider) Sets the class of the RecordInfoProvider implementation

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CompositeInputFormat

public CompositeInputFormat()

Method Detail

configure

public void configure(JobConf conf)

getDelegateRecordReader

public abstract <any> getDelegateRecordReader(FileSplit split,
                                              JobConf conf,
                                              Reporter reporter)
                                       throws java.io.IOException

Creates a record reader instance. The returned record instance can be a wrapper of the internalInputFormat's record reader. This method is called after getRecordReader(). If the internalInputFormat is a CombineFileInputFormat subclass, this method can be called as many times as paths are contained in the CombineFileSplits.

Parameters:

split - a FileSplit instance

conf - the job configuration

reporter - a Reporter instance

Returns:

a RecordReader instance.

Throws:

java.io.IOException

getFittingInputSplit

public static InputSplit getFittingInputSplit(JobConf conf,
                                              <any> inputFormat,
                                              InputSplit split)
                                       throws java.io.IOException

Returns an instance of an InputSplit subclass appropriate for the given InputFormat. For example, if the passed split is a FileSplit instance and the InputFormat is a CombineFileInputFormat implementation, the returned split will be a instance of CombineFileSplit.

Parameters:

conf - the job configuration

inputFormat - an InputFormat instance

split - an InputSplit instance

Returns:

if the passed split fits the input format, it is returned, otherwise a new one that fits is returned.

Throws:

java.io.IOException

getInternalInputFormat

public <any> getInternalInputFormat(JobConf jobConf)

Gets an instance of the class set as the internal input format

Specified by:

getInternalInputFormat in interface AbstractCompositeInputFormat<K,V>

Parameters:

jobConf - the job configuration

Returns:

an InputFormat instance

getInternalInputFormatClass

public static <K,V> java.lang.Class<? extends <any>> getInternalInputFormatClass(JobConf jobConf)

Gets the internal input format type which is used to actually read the data.

Parameters:

jobConf - the job configuration

Returns:

a class which is a subclass of FileInputFormat or CombineFileInputFormat

getRecordInfoProvider

public static <K,V> RecordInfoProvider<K,V> getRecordInfoProvider(RecordInfoProvider<K,V> provider,
                                                                  JobConf conf)

Gets an instance of the specified RecordInfoProvider implementation

Parameters:

provider - if null, a new instance will be returned, otherwise, the same instance will be returned. It is used to release the caller from checking if the instance is null.

conf - the job configuration

Returns:

a RecordInfoProvider instance

getRecordInfoProviderClass

public static <K,V> java.lang.Class<? extends RecordInfoProvider<K,V>> getRecordInfoProviderClass(JobConf jobConf)

Gets the class of the RecordInfoProvider implementation

Parameters:

jobConf - the job configuration

Returns:

a class that extends from RecordInfoProvider

getRecordReader

public <any> getRecordReader(InputSplit split,
                             JobConf conf,
                             Reporter reporter)
                      throws java.io.IOException

Throws:

java.io.IOException

getSplits

public InputSplit[] getSplits(JobConf conf,
                              int splitCount)
                       throws java.io.IOException

Throws:

java.io.IOException

setInternalInputFormatClass

public static <K,V> void setInternalInputFormatClass(JobConf jobConf,
                                                     java.lang.Class<? extends <any>> iInputFormat)

Sets the internal input format type which is used to actually read the data.

Parameters:

jobConf - the job configuration

iInputFormat - a class which is a subclass of FileInputFormat or CombineFileInputFormat

setRecordInfoProviderClass

public static <K,V> void setRecordInfoProviderClass(JobConf jobConf,
                                                    java.lang.Class<? extends RecordInfoProvider<K,V>> provider)

Sets the class of the RecordInfoProvider implementation

Parameters:

jobConf - the job configuration

provider - a class that extends from RecordInfoProvider