oracle.spatial.hadoop.vector.mapreduce.input

Class WrapperInputFormat<I,J,K,V>

java.lang.Object

org.apache.hadoop.mapreduce.InputFormat<K,V>

org.apache.hadoop.mapreduce.lib.input.FileInputFormat<I,J>

oracle.spatial.hadoop.vector.mapreduce.input.WrapperInputFormat<I,J,K,V>

Type Parameters:

K - The type of the record's key. The same used by internalInputFormat

V - The type of the record's value. The same used by internalInputFormat.

Direct Known Subclasses:

FileSplitInputFormat, PointKeyInputFormat, SpatialFilterInputFormat

public abstract class WrapperInputFormat<I,J,K,V>
extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<I,J>

Abstract InputFormat which uses another InputFormat (internalInputFormat) to read data and allows adding additional functionality such as spatial filtering. The internal input format must be a subclass of FileInputFormat.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

org.apache.hadoop.mapreduce.lib.input.FileInputFormat.Counter

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>	iInputFormat

Fields inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

DEFAULT_LIST_STATUS_NUM_THREADS, INPUT_DIR, INPUT_DIR_RECURSIVE, LIST_STATUS_NUM_THREADS, NUM_INPUT_FILES, PATHFILTER_CLASS, SPLIT_MAXSIZE, SPLIT_MINSIZE

Constructor Summary

Constructors

Constructor and Description
WrapperInputFormat()

Method Summary

Methods

Modifier and Type	Method and Description
static <K,V> org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>	createInternalInputFormat(org.apache.hadoop.mapreduce.JobContext context)
static org.apache.hadoop.mapreduce.InputSplit	getFittingInputSplit(org.apache.hadoop.mapreduce.JobContext context, org.apache.hadoop.mapreduce.lib.input.FileInputFormat<?,?> inputFormat, org.apache.hadoop.mapreduce.InputSplit split) Returns an instance of an InputSplit subclass appropriate for the given InputFormat.
org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>	getInternalInputFormat(org.apache.hadoop.mapreduce.JobContext context) Gets an instance of the class set as the internal input format
static <K,V> java.lang.Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>>	getInternalInputFormatClass(org.apache.hadoop.mapreduce.JobContext context) Gets the internal input format type which is used to actually read the data.
static <K,V> RecordInfoProvider<K,V>	getRecordInfoProvider(RecordInfoProvider<K,V> provider, org.apache.hadoop.mapreduce.JobContext job) Gets an instance of the specified RecordInfoProvider implementation
static <K,V> java.lang.Class<? extends RecordInfoProvider<K,V>>	getRecordInfoProviderClass(org.apache.hadoop.mapreduce.JobContext job) Gets the class of the RecordInfoProvider implementation
java.util.List<org.apache.hadoop.mapreduce.InputSplit>	getSplits(org.apache.hadoop.mapreduce.JobContext context)
static <K,V> void	setInternalInputFormatClass(org.apache.hadoop.mapreduce.JobContext job, java.lang.Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>> iInputFormat) Sets the internal input format type which is used to actually read the data.
static <K,V> void	setRecordInfoProviderClass(org.apache.hadoop.mapreduce.JobContext job, java.lang.Class<? extends RecordInfoProvider<K,V>> provider) Sets the class of the RecordInfoProvider implementation

Methods inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getFormatMinSplitSize, getInputDirRecursive, getInputPathFilter, getInputPaths, getMaxSplitSize, getMinSplitSize, isSplitable, listStatus, makeSplit, makeSplit, setInputDirRecursive, setInputPathFilter, setInputPaths, setInputPaths, setMaxInputSplitSize, setMinInputSplitSize

Methods inherited from class org.apache.hadoop.mapreduce.InputFormat

createRecordReader

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

iInputFormat

protected org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V> iInputFormat

Constructor Detail

WrapperInputFormat

public WrapperInputFormat()

Method Detail

setInternalInputFormatClass

public static <K,V> void setInternalInputFormatClass(org.apache.hadoop.mapreduce.JobContext job,
                                     java.lang.Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>> iInputFormat)

Sets the internal input format type which is used to actually read the data.

Parameters:

job - the job configuration

iInputFormat - a class which is a subclass of FileInputFormat or CombineFileInputFormat

getInternalInputFormatClass

public static <K,V> java.lang.Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>> getInternalInputFormatClass(org.apache.hadoop.mapreduce.JobContext context)

Gets the internal input format type which is used to actually read the data.

Parameters:

context - the job configuration

Returns:

a class which is a subclass of FileInputFormat or CombineFileInputFormat

createInternalInputFormat

public static <K,V> org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V> createInternalInputFormat(org.apache.hadoop.mapreduce.JobContext context)

getInternalInputFormat

public org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V> getInternalInputFormat(org.apache.hadoop.mapreduce.JobContext context)

Gets an instance of the class set as the internal input format

Parameters:

context - the job configuration

Returns:

an InputFormat instance

setRecordInfoProviderClass

public static <K,V> void setRecordInfoProviderClass(org.apache.hadoop.mapreduce.JobContext job,
                                    java.lang.Class<? extends RecordInfoProvider<K,V>> provider)

Sets the class of the RecordInfoProvider implementation

Parameters:

job - the job configuration

provider - a class that extends from RecordInfoProvider

getRecordInfoProviderClass

public static <K,V> java.lang.Class<? extends RecordInfoProvider<K,V>> getRecordInfoProviderClass(org.apache.hadoop.mapreduce.JobContext job)

Gets the class of the RecordInfoProvider implementation

Parameters:

job - the job configuration

Returns:

a class that extends from RecordInfoProvider

getRecordInfoProvider

public static <K,V> RecordInfoProvider<K,V> getRecordInfoProvider(RecordInfoProvider<K,V> provider,
                                                  org.apache.hadoop.mapreduce.JobContext job)

Gets an instance of the specified RecordInfoProvider implementation

Parameters:

provider - if null, a new instance will be returned, otherwise, the same instance will be returned. It is used to release the caller from checking if the instance is null.

job - the job configuration

Returns:

a RecordInfoProvider instance

getSplits

public java.util.List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext context)
                                                                 throws java.io.IOException

Overrides:

getSplits in class org.apache.hadoop.mapreduce.lib.input.FileInputFormat<I,J>

Throws:

java.io.IOException

getFittingInputSplit

public static org.apache.hadoop.mapreduce.InputSplit getFittingInputSplit(org.apache.hadoop.mapreduce.JobContext context,
                                                          org.apache.hadoop.mapreduce.lib.input.FileInputFormat<?,?> inputFormat,
                                                          org.apache.hadoop.mapreduce.InputSplit split)
                                                                   throws java.io.IOException

Returns an instance of an InputSplit subclass appropriate for the given InputFormat. For example, if the passed split is a FileSplit instance and the InputFormat is a CombineFileInputFormat implementation, the returned split will be a instance of CombineFileSplit.

Parameters:

context - the job configuration

inputFormat - an InputFormat instance

split - an InputSplit instance

Returns:

if the passed split fits the input format, it is returned, otherwise a new one that fits is returned.

Throws:

java.io.IOException