oracle.spatial.hadoop.vector.mapreduce.input

Class MultipleInputsFileSplitInputFormat

java.lang.Object

org.apache.hadoop.mapreduce.InputFormat<K,V>

org.apache.hadoop.mapreduce.lib.input.FileInputFormat<I,J>

oracle.spatial.hadoop.vector.mapreduce.input.WrapperInputFormat<org.apache.hadoop.io.NullWritable,org.apache.hadoop.mapreduce.lib.input.FileSplit,K,V>

oracle.spatial.hadoop.vector.mapreduce.input.FileSplitInputFormat<java.lang.Object,java.lang.Object>

oracle.spatial.hadoop.vector.mapreduce.input.MultipleInputsFileSplitInputFormat

Deprecated.

FileSplitInputFormat subclass that can take more than one data set.
Input data sets are added through MultipleInputsConfig in the following way:

 
            //create MultipleInputsConfig
        MultipleInputsConfig miConf = new MultipleInputsConfig();
        
        //setup and add dataset 1
        InputDataSet dataSet1 = new InputDataSet();
        dataSet1.setInputString("/user/someUser/dataset1/*");
        dataSet1.setInputFormatClass(GeoJsonInputFormat.class);
        dataSet1.setRecordInfoProviderClass(GeoJsonRecordInfoProvider.class);
        miConf.addInputDataSet(dataSet1, jobConf);
        
        //setup and add dataset 2
        InputDataSet dataSet2 = new InputDataSet();
        dataSet1.setIndexName("dataset2_index");
        miConf.addInputDataSet(dataSet2, jobConf);
        
        //save MultipleInputsConfig to the job's configuration
        miConf.store(jobConf);
   
 

public class MultipleInputsFileSplitInputFormat
extends FileSplitInputFormat<java.lang.Object,java.lang.Object>

Nested Class Summary

Nested classes/interfaces inherited from class oracle.spatial.hadoop.vector.mapreduce.input.FileSplitInputFormat

FileSplitInputFormat.FileSplitRecordReader

Field Summary

Fields inherited from class oracle.spatial.hadoop.vector.mapreduce.input.WrapperInputFormat

iInputFormat

Constructor Summary

Constructors

Constructor and Description
MultipleInputsFileSplitInputFormat() Deprecated.

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.NullWritable,org.apache.hadoop.mapreduce.lib.input.FileSplit>	createRecordReader(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context) Deprecated.
int	getDataSetId(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf) Deprecated. Gets the id which identifies the path's data set
org.apache.hadoop.mapreduce.InputFormat<?,?>	getInputFormatForPath(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf) Deprecated. Gets an instance of the InputFormat class used to read the data set from the given path
RecordInfoProvider<?,?>	getRecordInfoProviderForPath(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf) Deprecated. Gets an instance of the RecordInfoProvider class used to interpret records of the data set from the given path
java.util.List<org.apache.hadoop.mapreduce.InputSplit>	getSplits(org.apache.hadoop.mapreduce.JobContext context) Deprecated.

Methods inherited from class oracle.spatial.hadoop.vector.mapreduce.input.WrapperInputFormat

createInternalInputFormat, getFittingInputSplit, getInternalInputFormat, getInternalInputFormatClass, getRecordInfoProvider, getRecordInfoProviderClass, setInternalInputFormatClass, setRecordInfoProviderClass

Methods inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

addInputPath, addInputPaths, computeSplitSize, getBlockIndex, getFormatMinSplitSize, getInputPathFilter, getInputPaths, getMaxSplitSize, getMinSplitSize, isSplitable, listStatus, setInputPathFilter, setInputPaths, setInputPaths, setMaxInputSplitSize, setMinInputSplitSize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MultipleInputsFileSplitInputFormat

public MultipleInputsFileSplitInputFormat()

Deprecated.

Method Detail

getInputFormatForPath

public org.apache.hadoop.mapreduce.InputFormat<?,?> getInputFormatForPath(org.apache.hadoop.fs.Path path,
                                                                          org.apache.hadoop.conf.Configuration conf)
                                                                   throws java.io.IOException

Deprecated.

Gets an instance of the InputFormat class used to read the data set from the given path

Parameters:

path - a data set path

conf - the job configuration

Returns:

an InputFormat instance or null if the given path does not belong to a configured input data set.

Throws:

java.io.IOException

getRecordInfoProviderForPath

public RecordInfoProvider<?,?> getRecordInfoProviderForPath(org.apache.hadoop.fs.Path path,
                                                            org.apache.hadoop.conf.Configuration conf)
                                                     throws java.io.IOException

Deprecated.

Gets an instance of the RecordInfoProvider class used to interpret records of the data set from the given path

Parameters:

path - a data set path

conf - the job configuration

Returns:

an RecordInfoProvider instance or null if the given path does not belong to a configured input data set.

Throws:

java.io.IOException

getDataSetId

public int getDataSetId(org.apache.hadoop.fs.Path path,
                        org.apache.hadoop.conf.Configuration conf)
                 throws java.io.IOException

Deprecated.

Gets the id which identifies the path's data set

Parameters:

path - a data set path

conf - the job configuration

Returns:

a number equal or greater than zero or -1 if the given path does not belong to a configured input data set.

Throws:

java.io.IOException

createRecordReader

public org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.NullWritable,org.apache.hadoop.mapreduce.lib.input.FileSplit> createRecordReader(org.apache.hadoop.mapreduce.InputSplit split,
                                                                                                                                                      org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                                                                                               throws java.io.IOException,
                                                                                                                                                      java.lang.InterruptedException

Deprecated.

Overrides:

createRecordReader in class FileSplitInputFormat<java.lang.Object,java.lang.Object>

Throws:

java.io.IOException

java.lang.InterruptedException

getSplits

public java.util.List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext context)
                                                                 throws java.io.IOException

Deprecated.

Overrides:

getSplits in class WrapperInputFormat<org.apache.hadoop.io.NullWritable,org.apache.hadoop.mapreduce.lib.input.FileSplit,java.lang.Object,java.lang.Object>

Throws:

java.io.IOException