K
- The type of the record's key. The same used by internalInputFormatV
- The type of the record's value. The same used by internalInputFormat.public abstract class CompositeInputFormat<K,V> extends <any> implements AbstractCompositeInputFormat<K,V>
Constructor and Description |
---|
CompositeInputFormat() |
Modifier and Type | Method and Description |
---|---|
void |
configure(JobConf conf) |
abstract <any> |
getDelegateRecordReader(FileSplit split, JobConf conf, Reporter reporter)
Creates a record reader instance.
|
static InputSplit |
getFittingInputSplit(JobConf conf, <any> inputFormat, InputSplit split)
Returns an instance of an InputSplit subclass appropriate for the given InputFormat.
|
<any> |
getInternalInputFormat(JobConf jobConf)
Gets an instance of the class set as the internal input format
|
static <K,V> java.lang.Class<? extends <any>> |
getInternalInputFormatClass(JobConf jobConf)
Gets the internal input format type which is used to actually read the data.
|
static <K,V> RecordInfoProvider<K,V> |
getRecordInfoProvider(RecordInfoProvider<K,V> provider, JobConf conf)
Gets an instance of the specified RecordInfoProvider implementation
|
static <K,V> java.lang.Class<? extends RecordInfoProvider<K,V>> |
getRecordInfoProviderClass(JobConf jobConf)
Gets the class of the RecordInfoProvider implementation
|
<any> |
getRecordReader(InputSplit split, JobConf conf, Reporter reporter) |
InputSplit[] |
getSplits(JobConf conf, int splitCount) |
static <K,V> void |
setInternalInputFormatClass(JobConf jobConf, java.lang.Class<? extends <any>> iInputFormat)
Sets the internal input format type which is used to actually read the data.
|
static <K,V> void |
setRecordInfoProviderClass(JobConf jobConf, java.lang.Class<? extends RecordInfoProvider<K,V>> provider)
Sets the class of the RecordInfoProvider implementation
|
public void configure(JobConf conf)
public abstract <any> getDelegateRecordReader(FileSplit split, JobConf conf, Reporter reporter) throws java.io.IOException
split
- a FileSplit instanceconf
- the job configurationreporter
- a Reporter instancejava.io.IOException
public static InputSplit getFittingInputSplit(JobConf conf, <any> inputFormat, InputSplit split) throws java.io.IOException
conf
- the job configurationinputFormat
- an InputFormat instancesplit
- an InputSplit instancejava.io.IOException
public <any> getInternalInputFormat(JobConf jobConf)
getInternalInputFormat
in interface AbstractCompositeInputFormat<K,V>
jobConf
- the job configurationpublic static <K,V> java.lang.Class<? extends <any>> getInternalInputFormatClass(JobConf jobConf)
jobConf
- the job configurationpublic static <K,V> RecordInfoProvider<K,V> getRecordInfoProvider(RecordInfoProvider<K,V> provider, JobConf conf)
provider
- if null, a new instance will be returned, otherwise, the same instance will be returned. It is used to release the caller from checking if the instance is null.conf
- the job configurationpublic static <K,V> java.lang.Class<? extends RecordInfoProvider<K,V>> getRecordInfoProviderClass(JobConf jobConf)
jobConf
- the job configurationpublic <any> getRecordReader(InputSplit split, JobConf conf, Reporter reporter) throws java.io.IOException
java.io.IOException
public InputSplit[] getSplits(JobConf conf, int splitCount) throws java.io.IOException
java.io.IOException
public static <K,V> void setInternalInputFormatClass(JobConf jobConf, java.lang.Class<? extends <any>> iInputFormat)
jobConf
- the job configurationiInputFormat
- a class which is a subclass of FileInputFormat or CombineFileInputFormatpublic static <K,V> void setRecordInfoProviderClass(JobConf jobConf, java.lang.Class<? extends RecordInfoProvider<K,V>> provider)
jobConf
- the job configurationprovider
- a class that extends from RecordInfoProviderCopyright © 2017, 2019 Oracle and/or its affiliates. All Rights Reserved.