oracle.hadoop.loader.examples

Class CSVInputFormat.CSVRecordReader

java.lang.Object

org.apache.hadoop.mapreduce.RecordReader<java.lang.Object,org.apache.avro.generic.IndexedRecord>

oracle.hadoop.loader.examples.CSVInputFormat.CSVRecordReader

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable

Enclosing class:

CSVInputFormat

public static class CSVInputFormat.CSVRecordReader
extends org.apache.hadoop.mapreduce.RecordReader<java.lang.Object,org.apache.avro.generic.IndexedRecord>

The record reader parses the input data into key/value pairs which are read by OraLoaderMapper. It produces an Object for the key and a org.apache.avro.generic.IndexedRecord for the value.

Constructor Summary

Constructors

Constructor and Description
CSVRecordReader(org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> textRecordReader)

Method Summary

Modifier and Type	Method and Description
void	close() Close the record reader.
java.lang.Object	getCurrentKey() The key should be be an Object whose toString() method return value must help identify the IndexedRecord returned by getCurrentValue().
org.apache.avro.generic.IndexedRecord	getCurrentValue() The next line of text is read from the input and split into tokens using the ',' delimiter.
float	getProgress() The current progress of the record reader through its data.
void	initialize(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context) Called once at initialization.
boolean	nextKeyValue() Read the next key, value pair.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CSVRecordReader

public CSVRecordReader(org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> textRecordReader)

Method Detail

getCurrentValue

public org.apache.avro.generic.IndexedRecord getCurrentValue()
                                                      throws java.io.IOException,
                                                             java.lang.InterruptedException

The next line of text is read from the input and split into tokens using the ',' delimiter. Token values are put into an Avro IndexedRecord which is returned as the key value.

Specified by:

getCurrentValue in class org.apache.hadoop.mapreduce.RecordReader<java.lang.Object,org.apache.avro.generic.IndexedRecord>

Returns:

the current value or null if there is no current value

Throws:

java.io.IOException

java.lang.InterruptedException

getCurrentKey

public java.lang.Object getCurrentKey()
                               throws java.io.IOException,
                                      java.lang.InterruptedException

The key should be be an Object whose toString() method return value must help identify the IndexedRecord returned by getCurrentValue().

This implementation returns the actual Text object that getCurrentValue() tokenizes into an IndexedRecord.

Specified by:

getCurrentKey in class org.apache.hadoop.mapreduce.RecordReader<java.lang.Object,org.apache.avro.generic.IndexedRecord>

Throws:

java.io.IOException

java.lang.InterruptedException

initialize

public void initialize(org.apache.hadoop.mapreduce.InputSplit split,
                       org.apache.hadoop.mapreduce.TaskAttemptContext context)
                throws java.io.IOException,
                       java.lang.InterruptedException

Called once at initialization.

Specified by:

initialize in class org.apache.hadoop.mapreduce.RecordReader<java.lang.Object,org.apache.avro.generic.IndexedRecord>

Parameters:

split - the split that defines the range of records to read

context - the information about the task

Throws:

java.io.IOException

java.lang.InterruptedException

close

public void close()
           throws java.io.IOException

Close the record reader.

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in class org.apache.hadoop.mapreduce.RecordReader<java.lang.Object,org.apache.avro.generic.IndexedRecord>

Throws:

java.io.IOException

nextKeyValue

public boolean nextKeyValue()
                     throws java.io.IOException,
                            java.lang.InterruptedException

Read the next key, value pair.

Specified by:

nextKeyValue in class org.apache.hadoop.mapreduce.RecordReader<java.lang.Object,org.apache.avro.generic.IndexedRecord>

Returns:

true if a key/value pair was read

Throws:

java.io.IOException

java.lang.InterruptedException

getProgress

public float getProgress()
                  throws java.io.IOException,
                         java.lang.InterruptedException

The current progress of the record reader through its data.

Specified by:

getProgress in class org.apache.hadoop.mapreduce.RecordReader<java.lang.Object,org.apache.avro.generic.IndexedRecord>

Returns:

a number between 0.0 and 1.0 that is the fraction of the data read

Throws:

java.io.IOException

java.lang.InterruptedException