public class CombineFileRecordReader<K,V> extends RecordReader<K,V>
CombineFileSplit
.
A CombineFileSplit can combine data chunks from multiple files.
This class allows using different RecordReaders for processing
these data chunks from different files.CombineFileSplit
Modifier and Type | Field and Description |
---|---|
protected TaskAttemptContext |
context |
protected RecordReader<K,V> |
curReader |
protected org.apache.hadoop.fs.FileSystem |
fs |
protected int |
idx |
protected long |
progress |
protected java.lang.Class<? extends RecordReader<K,V>> |
rrClass |
protected java.lang.reflect.Constructor<? extends RecordReader<K,V>> |
rrConstructor |
protected CombineFileSplit |
split |
Constructor and Description |
---|
CombineFileRecordReader(CombineFileSplit split,
TaskAttemptContext context,
java.lang.Class<? extends RecordReader<K,V>> rrClass)
A generic RecordReader that can hand out different recordReaders
for each chunk in the CombineFileSplit.
|
Modifier and Type | Method and Description |
---|---|
void |
close()
Close the record reader.
|
K |
getCurrentKey()
Get the current key
|
V |
getCurrentValue()
Get the current value.
|
float |
getProgress()
return progress based on the amount of data processed so far.
|
void |
initialize(InputSplit split,
TaskAttemptContext context)
Called once at initialization.
|
protected boolean |
initNextRecordReader()
Get the record reader for the next chunk in this CombineFileSplit.
|
boolean |
nextKeyValue()
Read the next key, value pair.
|
protected CombineFileSplit split
protected java.lang.Class<? extends RecordReader<K,V>> rrClass
protected java.lang.reflect.Constructor<? extends RecordReader<K,V>> rrConstructor
protected org.apache.hadoop.fs.FileSystem fs
protected TaskAttemptContext context
protected int idx
protected long progress
protected RecordReader<K,V> curReader
public CombineFileRecordReader(CombineFileSplit split, TaskAttemptContext context, java.lang.Class<? extends RecordReader<K,V>> rrClass) throws java.io.IOException
java.io.IOException
public void initialize(InputSplit split, TaskAttemptContext context) throws java.io.IOException, java.lang.InterruptedException
RecordReader
initialize
in class RecordReader<K,V>
split
- the split that defines the range of records to readcontext
- the information about the taskjava.io.IOException
java.lang.InterruptedException
public boolean nextKeyValue() throws java.io.IOException, java.lang.InterruptedException
RecordReader
nextKeyValue
in class RecordReader<K,V>
java.io.IOException
java.lang.InterruptedException
public K getCurrentKey() throws java.io.IOException, java.lang.InterruptedException
RecordReader
getCurrentKey
in class RecordReader<K,V>
java.io.IOException
java.lang.InterruptedException
public V getCurrentValue() throws java.io.IOException, java.lang.InterruptedException
RecordReader
getCurrentValue
in class RecordReader<K,V>
java.io.IOException
java.lang.InterruptedException
public void close() throws java.io.IOException
RecordReader
close
in interface java.io.Closeable
close
in interface java.lang.AutoCloseable
close
in class RecordReader<K,V>
java.io.IOException
public float getProgress() throws java.io.IOException, java.lang.InterruptedException
getProgress
in class RecordReader<K,V>
java.io.IOException
java.lang.InterruptedException
protected boolean initNextRecordReader() throws java.io.IOException
java.io.IOException
Copyright © 2009 The Apache Software Foundation