public abstract class MultipleOutputFormat<K,V> extends FileOutputFormat<K,V>
Constructor and Description |
---|
MultipleOutputFormat() |
Modifier and Type | Method and Description |
---|---|
protected K |
generateActualKey(K key,
V value)
Generate the actual key from the given key/value.
|
protected V |
generateActualValue(K key,
V value)
Generate the actual value from the given key and value.
|
protected java.lang.String |
generateFileNameForKeyValue(K key,
V value,
java.lang.String name)
Generate the file output file name based on the given key and the leaf file
name.
|
protected java.lang.String |
generateLeafFileName(java.lang.String name)
Generate the leaf name for the output file name.
|
protected abstract RecordWriter<K,V> |
getBaseRecordWriter(org.apache.hadoop.fs.FileSystem fs,
JobConf job,
java.lang.String name,
org.apache.hadoop.util.Progressable arg3) |
protected java.lang.String |
getInputFileBasedOutputFileName(JobConf job,
java.lang.String name)
Generate the outfile name based on a given anme and the input file name.
|
RecordWriter<K,V> |
getRecordWriter(org.apache.hadoop.fs.FileSystem fs,
JobConf job,
java.lang.String name,
org.apache.hadoop.util.Progressable arg3)
Create a composite record writer that can write key/value data to different
output files
|
checkOutputSpecs, getCompressOutput, getOutputCompressorClass, getOutputPath, getPathForCustomFile, getTaskOutputPath, getUniqueName, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputPath
public RecordWriter<K,V> getRecordWriter(org.apache.hadoop.fs.FileSystem fs, JobConf job, java.lang.String name, org.apache.hadoop.util.Progressable arg3) throws java.io.IOException
getRecordWriter
in interface OutputFormat<K,V>
getRecordWriter
in class FileOutputFormat<K,V>
fs
- the file system to usejob
- the job conf for the jobname
- the leaf file name for the output file (such as part-00000")arg3
- a progressable for reporting progress.java.io.IOException
protected java.lang.String generateLeafFileName(java.lang.String name)
name
- the leaf file name for the output fileprotected java.lang.String generateFileNameForKeyValue(K key, V value, java.lang.String name)
key
- the key of the output dataname
- the leaf file nameprotected K generateActualKey(K key, V value)
key
- the key of the output datavalue
- the value of the output dataprotected V generateActualValue(K key, V value)
key
- the key of the output datavalue
- the value of the output dataprotected java.lang.String getInputFileBasedOutputFileName(JobConf job, java.lang.String name)
job
- the job configname
- the output file nameprotected abstract RecordWriter<K,V> getBaseRecordWriter(org.apache.hadoop.fs.FileSystem fs, JobConf job, java.lang.String name, org.apache.hadoop.util.Progressable arg3) throws java.io.IOException
fs
- the file system to usejob
- a job conf objectname
- the name of the file over which a record writer object will be
constructedarg3
- a progressable objectjava.io.IOException
Copyright © 2009 The Apache Software Foundation