public class CompositeInputFormat<K extends org.apache.hadoop.io.WritableComparable> extends java.lang.Object implements ComposableInputFormat<K,TupleWritable>
A user may define new join types by setting the property
mapred.join.define.<ident> to a classname. In the expression
mapred.join.expr, the identifier will be assumed to be a
ComposableRecordReader.
mapred.join.keycomparator can be a classname used to compare keys
in the join.
,
JoinRecordReader
,
MultiFilterRecordReader
Constructor and Description |
---|
CompositeInputFormat() |
Modifier and Type | Method and Description |
---|---|
protected void |
addDefaults()
Adds the default set of identifiers to the parser.
|
static java.lang.String |
compose(java.lang.Class<? extends InputFormat> inf,
java.lang.String path)
Convenience method for constructing composite formats.
|
static java.lang.String |
compose(java.lang.String op,
java.lang.Class<? extends InputFormat> inf,
org.apache.hadoop.fs.Path... path)
Convenience method for constructing composite formats.
|
static java.lang.String |
compose(java.lang.String op,
java.lang.Class<? extends InputFormat> inf,
java.lang.String... path)
Convenience method for constructing composite formats.
|
ComposableRecordReader<K,TupleWritable> |
getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
Construct a CompositeRecordReader for the children of this InputFormat
as defined in the init expression.
|
InputSplit[] |
getSplits(JobConf job,
int numSplits)
Build a CompositeInputSplit from the child InputFormats by assigning the
ith split from each child to the ith composite split.
|
void |
setFormat(JobConf job)
Interpret a given string as a composite expression.
|
public void setFormat(JobConf job) throws java.io.IOException
func ::= <ident>([<func>,]*<func>)
func ::= tbl(<class>,"<path>")
class ::= @see java.lang.Class#forName(java.lang.String)
path ::= @see org.apache.hadoop.fs.Path#Path(java.lang.String)
Reads expression from the mapred.join.expr property and
user-supplied join types from mapred.join.define.<ident>
types. Paths supplied to tbl are given as input paths to the
InputFormat class listed.java.io.IOException
compose(java.lang.String, java.lang.Class, java.lang.String...)
protected void addDefaults()
public InputSplit[] getSplits(JobConf job, int numSplits) throws java.io.IOException
getSplits
in interface InputFormat<K extends org.apache.hadoop.io.WritableComparable,TupleWritable>
job
- job configuration.numSplits
- the desired number of splits, a hint.InputSplit
s for the job.java.io.IOException
public ComposableRecordReader<K,TupleWritable> getRecordReader(InputSplit split, JobConf job, Reporter reporter) throws java.io.IOException
getRecordReader
in interface InputFormat<K extends org.apache.hadoop.io.WritableComparable,TupleWritable>
getRecordReader
in interface ComposableInputFormat<K extends org.apache.hadoop.io.WritableComparable,TupleWritable>
split
- the InputSplit
job
- the job that this split belongs toRecordReader
java.io.IOException
public static java.lang.String compose(java.lang.Class<? extends InputFormat> inf, java.lang.String path)
tbl(<inf>, <p>)
public static java.lang.String compose(java.lang.String op, java.lang.Class<? extends InputFormat> inf, java.lang.String... path)
<op>(tbl(<inf>,<p1>),tbl(<inf>,<p2>),...,tbl(<inf>,<pn>))
public static java.lang.String compose(java.lang.String op, java.lang.Class<? extends InputFormat> inf, org.apache.hadoop.fs.Path... path)
<op>(tbl(<inf>,<p1>),tbl(<inf>,<p2>),...,tbl(<inf>,<pn>))
Copyright © 2009 The Apache Software Foundation