Package | Description |
---|---|
org.apache.hadoop.contrib.index.example | |
org.apache.hadoop.contrib.index.lucene | |
org.apache.hadoop.contrib.index.mapred |
Modifier and Type | Method and Description |
---|---|
void |
HashingDistributionPolicy.init(Shard[] shards) |
void |
RoundRobinDistributionPolicy.init(Shard[] shards) |
Constructor and Description |
---|
ShardWriter(org.apache.hadoop.fs.FileSystem fs,
Shard shard,
java.lang.String tempDir,
IndexUpdateConfiguration iconf)
Constructor
|
Modifier and Type | Method and Description |
---|---|
static Shard[] |
Shard.getIndexShards(IndexUpdateConfiguration conf) |
Modifier and Type | Method and Description |
---|---|
RecordWriter<Shard,org.apache.hadoop.io.Text> |
IndexUpdateOutputFormat.getRecordWriter(org.apache.hadoop.fs.FileSystem fs,
JobConf job,
java.lang.String name,
org.apache.hadoop.util.Progressable progress) |
Modifier and Type | Method and Description |
---|---|
int |
Shard.compareTo(Shard other)
Compare to another shard.
|
int |
IndexUpdatePartitioner.getPartition(Shard key,
IntermediateForm value,
int numPartitions) |
void |
IDistributionPolicy.init(Shard[] shards)
Initialization.
|
void |
IndexUpdateCombiner.reduce(Shard key,
java.util.Iterator<IntermediateForm> values,
OutputCollector<Shard,IntermediateForm> output,
Reporter reporter) |
void |
IndexUpdateReducer.reduce(Shard key,
java.util.Iterator<IntermediateForm> values,
OutputCollector<Shard,org.apache.hadoop.io.Text> output,
Reporter reporter) |
void |
IIndexUpdater.run(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path[] inputPaths,
org.apache.hadoop.fs.Path outputPath,
int numMapTasks,
Shard[] shards)
Create a Map/Reduce job configuration and run the Map/Reduce job to
analyze documents and update Lucene instances in parallel.
|
void |
IndexUpdater.run(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path[] inputPaths,
org.apache.hadoop.fs.Path outputPath,
int numMapTasks,
Shard[] shards) |
static void |
Shard.setIndexShards(IndexUpdateConfiguration conf,
Shard[] shards) |
Modifier and Type | Method and Description |
---|---|
void |
IndexUpdateMapper.map(K key,
V value,
OutputCollector<Shard,IntermediateForm> output,
Reporter reporter)
Map a key-value pair to a shard-and-intermediate form pair.
|
void |
IndexUpdateCombiner.reduce(Shard key,
java.util.Iterator<IntermediateForm> values,
OutputCollector<Shard,IntermediateForm> output,
Reporter reporter) |
void |
IndexUpdateReducer.reduce(Shard key,
java.util.Iterator<IntermediateForm> values,
OutputCollector<Shard,org.apache.hadoop.io.Text> output,
Reporter reporter) |
Constructor and Description |
---|
Shard(Shard shard)
Construct using a shard object.
|
Copyright © 2009 The Apache Software Foundation