public class FileOutputCommitter extends OutputCommitter
OutputCommitter
that commits files specified
in job output directory i.e. ${mapred.output.dir}.Modifier and Type | Field and Description |
---|---|
static java.lang.String |
SUCCEEDED_FILE_NAME |
protected static java.lang.String |
TEMP_DIR_NAME
Temporary directory name
|
Constructor and Description |
---|
FileOutputCommitter(org.apache.hadoop.fs.Path outputPath,
TaskAttemptContext context)
Create a file output committer
|
Modifier and Type | Method and Description |
---|---|
void |
abortJob(JobContext context,
JobStatus.State state)
Delete the temporary directory, including all of the work directories.
|
void |
abortTask(TaskAttemptContext context)
Delete the work directory
|
void |
cleanupJob(JobContext context)
Deprecated.
|
void |
commitJob(JobContext context)
Delete the temporary directory, including all of the work directories.
|
void |
commitTask(TaskAttemptContext context)
Move the files from the work directory to the job output directory
|
org.apache.hadoop.fs.Path |
getWorkPath()
Get the directory that the task should write results into
|
boolean |
needsTaskCommit(TaskAttemptContext context)
Did this task write any files in the work directory?
|
void |
setupJob(JobContext context)
Create the temporary directory that is the root of all of the task
work directories.
|
void |
setupTask(TaskAttemptContext context)
No task setup required.
|
protected static final java.lang.String TEMP_DIR_NAME
public static final java.lang.String SUCCEEDED_FILE_NAME
public FileOutputCommitter(org.apache.hadoop.fs.Path outputPath, TaskAttemptContext context) throws java.io.IOException
outputPath
- the job's output pathcontext
- the task's contextjava.io.IOException
public void setupJob(JobContext context) throws java.io.IOException
setupJob
in class OutputCommitter
context
- the job's contextjava.io.IOException
- if temporary output could not be createdpublic void commitJob(JobContext context) throws java.io.IOException
commitJob
in class OutputCommitter
context
- the job's context.java.io.IOException
@Deprecated public void cleanupJob(JobContext context) throws java.io.IOException
OutputCommitter
cleanupJob
in class OutputCommitter
java.io.IOException
public void abortJob(JobContext context, JobStatus.State state) throws java.io.IOException
abortJob
in class OutputCommitter
context
- the job's contextstate
- final run state of the job, should be FAILED or KILLEDjava.io.IOException
public void setupTask(TaskAttemptContext context) throws java.io.IOException
setupTask
in class OutputCommitter
context
- Context of the task whose output is being written.java.io.IOException
public void commitTask(TaskAttemptContext context) throws java.io.IOException
commitTask
in class OutputCommitter
context
- the task contextjava.io.IOException
- if commit is notpublic void abortTask(TaskAttemptContext context) throws java.io.IOException
abortTask
in class OutputCommitter
java.io.IOException
public boolean needsTaskCommit(TaskAttemptContext context) throws java.io.IOException
needsTaskCommit
in class OutputCommitter
context
- the task's contextjava.io.IOException
public org.apache.hadoop.fs.Path getWorkPath() throws java.io.IOException
java.io.IOException
Copyright © 2009 The Apache Software Foundation