org.apache.hadoop.mapred
Class MapOutputFile

java.lang.Object
  extended by org.apache.hadoop.mapred.MapOutputFile

@InterfaceAudience.LimitedPrivate(value="MapReduce")
@InterfaceStability.Unstable
public class MapOutputFile
extends java.lang.Object

Manipulate the working area for the transient store for maps and reduces. This class is used by map and reduce tasks to identify the directories that they need to write to/read from for intermediate files. The callers of these methods are from child space and see mapreduce.cluster.local.dir as taskTracker/jobCache/jobId/attemptId This class should not be used from TaskTracker space.


Method Summary
 org.apache.hadoop.fs.Path getInputFile(int mapId)
          Return a local reduce input file created earlier
 org.apache.hadoop.fs.Path getInputFileForWrite(TaskID mapId, long size)
          Create a local reduce input file name.
 org.apache.hadoop.fs.Path getOutputFile()
          Return the path to local map output file created earlier
 org.apache.hadoop.fs.Path getOutputFileForWrite(long size)
          Create a local map output file name.
 org.apache.hadoop.fs.Path getOutputIndexFile()
          Return the path to a local map output index file created earlier
 org.apache.hadoop.fs.Path getOutputIndexFileForWrite(long size)
          Create a local map output index file name.
 org.apache.hadoop.fs.Path getSpillFile(int spillNumber)
          Return a local map spill file created earlier.
 org.apache.hadoop.fs.Path getSpillFileForWrite(int spillNumber, long size)
          Create a local map spill file name.
 org.apache.hadoop.fs.Path getSpillIndexFile(int spillNumber)
          Return a local map spill index file created earlier
 org.apache.hadoop.fs.Path getSpillIndexFileForWrite(int spillNumber, long size)
          Create a local map spill index file name.
 void removeAll()
          Removes all of the files related to a task.
 void setConf(org.apache.hadoop.conf.Configuration conf)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Method Detail

getOutputFile

public org.apache.hadoop.fs.Path getOutputFile()
                                        throws java.io.IOException
Return the path to local map output file created earlier

Returns:
path
Throws:
java.io.IOException

getOutputFileForWrite

public org.apache.hadoop.fs.Path getOutputFileForWrite(long size)
                                                throws java.io.IOException
Create a local map output file name.

Parameters:
size - the size of the file
Returns:
path
Throws:
java.io.IOException

getOutputIndexFile

public org.apache.hadoop.fs.Path getOutputIndexFile()
                                             throws java.io.IOException
Return the path to a local map output index file created earlier

Returns:
path
Throws:
java.io.IOException

getOutputIndexFileForWrite

public org.apache.hadoop.fs.Path getOutputIndexFileForWrite(long size)
                                                     throws java.io.IOException
Create a local map output index file name.

Parameters:
size - the size of the file
Returns:
path
Throws:
java.io.IOException

getSpillFile

public org.apache.hadoop.fs.Path getSpillFile(int spillNumber)
                                       throws java.io.IOException
Return a local map spill file created earlier.

Parameters:
spillNumber - the number
Returns:
path
Throws:
java.io.IOException

getSpillFileForWrite

public org.apache.hadoop.fs.Path getSpillFileForWrite(int spillNumber,
                                                      long size)
                                               throws java.io.IOException
Create a local map spill file name.

Parameters:
spillNumber - the number
size - the size of the file
Returns:
path
Throws:
java.io.IOException

getSpillIndexFile

public org.apache.hadoop.fs.Path getSpillIndexFile(int spillNumber)
                                            throws java.io.IOException
Return a local map spill index file created earlier

Parameters:
spillNumber - the number
Returns:
path
Throws:
java.io.IOException

getSpillIndexFileForWrite

public org.apache.hadoop.fs.Path getSpillIndexFileForWrite(int spillNumber,
                                                           long size)
                                                    throws java.io.IOException
Create a local map spill index file name.

Parameters:
spillNumber - the number
size - the size of the file
Returns:
path
Throws:
java.io.IOException

getInputFile

public org.apache.hadoop.fs.Path getInputFile(int mapId)
                                       throws java.io.IOException
Return a local reduce input file created earlier

Parameters:
mapId - a map task id
Returns:
path
Throws:
java.io.IOException

getInputFileForWrite

public org.apache.hadoop.fs.Path getInputFileForWrite(TaskID mapId,
                                                      long size)
                                               throws java.io.IOException
Create a local reduce input file name.

Parameters:
mapId - a map task id
size - the size of the file
Returns:
path
Throws:
java.io.IOException

removeAll

public void removeAll()
               throws java.io.IOException
Removes all of the files related to a task.

Throws:
java.io.IOException

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)


Copyright © 2009 The Apache Software Foundation