|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.apache.hadoop.mapred.lib.CombineFileSplit
public class CombineFileSplit
A sub-collection of input files. Unlike FileSplit
,
CombineFileSplit * class does not represent a split of a file, but a split of input files
into smaller sets. A split may contain blocks from different file but all
the blocks in the same split are probably local to some rack
CombineFileSplit can be used to implement RecordReader
's,
with reading one record per file.
FileSplit
,
CombineFileInputFormat
Constructor Summary | |
---|---|
CombineFileSplit()
default constructor |
|
CombineFileSplit(CombineFileSplit old)
Copy constructor |
|
CombineFileSplit(JobConf job,
org.apache.hadoop.fs.Path[] files,
long[] lengths)
|
|
CombineFileSplit(JobConf job,
org.apache.hadoop.fs.Path[] files,
long[] start,
long[] lengths,
java.lang.String[] locations)
|
Method Summary | |
---|---|
JobConf |
getJob()
|
long |
getLength()
Get the total number of bytes in the data of the InputSplit . |
long |
getLength(int i)
Returns the length of the ith Path |
long[] |
getLengths()
Returns an array containing the lengths of the files in the split |
java.lang.String[] |
getLocations()
Returns all the Paths where this input-split resides |
int |
getNumPaths()
Returns the number of Paths in the split |
long |
getOffset(int i)
Returns the start offset of the ith Path |
org.apache.hadoop.fs.Path |
getPath(int i)
Returns the ith Path |
org.apache.hadoop.fs.Path[] |
getPaths()
Returns all the Paths in the split |
long[] |
getStartOffsets()
Returns an array containing the startoffsets of the files in the split |
void |
readFields(java.io.DataInput in)
|
java.lang.String |
toString()
|
void |
write(java.io.DataOutput out)
|
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait |
Constructor Detail |
---|
public CombineFileSplit()
public CombineFileSplit(JobConf job, org.apache.hadoop.fs.Path[] files, long[] start, long[] lengths, java.lang.String[] locations)
public CombineFileSplit(JobConf job, org.apache.hadoop.fs.Path[] files, long[] lengths)
public CombineFileSplit(CombineFileSplit old) throws java.io.IOException
java.io.IOException
Method Detail |
---|
public JobConf getJob()
public long getLength()
InputSplit
InputSplit
.
getLength
in interface InputSplit
public long[] getStartOffsets()
public long[] getLengths()
public long getOffset(int i)
public long getLength(int i)
public int getNumPaths()
public org.apache.hadoop.fs.Path getPath(int i)
public org.apache.hadoop.fs.Path[] getPaths()
public java.lang.String[] getLocations() throws java.io.IOException
getLocations
in interface InputSplit
InputSplit
is
located as an array of String
s.
java.io.IOException
public void readFields(java.io.DataInput in) throws java.io.IOException
readFields
in interface org.apache.hadoop.io.Writable
java.io.IOException
public void write(java.io.DataOutput out) throws java.io.IOException
write
in interface org.apache.hadoop.io.Writable
java.io.IOException
public java.lang.String toString()
toString
in class java.lang.Object
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |