|
||||||||||
PREV NEXT | FRAMES NO FRAMES |
Path
to the list of inputs for the map-reduce job.
Path
with a custom InputFormat
to the list of
inputs for the map-reduce job.
Path
with a custom InputFormat
and
Mapper
to the list of inputs for the map-reduce job.
Path
to the list of inputs for the map-reduce job.
Path
with a custom InputFormat
to the list of
inputs for the map-reduce job.
Path
with a custom InputFormat
and
Mapper
to the list of inputs for the map-reduce job.
UserLogEvent
for processing.
InputFormat
that tries to deduce the types of the input files
automatically.BinaryComparable
keys using a configurable part of
the bytes array returned by BinaryComparable.getBytes()
.BinaryComparable
keys using a configurable part of
the bytes array returned by BinaryComparable.getBytes()
.DistributedCache.CACHE_ARCHIVES
is not a *public* constant.
DistributedCache.CACHE_ARCHIVES_SIZES
is not a *public* constant.
DistributedCache.CACHE_ARCHIVES_TIMESTAMPS
is not a *public* constant.
DistributedCache.CACHE_FILES
is not a *public* constant.
DistributedCache.CACHE_FILES_SIZES
is not a *public* constant.
DistributedCache.CACHE_FILES_TIMESTAMPS
is not a *public* constant.
DistributedCache.CACHE_LOCALARCHIVES
is not a *public* constant.
DistributedCache.CACHE_LOCALFILES
is not a *public* constant.
DistributedCache.CACHE_SYMLINK
is not a *public* constant.
TaskTracker
is declared as 'lost/blacklisted'
by the JobTracker.
OutputCommitter.commitJob(JobContext)
or
OutputCommitter.abortJob(JobContext, int)
instead
OutputCommitter.commitJob(JobContext)
or
OutputCommitter.abortJob(JobContext, JobStatus.State)
instead
JobContext
or TaskAttemptContext
with a
new configuration.
JobClient
.
RecordWriter
to future operations.
InputSplit
to future operations.
RecordWriter
to future operations.
RecordWriter
to future operations.
RecordWriter
to future operations.
MultiFilterRecordReader.emit(org.apache.hadoop.mapred.join.TupleWritable)
every Tuple from the
collector (the outer join of child RRs).
InputFormat
that returns CombineFileSplit
's
in InputFormat.getSplits(JobConf, int)
method.InputFormat
that returns CombineFileSplit
's in
InputFormat.getSplits(JobContext)
method.CombineFileSplit
.CombineFileSplit
.JobConf
.
JobConf
.
JobConf
.
Configuration
.
Counter
s that logically belong together.Counters
instead.Counters
holds per job/task counters, defined either by the
Map-Reduce framework or applications.Group
of counters, comprising of counters from a particular
counter Enum
class.DBWritable
.InputFormat
that delegates behaviour of paths to multiple other
InputFormats.InputFormat
that delegates behavior of paths to multiple other
InputFormats.Mapper
that delegates behaviour of paths to multiple other
mappers.Mapper
that delegates behavior of paths to multiple other
mappers.TaggedInputSplit
UserLogEvent
sent when job logs should be deleted.InputFormat
.InputFormat
s.OutputCommitter
that commits files specified
in job output directory i.e.OutputCommitter
that commits files specified
in job output directory i.e.OutputFormat
.OutputFormat
s that read from FileSystem
s.FilterRecordWriter
is a convenience wrapper
class that extends the RecordWriter
.UTF8ByteArrayUtils.findByte(byte[], int,
int, byte)
UTF8ByteArrayUtils.findBytes(byte[], int,
int, byte[])
Counters.findCounter(String, String)
instead
UTF8ByteArrayUtils.findNthByte(byte[], int,
int, byte, int)
UTF8ByteArrayUtils.findNthByte(byte[],
byte, int)
StreamKeyValUtil.findTab(byte[], int, int)
StreamKeyValUtil.findTab(byte[])
Counters.makeEscapedCompactString()
counter
representation into a counter object.
DataInput
.
DataOutput
.
TypedBytesInput
.
DataInput
.
TypedBytesOutput
.
DataOutput
.
TypedBytesInput
.
DataInput
.
TypedBytesOutput
.
DataOutput
.
SkipBadRecords.COUNTER_MAP_PROCESSED_RECORDS
is incremented
by MapRunner after invoking the map function.
SkipBadRecords.COUNTER_REDUCE_PROCESSED_GROUPS
is incremented
by framework after invoking the reduce function.
JobTracker.getClusterStatus(boolean)
WritableComparable
comparator for
grouping keys of inputs to the combiner.
RawComparator
comparator for
grouping keys of inputs to the combiner.
RawComparator
comparator for
grouping keys of inputs to the combiner.
Counters.Group.findCounter(String)
instead
Counters.Counter
of the given group with the given name.
Counters.Counter
of the given group with the given name.
Counter
for the given counterName
.
Counter
for the given groupName
and
counterName
.
UserLogEvent.EventType
.
FileStatus
of a given cache file on hdfs.
RawComparator
comparator for
grouping keys of inputs to the reduce.
RawComparator
comparator for
grouping keys of inputs to the reduce.
InputFormat
implementation for the map-reduce job,
defaults to TextInputFormat
if not specified explicity.
InputFormat
class for the job.
InputFormat
class for the job.
Path
s for the map-reduce job.
Path
s for the map-reduce job.
InputSplit
object for a map.
InputWriter
class.
Job
A Job will be created with a generic Configuration
.
Job
with a given Configuration
.
Job
with a given Configuration
and a given jobName.
RunningJob
object to track an ongoing job.
JobClient.getJob(JobID)
.
JobInProgress
for which the fallow slot(s) are held.
RunningJob.getID()
.
JobID
object that this task attempt belongs to
JobID
object that this tip belongs to
JobPriority
for this job.
JobStatus
, of the Job.
addressKey
with the the address of the local node.
SequenceFileRecordReader.next(Object, Object)
..
KeyFieldBasedComparator
options
KeyFieldBasedComparator
options
KeyFieldBasedPartitioner
options
KeyFieldBasedPartitioner
options
InputSplit
.
WrappedMapper.Context
for custom implementations.
CompressionCodec
for compressing the map outputs.
Mapper
class for the job.
Mapper
class for the job.
Mapper
class for the job.
MapRunnable
class for the job.
true
.
JobClient.getMapTaskReports(JobID)
mapred.map.max.attempts
property.
mapred.map.max.attempts
property.
mapred.reduce.max.attempts
property.
mapred.reduce.max.attempts
property.
TaskStatus.getMaxStringSize()
to control the max-size
of strings in TaskStatus
.
JobConf.getMemoryForMapTask()
and
JobConf.getMemoryForReduceTask()
OutputCommitter
implementation for the map-reduce job,
defaults to FileOutputCommitter
if not specified explicitly.
OutputCommitter
for the task-attempt.
SequenceFile.CompressionType
for the output SequenceFile
.
SequenceFile.CompressionType
for the output SequenceFile
.
CompressionCodec
for compressing the job outputs.
CompressionCodec
for compressing the job outputs.
OutputFormat
implementation for the map-reduce job,
defaults to TextOutputFormat
if not specified explicity.
OutputFormat
class for the job.
OutputFormat
class for the job.
RawComparator
comparator used to compare keys.
Path
to the output directory for the map-reduce job.
Path
to the output directory for the map-reduce job.
OutputReader
class.
WritableComparable
comparator for
grouping keys of inputs to the reduce.
Object.hashCode()
to partition.
BinaryComparable.getBytes()
to partition.
Object.hashCode()
to partition.
Partitioner
used to partition Mapper
-outputs
to be sent to the Reducer
s.
Partitioner
class for the job.
Partitioner
class for the job.
Path
for a file that is unique for
the task within the job output directory.
Path
for a file that is unique for
the task within the job output directory.
RecordReader
consumed i.e.
GetUserMappingsProtocol
implementation is running.
QueueManager
associated with the JobTracker.
RecordReader
for the given InputSplit
.
RecordReader
for the given InputSplit
.
RecordWriter
for the given job.
RecordWriter
for the given job.
RecordWriter
for the given task.
RecordWriter
for the given task.
Reducer
class for the job.
Reducer
class for the job.
Reducer
class for the job.
WrappedReducer.Context
for custom implementations.
true
.
JobClient.getReduceTaskReports(JobID)
TaskTrackerStatus.ResourceStatus
object configured with this
status.
SequenceFile
SequenceFile
SequenceFile
SequenceFile
RawComparator
comparator used to compare keys.
RawComparator
comparator used to compare keys.
true
.
FileInputFormat.listStatus(JobConf)
when
they're too big.
Reporter
.
TaskTrackerStatus
of the TaskTracker
.
TaskCompletionEvent.getTaskAttemptId()
instead.
TaskReport.getTaskID()
instead
TaskID
object that this task attempt belongs to
TaskTracker
GetUserMappingsProtocol
.
UserLogCleaner
.
SequenceFileRecordReader.next(Object, Object)
..
Path
to the task's temporary output directory
for the map-reduce job
Path
to the task's temporary output directory
for the map-reduce job
StreamJob.run(String[])
instead.
Object.hashCode()
.Object.hashCode()
.TaskTracker
and
the JobTracker
.
Enum
type, by the specified amount.
JobTracker
.
InputFormat
describes the input-specification for a
Map-Reduce job.InputFormat
describes the input-specification for a
Map-Reduce job.TotalOrderPartitioner
.TotalOrderPartitioner
.InputFormat
.InputFormat
.InputSplit
represents the data to be processed by an
individual Mapper
.InputSplit
represents the data to be processed by an
individual Mapper
.Mapper
that swaps keys and values.Mapper
that swaps keys and values.ControlledJob
instead.JobClient
is the primary interface for the user-job to interact
with the JobTracker
.JobConf
, and connect to the
default JobTracker
.
Configuration
,
and connect to the default cluster
UserLogEvent
sent when the job completesJobContext
instead.JobControl
insteadJobProfile
.
JobProfile
the userid, jobid,
job config-file, job-details url and job name.
JobProfile
the userid, jobid,
job config-file, job-details url and job name.
UserLogEvent
sent when the job starts.UserLogEvent
sent when the jvm finishes.KeyFieldBasedComparator
.KeyFieldBasedComparator
.InputFormat
for plain text files.InputFormat
for plain text files.UnsupportedOperationException
RunningJob.killTask(TaskAttemptID, boolean)
LineReader
instead.JobHistory.MapAttempt.logFailed(TaskAttemptID, long, String, String, String)
JobHistory.ReduceAttempt.logFailed(TaskAttemptID, long, String, String, String)
JobHistory.MapAttempt.logFinished(TaskAttemptID, long, String, String, String, Counters)
JobHistory.ReduceAttempt.logFinished(TaskAttemptID, long, long, long, String, String, String, Counters)
JobHistory.JobInfo.logJobInfo(JobID, long, long)
instead.
JobHistory.MapAttempt.logKilled(TaskAttemptID, long, String, String, String)
JobHistory.ReduceAttempt.logKilled(TaskAttemptID, long, String, String, String)
JobHistory.JobInfo.logInited(JobID, long, int, int)
and
JobHistory.JobInfo.logStarted(JobID)
JobHistory.MapAttempt.logStarted(TaskAttemptID, long, String, int, String)
JobHistory.ReduceAttempt.logStarted(TaskAttemptID, long, String, int, String)
JobHistory.JobInfo.logSubmitted(JobID, JobConf, String, long, boolean)
instead.
Reducer
that sums long values.LinuxMemoryCalculatorPlugin
LinuxResourceCalculatorPlugin
map(...)
methods of the Mappers in the chain.
Mapper
.Mapper
.OutputFormat
that writes MapFile
s.OutputFormat
that writes
MapFile
s.Context
passed on to the Mapper
implementations.Level
for the map task.
Level
for the reduce task.
JobConf.MAPRED_MAP_TASK_ENV
or
JobConf.MAPRED_REDUCE_TASK_ENV
JobConf.MAPRED_MAP_TASK_JAVA_OPTS
or
JobConf.MAPRED_REDUCE_TASK_JAVA_OPTS
JobConf.MAPRED_JOB_MAP_MEMORY_MB_PROPERTY
and
JobConf.MAPRED_JOB_REDUCE_MEMORY_MB_PROPERTY
JobConf.MAPRED_MAP_TASK_ULIMIT
or
JobConf.MAPRED_REDUCE_TASK_ULIMIT
Mapper
and Reducer
implementations.PolicyProvider
for Map-Reduce protocols.Mapper
s.MapRunnable
implementation.CombineFileInputFormat
insteadCombineFileSplit
insteadMultiFileWordCount.MapClass
.CombineFileInputFormat
, one should extend it, to return a
(custom) RecordReader
.InputFormat
and Mapper
for each pathInputFormat
and Mapper
for each pathOutputCollector
passed to
the map()
and reduce()
methods of the
Mapper
and Reducer
implementations.DBRecordReader.nextKeyValue()
<key, value>
pairs output by Mapper
s
and Reducer
s.OutputCommitter
describes the commit of task output for a
Map-Reduce job.OutputCommitter
describes the commit of task output for a
Map-Reduce job.OutputFormat
describes the output-specification for a
Map-Reduce job.OutputFormat
describes the output-specification for a
Map-Reduce job.Utils.OutputFileUtils.OutputLogFilter
instead.RawKeyValueIterator
is an iterator used to iterate over
the raw keys and values during sort/merge of intermediate data.Type.BOOL
code.
Type.BYTE
code.
Type.BYTES
code.
Type.DOUBLE
code.
ResultSet
.
Type.FLOAT
code.
Type.INT
code.
StreamKeyValUtil.readLine(LineReader, Text)
Type.LIST
code.
Type.LONG
code.
Type.MAP
code.
Type.MAP
code.
Type.BOOL
code.
Type.BYTE
code.
Type.BYTES
code.
Type.DOUBLE
code.
Type.FLOAT
code.
Type.INT
code.
Type.LIST
code.
Type.LONG
code.
Type.MAP
code.
Type.STRING
code.
Type.VECTOR
code.
Type.STRING
code.
Type
.
Type.VECTOR
code.
Type.VECTOR
code.
RecordReader
reads <key, value> pairs from an
InputSplit
.Mapper
.RecordWriter
writes the output <key, value> pairs
to an output file.RecordWriter
writes the output <key, value> pairs
to an output file.reduce(...)
method of the Reducer with the
map(...)
methods of the Mappers in the chain.
Reducer
.Iterator
to iterate over values for a given group of records.Reducer
.Context
passed on to the Reducer
implementations.JobTracker
JobTracker
.
Mapper
that extracts text matching a regular expression.Mapper
that extracts text matching a regular expression.TrackerDistributedCacheManager
.
job
.
Reducer.run(org.apache.hadoop.mapreduce.Reducer.Context)
method to
control how the reduce task works.
PrintStream
configured earlier.
DumpTypedBytes
.
LoadTypedBytes
.
RunningJob
is the user-interface to query for details on a
running Map-Reduce job.OutputFormat
that writes keys, values to
SequenceFile
s in binary(raw) formatOutputFormat
that writes keys,
values to SequenceFile
s in binary(raw) formatInputFormat
for SequenceFile
s.InputFormat
for SequenceFile
s.OutputFormat
that writes SequenceFile
s.OutputFormat
that writes SequenceFile
s.RecordReader
for SequenceFile
s.RecordReader
for SequenceFile
s.SkipBadRecords.COUNTER_MAP_PROCESSED_RECORDS
is incremented
by MapRunner after invoking the map function.
SkipBadRecords.COUNTER_REDUCE_PROCESSED_GROUPS
is incremented
by framework after invoking the reduce function.
RawComparator
comparator for
grouping keys in the input to the combiner.
Reducer.reduce(Object, Iterable,
org.apache.hadoop.mapreduce.Reducer.Context)
Reducer.reduce(Object, Iterable,
org.apache.hadoop.mapreduce.Reducer.Context)
InputFormat
implementation for the map-reduce job.
InputFormat
for the job.
Path
s as the list of inputs
for the map-reduce job.
Path
s as the list of inputs
for the map-reduce job.
InputWriter
class.
JobPriority
for this job.
KeyFieldBasedComparator
options used to compare keys.
KeyFieldBasedComparator
options used to compare keys.
KeyFieldBasedPartitioner
options used for
Partitioner
KeyFieldBasedPartitioner
options used for
Partitioner
bytes[offset:]
in Python syntax.
CompressionCodec
for the map outputs.
Mapper
class for the job.
Mapper
for the job.
MapRunnable
class for the job.
JobConf.setMemoryForMapTask(long mem)
and
Use JobConf.setMemoryForReduceTask(long mem)
bytes[left:(right+1)]
in Python syntax.
OutputCommitter
implementation for the map-reduce job.
SequenceFile.CompressionType
for the output SequenceFile
.
SequenceFile.CompressionType
for the output SequenceFile
.
CompressionCodec
to be used to compress job outputs.
CompressionCodec
to be used to compress job outputs.
OutputFormat
implementation for the map-reduce job.
OutputFormat
for the job.
RawComparator
comparator used to compare keys.
Path
of the output directory for the map-reduce job.
Path
of the output directory for the map-reduce job.
OutputReader
class.
RawComparator
comparator for
grouping keys in the input to the reduce.
Partitioner
class used to partition
Mapper
-outputs to be sent to the Reducer
s.
Partitioner
for the job.
Reducer
class for the job.
Reducer
for the job.
bytes[:(offset+1)]
in Python syntax.
SequenceFile
SequenceFile
SequenceFile
SequenceFile
ProcfsBasedProcessTree.ProcfsBasedProcessTree(
String, boolean, long)
instead
Reducer
.
TaskStatus
.
TaskTrackerStatus
of the TaskTracker
.
TaskCompletionEvent.setTaskID(TaskAttemptID)
instead.
AbstractCounters.countCounters()
instead
StreamKeyValUtil.splitKeyVal(byte[], int, int, Text, Text,
int, int)
StreamKeyValUtil.splitKeyVal(byte[], int, int, Text, Text, int)
StreamKeyValUtil.splitKeyVal(byte[], Text, Text, int, int)
StreamKeyValUtil.splitKeyVal(byte[], Text, Text, int)
StreamJob.setConf(Configuration)
and
run with StreamJob.run(String[])
.
Submitter.runJob(JobConf)
TaskAttemptContext
instead.TaskAttemptContextImpl
instead.TaskID
.
TaskID
.
TrackerDistributedCacheManager
that represents
the cached files of a single job.JobID
.
JobID
.
TaskTracker
as seen by
the JobTracker
.TaskTracker
.
InputFormat
for plain text files.InputFormat
for plain text files.OutputFormat
that writes plain text files.OutputFormat
that writes plain text files.TaskInProgress
as seen by the JobTracker.Mapper
that maps text values into Writable
s.TaskTracker
which were reserved for
taskType
.
JobTracker
is locked on entry.
TaskTracker
to the
UserLogManager
to inform about an event.TaskTracker
.TaskTracker
.
TaskTracker
.
UTF8ByteArrayUtils
and
StreamKeyValUtil
insteadMapper
which wraps a given one to allow custom
WrappedMapper.Context
implementations.Reducer
which wraps a given one to allow for custom
WrappedReducer.Context
implementations.out
.
PreparedStatement
.
|
||||||||||
PREV NEXT | FRAMES NO FRAMES |