org.apache.hadoop.mapreduce.lib.partition
Class InputSampler<K,V>
java.lang.Object
org.apache.hadoop.conf.Configured
org.apache.hadoop.mapreduce.lib.partition.InputSampler<K,V>
- All Implemented Interfaces:
- org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool
public class InputSampler<K,V>
- extends org.apache.hadoop.conf.Configured
- implements org.apache.hadoop.util.Tool
Utility for collecting samples and writing a partition file for
TotalOrderPartitioner
.
Constructor Summary |
InputSampler(org.apache.hadoop.conf.Configuration conf)
|
Method Summary |
static void |
main(java.lang.String[] args)
|
int |
run(java.lang.String[] args)
Driver for InputSampler from the command line. |
static
|
writePartitionFile(Job job,
InputSampler.Sampler<K,V> sampler)
Write a partition file for the given job, using the Sampler provided. |
Methods inherited from class org.apache.hadoop.conf.Configured |
getConf, setConf |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Methods inherited from interface org.apache.hadoop.conf.Configurable |
getConf, setConf |
InputSampler
public InputSampler(org.apache.hadoop.conf.Configuration conf)
writePartitionFile
public static <K,V> void writePartitionFile(Job job,
InputSampler.Sampler<K,V> sampler)
throws java.io.IOException,
java.lang.ClassNotFoundException,
java.lang.InterruptedException
- Write a partition file for the given job, using the Sampler provided.
Queries the sampler for a sample keyset, sorts by the output key
comparator, selects the keys for each rank, and writes to the destination
returned from
TotalOrderPartitioner.getPartitionFile(org.apache.hadoop.conf.Configuration)
.
- Throws:
java.io.IOException
java.lang.ClassNotFoundException
java.lang.InterruptedException
run
public int run(java.lang.String[] args)
throws java.lang.Exception
- Driver for InputSampler from the command line.
Configures a JobConf instance and calls
writePartitionFile(org.apache.hadoop.mapreduce.Job, org.apache.hadoop.mapreduce.lib.partition.InputSampler.Sampler)
.
- Specified by:
run
in interface org.apache.hadoop.util.Tool
- Throws:
java.lang.Exception
main
public static void main(java.lang.String[] args)
throws java.lang.Exception
- Throws:
java.lang.Exception
Copyright © 2009 The Apache Software Foundation