org.apache.hadoop.mapreduce.lib.db
Class DBRecordReader<T extends DBWritable>

java.lang.Object
  extended by org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,T>
      extended by org.apache.hadoop.mapreduce.lib.db.DBRecordReader<T>
All Implemented Interfaces:
java.io.Closeable
Direct Known Subclasses:
DataDrivenDBRecordReader, DBInputFormat.DBRecordReader, MySQLDBRecordReader, OracleDBRecordReader

public class DBRecordReader<T extends DBWritable>
extends RecordReader<org.apache.hadoop.io.LongWritable,T>

A RecordReader that reads records from a SQL table. Emits LongWritables containing the record number as key and DBWritables as value.


Constructor Summary
DBRecordReader(DBInputFormat.DBInputSplit split, java.lang.Class<T> inputClass, org.apache.hadoop.conf.Configuration conf, java.sql.Connection conn, DBConfiguration dbConfig, java.lang.String cond, java.lang.String[] fields, java.lang.String table)
           
 
Method Summary
 void close()
          Close the record reader.
 T createValue()
          Deprecated.  
protected  java.sql.ResultSet executeQuery(java.lang.String query)
           
protected  java.lang.String getConditions()
           
protected  java.sql.Connection getConnection()
           
 org.apache.hadoop.io.LongWritable getCurrentKey()
          Get the current key
 T getCurrentValue()
          Get the current value.
protected  DBConfiguration getDBConf()
           
protected  java.lang.String[] getFieldNames()
           
 long getPos()
          Deprecated.  
 float getProgress()
          The current progress of the record reader through its data.
protected  java.lang.String getSelectQuery()
          Returns the query for selecting the records, subclasses can override this for custom behaviour.
protected  DBInputFormat.DBInputSplit getSplit()
           
protected  java.sql.PreparedStatement getStatement()
           
protected  java.lang.String getTableName()
           
 void initialize(InputSplit split, TaskAttemptContext context)
          Called once at initialization.
 boolean next(org.apache.hadoop.io.LongWritable key, T value)
          Deprecated. Use nextKeyValue()
 boolean nextKeyValue()
          Read the next key, value pair.
protected  void setStatement(java.sql.PreparedStatement stmt)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

DBRecordReader

public DBRecordReader(DBInputFormat.DBInputSplit split,
                      java.lang.Class<T> inputClass,
                      org.apache.hadoop.conf.Configuration conf,
                      java.sql.Connection conn,
                      DBConfiguration dbConfig,
                      java.lang.String cond,
                      java.lang.String[] fields,
                      java.lang.String table)
               throws java.sql.SQLException
Parameters:
split - The InputSplit to read data for
Throws:
java.sql.SQLException
Method Detail

executeQuery

protected java.sql.ResultSet executeQuery(java.lang.String query)
                                   throws java.sql.SQLException
Throws:
java.sql.SQLException

getSelectQuery

protected java.lang.String getSelectQuery()
Returns the query for selecting the records, subclasses can override this for custom behaviour.


close

public void close()
           throws java.io.IOException
Close the record reader.

Specified by:
close in interface java.io.Closeable
Specified by:
close in class RecordReader<org.apache.hadoop.io.LongWritable,T extends DBWritable>
Throws:
java.io.IOException

initialize

public void initialize(InputSplit split,
                       TaskAttemptContext context)
                throws java.io.IOException,
                       java.lang.InterruptedException
Description copied from class: RecordReader
Called once at initialization.

Specified by:
initialize in class RecordReader<org.apache.hadoop.io.LongWritable,T extends DBWritable>
Parameters:
split - the split that defines the range of records to read
context - the information about the task
Throws:
java.io.IOException
java.lang.InterruptedException

getCurrentKey

public org.apache.hadoop.io.LongWritable getCurrentKey()
Get the current key

Specified by:
getCurrentKey in class RecordReader<org.apache.hadoop.io.LongWritable,T extends DBWritable>
Returns:
the current key or null if there is no current key

getCurrentValue

public T getCurrentValue()
Get the current value.

Specified by:
getCurrentValue in class RecordReader<org.apache.hadoop.io.LongWritable,T extends DBWritable>
Returns:
the object that was read

createValue

@Deprecated
public T createValue()
Deprecated. 


getPos

@Deprecated
public long getPos()
            throws java.io.IOException
Deprecated. 

Throws:
java.io.IOException

next

@Deprecated
public boolean next(org.apache.hadoop.io.LongWritable key,
                               T value)
             throws java.io.IOException
Deprecated. Use nextKeyValue()

Throws:
java.io.IOException

getProgress

public float getProgress()
                  throws java.io.IOException
The current progress of the record reader through its data.

Specified by:
getProgress in class RecordReader<org.apache.hadoop.io.LongWritable,T extends DBWritable>
Returns:
a number between 0.0 and 1.0 that is the fraction of the data read
Throws:
java.io.IOException

nextKeyValue

public boolean nextKeyValue()
                     throws java.io.IOException
Read the next key, value pair.

Specified by:
nextKeyValue in class RecordReader<org.apache.hadoop.io.LongWritable,T extends DBWritable>
Returns:
true if a key/value pair was read
Throws:
java.io.IOException

getSplit

protected DBInputFormat.DBInputSplit getSplit()

getFieldNames

protected java.lang.String[] getFieldNames()

getTableName

protected java.lang.String getTableName()

getConditions

protected java.lang.String getConditions()

getDBConf

protected DBConfiguration getDBConf()

getConnection

protected java.sql.Connection getConnection()

getStatement

protected java.sql.PreparedStatement getStatement()

setStatement

protected void setStatement(java.sql.PreparedStatement stmt)


Copyright © 2009 The Apache Software Foundation