public class HdfsBolt extends AbstractHdfsBolt
collector, configKey, fileNameFormat, fileRetryCount, fs, fsUrl, hdfsConfig, offset, rotation, rotationActions, rotationPolicy, rotationTimer, syncPolicy, tickTupleInterval, writeLock
Constructor and Description |
---|
HdfsBolt() |
Modifier and Type | Method and Description |
---|---|
HdfsBolt |
addRotationAction(RotationAction action) |
protected void |
closeOutputFile() |
protected org.apache.hadoop.fs.Path |
createOutputFile() |
void |
doPrepare(Map conf,
TopologyContext topologyContext,
OutputCollector collector) |
protected void |
syncTuples()
Make the best effort to sync written data to the underlying file system.
|
HdfsBolt |
withConfigKey(String configKey) |
HdfsBolt |
withFileNameFormat(FileNameFormat fileNameFormat) |
HdfsBolt |
withFsUrl(String fsUrl) |
HdfsBolt |
withRecordFormat(RecordFormat format) |
HdfsBolt |
withRetryCount(int fileRetryCount) |
HdfsBolt |
withRotationPolicy(FileRotationPolicy rotationPolicy) |
HdfsBolt |
withSyncPolicy(SyncPolicy syncPolicy) |
HdfsBolt |
withTickTupleIntervalSeconds(int interval) |
protected void |
writeTuple(Tuple tuple)
writes a tuple to the underlying filesystem but makes no guarantees about syncing data.
|
declareOutputFields, execute, getComponentConfiguration, prepare, rotateOutputFile
cleanup
public HdfsBolt withFileNameFormat(FileNameFormat fileNameFormat)
public HdfsBolt withRecordFormat(RecordFormat format)
public HdfsBolt withSyncPolicy(SyncPolicy syncPolicy)
public HdfsBolt withRotationPolicy(FileRotationPolicy rotationPolicy)
public HdfsBolt addRotationAction(RotationAction action)
public HdfsBolt withTickTupleIntervalSeconds(int interval)
public HdfsBolt withRetryCount(int fileRetryCount)
public void doPrepare(Map conf, TopologyContext topologyContext, OutputCollector collector) throws IOException
doPrepare
in class AbstractHdfsBolt
IOException
protected void syncTuples() throws IOException
AbstractHdfsBolt
Make the best effort to sync written data to the underlying file system. Concrete classes should very clearly state the file state that sync guarantees. For example, HdfsBolt can make a much stronger guarantee than SequenceFileBolt.
syncTuples
in class AbstractHdfsBolt
IOException
protected void writeTuple(Tuple tuple) throws IOException
AbstractHdfsBolt
writes a tuple to the underlying filesystem but makes no guarantees about syncing data.
this.offset is also updated to reflect additional data written
writeTuple
in class AbstractHdfsBolt
IOException
protected void closeOutputFile() throws IOException
closeOutputFile
in class AbstractHdfsBolt
IOException
protected org.apache.hadoop.fs.Path createOutputFile() throws IOException
createOutputFile
in class AbstractHdfsBolt
IOException
Copyright © 2016 The Apache Software Foundation. All Rights Reserved.