public static class KafkaSpoutConfig.Builder<K,V> extends Object
Constructor and Description |
---|
Builder(Map<String,Object> kafkaProps,
KafkaSpoutStreams kafkaSpoutStreams,
KafkaSpoutTuplesBuilder<K,V> tuplesBuilder)
Please refer to javadoc in
#Builder(Map, KafkaSpoutStreams, KafkaSpoutTuplesBuilder, KafkaSpoutRetryService) . This constructor uses by the default the following implementation for KafkaSpoutRetryService : new KafkaSpoutRetryExponentialBackoff(TimeInterval.seconds(0), TimeInterval.milliSeconds(2),
DEFAULT_MAX_RETRIES, TimeInterval.seconds(10))) |
Builder(Map<String,Object> kafkaProps,
KafkaSpoutStreams kafkaSpoutStreams,
KafkaSpoutTuplesBuilder<K,V> tuplesBuilder,
KafkaSpoutRetryService retryService)
KafkaSpoutConfig defines the required configuration to connect a consumer to a consumer group, as well as the subscribing topics The optional configuration can be specified using the set methods of this builder
|
Modifier and Type | Method and Description |
---|---|
KafkaSpoutConfig<K,V> |
build() |
KafkaSpoutConfig.Builder<K,V> |
setFirstPollOffsetStrategy(KafkaSpoutConfig.FirstPollOffsetStrategy firstPollOffsetStrategy)
Sets the offset used by the Kafka spout in the first poll to Kafka broker upon process start.
|
KafkaSpoutConfig.Builder<K,V> |
setKeyDeserializer(org.apache.kafka.common.serialization.Deserializer<K> keyDeserializer)
Specifying this key deserializer overrides the property key.deserializer
|
KafkaSpoutConfig.Builder<K,V> |
setMaxRetries(int maxRetries)
Defines the max number of retrials in case of tuple failure.
|
KafkaSpoutConfig.Builder<K,V> |
setMaxUncommittedOffsets(int maxUncommittedOffsets)
Defines the max number of polled offsets (records) that can be pending commit, before another poll can take place.
|
KafkaSpoutConfig.Builder<K,V> |
setOffsetCommitPeriodMs(long offsetCommitPeriodMs)
Specifies the period, in milliseconds, the offset commit task is periodically called.
|
KafkaSpoutConfig.Builder<K,V> |
setPollTimeoutMs(long pollTimeoutMs)
Specifies the time, in milliseconds, spent waiting in poll if data is not available.
|
KafkaSpoutConfig.Builder<K,V> |
setValueDeserializer(org.apache.kafka.common.serialization.Deserializer<V> valueDeserializer)
Specifying this value deserializer overrides the property value.deserializer
|
public Builder(Map<String,Object> kafkaProps, KafkaSpoutStreams kafkaSpoutStreams, KafkaSpoutTuplesBuilder<K,V> tuplesBuilder)
Please refer to javadoc in #Builder(Map, KafkaSpoutStreams, KafkaSpoutTuplesBuilder, KafkaSpoutRetryService)
.
KafkaSpoutRetryService
: new KafkaSpoutRetryExponentialBackoff(TimeInterval.seconds(0), TimeInterval.milliSeconds(2),
DEFAULT_MAX_RETRIES, TimeInterval.seconds(10)))
public Builder(Map<String,Object> kafkaProps, KafkaSpoutStreams kafkaSpoutStreams, KafkaSpoutTuplesBuilder<K,V> tuplesBuilder, KafkaSpoutRetryService retryService)
KafkaSpoutConfig defines the required configuration to connect a consumer to a consumer group, as well as the subscribing topics The optional configuration can be specified using the set methods of this builder
kafkaProps
- properties defining consumer connection to Kafka broker as specified in @see KafkaConsumerkafkaSpoutStreams
- streams to where the tuples are emitted for each tuple. Multiple topics can emit in the same stream.tuplesBuilder
- logic to build tuples from ConsumerRecord
s.retryService
- logic that manages the retrial of failed tuplespublic KafkaSpoutConfig.Builder<K,V> setKeyDeserializer(org.apache.kafka.common.serialization.Deserializer<K> keyDeserializer)
Specifying this key deserializer overrides the property key.deserializer
public KafkaSpoutConfig.Builder<K,V> setValueDeserializer(org.apache.kafka.common.serialization.Deserializer<V> valueDeserializer)
Specifying this value deserializer overrides the property value.deserializer
public KafkaSpoutConfig.Builder<K,V> setPollTimeoutMs(long pollTimeoutMs)
Specifies the time, in milliseconds, spent waiting in poll if data is not available. Default is 2s
pollTimeoutMs
- time in mspublic KafkaSpoutConfig.Builder<K,V> setOffsetCommitPeriodMs(long offsetCommitPeriodMs)
Specifies the period, in milliseconds, the offset commit task is periodically called. Default is 15s.
offsetCommitPeriodMs
- time in mspublic KafkaSpoutConfig.Builder<K,V> setMaxRetries(int maxRetries)
Defines the max number of retrials in case of tuple failure. The default is to retry forever, which means that no new records are committed until the previous polled records have been acked. This guarantees at once delivery of all the previously polled records. By specifying a finite value for maxRetries, the user decides to sacrifice guarantee of delivery for the previous polled records in favor of processing more records.
maxRetries
- max number of retrialspublic KafkaSpoutConfig.Builder<K,V> setMaxUncommittedOffsets(int maxUncommittedOffsets)
Defines the max number of polled offsets (records) that can be pending commit, before another poll can take place. Once this limit is reached, no more offsets (records) can be polled until the next successful commit(s) sets the number of pending offsets bellow the threshold. The default is KafkaSpoutConfig.DEFAULT_MAX_UNCOMMITTED_OFFSETS
.
maxUncommittedOffsets
- max number of records that can be be pending commitpublic KafkaSpoutConfig.Builder<K,V> setFirstPollOffsetStrategy(KafkaSpoutConfig.FirstPollOffsetStrategy firstPollOffsetStrategy)
Sets the offset used by the Kafka spout in the first poll to Kafka broker upon process start. Please refer to to the documentation in KafkaSpoutConfig.FirstPollOffsetStrategy
firstPollOffsetStrategy
- Offset used by Kafka spout first pollpublic KafkaSpoutConfig<K,V> build()
Copyright © 2016 The Apache Software Foundation. All Rights Reserved.