You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When subscribing to a topic the KafkaConsumerActor adds a rebalancing hook that registers FetchRequests.
Then when calling stream the KafkaConsumer will requests partition assignments and - that's the race condition - depending on the timing might get some partition ids for which it will also register FetchRequests.
This is aggravated by the fact that the KafkaConsumerActor.State does not restrict the amount of FetchRequests per (partition, consumer), leading to duplicated records sent downstream.
The text was updated successfully, but these errors were encountered:
When subscribing to a topic the
KafkaConsumerActor
adds a rebalancing hook that registersFetchRequest
s.Then when calling
stream
theKafkaConsumer
will requests partition assignments and - that's the race condition - depending on the timing might get some partition ids for which it will also registerFetchRequest
s.This is aggravated by the fact that the
KafkaConsumerActor.State
does not restrict the amount ofFetchRequest
s per (partition, consumer), leading to duplicated records sent downstream.The text was updated successfully, but these errors were encountered: