How to handle a closed message channel?

[4578395263256]

When in the consume loop from a PartitionConsumer, in some cases the message channel gets closed (this usually happens when killing brokers). In this situation, would it be safe to call ConsumePartition again or do you need to wait for something? Also it looks like it should be safe to call AsyncClose, but is it necessary to do so?

[eapache]

The only time a PartitionConsumer should shut itself down is when the offset it is consuming is no longer in range. I'm not sure why this would happen when killing brokers... that seems odd.

Regardless, it should be safe to re-consume that partition, and it is probably not safe (or necessary) to call AsyncClose again

[4578395263256]

Thanks for the information. FWIW our java consumers are getting OffsetOutOfRange. To clarify what we are doing, we are killing the whole cluster (first SIGTERM, then after a few min SIGKILL) and then bringing it back up after a bit, to see if problems occur.

[eapache]

FWIW our java consumers are getting OffsetOutOfRange

Then that is almost certainly what Sarama is getting as well. I'm not sure why that's happening, but that's a question for the upstream Kafka folks.

In this case you do not need to (and should not) call AsyncClose, you can simply re-consume that partition and let the old PartitionConsumer pass out of scope.