Producer: AttributeError - 'tuple' object has no attribute 'size'

[petee-d]

I've had an AttributeError - 'tuple' object has no attribute 'size' raised from producer.send in method ProducerBatch.try_append. I won't post a stack trace as I already located the primary issue:

https://github.com/dpkp/kafka-python/blob/master/kafka/record/memory_records.py#L131

MemoryRecordsBuilder.append returns None, 0 in case self._closed, but the code using this function expects the return value to be either None or some metadata object. Also the documentation of the method falsely claims the return type is (int, int), it all probably stems in unfinished refactoring.

Furthermore, I'm scared what will happen when that bug is fixed, because if it properly returns None and it gets to the following code, it will raise Exception(), which just sounds incredibly wrong, I don't think it's handled by the library.
https://github.com/dpkp/kafka-python/blob/master/kafka/producer/record_accumulator.py#L266

I'm afraid I don't have the time to make a PR with tests and also probably wouldn't have proper understanding of what I'm doing anyway.

[tvoinarovskyi]

Yup, my bad (

[tvoinarovskyi]

Furthermore, I'm scared what will happen when that bug is fixed, because if it properly returns None and it gets to the following code, it will raise Exception(), which just sounds incredibly wrong, I don't think it's handled by the library.
https://github.com/dpkp/kafka-python/blob/master/kafka/producer/record_accumulator.py#L266

Don't worry, that code path is an assertion for threading issues. You will hit the try_append above.

[petee-d]

Thank you, good to know. Btw, I probably don't need to tell you, but an assert with a message would be nicer than raise Exception(). :)

Any estimate for release of a fix? For an estimate of severity, this caused hundreds of crashes of producers for our project in the last 9 days since we've upgraded, out of a few billions of produced messages. So it happens quite rarely, but Kafka producing is a very sensitive area...

[dpkp]

Fixed via #1448