Force lz4 to disable Kafka-unsupported block linking when encoding

[mnito]

Kafka does not support LZ4 compression with dependent blocks

https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/record/KafkaLZ4BlockOutputStream.java#L343

This issue was first noticed when encoding larger-than-usual messages in our application.

This change appears to solve the problem by enforcing compression with independent blocks in the underlying LZ4 package.

[dpkp]

It looks like block_linked was added as a kwarg in Dec 2017 (v0.12.0). That means that anyone using an earlier release of python-lz4 will get a TypeError (TypeError: 'block_linked' is an invalid keyword argument for this function). In earlier versions the options was block_mode=1How about adding a try / except and falling back toblock_mode=1` on a TypeError?

[mnito]

You got it. Thanks for the heads up.

[dpkp]

Thanks!