Failed to decompress gzip

[winbatch]

I'm setting gzip in my publisher and am receiving this in rdkafka_example:

1389132516.572 RDKAFKA-7-GZIP: rdkafka#consumer-0: d145931-002.masked.com:5757/0: Failed to decompress Gzip message at offset 3145 of 216 bytes: ignoring message

I get this whether -z gzip is set or not. Any idea?

[winbatch]

Us there a minimum size required for compression to work?

[edenhill]

Do you know what produced this message? Scala Kafka producer? rdkafka producer?

I dont think gzip has any limits on the size of the uncompressed data.

[winbatch]

My app using rdkafka. When I do no compression it works fine, when I do gzip it doesnot complain on the publisher side (but does in the consumer), and when I do snappy I get a message timed out error.

[winbatch]

Reproduced this using rdkafka example on both sides

[edenhill]

Perfect! Can you provide the exact command lines for both producer and consumer as well as your sample message(s)?

[winbatch]

Note I have not updated from master since Dec 28 in case it matters.

My message was on stdin and was simple like 'hi'.

./rdkafka_example -C -b kafkadevcluster1-1.masked.com:5757,
kafkadevcluster1-2.masked.com:5757,kafkadevcluster1-3.masked.com:5757 -p 8
-t LaraReplicator_kafkacluster4 -o -1

./rdkafka_example -P -b kafkadevcluster1-1.masked.com:5757,
kafkadevcluster1-2.masked.com:5757,kafkadevcluster1-3.masked.com:5757 -p 8
-t LaraReplicator_kafkacluster4

With either –z gzip, -z snappy or (nothing)

On Tuesday, January 7, 2014, Magnus Edenhill wrote:

Perfect! Can you provide the exact command lines for both producer and
consumer as well as your sample message(s)?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/54#issuecomment-31787010
.

[edenhill]

Gzip decompression should now be working.
I will look more into Snappy tomorrow.

[winbatch]

Ok cool. The snappy seemed to be a publisher side problem since it
received a local message timeout

On Tuesday, January 7, 2014, Magnus Edenhill wrote:

Gzip decompression should now be working.
I will look more into Snappy tomorrow.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/54#issuecomment-31792333
.

[winbatch]

Magnus- let me know if you think the snappy fix will be a straightforward one or will require some time- so that I can decide whether to update the code base now to get the gzip fix or to wait to get both. Thanks in advance.

[edenhill]

I will hopefully have time to look at the snappy issue tonight (utc)
Den 8 jan 2014 13:09 skrev "winbatch" notifications@github.com:

Magnus- let me know if you think the snappy fix will be a straightforward
one or will require some time- so that I can decide whether to update the
code base now to get the gzip fix or to wait to get both. Thanks in advance.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/54#issuecomment-31826103
.

[winbatch]

Updated, gzip now works.

On Wednesday, January 8, 2014, Magnus Edenhill wrote:

I will hopefully have time to look at the snappy issue tonight (utc)
Den 8 jan 2014 13:09 skrev "winbatch" <notifications@github.com<javascript:_e({}, 'cvml', 'notifications@github.com');>>:

Magnus- let me know if you think the snappy fix will be a
straightforward
one or will require some time- so that I can decide whether to update
the
code base now to get the gzip fix or to wait to get both. Thanks in
advance.

—
Reply to this email directly or view it on GitHub<
https://github.com/edenhill/librdkafka/issues/54#issuecomment-31826103>
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/54#issuecomment-31831197
.

[edenhill]

Creating a separate issue for snappy.