Zlib::BufError

[jvenezia]

This code raises an Zlib::BufError, for this specific url.

url = 'https://www.mediquality.net/fr/web/MediQuality/-/en-suede-la-cigarette-electronique-est-un-medicament-selon-la-justice'
title = MetaInspector.new(url, retries: 0, download_images: false).title

Is it possible to find a workaround? Or is the response is just not acceptable to be able to extract something?

Thanks

[jaimeiniesta]

Thanks for the report. I confirm I get this on Ruby 2.1, this is the related exception:

http://ruby-doc.org/stdlib-2.2.0/libdoc/zlib/rdoc/Zlib/BufError.html

On Ruby 2.2, I get this instead:

TypeError: exception object expected
from /Users/jaime/.rvm/rubies/ruby-2.2.0/lib/ruby/2.2.0/net/http/response.rb:266:in `raise'

What's your Ruby version?

[jaimeiniesta]

There seems to be a bug on Ruby's Net::HTTP. We need to understand this in order to find the cure, as Net:HTTP is the adapter we're using:

https://github.com/jaimeiniesta/metainspector/blob/master/lib/meta_inspector/request.rb#L56

I've tried using Excon as the adapter, and in that case the exception raised is FaradayMiddleware::RedirectLimitReached. But we cannot just simply replace Net::HTTP by Excon, this would require changing most of the specs that mock net requests (will do if needed).

[jaimeiniesta]

Found it! It seems to be related with servers returning malformed compressed responses and Net::HTTP trying to decompress them. A better explanation can be found here:

lostisland/faraday#337

A workaround is using these headers:

MetaInspector.new(url, headers: {'Accept-Encoding' => 'identity'})

[jaimeiniesta]

I think this is a good default header, so I've released 4.4.0 that includes this.

[jvenezia]

@jaimeiniesta Awesome :) Thanks a lot for your reactivity and this awesome gem !

[jaimeiniesta]

You're welcome! Thanks for catching this.