Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zlib.createGunzip "unknown compression method" #5852

Closed
dpicky opened this issue Mar 22, 2016 · 10 comments
Closed

zlib.createGunzip "unknown compression method" #5852

dpicky opened this issue Mar 22, 2016 · 10 comments
Labels
zlib Issues and PRs related to the zlib subsystem.

Comments

@dpicky
Copy link

dpicky commented Mar 22, 2016

Since upgrading to 5.9.0 getting lots of "unknown compression method" errors when trying to uncompress files (using zlib.createGunzip) that work in previous versions of Node include 5.8

  • Version: v5.9.0
  • Platform: Both OSX and Linux (CentOS)
  • Subsystem: zlib
@jasnell jasnell added the zlib Issues and PRs related to the zlib subsystem. label Mar 22, 2016
@jasnell
Copy link
Member

jasnell commented Mar 22, 2016

@evanlucas @thealphanerd ... possible zlib regression in v5.9? Could #5120 be to blame?

@dpicky ... can you provide an example test case that demonstrates the problem reliably?

@Trott
Copy link
Member

Trott commented Mar 22, 2016

@dpicky ... can you provide an example test case that demonstrates the problem reliably?

More details would be useful. Straightforward use case does not have this problem as far as I can tell:

$ cat test.js
'use strict';
var zlib = require('zlib');

zlib.gzip('hello', function(err, data) {
  if (err) { 
    return console.log('err', err);
  }
  var unzip = zlib.createGunzip();
  unzip.write(data);
  unzip.on('data', function (data) {
    console.log(data.toString());
  });
});
$ node -v
v5.9.0
$ uname -a
Darwin LIB-0S8DV17-LT 13.4.0 Darwin Kernel Version 13.4.0: Wed Mar 18 16:20:14 PDT 2015; root:xnu-2422.115.14~1/RELEASE_X86_64 x86_64
$ node test.js
hello
$

@Fishrock123
Copy link
Contributor

@dpicky Could you give us some more info on what sort of files these are?

@floatdrop
Copy link
Contributor

@Fishrock123 I have same issue on 5.9.0, but not on 5.8.0. Seems like to reproduce you need quite big file (I get error on about 150Mb point of gzipped file). File was created through standard gzip util without any additional parameters.

@addaleax
Copy link
Member

Just checked, this definitely was introduced in f380db2.

@addaleax
Copy link
Member

I can reproduce this by passing the gcc-5.3.0 tarball to createUnzip():

const fs = require('fs');
const zlib = require('zlib');

let l = 0;

fs.createReadStream('gcc-5.3.0.tar.gz')
  .pipe(zlib.createGunzip())
  .on('data', d => l += d.length);

process.on('exit', () => console.log(l));

results in

598310912
events.js:155
      throw er; // Unhandled 'error' event
      ^

Error: unknown compression method
    at Zlib._handle.onerror (zlib.js:363:17)

@bnoordhuis
Copy link
Member

/cc @kthelgason

@addaleax
Copy link
Member

I think I got this, PR incoming. 😄

addaleax added a commit to addaleax/node that referenced this issue Mar 23, 2016
Add test files that reliably reproduce nodejs#5852. The gzipped file
in test/fixtures/pseudo-multimember-gzip.gz contains the gzip
magic bytes exactly at the position that node encounters after having
read a single block, leading it to believe that a new data
member is starting.
addaleax added a commit to addaleax/node that referenced this issue Mar 23, 2016
Only treat the gzip magic bytes, when encountered within the file
after reading a single block, as the start of a new member when
the previous member has ended.

Fixes: nodejs#5852
@Trott Trott removed the unconfirmed label Mar 23, 2016
evanlucas pushed a commit that referenced this issue Mar 30, 2016
Only treat the gzip magic bytes, when encountered within the file
after reading a single block, as the start of a new member when
the previous member has ended.

Add test files that reliably reproduce #5852. The gzipped file
in test/fixtures/pseudo-multimember-gzip.gz contains the gzip
magic bytes exactly at the position that node encounters after having
read a single block, leading it to believe that a new data
member is starting.

Fixes: #5852
PR-URL: #5863
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
evanlucas pushed a commit that referenced this issue Mar 31, 2016
Only treat the gzip magic bytes, when encountered within the file
after reading a single block, as the start of a new member when
the previous member has ended.

Add test files that reliably reproduce #5852. The gzipped file
in test/fixtures/pseudo-multimember-gzip.gz contains the gzip
magic bytes exactly at the position that node encounters after having
read a single block, leading it to believe that a new data
member is starting.

Fixes: #5852
PR-URL: #5863
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
mjackson added a commit to mjackson/unpkg that referenced this issue Apr 7, 2016
A bug was introduced in node v5.9 that causes lots of "unknown
compression method" errors, see
nodejs/node#5852

Hopefully this is fixed in the next patch release.
@thomasmodeneis
Copy link

Hi,
I just ran into this thing, is there any specific version where this is fixed ?

events.js:154
      throw er; // Unhandled 'error' event
      ^

Error: unknown compression method
    at Zlib._handle.onerror (zlib.js:363:17)
$ node --version
v5.9.1 

Cheers.

@Trott
Copy link
Member

Trott commented Dec 22, 2016

@thomasmodeneis Fixed in 6.0.0 and above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
zlib Issues and PRs related to the zlib subsystem.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants