Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multi member / bgzip compatibility #145

Closed
wants to merge 8 commits into from

Conversation

rbuels
Copy link

@rbuels rbuels commented Sep 1, 2018

This pull req expands upon @Kirill89's 15e220c, re-enabling and fixing the SYNC test that was failing, and adding 2 more tests cases for inflating content that was compressed by bgzip, which is a compression tool widely used in bioinformatics.

The most significant change here is that calling inflateReset now discards and re-creates the InflateState of the stream, where before it was leaving it unchanged from how it was at the end of decompressing the previous compression member in the file.

fixes #139

@rbuels
Copy link
Author

rbuels commented Sep 1, 2018

fyi, I ran the browserify-zlib tests against this code and they passed as well.

@rbuels rbuels changed the title WIP multi member multi member / bgzip compatibility Sep 1, 2018
@rbuels
Copy link
Author

rbuels commented Sep 1, 2018

@abetusk I think this PR might address your problem, if you want to give it a try

Copy link
Member

@puzrin puzrin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some notes.

Also see #139 (comment), that's the main reason of freeze. I can merge Kirill's patch without tests, but can not merge without that fix.

lib/inflate.js Outdated Show resolved Hide resolved
test/fixtures/gzip-joined Show resolved Hide resolved
@@ -86,4 +86,17 @@ describe('Gzip special cases', function () {
assert.deepEqual(inflator.result, expectedDataArray, 'inflator produced expected data');
});

it('Read bgzipped file 1', function () {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will be more convenient if all 3 tests will have similar names and fixtures:

  • "Read streamed file 1" + "streamed_gzip_java.txt.gz
  • "Read streamed file 2" + "streamed_bgzip_1.txt.gz
  • "Read streamed file 3" + "streamed_bgzip_2.txt.gz

@@ -207,7 +207,7 @@ function inflateReset(strm) {
var state;

if (!strm || !strm.state) { return Z_STREAM_ERROR; }
state = strm.state;
state = new InflateState();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess, that's wrong. everything in /zlib folder is ported from upstream, and does exactly what it should. Problems on upper wrapped should not affect this code.

@@ -237,7 +241,7 @@ Inflate.prototype.push = function (data, mode) {
}

if (strm.next_out) {
if (strm.avail_out === 0 || status === c.Z_STREAM_END || (strm.avail_in === 0 && (_mode === c.Z_FINISH || _mode === c.Z_SYNC_FLUSH))) {
if (strm.avail_out === 0 || (status === c.Z_STREAM_END && strm.avail_in === 0) || (strm.avail_in === 0 && (_mode === c.Z_FINISH || _mode === c.Z_SYNC_FLUSH))) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, now understand. I don't remember all black magick in this conditions, and how should it work. Probably, "make cover" should generate report and show if this condition is covered or not.

Did some test failed without this?

@rbuels
Copy link
Author

rbuels commented Sep 1, 2018

OK yeah, I think this needs a different approach. I'll open a new PR if I figure something else out.

@rbuels rbuels closed this Sep 1, 2018
@puzrin
Copy link
Member

puzrin commented Sep 1, 2018

Probably, final PR could be something like this:

That will be easy to read in history.

I see no problems to continue use this PR as temporary sandbox.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Failing for BGZIP'd streaming files
2 participants