Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

7zip 'Warnings: Headers Error' #13

Closed
schmity opened this issue Dec 28, 2022 · 4 comments
Closed

7zip 'Warnings: Headers Error' #13

schmity opened this issue Dec 28, 2022 · 4 comments
Labels
bug Something isn't working

Comments

@schmity
Copy link

schmity commented Dec 28, 2022

Hi,
I am using zipstream-ng to stream data from several URL-endpoints into one zip stream. For that I adapted your code posted in #11 (comment)

The zip file is created fine. However, on extraction with 7zip on a Windows PC it shows 'Warnings: Headers Error' even though it extracts the files OK. This happens both when writing the zipstream to disk or sending it as Flask Response object.

Have you encountered this before and do you have any ideas on how to get rid of that warning?

Best, Sebastian
@pR0Ps
Copy link
Owner

pR0Ps commented Dec 28, 2022 •
edited
Loading

No, I haven't seen that happen. From some searching around, it looks like a somewhat-generic message that 7zip emits when some metadata (CRC, zip64, file sizes, etc.) doesn't match the actual file data, but not in a way that prevents extraction.

I'd like to attempt to fix the issue though. If you have a zip file that exhibits the issue (that doesn't have anything sensitive in it) could you upload it here (or externally if it's too large) so I can take a look?

@pR0Ps pR0Ps added the bug Something isn't working label Dec 28, 2022
@schmity
Copy link
Author

schmity commented Dec 28, 2022

Thanks for looking into this. Find a generated zip-archive attached along with my little Flask app that generated it.

requirements.txt
download_app.py.txt
HH_WFS_abfall_recycling.zip

pR0Ps added a commit that referenced this issue Dec 29, 2022
@pR0Ps
Fix Zip64 and data descriptor interactions
8924cda 
The Python stdlib `zipfile.ZipInfo.FileHeader` function has an issue
where a Zip64 extra field can be added without the `min_version` and
file sizes being updated to reflect this.

To fix the issue, the `FileHeader` function was overridden on the
`ZipStreamInfo` class with a custom implementation. As the zip spec is
somewhat-ambiguous in how data descriptors and Zip64 extra field
interact, the assumptions were documented in a comment.

In addition to fixing issue #13 (7zip emitting header warnings when
extracting zip generated from iterable data), this also fixes an
issue where the `min_version` wouldn't properly indicate zip64 support
being required.
pR0Ps added a commit that referenced this issue Dec 29, 2022
@pR0Ps
Fix Zip64 and data descriptor interactions
64c3ba4 
The Python stdlib `zipfile.ZipInfo.FileHeader` function has an issue
where a Zip64 extra field can be added without the `min_version` and
file sizes being updated to reflect this.

To fix the issue, the `FileHeader` function was overridden on the
`ZipStreamInfo` class with a custom implementation. As the zip spec is
somewhat-ambiguous in how data descriptors and Zip64 extra field
interact, the assumptions were documented in a comment.

In addition to fixing issue #13 (7zip emitting header warnings when
extracting zip generated from iterable data), this also fixes an
issue where the `min_version` wouldn't properly indicate zip64 support
being required.
pR0Ps added a commit that referenced this issue Dec 29, 2022
@pR0Ps
Fix Zip64 and data descriptor interactions
6eef3b3 
The Python stdlib `zipfile.ZipInfo.FileHeader` function has an issue
where a Zip64 extra field can be added without the `min_version` and
file sizes being updated to reflect this.

To fix the issue, the `FileHeader` function was overridden on the
`ZipStreamInfo` class with a custom implementation. As the zip spec is
somewhat-ambiguous in how data descriptors and Zip64 extra field
interact, the assumptions were documented in a comment.

In addition to fixing issue #13 (7zip emitting header warnings when
extracting zip generated from iterable data), this also fixes an
issue where the `min_version` wouldn't properly indicate zip64 support
being required.
@pR0Ps
Copy link
Owner

pR0Ps commented Dec 29, 2022

Fixed as of 6eef3b3 (released in v1.3.5). The issue was that the data sizes in the file headers weren't properly being set to 0xFFFFFFFF when a Zip64 extra field was added. Not something that most (all?) unzipping programs would have an issue with since the size was still where it should've been, but still something that doesn't conform to spec.

@pR0Ps pR0Ps closed this as completed Dec 29, 2022
@schmity
Copy link
Author

schmity commented Dec 29, 2022

Awesome, thanks for the quick fix! Unzipping now works without warning in 7Zip.

@pR0Ps pR0Ps mentioned this issue Jan 5, 2023
Closed
@pR0Ps pR0Ps mentioned this issue Apr 26, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@pR0Ps @schmity