Read only the necessary bytes when parsing the header #36
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I use this library in a project i'm currently working on. It works beautifully, thank you very much!
The files i use it on are around 1.5GB. The streaming of the body is super cool, however, i noticed that to parse the header, the entire file was being read in memory
This is due to the use of
fs.readFile
insrc/header.js
In this PR, the code is a bit lower-level to be able to read only the necessary bytes. The idea is:
start
field of the header,I'm happy to discuss the change further if what i did here is unclear or if it doesn't adhere to the project standard