-
Notifications
You must be signed in to change notification settings - Fork 38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can we merge all dictionaries into single compressed stream? #397
Comments
I agree, we definitely need to investigate this. |
I'll run a quick test on this. |
My quick test seems to indicate a small decrease in total file size. Unfortunately, I didn't write down a before/after, but it's encouraging. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Just an idea for the next format.
Currently prelude and content sections have multiple small streams (1 stream for each dictionary),
and when decoding the file, decoder has to allocate buffer for each stream.
also, currently there's no information about uncompressed size for brotli stream, afaik.
That means decoder has to do allocation/reallocation multiple times while decoding prelude/content sections.
If there's no merit for compressing each stream, using single stream can make the decoder simpler.
Also, adding uncompressed size might make it simpler (of course we need to validate the size before allocation tho)
The text was updated successfully, but these errors were encountered: