deflate: Better Huffman encoding #374
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Speed up and improve huffman compression: ``` λ benchcmp before.txt after.txt benchmark old ns/op new ns/op delta BenchmarkEncodeDigitsConstant1e4-32 32925 20138 -38.84% BenchmarkEncodeDigitsConstant1e5-32 425414 218386 -48.67% BenchmarkEncodeDigitsConstant1e6-32 4261446 1866023 -56.21% BenchmarkEncodeDigitsSpeed1e4-32 66777 60683 -9.13% BenchmarkEncodeDigitsSpeed1e5-32 855737 807328 -5.66% BenchmarkEncodeDigitsSpeed1e6-32 8584307 7505546 -12.57% BenchmarkEncodeDigitsDefault1e4-32 124753 123101 -1.32% BenchmarkEncodeDigitsDefault1e5-32 1536784 1507136 -1.93% BenchmarkEncodeDigitsDefault1e6-32 15765790 14838850 -5.88% BenchmarkEncodeDigitsCompress1e4-32 185589 186598 +0.54% BenchmarkEncodeDigitsCompress1e5-32 3264706 3277041 +0.38% BenchmarkEncodeDigitsCompress1e6-32 35219900 35308128 +0.25% BenchmarkEncodeDigitsSL1e4-32 59526 54858 -7.84% BenchmarkEncodeDigitsSL1e5-32 916883 896292 -2.25% BenchmarkEncodeDigitsSL1e6-32 9180701 8873708 -3.34% BenchmarkEncodeTwainConstant1e4-32 41059 29454 -28.26% BenchmarkEncodeTwainConstant1e5-32 486514 248799 -48.86% BenchmarkEncodeTwainConstant1e6-32 3938046 2547548 -35.31% BenchmarkEncodeTwainSpeed1e4-32 87027 82783 -4.88% BenchmarkEncodeTwainSpeed1e5-32 851805 803264 -5.70% BenchmarkEncodeTwainSpeed1e6-32 7885728 7452326 -5.50% BenchmarkEncodeTwainDefault1e4-32 126807 126695 -0.09% BenchmarkEncodeTwainDefault1e5-32 1371597 1373745 +0.16% BenchmarkEncodeTwainDefault1e6-32 13067533 13027351 -0.31% BenchmarkEncodeTwainCompress1e4-32 237083 234776 -0.97% BenchmarkEncodeTwainCompress1e5-32 4430928 4396044 -0.79% BenchmarkEncodeTwainCompress1e6-32 48377762 48015133 -0.75% BenchmarkEncodeTwainSL1e4-32 80816 81162 +0.43% BenchmarkEncodeTwainSL1e5-32 889941 868247 -2.44% BenchmarkEncodeTwainSL1e6-32 8740752 8356943 -4.39% ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Speed up and improve huffman compression:
Lower levels also slightly affected.
Sizes:
nyc-taxi-data-10M.csv: 1877917504 -> 1872115496 bytes
consensus.db.10gb: 7774876313 -> 7769853327 bytes
github-june-2days-2019.json: 4104164737 -> 4097019597 bytes.