Reduce allocations in flate decompressor and minor code improvements

[flrdv]

• decompressor.step is now enumeric instead of function pointer. This reduces ALL allocations in decompressor benchmarks to constant 1 (previously this number differed for each case). However, it doesn't bring much gains in terms of throughtput.
• huffmanBlockDecoder(), generated by _gen/gen_inflate.go, also used to return a function pointer. Now calling the method directly inside of it. Also doesn't bring much gains, but in microbenchmarks may be visible
• Improved decompressor benchmarks by replacing io.Copy() with io.CopyBuffer(), because io.Copy() allocates internally a buffer for reading
• Removed some expressions, that are useless

[klauspost]

Thanks!

Could you post times in a (comparable) benchmark? Just want to ensure there isn't a speed regression.

[flrdv]

@klauspost in doStep() method, if current enum is unknown - nothing will be done. This may result in undefined behaviour, up to infinite loop. Currently there's just a debug print in default case, however this doesn't help much either. Shall we rise a panic there?

[klauspost]

if current enum is unknown - nothing will be done. This may result in undefined behaviour, up to infinite loop. Currently there's just a debug print in default case, however this doesn't help much either. Shall we rise a panic there?

Sure. Should be easier to pick up by a Fuzzer then.

[flrdv]

Full benchmarks output can be found here: https://gist.github.com/fakefloordiv/395a15a982cda3e43dc4f4833d3b2aac

Here's short conclusion:

Time per operation (in ns/op):

DecodeDigitsSpeed1e4:   36650   -> 33963 (1.079x)
DecodeDigitsSpeed1e5:   346724  -> 340885 (1.017x)
DecodeDigitsSpeed1e6:   3576404 -> 3515543 (1.017x)
DecodeDigitsDefault1e4:   32550   -> 31307 (1.04x)
DecodeDigitsDefault1e5:   388516  -> 383745 (1.012x)
DecodeDigitsDefault1e6:   3847923 -> 3825121 (1.006x)
DecodeDigitsCompress1e4:   33756   -> 31259 (1.08x)
DecodeDigitsCompress1e5:   367638  -> 359207 (1.023x)
DecodeDigitsCompress1e6:   3618779 -> 3546951 (1.02x)
DecodeTwainSpeed1e4:   28407   -> 27533 (1.032x)
DecodeTwainSpeed1e5:   377619  -> 371688 (1.016x)
DecodeTwainSpeed1e6:   3810692 -> 3726031 (1.023x)
DecodeTwainDefault1e4:   29461   -> 27603 (1.067x)
DecodeTwainDefault1e5:   393549  -> 385159 (1.022x)
DecodeTwainDefault1e6:   3926477 -> 3854784 (1.019x)
DecodeTwainCompress1e4:   26872   -> 25846 (1.04x)
DecodeTwainCompress1e5:   365831  -> 358215 (1.021x)
DecodeTwainCompress1e6:   3651813 -> 3577634 (1.021x)
DecodeRandomSpeed1e4:   214     -> 202.8 (1.055x)
DecodeRandomSpeed1e5:   1659    -> 1626 (1.02x)
DecodeRandomSpeed1e6:   17198   -> 17274 (0.996x)

Throughtput (in mb/s):

DecodeDigitsSpeed1e4:   272.85 -> 294.43 (1.079x)
DecodeDigitsSpeed1e5:   288.41 -> 293.35 (1.017x)
DecodeDigitsSpeed1e6:   279.61 -> 284.45 (1.017x)
DecodeDigitsDefault1e4:   307.22 -> 319.42 (1.04x)
DecodeDigitsDefault1e5:   257.39 -> 260.59 (1.012x)
DecodeDigitsDefault1e6:   259.88 -> 261.43 (1.006x)
DecodeDigitsCompress1e4:   296.24 -> 319.91 (1.08x)
DecodeDigitsCompress1e5:   272.01 -> 278.39 (1.023x)
DecodeDigitsCompress1e6:   276.34 -> 281.93 (1.02x)
DecodeTwainSpeed1e4:   352.03 -> 363.2 (1.032x)
DecodeTwainSpeed1e5:   264.82 -> 269.04 (1.016x)
DecodeTwainSpeed1e6:   262.42 -> 268.38 (1.023x)
DecodeTwainDefault1e4:   339.44 -> 362.28 (1.067x)
DecodeTwainDefault1e5:   254.1 -> 259.63 (1.022x)
DecodeTwainDefault1e6:   254.68 -> 259.42 (1.019x)
DecodeTwainCompress1e4:   372.13 -> 386.91 (1.04x)
DecodeTwainCompress1e5:   273.35 -> 279.16 (1.021x)
DecodeTwainCompress1e6:   273.84 -> 279.51 (1.021x)
DecodeRandomSpeed1e4:   46695.1 -> 49300.24 (1.056x)
DecodeRandomSpeed1e5:   60272.24 -> 61516.82 (1.021x)
DecodeRandomSpeed1e6:   58144.68 -> 57889.09 (0.996x)

Please note: benchmarks were also updated (replaced io.Copy with io.CopyBuffer), this may affect results, too. However, even in this case they simply became a bit more accurate

[klauspost]

benchmarks were also updated

Yes. That is why I asked for "comparable benchmarks", meaning the same code. :)

I don't think the "CopyBuffer" will make any difference, since both will use the io.WriterTo interface, so you should be able to take that out.

[flrdv]

benchmarks were also updated

Yes. That is why I asked for "comparable benchmarks", meaning the same code. :)

I don't think the "CopyBuffer" will make any difference, since both will use the io.WriterTo interface, so you should be able to take that out.

You are right, there's completely no difference. Here are results with io.Copy: https://gist.github.com/fakefloordiv/d32971935516f566006f28c6c000a330

[klauspost]

LGTM