encode: Tweak compression settings

[klauspost]

zstd: Limit window sizes to 128K to keep memory in control both server and client size.
zstd: Write 0 length frames. This may be needed for compatibility.
zstd: Create fewer encoders. Small memory improvement.
gzip: Allow -2 (Huffman only) and -3 (stateless) compression modes.

[CLAassistant]

All committers have signed the CLA.

[mholt]

Ah, thanks. Always a pleasure to accept your contributions. Appreciate the work you do on your compress libs!

[polarathene]

zstd: Limit window sizes to 128K to keep memory in control both server and client size.

Is there any citation for clients that have trouble with the 8MB window size?

Since Chrome landed stable support for zstd content encoding in March 2024 releases of Chrome, they've been following RFC 8878 guidance and limiting the allowed window size to 8MB: #6140 (comment)

Their brotli support has a 16MB window, so for browsers there's likely no concern about the 8MB window limit? Where is the 128KB limit needed? Some embedded devices? Perhaps it'd be better to clarify/document that, potentially allowing for this to be configurable in some manner?

zstd: Create fewer encoders. Small memory improvement.

Likewise it could be useful to support more than a single thread, although I guess larger static content could now be supported with pre-compressed zstd assets instead. This doesn't affect the client decoder right? Is the memory improvement likely to make that much of an improvement on the server at the expense of reduced encoding speed?

---

Granted that for many small files it's unlikely to matter much adjusting these. Just curious about these fixed settings and decisions behind them.

[klauspost]

@polarathene

Where is the 128KB limit needed?

It is not needed - but you don't gain much from the additional window, so it is just to avoid mostly needless memory use, since most encodes will be small anyway.

Likewise it could be useful to support more than a single thread

Likewise it is pointless in this context. There is currently limited concurrency for streaming encodes anyway - and it will just add overhead.

Smooth operation should go ahead of "complete efficiency".

[mholt]

Thanks for chiming in @klauspost ! Your info has been very helpful.