zstd.go initializes shared global encoder+decoder regardless of whether zstd is being used #1937
Labels
needs-investigation
Issues that require followup from maintainers
stale
Issues and pull requests without any recent activity
Currently we initialise a global zstd encoder+decoder at import time:
https://github.com/Shopify/sarama/blob/8dbbfb50f4698c577c49ff8142c90a9185eb3114/zstd.go#L7-L8
Under the covers the compress library spins up goroutines to handle the decompression/compression
If the Sarama user doesn't ever intend to use zstd then this is a bunch of wasted resource.
I'm also not sure if a shared global encoder+decoder is the correct behaviour, or whether we should have per-producer encoders and
WithEncoderConcurrency(1)
and per-consumer decoders andWithDecoderConcurrency(1)
I also wonder if we should be surfacing the
WithDecoderLowmem
andWithWindowSize
options in sarama.Config{...} so that users can tune the performance vs memory usage tradeoffs themselves.The text was updated successfully, but these errors were encountered: