Any completely 100% zero allocation / no heap escape / no garbage generated compression for golang? #808
-
|
knowing u are the expert in this area, do u know of anything like this? i dont mind the compression ratio, even if 10% lesser is fine. or smaller for lesser than 4mb (which seemed to not escape to heap that much) or something similar? anything u know of or can provide a clue on how to get this done? will be compressing and decompressing lots of such data and would prefer not having any memory allocation for garbage collection. just anything. appreciate this help. thx. |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments
-
|
S2 blocks only allocate on the stack. S2 streams with FlushOnWrite and "WriterConcurrency(1)" should also be fully allocation-less. Of course you must reuse the Writers you allocate. |
Beta Was this translation helpful? Give feedback.
-
|
zstd will also be mostly allocationless when using EncodeAll and providing properly sized buffers, once the internal buffers have been allocated. Using WithLowerEncoderMem(false) will make it allocate buffers that are unlikely to need re-allocation. |
Beta Was this translation helpful? Give feedback.
S2 blocks only allocate on the stack.
S2 streams with FlushOnWrite and "WriterConcurrency(1)" should also be fully allocation-less. Of course you must reuse the Writers you allocate.