-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Closed
Milestone
Description
There's a lot of complicated logic around trying to re-use byte buffers as well as allocating slabs before they are needed as well as shrinking / consolidating.
After discussing with @julienledem we should simplify and not bother reusing the byte buffers, and not allocate buffers until they are needed. The hope is this will reduce the memory overhead and that the number of byte buffers should be small so they should not cause trouble for GC.
Reporter: Alex Levenson / @isnotinvain
Related issues:
- ColumnChunkPageWriter uses only heap memory. (relates to)
Note: This issue was originally created as PARQUET-160. Please see the migration documentation for further details.