You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
The ListArrayReader currently uses a custom remove_indices function in order to strip nulls from its values array.
Describe the solution you'd like
This function is very similar to the arrow take kernel, and with some minor tweaks we can likely eliminate a non-trivial amount of code from the parquet crate, whilst also potentially improving performance
alamb
added
arrow
Changes to the arrow crate
parquet
Changes to the parquet crate
and removed
enhancement
Any new improvement worthy of a entry in the changelog
arrow
Changes to the arrow crate
labels
Mar 31, 2022
alamb
changed the title
Use Arrow take kernel Within ListArrayReader
Cleanup: Use Arrow take kernel Within parquet ListArrayReader
Mar 31, 2022
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
The
ListArrayReader
currently uses a customremove_indices
function in order to strip nulls from its values array.Describe the solution you'd like
This function is very similar to the arrow take kernel, and with some minor tweaks we can likely eliminate a non-trivial amount of code from the parquet crate, whilst also potentially improving performance
Describe alternatives you've considered
The code could remain unchanged
Additional context
Originally pointed out here
The text was updated successfully, but these errors were encountered: