Support Tables.partitions when reading "arrow file" format in addition to "stream" format #293
Comments
quinnj
added a commit
that referenced
this issue
May 24, 2023
We had this functionality w/ `Arrow.Stream`, but it's convenient and not that expensive to define it for `Arrow.Table` as well. Fixes #293.
|
PR up: #443 |
quinnj
added a commit
that referenced
this issue
May 25, 2023
We had this functionality w/ `Arrow.Stream`, but it's convenient and not that expensive to define it for `Arrow.Table` as well. Fixes #293.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I see #23 closed but only stream format got supported.
How about files in "arrow file" format, each with multiple record batches inside?
With released version I get a single
Tableas the result ofTables.partitions(tbl)is a::Tuple{Arrow.Table}, however the columns there are chunked.Can we get partition tables with contiguous column arrays via
Tables.partitions?As for workarounds before that's supported, any public API I can use to obtain contiguous column arrays per each record batches?
The text was updated successfully, but these errors were encountered: