Refactor parquet::arrow module #1827
Conversation
|
|
||
| #[cfg(feature = "async")] | ||
| pub mod async_reader; | ||
|
|
||
| experimental_mod!(converter); |
There was a problem hiding this comment.
This module is slated for eventual removal as part of #1661
It is marked experimental so hiding it from the public API is not a breaking change
parquet/src/arrow/buffer/mod.rs
Outdated
| pub mod offset_buffer; | ||
| pub mod dictionary_buffer; | ||
| pub mod converter; |
There was a problem hiding this comment.
I would like to eventually upstream many of these into arrow-rs proper
Codecov Report
@@ Coverage Diff @@
## master #1827 +/- ##
==========================================
- Coverage 83.45% 83.45% -0.01%
==========================================
Files 200 200
Lines 56697 56696 -1
==========================================
- Hits 47315 47313 -2
- Misses 9382 9383 +1
Continue to review full report at Codecov.
|
| @@ -355,27 +355,6 @@ fn create_string_byte_array_dictionary_reader( | |||
| .unwrap() | |||
| } | |||
|
|
|||
| fn create_complex_object_byte_array_dictionary_reader( | |||
There was a problem hiding this comment.
ComplexObjectArrayReader is no longer used and is slated for removal (#1661 )
Yes, the high level ticket is #1163. I'm hoping to pick it up soon, but large amounts of this API are public so needs some careful thought |
Great! 👍 After implement send row_selection in |
Which issue does this PR close?
Closes #.
Rationale for this change
As the reader logic in particular has gotten more sophisticated, the module has gotten rather large. This moves some files around to make the structure easier to understand
What changes are included in this PR?
module_name/mod.rsinstead of havingmodule_name.rsandmodule_name/Are there any user-facing changes?
No