-
Notifications
You must be signed in to change notification settings - Fork 810
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make rle
decoder public under experimental
feature
#1271
Conversation
Make 'encodings' mod public
Make the 'rle' mod public outside the crate.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the contribution @zeevm
This is fine with me, though I wonder what your expectation is for the stability of the API within the rle
module? I don't think it has changed much recently but it may in the upcoming releases.
What would you think if we made it experimental
as a signal that we may change it (meaning you would have to enable the experimental
arrow feature )
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM too, I'd also suggest to consider putting it under experimental
flag.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this would be a perfect fit for what the experimental feature flag was created for (#1032) - namely lower-level APIs that may change, and that we don't want to overburden the average user who "just wants to read a parquet file" with the cognitive load of 👍
FWIW I do hope to at some point do a cleanup of some of the lower-level APIs in the parquet crate, in particular a lot of the trait plumbing currently leaks out, but who knows when I'll find time for that 😅
parquet/src/encodings/mod.rs
Outdated
@@ -18,4 +18,4 @@ | |||
pub mod decoding; | |||
pub mod encoding; | |||
pub mod levels; | |||
pub(crate) mod rle; | |||
pub mod rle; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps if you're only interested in rle
we could make the above modules pub(crate)
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tustvold I'd like to use 'decoding' as well, it makes it really easy to decode RLE encoded pages I get from a column page reader.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense 👍
What's the process for completing the PR? |
Codecov Report
@@ Coverage Diff @@
## master #1271 +/- ##
==========================================
- Coverage 83.02% 83.02% -0.01%
==========================================
Files 180 180
Lines 52269 52269
==========================================
- Hits 43398 43396 -2
- Misses 8871 8873 +2
Continue to review full report at Codecov.
|
LGTM, merging to master. |
rle
decoder public under experimental
feature
Closes #1270
Are there any user-facing changes?
'rle' decoder is exposed outside the crate.