Skip to content
This repository has been archived by the owner on Feb 18, 2024. It is now read-only.

Added support to read and write Parquet's delta-bitpacked (integer encoding) #1226

Merged
merged 3 commits into from
Aug 19, 2022

Conversation

jorgecarleitao
Copy link
Owner

@jorgecarleitao jorgecarleitao commented Aug 15, 2022

This PR adds support to read Encoding::DeltaLengthByteArray, an important encoding for integer types. For now this is only supported on non-nested types.

@jorgecarleitao jorgecarleitao added the feature A new feature label Aug 15, 2022
@jorgecarleitao jorgecarleitao changed the title Added support to read and write delta-bitpacked Added support to read and write Parquet's delta-bitpacked (integer encoding) Aug 15, 2022
@codecov
Copy link

codecov bot commented Aug 15, 2022

Codecov Report

Merging #1226 (9d56c5d) into main (052a80a) will increase coverage by 0.02%.
The diff coverage is 88.55%.

@@            Coverage Diff             @@
##             main    #1226      +/-   ##
==========================================
+ Coverage   83.31%   83.33%   +0.02%     
==========================================
  Files         358      359       +1     
  Lines       37619    37881     +262     
==========================================
+ Hits        31342    31568     +226     
- Misses       6277     6313      +36     
Impacted Files Coverage Δ
src/io/parquet/mod.rs 0.00% <0.00%> (ø)
src/io/parquet/read/deserialize/binary/basic.rs 80.43% <57.14%> (-1.16%) ⬇️
src/io/parquet/read/deserialize/binary/nested.rs 80.86% <58.33%> (-1.54%) ⬇️
src/io/parquet/read/deserialize/dictionary/mod.rs 76.50% <75.00%> (-0.04%) ⬇️
...c/io/parquet/read/deserialize/dictionary/nested.rs 66.11% <75.00%> (+0.28%) ⬆️
...c/io/parquet/read/deserialize/primitive/integer.rs 85.27% <85.27%> (ø)
src/io/parquet/write/mod.rs 86.35% <88.46%> (-0.16%) ⬇️
src/io/parquet/read/deserialize/nested_utils.rs 85.79% <93.75%> (+0.54%) ⬆️
src/io/parquet/write/primitive/basic.rs 98.77% <98.85%> (-0.20%) ⬇️
src/io/parquet/read/deserialize/boolean/nested.rs 74.41% <100.00%> (+0.30%) ⬆️
... and 17 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@jorgecarleitao jorgecarleitao merged commit cfb0d58 into main Aug 19, 2022
@jorgecarleitao jorgecarleitao deleted the delta_bitpacked branch August 19, 2022 06:36
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
feature A new feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant