Skip to content
This repository has been archived by the owner on Feb 18, 2024. It is now read-only.

Fixed reading multiple parquet pages. #374

Merged
merged 1 commit into from
Sep 3, 2021
Merged

Fixed reading multiple parquet pages. #374

merged 1 commit into from
Sep 3, 2021

Conversation

jorgecarleitao
Copy link
Owner

There was a mistake from my part in computing the remaining number of elements when reading parquet pages that caused the skip of the last elements of the last page. :/

Thank you very much to @vincev for reporting this at #373 .

Closes #373

PS. I will be working in creating random tests to make us a bit more resilient on large scale parquet files created by external sources.

@codecov
Copy link

codecov bot commented Sep 3, 2021

Codecov Report

Merging #374 (8a79349) into main (dbb7b8a) will increase coverage by 0.00%.
The diff coverage is 88.88%.

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #374   +/-   ##
=======================================
  Coverage   81.08%   81.09%           
=======================================
  Files         328      328           
  Lines       21748    21750    +2     
=======================================
+ Hits        17635    17638    +3     
+ Misses       4113     4112    -1     
Impacted Files Coverage Δ
src/io/parquet/read/fixed_size_binary.rs 50.63% <75.00%> (+0.63%) ⬆️
src/io/parquet/read/binary/basic.rs 81.95% <100.00%> (ø)
src/io/parquet/read/binary/dictionary.rs 81.03% <100.00%> (+0.33%) ⬆️
src/io/parquet/read/primitive/basic.rs 82.85% <100.00%> (+0.50%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dbb7b8a...8a79349. Read the comment docs.

@jorgecarleitao jorgecarleitao merged commit 43d8cf5 into main Sep 3, 2021
@jorgecarleitao jorgecarleitao deleted the fix_parquet branch September 3, 2021 20:58
@jorgecarleitao jorgecarleitao added the bug Something isn't working label Sep 23, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Parquet read skips a few rows at the end of the page
1 participant