Skip to content
This repository has been archived by the owner on Feb 18, 2024. It is now read-only.

Improved support for date64 written by pyarrow to parquet #1402

Merged
merged 1 commit into from
Feb 14, 2023

Conversation

jorgecarleitao
Copy link
Owner

Writing date64 to parquet is implementation-defined. This PR adds support for the date64 written by pyarrow.

Also protects us in the future against variations that we do not support with an error.

Closes #1400

@jorgecarleitao jorgecarleitao added the bug Something isn't working label Feb 14, 2023
@codecov
Copy link

codecov bot commented Feb 14, 2023

Codecov Report

Base: 83.63% // Head: 83.61% // Decreases project coverage by -0.03% ⚠️

Coverage data is based on head (a83d6b0) compared to base (3a8da98).
Patch coverage: 77.98% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1402      +/-   ##
==========================================
- Coverage   83.63%   83.61%   -0.03%     
==========================================
  Files         373      373              
  Lines       40284    40289       +5     
==========================================
- Hits        33692    33687       -5     
- Misses       6592     6602      +10     
Impacted Files Coverage Δ
src/io/parquet/read/statistics/mod.rs 87.31% <14.28%> (-1.30%) ⬇️
src/io/parquet/read/deserialize/simple.rs 83.03% <82.35%> (-2.07%) ⬇️
src/bitmap/utils/slice_iterator.rs 97.56% <0.00%> (-1.22%) ⬇️
src/offset.rs 86.11% <0.00%> (+0.34%) ⬆️
src/io/ipc/read/file.rs 97.32% <0.00%> (+0.89%) ⬆️
src/array/binary/mod.rs 94.08% <0.00%> (+1.47%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@jorgecarleitao jorgecarleitao added enhancement An improvement to an existing feature and removed bug Something isn't working labels Feb 14, 2023
@jorgecarleitao jorgecarleitao merged commit b457d68 into main Feb 14, 2023
@jorgecarleitao jorgecarleitao deleted the date64 branch February 14, 2023 10:44
ritchie46 pushed a commit to ritchie46/arrow2 that referenced this pull request Mar 29, 2023
ritchie46 pushed a commit to ritchie46/arrow2 that referenced this pull request Apr 5, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement An improvement to an existing feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

parquet_read panics when working with date64s
1 participant