Skip to content
This repository has been archived by the owner on Feb 18, 2024. It is now read-only.

Added support to write parquet _metadata sidecar #1063

Merged
merged 1 commit into from
Jun 10, 2022
Merged

Conversation

jorgecarleitao
Copy link
Owner

@jorgecarleitao jorgecarleitao commented Jun 10, 2022

This PR exposes parquet2's API to write partitioned parquet files' metadata on a sidecar, _common_metadata and _metadata. See jorgecarleitao/parquet2#146 for context within parquet2 and pyarrow documentation. Thanks to @kylebarron for the idea and thorough explanation!

@jorgecarleitao jorgecarleitao added the feature A new feature label Jun 10, 2022
@codecov
Copy link

codecov bot commented Jun 10, 2022

Codecov Report

Merging #1063 (0da8601) into main (b0c601e) will decrease coverage by 0.01%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main    #1063      +/-   ##
==========================================
- Coverage   81.32%   81.30%   -0.02%     
==========================================
  Files         363      363              
  Lines       34685    34692       +7     
==========================================
  Hits        28206    28206              
- Misses       6479     6486       +7     
Impacted Files Coverage Δ
src/io/parquet/write/file.rs 76.47% <0.00%> (-4.78%) ⬇️
src/io/parquet/write/mod.rs 62.88% <ø> (ø)
src/io/parquet/read/deserialize/null.rs 0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b0c601e...0da8601. Read the comment docs.

@jorgecarleitao jorgecarleitao changed the title Added support for parquet _metadata sidecar Added support to write parquet _metadata sidecar Jun 10, 2022
@jorgecarleitao jorgecarleitao merged commit 5f657aa into main Jun 10, 2022
@jorgecarleitao jorgecarleitao deleted the sidecar branch June 10, 2022 15:09
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
feature A new feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant