Skip to content
This repository has been archived by the owner on Feb 18, 2024. It is now read-only.

Added more serialization options for csv writer. #453

Merged
merged 3 commits into from
Sep 27, 2021

Conversation

ritchie46
Copy link
Collaborator

@ritchie46 ritchie46 commented Sep 25, 2021

This changes csv output for the Date64 datatype to also include the time information. So that a round trip of Date64 -> csv -> Date64 would not lose information.

@codecov
Copy link

codecov bot commented Sep 25, 2021

Codecov Report

Merging #453 (7cf0c61) into main (db92910) will decrease coverage by 0.64%.
The diff coverage is 58.33%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #453      +/-   ##
==========================================
- Coverage   80.64%   80.00%   -0.65%     
==========================================
  Files         372      371       -1     
  Lines       22691    22836     +145     
==========================================
- Hits        18300    18270      -30     
- Misses       4391     4566     +175     
Impacted Files Coverage Δ
src/io/csv/write/serialize.rs 47.31% <50.00%> (+1.15%) ⬆️
tests/it/io/csv/write.rs 95.83% <100.00%> (+0.08%) ⬆️
src/alloc/mod.rs 0.00% <0.00%> (-93.03%) ⬇️
src/types/mod.rs 22.22% <0.00%> (-6.10%) ⬇️
src/array/dictionary/mutable.rs 76.66% <0.00%> (-5.48%) ⬇️
src/array/list/mutable.rs 70.96% <0.00%> (-4.90%) ⬇️
src/array/fixed_size_binary/mutable.rs 71.76% <0.00%> (-4.49%) ⬇️
src/array/fixed_size_list/mutable.rs 36.84% <0.00%> (-3.55%) ⬇️
src/array/boolean/mutable.rs 85.23% <0.00%> (-2.96%) ⬇️
src/array/utf8/mutable.rs 87.70% <0.00%> (-2.22%) ⬇️
... and 26 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update db92910...7cf0c61. Read the comment docs.

@ritchie46 ritchie46 force-pushed the csv_out branch 4 times, most recently from e117ad3 to 9784618 Compare September 27, 2021 09:10
@ritchie46
Copy link
Collaborator Author

To stay compliant with arrow-spec, I kept the default the same. I just give user more granular control in how to serialize different data types. Which makes sense as uses might want to serialize a time64 with more precision than a time32.

@ritchie46 ritchie46 changed the title date64 csv write fmt More granular dtype serialization options for csv writer. Sep 27, 2021
@jorgecarleitao jorgecarleitao merged commit 3f3d76c into jorgecarleitao:main Sep 27, 2021
@jorgecarleitao jorgecarleitao changed the title More granular dtype serialization options for csv writer. More granular serialization options for csv writer. Sep 27, 2021
@jorgecarleitao jorgecarleitao changed the title More granular serialization options for csv writer. Added more serialization options for csv writer. Sep 27, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants