Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add human readable Format for parquet ByteArray #642

Merged
merged 1 commit into from
Jul 31, 2021

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Jul 31, 2021

Which issue does this PR close?

re #641

Rationale for this change

While working on #641 I found it challenging to determine what values were in what ByteArray while debugging with "{:?}"

What changes are included in this PR?

  1. Add more human readable std::debug::Format impl for ByteArray

Before this PR the output for a string looks like this

ByteArray { data: Some(BufferPtr { data: [97, 110, 100, 111, 118, 101, 114], start: 0, len: 7, mem_tracker: None }) }

After this PR, the output looks like:

ByteArray { data: "andover" }

Are there any user-facing changes?

Debugging output is different for ByteArray

It also makes test failure output easier to understand:


---- column::writer::tests::test_byte_array_statistics stdout ----
thread 'column::writer::tests::test_byte_array_statistics' panicked at 'assertion failed: `(left == right)`
  left: `ByteArray { data: "lexington" }`,
 right: `ByteArray { data: "tewsbury" }`', parquet/src/column/writer.rs:1708:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Rather than a list of individual bytes

@github-actions github-actions bot added the parquet Changes to the parquet crate label Jul 31, 2021
@codecov-commenter
Copy link

Codecov Report

Merging #642 (087402a) into master (e84fe20) will increase coverage by 0.02%.
The diff coverage is 0.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #642      +/-   ##
==========================================
+ Coverage   82.47%   82.50%   +0.02%     
==========================================
  Files         167      168       +1     
  Lines       46454    47237     +783     
==========================================
+ Hits        38314    38972     +658     
- Misses       8140     8265     +125     
Impacted Files Coverage Δ
parquet/src/data_type.rs 76.54% <0.00%> (-1.11%) ⬇️
arrow/src/array/equal_json.rs 88.69% <0.00%> (-2.52%) ⬇️
parquet/src/arrow/schema.rs 87.35% <0.00%> (-1.18%) ⬇️
arrow/src/datatypes/datatype.rs 66.08% <0.00%> (-0.74%) ⬇️
arrow/src/array/equal/mod.rs 93.45% <0.00%> (-0.48%) ⬇️
arrow/src/datatypes/field.rs 51.36% <0.00%> (-0.47%) ⬇️
arrow/src/array/builder.rs 85.97% <0.00%> (-0.29%) ⬇️
arrow/src/ipc/convert.rs 92.93% <0.00%> (-0.06%) ⬇️
arrow/src/json/reader.rs 83.99% <0.00%> (-0.04%) ⬇️
arrow/src/ipc/writer.rs 86.22% <0.00%> (ø)
... and 13 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e84fe20...087402a. Read the comment docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
parquet Changes to the parquet crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants