We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Describe the bug When serializing an all-null arrow array to parquet, the null-count in the stats is always 0.
To Reproduce Steps to reproduce the behavior:
#[test] fn statistics_null_counts_only_nulls() { // check that null-count statistics for "only NULL"-columns are correct let values = Arc::new(UInt64Array::from(vec![ None, None, ])); let file = one_column_roundtrip("null_counts", values, true); // check statistics are valid let reader = SerializedFileReader::new(file).unwrap(); let metadata = reader.metadata(); assert_eq!(metadata.num_row_groups(), 1); let row_group = metadata.row_group(0); assert_eq!(row_group.num_columns(), 1); let column = row_group.column(0); let stats = column.statistics().unwrap(); assert_eq!(stats.null_count(), 2); // <<< this fails, null count is 0 }
Expected behavior For all-null columns the null-count should be the same as the number of rows.
Additional context Tested on c863a2c44bffa5c092a49e07910d5e9225483193.
c863a2c44bffa5c092a49e07910d5e9225483193
I am claiming this issue since I have a fix ready.
The text was updated successfully, but these errors were encountered:
ensure null-counts are written for all-null columns
4c628f8
Fixes apache#306.
ensure null-counts are written for all-null columns (#307)
94a82cd
Fixes #306.
46b771c
ensure null-counts are written for all-null columns (#307) (#404)
4d216f3
Fixes #306. Co-authored-by: Marco Neumann <marco@crepererum.net>
Successfully merging a pull request may close this issue.
Describe the bug
When serializing an all-null arrow array to parquet, the null-count in the stats is always 0.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
For all-null columns the null-count should be the same as the number of rows.
Additional context
Tested on
c863a2c44bffa5c092a49e07910d5e9225483193
.I am claiming this issue since I have a fix ready.
The text was updated successfully, but these errors were encountered: