Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RecordBatch From StructArray Silently Discards Nulls #3952

Closed
tustvold opened this issue Mar 26, 2023 · 5 comments · Fixed by #3953
Closed

RecordBatch From StructArray Silently Discards Nulls #3952

tustvold opened this issue Mar 26, 2023 · 5 comments · Fixed by #3953
Labels
arrow Changes to the arrow crate bug parquet Changes to the parquet crate

Comments

@tustvold
Copy link
Contributor

Describe the bug

To Reproduce

#[test]
fn test_from_struct() {
    let s = StructArray::from(ArrayData::new_null(
        &DataType::Struct(vec![Field::new("foo", DataType::Int32, false)]),
        2,
    ));

    let batch = RecordBatch::from(&s);
    assert_eq!(batch.column(0).null_count(), 2);
}

Expected behavior

The above should fail

Additional context

@tustvold tustvold added the bug label Mar 26, 2023
@SinanGncgl
Copy link
Contributor

Hi, do we expect a failure something like => left = 0 and right = 2 assertion failed: (left == right) so should we expect an empty batch from RecordBatch::from? Could you please elaborate more so that maybe I can take it? Thanks for your time and answer in advance.

@tustvold
Copy link
Contributor Author

There is already a PR to fix this, I just accidentally linked the wrong issue 😅

@SinanGncgl
Copy link
Contributor

There is already a PR to fix this, I just accidentally linked the wrong issue sweat_smile

Ah okay, thank you for your swift response 😃

@tustvold tustvold added the parquet Changes to the parquet crate label Apr 7, 2023
@tustvold
Copy link
Contributor Author

tustvold commented Apr 7, 2023

label_issue.py automatically added labels {'parquet'} from #3953

@tustvold tustvold added the arrow Changes to the arrow crate label Apr 7, 2023
@tustvold
Copy link
Contributor Author

tustvold commented Apr 7, 2023

label_issue.py automatically added labels {'arrow'} from #3953

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate bug parquet Changes to the parquet crate
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants