Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect null count for cast kernel for list arrays #815

Closed
alamb opened this issue Oct 6, 2021 · 0 comments · Fixed by #816
Closed

Incorrect null count for cast kernel for list arrays #815

alamb opened this issue Oct 6, 2021 · 0 comments · Fixed by #816
Labels

Comments

@alamb
Copy link
Contributor

alamb commented Oct 6, 2021

Describe the bug
When casting a ListArray using the cast kernel, the resulting null count is incorrect

To Reproduce

use std::sync::Arc;

use arrow::{array::{Array, ArrayRef, ListArray}, datatypes::{DataType, Field, Int32Type, UInt32Type}};

fn main() {

    // Form the list [[0, -11, -22], null]
    let data = vec![
        Some(vec![Some(0), Some(-11), Some(-22)]),
        None,
    ];
    let list_array = ListArray::from_iter_primitive::<Int32Type, _, _>(data);

    assert_eq!(DataType::Int32, list_array.value_type());
    assert_eq!(2, list_array.len());
    assert_eq!(1, list_array.null_count());

    let list_array: ArrayRef = Arc::new(list_array);

    // Cast to UInt32, resulting in
    // the list [[0, null, null], null]
    let new_datatype = DataType::List(Box::new(Field::new("item", DataType::UInt32, true)));
    let cast_array = arrow::compute::cast(&list_array, &new_datatype).unwrap();
    let cast_array = cast_array.as_any().downcast_ref::<ListArray>().unwrap();

    let data = vec![
        Some(vec![Some(0), None, None]),
        None,
    ];
    let expected_array = ListArray::from_iter_primitive::<UInt32Type, _, _>(data);

    assert_eq!(expected_array.null_count(), 1);
    // currently fails cast_array has a null count of 2 (not 1)
    assert_eq!(cast_array.null_count(), 1);
    // the expected array should be the same as well
    assert_eq!(cast_array, &expected_array);
}

Results in this failure:

thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `2`,
 right: `1`', src/main.rs:34:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Expected behavior
The test program above should pass

Additional context
Found while adding validation to #810

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant