Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using a take kernel on a dense union can result in reaching "unreachable" code #6206

Closed
mhilton opened this issue Aug 7, 2024 · 2 comments · Fixed by #6209
Closed

Using a take kernel on a dense union can result in reaching "unreachable" code #6206

mhilton opened this issue Aug 7, 2024 · 2 comments · Fixed by #6209
Labels
arrow Changes to the arrow crate bug

Comments

@mhilton
Copy link

mhilton commented Aug 7, 2024

Describe the bug
If one applies a take kernel to a densely packed union then it is possible reach an unreachable! code branch. This causes a panic.

To Reproduce
The smallest reproducer I could find is the following application:

use arrow::{
    array::{Int64Array, UnionArray},
    buffer::ScalarBuffer,
    compute::take,
    datatypes::{DataType, Field, UnionFields},
};
use std::sync::Arc;

fn main() {
    let fields = UnionFields::new(vec![0], vec![Field::new("a", DataType::Int64, false)]);
    let ints = Arc::new(Int64Array::from(vec![1, 2, 3, 4, 5]));

    let array = UnionArray::try_new(
        fields,
        ScalarBuffer::from_iter(vec![0_i8, 0, 0, 0, 0].into_iter()),
        Some(ScalarBuffer::from_iter(0_i32..5)),
        vec![ints],
    )
    .unwrap();

    let indicies = Int64Array::from(vec![0, 2, 4]);
    let array = take(&array, &indicies, None).unwrap();
    assert_eq!(array.len(), 3);
}

Expected behavior
The take kernel should take the requested values from the UnionArray.

Additional context
Here's the full backtrace of the panic:

thread 'main' panicked at /Users/mhilton/.cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-select-52.2.0/src/filter.rs:548:61:
internal error: entered unreachable code
stack backtrace:
   0:        0x102282d38 - std::backtrace_rs::backtrace::libunwind::trace::h6196ef3b0a4e77ef
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/../../backtrace/src/backtrace/libunwind.rs:116:5
   1:        0x102282d38 - std::backtrace_rs::backtrace::trace_unsynchronized::hac375eab426f00dd
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:        0x102282d38 - std::sys_common::backtrace::_print_fmt::h2a4f1f15852b7d46
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/sys_common/backtrace.rs:68:5
   3:        0x102282d38 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h205eab4daa59a56a
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/sys_common/backtrace.rs:44:22
   4:        0x10229c2d8 - core::fmt::rt::Argument::fmt::h391b30ab7a26d7be
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/fmt/rt.rs:165:63
   5:        0x10229c2d8 - core::fmt::write::ha28d0fbf1a1eeba9
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/fmt/mod.rs:1168:21
   6:        0x102281078 - std::io::Write::write_fmt::h54f6960924d6c6c0
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/io/mod.rs:1835:15
   7:        0x102282b90 - std::sys_common::backtrace::_print::hfe43851cb14e36df
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/sys_common/backtrace.rs:47:5
   8:        0x102282b90 - std::sys_common::backtrace::print::hb748761a345859f9
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/sys_common/backtrace.rs:34:9
   9:        0x102283d68 - std::panicking::default_hook::{{closure}}::h6cb9ca91838de054
  10:        0x102283a34 - std::panicking::default_hook::h83345dc5126de57b
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/panicking.rs:298:9
  11:        0x102284614 - std::panicking::rust_panic_with_hook::h9305dc63490c537b
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/panicking.rs:795:13
  12:        0x102284024 - std::panicking::begin_panic_handler::{{closure}}::h0d6c6c01f5801c1a
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/panicking.rs:656:13
  13:        0x1022831bc - std::sys_common::backtrace::__rust_end_short_backtrace::h2ace9f7ada66a76a
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/sys_common/backtrace.rs:171:18
  14:        0x102283dc0 - rust_begin_unwind
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/panicking.rs:652:5
  15:        0x1022a8188 - core::panicking::panic_fmt::hf3031f82c202a80d
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/panicking.rs:72:14
  16:        0x1022a8208 - core::panicking::panic::hfb97fcfe2d432218
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/panicking.rs:146:5
  17:        0x102169458 - arrow_select::filter::filter_native::h9f28851713a0b08f
                               at /Users/mhilton/.cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-select-52.2.0/src/filter.rs:548:61
  18:        0x102169930 - arrow_select::filter::filter_primitive::hcdf550095906b537
                               at /Users/mhilton/.cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-select-52.2.0/src/filter.rs:563:18
  19:        0x102112f88 - arrow_select::take::take_impl::{{closure}}::h239177a4c6a8bec8
                               at /Users/mhilton/.cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-select-52.2.0/src/take.rs:256:35
  20:        0x102121220 - core::iter::adapters::map::map_try_fold::{{closure}}::hab6993c506059229
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/iter/adapters/map.rs:96:28
  21:        0x102120e40 - core::iter::adapters::map::map_try_fold::{{closure}}::h3d41d09047cc2b32
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/iter/adapters/map.rs:96:21
  22:        0x10206995c - core::iter::traits::iterator::Iterator::try_fold::h72e753c0a4af1044
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/iter/traits/iterator.rs:2411:21
  23:        0x10211aef4 - <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold::h75e4ab88b8caf0c6
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/iter/adapters/map.rs:122:9
  24:        0x10211ae00 - <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold::h3043137ca45b49b3
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/iter/adapters/map.rs:122:9
  25:        0x10214eb84 - <core::iter::adapters::GenericShunt<I,R> as core::iter::traits::iterator::Iterator>::try_fold::h5e2f7ab5cb4ee7b9
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/iter/adapters/mod.rs:204:9
  26:        0x10214e530 - core::iter::traits::iterator::Iterator::try_for_each::h597277cdd977e2d5
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/iter/traits/iterator.rs:2473:9
  27:        0x10214e530 - <core::iter::adapters::GenericShunt<I,R> as core::iter::traits::iterator::Iterator>::next::h8aa312b9f15e2e1c
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/iter/adapters/mod.rs:187:14
  28:        0x102083ea0 - <alloc::vec::Vec<T> as alloc::vec::spec_from_iter_nested::SpecFromIterNested<T,I>>::from_iter::h3cd7e10537f2b4ae
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/alloc/src/vec/spec_from_iter_nested.rs:26:32
  29:        0x1020a7598 - <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter::h7b756cb59644d8d3
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/alloc/src/vec/spec_from_iter.rs:33:9
  30:        0x1020a4320 - <alloc::vec::Vec<T> as core::iter::traits::collect::FromIterator<T>>::from_iter::h41de20caa2f31fb6
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/alloc/src/vec/mod.rs:2970:9
  31:        0x10215c8c0 - core::iter::traits::iterator::Iterator::collect::h0512061a5d22f37a
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/iter/traits/iterator.rs:2005:9
  32:        0x10215c8c0 - <core::result::Result<V,E> as core::iter::traits::collect::FromIterator<core::result::Result<A,E>>>::from_iter::{{closure}}::ha43fd813a35c4443
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/result.rs:1960:51
  33:        0x1021589a8 - core::iter::adapters::try_process::h9354d10d3933a934
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/iter/adapters/mod.rs:173:17
  34:        0x10215c5fc - <core::result::Result<V,E> as core::iter::traits::collect::FromIterator<core::result::Result<A,E>>>::from_iter::h20cdeefd686d375a
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/result.rs:1960:9
  35:        0x10211dc18 - core::iter::traits::iterator::Iterator::collect::h5c9250783e455d9f
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/iter/traits/iterator.rs:2005:9
  36:        0x1020d98dc - arrow_select::take::take_impl::hba5b9c3fe0878180
                               at /Users/mhilton/.cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-select-52.2.0/src/take.rs:251:28
  37:        0x1020c7d1c - arrow_select::take::take::h3cb3133ebe3f2b5b
                               at /Users/mhilton/.cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-select-52.2.0/src/take.rs:93:5
  38:        0x102063a2c - arrow_dense_array_unreachable::main::h12476859da44638b
                               at /Users/mhilton/src/arrow-dense-array-unreachable/src/main.rs:22:17
  39:        0x102063dd0 - core::ops::function::FnOnce::call_once::ha22a648cd6586e3f
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/ops/function.rs:250:5
  40:        0x102064850 - std::sys_common::backtrace::__rust_begin_short_backtrace::hcd25cd6cbcb964af
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/sys_common/backtrace.rs:155:18
  41:        0x1020648e4 - std::rt::lang_start::{{closure}}::he4496dc4748c591b
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/rt.rs:159:18
  42:        0x10227f740 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h9ac6e053b27a4e69
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/core/src/ops/function.rs:284:13
  43:        0x10227f740 - std::panicking::try::do_call::h09e23e6341cf3ffa
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/panicking.rs:559:40
  44:        0x10227f740 - std::panicking::try::hecb687885b834f0b
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/panicking.rs:523:19
  45:        0x10227f740 - std::panic::catch_unwind::hf02037d5d7fd99df
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/panic.rs:149:14
  46:        0x10227f740 - std::rt::lang_start_internal::{{closure}}::h2cb5afb5064cd3ee
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/rt.rs:141:48
  47:        0x10227f740 - std::panicking::try::do_call::h15529d58e6f83876
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/panicking.rs:559:40
  48:        0x10227f740 - std::panicking::try::h393140c707be8ad0
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/panicking.rs:523:19
  49:        0x10227f740 - std::panic::catch_unwind::h344c598b74f77cdb
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/panic.rs:149:14
  50:        0x10227f740 - std::rt::lang_start_internal::hd3ad173a069f2ae5
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/rt.rs:141:20
  51:        0x1020648b0 - std::rt::lang_start::h6054b03865accfe8
                               at /rustc/051478957371ee0084a7c0913941d2a8c4757bb9/library/std/src/rt.rs:158:17
  52:        0x102063c78 - _main
@mhilton mhilton added the bug label Aug 7, 2024
@gstvg
Copy link
Contributor

gstvg commented Aug 7, 2024

Thanks for your report @mhilton, i will investigate

@alamb
Copy link
Contributor

alamb commented Aug 31, 2024

label_issue.py automatically added labels {'arrow'} from #6209

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants