-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[Bug](agg) fix agg with topn limit coredump with ctor same key twice #54137
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
TPC-H: Total hot run time: 34458 ms |
TPC-DS: Total hot run time: 172804 ms |
ClickBench: Total hot run time: 33.02 s |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
2 similar comments
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
HappenLee
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
wm1581066
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
…pache#54137) Problem Summary: ``` doris_be: /mnt/disk8/zhangsida/doris/thirdparty/installed/include/parallel_hashmap/phmap.h:1511: void phmap::priv::raw_hash_set<phmap::priv::FlatHashMapPolicy<unsigned char, char *>, HashCRC32<doris::vectorized::UInt8>, phmap::EqualTo<unsigned char>, doris::vectorized::Allocator_<std::pair<const unsigned char, char *>>>::constructor::operator()(Args &&...) const [Policy = phmap::priv::FlatHashMapPolicy<unsigned char, char *>, Hash = HashCRC32<doris::vectorized::UInt8>, Eq = phmap::EqualTo<unsigned char>, Alloc = doris::vectorized::Allocator_<std::pair<const unsigned char, char *>>, Args = <unsigned char &, std::nullptr_t>]: Assertion `*slot_' failed. *** Query id: 6bda59ac672d4496-bcf117ae8ce3f894 *** *** is nereids: 1 *** *** tablet id: 0 *** *** Aborted at 1753928330 (unix time) try "date -d @1753928330" if you are using GNU date *** *** Current BE git commitID: 9634891 *** *** SIGABRT unknown detail explain (@0x3ef003cad02) received by PID 3976450 (TID 3979645 OR 0x7fad84c15700) from PID 3976450; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk8/zhangsida/doris/be/src/common/signal_handler.h:420 1# 0x00007FAF620E25B0 in /lib64/libc.so.6 2# __GI_raise in /lib64/libc.so.6 3# __GI_abort in /lib64/libc.so.6 4# _nl_load_domain.cold.0 in /lib64/libc.so.6 5# 0x00007FAF620DAE86 in /lib64/libc.so.6 6# void phmap::priv::raw_hash_set<phmap::priv::FlatHashMapPolicy<unsigned char, char*>, HashCRC32<unsigned char>, phmap::EqualTo<unsigned char>, doris::vectorized::Allocator_<std::pair<unsigned char const, char*> > >::constructor::operator()<unsigned char&, decltype(nullptr)>(unsigned char&, decltype(nullptr)&&) const at /mnt/disk8/zhangsida/doris/thirdparty/installed/include/parallel_hashmap/phmap.h:1511 7# _ZNSt8__detail9__variant17__gen_vtable_implINS0_12_Multi_arrayIPFNS0_21__deduce_visit_resultIbEEON5doris10vectorized8OverloadIJZNS5_8pipeline17AggSinkLocalState30_emplace_into_hash_table_limitEPPcPNS6_5BlockERKSt6vectorIiSaIiEERSE_IPKNS6_7IColumnESaISL_EEjE3$_0ZNS9_30_emplace_into_hash_table_limitESB_SD_SI_SO_jE3$_1EEERSt7variantIJSt9monostateNS6_16MethodSerializedI9PHHashMapINS5_9StringRefESA_11DefaultHashISX_vEEEENS6_15MethodOneNumberIhSW_IhSA_9HashCRC32IhEEEENS12_ItSW_ItSA_S13_ItEEEENS12_IjSW_IjSA_S13_IjEEEENS12_ImSW_ImSA_S13_ImEEEENS6_19MethodStringNoCacheINS5_13StringHashMapISA_NS5_9AllocatorILb1ELb1ELb0ENS5_22DefaultMemoryAllocatorELb1EEEEEEENS12_IN4wide7integerILm128EjEESW_IS1P_SA_S13_IS1P_EEEENS12_INS1O_ILm256EjEESW_IS1T_SA_S13_IS1T_EEEENS12_IjSW_IjSA_14HashMixWrapperIjS1A_EEEENS12_ImSW_ImSA_S1X_ImS1D_EEEENS6_26MethodSingleNullableColumnINS12_IhNS6_15DataWithNullKeyIS15_EEEEEENS24_INS12_ItNS25_IS18_EEEEEENS24_INS12_IjNS25_IS1B_EEEEEENS24_INS12_ImNS25_IS1E_EEEEEENS24_INS12_IjNS25_IS1Z_EEEEEENS24_INS12_ImNS25_IS22_EEEEEENS24_INS12_IS1P_NS25_IS1R_EEEEEENS24_INS12_IS1T_NS25_IS1V_EEEEEENS24_INS1G_INS25_IS1L_EEEEEENS6_15MethodKeysFixedIS1E_EENS2X_IS1R_EENS2X_IS1V_EENS2X_ISW_INS6_7UInt136ESA_S13_IS31_EEEEEEEJEEESt16integer_sequenceImJLm11EEEE14__visit_invokeESS_S36_ at /mnt/disk8/zhangsida/install_data/ldb_toolchain_robin/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/variant:1032 8# doris::pipeline::AggSinkLocalState::_emplace_into_hash_table_limit(char**, doris::vectorized::Block*, std::vector<int, std::allocator<int> > const&, std::vector<doris::vectorized::IColumn const*, std::allocator<doris::vectorized::IColumn const*> >&, unsigned int) at /mnt/disk8/zhangsida/doris/be/src/pipeline/exec/aggregation_sink_operator.cpp:584 9# doris::Status doris::pipeline::AggSinkLocalState::_execute_with_serialized_key_helper<true>(doris::vectorized::Block*) at /mnt/disk8/zhangsida/doris/be/src/pipeline/exec/aggregation_sink_operator.cpp:496 10# doris::pipeline::AggSinkLocalState::_execute_with_serialized_key(doris::vectorized::Block*) at /mnt/disk8/zhangsida/doris/be/src/pipeline/exec/aggregation_sink_operator.cpp:443 11# doris::pipeline::AggSinkLocalState::Executor<false, false>::execute(doris::pipeline::AggSinkLocalState*, doris::vectorized::Block*) at /mnt/disk8/zhangsida/doris/be/src/pipeline/exec/aggregation_sink_operator.h:62 ``` ``` try { HashMethodType::try_presis_key_and_origin(key, origin, _agg_arena_pool); auto mapped = _shared_state->aggregate_data_container->append_data( origin); auto st = _create_agg_status(mapped); if (!st) { throw Exception(st.code(), st.to_string()); } ctor(key, mapped); _shared_state->refresh_top_limit(i, key_columns); } catch (...) { // Exception-safety - if it can not allocate memory or create status, // the destructors will not be called. ctor(key, nullptr); throw; } ``` when _shared_state->refresh_top_limit(i, key_columns); throw exception will exectue ctor(key, nullptr);, but before have exectue ctor(key, mapped) None - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
…pache#54137) Problem Summary: ``` doris_be: /mnt/disk8/zhangsida/doris/thirdparty/installed/include/parallel_hashmap/phmap.h:1511: void phmap::priv::raw_hash_set<phmap::priv::FlatHashMapPolicy<unsigned char, char *>, HashCRC32<doris::vectorized::UInt8>, phmap::EqualTo<unsigned char>, doris::vectorized::Allocator_<std::pair<const unsigned char, char *>>>::constructor::operator()(Args &&...) const [Policy = phmap::priv::FlatHashMapPolicy<unsigned char, char *>, Hash = HashCRC32<doris::vectorized::UInt8>, Eq = phmap::EqualTo<unsigned char>, Alloc = doris::vectorized::Allocator_<std::pair<const unsigned char, char *>>, Args = <unsigned char &, std::nullptr_t>]: Assertion `*slot_' failed. *** Query id: 6bda59ac672d4496-bcf117ae8ce3f894 *** *** is nereids: 1 *** *** tablet id: 0 *** *** Aborted at 1753928330 (unix time) try "date -d @1753928330" if you are using GNU date *** *** Current BE git commitID: 9634891 *** *** SIGABRT unknown detail explain (@0x3ef003cad02) received by PID 3976450 (TID 3979645 OR 0x7fad84c15700) from PID 3976450; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk8/zhangsida/doris/be/src/common/signal_handler.h:420 1# 0x00007FAF620E25B0 in /lib64/libc.so.6 2# __GI_raise in /lib64/libc.so.6 3# __GI_abort in /lib64/libc.so.6 4# _nl_load_domain.cold.0 in /lib64/libc.so.6 5# 0x00007FAF620DAE86 in /lib64/libc.so.6 6# void phmap::priv::raw_hash_set<phmap::priv::FlatHashMapPolicy<unsigned char, char*>, HashCRC32<unsigned char>, phmap::EqualTo<unsigned char>, doris::vectorized::Allocator_<std::pair<unsigned char const, char*> > >::constructor::operator()<unsigned char&, decltype(nullptr)>(unsigned char&, decltype(nullptr)&&) const at /mnt/disk8/zhangsida/doris/thirdparty/installed/include/parallel_hashmap/phmap.h:1511 7# _ZNSt8__detail9__variant17__gen_vtable_implINS0_12_Multi_arrayIPFNS0_21__deduce_visit_resultIbEEON5doris10vectorized8OverloadIJZNS5_8pipeline17AggSinkLocalState30_emplace_into_hash_table_limitEPPcPNS6_5BlockERKSt6vectorIiSaIiEERSE_IPKNS6_7IColumnESaISL_EEjE3$_0ZNS9_30_emplace_into_hash_table_limitESB_SD_SI_SO_jE3$_1EEERSt7variantIJSt9monostateNS6_16MethodSerializedI9PHHashMapINS5_9StringRefESA_11DefaultHashISX_vEEEENS6_15MethodOneNumberIhSW_IhSA_9HashCRC32IhEEEENS12_ItSW_ItSA_S13_ItEEEENS12_IjSW_IjSA_S13_IjEEEENS12_ImSW_ImSA_S13_ImEEEENS6_19MethodStringNoCacheINS5_13StringHashMapISA_NS5_9AllocatorILb1ELb1ELb0ENS5_22DefaultMemoryAllocatorELb1EEEEEEENS12_IN4wide7integerILm128EjEESW_IS1P_SA_S13_IS1P_EEEENS12_INS1O_ILm256EjEESW_IS1T_SA_S13_IS1T_EEEENS12_IjSW_IjSA_14HashMixWrapperIjS1A_EEEENS12_ImSW_ImSA_S1X_ImS1D_EEEENS6_26MethodSingleNullableColumnINS12_IhNS6_15DataWithNullKeyIS15_EEEEEENS24_INS12_ItNS25_IS18_EEEEEENS24_INS12_IjNS25_IS1B_EEEEEENS24_INS12_ImNS25_IS1E_EEEEEENS24_INS12_IjNS25_IS1Z_EEEEEENS24_INS12_ImNS25_IS22_EEEEEENS24_INS12_IS1P_NS25_IS1R_EEEEEENS24_INS12_IS1T_NS25_IS1V_EEEEEENS24_INS1G_INS25_IS1L_EEEEEENS6_15MethodKeysFixedIS1E_EENS2X_IS1R_EENS2X_IS1V_EENS2X_ISW_INS6_7UInt136ESA_S13_IS31_EEEEEEEJEEESt16integer_sequenceImJLm11EEEE14__visit_invokeESS_S36_ at /mnt/disk8/zhangsida/install_data/ldb_toolchain_robin/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/variant:1032 8# doris::pipeline::AggSinkLocalState::_emplace_into_hash_table_limit(char**, doris::vectorized::Block*, std::vector<int, std::allocator<int> > const&, std::vector<doris::vectorized::IColumn const*, std::allocator<doris::vectorized::IColumn const*> >&, unsigned int) at /mnt/disk8/zhangsida/doris/be/src/pipeline/exec/aggregation_sink_operator.cpp:584 9# doris::Status doris::pipeline::AggSinkLocalState::_execute_with_serialized_key_helper<true>(doris::vectorized::Block*) at /mnt/disk8/zhangsida/doris/be/src/pipeline/exec/aggregation_sink_operator.cpp:496 10# doris::pipeline::AggSinkLocalState::_execute_with_serialized_key(doris::vectorized::Block*) at /mnt/disk8/zhangsida/doris/be/src/pipeline/exec/aggregation_sink_operator.cpp:443 11# doris::pipeline::AggSinkLocalState::Executor<false, false>::execute(doris::pipeline::AggSinkLocalState*, doris::vectorized::Block*) at /mnt/disk8/zhangsida/doris/be/src/pipeline/exec/aggregation_sink_operator.h:62 ``` ``` try { HashMethodType::try_presis_key_and_origin(key, origin, _agg_arena_pool); auto mapped = _shared_state->aggregate_data_container->append_data( origin); auto st = _create_agg_status(mapped); if (!st) { throw Exception(st.code(), st.to_string()); } ctor(key, mapped); _shared_state->refresh_top_limit(i, key_columns); } catch (...) { // Exception-safety - if it can not allocate memory or create status, // the destructors will not be called. ctor(key, nullptr); throw; } ``` when _shared_state->refresh_top_limit(i, key_columns); throw exception will exectue ctor(key, nullptr);, but before have exectue ctor(key, mapped) None - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
… key twice (#54137) (#54271) ### What problem does this PR solve? Problem Summary: cherry-pick from master (#54137) ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
What problem does this PR solve?
Problem Summary:
when _shared_state->refresh_top_limit(i, key_columns); throw exception
will exectue ctor(key, nullptr);, but before have exectue ctor(key, mapped)
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)