-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[bugfix](core) child block is shared between operator and node, it should be shared ptr #28106
Conversation
clang-tidy review says "All clean, LGTM! 👍" |
…ould be shared ptr related with apache#28106
run buildall |
(From new machine)TeamCity pipeline, clickbench performance test result: |
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
|
run buildall |
clang-tidy review says "All clean, LGTM! 👍" |
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
|
(From new machine)TeamCity pipeline, clickbench performance test result: |
PR approved by at least one committer and no changes requested. |
PR approved by anyone and no changes requested. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
…ould be shared ptr (apache#28106) _child_block in nest loop join , table value function, repeat node will be shared between ExecNode and related operator, but it should not be a unique ptr in operator, it belongs to exec node. It will double free the block, if operator's close method is not called correctly. It should be a shared ptr, then it will not core even if the opeartor's close method is not called.
Proposed changes
_child_block in nest loop join , table value function, repeat node will be shared between ExecNode and related operator, but it should not be a unique ptr in operator, it belongs to exec node.
It will double free the block, if operator's close method is not called correctly.
It should be a shared ptr, then it will not core even if the opeartor's close method is not called.
==30143==ERROR: AddressSanitizer: heap-use-after-free on address 0x61b0029a31d8 at pc 0x557c0b9cbf45 bp 0x7f3032029ed0 sp 0x7f3032029ec8
00:03:16 READ of size 8 at 0x61b0029a31d8 thread T284 (WithoutGroupTas)
00:03:16 #0 0x557c0b9cbf44 in std::_Bvector_base<std::allocator >::_M_deallocate() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_bvector.h:555:23
00:03:16 #1 0x557c0b9cbf44 in std::_Bvector_base<std::allocator >::~_Bvector_base() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_bvector.h:543:15
00:03:16 #2 0x557c0b9cbf44 in doris::vectorized::Block::~Block() /root/doris/be/src/vec/core/block.h:71:7
00:03:16 #3 0x557c0b9cbf44 in std::default_deletedoris::vectorized::Block::operator()(doris::vectorized::Block*) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:85:2
00:03:16 #4 0x557c0b9cbf44 in std::unique_ptr<doris::vectorized::Block, std::default_deletedoris::vectorized::Block >::
unique_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:361:4__shared_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1149:3100:03:16 #5 0x557c2b94d409 in doris::pipeline::StatefulOperatordoris::pipeline::NestLoopJoinProbeOperatorBuilder::
StatefulOperator() /root/doris/be/src/pipeline/exec/operator.h:441:41__shared_count() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:702:1100:03:16 #6 0x557c2b94d409 in void std::destroy_atdoris::pipeline::NestLoopJoinProbeOperator(doris::pipeline::NestLoopJoinProbeOperator*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15
00:03:16 #7 0x557c2b94d409 in void std::allocator_traits<std::allocatordoris::pipeline::NestLoopJoinProbeOperator >::destroydoris::pipeline::NestLoopJoinProbeOperator(std::allocatordoris::pipeline::NestLoopJoinProbeOperator&, doris::pipeline::NestLoopJoinProbeOperator*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:533:4
00:03:16 #8 0x557c2b94d409 in std::_Sp_counted_ptr_inplace<doris::pipeline::NestLoopJoinProbeOperator, std::allocatordoris::pipeline::NestLoopJoinProbeOperator, (__gnu_cxx::_Lock_policy)2>::_M_dispose() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:528:2
00:03:16 #9 0x557c2b8bae0d in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:168:6
00:03:16 #10 0x557c2b8bae0d in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::
00:03:16 #11 0x557c2b8bae0d in std::__shared_ptr<doris::pipeline::OperatorBase, (__gnu_cxx::_Lock_policy)2>::
00:03:16 #12 0x557c2b8bae0d in void std::destroy_at<std::shared_ptrdoris::pipeline::OperatorBase >(std::shared_ptrdoris::pipeline::OperatorBase) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15
00:03:16 #13 0x557c2b8bae0d in void std::_Destroy<std::shared_ptrdoris::pipeline::OperatorBase >(std::shared_ptrdoris::pipeline::OperatorBase) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:138:7
00:03:16 #14 0x557c2b8bae0d in void std::_Destroy_aux::__destroy<std::shared_ptrdoris::pipeline::OperatorBase>(std::shared_ptrdoris::pipeline::OperatorBase, std::shared_ptrdoris::pipeline::OperatorBase) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:152:6
00:03:16 #15 0x557c2b8bae0d in void std::_Destroy<std::shared_ptrdoris::pipeline::OperatorBase>(std::shared_ptrdoris::pipeline::OperatorBase, std::shared_ptrdoris::pipeline::OperatorBase) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:184:7
00:03:16 #16 0x557c2b8bae0d in void std::_Destroy<std::shared_ptrdoris::pipeline::OperatorBase, std::shared_ptrdoris::pipeline::OperatorBase >(std::shared_ptrdoris::pipeline::OperatorBase, std::shared_ptrdoris::pipeline::OperatorBase, std::allocator<std::shared_ptrdoris::pipeline::OperatorBase >&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:746:7
00:03:16 #17 0x557c2b8bae0d in std::vector<std::shared_ptrdoris::pipeline::OperatorBase, std::allocator<std::shared_ptrdoris::pipeline::OperatorBase > >::
vector() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:680:2__shared_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1149:3100:03:16 #18 0x557c2b8bae0d in doris::pipeline::Pipeline::
Pipeline() /root/doris/be/src/pipeline/pipeline.h:46:7__shared_count() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:702:1100:03:16 #19 0x557c2b87bf30 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:168:6
00:03:16 #20 0x557c2b87bf30 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::
00:03:16 #21 0x557c2b87bf30 in std::__shared_ptr<doris::pipeline::Pipeline, (__gnu_cxx::_Lock_policy)2>::
00:03:16 #22 0x557c2b87bf30 in void std::destroy_at<std::shared_ptrdoris::pipeline::Pipeline >(std::shared_ptrdoris::pipeline::Pipeline) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15
00:03:16 #23 0x557c2b87bf30 in void std::_Destroy<std::shared_ptrdoris::pipeline::Pipeline >(std::shared_ptrdoris::pipeline::Pipeline) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:138:7
00:03:16 #24 0x557c2b87bf30 in void std::_Destroy_aux::__destroy<std::shared_ptrdoris::pipeline::Pipeline>(std::shared_ptrdoris::pipeline::Pipeline, std::shared_ptrdoris::pipeline::Pipeline) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:152:6
00:03:16 #25 0x557c2b87bf30 in void std::_Destroy<std::shared_ptrdoris::pipeline::Pipeline>(std::shared_ptrdoris::pipeline::Pipeline, std::shared_ptrdoris::pipeline::Pipeline) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:184:7
00:03:16 #26 0x557c2b87bf30 in void std::_Destroy<std::shared_ptrdoris::pipeline::Pipeline, std::shared_ptrdoris::pipeline::Pipeline >(std::shared_ptrdoris::pipeline::Pipeline, std::shared_ptrdoris::pipeline::Pipeline, std::allocator<std::shared_ptrdoris::pipeline::Pipeline >&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:746:7
00:03:16 #27 0x557c2b87bf30 in std::vector<std::shared_ptrdoris::pipeline::Pipeline, std::allocator<std::shared_ptrdoris::pipeline::Pipeline > >::
vector() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:680:2__shared_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1149:3100:03:16 #28 0x557c2b87bf30 in doris::pipeline::PipelineFragmentContext::
PipelineFragmentContext() /root/doris/be/src/pipeline/pipeline_fragment_context.cpp:148:1__shared_count() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:702:1100:03:16 #29 0x557c2b9aecd7 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:168:6
00:03:16 #30 0x557c2b9aecd7 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::
00:03:16 #31 0x557c2b9aecd7 in std::__shared_ptr<doris::pipeline::PipelineFragmentContext, (__gnu_cxx::_Lock_policy)2>::
00:03:16 #32 0x557c2b9aecd7 in doris::pipeline::TaskScheduler::_try_close_task(doris::pipeline::PipelineTask*, doris::pipeline::PipelineTaskState) /root/doris/be/src/pipeline/task_scheduler.cpp:359:1
00:03:16 #33 0x557c2b9ac721 in doris::pipeline::TaskScheduler::_do_work(unsigned long) /root/doris/be/src/pipeline/task_scheduler.cpp:242:13
00:03:16 #34 0x557c0c61772a in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:533:24
00:03:16 #35 0x557c0c5f825d in std::function<void ()>::operator()() const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9
00:03:16 #36 0x557c0c5f825d in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:498:5
00:03:16 #37 0x7f32137f2608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8
00:03:16 #38 0x7f3213a9f132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95
00:03:16
00:03:16 0x61b0029a31d8 is located 1112 bytes inside of 1472-byte region [0x61b0029a2d80,0x61b0029a3340)
00:03:16 freed by thread T284 (WithoutGroupTas) here:
00:03:16 #0 0x557c09fdb80d in operator delete(void*) (/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0xe8c780d) (BuildId: 3c7aebf9dab039d1)
00:03:16 #1 0x557c0c153a61 in doris::ObjectPool::clear() /root/doris/be/src/common/object_pool.h:57:13
00:03:16 #2 0x557c0c153a61 in doris::RuntimeState::
RuntimeState() /root/doris/be/src/runtime/runtime_state.cpp:191:16__shared_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1149:3100:03:16 #3 0x557c2b87b41d in std::default_deletedoris::RuntimeState::operator()(doris::RuntimeState*) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:85:2
00:03:16 #4 0x557c2b87b41d in std::__uniq_ptr_impl<doris::RuntimeState, std::default_deletedoris::RuntimeState >::reset(doris::RuntimeState*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:182:4
00:03:16 #5 0x557c2b87b41d in std::unique_ptr<doris::RuntimeState, std::default_deletedoris::RuntimeState >::reset(doris::RuntimeState*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:456:7
00:03:16 #6 0x557c2b87b41d in doris::pipeline::PipelineFragmentContext::
PipelineFragmentContext() /root/doris/be/src/pipeline/pipeline_fragment_context.cpp:143:24__shared_count() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:702:1100:03:16 #7 0x557c2b9aecd7 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:168:6
00:03:16 #8 0x557c2b9aecd7 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::
00:03:16 #9 0x557c2b9aecd7 in std::__shared_ptr<doris::pipeline::PipelineFragmentContext, (__gnu_cxx::_Lock_policy)2>::
00:03:16 #10 0x557c2b9aecd7 in doris::pipeline::TaskScheduler::_try_close_task(doris::pipeline::PipelineTask*, doris::pipeline::PipelineTaskState) /root/doris/be/src/pipeline/task_scheduler.cpp:359:1
00:03:16 #11 0x557c2b9ac721 in doris::pipeline::TaskScheduler::_do_work(unsigned long) /root/doris/be/src/pipeline/task_scheduler.cpp:242:13
00:03:16 #12 0x557c0c61772a in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:533:24
00:03:16 #13 0x557c0c5f825d in std::function<void ()>::operator()() const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9
00:03:16 #14 0x557c0c5f825d in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:498:5
00:03:16 #15 0x7f32137f2608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8
Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...