Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fixed undefined behaviour in ThreadPool #4612

Merged
merged 6 commits into from
Mar 6, 2019
Merged

Fixed undefined behaviour in ThreadPool #4612

merged 6 commits into from
Mar 6, 2019

Conversation

alexey-milovidov
Copy link
Member

@alexey-milovidov alexey-milovidov commented Mar 6, 2019 •
edited
Loading

No changes to the previous release.

Detailed description (optional):

We lock mutex in one thread (when constructing ThreadFromGlobalPool object) and unlock in another (when shared_ptr<std::mutex> is destroyed in the task).

According to the C++ standard:

The mutex must be locked by the current thread of execution, otherwise, the behavior is undefined.

And this code failed in FreeBSD with Resource deadlock avoided message.

@alexey-milovidov alexey-milovidov requested a review from ztlpn March 6, 2019 17:57
ztlpn
ztlpn reviewed Mar 6, 2019
View reviewed changes
dbms/src/Common/ThreadPool.h
}

private:
std::shared_ptr<std::mutex> mutex; /// Object must be moveable.
/// The state used in this object and inside the thread job.
std::shared_ptr<Poco::Event> state;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe finished_event is more clear

@alexey-milovidov
Copy link
Member Author

alexey-milovidov commented Mar 6, 2019 •
edited
Loading

@ztlpn f49562b
Otherwise next query can reuse current_thread from previous query due to the fact that local ThreadPool is already joined but ThreadStatus is not destroyed and current_thread (thread-local pointer still has a value), and HTTPHandler or TCPHandler will create QueryScope for the next thread:

WARNING: ThreadSanitizer: data race (pid=35699)
  Write of size 8 at 0x7b40011b0050 by thread T968 (mutexes: write M843, write M1102110048648888344):
    #0 ProfileEvents::Counters::setParent(ProfileEvents::Counters*) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Common/ProfileEvents.h:77:20 (clickhouse+0xc16c4f1)
    #1 DB::ProcessList::insert(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::IAST const*, DB::Context&) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/ProcessList.cpp:168 (clickhouse+0xc16c4f1)
    #2 DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/executeQuery.cpp:231:59 (clickhouse+0xc249115)
    #3 DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, DB::Context&, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/executeQuery.cpp:497:30 (clickhouse+0xc24b871)
    #4 DB::HTTPHandler::processQuery(Poco::Net::HTTPServerRequest&, HTMLForm&, Poco::Net::HTTPServerResponse&, DB::HTTPHandler::Output&) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/programs/server/HTTPHandler.cpp:605:5 (clickhouse+0x64d7093)
    #5 DB::HTTPHandler::handleRequest(Poco::Net::HTTPServerRequest&, Poco::Net::HTTPServerResponse&) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/programs/server/HTTPHandler.cpp:716:9 (clickhouse+0x64da644)
    #6 Poco::Net::HTTPServerConnection::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Net/src/HTTPServerConnection.cpp:85:17 (clickhouse+0xce286dd)
    #7 Poco::Net::TCPServerConnection::start() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Net/src/TCPServerConnection.cpp:43:3 (clickhouse+0xce61eb2)
    #8 Poco::Net::TCPServerDispatcher::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Net/src/TCPServerDispatcher.cpp:114:20 (clickhouse+0xce6272a)
    #9 Poco::PooledThread::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/ThreadPool.cpp:214:14 (clickhouse+0xcf53fb1)
    #10 Poco::(anonymous namespace)::RunnableHolder::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread.cpp:43:11 (clickhouse+0xcf524df)
    #11 Poco::ThreadImpl::runnableEntry(void*) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread_STD.cpp:139:27 (clickhouse+0xcf50d4a)
    #12 decltype(std::__1::forward<void* (*)(void*)>(fp)(std::__1::forward<Poco::ThreadImpl*>(fp0))) std::__1::__invoke<void* (*)(void*), Poco::ThreadImpl*>(void* (*&&)(void*), Poco::ThreadImpl*&&) /usr/local/bin/../include/c++/v1/type_traits:4410:1 (clickhouse+0xcf52e8a)
    #13 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*>&, std::__1::__tuple_indices<2ul>) /usr/local/bin/../include/c++/v1/thread:341 (clickhouse+0xcf52e8a)
    #14 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*> >(void*) /usr/local/bin/../include/c++/v1/thread:351 (clickhouse+0xcf52e8a)

  Previous read of size 8 at 0x7b40011b0050 by thread T75:
    #0 ProfileEvents::Counters::increment(unsigned long, unsigned long) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Common/ProfileEvents.h:58:36 (clickhouse+0x64a56a0)
    #1 DB::RUsageCounters::incrementProfileEvents(DB::RUsageCounters const&, DB::RUsageCounters const&, ProfileEvents::Counters&) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Common/ThreadProfileEvents.h:104 (clickhouse+0x64a56a0)
    #2 DB::RUsageCounters::updateProfileEvents(DB::RUsageCounters&, ProfileEvents::Counters&) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Common/ThreadProfileEvents.h:115 (clickhouse+0x64a56a0)
    #3 DB::ThreadStatus::updatePerformanceCounters() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Common/ThreadStatus.cpp:95:9 (clickhouse+0x64a4dae)
    #4 DB::ThreadStatus::finalizePerformanceCounters() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/ThreadStatusExt.cpp:105:5 (clickhouse+0xc230589)
    #5 DB::ThreadStatus::detachQuery(bool, bool) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/ThreadStatusExt.cpp:132:5 (clickhouse+0xc22fb99)
    #6 DB::CurrentThread::defaultThreadDeleter() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/ThreadStatusExt.cpp:41 (clickhouse+0xc22fb99)
    #7 decltype(std::__1::forward<void (*&)()>(fp)()) std::__1::__invoke<void (*&)()>(void (*&)()) /usr/local/bin/../include/c++/v1/type_traits:4410:1 (clickhouse+0xc2324ed)
    #8 void std::__1::__invoke_void_return_wrapper<void>::__call<void (*&)()>(void (*&)()) /usr/local/bin/../include/c++/v1/__functional_base:348 (clickhouse+0xc2324ed)
    #9 std::__1::__function::__alloc_func<void (*)(), std::__1::allocator<void (*)()>, void ()>::operator()() /usr/local/bin/../include/c++/v1/functional:1527 (clickhouse+0xc2324ed)
    #10 std::__1::__function::__func<void (*)(), std::__1::allocator<void (*)()>, void ()>::operator()() /usr/local/bin/../include/c++/v1/functional:1651 (clickhouse+0xc2324ed)
    #11 std::__1::__function::__value_func<void ()>::operator()() const /usr/local/bin/../include/c++/v1/functional:1799:16 (clickhouse+0x64a47dc)
    #12 std::__1::function<void ()>::operator()() const /usr/local/bin/../include/c++/v1/functional:2347 (clickhouse+0x64a47dc)
    #13 DB::ThreadStatus::~ThreadStatus() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Common/ThreadStatus.cpp:54 (clickhouse+0x64a47dc)
    #14 ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&)::'lambda'()::operator()() const /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Common/ThreadPool.h:148:9 (clickhouse+0x64a32f5)
    #15 decltype(std::__1::forward<void>(fp)()) std::__1::__invoke<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&)::'lambda'()&>(void&&) /usr/local/bin/../include/c++/v1/type_traits:4410 (clickhouse+0x64a32f5)
    #16 void std::__1::__invoke_void_return_wrapper<void>::__call<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&)::'lambda'()&>(ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&)::'lambda'()&) /usr/local/bin/../include/c++/v1/__functional_base:348 (clickhouse+0x64a32f5)
    #17 std::__1::__function::__alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&)::'lambda'(), std::__1::allocator<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&)::'lambda'()>, void ()>::operator()() /usr/local/bin/../include/c++/v1/functional:1527 (clickhouse+0x64a32f5)
    #18 std::__1::__function::__func<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&)::'lambda'(), std::__1::allocator<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&)::'lambda'()>, void ()>::operator()() /usr/local/bin/../include/c++/v1/functional:1651 (clickhouse+0x64a32f5)
    #19 std::__1::__function::__value_func<void ()>::operator()() const /usr/local/bin/../include/c++/v1/functional:1799:16 (clickhouse+0x649e2b6)
    #20 std::__1::function<void ()>::operator()() const /usr/local/bin/../include/c++/v1/functional:2347 (clickhouse+0x649e2b6)
    #21 ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Common/ThreadPool.cpp:169 (clickhouse+0x649e2b6)
    #22 void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()::operator()() const /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Common/ThreadPool.cpp:65:73 (clickhouse+0x64a13fc)
    #23 decltype(std::__1::forward<void>(fp)()) std::__1::__invoke<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&) /usr/local/bin/../include/c++/v1/type_traits:4410 (clickhouse+0x64a13fc)
    #24 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(std::__1::tuple<void, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>&, std::__1::__tuple_indices<>) /usr/local/bin/../include/c++/v1/thread:341 (clickhouse+0x64a13fc)
    #25 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()> >(void*) /usr/local/bin/../include/c++/v1/thread:351 (clickhouse+0x64a13fc)

  Location is heap block of size 248 at 0x7b40011b0000 allocated by thread T968:
    #0 operator new(unsigned long) /home/milovidov/ClickHouse/ci/workspace/llvm/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cc:64 (clickhouse+0x63fec77)
    #1 std::__1::__libcpp_allocate(unsigned long, unsigned long) /usr/local/bin/../include/c++/v1/new:238:10 (clickhouse+0xc2319d3)
    #2 std::__1::allocator<std::__1::__shared_ptr_emplace<DB::ThreadGroupStatus, std::__1::allocator<DB::ThreadGroupStatus> > >::allocate(unsigned long, void const*) /usr/local/bin/../include/c++/v1/memory:1813 (clickhouse+0xc2319d3)
    #3 std::__1::shared_ptr<DB::ThreadGroupStatus> std::__1::shared_ptr<DB::ThreadGroupStatus>::make_shared<>() /usr/local/bin/../include/c++/v1/memory:4329 (clickhouse+0xc2319d3)
    #4 std::__1::enable_if<!(is_array<DB::ThreadGroupStatus>::value), std::__1::shared_ptr<DB::ThreadGroupStatus> >::type std::__1::make_shared<DB::ThreadGroupStatus>() /usr/local/bin/../include/c++/v1/memory:4709:12 (clickhouse+0xc22fe45)
    #5 DB::ThreadStatus::initializeQuery() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/ThreadStatusExt.cpp:48 (clickhouse+0xc22fe45)
    #6 DB::CurrentThread::initializeQuery() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/ThreadStatusExt.cpp:194:11 (clickhouse+0xc231243)
    #7 DB::CurrentThread::QueryScope::QueryScope(DB::Context&) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Interpreters/ThreadStatusExt.cpp:238 (clickhouse+0xc231243)
    #8 DB::HTTPHandler::processQuery(Poco::Net::HTTPServerRequest&, HTMLForm&, Poco::Net::HTTPServerResponse&, DB::HTTPHandler::Output&) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/programs/server/HTTPHandler.cpp:216:31 (clickhouse+0x64d3819)
    #9 DB::HTTPHandler::handleRequest(Poco::Net::HTTPServerRequest&, Poco::Net::HTTPServerResponse&) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/programs/server/HTTPHandler.cpp:716:9 (clickhouse+0x64da644)
    #10 Poco::Net::HTTPServerConnection::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Net/src/HTTPServerConnection.cpp:85:17 (clickhouse+0xce286dd)
    #11 Poco::Net::TCPServerConnection::start() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Net/src/TCPServerConnection.cpp:43:3 (clickhouse+0xce61eb2)
    #12 Poco::Net::TCPServerDispatcher::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Net/src/TCPServerDispatcher.cpp:114:20 (clickhouse+0xce6272a)
    #13 Poco::PooledThread::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/ThreadPool.cpp:214:14 (clickhouse+0xcf53fb1)
    #14 Poco::(anonymous namespace)::RunnableHolder::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread.cpp:43:11 (clickhouse+0xcf524df)
    #15 Poco::ThreadImpl::runnableEntry(void*) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread_STD.cpp:139:27 (clickhouse+0xcf50d4a)
    #16 decltype(std::__1::forward<void* (*)(void*)>(fp)(std::__1::forward<Poco::ThreadImpl*>(fp0))) std::__1::__invoke<void* (*)(void*), Poco::ThreadImpl*>(void* (*&&)(void*), Poco::ThreadImpl*&&) /usr/local/bin/../include/c++/v1/type_traits:4410:1 (clickhouse+0xcf52e8a)
    #17 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*>&, std::__1::__tuple_indices<2ul>) /usr/local/bin/../include/c++/v1/thread:341 (clickhouse+0xcf52e8a)
    #18 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*> >(void*) /usr/local/bin/../include/c++/v1/thread:351 (clickhouse+0xcf52e8a)

  Mutex M843 (0x7b8400000800) created at:
    #0 pthread_mutex_lock /home/milovidov/ClickHouse/ci/workspace/llvm/llvm/projects/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:4112 (clickhouse+0x6374b8b)
    #1 std::__1::mutex::lock() <null> (clickhouse+0x64471f5)
    #2 Poco::Util::Application::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Util/src/Application.cpp:335:8 (clickhouse+0xce738dd)
    #3 DB::Server::run() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/programs/server/Server.cpp:138:25 (clickhouse+0x6477297)
    #4 Poco::Util::ServerApplication::run(int, char**) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Util/src/ServerApplication.cpp:618:9 (clickhouse+0xce90bc8)
    #5 mainEntryClickHouseServer(int, char**) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/programs/server/Server.cpp:846:20 (clickhouse+0x6487243)
    #6 main /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/programs/main.cpp:175:12 (clickhouse+0x64026f2)

  Mutex M1102110048648888344 is already destroyed.

  Thread T968 'HTTPHandler' (tid=46483, running) created by thread T40 at:
    #0 pthread_create /home/milovidov/ClickHouse/ci/workspace/llvm/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:976 (clickhouse+0x635bd95)
    #1 std::__1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /usr/local/bin/../include/c++/v1/__threading_support:327:10 (clickhouse+0xcf52887)
    #2 std::__1::thread::thread<void* (&)(void*), Poco::ThreadImpl*, void>(void* (&)(void*), Poco::ThreadImpl*&&) /usr/local/bin/../include/c++/v1/thread:367 (clickhouse+0xcf52887)
    #3 Poco::ThreadImpl::startImpl(Poco::SharedPtr<Poco::Runnable, Poco::ReferenceCounter, Poco::ReleasePolicy<Poco::Runnable> >) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread_STD.cpp:58:53 (clickhouse+0xcf50812)
    #4 Poco::Thread::start(Poco::Runnable&) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread.cpp:116:2 (clickhouse+0xcf51ecc)
    #5 Poco::PooledThread::start(int) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/ThreadPool.cpp:88:10 (clickhouse+0xcf55a9e)
    #6 Poco::ThreadPool::getThread() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/ThreadPool.cpp:525 (clickhouse+0xcf55a9e)
    #7 Poco::ThreadPool::startWithPriority(Poco::Thread::Priority, Poco::Runnable&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/ThreadPool.cpp:429:2 (clickhouse+0xcf55e5e)
    #8 Poco::Net::TCPServerDispatcher::enqueue(Poco::Net::StreamSocket const&) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Net/src/TCPServerDispatcher.cpp:145:17 (clickhouse+0xce62d1f)
    #9 Poco::Net::TCPServer::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Net/src/TCPServer.cpp:148:21 (clickhouse+0xce618a5)
    #10 Poco::(anonymous namespace)::RunnableHolder::run() /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread.cpp:43:11 (clickhouse+0xcf524df)
    #11 Poco::ThreadImpl::runnableEntry(void*) /home/milovidov/ClickHouse/build_clang9_tsan/../contrib/poco/Foundation/src/Thread_STD.cpp:139:27 (clickhouse+0xcf50d4a)
    #12 decltype(std::__1::forward<void* (*)(void*)>(fp)(std::__1::forward<Poco::ThreadImpl*>(fp0))) std::__1::__invoke<void* (*)(void*), Poco::ThreadImpl*>(void* (*&&)(void*), Poco::ThreadImpl*&&) /usr/local/bin/../include/c++/v1/type_traits:4410:1 (clickhouse+0xcf52e8a)
    #13 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*>&, std::__1::__tuple_indices<2ul>) /usr/local/bin/../include/c++/v1/thread:341 (clickhouse+0xcf52e8a)
    #14 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void* (*)(void*), Poco::ThreadImpl*> >(void*) /usr/local/bin/../include/c++/v1/thread:351 (clickhouse+0xcf52e8a)

  Thread T75 'AsyncBlockInput' (tid=38234, running) created by thread T24 at:
    #0 pthread_create /home/milovidov/ClickHouse/ci/workspace/llvm/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:976 (clickhouse+0x635bd95)
    #1 std::__1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /usr/local/bin/../include/c++/v1/__threading_support:327:10 (clickhouse+0x64a09e1)
    #2 std::__1::thread::thread<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'(), void>(void&&) /usr/local/bin/../include/c++/v1/thread:367 (clickhouse+0x64a09e1)
    #3 void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Common/ThreadPool.cpp:65:35 (clickhouse+0x649cfd1)
    #4 ThreadPoolImpl<std::__1::thread>::scheduleOrThrow(std::__1::function<void ()>, int, unsigned long) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Common/ThreadPool.cpp:92:5 (clickhouse+0x649d8d8)
    #5 ThreadFromGlobalPool::ThreadFromGlobalPool<DB::ParallelInputsProcessor<DB::ParallelAggregatingBlockInputStream::Handler>::process()::'lambda'()>(DB::ParallelInputsProcessor<DB::ParallelAggregatingBlockInputStream::Handler>::process()::'lambda'()&&) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Common/ThreadPool.h:140:38 (clickhouse+0xc04253a)
    #6 void std::__1::allocator<ThreadFromGlobalPool>::construct<ThreadFromGlobalPool, DB::ParallelInputsProcessor<DB::ParallelAggregatingBlockInputStream::Handler>::process()::'lambda'()>(ThreadFromGlobalPool*, DB::ParallelInputsProcessor<DB::ParallelAggregatingBlockInputStream::Handler>::process()::'lambda'()&&) /usr/local/bin/../include/c++/v1/memory:1825:31 (clickhouse+0xc03dda4)
    #7 void std::__1::allocator_traits<std::__1::allocator<ThreadFromGlobalPool> >::__construct<ThreadFromGlobalPool, DB::ParallelInputsProcessor<DB::ParallelAggregatingBlockInputStream::Handler>::process()::'lambda'()>(std::__1::integral_constant<bool, true>, std::__1::allocator<ThreadFromGlobalPool>&, ThreadFromGlobalPool*, DB::ParallelInputsProcessor<DB::ParallelAggregatingBlockInputStream::Handler>::process()::'lambda'()&&) /usr/local/bin/../include/c++/v1/memory:1717 (clickhouse+0xc03dda4)
    #8 void std::__1::allocator_traits<std::__1::allocator<ThreadFromGlobalPool> >::construct<ThreadFromGlobalPool, DB::ParallelInputsProcessor<DB::ParallelAggregatingBlockInputStream::Handler>::process()::'lambda'()>(std::__1::allocator<ThreadFromGlobalPool>&, ThreadFromGlobalPool*, DB::ParallelInputsProcessor<DB::ParallelAggregatingBlockInputStream::Handler>::process()::'lambda'()&&) /usr/local/bin/../include/c++/v1/memory:1560 (clickhouse+0xc03dda4)
    #9 ThreadFromGlobalPool& std::__1::vector<ThreadFromGlobalPool, std::__1::allocator<ThreadFromGlobalPool> >::emplace_back<DB::ParallelInputsProcessor<DB::ParallelAggregatingBlockInputStream::Handler>::process()::'lambda'()>(DB::ParallelInputsProcessor<DB::ParallelAggregatingBlockInputStream::Handler>::process()::'lambda'()&&) /usr/local/bin/../include/c++/v1/vector:1687 (clickhouse+0xc03dda4)
    #10 DB::ParallelInputsProcessor<DB::ParallelAggregatingBlockInputStream::Handler>::process() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/DataStreams/ParallelInputsProcessor.h:104 (clickhouse+0xc03dda4)
    #11 DB::ParallelAggregatingBlockInputStream::execute() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/DataStreams/ParallelAggregatingBlockInputStream.cpp:174:15 (clickhouse+0xc03b808)
    #12 DB::ParallelAggregatingBlockInputStream::readImpl() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/DataStreams/ParallelAggregatingBlockInputStream.cpp:58:9 (clickhouse+0xc03a3d6)
    #13 DB::IBlockInputStream::read() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/DataStreams/IBlockInputStream.cpp:56:15 (clickhouse+0xbc41bd5)
    #14 DB::ExpressionBlockInputStream::readImpl() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/DataStreams/ExpressionBlockInputStream.cpp:33:34 (clickhouse+0xc021611)
    #15 DB::IBlockInputStream::read() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/DataStreams/IBlockInputStream.cpp:56:15 (clickhouse+0xbc41bd5)
    #16 DB::ExpressionBlockInputStream::readImpl() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/DataStreams/ExpressionBlockInputStream.cpp:33:34 (clickhouse+0xc021611)
    #17 DB::IBlockInputStream::read() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/DataStreams/IBlockInputStream.cpp:56:15 (clickhouse+0xbc41bd5)
    #18 DB::AsynchronousBlockInputStream::calculate() /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/DataStreams/AsynchronousBlockInputStream.cpp:74:34 (clickhouse+0xbc3f2c3)
    #19 DB::AsynchronousBlockInputStream::next()::$_0::operator()() const /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/DataStreams/AsynchronousBlockInputStream.cpp:59:9 (clickhouse+0xbc3fa8f)
    #20 decltype(std::__1::forward<DB::AsynchronousBlockInputStream::next()::$_0&>(fp)()) std::__1::__invoke<DB::AsynchronousBlockInputStream::next()::$_0&>(DB::AsynchronousBlockInputStream::next()::$_0&) /usr/local/bin/../include/c++/v1/type_traits:4410 (clickhouse+0xbc3fa8f)
    #21 void std::__1::__invoke_void_return_wrapper<void>::__call<DB::AsynchronousBlockInputStream::next()::$_0&>(DB::AsynchronousBlockInputStream::next()::$_0&) /usr/local/bin/../include/c++/v1/__functional_base:348 (clickhouse+0xbc3fa8f)
    #22 std::__1::__function::__alloc_func<DB::AsynchronousBlockInputStream::next()::$_0, std::__1::allocator<DB::AsynchronousBlockInputStream::next()::$_0>, void ()>::operator()() /usr/local/bin/../include/c++/v1/functional:1527 (clickhouse+0xbc3fa8f)
    #23 std::__1::__function::__func<DB::AsynchronousBlockInputStream::next()::$_0, std::__1::allocator<DB::AsynchronousBlockInputStream::next()::$_0>, void ()>::operator()() /usr/local/bin/../include/c++/v1/functional:1651 (clickhouse+0xbc3fa8f)
    #24 std::__1::__function::__value_func<void ()>::operator()() const /usr/local/bin/../include/c++/v1/functional:1799:16 (clickhouse+0x64a05b6)
    #25 std::__1::function<void ()>::operator()() const /usr/local/bin/../include/c++/v1/functional:2347 (clickhouse+0x64a05b6)
    #26 ThreadPoolImpl<ThreadFromGlobalPool>::worker(std::__1::__list_iterator<ThreadFromGlobalPool, void*>) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Common/ThreadPool.cpp:169 (clickhouse+0x64a05b6)
    #27 void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()::operator()() const /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Common/ThreadPool.cpp:65:73 (clickhouse+0x64a32d7)
    #28 decltype(std::__1::forward<void>(fp)()) std::__1::__invoke_constexpr<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'() const&>(void&&) /usr/local/bin/../include/c++/v1/type_traits:4416 (clickhouse+0x64a32d7)
    #29 decltype(auto) std::__1::__apply_tuple_impl<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'() const&, std::__1::tuple<> const&>(void&&, std::__1::tuple<> const&, std::__1::__tuple_indices<>) /usr/local/bin/../include/c++/v1/tuple:1358 (clickhouse+0x64a32d7)
    #30 decltype(auto) std::__1::apply<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'() const&, std::__1::tuple<> const&>(void&&, std::__1::tuple<> const&) /usr/local/bin/../include/c++/v1/tuple:1367 (clickhouse+0x64a32d7)
    #31 ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&)::'lambda'()::operator()() const /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Common/ThreadPool.h:146 (clickhouse+0x64a32d7)
    #32 decltype(std::__1::forward<void>(fp)()) std::__1::__invoke<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&)::'lambda'()&>(void&&) /usr/local/bin/../include/c++/v1/type_traits:4410 (clickhouse+0x64a32d7)
    #33 void std::__1::__invoke_void_return_wrapper<void>::__call<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&)::'lambda'()&>(ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&)::'lambda'()&) /usr/local/bin/../include/c++/v1/__functional_base:348 (clickhouse+0x64a32d7)
    #34 std::__1::__function::__alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&)::'lambda'(), std::__1::allocator<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&)::'lambda'()>, void ()>::operator()() /usr/local/bin/../include/c++/v1/functional:1527 (clickhouse+0x64a32d7)
    #35 std::__1::__function::__func<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&)::'lambda'(), std::__1::allocator<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&)::'lambda'()>, void ()>::operator()() /usr/local/bin/../include/c++/v1/functional:1651 (clickhouse+0x64a32d7)
    #36 std::__1::__function::__value_func<void ()>::operator()() const /usr/local/bin/../include/c++/v1/functional:1799:16 (clickhouse+0x649e2b6)
    #37 std::__1::function<void ()>::operator()() const /usr/local/bin/../include/c++/v1/functional:2347 (clickhouse+0x649e2b6)
    #38 ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Common/ThreadPool.cpp:169 (clickhouse+0x649e2b6)
    #39 void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()::operator()() const /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Common/ThreadPool.cpp:65:73 (clickhouse+0x64a13fc)
    #40 decltype(std::__1::forward<void>(fp)()) std::__1::__invoke<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&) /usr/local/bin/../include/c++/v1/type_traits:4410 (clickhouse+0x64a13fc)
    #41 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(std::__1::tuple<void, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>&, std::__1::__tuple_indices<>) /usr/local/bin/../include/c++/v1/thread:341 (clickhouse+0x64a13fc)
    #42 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()> >(void*) /usr/local/bin/../include/c++/v1/thread:351 (clickhouse+0x64a13fc)

SUMMARY: ThreadSanitizer: data race /home/milovidov/ClickHouse/build_clang9_tsan/../dbms/src/Common/ProfileEvents.h:77:20 in ProfileEvents::Counters::setParent(ProfileEvents::Counters*)=

@alexey-milovidov alexey-milovidov merged commit b403419 into master Mar 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ztlpn ztlpn ztlpn approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@alexey-milovidov @ztlpn @proller