Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deadlock when shutdown with more than one running http request and OTLP HTTP exporter #1548

Closed
owent opened this issue Aug 9, 2022 · 0 comments · Fixed by #1549
Closed
Labels
bug Something isn't working

Comments

@owent
Copy link
Member

owent commented Aug 9, 2022

Describe your environment

OS: CentOS8
Compiler: GCC 8.4.1
Version: v1.5.0

Steps to reproduce

Using http exporter and shutdown when more than one running request.

What is the expected behavior?

Shutdown after timeout.

What is the actual behavior?

Infinitely waiting for session closing.

Additional context

(gdb) bt
#0  0x00007f73df4989aa in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00000000004cbd83 in __gthread_cond_timedwait (__abs_timeout=0x7fffa6547f20, __mutex=0x1165968, __cond=0x1165990) at /usr/include/c++/8/x86_64-redhat-linux/bits/gthr-default.h:871
#2  std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=<synthetic pointer>..., this=0x1165990) at /usr/include/c++/8/condition_variable:178
#3  std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=<synthetic pointer>..., this=0x1165990) at /usr/include/c++/8/condition_variable:106
#4  std::condition_variable::wait_for<long, std::ratio<1l, 1000000000l> > (__rtime=..., __lock=<synthetic pointer>..., this=0x1165990) at /usr/include/c++/8/condition_variable:143
#5  opentelemetry::v1::exporter::otlp::OtlpHttpClient::ForceFlush (this=this@entry=0x1165840, timeout=..., timeout@entry=...)
    at /data/home/owent/workspace/owent-contrib/opentelemetry-cpp-ecosystem/third_party/packages/opentelemetry-cpp-v1.5.0/exporters/otlp/src/otlp_http_client.cc:785
#6  0x00000000004cdae4 in opentelemetry::v1::exporter::otlp::OtlpHttpClient::Shutdown (this=0x1165840, timeout=...)
    at /data/home/owent/workspace/owent-contrib/opentelemetry-cpp-ecosystem/third_party/packages/opentelemetry-cpp-v1.5.0/exporters/otlp/src/otlp_http_client.cc:809
#7  0x00000000005bba7c in opentelemetry::v1::sdk::trace::MultiSpanProcessor::Shutdown (timeout=..., this=<optimized out>)
    at /data/home/owent/workspace/owent-contrib/opentelemetry-cpp-ecosystem/third_party/packages/opentelemetry-cpp-v1.5.0/sdk/include/opentelemetry/sdk/trace/multi_span_processor.h:131
#8  opentelemetry::v1::sdk::trace::TracerContext::Shutdown (this=<optimized out>) at /data/home/owent/workspace/owent-contrib/opentelemetry-cpp-ecosystem/third_party/packages/opentelemetry-cpp-v1.5.0/sdk/src/trace/tracer_context.cc:58
#9  0x00000000005b6967 in opentelemetry::v1::sdk::trace::TracerProvider::~TracerProvider (this=0x11655b0, __in_chrg=<optimized out>) at /usr/include/c++/8/bits/shared_ptr_base.h:1010
#10 0x00000000005b6ac9 in opentelemetry::v1::sdk::trace::TracerProvider::~TracerProvider (this=0x11655b0, __in_chrg=<optimized out>)
    at /data/home/owent/workspace/owent-contrib/opentelemetry-cpp-ecosystem/third_party/packages/opentelemetry-cpp-v1.5.0/sdk/src/trace/tracer_provider.cc:39
#11 0x00000000004bdb6f in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x1165000) at /usr/include/c++/8/bits/shared_ptr_base.h:148
#12 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x1165000) at /usr/include/c++/8/bits/shared_ptr_base.h:148
#13 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/8/bits/shared_ptr_base.h:728
#14 std::__shared_ptr<opentelemetry::v1::trace::TracerProvider, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/8/bits/shared_ptr_base.h:1167
#15 std::shared_ptr<opentelemetry::v1::trace::TracerProvider>::~shared_ptr (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/8/bits/shared_ptr.h:103
#16 0x00007f73de83060c in __run_exit_handlers () from /lib64/libc.so.6
#17 0x00007f73de830740 in exit () from /lib64/libc.so.6
#18 0x00007f73de819f9a in __libc_start_main () from /lib64/libc.so.6
#19 0x00000000004ba8ce in _start () at /usr/include/c++/8/ext/atomicity.h:69
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
1 participant