Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

quic integration test Asan violation #32340

Open
kyessenov opened this issue Feb 12, 2024 · 5 comments
Open

quic integration test Asan violation #32340

kyessenov opened this issue Feb 12, 2024 · 5 comments
Labels
area/test flakes bug no stalebot Disables stalebot from closing an issue

Comments

@kyessenov
Copy link
Contributor

[ RUN      ] QuicHttpIntegrationTests/QuicHttpIntegrationTest.MultipleQuicConnectionsNoBPF/IPv4
source/common/http/http1/codec_impl.cc:1067:9: runtime error: member call on address 0x61c0001a2880 which does not point to an object of type 'Envoy::Http::Http1::ServerConnectionImpl'
0x61c0001a2880: note: object has invalid vptr
 b5 00 00 00  12 8a 0c 00 00 00 00 00  be be be be be be be be  b8 d8 44 0f 00 00 00 00  e8 7d 95 0f
              ^~~~~~~~~~~~~~~~~~~~~~~
              invalid vptr
    #0 0xa6843a6 in void std::__1::__invoke_void_return_wrapper<void, true>::__call<Envoy::Http::Http1::ServerConnectionImpl::ServerConnectionImpl(Envoy::Network::Connection&, Envoy::Http::Http1::CodecStats&, Envoy::Http::ServerConnectionCallbacks&, Envoy::Http::Http1Settings const&, unsigned int, unsigned int, envoy::config::core::v3::HttpProtocolOptions_HeadersWithUnderscoresAction, Envoy::Server::OverloadManager&)::$_5&, Envoy::Buffer::OwnedBufferFragmentImpl const*>(Envoy::Http::Http1::ServerConnectionImpl::ServerConnectionImpl(Envoy::Network::Connection&, Envoy::Http::Http1::CodecStats&, Envoy::Http::ServerConnectionCallbacks&, Envoy::Http::Http1Settings const&, unsigned int, unsigned int, envoy::config::core::v3::HttpProtocolOptions_HeadersWithUnderscoresAction, Envoy::Server::OverloadManager&)::$_5&, Envoy::Buffer::OwnedBufferFragmentImpl const*&&) /proc/self/cwd/source/common/http/http1/codec_impl.cc:1067:9
    #1 0xa68408a in std::__1::__function::__func<Envoy::Http::Http1::ServerConnectionImpl::ServerConnectionImpl(Envoy::Network::Connection&, Envoy::Http::Http1::CodecStats&, Envoy::Http::ServerConnectionCallbacks&, Envoy::Http::Http1Settings const&, unsigned int, unsigned int, envoy::config::core::v3::HttpProtocolOptions_HeadersWithUnderscoresAction, Envoy::Server::OverloadManager&)::$_5, std::__1::allocator<Envoy::Http::Http1::ServerConnectionImpl::ServerConnectionImpl(Envoy::Network::Connection&, Envoy::Http::Http1::CodecStats&, Envoy::Http::ServerConnectionCallbacks&, Envoy::Http::Http1Settings const&, unsigned int, unsigned int, envoy::config::core::v3::HttpProtocolOptions_HeadersWithUnderscoresAction, Envoy::Server::OverloadManager&)::$_5>, void (Envoy::Buffer::OwnedBufferFragmentImpl const*)>::operator()(Envoy::Buffer::OwnedBufferFragmentImpl const*&&) /opt/llvm/bin/../include/c++/v1/__functional/function.h:180:16
    #2 0xa691600 in std::__1::__function::__value_func<void (Envoy::Buffer::OwnedBufferFragmentImpl const*)>::operator()(Envoy::Buffer::OwnedBufferFragmentImpl const*&&) const /opt/llvm/bin/../include/c++/v1/__functional/function.h:507:16
    #3 0xa6911bd in Envoy::Buffer::OwnedBufferFragmentImpl::done() /opt/llvm/bin/../include/c++/v1/__functional/function.h:1184:12

https://dev.azure.com/cncf/envoy/_build/results?buildId=162155&view=logs&j=1439b9f7-a348-5b50-b5fe-ea612ea91241&t=1002ac43-da84-5fae-70b2-98833b702d09

@kyessenov kyessenov added bug triage Issue requires triage area/test flakes and removed triage Issue requires triage labels Feb 12, 2024
@kyessenov
Copy link
Contributor Author

CC @danzh2010

Copy link

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale stalebot believes this issue/PR has not been touched recently label Mar 14, 2024
@kyessenov kyessenov added no stalebot Disables stalebot from closing an issue and removed stale stalebot believes this issue/PR has not been touched recently labels Mar 14, 2024
@danzh2010
Copy link
Contributor

danzh2010 commented Mar 15, 2024

The ASAN stack doesn't contain QUIC code:

[ RUN      ] QuicHttpIntegrationTests/QuicHttpIntegrationTest.MultipleQuicConnectionsNoBPF/IPv4
source/common/http/http1/codec_impl.cc:1067:9: runtime error: member call on address 0x61c0001a2880 which does not point to an object of type 'Envoy::Http::Http1::ServerConnectionImpl'
0x61c0001a2880: note: object has invalid vptr
 b5 00 00 00  12 8a 0c 00 00 00 00 00  be be be be be be be be  b8 d8 44 0f 00 00 00 00  e8 7d 95 0f
              ^~~~~~~~~~~~~~~~~~~~~~~
              invalid vptr
    #0 0xa6843a6 in void std::__1::__invoke_void_return_wrapper<void, true>::__call<Envoy::Http::Http1::ServerConnectionImpl::ServerConnectionImpl(Envoy::Network::Connection&, Envoy::Http::Http1::CodecStats&, Envoy::Http::ServerConnectionCallbacks&, Envoy::Http::Http1Settings const&, unsigned int, unsigned int, envoy::config::core::v3::HttpProtocolOptions_HeadersWithUnderscoresAction, Envoy::Server::OverloadManager&)::$_5&, Envoy::Buffer::OwnedBufferFragmentImpl const*>(Envoy::Http::Http1::ServerConnectionImpl::ServerConnectionImpl(Envoy::Network::Connection&, Envoy::Http::Http1::CodecStats&, Envoy::Http::ServerConnectionCallbacks&, Envoy::Http::Http1Settings const&, unsigned int, unsigned int, envoy::config::core::v3::HttpProtocolOptions_HeadersWithUnderscoresAction, Envoy::Server::OverloadManager&)::$_5&, Envoy::Buffer::OwnedBufferFragmentImpl const*&&) /proc/self/cwd/source/common/http/http1/codec_impl.cc:1067:9
    #1 0xa68408a in std::__1::__function::__func<Envoy::Http::Http1::ServerConnectionImpl::ServerConnectionImpl(Envoy::Network::Connection&, Envoy::Http::Http1::CodecStats&, Envoy::Http::ServerConnectionCallbacks&, Envoy::Http::Http1Settings const&, unsigned int, unsigned int, envoy::config::core::v3::HttpProtocolOptions_HeadersWithUnderscoresAction, Envoy::Server::OverloadManager&)::$_5, std::__1::allocator<Envoy::Http::Http1::ServerConnectionImpl::ServerConnectionImpl(Envoy::Network::Connection&, Envoy::Http::Http1::CodecStats&, Envoy::Http::ServerConnectionCallbacks&, Envoy::Http::Http1Settings const&, unsigned int, unsigned int, envoy::config::core::v3::HttpProtocolOptions_HeadersWithUnderscoresAction, Envoy::Server::OverloadManager&)::$_5>, void (Envoy::Buffer::OwnedBufferFragmentImpl const*)>::operator()(Envoy::Buffer::OwnedBufferFragmentImpl const*&&) /opt/llvm/bin/../include/c++/v1/__functional/function.h:180:16
    #2 0xa691600 in std::__1::__function::__value_func<void (Envoy::Buffer::OwnedBufferFragmentImpl const*)>::operator()(Envoy::Buffer::OwnedBufferFragmentImpl const*&&) const /opt/llvm/bin/../include/c++/v1/__functional/function.h:507:16
    #3 0xa6911bd in Envoy::Buffer::OwnedBufferFragmentImpl::done() /opt/llvm/bin/../include/c++/v1/__functional/function.h:1184:12
    #4 0xe4cff88 in void std::__1::__invoke_void_return_wrapper<void, true>::__call<Envoy::Buffer::Slice::Slice(Envoy::Buffer::BufferFragment&)::'lambda'()&>(Envoy::Buffer::Slice::Slice(Envoy::Buffer::BufferFragment&)::'lambda'()&) /proc/self/cwd/./source/common/buffer/buffer_impl.h:95:42
    #5 0xe4cfed8 in std::__1::__function::__alloc_func<Envoy::Buffer::Slice::Slice(Envoy::Buffer::BufferFragment&)::'lambda'(), std::__1::allocator<Envoy::Buffer::Slice::Slice(Envoy::Buffer::BufferFragment&)::'lambda'()>, void ()>::operator()() /opt/llvm/bin/../include/c++/v1/__functional/function.h:180:16
    #6 0xe4cf653 in std::__1::__function::__func<Envoy::Buffer::Slice::Slice(Envoy::Buffer::BufferFragment&)::'lambda'(), std::__1::allocator<Envoy::Buffer::Slice::Slice(Envoy::Buffer::BufferFragment&)::'lambda'()>, void ()>::operator()() /opt/llvm/bin/../include/c++/v1/__functional/function.h:354:12
    #7 0x59d93a9 in std::__1::__function::__value_func<void ()>::operator()() const /opt/llvm/bin/../include/c++/v1/__functional/function.h:507:16
    #8 0xa7b3e14 in Envoy::Buffer::Slice::operator=(Envoy::Buffer::Slice&&) /opt/llvm/bin/../include/c++/v1/__functional/function.h:1184:12
    #9 0xe4c7850 in Envoy::Buffer::SliceDeque::pop_front() /proc/self/cwd/./source/common/buffer/buffer_impl.h:506:13
    #10 0xe4b5c33 in Envoy::Buffer::OwnedImpl::drainImpl(unsigned long) /proc/self/cwd/source/common/buffer/buffer_impl.cc:181:13
    #11 0xe4b5796 in Envoy::Buffer::OwnedImpl::drain(unsigned long) /proc/self/cwd/source/common/buffer/buffer_impl.cc:160:40
    #12 0xc31d37a in Envoy::Buffer::WatermarkBuffer::drain(unsigned long) /proc/self/cwd/source/common/buffer/watermark_buffer.cc:55:14
    #13 0xcb36028 in Envoy::Network::IoSocketHandleImpl::write(Envoy::Buffer::Instance&) /proc/self/cwd/source/common/network/io_socket_handle_impl.cc:140:12
    #14 0xc66d4bd in Envoy::Network::RawBufferSocket::doWrite(Envoy::Buffer::Instance&, bool) /proc/self/cwd/source/common/network/raw_buffer_socket.cc:67:61
    #15 0xc2d8a34 in Envoy::Network::ConnectionImpl::onWriteReady() /proc/self/cwd/source/common/network/connection_impl.cc:765:40
    #16 0xc2d4c2f in Envoy::Network::ConnectionImpl::onFileEvent(unsigned int) /proc/self/cwd/source/common/network/connection_impl.cc:644:5
    #17 0xc2f95c3 in void std::__1::__invoke_void_return_wrapper<void, true>::__call<Envoy::Network::ConnectionImpl::ConnectionImpl(Envoy::Event::Dispatcher&, std::__1::unique_ptr<Envoy::Network::ConnectionSocket, std::__1::default_delete<Envoy::Network::ConnectionSocket> >&&, std::__1::unique_ptr<Envoy::Network::TransportSocket, std::__1::default_delete<Envoy::Network::TransportSocket> >&&, Envoy::StreamInfo::StreamInfo&, bool)::$_6&, unsigned int>(Envoy::Network::ConnectionImpl::ConnectionImpl(Envoy::Event::Dispatcher&, std::__1::unique_ptr<Envoy::Network::ConnectionSocket, std::__1::default_delete<Envoy::Network::ConnectionSocket> >&&, std::__1::unique_ptr<Envoy::Network::TransportSocket, std::__1::default_delete<Envoy::Network::TransportSocket> >&&, Envoy::StreamInfo::StreamInfo&, bool)::$_6&, unsigned int&&) /proc/self/cwd/source/common/network/connection_impl.cc:101:54
    #18 0xc2f92ba in std::__1::__function::__func<Envoy::Network::ConnectionImpl::ConnectionImpl(Envoy::Event::Dispatcher&, std::__1::unique_ptr<Envoy::Network::ConnectionSocket, std::__1::default_delete<Envoy::Network::ConnectionSocket> >&&, std::__1::unique_ptr<Envoy::Network::TransportSocket, std::__1::default_delete<Envoy::Network::TransportSocket> >&&, Envoy::StreamInfo::StreamInfo&, bool)::$_6, std::__1::allocator<Envoy::Network::ConnectionImpl::ConnectionImpl(Envoy::Event::Dispatcher&, std::__1::unique_ptr<Envoy::Network::ConnectionSocket, std::__1::default_delete<Envoy::Network::ConnectionSocket> >&&, std::__1::unique_ptr<Envoy::Network::TransportSocket, std::__1::default_delete<Envoy::Network::TransportSocket> >&&, Envoy::StreamInfo::StreamInfo&, bool)::$_6>, void (unsigned int)>::operator()(unsigned int&&) /opt/llvm/bin/../include/c++/v1/__functional/function.h:180:16
    #19 0x6aa9340 in std::__1::__function::__value_func<void (unsigned int)>::operator()(unsigned int&&) const /opt/llvm/bin/../include/c++/v1/__functional/function.h:507:16
    #20 0xc237b4d in decltype(static_cast<Envoy::Event::DispatcherImpl::createFileEvent(int, std::__1::function<void (unsigned int)>, Envoy::Event::FileTriggerType, unsigned int)::$_5&>(fp)(static_cast<unsigned int>(fp0))) std::__1::__invoke<Envoy::Event::DispatcherImpl::createFileEvent(int, std::__1::function<void (unsigned int)>, Envoy::Event::FileTriggerType, unsigned int)::$_5&, unsigned int>(Envoy::Event::DispatcherImpl::createFileEvent(int, std::__1::function<void (unsigned int)>, Envoy::Event::FileTriggerType, unsigned int)::$_5&, unsigned int&&) /opt/llvm/bin/../include/c++/v1/__functional/function.h:1184:12
    #21 0xc2379d0 in void std::__1::__invoke_void_return_wrapper<void, true>::__call<Envoy::Event::DispatcherImpl::createFileEvent(int, std::__1::function<void (unsigned int)>, Envoy::Event::FileTriggerType, unsigned int)::$_5&, unsigned int>(Envoy::Event::DispatcherImpl::createFileEvent(int, std::__1::function<void (unsigned int)>, Envoy::Event::FileTriggerType, unsigned int)::$_5&, unsigned int&&) /opt/llvm/bin/../include/c++/v1/__functional/invoke.h:61:9
    #22 0xc2376ba in std::__1::__function::__func<Envoy::Event::DispatcherImpl::createFileEvent(int, std::__1::function<void (unsigned int)>, Envoy::Event::FileTriggerType, unsigned int)::$_5, std::__1::allocator<Envoy::Event::DispatcherImpl::createFileEvent(int, std::__1::function<void (unsigned int)>, Envoy::Event::FileTriggerType, unsigned int)::$_5>, void (unsigned int)>::operator()(unsigned int&&) /opt/llvm/bin/../include/c++/v1/__functional/function.h:180:16
    #23 0x6aa9340 in std::__1::__function::__value_func<void (unsigned int)>::operator()(unsigned int&&) const /opt/llvm/bin/../include/c++/v1/__functional/function.h:507:16
    #24 0xc250dec in Envoy::Event::FileEventImpl::mergeInjectedEventsAndRunCb(unsigned int) /opt/llvm/bin/../include/c++/v1/__functional/function.h:1184:12
    #25 0xc252a64 in decltype(static_cast<Envoy::Event::FileEventImpl::FileEventImpl(Envoy::Event::DispatcherImpl&, int, std::__1::function<void (unsigned int)>, Envoy::Event::FileTriggerType, unsigned int)::$_0&>(fp)()) std::__1::__invoke<Envoy::Event::FileEventImpl::FileEventImpl(Envoy::Event::DispatcherImpl&, int, std::__1::function<void (unsigned int)>, Envoy::Event::FileTriggerType, unsigned int)::$_0&>(Envoy::Event::FileEventImpl::FileEventImpl(Envoy::Event::DispatcherImpl&, int, std::__1::function<void (unsigned int)>, Envoy::Event::FileTriggerType, unsigned int)::$_0&) /proc/self/cwd/source/common/event/file_event_impl.cc:18:9
    #26 0xc252613 in std::__1::__function::__func<Envoy::Event::FileEventImpl::FileEventImpl(Envoy::Event::DispatcherImpl&, int, std::__1::function<void (unsigned int)>, Envoy::Event::FileTriggerType, unsigned int)::$_0, std::__1::allocator<Envoy::Event::FileEventImpl::FileEventImpl(Envoy::Event::DispatcherImpl&, int, std::__1::function<void (unsigned int)>, Envoy::Event::FileTriggerType, unsigned int)::$_0>, void ()>::operator()() /opt/llvm/bin/../include/c++/v1/__functional/invoke.h:61:9
    #27 0x59d93a9 in std::__1::__function::__value_func<void ()>::operator()() const /opt/llvm/bin/../include/c++/v1/__functional/function.h:507:16
    #28 0xc238857 in decltype(static_cast<Envoy::Event::DispatcherImpl::createSchedulableCallback(std::__1::function<void ()>)::$_6&>(fp)()) std::__1::__invoke<Envoy::Event::DispatcherImpl::createSchedulableCallback(std::__1::function<void ()>)::$_6&>(Envoy::Event::DispatcherImpl::createSchedulableCallback(std::__1::function<void ()>)::$_6&) /opt/llvm/bin/../include/c++/v1/__functional/function.h:1184:12
    #29 0xc2384d3 in std::__1::__function::__func<Envoy::Event::DispatcherImpl::createSchedulableCallback(std::__1::function<void ()>)::$_6, std::__1::allocator<Envoy::Event::DispatcherImpl::createSchedulableCallback(std::__1::function<void ()>)::$_6>, void ()>::operator()() /opt/llvm/bin/../include/c++/v1/__functional/invoke.h:61:9
    #30 0x59d93a9 in std::__1::__function::__value_func<void ()>::operator()() const /opt/llvm/bin/../include/c++/v1/__functional/function.h:507:16
    #31 0xcb630c4 in Envoy::Event::SchedulableCallbackImpl::SchedulableCallbackImpl(Envoy::CSmartPtr<event_base, &(event_base_free)>&, std::__1::function<void ()>)::$_0::__invoke(int, short, void*) /opt/llvm/bin/../include/c++/v1/__functional/function.h:1184:12
    #32 0xe514eee in event_process_active_single_queue /b/f/w/external/com_github_libevent_libevent/event.c:1713:4
    #33 0xe4f64e2 in event_process_active /b/f/w/external/com_github_libevent_libevent/event.c
    #34 0xe4f64e2 in event_base_loop /b/f/w/external/com_github_libevent_libevent/event.c:2047:12
    #35 0xcb5e44c in Envoy::Event::LibeventScheduler::run(Envoy::Event::Dispatcher::RunType) /proc/self/cwd/source/common/event/libevent_scheduler.cc:60:3
    #36 0xc22bd01 in Envoy::Event::DispatcherImpl::run(Envoy::Event::Dispatcher::RunType) /proc/self/cwd/source/common/event/dispatcher_impl.cc:277:19
    #37 0x5e3a0f8 in Envoy::FakeUpstream::threadRoutine() /proc/self/cwd/test/integration/fake_upstream.cc:748:16
    #38 0x5e5fcc7 in void std::__1::__invoke_void_return_wrapper<void, true>::__call<Envoy::FakeUpstream::initializeServer()::$_27&>(Envoy::FakeUpstream::initializeServer()::$_27&) /proc/self/cwd/test/integration/fake_upstream.cc:695:67
    #39 0x5e5f9e3 in std::__1::__function::__func<Envoy::FakeUpstream::initializeServer()::$_27, std::__1::allocator<Envoy::FakeUpstream::initializeServer()::$_27>, void ()>::operator()() /opt/llvm/bin/../include/c++/v1/__functional/function.h:180:16
    #40 0x59d93a9 in std::__1::__function::__value_func<void ()>::operator()() const /opt/llvm/bin/../include/c++/v1/__functional/function.h:507:16
    #41 0xe4ad89c in Envoy::Thread::ThreadImplPosix::ThreadImplPosix(std::__1::function<void ()>, std::__1::optional<Envoy::Thread::Options> const&)::'lambda'(void*)::operator()(void*) const /opt/llvm/bin/../include/c++/v1/__functional/function.h:1184:12
    #42 0xe4ad813 in Envoy::Thread::ThreadImplPosix::ThreadImplPosix(std::__1::function<void ()>, std::__1::optional<Envoy::Thread::Options> const&)::'lambda'(void*)::__invoke(void*) /proc/self/cwd/source/common/common/posix/thread_impl.cc:48:9
    #43 0x7fcc7ebb6608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x8608) (BuildId: 0c044ba611aeeeaebb8374e660061f341ebc0bac)
    #44 0x7fcc7eab4352 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11f352) (BuildId: eebe5d5f4b608b8a53ec446b63981bba373ca0ca)

It looks like an ASAN issue in upstream HTTP/1.

@danzh2010 danzh2010 removed their assignment Jun 5, 2024
@danzh2010
Copy link
Contributor

danzh2010 commented Jun 5, 2024

The ASAN call stack is the FakeUpstream with Envoy::Http::Http1::ServerConnectionImpl::ServerConnectionImpl. Not HTTP/3 related.

@danzh2010
Copy link
Contributor

This is a duplicate of #34353

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/test flakes bug no stalebot Disables stalebot from closing an issue
Projects
None yet
Development

No branches or pull requests

2 participants