TSAN test flake in //test/integration:h1_capture_persistent_fuzz_test

[ggreenway]

//test/integration:h1_capture_persistent_fuzz_test
 ==================
2022-03-16T14:17:37.7351141Z WARNING: ThreadSanitizer: data race (pid=18)
2022-03-16T14:17:37.7351687Z   Write of size 8 at 0x7b540000cbb8 by main thread:
2022-03-16T14:17:37.7353214Z     #0 testing::internal::UnitTestImpl::set_current_test_info(testing::TestInfo*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest-internal-inl.h:729:24 (h1_capture_persistent_fuzz_test+0x8dfd524)
2022-03-16T14:17:37.7354444Z     #1 testing::TestInfo::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2850:9 (h1_capture_persistent_fuzz_test+0x8de0e79)
2022-03-16T14:17:37.7355482Z     #2 testing::TestSuite::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2986:28 (h1_capture_persistent_fuzz_test+0x8de1945)
2022-03-16T14:17:37.7356569Z     #3 testing::internal::UnitTestImpl::RunAllTests() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:5697:44 (h1_capture_persistent_fuzz_test+0x8df1e7b)
2022-03-16T14:17:37.7358101Z     #4 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2580:10 (h1_capture_persistent_fuzz_test+0x8e1ff77)
2022-03-16T14:17:37.7360032Z     #5 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2616:14 (h1_capture_persistent_fuzz_test+0x8e006b0)
2022-03-16T14:17:37.7361497Z     #6 testing::UnitTest::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:5280:10 (h1_capture_persistent_fuzz_test+0x8df1723)
2022-03-16T14:17:37.7362502Z     #7 RUN_ALL_TESTS() /proc/self/cwd/external/com_google_googletest/googletest/include/gtest/gtest.h:2485:46 (h1_capture_persistent_fuzz_test+0x684569a)
2022-03-16T14:17:37.7363339Z     #8 main /proc/self/cwd/test/fuzz/main.cc:97:10 (h1_capture_persistent_fuzz_test+0x683c9a2)
2022-03-16T14:17:37.7363704Z 
2022-03-16T14:17:37.7364120Z   Previous read of size 8 at 0x7b540000cbb8 by thread T1:
2022-03-16T14:17:37.7364623Z     [failed to restore the stack]
2022-03-16T14:17:37.7364824Z 
2022-03-16T14:17:37.7365283Z   Location is heap block of size 552 at 0x7b540000ca80 allocated by main thread:
2022-03-16T14:17:37.7366398Z     #0 malloc /tmp/llvm/utils/release/final/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:651:5 (h1_capture_persistent_fuzz_test+0x286727b)
2022-03-16T14:17:37.7367663Z     #1 operator new(unsigned long) /tmp/tmp.9oihhy9ANQ/llvm-project-llvmorg-12.0.1/libcxxabi/src/stdlib_new_delete.cpp:30:17 (libc++abi.so.1+0x4ed37)
2022-03-16T14:17:37.7368657Z     #2 testing::UnitTest::UnitTest() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:5329:11 (h1_capture_persistent_fuzz_test+0x8df25ff)
2022-03-16T14:17:37.7369868Z     #3 testing::UnitTest::GetInstance() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:4974:19 (h1_capture_persistent_fuzz_test+0x8dd1280)
2022-03-16T14:17:37.7370991Z     #4 Envoy::(anonymous namespace)::FuzzerCorpusTest_RunOneCorpusFile_Test::AddToRegistry() /proc/self/cwd/test/fuzz/main.cc:46:1 (h1_capture_persistent_fuzz_test+0x683b728)
2022-03-16T14:17:37.7371989Z     #5 __cxx_global_var_init.2 /proc/self/cwd/test/fuzz/main.cc:46:1 (h1_capture_persistent_fuzz_test+0x6844d65)
2022-03-16T14:17:37.7372796Z     #6 _GLOBAL__sub_I_main.cc /proc/self/cwd/test/fuzz/main.cc (h1_capture_persistent_fuzz_test+0x6844f79)
2022-03-16T14:17:37.7373519Z     #7 __libc_csu_init <null> (h1_capture_persistent_fuzz_test+0x8e7ad7c)
2022-03-16T14:17:37.7373875Z 
2022-03-16T14:17:37.7374338Z   Thread T1 (tid=25, running) created by main thread at:
2022-03-16T14:17:37.7375473Z     #0 pthread_create /tmp/llvm/utils/release/final/llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:965:3 (h1_capture_persistent_fuzz_test+0x2868aab)
2022-03-16T14:17:37.7376771Z     #1 Envoy::Thread::ThreadImplPosix::ThreadImplPosix(std::__1::function<void ()>, absl::optional<Envoy::Thread::Options> const&) /proc/self/cwd/source/common/common/posix/thread_impl.cc:46:20 (h1_capture_persistent_fuzz_test+0x766485d)
2022-03-16T14:17:37.7378742Z     #2 std::__1::__unique_if<Envoy::Thread::ThreadImplPosix>::__unique_single std::__1::make_unique<Envoy::Thread::ThreadImplPosix, std::__1::function<void ()>&, absl::optional<Envoy::Thread::Options> const&>(std::__1::function<void ()>&, absl::optional<Envoy::Thread::Options> const&) /opt/llvm/bin/../include/c++/v1/memory:2094:32 (h1_capture_persistent_fuzz_test+0x7663e9f)
2022-03-16T14:17:37.7452866Z     #3 Envoy::Thread::ThreadFactoryImplPosix::createThread(std::__1::function<void ()>, absl::optional<Envoy::Thread::Options> const&) /proc/self/cwd/source/common/common/posix/thread_impl.cc:121:10 (h1_capture_persistent_fuzz_test+0x7663c0d)
2022-03-16T14:17:37.7455032Z     #4 Envoy::FakeUpstream::FakeUpstream(std::__1::unique_ptr<Envoy::Network::TransportSocketFactory, std::__1::default_delete<Envoy::Network::TransportSocketFactory> >&&, std::__1::unique_ptr<Envoy::Network::Socket, std::__1::default_delete<Envoy::Network::Socket> >&&, Envoy::FakeUpstreamConfig const&) /proc/self/cwd/test/integration/fake_upstream.cc:580:35 (h1_capture_persistent_fuzz_test+0x2c8627e)
2022-03-16T14:17:37.7458290Z     #5 Envoy::FakeUpstream::FakeUpstream(std::__1::unique_ptr<Envoy::Network::TransportSocketFactory, std::__1::default_delete<Envoy::Network::TransportSocketFactory> >&&, std::__1::shared_ptr<Envoy::Network::Address::Instance const> const&, Envoy::FakeUpstreamConfig const&) /proc/self/cwd/test/integration/fake_upstream.cc:550:7 (h1_capture_persistent_fuzz_test+0x2c86856)
2022-03-16T14:17:37.7460251Z     #6 Envoy::BaseIntegrationTest::createUpstreams() /proc/self/cwd/test/integration/base_integration_test.cc:155:15 (h1_capture_persistent_fuzz_test+0x2b80c68)
2022-03-16T14:17:37.7461458Z     #7 Envoy::BaseIntegrationTest::initialize() /proc/self/cwd/test/integration/base_integration_test.cc:100:3 (h1_capture_persistent_fuzz_test+0x2b7fb33)
2022-03-16T14:17:37.7462602Z     #8 Envoy::HttpIntegrationTest::initialize() /proc/self/cwd/test/integration/http_integration.cc:343:33 (h1_capture_persistent_fuzz_test+0x29afa5c)
2022-03-16T14:17:37.7463773Z     #9 Envoy::H1FuzzIntegrationTest::initialize() /proc/self/cwd/test/integration/h1_capture_fuzz_test.cc:4:65 (h1_capture_persistent_fuzz_test+0x28d8915)
2022-03-16T14:17:37.7464989Z     #10 Envoy::H1FuzzIntegrationTest::replay(test::integration::CaptureFuzzTestCase const&, bool)::$_0::operator()() const /proc/self/cwd/test/integration/h1_fuzz.cc:16:5 (h1_capture_persistent_fuzz_test+0x2986776)
2022-03-16T14:17:37.7466317Z     #11 Envoy::H1FuzzIntegrationTest::replay(test::integration::CaptureFuzzTestCase const&, bool) /proc/self/cwd/test/integration/h1_fuzz.cc:15:42 (h1_capture_persistent_fuzz_test+0x29841bf)
2022-03-16T14:17:37.7467552Z     #12 Envoy::TestOneProtoInput(test::integration::CaptureFuzzTestCase const&) /proc/self/cwd/test/integration/h1_capture_fuzz_test.cc:11:28 (h1_capture_persistent_fuzz_test+0x28d92ca)
2022-03-16T14:17:37.7468519Z     #13 LLVMFuzzerTestOneInput /proc/self/cwd/test/integration/h1_capture_fuzz_test.cc:6:1 (h1_capture_persistent_fuzz_test+0x28d8c7f)
2022-03-16T14:17:37.7469442Z     #14 Envoy::(anonymous namespace)::FuzzerCorpusTest_RunOneCorpusFile_Test::TestBody() /proc/self/cwd/test/fuzz/main.cc:50:3 (h1_capture_persistent_fuzz_test+0x6840689)
2022-03-16T14:17:37.7470668Z     #15 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2580:10 (h1_capture_persistent_fuzz_test+0x8e19877)
2022-03-16T14:17:37.7472437Z     #16 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2616:14 (h1_capture_persistent_fuzz_test+0x8dfce30)
2022-03-16T14:17:37.7473778Z     #17 testing::Test::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2655:5 (h1_capture_persistent_fuzz_test+0x8ddfec7)
2022-03-16T14:17:37.7474906Z     #18 testing::TestInfo::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2832:11 (h1_capture_persistent_fuzz_test+0x8de0d7a)
2022-03-16T14:17:37.7475969Z     #19 testing::TestSuite::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2986:28 (h1_capture_persistent_fuzz_test+0x8de1945)
2022-03-16T14:17:37.7477067Z     #20 testing::internal::UnitTestImpl::RunAllTests() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:5697:44 (h1_capture_persistent_fuzz_test+0x8df1e7b)
2022-03-16T14:17:37.7478642Z     #21 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2580:10 (h1_capture_persistent_fuzz_test+0x8e1ff77)
2022-03-16T14:17:37.7480574Z     #22 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2616:14 (h1_capture_persistent_fuzz_test+0x8e006b0)
2022-03-16T14:17:37.7482046Z     #23 testing::UnitTest::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:5280:10 (h1_capture_persistent_fuzz_test+0x8df1723)
2022-03-16T14:17:37.7483089Z     #24 RUN_ALL_TESTS() /proc/self/cwd/external/com_google_googletest/googletest/include/gtest/gtest.h:2485:46 (h1_capture_persistent_fuzz_test+0x684569a)
2022-03-16T14:17:37.7483986Z     #25 main /proc/self/cwd/test/fuzz/main.cc:97:10 (h1_capture_persistent_fuzz_test+0x683c9a2)
2022-03-16T14:17:37.7484401Z 
2022-03-16T14:17:37.7485927Z SUMMARY: ThreadSanitizer: data race /proc/self/cwd/external/com_google_googletest/googletest/src/gtest-internal-inl.h:729:24 in testing::internal::UnitTestImpl::set_current_test_info(testing::TestIn

[alyssawilk]

@adisuissa can you assign this to whoever you think should own fuzz bugs nowadays?

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.

[alyssawilk]

reopening. Adi, I'm still seeing failures here. can you please bump in priority?

[adisuissa]

Trying to plumb the real runtime is more complicated than having a fake runtime (I guess its because the Runtime Loader wasn't designed for multiple FakeUpstreams). I'll get some more time to work on it this week.

[alyssawilk]

https://dev.azure.com/cncf/4684fb3d-0389-4e0b-8251-221942316e06/_apis/build/builds/107022/logs/270

[RyanTheOptimist]

Can we disable this test until we have a solution that doesn't have such a high flake rate, perhaps? I'm regularly having PRs fail CI because of this failure and it looks like it takes another 40 minutes to re-run.

[jmarantz]

+1 to disabling flaky tests.

[adisuissa]

Should be fixed by #21552, closing.