[SYS-2605] Fix json serialization double conversion may overflow int64_t #3
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Current dependencies on/for this PR: This comment was auto-generated by Graphite. |
chaoz2
changed the title
chaoz2
changed the title
chaoz2
changed the title
chaoz2
force-pushed
the
chaoz_fix_json
branch
2 times, most recently
from
80a337d to
29a7b37
Compare
|
going to land https://github.com/rockset/folly/pull/4/files to add double conversion flags. |
nbronson
pushed a commit
that referenced
this pull request
May 26, 2023
Summary:
AsyncUDPSocket test cases are crashing when running under llvm15. During
debugging it seems that the issue is the fact that the code tries to allocated 0
size array. Changing the code to prevent such allocation.
This is not very clean why to fix, but I am not sure if there is better one.
Please let me know if you have any suggestions.
Sample crash:
```
$ buck test //folly/io/async/test:async_udp_socket_test
...
stdout:
Note: Google Test filter = AsyncUDPSocketTest.TestDetachAttach
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from AsyncUDPSocketTest
[ RUN ] AsyncUDPSocketTest.TestDetachAttach
stderr:
fbcode/folly/io/async/AsyncUDPSocket.cpp:699:10: runtime error: variable length array bound evaluates to non-positive value 0
#0 0x7f4d8ed93704 in folly::AsyncUDPSocket::writev(folly::SocketAddress const&, iovec const*, unsigned long, int) fbcode/folly/io/async/AsyncUDPSocket.cpp:698
#1 0x7f4d8ed9081f in folly::AsyncUDPSocket::writeGSO(folly::SocketAddress const&, std::unique_ptr<folly::IOBuf, std::default_delete<folly::IOBuf>> const&, int) fbcode/folly/io/async/AsyncUDPSocket.cpp:528
#2 0x7f4d8ed900b2 in folly::AsyncUDPSocket::write(folly::SocketAddress const&, std::unique_ptr<folly::IOBuf, std::default_delete<folly::IOBuf>> const&) fbcode/folly/io/async/AsyncUDPSocket.cpp:660
#3 0x350a05 in AsyncUDPSocketTest_TestDetachAttach_Test::TestBody() fbcode/folly/io/async/test/AsyncUDPSocketTest.cpp:914
#4 0x7f4d90dd1ad5 in testing::Test::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2682:50
#5 0x7f4d90dd1ad5 in testing::Test::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2672:6
facebook#6 0x7f4d90dd1c64 in testing::TestInfo::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2861:14
facebook#7 0x7f4d90dd1c64 in testing::TestInfo::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2833:6
facebook#8 0x7f4d90dd2321 in testing::TestSuite::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:3015:31
facebook#9 0x7f4d90dd2321 in testing::TestSuite::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2993:6
facebook#10 0x7f4d90dd2b1e in testing::internal::UnitTestImpl::RunAllTests() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:5855:47
facebook#11 0x7f4d90dd1d87 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:2665:29
facebook#12 0x7f4d90dd1d87 in testing::UnitTest::Run() /home/engshare/third-party2/googletest/1.11.0/src/googletest/googletest/src/gtest.cc:5438:55
facebook#13 0x7f4d90d5c990 in RUN_ALL_TESTS() fbcode/third-party-buck/platform010/build/googletest/include/gtest/gtest.h:2490
facebook#14 0x7f4d90d5c618 in main fbcode/common/gtest/LightMain.cpp:20
facebook#15 0x7f4d8ea2c656 in __libc_start_call_main /home/engshare/third-party2/glibc/2.34/src/glibc-2.34/csu/../sysdeps/nptl/libc_start_call_main.h:58:16
facebook#16 0x7f4d8ea2c717 in __libc_start_main@GLIBC_2.2.5 /home/engshare/third-party2/glibc/2.34/src/glibc-2.34/csu/../csu/libc-start.c:409:3
facebook#17 0x33ea60 in _start /home/engshare/third-party2/glibc/2.34/src/glibc-2.34/csu/../sysdeps/x86_64/start.S:116
...
```
Reviewed By: russoue, dmm-fb
Differential Revision: D43858875
fbshipit-source-id: 93749bab17027b6dfc0dbc01b6c183e501a5494c
igorcanadi
pushed a commit
that referenced
this pull request
May 28, 2024
Summary:
ThreadLocalDetailTest.MultiThreadedTest had a data race on std::vector because multiple threads were accessing and modifying the std::vector without a mutex. I update the code to use indexes such that the vector is not updated concurrently.
Previous run failure that this fixes:
```
stderr:
==================
WARNING: ThreadSanitizer: data race (pid=1249005)
Write of size 8 at 0x7fff195936d0 by main thread:
#0 std::vector<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>, std::allocator<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>>>::pop_back() fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/stl_vector.h:1228 (thread_local_detail_test+0x116c28)
#1 folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody() fbcode/folly/detail/test/ThreadLocalDetailTest.cpp:121 (thread_local_detail_test+0x101aba)
#2 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) fbsource/src/gtest.cc:2675 (libthird-party_googletest_1.14.0_gtest.so+0xacf3c)
#3 testing::Test::Run() fbsource/src/gtest.cc:2692 (libthird-party_googletest_1.14.0_gtest.so+0xacb9c)
#4 testing::TestInfo::Run() fbsource/src/gtest.cc:2841 (libthird-party_googletest_1.14.0_gtest.so+0xafa4a)
#5 testing::TestSuite::Run() fbsource/src/gtest.cc:3020 (libthird-party_googletest_1.14.0_gtest.so+0xb4303)
facebook#6 testing::internal::UnitTestImpl::RunAllTests() fbsource/src/gtest.cc:5925 (libthird-party_googletest_1.14.0_gtest.so+0xd105e)
facebook#7 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) fbsource/src/gtest.cc:2675 (libthird-party_googletest_1.14.0_gtest.so+0xd08f1)
facebook#8 testing::UnitTest::Run() fbsource/src/gtest.cc:5489 (libthird-party_googletest_1.14.0_gtest.so+0xd037e)
facebook#9 RUN_ALL_TESTS() fbsource/gtest/gtest.h:2317 (libcommon_gtest_light_main.so+0x22a7)
facebook#10 main fbcode/common/gtest/LightMain.cpp:20 (libcommon_gtest_light_main.so+0x2131)
Previous read of size 8 at 0x7fff195936d0 by thread T123:
#0 std::vector<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>, std::allocator<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>>>::size() const fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/stl_vector.h:919 (thread_local_detail_test+0x11b0bd)
#1 std::vector<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>, std::allocator<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>>>::operator[](unsigned long) fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/stl_vector.h:1045 (thread_local_detail_test+0x11d101)
#2 folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0::operator()() const fbcode/folly/detail/test/ThreadLocalDetailTest.cpp:97 (thread_local_detail_test+0x11d08c)
#3 void std::__invoke_impl<void, folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0>(std::__invoke_other, folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0&&) fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/invoke.h:61 (thread_local_detail_test+0x11cf95)
#4 std::__invoke_result<folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0>::type std::__invoke<folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0>(folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0&&) fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/invoke.h:96 (thread_local_detail_test+0x11cf05)
#5 void std::thread::_Invoker<std::tuple<folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0>>::_M_invoke<0ul>(std::_Index_tuple<0ul>) fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/std_thread.h:253 (thread_local_detail_test+0x11cebd)
facebook#6 std::thread::_Invoker<std::tuple<folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0>>::operator()() fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/std_thread.h:260 (thread_local_detail_test+0x11ce65)
facebook#7 std::thread::_State_impl<std::thread::_Invoker<std::tuple<folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody()::$_0>>>::_M_run() fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/std_thread.h:211 (thread_local_detail_test+0x11cd79)
facebook#8 execute_native_thread_routine /home/engshare/third-party2/libgcc/11.x/src/gcc-11.x/x86_64-facebook-linux/libstdc++-v3/src/c++11/../../../.././libstdc++-v3/src/c++11/thread.cc:82:18 (libstdc++.so.6+0xdf4e4) (BuildId: 452d1cdae868baeeb2fdf1ab140f1c219bf50c6e)
Location is stack of main thread.
Location is global '??' at 0x7fff19575000 ([stack]+0x1e6d0)
Thread T123 (tid=1249233, running) created by main thread at:
#0 pthread_create <null> (thread_local_detail_test+0x156bef)
#1 __gthread_create /home/engshare/third-party2/libgcc/11.x/src/gcc-11.x/x86_64-facebook-linux/libstdc++-v3/include/x86_64-facebook-linux/bits/gthr-default.h:663:35 (libstdc++.so.6+0xdf80e) (BuildId: 452d1cdae868baeeb2fdf1ab140f1c219bf50c6e)
#2 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State>>, void (*)()) /home/engshare/third-party2/libgcc/11.x/src/gcc-11.x/x86_64-facebook-linux/libstdc++-v3/src/c++11/../../../.././libstdc++-v3/src/c++11/thread.cc:147:37 (libstdc++.so.6+0xdf80e)
#3 folly::threadlocal_detail::ThreadLocalDetailTest_MultiThreadedTest_Test::TestBody() fbcode/folly/detail/test/ThreadLocalDetailTest.cpp:93 (thread_local_detail_test+0x1015cd)
#4 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) fbsource/src/gtest.cc:2675 (libthird-party_googletest_1.14.0_gtest.so+0xacf3c)
#5 testing::Test::Run() fbsource/src/gtest.cc:2692 (libthird-party_googletest_1.14.0_gtest.so+0xacb9c)
facebook#6 testing::TestInfo::Run() fbsource/src/gtest.cc:2841 (libthird-party_googletest_1.14.0_gtest.so+0xafa4a)
facebook#7 testing::TestSuite::Run() fbsource/src/gtest.cc:3020 (libthird-party_googletest_1.14.0_gtest.so+0xb4303)
facebook#8 testing::internal::UnitTestImpl::RunAllTests() fbsource/src/gtest.cc:5925 (libthird-party_googletest_1.14.0_gtest.so+0xd105e)
facebook#9 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) fbsource/src/gtest.cc:2675 (libthird-party_googletest_1.14.0_gtest.so+0xd08f1)
facebook#10 testing::UnitTest::Run() fbsource/src/gtest.cc:5489 (libthird-party_googletest_1.14.0_gtest.so+0xd037e)
facebook#11 RUN_ALL_TESTS() fbsource/gtest/gtest.h:2317 (libcommon_gtest_light_main.so+0x22a7)
facebook#12 main fbcode/common/gtest/LightMain.cpp:20 (libcommon_gtest_light_main.so+0x2131)
ThreadSanitizer: data race fbcode/third-party-buck/platform010/build/libgcc/include/c++/trunk/bits/stl_vector.h:1228 in std::vector<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>, std::allocator<std::unique_ptr<folly::test::Barrier, std::default_delete<folly::test::Barrier>>>>::pop_back()
==================
ThreadSanitizer: reported 1 warnings
```
Reviewed By: yfeldblum
Differential Revision: D56643303
fbshipit-source-id: cc364817ae79bc6418cc07faa35e1bcd21830c66
mpatou-openai
pushed a commit
that referenced
this pull request
Oct 16, 2024
Summary: Ucache with io_uring is crashing with this stack trace: ``` (lldb) bt * thread #1, name = 'ucache', stop reason = signal SIGSEGV: address not mapped to object * frame #0: 0x0000000000000000 frame #1: 0x00007f517ba44560 libc.so.6`__restore_rt at libc_sigaction.c:13 frame #2: 0x000000000a16e249 ucache`folly::AsyncIoUringSocket::ReadSqe::callback(this=0x00007f498c5955e0, cqe=<unavailable>) at AsyncIoUringSocket.cpp:639 frame #3: 0x000000000964196a ucache`folly::IoUringBackend::getActiveEvents(folly::IoUringBackend::WaitForEventsMode) [inlined] folly::IoSqeBase::internalCallback(this=<unavailable>, cqe=<unavailable>) at IoUringBackend.cpp:0 frame #4: 0x000000000964195c ucache`folly::IoUringBackend::getActiveEvents(folly::IoUringBackend::WaitForEventsMode) [inlined] folly::IoUringBackend::internalProcessCqe(this=0x00007f4a8f989c00, maxGet=4294967295, mode=NORMAL) at IoUringBackend.cpp:1556 frame #5: 0x000000000964190f ucache`folly::IoUringBackend::getActiveEvents(this=0x00007f4a8f989c00, waitForEvents=<unavailable>) at IoUringBackend.cpp:1527 frame facebook#6: 0x0000000009640ac9 ucache`folly::IoUringBackend::eb_event_base_loop(this=0x00007f4a8f989c00, flags=1) at IoUringBackend.cpp:1178 frame facebook#7: 0x000000001b61547c ucache`folly::EventBase::loopMain(this=0x00007f4a8f97af00, flags=0, options=<unavailable>) at EventBase.cpp:0 frame facebook#8: 0x000000001bab9a2d ucache`folly::EventBase::loopBody(this=0x00007f4a8f97af00, flags=0, options=(ignoreKeepAlive = false, allowSuspension = false)) at EventBase.cpp:513 frame facebook#9: 0x000000000a3a6c90 ucache`folly::EventBase::loop(this=<unavailable>) at EventBase.cpp:474 frame facebook#10: 0x000000000a3a75f7 ucache`folly::EventBase::loopForever(this=<unavailable>) at EventBase.cpp:781 frame facebook#11: 0x000000000a436907 ucache`folly::IOThreadPoolExecutor::threadRun(this=0x00007f4da3fce1c0, thread=<unavailable>) at IOThreadPoolExecutor.cpp:240 frame facebook#12: 0x0000000009ed68b8 ucache`void std::__invoke_impl<void, void (folly::ThreadPoolExecutor::*&)(std::shared_ptr<folly::ThreadPoolExecutor::Thread>), folly::ThreadPoolExecutor*&, std::shared_ptr<folly::ThreadPoolExecutor::Thread>&>((null)=<unavailable>, __f=<unavailable>, __t=<unavailable>, __args=<unavailable>) at invoke.h:74 frame facebook#13: 0x00007f517b6df4e5 libstdc++.so.6`std::execute_native_thread_routine(__p=0x00007f4da01bdbe0) at thread.cc:82:18 frame facebook#14: 0x00007f517ba9abc9 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:434:8 frame facebook#15: 0x00007f517bb2cf5c libc.so.6`__clone3 at clone3.S:81 ``` In frame 2 the offending function is `ReadSqe::callback()`: https://fburl.com/code/4fb46hdq The problem is that `readCallback_` is null: ``` (lldb) fr v readCallback_ (folly::AsyncReader::ReadCallback *) readCallback_ = nullptr ``` However, `readCallback_` is checked earlier in the the function `ReadSqe::callback()`: https://fburl.com/code/dhnndk29 `AsyncIoUringSocket::readError()` now fails all pending writes and calls `writeErr()`, whose implementation can then close out a socket and call `AsyncIoUringSocket::setReadCB(nullptr)` to change the callback. Move the call to `AsyncIoUringSocket::readError()` after the call to `readCallback_`. Reviewed By: dmm-fb Differential Revision: D60880377 fbshipit-source-id: 42d2f430cd3dc3f4787d0e4c29da7c8ef31ba9a8
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
as title, we must set
kConvMaxDecimalInShortestHighlower or equal to 18examples of how this arg is used in
double-conversionhttps://github.com/google/double-conversion/blob/master/double-conversion/double-to-string.h#L116-L118
test added in rs https://github.com/rockset/rs/pull/12908