Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Im/ember array unpack (local test) #1

Closed
wants to merge 2 commits into from
Closed

Im/ember array unpack (local test) #1

wants to merge 2 commits into from

Conversation

erjiaqing
Copy link
Owner

Problem

What is being fixed? Examples:

Change overview

What's in this PR

Testing

How was this tested? (at least one bullet point required)

  • If unit tests were added, how do they cover this issue?
  • If unit tests existed, how were they fixed/modified to prevent this in future?
  • If new unit tests are not added, why not?
  • If integration tests were added, how do they verify this change?
  • If new integration tests are not added, why not?
  • If manually tested, what platforms controller and device platforms were manually tested, and how?
  • If no testing is required, why not?

@erjiaqing erjiaqing force-pushed the im/ember-array-unpack branch 14 times, most recently from 85c2a07 to 86da4f1 Compare September 9, 2021 03:09
@erjiaqing erjiaqing force-pushed the im/ember-array-unpack branch 4 times, most recently from 4ba918d to 2bdfb45 Compare September 13, 2021 02:44
@erjiaqing erjiaqing closed this Sep 13, 2021
woody-apple pushed a commit that referenced this pull request Jul 16, 2022
@mspang
Remove racy access to readline (project-chip#20466)
5745a46 
It's not safe to access line editing state from the IO thread while
inside readline() on the main thread.

Remove the code that attempts to redraw readline after printing logs.
This avoids segfaults during logging at the cost of those logs
overwriting the prompt (this is not trivial to fix as readline
is a blocking API).

==================
WARNING: ThreadSanitizer: data race (pid=63005)
  Write of size 1 at 0x55f81c7745ff by main thread:
    #0 InteractiveStartCommand::ParseCommand(char*) ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:127 (chip-tool+0x874911)
    #1 InteractiveStartCommand::RunCommand() ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:85 (chip-tool+0x874594)
    #2 CHIPCommand::StartWaiting(std::chrono::duration<unsigned int, std::ratio<1l, 1000l> >) ../../examples/chip-tool/commands/common/CHIPCommand.cpp:408 (chip-tool+0x83e478)
    #3 CHIPCommand::Run() ../../examples/chip-tool/commands/common/CHIPCommand.cpp:187 (chip-tool+0x83c839)
    #4 Commands::RunCommand(int, char**, bool) ../../examples/chip-tool/commands/common/Commands.cpp:147 (chip-tool+0x85d4f7)
    #5 Commands::Run(int, char**) ../../examples/chip-tool/commands/common/Commands.cpp:51 (chip-tool+0x85c288)
    #6 main <null> (chip-tool+0x569c0a)

  Previous read of size 1 at 0x55f81c7745ff by thread T5 (mutexes: write M185):
    #0 LoggingCallback ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:46 (chip-tool+0x874479)
    #1 chip::Logging::LogV(unsigned char, unsigned char, char const*, __va_list_tag*) ../../src/lib/support/logging/CHIPLogging.cpp:221 (chip-tool+0x8ee4dc)
    #2 chip::Logging::Log(unsigned char, unsigned char, char const*, ...) ../../src/lib/support/logging/CHIPLogging.cpp:172 (chip-tool+0x8ee30a)
    #3 chip::app::ReadClient::RefreshLivenessCheckTimer() <null> (chip-tool+0x8b1746)
    #4 chip::app::ReadClient::ProcessSubscribeResponse(chip::System::PacketBufferHandle&&) ../../src/app/ReadClient.cpp:845 (chip-tool+0x8b20ec)
    #5 chip::app::ReadClient::OnMessageReceived(chip::Messaging::ExchangeContext*, chip::PayloadHeader const&, chip::System::PacketBufferHandle&&) ../../src/app/ReadClient.cpp:409 (chip-tool+0x8ae2a4)
    #6 chip::Messaging::ExchangeContext::HandleMessage(unsigned int, chip::PayloadHeader const&, chip::BitFlags<chip::Messaging::MessageFlagValues, unsigned int>, chip::System::PacketBufferHandle&&) <null> (chip-tool+0xa0517a)
    #7 operator()<chip::Messaging::ExchangeContext> ../../src/messaging/ExchangeMgr.cpp:219 (chip-tool+0xa08c73)
    #8 Call ../../src/lib/support/Pool.h:126 (chip-tool+0xa0912d)
    #9 chip::internal::HeapObjectList::ForEachNode(void*, chip::Loop (*)(void*, void*)) ../../src/lib/support/Pool.cpp:127 (chip-tool+0x8ee05a)
    #10 ForEachActiveObject<chip::Messaging::ExchangeManager::OnMessageReceived(const chip::PacketHeader&, const chip::PayloadHeader&, const chip::SessionHandle&, chip::SessionMessageDelegate::DuplicateMessage, chip::System::PacketBufferHandle&&)::<lambda(auto:2*)> > ../../src/lib/support/Pool.h:396 (chip-tool+0xa08d10)
    #11 chip::Messaging::ExchangeManager::OnMessageReceived(chip::PacketHeader const&, chip::PayloadHeader const&, chip::SessionHandle const&, chip::SessionMessageDelegate::DuplicateMessage, chip::System::PacketBufferHandle&&) ../../src/messaging/ExchangeMgr.cpp:212 (chip-tool+0xa07e91)
    #12 chip::SessionManager::SecureUnicastMessageDispatch(chip::PacketHeader const&, chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/SessionManager.cpp:616 (chip-tool+0xa1548b)
    #13 chip::SessionManager::OnMessageReceived(chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/SessionManager.cpp:443 (chip-tool+0xa14426)
    #14 chip::TransportMgrBase::HandleMessageReceived(chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/TransportMgrBase.cpp:76 (chip-tool+0xa17dfa)
    #15 chip::Transport::Base::HandleMessageReceived(chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/raw/Base.h:102 (chip-tool+0xb19728)
    #16 chip::Transport::UDP::OnUdpReceive(chip::Inet::UDPEndPoint*, chip::System::PacketBufferHandle&&, chip::Inet::IPPacketInfo const*) ../../src/transport/raw/UDP.cpp:122 (chip-tool+0xb1a48b)
    #17 chip::Inet::UDPEndPointImplSockets::HandlePendingIO(chip::BitFlags<chip::System::SocketEventFlags, unsigned char>) ../../src/inet/UDPEndPointImplSockets.cpp:688 (chip-tool+0xb00aa0)
    #18 chip::Inet::UDPEndPointImplSockets::HandlePendingIO(chip::BitFlags<chip::System::SocketEventFlags, unsigned char>, long) ../../src/inet/UDPEndPointImplSockets.cpp:569 (chip-tool+0xafff89)
    #19 chip::System::LayerImplSelect::HandleEvents() ../../src/system/SystemLayerImplSelect.cpp:406 (chip-tool+0xb07563)
    #20 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_RunEventLoop() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:181 (chip-tool+0x98a227)
    #21 chip::DeviceLayer::PlatformManager::RunEventLoop() ../../src/include/platform/PlatformManager.h:362 (chip-tool+0x988f75)
    #22 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::EventLoopTaskMain(void*) ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:205 (chip-tool+0x98a87c)

  Location is global '(anonymous namespace)::gIsCommandRunning' of size 1 at 0x55f81c7745ff (chip-tool+0x000000c485ff)

  Mutex M185 (0x55f81c776180) created at:
    #0 pthread_mutex_lock ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:4240 (libtsan.so.0+0x4f30a)
    #1 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_LockChipStack() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:78 (chip-tool+0x989e90)
    #2 chip::DeviceLayer::PlatformManager::LockChipStack() ../../src/include/platform/PlatformManager.h:410 (chip-tool+0x988fa5)
    #3 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_RunEventLoop() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:170 (chip-tool+0x98a147)
    #4 chip::DeviceLayer::PlatformManager::RunEventLoop() ../../src/include/platform/PlatformManager.h:362 (chip-tool+0x988f75)
    #5 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::EventLoopTaskMain(void*) ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:205 (chip-tool+0x98a87c)

  Thread T5 (tid=63013, running) created by main thread at:
    #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:969 (libtsan.so.0+0x5ad75)
    #1 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_StartEventLoopTask() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:231 (chip-tool+0x98a40a)
    #2 chip::DeviceLayer::PlatformManager::StartEventLoopTask() ../../src/include/platform/PlatformManager.h:375 (chip-tool+0xaacca2)
    #3 chip::Controller::DeviceControllerFactory::ServiceEvents() ../../src/controller/CHIPDeviceControllerFactory.cpp:331 (chip-tool+0xab0417)
    #4 CHIPCommand::StartWaiting(std::chrono::duration<unsigned int, std::ratio<1l, 1000l> >) ../../examples/chip-tool/commands/common/CHIPCommand.cpp:403 (chip-tool+0x83e353)
    #5 CHIPCommand::Run() ../../examples/chip-tool/commands/common/CHIPCommand.cpp:187 (chip-tool+0x83c839)
    #6 Commands::RunCommand(int, char**, bool) ../../examples/chip-tool/commands/common/Commands.cpp:147 (chip-tool+0x85d4f7)
    #7 Commands::Run(int, char**) ../../examples/chip-tool/commands/common/Commands.cpp:51 (chip-tool+0x85c288)
    #8 main <null> (chip-tool+0x569c0a)

SUMMARY: ThreadSanitizer: data race ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:127 in InteractiveStartCommand::ParseCommand(char*)
==================
erjiaqing pushed a commit that referenced this pull request Jun 6, 2023
@bzbarsky-apple
Fix ThreadSanitizer failure in controller factory. (project-chip#26824)
9f1a470 
* Fix ThreadSanitizer failure in controller factory.

The failure looks like this:

  WARNING: ThreadSanitizer: race on NSMutableArray (pid=11619)
    Read-only access of NSMutableArray at 0x7b0c0005f5b0 by thread T3:
      #0 -[__NSArrayM countByEnumeratingWithState:objects:count:] <null>:2 (CoreFoundation:x86_64+0x4a338)
      #1 -[MTRDeviceControllerFactory(InternalMethods) operationalInstanceAdded:] MTRDeviceControllerFactory.mm:855 (Matter:x86_64+0x1fd2a)
      #2 MTROperationalBrowser::OnBrowse(_DNSServiceRef_t*, unsigned int, unsigned int, int, char const*, char const*, char const*, void*) MTROperationalBrowser.mm:100 (Matter:x86_64+0x20ee63c)
      #3 handle_browse_response <null>:2 (libsystem_dnssd.dylib:x86_64+0x3733)
      #4 _dispatch_client_callout <null>:2 (libdispatch.dylib:x86_64+0x3316)

    Previous modifying access of NSMutableArray at 0x7b0c0005f5b0 by main thread:
      #0 -[__NSArrayM addObject:] <null>:2 (CoreFoundation:x86_64+0x2457a)
      #1 -[MTRDeviceControllerFactory createController] MTRDeviceControllerFactory.mm:719 (Matter:x86_64+0x1cee3)
      #2 -[MTRDeviceControllerFactory createControllerOnExistingFabric:error:] MTRDeviceControllerFactory.mm:534 (Matter:x86_64+0x19792)

The basic problem is that we are in the middle of adding an object to
_controllers on the API consumer thread when on the Matter thread we get our
browse notification.

The changes here don't aim to lock around all access to _controllers, but just
to make sure that our mutations of it can't race with the access on the Matter
thread.  More coarse locking would need to be done very carefully, given the
amount of dispath_sync to the Matter thread we have going on.

* Address review comments.
erjiaqing pushed a commit that referenced this pull request Jun 6, 2023
@bzbarsky-apple
Fix thread race in MTRXPCListenerSampleTests. (project-chip#26862)
07a7ea5 
WARNING: ThreadSanitizer: race on NSMutableDictionary (pid=72050)
      Modifying access of NSMutableDictionary at 0x7b0800149020 by thread T7:
        #0 -[__NSDictionaryM setObject:forKey:] <null>:2 (CoreFoundation:x86_64+0x40d4e)
        #1 -[MTRXPCListenerSample listener:shouldAcceptNewConnection:] MTRXPCListenerSampleTests.m:116 (MatterTests:x86_64+0x48437b)
        #2 service_connection_handler_make_connection <null>:2 (Foundation:x86_64+0xdea93)

      Previous modifying access of NSMutableDictionary at 0x7b0800149020 by thread T4:
        #0 -[__NSDictionaryM removeObjectForKey:] <null>:2 (CoreFoundation:x86_64+0x6cd6d)
        #1 __59-[MTRXPCListenerSample listener:shouldAcceptNewConnection:]_block_invoke MTRXPCListenerSampleTests.m:119 (MatterTests:x86_64+0x484630)
erjiaqing pushed a commit that referenced this pull request Nov 7, 2023
@bzbarsky-apple
Fix intermittent LSan failures running "darwin-framework-tool tests l…
b846dbe 
…ist". (project-chip#29666)

The typical failure there looks like this:

==29620==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 88 byte(s) in 1 object(s) allocated from:
    #0 0x106396e12 in calloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x51e12)
    #1 0x7ff800dc9789 in map_images_nolock+0x24b (libobjc.A.dylib:x86_64h+0x1789)
    #2 0x7ff800dc94db in map_images+0x42 (libobjc.A.dylib:x86_64h+0x14db)
    #3 0x113d721fa in invocation function for block in dyld4::RuntimeState::setObjCNotifiers(void (*)(unsigned int, char const* const*, mach_header const* const*), void (*)(char const*, mach_header const*), void (*)(char const*, mach_header const*))+0x112 (dyld:x86_64+0xf1fa)
    #4 0x113d6d6c8 in dyld4::RuntimeState::withLoadersReadLock(void () block_pointer)+0x28 (dyld:x86_64+0xa6c8)
    #5 0x113d720e1 in dyld4::RuntimeState::setObjCNotifiers(void (*)(unsigned int, char const* const*, mach_header const* const*), void (*)(char const*, mach_header const*), void (*)(char const*, mach_header const*))+0x51 (dyld:x86_64+0xf0e1)
    #6 0x113d85d44 in dyld4::APIs::_dyld_objc_notify_register(void (*)(unsigned int, char const* const*, mach_header const* const*), void (*)(char const*, mach_header const*), void (*)(char const*, mach_header const*))+0x4e (dyld:x86_64+0x22d44)
    #7 0x7ff800dc9343 in _objc_init+0x4fe (libobjc.A.dylib:x86_64h+0x1343)
    #8 0x7ff800d83992 in _os_object_init+0xc (libdispatch.dylib:x86_64+0x2992)
    #9 0x7ff800d911b7 in libdispatch_init+0x136 (libdispatch.dylib:x86_64+0x101b7)
    #10 0x7ff80bd34894 in libSystem_initializer+0xed (libSystem.B.dylib:x86_64+0x1894)
    #11 0x113d77e4e in invocation function for block in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const+0xb5 (dyld:x86_64+0x14e4e)
    #12 0x113d9eaac in invocation function for block in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const+0xf1 (dyld:x86_64+0x3baac)
    #13 0x113d95e25 in invocation function for block in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const+0x22c (dyld:x86_64+0x32e25)
    #14 0x113d64db2 in dyld3::MachOFile::forEachLoadCommand(Diagnostics&, void (load_command const*, bool&) block_pointer) const+0x80 (dyld:x86_64+0x1db2)
    #15 0x113d95bb6 in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const+0xb2 (dyld:x86_64+0x32bb6)
    #16 0x113d9e603 in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const+0x1d1 (dyld:x86_64+0x3b603)
    #17 0x113d77d81 in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const+0x8f (dyld:x86_64+0x14d81)
    #18 0x113d7e659 in dyld4::PrebuiltLoader::runInitializers(dyld4::RuntimeState&) const+0x1d (dyld:x86_64+0x1b659)
    #19 0x113d8b76d in dyld4::APIs::runAllInitializersForMain()+0x25 (dyld:x86_64+0x2876d)
    #20 0x113d6938c in dyld4::prepare(dyld4::APIs&, dyld3::MachOAnalyzer const*)+0xd72 (dyld:x86_64+0x638c)
    #21 0x113d684e3 in start+0x183 (dyld:x86_64+0x54e3)
erjiaqing pushed a commit that referenced this pull request May 7, 2024
@Alami-Amine
pw_unit_test migration: lib support batch #1 (project-chip#33091)
3380876 
* pw_unit_test migration: lib support batch #1

* apply restyled patch

* integrate comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@erjiaqing