Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix conflicts in PR 9597 #1

Conversation

tcarmelveilleux
Copy link

Note that a couple test cases fail after update...

mspang and others added 30 commits September 13, 2021 09:54
Make sure to unambiguously specify the chip namespace in macros
as they can be evaluated in places where 'chip::' is ambiguous.
* Change temperature delta label to degrees celsius

* Rename Fahrenheit in identifiers to Celsius

* Restyled by clang-format

Co-authored-by: Restyled.io <commits@restyled.io>
Signed-off-by: Victor Morales <v.morales@samsung.com>
This is unused in Matter.  The only use case is ICMP, and Matter
doesn't do ICMP.
…ect-chip#9592)

It was unused, and if we can guarantee a contiguous buffer consumers can be safer
This reverts commit 3da26cf.

Tizen builds seem broken since this change:

```
2021-09-13 15:22:31 INFO
/__w/connectedhomeip/connectedhomeip/out/tizen-arm-light/../../examples/lighting-app/linux/third_party/connectedhomeip/src/lib/mdns/Discovery_ImplPlatform.h:46:
undefined reference to `chip::Mdns::ChipMdnsShutdown()'
409
2021-09-13 15:22:31 INFO    collect2: error: ld returned 1 exit status
```
…cket buffers. (project-chip#9599)

None of our upper-layer core really deals with chained packet buffers,
starting with the way we do in-place decryption that assumes that we
have a contiguous buffer.

The changes here:

1) Flatten out possibly-chained buffers in the LwIP case in the UDP
   endpoint.
2) Ensure that we don't have a chained buffer in BTP.  There was
   already a check for this in the "in progress" case; this change just
   adds the same check in the "idle" case.
* Use RTC memory as heap in unicore mode and readme fixes

Signed-off-by: Shubham Patil <shubham.patil@espressif.com>

* Wait till NimBLE host-controller is synced

Signed-off-by: Shubham Patil <shubham.patil@espressif.com>
* Small-artifact bloat reports

#### Problem

The current bloat report requires preserving large binaries, and has
trouble matching tree parents for comparison.

#### Change overview

This change makes example builds generate small artifacts containing
json file(s) containing build item identification and section sizes.

Artifacts names have the form “Size,PR,WORKFLOW,CURRENT_SHA,PARENT_SHA”;
these contain one or more json files, each containing a report on one
build target.

The reporting scripts then examines the list of available artifacts to
list of artifacts to identify pairs with the same $WORKFLOW where one's
$CURRENT_SHA is the other's $PARENT_SHA, and only then downloads and
processes those artifacts to generate comparisons for matching builds.

- Changes to `examples-…` workflows:

  - Adds some `GH_EVENT_…` variables to workflow `env`, to identify
    the current run.
  - Uses the `gh_sizes.py` script to generate size report json files.
  - Uploads size report artifacts.

- Changes to `scripts/tools/memory`:
  - Adds minimal platform config files for recently-added platforms.
  - Adds markdown output options (required for github comments) along
    with some associated cleanup.
  - Adds a script `scripts/tools/memory/gh_sizes.py` for use by workflows;
    this is similar to `report_summary.py` with a suitable consistent set
    of arguments.
  - Adds a script `scripts/tools/memory/gh_report.py` to analyze size
    report artifacts.

- Modifies `bloat_check.py` to ignore the size report artifacts.

The github comments produced by `gh_report.py` are slightly different
fromt the existing reports. Since this change enables reports for many
more builds, a full report will be over a hundred lines, and is placed
inside a details tag. Only size changes above a configurable threshold
are called out ‘above the fold’.

Note that this PR does _not_ include a change to actually send size
report comments on github; this is left to a followup after that code
has been verified on real-world size artifacts.

#### Testing

Fork-CI runs and offline verification on resulting artifacts.

* remove leftover debug print

* small json format change

* changes from review

- restore original scripts/examples/esp_example.sh
  and scripts/examples/gn_efr32_example.sh
- avoid a problem using pyelftools `describe_p_type`

* telink output moved

* report sections directly rather than aggregated symbols by section
* WC: Update ClusterRevision to Matter initial release

* WC: Update rev # for all-cluster-app
#### Problem

Linux/MdnsImpl currently uses ad-hoc coupling with the POSIX I/O event
loop, which intrudes on and constrains the latter's implementation.

#### Change overview

Replace the `GetMdnsTimeout()`/`HandleMdnsTimeout()` pair with a
`System::Layer` timer tracking the earliest mDNS timeout.

The last remnants of ~the Old Republic~ `select()` have been swept away.

#### Testing

Existing unit test TestMdns
* Update telink build instructions to work with latest vscode image.

Latest vscode image contains TELINK_ZEPHYR_SDK_DIR, no need
to hardcode the path to it in the build script anymore.

* Restyle fixes
…roject-chip#9638)

1) Modify our CHIPClientCallbacks template to suppress the "possibly
   unbounded stack" warning from a VLA we know we want to remove.
2) Don't try to link in BLELayer in the TestDevice test if BLE
   networking is disabled (so that BLELayer symbols don't actually
   exist).
3) Heap-allocate the huge FabricTable so we don't run into compile
   errors due to a stack frame being too big in TestDevice.
4) Fix the fact that SystemLayerImplSelect did not default-initialize
   mDispatchQueue.  This matters because TestDevice does weird things
   with a separate SystemLayer that the PlatformManager does not know
   about, nor does it call InitChipStack in any case.

The tests are still disabled on nrfconnect because I could not get
them to link there so far.

Fixes project-chip#9173
…-chip#9582)

* Update PASE state machine to match the latest specifications

* Use TLV formatted messages
* Use StatusReport to indicate errors and completion

* Address review comments

* Address comments

* some cleanup
…hip#9571)

* Expire associated secure sessions when a fabric is deleted

* regen zap files

* another zap regen

* address review comments

* address review comments

* updates

* update CASESession error handling

* fix build error
* Add tests for CHIPMemString.

* Update src/lib/support/tests/TestCHIPMemString.cpp

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Address review comments

Use buf len instead of strlen to make it more clear about what the
buffer sizes really mean.

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>
* [testsuite] Refactor test suite simulated cluster command

* Add initial IM subscriber server change

--Extend IM ReadHandler with subscribe capability, and update the
corresponding reporting engine, which can process multiple subscriber,
process subscribe request and generate subscribe response.
--Hook setDirty API to ember so that ember can generate the change set
and send it out via IM report.

* Add initial IM subscription client change

--Extends IM read client with subscribe capability and it can send
subscribe request and process subscribe process and further maintain
subscription with livess check.
--Disable path filter for IM read/subscribe request.
--Update templates to use interaction model APIs for sending subscribe
requests.
--Update DeviceControllerInteractionModelDelegate to handle report
messages from subscription.
--Update ChipCallback Mgr to bridge subscribe responses to existing
callbacks, and handle TLV message just as read responses.
--Update TestSuite for basic Subscription protocol tests, test routing
looks like follows:
Send subscribe request, wait for response.
Wait for a few seconds, and execute "kick" commands.
The test is expected to receive the same number of Reports as the
number of kick commands.
The test will fail when not receiving enough report data from server
in global timeout.
--Update python script test to (in cirque) include a similar test
routine as test suite.
--Add unit test for positive and negative subscribe test
--Add integrated cirque test for subscribe

* Update test cases, add darwin test

* run codegen

Co-authored-by: Song Guo <songguo@google.com>
* linux: Restrict search to BLE only

For BlueZ, set "Transport" discovery filter to "le" avoiding
interleaved discovery. There is no need to inquiry for traditional
Bluetooth devices.

* linux: Restrict search to chip UUID

For BlueZ, set discovery "UUIDs" filter to chip UUID to avoid
reporting BLE devices that doesn't support chip/matter.
* unauthenticated-session

* Resolve comments
This fixes errors like:

```
FileNotFoundError: [Errno 2] No such file or directory:
'/workspaces/connectedhomeip/out/tizen-arm-light/chip-tizen-lighting-example.out'
```

When trying to copy artifacts.

Tested running:

```
./scripts/build/build_examples.py --platform tizen build --copy-artifacts-to /tmp
```
* Fix documentation's typos

Signed-off-by: Victor Morales <v.morales@samsung.com>

* Add Spell checker CI action

Signed-off-by: Victor Morales <v.morales@samsung.com>
* fix ota-requestor attribute definitions and add write to chip-tool

* add ota requestor attributes to all-clusters-app

* generate for missing files

* add OTA Requestor Cluster to Endpoint 0 for Darwin tests

* regenerate files after darwin endpoint change

* regen files after master merge

* regen after master merge
…p#9573)

* Make platform PlatformManager::PostEvent() return status

#### Problem

`void PlatformManager::PostEvent()` can fail on some platforms, but does
not report this to callers.

In the project-chip#9543 case, an intermediate layer unconditionally returned
`CHIP_NO_ERROR`, and `Inet` code assumed it could actually test whether
`PostEvent()` succeeded.

Fixes project-chip#9543 _PacketBuffer leak_

#### Change overview

Made `PlatformManager::PostEvent()` return `CHIP_ERROR`.
Marked it `[[nodiscard]]` to catch all uses, and made callers
propagate or at least log any error.

#### Testing

CI.

* address review comments

* restyle

* replace PostEventLoggingErrors with PostEventOrDie

* fix P6
kpschoedel and others added 17 commits October 1, 2021 09:15
* Indirect access for System::Clock functions

#### Problem

Historically `System::Clock` operations were fixed global functions.
Recently PR#9547 converted to a mockable abstract base class, but
transitionally retained the global function API externally.

#### Change overview

- Added `SystemClock()`, following the model of `SystemLayer()`.
- Removed the old global functions.

#### Testing

CI. Revised `TestSystemClock` to use a `SetSystemClockForTesting()`
function, again following the System::Layer model.

* Use IsEarlier() in more places with a new AddOffset()

* restyle and fix copy-paste

* fix crossed paths and add transitional shims
…t-chip#10131)

* Change codegen templates to use namespaced accessors.

* Regenerate generated files

* Mass-replace consumers of attribute accessors with namespaced API.

This was generated by installing the git-extras package and running:

  git sed -f g "Attributes::\([GS]\)et\([A-Za-z0-9]*\)" "Attributes::\2::\1et"

* Update thermostat cluster to use namespaced accessors.

This changeset was generated by installing the git-extras package and then running:

  git sed -f g "\([^:]\)Get\([A-Za-z0-9]*\)(endpoint" "\1\2::Get(endpoint" -- src/app/clusters/thermostat-server/thermostat-server.cpp
  git sed -f g "\([^:A-Za-z]\)Set\([A-Za-z0-9]*\)(aEndpointId" "\1\2::Set(aEndpointId" -- src/app/clusters/thermostat-server/thermostat-server.cpp
  git sed -f g "\([^:]\)Get\([A-Za-z0-9]*\)(aEndpointId" "\1\2::Get(aEndpointId" -- src/app/clusters/thermostat-server/thermostat-server.cpp
Should prevent re-introduction of the sort of errors
project-chip#10064 fixed.
…#9989)

* Swith enable ipv4 to default to false

* Disable some more IPv4: in linux/darwin/tizen

* Add some ifdefs in linux mdns for IPv4

* Fix ifdef to if in ipv4 conditional

* Make minmdns server compile with IPv4 disabled

* Fix DNSResolver compilation with IPv4 disabled

* More compilation fixes for android arm64 working

* Do not allow platformconfig to control ipv4 - this should be using build flags

* Make TestInetLayerDNS compile with IPv4 off

* Convert and conditional-change several tests for IPv6 compatiblity

* Also update chip_im_initiator to use ipv6

* Move echo apps to ipv6, make minmdns server compile with clang when ipv4 is disabled

* Restyle fixes

* Update darwin mdns to enforce IPv6 only if ipv4 is disabled

* Allow more time for happy tests CI, fix typo in integration test name

* Change happy to only test IPv6

* Update happy comment regarding tests being run

* Restyle fixes

* Undo the "interation" typo fix, since the github required runs seems to be name based

* Undo the disabling ipv4 by default. Also change infineon P6 to inherit ipv4 defaults (i.e. true for now, but potentially false in the future)

* Allow TestDevice.cpp to be compiled with ipv4 turned on

* Update listening for IM/Echo requester/responder to use IPv4 if enabled, to preserve cirque compatibility

* Use input address for deciding IP types for requester/initiator apps

* Fix typo for previous commit: use Type() instead of GetAddressType

* Fix typo

* Change error to detail for logging IPv4 skip

* Change PtrResponse to use IPv6 instead of IPv4

* Do not callback on IPv4 result from avahi when IPv4 not available

* Ensure mdns callback on unexpected ipv4 is reported as an INVALID_ADDRESS error
…10142)

* add missing identify type & trigger effect command to chip-device-ctrl

* regen all
…Manager (project-chip#10122)

* Implement the ResetEthNetworkDiagnosticsCounts method for the zcl command

* Address the review comments
We're only generating them for the ones enabled in controller-clusters.zap.
…hip#10155)

* Consistently use the "Clusters" namespace, not "clusters".

* Regenerate generated files.
…t-chip#10167)

This was missed when RemoveAllFabrics was removed and UpdateOpCert renamed.
…ode) API that looks for the device over supported networks resolves project-chip#9343 (project-chip#9847)

* Add DeviceDiscoveryDelegate and use it in chip-tool

* [ChipTool] Add chip-tool commands to connect to a device using mdns, and possibly specifying subtypes

* Add SetUpCodePairer::PairDevice to handle scanning from a setup code
bzbarsky-apple pushed a commit that referenced this pull request Jul 11, 2022
It's not safe to access line editing state from the IO thread while
inside readline() on the main thread.

Remove the code that attempts to redraw readline after printing logs.
This avoids segfaults during logging at the cost of those logs
overwriting the prompt (this is not trivial to fix as readline
is a blocking API).

==================
WARNING: ThreadSanitizer: data race (pid=63005)
  Write of size 1 at 0x55f81c7745ff by main thread:
    #0 InteractiveStartCommand::ParseCommand(char*) ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:127 (chip-tool+0x874911)
    #1 InteractiveStartCommand::RunCommand() ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:85 (chip-tool+0x874594)
    #2 CHIPCommand::StartWaiting(std::chrono::duration<unsigned int, std::ratio<1l, 1000l> >) ../../examples/chip-tool/commands/common/CHIPCommand.cpp:408 (chip-tool+0x83e478)
    #3 CHIPCommand::Run() ../../examples/chip-tool/commands/common/CHIPCommand.cpp:187 (chip-tool+0x83c839)
    #4 Commands::RunCommand(int, char**, bool) ../../examples/chip-tool/commands/common/Commands.cpp:147 (chip-tool+0x85d4f7)
    #5 Commands::Run(int, char**) ../../examples/chip-tool/commands/common/Commands.cpp:51 (chip-tool+0x85c288)
    #6 main <null> (chip-tool+0x569c0a)

  Previous read of size 1 at 0x55f81c7745ff by thread T5 (mutexes: write M185):
    #0 LoggingCallback ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:46 (chip-tool+0x874479)
    #1 chip::Logging::LogV(unsigned char, unsigned char, char const*, __va_list_tag*) ../../src/lib/support/logging/CHIPLogging.cpp:221 (chip-tool+0x8ee4dc)
    #2 chip::Logging::Log(unsigned char, unsigned char, char const*, ...) ../../src/lib/support/logging/CHIPLogging.cpp:172 (chip-tool+0x8ee30a)
    #3 chip::app::ReadClient::RefreshLivenessCheckTimer() <null> (chip-tool+0x8b1746)
    #4 chip::app::ReadClient::ProcessSubscribeResponse(chip::System::PacketBufferHandle&&) ../../src/app/ReadClient.cpp:845 (chip-tool+0x8b20ec)
    #5 chip::app::ReadClient::OnMessageReceived(chip::Messaging::ExchangeContext*, chip::PayloadHeader const&, chip::System::PacketBufferHandle&&) ../../src/app/ReadClient.cpp:409 (chip-tool+0x8ae2a4)
    #6 chip::Messaging::ExchangeContext::HandleMessage(unsigned int, chip::PayloadHeader const&, chip::BitFlags<chip::Messaging::MessageFlagValues, unsigned int>, chip::System::PacketBufferHandle&&) <null> (chip-tool+0xa0517a)
    #7 operator()<chip::Messaging::ExchangeContext> ../../src/messaging/ExchangeMgr.cpp:219 (chip-tool+0xa08c73)
    #8 Call ../../src/lib/support/Pool.h:126 (chip-tool+0xa0912d)
    #9 chip::internal::HeapObjectList::ForEachNode(void*, chip::Loop (*)(void*, void*)) ../../src/lib/support/Pool.cpp:127 (chip-tool+0x8ee05a)
    #10 ForEachActiveObject<chip::Messaging::ExchangeManager::OnMessageReceived(const chip::PacketHeader&, const chip::PayloadHeader&, const chip::SessionHandle&, chip::SessionMessageDelegate::DuplicateMessage, chip::System::PacketBufferHandle&&)::<lambda(auto:2*)> > ../../src/lib/support/Pool.h:396 (chip-tool+0xa08d10)
    #11 chip::Messaging::ExchangeManager::OnMessageReceived(chip::PacketHeader const&, chip::PayloadHeader const&, chip::SessionHandle const&, chip::SessionMessageDelegate::DuplicateMessage, chip::System::PacketBufferHandle&&) ../../src/messaging/ExchangeMgr.cpp:212 (chip-tool+0xa07e91)
    #12 chip::SessionManager::SecureUnicastMessageDispatch(chip::PacketHeader const&, chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/SessionManager.cpp:616 (chip-tool+0xa1548b)
    #13 chip::SessionManager::OnMessageReceived(chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/SessionManager.cpp:443 (chip-tool+0xa14426)
    #14 chip::TransportMgrBase::HandleMessageReceived(chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/TransportMgrBase.cpp:76 (chip-tool+0xa17dfa)
    #15 chip::Transport::Base::HandleMessageReceived(chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/raw/Base.h:102 (chip-tool+0xb19728)
    #16 chip::Transport::UDP::OnUdpReceive(chip::Inet::UDPEndPoint*, chip::System::PacketBufferHandle&&, chip::Inet::IPPacketInfo const*) ../../src/transport/raw/UDP.cpp:122 (chip-tool+0xb1a48b)
    #17 chip::Inet::UDPEndPointImplSockets::HandlePendingIO(chip::BitFlags<chip::System::SocketEventFlags, unsigned char>) ../../src/inet/UDPEndPointImplSockets.cpp:688 (chip-tool+0xb00aa0)
    #18 chip::Inet::UDPEndPointImplSockets::HandlePendingIO(chip::BitFlags<chip::System::SocketEventFlags, unsigned char>, long) ../../src/inet/UDPEndPointImplSockets.cpp:569 (chip-tool+0xafff89)
    #19 chip::System::LayerImplSelect::HandleEvents() ../../src/system/SystemLayerImplSelect.cpp:406 (chip-tool+0xb07563)
    #20 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_RunEventLoop() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:181 (chip-tool+0x98a227)
    #21 chip::DeviceLayer::PlatformManager::RunEventLoop() ../../src/include/platform/PlatformManager.h:362 (chip-tool+0x988f75)
    #22 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::EventLoopTaskMain(void*) ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:205 (chip-tool+0x98a87c)

  Location is global '(anonymous namespace)::gIsCommandRunning' of size 1 at 0x55f81c7745ff (chip-tool+0x000000c485ff)

  Mutex M185 (0x55f81c776180) created at:
    #0 pthread_mutex_lock ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:4240 (libtsan.so.0+0x4f30a)
    #1 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_LockChipStack() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:78 (chip-tool+0x989e90)
    #2 chip::DeviceLayer::PlatformManager::LockChipStack() ../../src/include/platform/PlatformManager.h:410 (chip-tool+0x988fa5)
    #3 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_RunEventLoop() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:170 (chip-tool+0x98a147)
    #4 chip::DeviceLayer::PlatformManager::RunEventLoop() ../../src/include/platform/PlatformManager.h:362 (chip-tool+0x988f75)
    #5 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::EventLoopTaskMain(void*) ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:205 (chip-tool+0x98a87c)

  Thread T5 (tid=63013, running) created by main thread at:
    #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:969 (libtsan.so.0+0x5ad75)
    #1 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_StartEventLoopTask() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:231 (chip-tool+0x98a40a)
    #2 chip::DeviceLayer::PlatformManager::StartEventLoopTask() ../../src/include/platform/PlatformManager.h:375 (chip-tool+0xaacca2)
    #3 chip::Controller::DeviceControllerFactory::ServiceEvents() ../../src/controller/CHIPDeviceControllerFactory.cpp:331 (chip-tool+0xab0417)
    #4 CHIPCommand::StartWaiting(std::chrono::duration<unsigned int, std::ratio<1l, 1000l> >) ../../examples/chip-tool/commands/common/CHIPCommand.cpp:403 (chip-tool+0x83e353)
    #5 CHIPCommand::Run() ../../examples/chip-tool/commands/common/CHIPCommand.cpp:187 (chip-tool+0x83c839)
    #6 Commands::RunCommand(int, char**, bool) ../../examples/chip-tool/commands/common/Commands.cpp:147 (chip-tool+0x85d4f7)
    #7 Commands::Run(int, char**) ../../examples/chip-tool/commands/common/Commands.cpp:51 (chip-tool+0x85c288)
    #8 main <null> (chip-tool+0x569c0a)

SUMMARY: ThreadSanitizer: data race ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:127 in InteractiveStartCommand::ParseCommand(char*)
==================
bzbarsky-apple added a commit that referenced this pull request May 25, 2023
The failure looks like this:

  WARNING: ThreadSanitizer: race on NSMutableArray (pid=11619)
    Read-only access of NSMutableArray at 0x7b0c0005f5b0 by thread T3:
      #0 -[__NSArrayM countByEnumeratingWithState:objects:count:] <null>:2 (CoreFoundation:x86_64+0x4a338)
      #1 -[MTRDeviceControllerFactory(InternalMethods) operationalInstanceAdded:] MTRDeviceControllerFactory.mm:855 (Matter:x86_64+0x1fd2a)
      #2 MTROperationalBrowser::OnBrowse(_DNSServiceRef_t*, unsigned int, unsigned int, int, char const*, char const*, char const*, void*) MTROperationalBrowser.mm:100 (Matter:x86_64+0x20ee63c)
      #3 handle_browse_response <null>:2 (libsystem_dnssd.dylib:x86_64+0x3733)
      #4 _dispatch_client_callout <null>:2 (libdispatch.dylib:x86_64+0x3316)

    Previous modifying access of NSMutableArray at 0x7b0c0005f5b0 by main thread:
      #0 -[__NSArrayM addObject:] <null>:2 (CoreFoundation:x86_64+0x2457a)
      #1 -[MTRDeviceControllerFactory createController] MTRDeviceControllerFactory.mm:719 (Matter:x86_64+0x1cee3)
      #2 -[MTRDeviceControllerFactory createControllerOnExistingFabric:error:] MTRDeviceControllerFactory.mm:534 (Matter:x86_64+0x19792)

The basic problem is that we are in the middle of adding an object to
_controllers on the API consumer thread when on the Matter thread we get our
browse notification.

The changes here don't aim to lock around all access to _controllers, but just
to make sure that our mutations of it can't race with the access on the Matter
thread.  More coarse locking would need to be done very carefully, given the
amount of dispath_sync to the Matter thread we have going on.
bzbarsky-apple added a commit that referenced this pull request May 25, 2023
The failure looks like this:

  WARNING: ThreadSanitizer: race on NSMutableArray (pid=11619)
    Read-only access of NSMutableArray at 0x7b0c0005f5b0 by thread T3:
      #0 -[__NSArrayM countByEnumeratingWithState:objects:count:] <null>:2 (CoreFoundation:x86_64+0x4a338)
      #1 -[MTRDeviceControllerFactory(InternalMethods) operationalInstanceAdded:] MTRDeviceControllerFactory.mm:855 (Matter:x86_64+0x1fd2a)
      #2 MTROperationalBrowser::OnBrowse(_DNSServiceRef_t*, unsigned int, unsigned int, int, char const*, char const*, char const*, void*) MTROperationalBrowser.mm:100 (Matter:x86_64+0x20ee63c)
      #3 handle_browse_response <null>:2 (libsystem_dnssd.dylib:x86_64+0x3733)
      #4 _dispatch_client_callout <null>:2 (libdispatch.dylib:x86_64+0x3316)

    Previous modifying access of NSMutableArray at 0x7b0c0005f5b0 by main thread:
      #0 -[__NSArrayM addObject:] <null>:2 (CoreFoundation:x86_64+0x2457a)
      #1 -[MTRDeviceControllerFactory createController] MTRDeviceControllerFactory.mm:719 (Matter:x86_64+0x1cee3)
      #2 -[MTRDeviceControllerFactory createControllerOnExistingFabric:error:] MTRDeviceControllerFactory.mm:534 (Matter:x86_64+0x19792)

The basic problem is that we are in the middle of adding an object to
_controllers on the API consumer thread when on the Matter thread we get our
browse notification.

The changes here don't aim to lock around all access to _controllers, but just
to make sure that our mutations of it can't race with the access on the Matter
thread.  More coarse locking would need to be done very carefully, given the
amount of dispath_sync to the Matter thread we have going on.
bzbarsky-apple added a commit that referenced this pull request May 25, 2023
The failure looks like this:

  WARNING: ThreadSanitizer: race on NSMutableArray (pid=11619)
    Read-only access of NSMutableArray at 0x7b0c0005f5b0 by thread T3:
      #0 -[__NSArrayM countByEnumeratingWithState:objects:count:] <null>:2 (CoreFoundation:x86_64+0x4a338)
      #1 -[MTRDeviceControllerFactory(InternalMethods) operationalInstanceAdded:] MTRDeviceControllerFactory.mm:855 (Matter:x86_64+0x1fd2a)
      #2 MTROperationalBrowser::OnBrowse(_DNSServiceRef_t*, unsigned int, unsigned int, int, char const*, char const*, char const*, void*) MTROperationalBrowser.mm:100 (Matter:x86_64+0x20ee63c)
      #3 handle_browse_response <null>:2 (libsystem_dnssd.dylib:x86_64+0x3733)
      #4 _dispatch_client_callout <null>:2 (libdispatch.dylib:x86_64+0x3316)

    Previous modifying access of NSMutableArray at 0x7b0c0005f5b0 by main thread:
      #0 -[__NSArrayM addObject:] <null>:2 (CoreFoundation:x86_64+0x2457a)
      #1 -[MTRDeviceControllerFactory createController] MTRDeviceControllerFactory.mm:719 (Matter:x86_64+0x1cee3)
      #2 -[MTRDeviceControllerFactory createControllerOnExistingFabric:error:] MTRDeviceControllerFactory.mm:534 (Matter:x86_64+0x19792)

The basic problem is that we are in the middle of adding an object to
_controllers on the API consumer thread when on the Matter thread we get our
browse notification.

The changes here don't aim to lock around all access to _controllers, but just
to make sure that our mutations of it can't race with the access on the Matter
thread.  More coarse locking would need to be done very carefully, given the
amount of dispath_sync to the Matter thread we have going on.
bzbarsky-apple added a commit that referenced this pull request May 26, 2023
* Fix ThreadSanitizer failure in controller factory.

The failure looks like this:

  WARNING: ThreadSanitizer: race on NSMutableArray (pid=11619)
    Read-only access of NSMutableArray at 0x7b0c0005f5b0 by thread T3:
      #0 -[__NSArrayM countByEnumeratingWithState:objects:count:] <null>:2 (CoreFoundation:x86_64+0x4a338)
      #1 -[MTRDeviceControllerFactory(InternalMethods) operationalInstanceAdded:] MTRDeviceControllerFactory.mm:855 (Matter:x86_64+0x1fd2a)
      #2 MTROperationalBrowser::OnBrowse(_DNSServiceRef_t*, unsigned int, unsigned int, int, char const*, char const*, char const*, void*) MTROperationalBrowser.mm:100 (Matter:x86_64+0x20ee63c)
      #3 handle_browse_response <null>:2 (libsystem_dnssd.dylib:x86_64+0x3733)
      #4 _dispatch_client_callout <null>:2 (libdispatch.dylib:x86_64+0x3316)

    Previous modifying access of NSMutableArray at 0x7b0c0005f5b0 by main thread:
      #0 -[__NSArrayM addObject:] <null>:2 (CoreFoundation:x86_64+0x2457a)
      #1 -[MTRDeviceControllerFactory createController] MTRDeviceControllerFactory.mm:719 (Matter:x86_64+0x1cee3)
      #2 -[MTRDeviceControllerFactory createControllerOnExistingFabric:error:] MTRDeviceControllerFactory.mm:534 (Matter:x86_64+0x19792)

The basic problem is that we are in the middle of adding an object to
_controllers on the API consumer thread when on the Matter thread we get our
browse notification.

The changes here don't aim to lock around all access to _controllers, but just
to make sure that our mutations of it can't race with the access on the Matter
thread.  More coarse locking would need to be done very carefully, given the
amount of dispath_sync to the Matter thread we have going on.

* Address review comments.
bzbarsky-apple added a commit that referenced this pull request May 26, 2023
    WARNING: ThreadSanitizer: race on NSMutableDictionary (pid=72050)
      Modifying access of NSMutableDictionary at 0x7b0800149020 by thread T7:
        #0 -[__NSDictionaryM setObject:forKey:] <null>:2 (CoreFoundation:x86_64+0x40d4e)
        #1 -[MTRXPCListenerSample listener:shouldAcceptNewConnection:] MTRXPCListenerSampleTests.m:116 (MatterTests:x86_64+0x48437b)
        #2 service_connection_handler_make_connection <null>:2 (Foundation:x86_64+0xdea93)

      Previous modifying access of NSMutableDictionary at 0x7b0800149020 by thread T4:
        #0 -[__NSDictionaryM removeObjectForKey:] <null>:2 (CoreFoundation:x86_64+0x6cd6d)
        #1 __59-[MTRXPCListenerSample listener:shouldAcceptNewConnection:]_block_invoke MTRXPCListenerSampleTests.m:119 (MatterTests:x86_64+0x484630)
bzbarsky-apple added a commit that referenced this pull request May 26, 2023
WARNING: ThreadSanitizer: race on NSMutableDictionary (pid=72050)
      Modifying access of NSMutableDictionary at 0x7b0800149020 by thread T7:
        #0 -[__NSDictionaryM setObject:forKey:] <null>:2 (CoreFoundation:x86_64+0x40d4e)
        #1 -[MTRXPCListenerSample listener:shouldAcceptNewConnection:] MTRXPCListenerSampleTests.m:116 (MatterTests:x86_64+0x48437b)
        #2 service_connection_handler_make_connection <null>:2 (Foundation:x86_64+0xdea93)

      Previous modifying access of NSMutableDictionary at 0x7b0800149020 by thread T4:
        #0 -[__NSDictionaryM removeObjectForKey:] <null>:2 (CoreFoundation:x86_64+0x6cd6d)
        #1 __59-[MTRXPCListenerSample listener:shouldAcceptNewConnection:]_block_invoke MTRXPCListenerSampleTests.m:119 (MatterTests:x86_64+0x484630)
bzbarsky-apple added a commit that referenced this pull request Oct 9, 2023
…ist".

The typical failure there looks like this:

==29620==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 88 byte(s) in 1 object(s) allocated from:
    #0 0x106396e12 in calloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x51e12)
    #1 0x7ff800dc9789 in map_images_nolock+0x24b (libobjc.A.dylib:x86_64h+0x1789)
    #2 0x7ff800dc94db in map_images+0x42 (libobjc.A.dylib:x86_64h+0x14db)
    #3 0x113d721fa in invocation function for block in dyld4::RuntimeState::setObjCNotifiers(void (*)(unsigned int, char const* const*, mach_header const* const*), void (*)(char const*, mach_header const*), void (*)(char const*, mach_header const*))+0x112 (dyld:x86_64+0xf1fa)
    #4 0x113d6d6c8 in dyld4::RuntimeState::withLoadersReadLock(void () block_pointer)+0x28 (dyld:x86_64+0xa6c8)
    #5 0x113d720e1 in dyld4::RuntimeState::setObjCNotifiers(void (*)(unsigned int, char const* const*, mach_header const* const*), void (*)(char const*, mach_header const*), void (*)(char const*, mach_header const*))+0x51 (dyld:x86_64+0xf0e1)
    #6 0x113d85d44 in dyld4::APIs::_dyld_objc_notify_register(void (*)(unsigned int, char const* const*, mach_header const* const*), void (*)(char const*, mach_header const*), void (*)(char const*, mach_header const*))+0x4e (dyld:x86_64+0x22d44)
    #7 0x7ff800dc9343 in _objc_init+0x4fe (libobjc.A.dylib:x86_64h+0x1343)
    #8 0x7ff800d83992 in _os_object_init+0xc (libdispatch.dylib:x86_64+0x2992)
    #9 0x7ff800d911b7 in libdispatch_init+0x136 (libdispatch.dylib:x86_64+0x101b7)
    #10 0x7ff80bd34894 in libSystem_initializer+0xed (libSystem.B.dylib:x86_64+0x1894)
    #11 0x113d77e4e in invocation function for block in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const+0xb5 (dyld:x86_64+0x14e4e)
    #12 0x113d9eaac in invocation function for block in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const+0xf1 (dyld:x86_64+0x3baac)
    #13 0x113d95e25 in invocation function for block in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const+0x22c (dyld:x86_64+0x32e25)
    #14 0x113d64db2 in dyld3::MachOFile::forEachLoadCommand(Diagnostics&, void (load_command const*, bool&) block_pointer) const+0x80 (dyld:x86_64+0x1db2)
    #15 0x113d95bb6 in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const+0xb2 (dyld:x86_64+0x32bb6)
    #16 0x113d9e603 in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const+0x1d1 (dyld:x86_64+0x3b603)
    #17 0x113d77d81 in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const+0x8f (dyld:x86_64+0x14d81)
    #18 0x113d7e659 in dyld4::PrebuiltLoader::runInitializers(dyld4::RuntimeState&) const+0x1d (dyld:x86_64+0x1b659)
    #19 0x113d8b76d in dyld4::APIs::runAllInitializersForMain()+0x25 (dyld:x86_64+0x2876d)
    #20 0x113d6938c in dyld4::prepare(dyld4::APIs&, dyld3::MachOAnalyzer const*)+0xd72 (dyld:x86_64+0x638c)
    #21 0x113d684e3 in start+0x183 (dyld:x86_64+0x54e3)
bzbarsky-apple added a commit that referenced this pull request Oct 9, 2023
…ist".

The typical failure there looks like this:

==29620==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 88 byte(s) in 1 object(s) allocated from:
    #0 0x106396e12 in calloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x51e12)
    #1 0x7ff800dc9789 in map_images_nolock+0x24b (libobjc.A.dylib:x86_64h+0x1789)
    #2 0x7ff800dc94db in map_images+0x42 (libobjc.A.dylib:x86_64h+0x14db)
    #3 0x113d721fa in invocation function for block in dyld4::RuntimeState::setObjCNotifiers(void (*)(unsigned int, char const* const*, mach_header const* const*), void (*)(char const*, mach_header const*), void (*)(char const*, mach_header const*))+0x112 (dyld:x86_64+0xf1fa)
    #4 0x113d6d6c8 in dyld4::RuntimeState::withLoadersReadLock(void () block_pointer)+0x28 (dyld:x86_64+0xa6c8)
    #5 0x113d720e1 in dyld4::RuntimeState::setObjCNotifiers(void (*)(unsigned int, char const* const*, mach_header const* const*), void (*)(char const*, mach_header const*), void (*)(char const*, mach_header const*))+0x51 (dyld:x86_64+0xf0e1)
    #6 0x113d85d44 in dyld4::APIs::_dyld_objc_notify_register(void (*)(unsigned int, char const* const*, mach_header const* const*), void (*)(char const*, mach_header const*), void (*)(char const*, mach_header const*))+0x4e (dyld:x86_64+0x22d44)
    #7 0x7ff800dc9343 in _objc_init+0x4fe (libobjc.A.dylib:x86_64h+0x1343)
    #8 0x7ff800d83992 in _os_object_init+0xc (libdispatch.dylib:x86_64+0x2992)
    #9 0x7ff800d911b7 in libdispatch_init+0x136 (libdispatch.dylib:x86_64+0x101b7)
    #10 0x7ff80bd34894 in libSystem_initializer+0xed (libSystem.B.dylib:x86_64+0x1894)
    #11 0x113d77e4e in invocation function for block in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const+0xb5 (dyld:x86_64+0x14e4e)
    #12 0x113d9eaac in invocation function for block in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const+0xf1 (dyld:x86_64+0x3baac)
    #13 0x113d95e25 in invocation function for block in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const+0x22c (dyld:x86_64+0x32e25)
    #14 0x113d64db2 in dyld3::MachOFile::forEachLoadCommand(Diagnostics&, void (load_command const*, bool&) block_pointer) const+0x80 (dyld:x86_64+0x1db2)
    #15 0x113d95bb6 in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const+0xb2 (dyld:x86_64+0x32bb6)
    #16 0x113d9e603 in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const+0x1d1 (dyld:x86_64+0x3b603)
    #17 0x113d77d81 in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const+0x8f (dyld:x86_64+0x14d81)
    #18 0x113d7e659 in dyld4::PrebuiltLoader::runInitializers(dyld4::RuntimeState&) const+0x1d (dyld:x86_64+0x1b659)
    #19 0x113d8b76d in dyld4::APIs::runAllInitializersForMain()+0x25 (dyld:x86_64+0x2876d)
    #20 0x113d6938c in dyld4::prepare(dyld4::APIs&, dyld3::MachOAnalyzer const*)+0xd72 (dyld:x86_64+0x638c)
    #21 0x113d684e3 in start+0x183 (dyld:x86_64+0x54e3)
bzbarsky-apple added a commit that referenced this pull request Oct 10, 2023
…ist". (project-chip#29666)

The typical failure there looks like this:

==29620==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 88 byte(s) in 1 object(s) allocated from:
    #0 0x106396e12 in calloc+0xa2 (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x51e12)
    #1 0x7ff800dc9789 in map_images_nolock+0x24b (libobjc.A.dylib:x86_64h+0x1789)
    #2 0x7ff800dc94db in map_images+0x42 (libobjc.A.dylib:x86_64h+0x14db)
    #3 0x113d721fa in invocation function for block in dyld4::RuntimeState::setObjCNotifiers(void (*)(unsigned int, char const* const*, mach_header const* const*), void (*)(char const*, mach_header const*), void (*)(char const*, mach_header const*))+0x112 (dyld:x86_64+0xf1fa)
    #4 0x113d6d6c8 in dyld4::RuntimeState::withLoadersReadLock(void () block_pointer)+0x28 (dyld:x86_64+0xa6c8)
    #5 0x113d720e1 in dyld4::RuntimeState::setObjCNotifiers(void (*)(unsigned int, char const* const*, mach_header const* const*), void (*)(char const*, mach_header const*), void (*)(char const*, mach_header const*))+0x51 (dyld:x86_64+0xf0e1)
    #6 0x113d85d44 in dyld4::APIs::_dyld_objc_notify_register(void (*)(unsigned int, char const* const*, mach_header const* const*), void (*)(char const*, mach_header const*), void (*)(char const*, mach_header const*))+0x4e (dyld:x86_64+0x22d44)
    #7 0x7ff800dc9343 in _objc_init+0x4fe (libobjc.A.dylib:x86_64h+0x1343)
    #8 0x7ff800d83992 in _os_object_init+0xc (libdispatch.dylib:x86_64+0x2992)
    #9 0x7ff800d911b7 in libdispatch_init+0x136 (libdispatch.dylib:x86_64+0x101b7)
    #10 0x7ff80bd34894 in libSystem_initializer+0xed (libSystem.B.dylib:x86_64+0x1894)
    #11 0x113d77e4e in invocation function for block in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const+0xb5 (dyld:x86_64+0x14e4e)
    #12 0x113d9eaac in invocation function for block in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const+0xf1 (dyld:x86_64+0x3baac)
    #13 0x113d95e25 in invocation function for block in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const+0x22c (dyld:x86_64+0x32e25)
    #14 0x113d64db2 in dyld3::MachOFile::forEachLoadCommand(Diagnostics&, void (load_command const*, bool&) block_pointer) const+0x80 (dyld:x86_64+0x1db2)
    #15 0x113d95bb6 in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const+0xb2 (dyld:x86_64+0x32bb6)
    #16 0x113d9e603 in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const+0x1d1 (dyld:x86_64+0x3b603)
    #17 0x113d77d81 in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const+0x8f (dyld:x86_64+0x14d81)
    #18 0x113d7e659 in dyld4::PrebuiltLoader::runInitializers(dyld4::RuntimeState&) const+0x1d (dyld:x86_64+0x1b659)
    #19 0x113d8b76d in dyld4::APIs::runAllInitializersForMain()+0x25 (dyld:x86_64+0x2876d)
    #20 0x113d6938c in dyld4::prepare(dyld4::APIs&, dyld3::MachOAnalyzer const*)+0xd72 (dyld:x86_64+0x638c)
    #21 0x113d684e3 in start+0x183 (dyld:x86_64+0x54e3)
bzbarsky-apple pushed a commit that referenced this pull request Apr 24, 2024
* pw_unit_test migration: lib support batch #1

* apply restyled patch

* integrate comments
bzbarsky-apple pushed a commit that referenced this pull request Jul 27, 2024
…t-chip#34322)

* Fix the metadata module to make the wheel buildable

* Added the fix for metadata import

* Added fixes as per module build for metadata_testing_infrastructure

* Restyled by shfmt

* Make CI checks run

* Update build_python.sh

* Delete admin_storage.json

* Make metadata_parser python module wheel buildable within src/python_testing/matter_testing_infrastructure/ (#1)

* Adding metadata buildable wheel to src/python_testing/matter_testing_infrastructure
* Update build_python.sh
* Delete src/python_testing/matter_testing_infrastructure/build/lib/metadata_parser directory

---------

Co-authored-by: Restyled.io <commits@restyled.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.