Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve detection of "historical" events in Matter.framework. #36506

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

bzbarsky-apple
Copy link
Contributor

We were marking events as "historical" if there were any attributes in the same report that have the C quality. This had a few problems:

  • Some devices randomly report C-quality attributes, for various reasons.
  • Some attributes stop having the C quality in favor of Q, in some cases, so this is not very forward-compatible.

The fix is to remove this C-quality check altogether. At that point we have the following situations:

  1. We're setting up a new subscription and getting our initial priming report. This case will still have _receivingPrimingReport set, and will cause any events reported as part of priming to be marked "historical".
  2. We are getting a "priming" report from a server-side subscription resumption after the server timed out on a subscription. If this happens before the max-interval elapses, we will see this as a normal report on our subscription, but with all attributes and events the server knows about included. In this case, we mark ourselves as being in a "priming report" if we get an event that has an event number we have already seen, so that we mark any not-seen-before events in that report as "historical", since we don't know how long ago they are from.
  3. We are getting a normal incremental report, and will not mark any such events as "historical".

Since we are now keeping track of last-received event numbers, we can also use those for our subscriptions and can filter out any events that have an event number no greater than our last-observed one.

@bzbarsky-apple bzbarsky-apple requested a review from a team as a code owner November 14, 2024 21:23
Copy link

Review changes with  SemanticDiff

Copy link

github-actions bot commented Nov 14, 2024

PR #36506: Size comparison from 73ff58f to 8a9833a

Full report (68 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
platform target config section 73ff58f 8a9833a change % change
bl602 lighting-app bl602+mfd+littlefs+rpc FLASH 1349448 1349448 0 0.0
RAM 104080 104080 0 0.0
bl702 lighting-app bl702+eth FLASH 649334 649334 0 0.0
RAM 25329 25329 0 0.0
bl702+wifi FLASH 826914 826914 0 0.0
RAM 14069 14069 0 0.0
bl706+mfd+rpc+littlefs FLASH 1055112 1055112 0 0.0
RAM 23909 23909 0 0.0
bl702l lighting-app bl702l+mfd+littlefs FLASH 976486 976486 0 0.0
RAM 16556 16556 0 0.0
cc13x4_26x4 lighting-app LP_EM_CC1354P10_6 FLASH 837852 837852 0 0.0
RAM 123632 123632 0 0.0
lock-ftd LP_EM_CC1354P10_6 FLASH 823768 823768 0 0.0
RAM 125520 125520 0 0.0
pump-app LP_EM_CC1354P10_6 FLASH 770492 770492 0 0.0
RAM 113988 113988 0 0.0
pump-controller-app LP_EM_CC1354P10_6 FLASH 754704 754704 0 0.0
RAM 114196 114196 0 0.0
cc32xx air-purifier CC3235SF_LAUNCHXL FLASH 628706 628706 0 0.0
RAM 205784 205784 0 0.0
lock CC3235SF_LAUNCHXL FLASH 667242 667242 0 0.0
RAM 205936 205936 0 0.0
cyw30739 light CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 679489 679489 0 0.0
RAM 78692 78692 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 699333 699333 0 0.0
RAM 81324 81324 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 699333 699333 0 0.0
RAM 81324 81324 0 0.0
CYW930739M2EVB-02 unknown 2040 2040 0 0.0
FLASH 656277 656277 0 0.0
RAM 73760 73760 0 0.0
light-switch CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 616345 616345 0 0.0
RAM 71644 71644 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 635973 635973 0 0.0
RAM 74196 74196 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 635973 635973 0 0.0
RAM 74196 74196 0 0.0
lock CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 635801 635801 0 0.0
RAM 74692 74692 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 655509 655509 0 0.0
RAM 77244 77244 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 655509 655509 0 0.0
RAM 77244 77244 0 0.0
thermostat CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 612285 612285 0 0.0
RAM 68780 68780 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 632137 632137 0 0.0
RAM 71412 71412 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 632137 632137 0 0.0
RAM 71412 71412 0 0.0
efr32 lock-app BRD4187C FLASH 927596 927596 0 0.0
RAM 160164 160164 0 0.0
BRD4338a FLASH 743696 743688 -8 -0.0
RAM 233296 233296 0 0.0
window-app BRD4187C FLASH 1018816 1018808 -8 -0.0
RAM 128264 128264 0 0.0
esp32 all-clusters-app c3devkit DRAM 95336 95336 0 0.0
FLASH 1541134 1541134 0 0.0
IRAM 82542 82542 0 0.0
m5stack DRAM 116264 116264 0 0.0
FLASH 1549822 1549822 0 0.0
IRAM 117039 117039 0 0.0
linux air-purifier-app debug unknown 4712 4712 0 0.0
FLASH 2702273 2702273 0 0.0
RAM 129728 129728 0 0.0
all-clusters-app debug unknown 5552 5552 0 0.0
FLASH 5993494 5993494 0 0.0
RAM 523456 523456 0 0.0
all-clusters-minimal-app debug unknown 5448 5448 0 0.0
FLASH 5331632 5331632 0 0.0
RAM 242512 242512 0 0.0
bridge-app debug unknown 5432 5432 0 0.0
FLASH 4671272 4671272 0 0.0
RAM 218464 218464 0 0.0
chip-tool debug unknown 5984 5984 0 0.0
FLASH 12830216 12830216 0 0.0
RAM 582226 582226 0 0.0
chip-tool-ipv6only arm64 unknown 21288 21288 0 0.0
FLASH 10966576 10966576 0 0.0
RAM 633144 633144 0 0.0
fabric-admin debug unknown 5808 5808 0 0.0
FLASH 1123780 1123780 0 0.0
RAM 582578 582578 0 0.0
fabric-bridge-app debug unknown 4688 4688 0 0.0
FLASH 4495542 4495542 0 0.0
RAM 205424 205424 0 0.0
fabric-sync debug unknown 4896 4896 0 0.0
FLASH 5456661 5456661 0 0.0
RAM 470512 470512 0 0.0
lighting-app debug+rpc+ui unknown 6096 6096 0 0.0
FLASH 5607201 5607201 0 0.0
RAM 228616 228616 0 0.0
lock-app debug unknown 5368 5368 0 0.0
FLASH 4720684 4720684 0 0.0
RAM 204632 204632 0 0.0
ota-provider-app debug unknown 4744 4744 0 0.0
FLASH 4346146 4346146 0 0.0
RAM 198304 198304 0 0.0
ota-requestor-app debug unknown 4680 4680 0 0.0
FLASH 4484580 4484580 0 0.0
RAM 202888 202888 0 0.0
shell debug unknown 4240 4240 0 0.0
FLASH 3016701 3016701 0 0.0
RAM 160360 160360 0 0.0
thermostat-no-ble arm64 unknown 9496 9496 0 0.0
FLASH 4093392 4093392 0 0.0
RAM 242984 242984 0 0.0
tv-app debug unknown 5664 5664 0 0.0
FLASH 5943813 5943813 0 0.0
RAM 595920 595920 0 0.0
tv-casting-app debug unknown 5232 5232 0 0.0
FLASH 11035981 11035981 0 0.0
RAM 691960 691960 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 FLASH 915876 915876 0 0.0
RAM 143289 143289 0 0.0
nrf7002dk_nrf5340_cpuapp FLASH 888100 888100 0 0.0
RAM 141476 141476 0 0.0
all-clusters-minimal-app nrf52840dk_nrf52840 FLASH 850072 850072 0 0.0
RAM 142197 142197 0 0.0
nxp contact k32w0+release FLASH 584128 584128 0 0.0
RAM 71048 71048 0 0.0
mcxw71+release FLASH 598664 598664 0 0.0
RAM 63144 63144 0 0.0
light k32w0+release FLASH 611172 611172 0 0.0
RAM 70440 70440 0 0.0
k32w1+release FLASH 684976 684976 0 0.0
RAM 48776 48776 0 0.0
lock mcxw71+release FLASH 748512 748512 0 0.0
RAM 67300 67300 0 0.0
psoc6 all-clusters cy8ckit_062s2_43012 FLASH 1643604 1643604 0 0.0
RAM 212064 212064 0 0.0
all-clusters-minimal cy8ckit_062s2_43012 FLASH 1551596 1551596 0 0.0
RAM 208864 208864 0 0.0
light cy8ckit_062s2_43012 FLASH 1466620 1466620 0 0.0
RAM 200848 200848 0 0.0
lock cy8ckit_062s2_43012 FLASH 1464812 1464812 0 0.0
RAM 225208 225208 0 0.0
qpg lighting-app qpg6105+debug FLASH 662136 662136 0 0.0
RAM 105384 105384 0 0.0
lock-app qpg6105+debug FLASH 620244 620244 0 0.0
RAM 99836 99836 0 0.0
stm32 light STM32WB5MM-DK FLASH 482944 482944 0 0.0
RAM 144848 144848 0 0.0
telink bridge-app tlsr9258a FLASH 684686 684686 0 0.0
RAM 91536 91536 0 0.0
contact-sensor-app tlsr9528a_retention FLASH 622694 622694 0 0.0
RAM 50472 50472 0 0.0
light-switch-app-ota-compress-lzma-shell-factory-data tlsr9528a FLASH 710624 710624 0 0.0
RAM 73812 73812 0 0.0
lighting-app-ota-factory-data tlsr9118bdk40d FLASH 628992 628992 0 0.0
RAM 145392 145392 0 0.0
lighting-app-ota-rpc-factory-data-4mb tlsr9518adk80d FLASH 814880 814880 0 0.0
RAM 100012 100012 0 0.0
tizen all-clusters-app arm unknown 4968 4968 0 0.0
FLASH 1723900 1723900 0 0.0
RAM 90628 90628 0 0.0
chip-tool-ubsan arm unknown 10776 10776 0 0.0
FLASH 17938758 17938758 0 0.0
RAM 7825888 7825888 0 0.0

We were marking events as "historical" if there were any attributes in the same
report that have the C quality. This had a few problems:

* Some devices randomly report C-quality attributes, for various reasons.
* Some attributes stop having the C quality in favor of Q, in some cases, so
  this is not very forward-compatible.

The fix is to remove this C-quality check altogether.  At that point we have the
following situations:

1. We're setting up a new subscription and getting our initial priming report.
   This case will still have _receivingPrimingReport set, and will cause any
   events reported as part of priming to be marked "historical".
2. We are getting a "priming" report from a server-side subscription resumption
   after the server timed out on a subscription.  If this happens before the
   max-interval elapses, we will see this as a normal report on our
   subscription, but with all attributes and events the server knows about
   included.  In this case, we mark ourselves as being in a "priming report" if
   we get an event that has an event number we have already seen, so that we
   mark any not-seen-before events in that report as "historical", since we
   don't know how long ago they are from.
3. We are getting a normal incremental report, and will not mark any such events
   as "historical".

Since we are now keeping track of last-received event numbers, we can also use
those for our subscriptions and can filter out any events that have an event
number no greater than our last-observed one.
Copy link

github-actions bot commented Nov 14, 2024

PR #36506: Size comparison from 1bdcb74 to 31c5839

Full report (68 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
platform target config section 1bdcb74 31c5839 change % change
bl602 lighting-app bl602+mfd+littlefs+rpc FLASH 1349448 1349448 0 0.0
RAM 104080 104080 0 0.0
bl702 lighting-app bl702+eth FLASH 649334 649334 0 0.0
RAM 25329 25329 0 0.0
bl702+wifi FLASH 826914 826914 0 0.0
RAM 14069 14069 0 0.0
bl706+mfd+rpc+littlefs FLASH 1055112 1055112 0 0.0
RAM 23909 23909 0 0.0
bl702l lighting-app bl702l+mfd+littlefs FLASH 976486 976486 0 0.0
RAM 16556 16556 0 0.0
cc13x4_26x4 lighting-app LP_EM_CC1354P10_6 FLASH 837852 837852 0 0.0
RAM 123632 123632 0 0.0
lock-ftd LP_EM_CC1354P10_6 FLASH 823768 823768 0 0.0
RAM 125520 125520 0 0.0
pump-app LP_EM_CC1354P10_6 FLASH 770492 770492 0 0.0
RAM 113988 113988 0 0.0
pump-controller-app LP_EM_CC1354P10_6 FLASH 754704 754704 0 0.0
RAM 114196 114196 0 0.0
cc32xx air-purifier CC3235SF_LAUNCHXL FLASH 628706 628706 0 0.0
RAM 205784 205784 0 0.0
lock CC3235SF_LAUNCHXL FLASH 667242 667242 0 0.0
RAM 205936 205936 0 0.0
cyw30739 light CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 679489 679489 0 0.0
RAM 78692 78692 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 699333 699333 0 0.0
RAM 81324 81324 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 699333 699333 0 0.0
RAM 81324 81324 0 0.0
CYW930739M2EVB-02 unknown 2040 2040 0 0.0
FLASH 656277 656277 0 0.0
RAM 73760 73760 0 0.0
light-switch CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 616345 616345 0 0.0
RAM 71644 71644 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 635973 635973 0 0.0
RAM 74196 74196 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 635973 635973 0 0.0
RAM 74196 74196 0 0.0
lock CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 635801 635801 0 0.0
RAM 74692 74692 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 655509 655509 0 0.0
RAM 77244 77244 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 655509 655509 0 0.0
RAM 77244 77244 0 0.0
thermostat CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 612285 612285 0 0.0
RAM 68780 68780 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 632137 632137 0 0.0
RAM 71412 71412 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 632137 632137 0 0.0
RAM 71412 71412 0 0.0
efr32 lock-app BRD4187C FLASH 927596 927596 0 0.0
RAM 160164 160164 0 0.0
BRD4338a FLASH 743696 743688 -8 -0.0
RAM 233296 233296 0 0.0
window-app BRD4187C FLASH 1018816 1018808 -8 -0.0
RAM 128264 128264 0 0.0
esp32 all-clusters-app c3devkit DRAM 95336 95336 0 0.0
FLASH 1541134 1541134 0 0.0
IRAM 82542 82542 0 0.0
m5stack DRAM 116264 116264 0 0.0
FLASH 1549822 1549822 0 0.0
IRAM 117039 117039 0 0.0
linux air-purifier-app debug unknown 4712 4712 0 0.0
FLASH 2702273 2702273 0 0.0
RAM 129728 129728 0 0.0
all-clusters-app debug unknown 5552 5552 0 0.0
FLASH 5993494 5993494 0 0.0
RAM 523456 523456 0 0.0
all-clusters-minimal-app debug unknown 5448 5448 0 0.0
FLASH 5331632 5331632 0 0.0
RAM 242512 242512 0 0.0
bridge-app debug unknown 5432 5432 0 0.0
FLASH 4671272 4671272 0 0.0
RAM 218464 218464 0 0.0
chip-tool debug unknown 5984 5984 0 0.0
FLASH 12830462 12830462 0 0.0
RAM 582226 582226 0 0.0
chip-tool-ipv6only arm64 unknown 21288 21288 0 0.0
FLASH 10966816 10966816 0 0.0
RAM 633144 633144 0 0.0
fabric-admin debug unknown 5808 5808 0 0.0
FLASH 1123780 1123780 0 0.0
RAM 582578 582578 0 0.0
fabric-bridge-app debug unknown 4688 4688 0 0.0
FLASH 4495542 4495542 0 0.0
RAM 205424 205424 0 0.0
fabric-sync debug unknown 4896 4896 0 0.0
FLASH 5456661 5456661 0 0.0
RAM 470512 470512 0 0.0
lighting-app debug+rpc+ui unknown 6096 6096 0 0.0
FLASH 5607201 5607201 0 0.0
RAM 228616 228616 0 0.0
lock-app debug unknown 5368 5368 0 0.0
FLASH 4720684 4720684 0 0.0
RAM 204632 204632 0 0.0
ota-provider-app debug unknown 4744 4744 0 0.0
FLASH 4346146 4346146 0 0.0
RAM 198304 198304 0 0.0
ota-requestor-app debug unknown 4680 4680 0 0.0
FLASH 4484580 4484580 0 0.0
RAM 202888 202888 0 0.0
shell debug unknown 4240 4240 0 0.0
FLASH 3016701 3016701 0 0.0
RAM 160360 160360 0 0.0
thermostat-no-ble arm64 unknown 9496 9496 0 0.0
FLASH 4093392 4093392 0 0.0
RAM 242984 242984 0 0.0
tv-app debug unknown 5664 5664 0 0.0
FLASH 5943813 5943813 0 0.0
RAM 595920 595920 0 0.0
tv-casting-app debug unknown 5232 5232 0 0.0
FLASH 11035981 11035981 0 0.0
RAM 691960 691960 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 FLASH 915876 915876 0 0.0
RAM 143289 143289 0 0.0
nrf7002dk_nrf5340_cpuapp FLASH 888100 888100 0 0.0
RAM 141476 141476 0 0.0
all-clusters-minimal-app nrf52840dk_nrf52840 FLASH 850072 850072 0 0.0
RAM 142197 142197 0 0.0
nxp contact k32w0+release FLASH 584128 584128 0 0.0
RAM 71048 71048 0 0.0
mcxw71+release FLASH 598664 598664 0 0.0
RAM 63144 63144 0 0.0
light k32w0+release FLASH 611172 611172 0 0.0
RAM 70440 70440 0 0.0
k32w1+release FLASH 684976 684976 0 0.0
RAM 48776 48776 0 0.0
lock mcxw71+release FLASH 748512 748512 0 0.0
RAM 67300 67300 0 0.0
psoc6 all-clusters cy8ckit_062s2_43012 FLASH 1643604 1643604 0 0.0
RAM 212064 212064 0 0.0
all-clusters-minimal cy8ckit_062s2_43012 FLASH 1551596 1551596 0 0.0
RAM 208864 208864 0 0.0
light cy8ckit_062s2_43012 FLASH 1466620 1466620 0 0.0
RAM 200848 200848 0 0.0
lock cy8ckit_062s2_43012 FLASH 1464812 1464812 0 0.0
RAM 225208 225208 0 0.0
qpg lighting-app qpg6105+debug FLASH 662136 662136 0 0.0
RAM 105384 105384 0 0.0
lock-app qpg6105+debug FLASH 620244 620244 0 0.0
RAM 99836 99836 0 0.0
stm32 light STM32WB5MM-DK FLASH 482944 482944 0 0.0
RAM 144848 144848 0 0.0
telink bridge-app tlsr9258a FLASH 684686 684686 0 0.0
RAM 91536 91536 0 0.0
contact-sensor-app tlsr9528a_retention FLASH 622694 622694 0 0.0
RAM 50472 50472 0 0.0
light-switch-app-ota-compress-lzma-shell-factory-data tlsr9528a FLASH 710624 710624 0 0.0
RAM 73812 73812 0 0.0
lighting-app-ota-factory-data tlsr9118bdk40d FLASH 628992 628992 0 0.0
RAM 145392 145392 0 0.0
lighting-app-ota-rpc-factory-data-4mb tlsr9518adk80d FLASH 814880 814880 0 0.0
RAM 100012 100012 0 0.0
tizen all-clusters-app arm unknown 4968 4968 0 0.0
FLASH 1723900 1723900 0 0.0
RAM 90628 90628 0 0.0
chip-tool-ubsan arm unknown 10776 10776 0 0.0
FLASH 17939086 17939086 0 0.0
RAM 7825976 7825976 0 0.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Todo
Development

Successfully merging this pull request may close these issues.

1 participant