Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use 5% tolerance when checking CurrentLevel attribute in TC-LVL-6.1 #20030

Merged
merged 2 commits into from
Jun 28, 2022
Merged

Use 5% tolerance when checking CurrentLevel attribute in TC-LVL-6.1 #20030

merged 2 commits into from
Jun 28, 2022

Conversation

manjunath-grl
Copy link
Contributor

@manjunath-grl manjunath-grl commented Jun 28, 2022 •
edited by andy31415
Loading

Modified script TC-LVl-6.1
We are observing inconsistent values for the CurrentLevel attribute after the wait time, instead of checking for specific value we have updated the script to check for a range with 5% tolerance
Added auto generated files

Attached execution logs for reference:
Execution_log.txt

Fixes #19869

@boring-cyborg boring-cyborg bot added the app label Jun 28, 2022
@github-actions
Copy link

github-actions bot commented Jun 28, 2022 •
edited
Loading

PR #20030: Size comparison from 9186af7 to c65f465

Increases (5 builds for cc13x2_26x2, esp32, linux, telink)
platform target config section 9186af7 c65f465 change % change
cc13x2_26x2 all-clusters-app LP_CC2652R7 (read/write) 188068 188076 8 0.0
esp32 all-clusters-app c3devkit (read only) 1018198 1018200 2 0.0
.flash.text 1018198 1018200 2 0.0
linux chip-tool debug (read only) 10172005 10172533 528 0.0
.text 8254117 8254645 528 0.0
chip-tool-no-interactive-ipv6only arm64 (read only) 9904116 9904580 464 0.0
.rodata 472164 472180 16 0.0
.text 7892372 7892820 448 0.0
telink light-switch-app tlsr9518adk80d (read/write) 797712 797720 8 0.0
text 564552 564554 2 0.0
Decreases (1 build for cc13x2_26x2)
platform target config section 9186af7 c65f465 change % change
cc13x2_26x2 all-clusters-app LP_CC2652R7 (read only) 664915 664907 -8 -0.0
.text 576708 576700 -8 -0.0
Full report (41 builds for cc13x2_26x2, cyw30739, efr32, esp32, k32w, linux, mbed, nrfconnect, p6, telink)
platform target config section 9186af7 c65f465 change % change
cc13x2_26x2 all-clusters-app LP_CC2652R7 (read only) 664915 664907 -8 -0.0
(read/write) 188068 188076 8 0.0
.bss 75876 75876 0 0.0
.data 3356 3356 0 0.0
.rodata 87891 87891 0 0.0
.text 576708 576700 -8 -0.0
all-clusters-minimal-app LP_CC2652R7 (read only) 653655 653655 0 0.0
(read/write) 198624 198624 0 0.0
.bss 75172 75172 0 0.0
.data 3356 3356 0 0.0
.rodata 91359 91359 0 0.0
.text 561980 561980 0 0.0
lock-ftd LP_CC2652R7 (read only) 667455 667455 0 0.0
(read/write) 175672 175672 0 0.0
.bss 72908 72908 0 0.0
.data 3280 3280 0 0.0
.rodata 75983 75983 0 0.0
.text 590992 590992 0 0.0
lock-mtd LP_CC2652R7 (read only) 616871 616871 0 0.0
(read/write) 146024 146024 0 0.0
.bss 68628 68628 0 0.0
.data 3280 3280 0 0.0
.rodata 75863 75863 0 0.0
.text 540520 540520 0 0.0
pump-app LP_CC2652R7 (read only) 677023 677023 0 0.0
(read/write) 166960 166960 0 0.0
.bss 72996 72996 0 0.0
.data 3280 3280 0 0.0
.rodata 88367 88367 0 0.0
.text 588172 588172 0 0.0
pump-controller-app LP_CC2652R7 (read only) 662927 662927 0 0.0
(read/write) 181176 181176 0 0.0
.bss 73116 73116 0 0.0
.data 3276 3276 0 0.0
.rodata 84215 84215 0 0.0
.text 578232 578232 0 0.0
shell LP_CC2652R7 (read only) 657370 657370 0 0.0
(read/write) 191116 191116 0 0.0
.bss 78180 78180 0 0.0
.data 3360 3360 0 0.0
.rodata 84650 84650 0 0.0
.text 572404 572404 0 0.0
cyw30739 light cyw930739m2evb_01 (read/write) 580462 580462 0 0.0
.app_xip_area 457472 457472 0 0.0
.bss 65952 65952 0 0.0
.data 716 716 0 0.0
.rodata 0 0 0 0.0
.text 112 112 0 0.0
lock cyw930739m2evb_01 (read/write) 582030 582030 0 0.0
.app_xip_area 458848 458848 0 0.0
.bss 66144 66144 0 0.0
.data 720 720 0 0.0
.rodata 0 0 0 0.0
.text 112 112 0 0.0
ota-requestor-no-progress-logging cyw930739m2evb_01 (read/write) 583502 583502 0 0.0
.app_xip_area 461360 461360 0 0.0
.bss 65160 65160 0 0.0
.data 660 660 0 0.0
.rodata 0 0 0 0.0
.text 112 112 0 0.0
efr32 lighting-app BRD4161A (read/write) 1081412 1081412 0 0.0
.bss 134756 134756 0 0.0
.data 2048 2048 0 0.0
.text 944588 944588 0 0.0
BRD4161A+rpc (read/write) 1135684 1135684 0 0.0
.bss 151444 151444 0 0.0
.data 2260 2260 0 0.0
.text 981960 981960 0 0.0
BRD4161A+rs911x (read/write) 943896 943896 0 0.0
.bss 140472 140472 0 0.0
.data 2048 2048 0 0.0
.text 801356 801356 0 0.0
lock-app BRD4161A+wf200 (read/write) 1121056 1121056 0 0.0
.bss 140920 140920 0 0.0
.data 2060 2060 0 0.0
.text 978056 978056 0 0.0
window-app BRD4161A (read/write) 1066676 1066676 0 0.0
.bss 134836 134836 0 0.0
.data 2076 2076 0 0.0
.text 929740 929740 0 0.0
esp32 all-clusters-app c3devkit (read only) 1018198 1018200 2 0.0
(read/write) 1486498 1486498 0 0.0
.dram0.bss 71824 71824 0 0.0
.dram0.data 14624 14624 0 0.0
.flash.rodata 214624 214624 0 0.0
.flash.text 1018198 1018200 2 0.0
.iram0.text 62902 62902 0 0.0
m5stack (read only) 1072383 1072383 0 0.0
(read/write) 488600 488600 0 0.0
.dram0.bss 77344 77344 0 0.0
.dram0.data 34144 34144 0 0.0
.flash.rodata 245116 245116 0 0.0
.flash.text 1066999 1066999 0 0.0
.iram0.text 123267 123267 0 0.0
k32w light k32w061+release (read/write) 659704 659704 0 0.0
.bss 71276 71276 0 0.0
.data 1992 1992 0 0.0
.text 580636 580636 0 0.0
lock k32w061+release (read/write) 685960 685960 0 0.0
.bss 71748 71748 0 0.0
.data 2004 2004 0 0.0
.text 606408 606408 0 0.0
linux all-clusters-app debug (read only) 2954457 2954457 0 0.0
(read/write) 178136 178136 0 0.0
.bss 84960 84960 0 0.0
.data 2048 2048 0 0.0
.data.rel.ro 84936 84936 0 0.0
.dynamic 608 608 0 0.0
.got 4536 4536 0 0.0
.init 27 27 0 0.0
.init_array 1040 1040 0 0.0
.rodata 263037 263037 0 0.0
.text 2514354 2514354 0 0.0
all-clusters-minimal-app debug (read only) 2808553 2808553 0 0.0
(read/write) 170072 170072 0 0.0
.bss 84288 84288 0 0.0
.data 2048 2048 0 0.0
.data.rel.ro 77576 77576 0 0.0
.dynamic 608 608 0 0.0
.got 4488 4488 0 0.0
.init 27 27 0 0.0
.init_array 1040 1040 0 0.0
.rodata 264701 264701 0 0.0
.text 2368962 2368962 0 0.0
bridge-app debug+rpc (read only) 2310961 2310961 0 0.0
(read/write) 148896 148896 0 0.0
.bss 72352 72352 0 0.0
.data 3824 3824 0 0.0
.data.rel.ro 66952 66952 0 0.0
.dynamic 608 608 0 0.0
.got 4392 4392 0 0.0
.init 27 27 0 0.0
.init_array 728 728 0 0.0
.rodata 197632 197632 0 0.0
.text 1951714 1951714 0 0.0
chip-tool debug (read only) 10172005 10172533 528 0.0
(read/write) 610824 610824 0 0.0
.bss 24384 24384 0 0.0
.data 1088 1088 0 0.0
.data.rel.ro 579056 579056 0 0.0
.dynamic 624 624 0 0.0
.got 5016 5016 0 0.0
.init 27 27 0 0.0
.init_array 640 640 0 0.0
.rodata 509157 509157 0 0.0
.text 8254117 8254645 528 0.0
chip-tool-no-interactive-ipv6only arm64 (read only) 9904116 9904580 464 0.0
(read/write) 675649 675649 0 0.0
.bss 42625 42625 0 0.0
.data 1152 1152 0 0.0
.data.rel.ro 614616 614616 0 0.0
.dynamic 528 528 0 0.0
.got 13440 13440 0 0.0
.init 24 24 0 0.0
.init_array 192 192 0 0.0
.rodata 472164 472180 16 0.0
.text 7892372 7892820 448 0.0
lighting-app debug+rpc (read only) 2548473 2548473 0 0.0
(read/write) 152952 152952 0 0.0
.bss 72896 72896 0 0.0
.data 2096 2096 0 0.0
.data.rel.ro 72088 72088 0 0.0
.dynamic 608 608 0 0.0
.got 4424 4424 0 0.0
.init 27 27 0 0.0
.init_array 816 816 0 0.0
.rodata 213416 213416 0 0.0
.text 2164850 2164850 0 0.0
lock-app debug (read only) 2509705 2509705 0 0.0
(read/write) 147896 147896 0 0.0
.bss 71264 71264 0 0.0
.data 1712 1712 0 0.0
.data.rel.ro 69064 69064 0 0.0
.dynamic 608 608 0 0.0
.got 4424 4424 0 0.0
.init 27 27 0 0.0
.init_array 784 784 0 0.0
.rodata 228104 228104 0 0.0
.text 2116578 2116578 0 0.0
ota-provider-app debug (read only) 2318825 2318825 0 0.0
(read/write) 141736 141736 0 0.0
.bss 70944 70944 0 0.0
.data 1944 1944 0 0.0
.data.rel.ro 63064 63064 0 0.0
.dynamic 608 608 0 0.0
.got 4488 4488 0 0.0
.init 27 27 0 0.0
.init_array 672 672 0 0.0
.rodata 203256 203256 0 0.0
.text 1952914 1952914 0 0.0
ota-requestor-app debug (read only) 2436073 2436073 0 0.0
(read/write) 148608 148608 0 0.0
.bss 73280 73280 0 0.0
.data 2232 2232 0 0.0
.data.rel.ro 67240 67240 0 0.0
.dynamic 608 608 0 0.0
.got 4480 4480 0 0.0
.init 27 27 0 0.0
.init_array 728 728 0 0.0
.rodata 207264 207264 0 0.0
.text 2057778 2057778 0 0.0
shell debug (read only) 2544233 2544233 0 0.0
(read/write) 164488 164488 0 0.0
.bss 80904 80904 0 0.0
.data 1264 1264 0 0.0
.data.rel.ro 76672 76672 0 0.0
.dynamic 608 608 0 0.0
.got 4096 4096 0 0.0
.init 27 27 0 0.0
.init_array 920 920 0 0.0
.rodata 227250 227250 0 0.0
.text 2160690 2160690 0 0.0
thermostat-no-ble arm64 (read only) 2591212 2591212 0 0.0
(read/write) 181761 181761 0 0.0
.bss 88705 88705 0 0.0
.data 1704 1704 0 0.0
.data.rel.ro 83256 83256 0 0.0
.dynamic 528 528 0 0.0
.got 5072 5072 0 0.0
.init 24 24 0 0.0
.init_array 400 400 0 0.0
.rodata 165036 165036 0 0.0
.text 2186448 2186448 0 0.0
tv-app debug (read only) 3099041 3099041 0 0.0
(read/write) 279240 279240 0 0.0
.bss 188648 188648 0 0.0
.data 4848 4848 0 0.0
.data.rel.ro 79312 79312 0 0.0
.dynamic 608 608 0 0.0
.got 4848 4848 0 0.0
.init 27 27 0 0.0
.init_array 952 952 0 0.0
.rodata 248992 248992 0 0.0
.text 2662290 2662290 0 0.0
tv-casting-app debug (read only) 5555681 5555681 0 0.0
(read/write) 185136 185136 0 0.0
.bss 73704 73704 0 0.0
.data 2416 2416 0 0.0
.data.rel.ro 102776 102776 0 0.0
.dynamic 608 608 0 0.0
.got 4720 4720 0 0.0
.init 27 27 0 0.0
.init_array 864 864 0 0.0
.rodata 343177 343177 0 0.0
.text 4935698 4935698 0 0.0
mbed lock-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2446672 2446672 0 0.0
.bss 215716 215716 0 0.0
.data 5872 5872 0 0.0
.text 1409316 1409316 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 (read/write) 1174579 1174579 0 0.0
bss 144650 144650 0 0.0
rodata 141284 141284 0 0.0
text 809712 809712 0 0.0
all-clusters-minimal-app nrf52840dk_nrf52840 (read/write) 1155447 1155447 0 0.0
bss 143886 143886 0 0.0
rodata 133216 133216 0 0.0
text 799416 799416 0 0.0
p6 all-clusters-app default (read/write) 2562600 2562600 0 0.0
.bss 150896 150896 0 0.0
.data 2776 2776 0 0.0
.text 1520864 1520864 0 0.0
all-clusters-minimal-app default (read/write) 2508448 2508448 0 0.0
.bss 150176 150176 0 0.0
.data 2776 2776 0 0.0
.text 1466712 1466712 0 0.0
light-app default (read/write) 2438600 2438600 0 0.0
.bss 142232 142232 0 0.0
.data 2592 2592 0 0.0
.text 1396864 1396864 0 0.0
lock-app default (read/write) 2464872 2464872 0 0.0
.bss 142080 142080 0 0.0
.data 2600 2600 0 0.0
.text 1423136 1423136 0 0.0
telink light-switch-app tlsr9518adk80d (read/write) 797712 797720 8 0.0
bss 72344 72344 0 0.0
noinit 40416 40416 0 0.0
text 564552 564554 2 0.0
lighting-app tlsr9518adk80d (read/write) 817788 817788 0 0.0
bss 73164 73164 0 0.0
noinit 40416 40416 0 0.0
text 581098 581098 0 0.0

@andy31415 andy31415 changed the title Addressing issue 19869 Use 5% tolerance when checking CurrentLevel attribute in TC-LVI Jun 28, 2022
@andy31415
Copy link
Contributor

@manjunath-grl for future issues please set a descriptive text of changes for PR requests. "fixing issue xyz" is generally not easy to parse as one would have to find the issue to even understand what is going on.

@manjunath-grl manjunath-grl changed the title Use 5% tolerance when checking CurrentLevel attribute in TC-LVI Use 5% tolerance when checking CurrentLevel attribute in TC-LVL-6.1 Jun 28, 2022
woody-apple
woody-apple approved these changes Jun 28, 2022
View reviewed changes
Copy link
Contributor

@woody-apple woody-apple left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fast tracking test updates

@woody-apple woody-apple merged commit b394bdf into project-chip:master Jun 28, 2022
@manjunath-grl manjunath-grl deleted the Addressing_Issue_19869 branch July 5, 2022 05:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@anush-apple anush-apple Awaiting requested review from anush-apple

@arkq arkq Awaiting requested review from arkq

@Byungjoo-Lee Byungjoo-Lee Awaiting requested review from Byungjoo-Lee

@bzbarsky-apple bzbarsky-apple Awaiting requested review from bzbarsky-apple

@carol-apple carol-apple Awaiting requested review from carol-apple

@chrisdecenzo chrisdecenzo Awaiting requested review from chrisdecenzo

@chshu chshu Awaiting requested review from chshu

@chulspro chulspro Awaiting requested review from chulspro

@Damian-Nordic Damian-Nordic Awaiting requested review from Damian-Nordic

@dhrishi dhrishi Awaiting requested review from dhrishi

@electrocucaracha electrocucaracha Awaiting requested review from electrocucaracha

@emargolis emargolis Awaiting requested review from emargolis

@erjiaqing erjiaqing Awaiting requested review from erjiaqing

@franck-apple franck-apple Awaiting requested review from franck-apple

@gjc13 gjc13 Awaiting requested review from gjc13

@harsha-rajendran harsha-rajendran Awaiting requested review from harsha-rajendran

@hawk248 hawk248 Awaiting requested review from hawk248

@isiu-apple isiu-apple Awaiting requested review from isiu-apple

@jelderton jelderton Awaiting requested review from jelderton

@jepenven-silabs jepenven-silabs Awaiting requested review from jepenven-silabs

@jmartinez-silabs jmartinez-silabs Awaiting requested review from jmartinez-silabs

@jtung-apple jtung-apple Awaiting requested review from jtung-apple

@kghost kghost Awaiting requested review from kghost

@kpschoedel kpschoedel Awaiting requested review from kpschoedel

@lazarkov lazarkov Awaiting requested review from lazarkov

@LuDuda LuDuda Awaiting requested review from LuDuda

@mrjerryjohns mrjerryjohns Awaiting requested review from mrjerryjohns

@msandstedt msandstedt Awaiting requested review from msandstedt

@rgoliver rgoliver Awaiting requested review from rgoliver

@saurabhst saurabhst Awaiting requested review from saurabhst

@selissia selissia Awaiting requested review from selissia

@tecimovic tecimovic Awaiting requested review from tecimovic

@turon turon Awaiting requested review from turon

@vijs vijs Awaiting requested review from vijs

@vivien-apple vivien-apple Awaiting requested review from vivien-apple

@wbschiller wbschiller Awaiting requested review from wbschiller

@andy31415 andy31415 Awaiting requested review from andy31415

@xylophone21 xylophone21 Awaiting requested review from xylophone21

@yufengwangca yufengwangca Awaiting requested review from yufengwangca

@tcarmelveilleux tcarmelveilleux Awaiting requested review from tcarmelveilleux

@woody-apple woody-apple Awaiting requested review from woody-apple

Assignees
No one assigned
Labels
app fast track review - pending
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Test_TC_LVL_6_1.yaml has disabled test step
4 participants
@manjunath-grl @andy31415 @tcarmelveilleux @woody-apple