Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[flashing scripts] Log errors from subprocesses #15593

Merged
merged 1 commit into from
Feb 25, 2022

Conversation

kpschoedel
Copy link
Contributor

Problem

It's difficult to troubleshoot errors frome external commands
run by the flashing scripts, because their output is captured
and not reported.

Change overview

Log command stdout and stderr in case of failure.

Testing

Manual run.

#### Problem

It's difficult to troubleshoot errors frome external commands
run by the flashing scripts, because their output is captured
and not reported.

#### Change overview

Log command stdout and stderr in case of failure.

#### Testing

Manual run.
@github-actions
Copy link

github-actions bot commented Feb 25, 2022

PR #15593: Size comparison from bb6bb37 to 64532f7

Full report (45 builds for cyw30739, efr32, esp32, k32w, linux, mbed, nrfconnect, p6, qpg, telink)
platform target config section bb6bb37 64532f7 change % change
cyw30739 light cyw930739m2evb_01 (read/write) 599614 599614 0 0.0
.app_xip_area 504460 504460 0 0.0
.bss 77844 77844 0 0.0
.data 652 652 0 0.0
.rodata 0 0 0 0.0
.text 0 0 0 0.0
lock cyw930739m2evb_01 (read/write) 557686 557686 0 0.0
.app_xip_area 464068 464068 0 0.0
.bss 76348 76348 0 0.0
.data 616 616 0 0.0
.rodata 0 0 0 0.0
.text 0 0 0 0.0
ota-requestor cyw930739m2evb_01 (read/write) 578210 578210 0 0.0
.app_xip_area 475160 475160 0 0.0
.bss 85452 85452 0 0.0
.data 560 560 0 0.0
.rodata 0 0 0 0.0
.text 112 112 0 0.0
efr32 lighting-app BRD4161A (read only) 916556 916556 0 0.0
(read/write) 128608 128608 0 0.0
.bss 126560 126560 0 0.0
.data 2048 2048 0 0.0
.text 916548 916548 0 0.0
BRD4161A+rpc (read only) 946644 946644 0 0.0
(read/write) 145548 145548 0 0.0
.bss 143336 143336 0 0.0
.data 2212 2212 0 0.0
.text 946636 946636 0 0.0
lock-app BRD4161A+wf200 (read only) 986476 986476 0 0.0
(read/write) 120360 120360 0 0.0
.bss 118436 118436 0 0.0
.data 1924 1924 0 0.0
.text 986468 986468 0 0.0
window-app BRD4161A (read only) 850620 850620 0 0.0
(read/write) 126844 126844 0 0.0
.bss 124928 124928 0 0.0
.data 1916 1916 0 0.0
.text 850612 850612 0 0.0
esp32 all-clusters-app c3devkit (read only) 956550 956550 0 0.0
(read/write) 1393010 1393010 0 0.0
.dram0.bss 62592 62592 0 0.0
.dram0.data 14228 14228 0 0.0
.flash.rodata 197272 197272 0 0.0
.flash.text 956550 956550 0 0.0
.iram0.text 62016 62016 0 0.0
m5stack (read only) 1004691 1004691 0 0.0
(read/write) 458288 458288 0 0.0
.dram0.bss 68096 68096 0 0.0
.dram0.data 34072 34072 0 0.0
.flash.rodata 224624 224624 0 0.0
.flash.text 999307 999307 0 0.0
.iram0.text 122767 122767 0 0.0
k32w light k32w061+release (read/write) 691972 691972 0 0.0
.bss 77480 77480 0 0.0
.data 1924 1924 0 0.0
.text 606768 606768 0 0.0
lock k32w061+release (read/write) 695244 695244 0 0.0
.bss 77688 77688 0 0.0
.data 1964 1964 0 0.0
.text 609792 609792 0 0.0
linux all-clusters-app debug (read only) 2394729 2394729 0 0.0
(read/write) 144696 144696 0 0.0
.bss 59232 59232 0 0.0
.data 1360 1360 0 0.0
.data.rel.ro 78392 78392 0 0.0
.dynamic 592 592 0 0.0
.got 4160 4160 0 0.0
.init 27 27 0 0.0
.init_array 912 912 0 0.0
.rodata 204709 204709 0 0.0
.text 2027250 2027250 0 0.0
bridge-app debug+rpc (read only) 1720805 1720805 0 0.0
(read/write) 92744 92744 0 0.0
.bss 47552 47552 0 0.0
.data 2042 2042 0 0.0
.data.rel.ro 38088 38088 0 0.0
.dynamic 592 592 0 0.0
.got 3912 3912 0 0.0
.init 27 27 0 0.0
.init_array 544 544 0 0.0
.rodata 141492 141492 0 0.0
.text 1463509 1463509 0 0.0
chip-tool debug (read only) 8994565 8994565 0 0.0
(read/write) 298496 298496 0 0.0
.bss 40384 40384 0 0.0
.data 1164 1164 0 0.0
.data.rel.ro 250872 250872 0 0.0
.dynamic 608 608 0 0.0
.got 4784 4784 0 0.0
.init 27 27 0 0.0
.init_array 632 632 0 0.0
.rodata 480533 480533 0 0.0
.text 7905557 7905557 0 0.0
chip-tool-ipv6only arm64 (read only) 8748796 8748796 0 0.0
(read/write) 411617 411617 0 0.0
.bss 58625 58625 0 0.0
.data 1216 1216 0 0.0
.data.rel.ro 296568 296568 0 0.0
.dynamic 560 560 0 0.0
.got 51408 51408 0 0.0
.init 24 24 0 0.0
.init_array 208 208 0 0.0
.rodata 456676 456676 0 0.0
.text 7427668 7427668 0 0.0
door-lock-app debug (read only) 1936273 1936273 0 0.0
(read/write) 119136 119136 0 0.0
.bss 50496 50496 0 0.0
.data 1050 1050 0 0.0
.data.rel.ro 62200 62200 0 0.0
.dynamic 592 592 0 0.0
.got 4096 4096 0 0.0
.init 27 27 0 0.0
.init_array 664 664 0 0.0
.rodata 173532 173532 0 0.0
.text 1614258 1614258 0 0.0
lighting-app debug+rpc (read only) 2071137 2071137 0 0.0
(read/write) 124496 124496 0 0.0
.bss 51424 51424 0 0.0
.data 1472 1472 0 0.0
.data.rel.ro 66144 66144 0 0.0
.dynamic 608 608 0 0.0
.got 4128 4128 0 0.0
.init 27 27 0 0.0
.init_array 712 712 0 0.0
.rodata 166129 166129 0 0.0
.text 1749554 1749554 0 0.0
ota-provider-app debug (read only) 1868393 1868393 0 0.0
(read/write) 114800 114800 0 0.0
.bss 50368 50368 0 0.0
.data 1320 1320 0 0.0
.data.rel.ro 57480 57480 0 0.0
.dynamic 608 608 0 0.0
.got 4352 4352 0 0.0
.init 27 27 0 0.0
.init_array 616 616 0 0.0
.rodata 158739 158739 0 0.0
.text 1559090 1559090 0 0.0
ota-requestor-app debug (read only) 1879385 1879385 0 0.0
(read/write) 116568 116568 0 0.0
.bss 51232 51232 0 0.0
.data 1192 1192 0 0.0
.data.rel.ro 58744 58744 0 0.0
.dynamic 592 592 0 0.0
.got 4152 4152 0 0.0
.init 27 27 0 0.0
.init_array 624 624 0 0.0
.rodata 152452 152452 0 0.0
.text 1578098 1578098 0 0.0
shell debug (read only) 2362961 2362961 0 0.0
(read/write) 147176 147176 0 0.0
.bss 67552 67552 0 0.0
.data 864 864 0 0.0
.data.rel.ro 73096 73096 0 0.0
.dynamic 592 592 0 0.0
.got 4144 4144 0 0.0
.init 27 27 0 0.0
.init_array 896 896 0 0.0
.rodata 204658 204658 0 0.0
.text 2001746 2001746 0 0.0
thermostat-no-ble arm64 (read only) 2162948 2162948 0 0.0
(read/write) 148577 148577 0 0.0
.bss 65265 65265 0 0.0
.data 1056 1056 0 0.0
.data.rel.ro 75080 75080 0 0.0
.dynamic 560 560 0 0.0
.got 4216 4216 0 0.0
.init 24 24 0 0.0
.init_array 352 352 0 0.0
.rodata 132980 132980 0 0.0
.text 1811664 1811664 0 0.0
tv-app debug (read only) 2589489 2589489 0 0.0
(read/write) 148632 148632 0 0.0
.bss 65824 65824 0 0.0
.data 3168 3168 0 0.0
.data.rel.ro 73624 73624 0 0.0
.dynamic 592 592 0 0.0
.got 4528 4528 0 0.0
.init 27 27 0 0.0
.init_array 880 880 0 0.0
.rodata 198653 198653 0 0.0
.text 2217074 2217074 0 0.0
mbed all-clusters-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2449728 2449728 0 0.0
.bss 191016 191016 0 0.0
.data 5512 5512 0 0.0
.text 1412300 1412300 0 0.0
lighting-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2414424 2414424 0 0.0
.bss 188188 188188 0 0.0
.data 5800 5800 0 0.0
.text 1376996 1376996 0 0.0
lock-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2350244 2350244 0 0.0
.bss 187092 187092 0 0.0
.data 5776 5776 0 0.0
.text 1312844 1312844 0 0.0
pigweed-app CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 1155428 1155428 0 0.0
.bss 11952 11952 0 0.0
.data 4512 4512 0 0.0
.text 118812 118812 0 0.0
shell CY8CPROTO_062_4343W+release (read only) 6224 6224 0 0.0
(read/write) 2340240 2340240 0 0.0
.bss 185736 185736 0 0.0
.data 5608 5608 0 0.0
.text 1302812 1302812 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 (read/write) 1038947 1038947 0 0.0
bss 125756 125756 0 0.0
rodata 135924 135924 0 0.0
text 700060 700060 0 0.0
lighting-app nrf52840dk_nrf52840 (read/write) 1002171 1002171 0 0.0
bss 122064 122064 0 0.0
rodata 116164 116164 0 0.0
text 686440 686440 0 0.0
nrf52840dk_nrf52840+rpc (read/write) 968503 968503 0 0.0
bss 119148 119148 0 0.0
rodata 107544 107544 0 0.0
text 663668 663668 0 0.0
nrf52840dongle_nrf52840 (read/write) 1018263 1018263 0 0.0
bss 123112 123112 0 0.0
rodata 115852 115852 0 0.0
text 691148 691148 0 0.0
nrf5340dk_nrf5340_cpuapp (read/write) 904986 904986 0 0.0
bss 119352 119352 0 0.0
rodata 108280 108280 0 0.0
text 596152 596152 0 0.0
lock-app nrf52840dk_nrf52840 (read/write) 931259 931259 0 0.0
bss 120280 120280 0 0.0
rodata 104996 104996 0 0.0
text 628928 628928 0 0.0
nrf5340dk_nrf5340_cpuapp (read/write) 834886 834886 0 0.0
bss 117600 117600 0 0.0
rodata 97028 97028 0 0.0
text 539404 539404 0 0.0
pigweed-app nrf52840dk_nrf52840 (read/write) 501603 501603 0 0.0
bss 52584 52584 0 0.0
rodata 44500 44500 0 0.0
text 342712 342712 0 0.0
pump-app nrf52840dk_nrf52840 (read/write) 929519 929519 0 0.0
bss 120012 120012 0 0.0
rodata 103976 103976 0 0.0
text 628356 628356 0 0.0
pump-controller-app nrf52840dk_nrf52840 (read/write) 925479 925479 0 0.0
bss 120016 120016 0 0.0
rodata 103672 103672 0 0.0
text 624612 624612 0 0.0
shell nrf52840dk_nrf52840 (read/write) 788811 788811 0 0.0
bss 112360 112360 0 0.0
rodata 74836 74836 0 0.0
text 525504 525504 0 0.0
p6 all-clusters-app default (read/write) 2488128 2488128 0 0.0
.bss 118664 118664 0 0.0
.data 2688 2688 0 0.0
.text 1446392 1446392 0 0.0
light-app default (read/write) 2395344 2395344 0 0.0
.bss 112984 112984 0 0.0
.data 2536 2536 0 0.0
.text 1353608 1353608 0 0.0
lock-app default (read/write) 2358920 2358920 0 0.0
.bss 112728 112728 0 0.0
.data 2496 2496 0 0.0
.text 1317184 1317184 0 0.0
qpg lighting-app qpg6105+debug (read only) 601248 601248 0 0.0
(read/write) 146936 146936 0 0.0
.bss 90904 90904 0 0.0
.data 1124 1124 0 0.0
.text 595928 595928 0 0.0
lock-app qpg6105+debug (read only) 567044 567044 0 0.0
(read/write) 146936 146936 0 0.0
.bss 90912 90912 0 0.0
.data 1076 1076 0 0.0
.text 561724 561724 0 0.0
persistent-storage-app qpg6105+debug (read only) 99536 99536 0 0.0
(read/write) 146941 146941 0 0.0
.bss 24001 24001 0 0.0
.data 180 180 0 0.0
.text 94216 94216 0 0.0
telink lighting-app tlsr9518adk80d (read/write) 883158 883158 0 0.0
bss 86336 86336 0 0.0
noinit 37160 37160 0 0.0
text 623996 623996 0 0.0

Comment on lines +228 to +232
self.log(fail_level, '--- stdout ---')
self.log(fail_level, exception.stdout)
self.log(fail_level, '--- stderr ---')
self.log(fail_level, exception.stderr)
self.log(fail_level, '---')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: shouldn't be better to have only one single log entry?

Suggested change
self.log(fail_level, '--- stdout ---')
self.log(fail_level, exception.stdout)
self.log(fail_level, '--- stderr ---')
self.log(fail_level, exception.stderr)
self.log(fail_level, '---')
self.log(fail_level, '--- stdout ---\n'
f'{exception.stdout}\n--- stderr ---\n'
f'{exception.stderr}\n---')

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets follow up in a separate PR if needed. I would like to have this version in to catch the reason of the flakyness. In most cases these logs will not be used.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unresolved after merge. @kpschoedel could you followup in a separate PR or argue that it is not needed?

there would also be the option of 'every stdout/stderr line should have its own log line' so that things align nicely in logs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I'll address this one way or another, when we catch the parttool error or separately.

@andy31415 andy31415 merged commit 660c745 into project-chip:master Feb 25, 2022
@kpschoedel kpschoedel deleted the flashing-log-errors branch June 3, 2024 15:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants