up_squared_adsp: Sanitycheck can not run test case on Up_Squared_ADSP board

[KangJianX]

Describe the bug
Sanitycheck can run on up_squared_adsp board.

To Reproduce
Steps to reproduce the behavior:
Run this command:
sanitycheck -W -p up_squared_adsp --device-testing -T samples/hello_world/ --west-flash="/home/ztest/work/zephyrproject/zephyr/boards/xtensa/up_squared_adsp/tools/up_squared_adsp_flash.sh, /home/ztest/work/sof/rimage/keys/otc_private_key.pem" --device-serial-pty="/home/ztest/work/ambient/set_ambient /home/ztest/work/zephyrproject/zephyr/boards/xtensa/up_squared_adsp/tools/mbterm.py" -x=CONFIG_IPM=y -x=CONFIG_CONSOLE=y -x=CONFIG_LOG_PRINTK=n

See sanitycheck will skip sample

Expected behavior
sanitycheck can run normal on adsp board

Impact
All test cases can not be run on up_squared_adsp

Logs and console output
DEBUG - platform filter: ['up_squared_adsp']
DEBUG - arch_filter: None
DEBUG - tag_filter: None
DEBUG - exclude_tag: None
INFO - Building initial testcase list...
DEBUG - up_squared_adsp samples/hello_world/sample.basic.helloworld SKIPPED: Excluded tags per platform (only_tags)
INFO - 1 test configurations selected, 1 configurations discarded due to filters.

Device testing on:

Platform	ID	Serial device
up_squared_adsp

INFO - Adding tasks to the queue...
DEBUG - DONE FEEDING

DEBUG - running footprint_reports
INFO - 0 of 0 tests passed (0.00%), 0 failed, 1 skipped with 0 warnings in 2.32 seconds
INFO - In total 0 test cases were executed on 1 out of total 297 platforms (0.34%)
INFO - 0 tests executed on platforms, 0 tests were only built.

Hardware distribution summary:

Board	ID	Counter
up_squared_adsp		0

Environment (please complete the following information):
OS: Fedora28
Toolchain: Zephyr-sdk-0.11.3
Commit ID: a8a92ce

[KangJianX]

I found (Commit ID: 544a38e) this commit remove up_squared_adsp_flash.sh. So sanitycheck can't run with this command.

[nashif]

does this work with kernel tests? samples right now are being excluded for different reasons.

[KangJianX]

Yes, we can not run kernel tests per sanitycheck, because up_squared_adsp_flash.sh shell script was removed, and sanitycheck run test cases need use --west-flash with this.
I saved a up_squared_adsp_flash.sh by previous commit and pull the latest code, want to run test cases with this shell but it still not work.

[andyross]

I'm just getting to this. Indeed, there's a west flash integration that was never committed to the sof tree and got lost with the rebase. I'm confused about where you got the remaining instructions though. The "mbterm" utility should work fine, though the adsplog.py utility is a little more recent and is what I use. What is the "set_ambient" tool you're using? I'm guessing that's a pty redirector? It's not part of the Zephyr tree, so can you point me to where you got it?

Broadly, it seems like the request here is for a working sanitycheck/west integration for the intel_adsp devices, which we really don't have. It seems like you have a partial solution, can you share it?

[nashif]

Broadly, it seems like the request here is for a working sanitycheck/west integration for the intel_adsp devices, which we really don't have. It seems like you have a partial solution, can you share it?

actually, we had all of this working AFAIK, @finikorg got this working at some point and it was integrated with sanitycheck, looks like we have drop those by mistake?

[KangJianX]

@andyross Yes we can run tests with sanitycheck, and we need west flash by a shell script which is dropped.
By the way, sanitycheck not work on ADSP even we use this script up_squared_adsp_flash.sh for now.

[finikorg]

I'm just getting to this. Indeed, there's a west flash integration that was never committed to the sof tree and got lost with the rebase. I'm confused about where you got the remaining instructions though. The "mbterm" utility should work fine, though the adsplog.py utility is a little more recent and is what I use. What is the "set_ambient" tool you're using? I'm guessing that's a pty redirector? It's not part of the Zephyr tree, so can you point me to where you got it?

pty is working with new sanitycheck option --device-serial-pty

zephyr/scripts/sanitycheck

Lines 606 to 613 in 5803ee1

	serial.add_argument("--device-serial-pty",
	help="""Script for controlling pseudoterminal.
	Sanitycheck believes that it interacts with a terminal
	when it actually interacts with the script.
	E.g "sanitycheck --device-testing
	--device-serial-pty <script>
	""")

set_ambient is here: https://github.com/finikorg/ambient, described also here: #22724 (comment)

adsplog.py should be working too, also it may be needed some tweaking.

Broadly, it seems like the request here is for a working sanitycheck/west integration for the intel_adsp devices, which we really don't have. It seems like you have a partial solution, can you share it?

Everything was merged upstream, see command how to run sanitycheck here: #27804 (comment)

[andyross]

Everything was merged upstream

But never to SOF, so it got clobbered with the rebase. :)

Also the setcap gadget was never merged, nor documented. Bear with me while I figure this out. I'd been working with SOF tooling and simply missed that this new non-SOF stuff got added.

[KangJianX]

Also the setcap gadget was never merged, nor documented. Bear with me while I figure this out. I'd been working with SOF tooling and simply missed that this new non-SOF stuff got added.

@andyross So can you share your way how to west flash on Audio DSP now, maybe it can help me verify some issue.

[finikorg]

Also the setcap gadget was never merged, nor documented. Bear with me while I figure this out. I'd been working with SOF tooling and simply missed that this new non-SOF stuff got added.

setcap is totally different from the sanitycheck, this is permission/capability approach to avoid using sudo, I am sure there are tens of other ways to achieve the same, in a worst case sudo will work, at least once ;)

[jocelyn-li]

@andyross I am interested to learn about the progress on this fix, thanks!

[andyross]

See changes to adsplog.py in #30797 which should resolve the root cause issue with running sanitycheck via the standard SOF trace output mechanism. I'm writing up docs on that right now and will push in that PR along with the west flash integration.

[KangJianX]

@andyross Hi Andy, it still have a problem. Run twister and check handle.log but log not correct, the result of test cases is that I used twister to flash last time. For example, use west to build and flash test cases of thread_api. Then use twister to run test cases of mutex, the result of twister is thread_api log, not mutex.