west build for zephyr/samples/net/sockets/echo_server/ on qemu_xtensa target outputs elf with panic

[antfobe]

Describe the bug
I am trying to get zephyr network examples to run under qemu_xtensa target - west outputs the elfs, however when trying to run them on qemu with west build -t run they generate the following panic:
`-- west build: running target run
[0/1] To exit from QEMU enter: 'CTRL+a, x'[QEMU] CPU: sample_controller

[00:00:00.010,000] os: ** FATAL EXCEPTION

[00:00:00.010,000] os: ** CPU 0 EXCCAUSE 28 (load prohibited)

[00:00:00.010,000] os: ** PC 0x6000986c VADDR 0x00000004

[00:00:00.010,000] os: ** PS 0x00060220

[00:00:00.010,000] os: ** (INTLEVEL:0 EXCM: 0 UM:1 RING:0 WOE:1 OWB:2 CALLINC:2)

[00:00:00.010,000] os: ** A0 0xa0019da3 SP 0x6001eff0 A2 0x0000001d A3 0x6000040c

[00:00:00.010,000] os: ** A4 0x60027714 A5 0x00000400 A6 0x6001e820 A7 0x00000001

[00:00:00.010,000] os: ** A8 0x600281b4 A9 0x6001efe0 A10 0x00000000 A11 0x00000000

[00:00:00.010,000] os: ** A12 0x00060420 A13 0xffffffff A14 0x00000001 A15 0x00000000

[00:00:00.010,000] os: ** SAR 0x00000000

[00:00:00.010,000] os: >>> ZEPHYR FATAL ERROR 0: CPU exception

[00:00:00.010,000] os: Current thread: 0x60028bf8 (unknown)

[00:00:00.010,000] os: Halting system
`

To Reproduce
Steps to reproduce the behavior:

1. Have zephyr-sdk and esp-idf installed;
2. build any network example with qemu_xtensa target
3. attempt to run with qemu
4. See panic

Expected behavior
It would be expected that the qemu_xtensa elf would run exactly like the qemu_x86 one for these examples.

Impact
This bug is a showstopper for my efforts.

Screenshots or console output

Environment (please complete the following information):

• OS: Linux (uname -a : Linux std4rch 5.3.7-arch1-2-ARCH #1 SMP PREEMPT @1572002934 x86_64 GNU/Linux)
• Toolchain: Zephyr v0.10.3; SDK, ESP-IDF v4.1-dev-592-g758db1e00-dirty
• Commit SHA: 5881f11

Additional context
Tested hello_world, synchronization and philosophers and they work fine;
My cmds:
(./tools/net-tools/loop-socat.sh &) && sudo ./tools/net-tools/loop-slip-tap.sh
ESP_IDF_PATH=/opt/esp-idf/ ESPRESSIF_TOOLCHAIN_PATH=/opt/zephyr-os/crosstool-NG/builds/xtensa-esp32-elf/xtensa-esp32-elf/ ZEPHYR_SDK_INSTALL_DIR=/opt/zephyr-os/zephyr-sdk/ ZEPHYR_TOOLCHAIN_VARIANT=zephyr west build -b qemu_xtensa zephyr/samples/net/sockets/echo_server/ -- -DCONF_FILE="prj.conf"

My prj.conf:
prj.conf.txt
CMake logs:
CMakeOutput.log
CMakeError.log

[antfobe]

Tried adding the QEMU settings at https://github.com/zephyrproject-rtos/net-tools to my prj.conf:
prj.conf.txt

Still getting the panics. Can't really dig into these sdk problems hinted at the CMake error log:
CMakeError.log

Will attempt to build from scratch with sdk-ng

[antfobe]

Welp got an error related to the settings CONFIG_NET_LOG=y and CONFIGT_LOG=y

/opt/zephyr-os/zproj/zephyr/subsys/logging/log_backend_xtensa_sim.c:31:2: error: invalid hard register usage between output operands 31 | __asm__ __volatile__ ( | ^~~~~~~

which probably is responsible for displaying qemu output to stdout - so still same panic.
CMakeError.log

• I used ./go.sh xtensa_sample_controller for sdk-ng and manually downloaded isl from waybackmachine

[antfobe]

@jukkar can I get any update on this?

[jukkar]

Sorry for late reply, just returned from vacation.

This could be stack size issue, can you try to increase the size of the networking thread stacks, the relevant options are:
CONFIG_NET_TX_STACK_SIZE (default value is 1200)
CONFIG_NET_RX_STACK_SIZE (default value is 1500)

[antfobe]

Increased them both to 2048, still getting the panic:

Isn't it related to the network stack? I was getting similar panics in FreeRTOS, until I fixed the esp-idf calls (esp_netif_init or something), the espressif guys changed them in the new release.

[jukkar]

Isn't it related to the network stack? I was getting similar panics in FreeRTOS, until I fixed the esp-idf calls (esp_netif_init or something), the espressif guys changed them in the new release.

So what you are saying above means that the issue is not in network stack but somewhere else.

[antfobe]

So what you are saying above means that the issue is not in network stack but somewhere else.

Yes, pretty much - how do you pass the network interfaces (like e1000, openeth) to zephyr?

[jukkar]

Yes, pretty much - how do you pass the network interfaces (like e1000, openeth) to zephyr?

First one needs to create zeth interface in host

$ cd $ZEPHYR_BASE/../net-tools
$ ./net-setup.sh

Then compile zephyr to use the e1000 driver. For this, there is a overlay-e1000.conf file in echo-server sample.

$ cd samples/net/sockets/echo_server
$ cmake -Bbuild -DBOARD=qemu_x86 -DOVERLAY_CONFIG=overlay-e1000.conf .
$ make -C build run

[jukkar]

Unassigning myself as this is not networking issue.

[antfobe]

Unassigning myself as this is not networking issue.

As pointed out in the issues' name, its an xtensa/qemu_xtensa bug - the build should work for other targets, not only qemu_x86.

Since hello_world and philosophers samples work on qemu_xtensa and I was not able to make any network samples work, I do believe its related to networking.

If you were able to get any of the network samples to work on the qemu_xtensa target and the esp32, please post the commands and modifications needed, since the wiki does not yet contain them.

[antfobe]

Well at least the error changed with -DOVERLAY_CONFIG=overlay-qemu_802154.conf:

[antfobe]

@jukkar, could you assign this issue to someone that is related to xtensa development in zephyr? - or if you were able to make any network example work, post the steps to reproduce your working example?

[jukkar]

$ grep xtensa CODEOWNERS 
/soc/xtensa/intel_s1000/                  @sathishkuttan @dcpleung
/arch/xtensa/                             @andrewboie @dcpleung @andyross
/soc/xtensa/                              @andrewboie @dcpleung @andyross
/boards/xtensa/                           @nashif @dcpleung
/boards/xtensa/intel_s1000_crb/           @sathishkuttan @dcpleung
/boards/xtensa/odroid_go/                 @ydamigos
/dts/xtensa/xtensa.dtsi                   @ydamigos
/include/arch/xtensa/                     @andrewboie

Lets pick dcpleung from the list.

[dcpleung]

The sample is not whitelist-ed to run on qemu_xtensa as indicated in sample.yaml. This is probably due to lack of UART driver for qemu_xtensa. Can you confirm that the fault instruction address (PC, in your example 0x600092d0) in zephyr/zephyr.lst points to api->poll_in(dev, p_char) in z_impl_uart_pool_in()?

[antfobe]

Yes, I think this confirms it:

So is the UART driver implementation planed for future releases?

[dcpleung]

According to http://wiki.osll.ru/doku.php/etc:users:jcmvbkbc:qemu-target-xtensa, the sim platform does not seem to have serial support (contrary to other platforms having 16550s).

Closing this as this is not a problem of Zephyr but a lack of feature in QEMU.

[antfobe]

Reading the page above, the lx200 machine should support UART, by modifying ninja's call to qemu in build/build.ninja to:
/opt/zephyr-os/zephyr-sdk/sysroots/x86_64-pokysdk-linux/usr/bin/qemu-system-xtensa -machine lx200 -semihosting -nographic -cpu sample_controller -net none -pidfile qemu.pid -serial mon:stdio -kernel /opt/zephyr-os/zproj/build/zephyr/zephyr.elf
The build still panicked:

[dcpleung]

There is still no driver in Zephyr, and so nothing for the shell to attach. This results in the shell calling UART function with a NULL device pointer.

[antfobe]

I see - well, at least I managed to get some networking with espressif's qemu
Maybe zephyr will have this driver in the future, thanks anyway.

[dcpleung]

If you can figure out the MMIO address, you can try enabling the existing 16550 driver and see if it works.

[jcmvbkbc]

UART base address is 0xfd050020 in lx60/lx200/ml605/kc705 machines.

[antfobe]

Have no idea what I'm doing, trying to follow this example on qemu. If I'm able to make it work i'll post my results here.