serial: uart_native_pty: IRQ support

[tpambor]

Add support for the interrupt-driven API to uart_native_pty.

Interrupts are emulated using a polling thread.

[aescolar]

Thanks @tpambor :)
A few comments below

[aescolar]

Thanks @tpambor

[tpambor]

Rebased on main to fix merge conflict in boards/native/native_sim/doc/index.rst.

[aescolar]

@tpambor something is not working as it should..
build ../samples/subsys/shell/shell_module/
and run it as:
echo -e "help\n" | zephyr/zephyr.exe -uart_stdinout | head -n 80
See that it gets stuck in an infinite loop in the shell
(note in main that works just fine, but with this PR the interrupt driven mode is enabled by default in that sample)

[aescolar]

@tpambor I can see that in this case I just put above, feof() never claims the input is finished, while select keeps saying stdin is ready and read keeps returning 0 (eof).
Meaning for stdin the logic that assumes that if select != 0 there is data is not holding, so first it appears there is data, and when the irq callback tries to read data there is no data. Leading to the infinite loop.

[tpambor]

@tpambor I can see that in this case I just put above, feof() never claims the input is finished, while select keeps saying stdin is ready and read keeps returning 0 (eof). Meaning for stdin the logic that assumes that if select != 0 there is data is not holding, so first it appears there is data, and when the irq callback tries to read data there is no data. Leading to the infinite loop.

@aescolar I fixed the end of file detection. I think feof() only works when characters a read using fgetc() etc. and not when using read. Your example no longer loops infinetly, but I still don't get output. I will investigate further.

[aescolar]

I think feof() only works when characters a read using fgetc()

I think you are right.

, but I still don't get output. I will investigate further.

it would seem the shell drops the initial input(?)
(sleep 0.01; echo -e "help\n") | zephyr/zephyr.exe -uart_stdinout | head -n 80
would seem to work

[tpambor]

I think feof() only works when characters a read using fgetc()

I think you are right.

, but I still don't get output. I will investigate further.

it would seem the shell drops the initial input(?) (sleep 0.01; echo -e "help\n") | zephyr/zephyr.exe -uart_stdinout | head -n 80 would seem to work

At least not in the uart driver. I added some debug prints and get identical following output with and without sleep. With the sleep I also get the help output.

np_uart_fifo_read(64) -> rc: 6 data: [104 101 108 112 10 10]
np_uart_fifo_read(58) -> rc: -1 data: []

Data from the first read is flushed on start of shell, see

zephyr/subsys/shell/shell.c

Line 1444 in 0e3cd5b

z_shell_backend_rx_buffer_flush(sh);

. So I would not consider this directly a problem of the uart driver.

[tpambor]

With polling mode, this currently works because data is polled every 10ms using a timer. The first data becomes available only after the timer elapses once. In contrast, with interrupt mode, data is received immediately and flushed (due to being data received before activating the shell).

To address this, I tested two solutions, both of which restore expected behavior for:

echo -e "help\n" | zephyr/zephyr.exe -uart_stdinout | head -n 80

1. Add a small startup delay to the irq thread (for stdin).

static void np_uart_irq_thread(void *arg1, void *arg2, void *arg3)
{
...
	if (data->on_stdinout) {
		k_sleep(K_MSEC(1));
	}

	while (1) {
...

2. Default to polling for the shell backend if native-pty is activated.

config SHELL_BACKEND_SERIAL_INTERRUPT_DRIVEN
	bool "Interrupt driven"
	default y if !UART_NATIVE_PTY
	depends on SERIAL_SUPPORT_INTERRUPT

[aescolar]

I think overall this is ok.
Let's wait a couple of hours to see what @jakub-uC says about
https://discord.com/channels/720317445772017664/937768560137220155/1404782521719328798
As I think we do not need to do anything here to work around that shell behavior

[tpambor]

@aescolar I detailed a bit more the documentation.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[henrikbrixandersen]

This somehow breaks interaction between pytest and the shell.

This is a snippet from running tests/drivers/can/host/ here. Some characters are left out, some are repeated when sending commands from pytest to the Zephyr native_sim shell.

DEBUG: #: uart:~$  can
DEBUG: #: can - CAN controller commands
DEBUG: #: Subcommands:
DEBUG: #:   start     : Start CAN controller
DEBUG: #: Usage: can start <device>
DEBUG: #:   stop      : Stop CAN controller
DEBUG: #: Usage: can stop <device>
DEBUG: #:   show      : Show CAN controller information
DEBUG: #: Usage: can show <device>
DEBUG: #:   bitrate   : Set CAN controller bitrate (sample point and SJW optional)
ERROR: Read from device timeout occurred
../../zephyr/tests/drivers/can/host/pytest/test_can.py::TestCanRxTx::test_host_to_dut[std_id_dlc_0] ERROR
../../zephyr/tests/drivers/can/host/pytest/test_can.py::TestCanRxTx::test_host_to_dut[std_id_dlc_4]
-------------------------------- live log setup --------------------------------
DEBUG: #: Usage: can bitrate <device> <bitrate> [sample point] [sjw]
DEBUG: #:   dbitrate  : Set CAN controller data phase bitrate (sample point and SJW
DEBUG: #: optional)
DEBUG: #: Usage: can dbitrate <device> <data phase bitrate> [sample point]
DEBUG: #: [sjw]
DEBUG: #:   timing    : Set CAN controller timing
DEBUG: #: Usage: can timing <device> <sjw> <prop_seg> <phase_seg1>
DEBUG: #: <phase_seg2> <prescaler>
DEBUG: #:   dtiming   : Set CAN controller data phase timing
ERROR: Read from device timeout occurred
ERROR
../../zephyr/tests/drivers/can/host/pytest/test_can.py::TestCanRxTx::test_host_to_dut[ext_id_dlc_8]
-------------------------------- live log setup --------------------------------
DEBUG: #: Usage: can dtiming <device> <sjw> <prop_seg> <phase_seg1>
DEBUG: #: <phase_seg2> <prescaler>
DEBUG: #:   mode      : Set CAN controller mode
DEBUG: #: Usage: can mode <device> <mode> [mode] [mode] [...]
DEBUG: #:   send      : Enqueue a CAN frame for sending
DEBUG: #: Usage: can send <device> [-e] [-r] [-f] [-b] <CAN ID> [data] [...]
DEBUG: #: -e  use extended (29-bit) CAN ID
DEBUG: #: -r  send Remote Transmission Request (RTR) frame
DEBUG: #: -f  use CAN FD frame format
ERROR: Read from device timeout occurred
ERROR
../../zephyr/tests/drivers/can/host/pytest/test_can.py::TestCanRxTx::test_host_to_dut[std_id_fdf_dlc_9]
-------------------------------- live log setup --------------------------------
DEBUG: #: -b  use CAN FD Bit Rate Switching (BRS)
DEBUG: #:   filter    : CAN rx filter commands
DEBUG: #: Usage: can filter <add|remove> <device> ...
DEBUG: #: uart:~$ 
DEBUG: #: uart:~$ cacan show can
DEBUG: #: cacan: command not found

[tpambor]

This somehow breaks interaction between pytest and the shell.

This is a snippet from running tests/drivers/can/host/ here. Some characters are left out, some are repeated when sending commands from pytest to the Zephyr native_sim shell.

@henrikbrixandersen There was a similar problem with the sensor shell. The problem was that the tests did not wait until the shell was ready, see b2de0dc. I imagine something similar is happening for the can shell.

I tried to do a hotfix #94424

[henrikbrixandersen]

I have opened a PR for reverting this until the problem is fully understood: #94426