Sending data from slave CPU to master CPU, without using the endpoint callback function

[alioumpas]

Hello,
I am working on a OpenAMP project based on the echo-demo.
On the remote (slave) CPU side, I create two endpoints, both with rpdev option "VIRTIO_DEV_SLAVE".
Each endpoint is bonded with a different callback function.

On the local (master) CPU side, the master C program loads the firmware on the remote CPU, resulting in the creation of the two rpmsg devices /sys/bus/rpmsg/devices/virtio0.rpmsg-channel.-1.0 and /sys/bus/rpmsg/devices/virtio0.rpmsg-channel.-1.1 on the Linux side.

Then I can send custom payload via the 2 Endpoints (using the write function to the rpmsg character device) and get the echoed data back (using the read function), since the transmitted data from the master CPU trigger the Callback functions on the remote CPU side.

The question is how I could send data from the remote CPU to the master CPU, independently of the callback function at the remote CPU.
I guess I have to use the "rpmsg_send(ept1, data, len)" function somewhere outside the call back function, e.g., before the "platform_poll_once(priv)" function?

What I miss completely is how the master CPU will be notified that some data have been sent from the remote CPU and how it will read it.

Thank you for your time and help.
Kind regards,
Athanasios

PS: the board I am using is the Zynq Ultrascale+.

[arnopo]

Hello,
No problem to send a message out of the rpmsg endpoint callback. The only constraint linked to a current limitation of the rpmsg is that the Linux has to

The question is how I could send data from the remote CPU to the master CPU, independently of the callback function at the remote CPU. I guess I have to use the "rpmsg_send(ept1, data, len)" function somewhere outside the call back function, e.g., before the "platform_poll_once(priv)" function?

I suppose that you are facing RPMsg limitation.
No synchronization point is defined by the RPMsg after which both sides can communicate reliably with each other. In the current protocol, at startup, the master sends notification to remote to let it know that it can receive name service announcement. However, master does not consider the fact that if the remote is ready to handle notification at this point in time.

To sum-up the master(host) has to send a first message to the remote endpoint. Then you can send messages from the remote endpoint out of the callback.

What I miss completely is how the master CPU will be notified that some data have been sent from the remote CPU and how it will read it.

Thank to the endpoint callback ( similar to the remote).

Thank you for your time and help. Kind regards, Athanasios

PS: the board I am using is the Zynq Ultrascale+.

[alioumpas]

Thank you very much for the clarifications.
So, based on your answer I understand that:

1. The master(host) must send the first message to "initiate" the channel. After that, the remote (slave) can send messages at any random time, for as long as needed.

2. The master (host) will be notified that the remote has sent some data, thanks to the callback function. Here, I would need some further help, please:
   In the openamp "echo" example, the master creates the rpmsg channel and binds it to a character device**. Then, it uses the write(device, payload, size)) function to write some data to that character device and the read(device, payload, size) function to read the data that (potentially) the remote has sent.

Based on your answer, I understand that we have to use a different implementation at the master, which will create the character device but it will also use a callback function, that we have to bind with the created endpoint? This would mean that we have to use the "rpmsg_create_ept" function at the master, which accepts the callback function as an argument? Or is there a different way to create this callback function at the master?
In short, how can we use a callback function at the master (host) side?

Thanks!

** steps:

• load the rpmsg char driver (modprobe rpmsg_char)
• rpmsg dev overrides path
• bind the rpmsg device to rpmsg char driver
• create endpoint eptinfo from rpmsg char driver using the "rpmsg_endpoint_info" from rpmsg.h
• create endpoint ioctl interface: ioctl(charfd, RPMSG_CREATE_EPT_IOCTL, eptinfo);
• create and open the character device: open(ept_dev_path, O_RDWR | O_NONBLOCK)

[arnopo]

Thank you very much for the clarifications. So, based on your answer I understand that:

1. The master(host) must send the first message to "initiate" the channel. After that, the remote (slave) can send messages at any random time, for as long as needed. But again the remote cannot be aware if the master is ready to handle the data.
2. The master (host) will be notified that the remote has sent some data, thanks to the callback function. Here, I would need some further help, please:
   In the openamp "echo" example, the master creates the rpmsg channel and binds it to a character device**. Then, it uses the write(device, payload, size)) function to write some data to that character device and the read(device, payload, size) function to read the data that (potentially) the remote has sent.

2 way to do this:

• The remote create the channel, and the host has to send a first meaage
• The host creates the channel and the remote send its messages

In both case the side which initiate the channel has to wait a first message from the other side.

Based on your answer, I understand that we have to use a different implementation at the master, which will not create the character device but it will use a callback function, that we have to bind with the created endpoint? This would mean that we have to use the "rpmsg_create_ept" function at the master, which accepts the callback function as an argument? Or is there a different way to create this callback function at the master?

If I well remember there is a specific Xilinx downstream implementation of the rpmsg_char. So difficult to answer...

if you use a recent Linux Kernel you can use the RPMSG_CREATE_DEV_IOCTL that creates the /dev/rpmsg and send a NS announcement to the remote to inform it of the creation of the channel.
In this case you will probably also need this patch on top of:
https://patchwork.kernel.org/project/linux-remoteproc/patch/20220316153001.662422-1-arnaud.pouliquen@foss.st.com/

Thanks!

** steps:

• load the rpmsg char driver (modprobe rpmsg_char)
• rpmsg dev overrides path
• bind the rpmsg device to rpmsg char driver
• create endpoint eptinfo from rpmsg char driver using the "rpmsg_endpoint_info" from rpmsg.h
• create endpoint ioctl interface: ioctl(charfd, RPMSG_CREATE_EPT_IOCTL, eptinfo);
• create and open the character device: open(ept_dev_path, O_RDWR | O_NONBLOCK)

[alioumpas]

Reading the rpmsg design document, I see that we can use the very same RPMsg User APIs (https://github.com/OpenAMP/open-amp/blob/main/docs/rpmsg-design.md),
e.g., "rpmsg_create_ept" at the master node application, in the same way we use this API at the remote node application.
So I guess it is possible to rewrite the echo master application using this API, right?

Furthermore, I understand that the specific master application (https://github.com/OpenAMP/meta-openamp/blob/master/recipes-openamp/rpmsg-examples/rpmsg-echo-test/echo_test.c)
chooses as an implementation option to:

• create the rpmsg char device
• create the endpoint using the ioctl(rpmsg character device, RPMSG_CREATE_EPT_IOCTL, endpoint info)
  The endpoint info structure includes only src, dst and service name, but no callback function.

Would it be possible to just modify the
struct rpmsg_endpoint_info { char name[32]; __u32 src; __u32 dst; };

to include also a callback function?

Thanks!

[Minerro886]

In my understanding, the host (Linux) does not need to call the callback function directly, because the data can be read and operated through the read function (unlike the slave, the data can only be obtained through the callback function)