RFC: Queued operations manager

[nordic-krch]

General idea

A generic module for managing access to single resource capable of performing certain asynchronous operation. A management is needed because there are multiple contexts (clients) trying to perform this operation with own data. A client is interested in scheduling asynchronous operation and then get notification when operation is performed. Scheduling may support prioritization.

Use cases

SPI

There are potentially multiple devices on single spi bus (e.g. display, memory, sensor) each attempts to randomly access the bus. Current SPI is synchronous so access management is achieved on API level: user will block (sleep) when device is busy. That is ok, but it limits API access only to the context which can sleep (thread). Thread only access has following consequences:

• transfers cannot be scheduled from interrupts (e.g. k_timer timeout handler) see RFC: asynchronous i2c and spi API #21538
• mutiple transfers from same context requires many context switches (evaulation shown 10-20% CPU load increase)

I2C

Similarly to SPI: multiple devices on single bus.

Others

ADC?

Client

On-off service has already defined an asynchronous client. Currently it is part of the on-off API but it can be extracted and used for all asynchronous access modules like queued operation manager.
Client defines 3 types of access:

• spin wait
• signal
• callback with user_data

Operations

Scheduling

Operation can be scheduled by a client, something like
int qop_mngr_schedule(struct qop_mngr *mngr, struct qop_op *op, struct async_client *cli)
Operation may return error is scheduling fails. If operation returns success it does not mean that operation will be successfully completed. Result of the operation is passed to the client when operation is completed.

Canceling

Operation can be cancelled if it is still in the queue and not yet started. Something like:
int qop_mngr_cancel(struct qop_mngr *mngr, struct qop_op *op, struct async_client *cli)
See include/sys/onoff.h for cancel behavior details.

Others

• onoff service supports case where operations (start, stop, reset) sleeps. Queued operation manager can also support that.

Last but not least - naming

working name queued operations manager (with qop_mngr as abbreviation)

[nordic-krch]

@pabigot @tbursztyka @carlescufi @anangl @alexanderwachter @MaureenHelm @nashif
I've created issues for queued operations manager and transaction manager (#22498)
If you see other requirements please add them in comments and i'll update the issue.
@pabigot can you comment what are the requirements coming from power management. You also mentioned before "mediated service" manager. Can you create issue for it as well as i guess that those 2 do not cover that.

[alexanderwachter]

Maybe timeouts could be useful.
Any ideas about the buffers for queued operations and where the contest lives?
Is the user responsible to place the context in a scope where it is valid until the callback happens or should we manage those things internally. In terms of safety and untrusted threads, maybe the context should be kept internally.

[nordic-krch]

Maybe timeouts could be useful.

I think that this can be achieved by using signal client. With current asynchronous client defined in onoff service it would be done like that:

async_client_init(&cli, &signal);
err = qop_mngr_schedule(&mngr, &cli, &op);
if (err == 0) {
  err = k_poll(&event, 1, K_MSEC(timeout));
  if (err == -EAGAIN) {
    err = qop_mngr_cancel(&mngr, &cli, &op);
    ...
  }
  ...
}

Any ideas about the buffers for queued operations and where the contest lives?

I was thinking that user will provide and keep operation data until operation is completed. As i didn't work with user mode I don't know how it looks there.

[pabigot]

@alexanderwachter @nordic-krch Regarding timeout: The solution in #22853 assumes that only operations that have not been started can be cancelled. I'm unaware of any precedent in Zephyr APIs for externally aborting an operation that is in progress other than k_thread_abort(), which is pretty violent.

Where aborting operations is possible the proposed implementation can be extended without API modification by adding a function to the service table that attempts to abort the in-progress operation, and if so completes the operation with -EABORTED. Services that don't provide that capability would continue to reject attempts to cancel the current operation.

If there's a desire to cancel the operation only if it has not been started, the API would have to change.

But I suspect the motivation for ability to cancel would be better addressed by detecting "stuckness" within the driver and failing at that level, which would provide recovery for all cases. Alternatively, any limit on how long to spend on an operation could be implemented within a specific queued operation service by adding a timeout to the operation description, with a handler that does the abort and completes the operation with an appropriate error.

[teburd]

@nordic-krch has this been fixed with rtio?

[nordic-krch]

yes