-
Notifications
You must be signed in to change notification settings - Fork 238
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Discuss: general case for Async calls into Rust. #1054
Comments
Thanks James. I haven't thought too deeply about this, but some 1/2-baked thoughts:
This seems to imply that this problem lives in the domain of the foreign binding generators? |
I think to a first approximation this analysis is correct; though I'm still not sure what happens in the Javascript or Python cases. Perhaps a good place to start with this would be to gather examples of what we're doing by hand, and what concurrency patterns and practices are common in the each language community; then hopefully concrete the cowpaths. |
As an example of interop, i'd like to mention pyo3-asyncio here as an interesting case, they chose to have 2 event loops running on different threads; a rust one, and the python one. Instead, async semantics are handled by suspending and passing callback messages between the two barriers using the help of some synchronisation primitives. There is an issue open that talks about in-event-loop rust-future awaiting (awestlake87/pyo3-asyncio#59), but this is a design which requires a ton of thought. In general, i think above design (different event loops that synchronise eachother) could work for a majority of usecases.
I'm not sure what Ultimately, the problem exists that an event loop implementation must be aware of the foreign loop to properly work with it, as it assumes its domineering in that aspect, that it has total control over the thread. In my opinion, above pattern (seperate event loops on different threads while they synchronise) could work for a majority of usecases, and could be a good stepping stone for other (more difficult) patterns. |
While I think that's true, it does assume that usecases have flexibility about how to implement event loops. If you consider desktop Firefox or Android apps, it's probably not going to be reasonable to assume that UniFFI-based projects can influence the event loop implementations. |
I'd like to revive this discussion now that we're hoping to add better async/callback support for Desktop JS in order to support Nimbus and viaduct. We've discussed several use cases that we think we need and features that would enable those use cases. However, I've never felt like we've completely mapped out the use cases and wondered if we're missing something. I'm hoping that we can group all uses cases into a table with 4 dimensions:
Based on those dimensions, let's try to actually map out all the possibilities:
Based on that, I believe there are 2 features that we need to add:
I think the Does this analysis make sense? Is there any use cases that aren't captured by the table above? |
Thanks for continuing to push on this Ben! I need more time to think about this because it's quite complicated 😅 . I'm also struggling to match up actual use-cases and what some of the table items mean in practice - eg, "Callback/Foreign/SameThread/Async" means, IIUC:
(That's just one example I'm noting more to highlight that some bits aren't quite clear and could maybe do with some expansion. Thinking about viaduct, we currently use callbacks, but that's only because we don't have async, right? ie, if we had some async support for the "Normal/Foreign/?/Async" case, Viaduct wouldn't use callbacks at all, right? I guess I'm wondering if it's worth fleshing out some realistic real-world use cases along with the table, including their UDL? Similarly for #1252, I'm struggling to draw the line between "this part is hand-written because UniFFI doesn't support it yet" vs "this is what it would look like if UniFFI did support it". I'm sure we can come up with contrived examples for most of the entries, but are they actually realistic? I'm also struggling a little with what the "thread switch" column means in practice - I know what it means, but don't quite understand all the use-cases, nor how it fits with a single-threaded executor. These are just my initial thoughts and might not make sense once I've thought this through a little more - I'll come back to this and will try and do some of my suggestions re use-cases etc. (Another random thought is that it's tricky to add comments/questions to the comment above - there were a few things I didn't fully understand or had minor suggestions for clarity. I wonder if this should be a PR to a document, so that we can add comments/suggestions to individual lines and/or entries in the table etc?) |
Closing as FIXED. |
This came from the discussion on delegates/decorators in #1051. That supports specializing method calls, perhaps with an async dispatch, but does not model every case.
The problem statement is relatively simple: the app would like to call into a Rust library via uniffi. The library performs some kind of computation, and returns the result back to the app (again, via uniffi).
The complexity/things to nail down is all the variations available, and/or we need to support:
All of these variations have different popularity and maturity. It's clear we need to support some variation by library owners and app owners; it's not clear we can pick winners in most cases.
Post your strawfox proposals here, and point any discussions of async uniffi here.
┆Issue is synchronized with this Jira Task
┆Issue Number: UNIFFI-89
The text was updated successfully, but these errors were encountered: