✨ Add alternative APIs that improve typing and tooling support (e.g. autocompletion) #2208
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
✨ Add alternative APIs that improve typing and tooling support (e.g. autocompletion in editors)
Description
I want to propose adding a new API style for three functions (additional to the current API) that would take advantage of new type annotations (PEP 612 - Parameter Specification Variables), to get better tooling and editor support.
In particular, this would allow/support mypy checks, inline editor errors, and ✨ autocompletion ✨ for positional and keyword arguments for:
nursery.start_soon()
)to_thread.run_sync()
)from_thread.run()
)This would also allow mypy and editors to provide type checks, autocompletion, etc. for the return values received when sending tasks to a worker thread and when receiving them from a worker thread:
to_thread.run_sync()
from_thread.run()
Previous Discussion
I proposed this in AnyIO here: agronholm/anyio#403
I was pointed out at the discussion here: #470
That discussion revolves mainly around accepting keyword arguments for functions called, and this is also related to that. But one of the main things I want from this is better tooling and editor support: autocompletion and inline errors... not only supporting kwargs. So I thought it was worth sharing this additional perspective.
Usage
The main difference is that instead of having a function that takes the function to call plus:
...this new API only takes the function to call plus Trio-specific configs (like the task name), and then returns another function that takes the positional and keyword arguments. And when that function is called, it does the actual work. So it's used like this:
And now it also supports keyword arguments, not only positional ones, so it's not necessary to use (and understand) partials to use keyword arguments:
...but the main benefit, from my point of view, is actually better typing/tooling support. Autocompletion and inline type errors in the editor and mypy.
Examples
Nursery
To Thread (asyncify)
From Thread (syncify)
Function names Bikeshedding
I wanted to have short function names that were descriptive enough, that's why the "somethingify".
I thought any more descriptive variant like
generate_async_function_that_calls_run_sync
would be too long. 😅My rationale is that if a user wants to call a blocking function while inside async stuff, the user might want to have a way to "asyncify" that blocking function. And also the "somethingify" doesn't imply it is being executed/called right away, but that something is "converted" in some way. I would hope that would help with the intuition that it just returns another function that is then called, so it's necessary to add the extra chained parenthesis with any possible needed arguments.
But of course, I can change the names if there's another preference.
Tests and Docs
I want to gather feedback first before writing tests and docs, but I'll add them if this or something like this is acceptable.
Type Annotations
I'm also adding the bare minimum type annotation stuff to make it all work with mypy.
Note on AnyIO
I want to have this on AnyIO: agronholm/anyio#403, but AnyIO would want any decision to be taken here first, so I'm making the PR here as well, even if just as a conversation starter.
Note on FastAPI
I want to have this type of interface for users and myself, I use autocompletion and tooling/types a lot. I thought of adding something like this to FastAPI itself, but I think this would really belong here in Trio and AnyIO, not in FastAPI.
I also intend to add it to the FastAPI docs to explain "advanced" use cases with concurrency and blocking code in threads run from async code, etc. I would like to document all those things in FastAPI referring to and explaining these libraries directly (AnyIO, Trio), instead of writing wrappers in FastAPI or anything else. 🤓
As I see this (in particular taking kwargs) is quite debated,
I'm considering buildingI built a small package (Asyncer) just with those wrappers, documenting it as highly experimental and subjective (to my point of view). Maybe that could help gauge how useful that API style is for Trio and AnyIO before making any commitments. I don't like the feel of "yet another library", but it might help separate what is more stable (Trio and AnyIO) and what is a Frankenstein temporary experiment (this idea I'm proposing).2022-01-08 Edit: I built Asyncer to try out this API design. 🤓