From 80249c9923dc434476e0deb6e39682de8ceeb505 Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Mon, 1 Jul 2024 17:42:13 -0400 Subject: [PATCH 01/25] Add async closure redux RFC --- text/0000-async-closure.md | 541 +++++++++++++++++++++++++++++++++++++ 1 file changed, 541 insertions(+) create mode 100644 text/0000-async-closure.md diff --git a/text/0000-async-closure.md b/text/0000-async-closure.md new file mode 100644 index 00000000000..df0f91b3f67 --- /dev/null +++ b/text/0000-async-closure.md @@ -0,0 +1,541 @@ +- Feature Name: `async_closure`[^rework][^plural] +[^rework]: This RFC reworks the "async closures" section of [RFC 2394] +[^plural]: The original feature name was not pluralized, and though it'd be more correct in the plural, it's probably too late to change at this point. +- Start Date: 2024-06-25 +- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- Tracking Issue: [rust-lang/rust#62290](https://github.com/rust-lang/rust/issues/62290) + +[RFC 2394]: https://rust-lang.github.io/rfcs/2394-async_await.html#async--closures + +# Summary +[summary]: #summary + +This RFC adds an `async` bound modifier to the `Fn` family of trait bounds. The combination desugars to a set of perma-unstable `AsyncFn{,Mut,Once}` traits that parallel the current `Fn{,Mut,Once}` traits. + +These traits give users the ability to express bounds for async callable types that are higher-ranked, and allow async closures to return futures which borrow from the closure's captures. + +This RFC also connects these traits to the `async || {}` closure syntax, as originally laid out in [RFC 2394][], and confirms the necessity of a first-class async closure syntax. + +[RFC 2394]: (https://rust-lang.github.io/rfcs/2394-async_await.html#async--closures) + +# Motivation +[motivation]: #motivation + +Users hit two major pitfalls when writing async code that uses closures and `Fn` trait bounds: + +- The inability to express higher-ranked async function signatures. +- That closures cannot return futures that borrow from the closure captures. + +We'll discuss each of these in the sections below. + +### Inability to express higher-ranked async function signatures + +Users often employ `Fn()` trait bounds to write more functional code and reduce code duplication by pulling out specific logic into callbacks. When adapting these idioms into async Rust, users find that they need to split their `Fn()` trait bounds[^alloc] into two to account for the fact that `async` blocks and async functions return anonymous futures. E.g.: +[^alloc]: Or return a concrete future type, like `F: Fn() -> Pin>>`. + +```rust +async fn for_each_city(mut f: F) +where + F: for<'c> FnMut(&'c str) -> Fut, + Fut: Future, +{ + for x in ["New York", "London", "Tokyo"] { + f(x).await; + } +} +``` + +However, when they try to call this code, users are often hit with mysterious higher-ranked lifetime errors, e.g.: + +```rust +async fn do_something(city_name: &str) { todo!() } + +async fn main() { + for_each_city(do_something).await; +} +``` + +``` +error: implementation of `FnMut` is not general enough + --> src/main.rs + | + | for_each_city(do_something); + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ implementation of `FnMut` is not general enough + | + = note: `for<'a> fn(&'a str) -> impl Future {do_something}` must implement `FnMut<(&str,)>` + = note: ...but it actually implements `FnMut<(&'0 str,)>`, for some specific lifetime `'0` +``` + +This happens because the type for the `Fut` generic parameter is chosen by the caller, in `main`, and it cannot reference the higher-ranked lifetime `for<'c>` in the `FnMut` trait bound, but the anonymous future produced by calling `do_something` does capture a generic lifetime parameter, and *must* capture it in order to use the `&str` argument. + +### Closures cannot return futures that borrow from their captures + +When users wants to *call* a function that takes an async callback argument, they often reach for `|| async {}` (a closure that returns an anonymous future) because closures can capture state from the local environment and are syntactically lightweight. + +However, users are quickly met with the limitation that they cannot use any of the closure's captures by reference in the async block. E.g.: + +```rust +async fn parent_country(city_name: &str) -> String { todo!() } + +async fn main() { + // Collect the country names of each city in our list. + let mut countries = vec![]; + for_each_city(|city_name| async { + countries.push(parent_country(city_name).await); + }) +} +``` + +``` +error: captured variable cannot escape `FnMut` closure body + --> src/main.rs + | + | let mut countries = vec![]; + | ------------- variable defined here + | for_each_city(|city_name| async { + | _____________________________-_^ + | | | + | | inferred to be a `FnMut` closure + | | countries.push(parent_country(city_name).await); + | | --------- variable captured here + | | }).await; + | |_____^ returns an `async` block that contains a reference to a captured variable, which then escapes the closure body + | + = note: `FnMut` closures only have access to their captured variables while they are executing... + = note: ...therefore, they cannot allow references to captured variables to escape +``` + +The future that is returned by the closure cannot reference any of the captures of the closure, which is a limitation that makes `|| async {}` quite unusable today without needing to, for example, clone data and declare the async block `move`. + +In order for this to work, the `FnMut` trait would need to be ["lending"](https://smallcultfollowing.com/babysteps/blog/2023/05/09/giving-lending-and-async-closures/#async-closures-are-a-lending-pattern); however, there are [complications](https://hackmd.io/@compiler-errors/async-closures#Lending-closures-are-not-typically-FnOnce) with implementing general lending closures. + +# Guide Level Explanation + +Just as you can write functions which accept closures, you can write functions which accept async closures: + +```rust +async fn takes_async_closure(f: impl async Fn(u64)) { + f(0).await; + f(1).await; +} + +takes_async_closure(async |i| { + core::future::ready(i).await; + println!("done with {i}."); +}); +``` + +We recommend using `async Fn` and `async ||` for async closures. This is more flexible than a closure returning a future for the reasons described elsewhere in this RFC. + +Async closures act similarly to closures, and can have parts of their their signatures specified: + +```rust +// They can have arguments annotated with types: +let arg = async |x: i32| { async_add(x, 1).await }; + +// They can have their return types annotated: +let ret = async || -> Vec { async_iterator.collect().await }; + +// They can be higher-ranked: +let hr = async |x: &str| { do_something(x).await }; +``` + +When called, they return an anonymous future type corresponding to the (not-yet-executed) body of the closure. These can be awaited like any other future. + +# Detailed Explanation + +### `AsyncFn*` + +This RFC introduces a family of `AsyncFn` traits. These traits are intended to remain perma-unstable to name or implement, just like the `Fn` traits. Nonetheless, we'll describe the details of these traits so as to explain the user-facing features enabled by them. + +The definition of the traits is (modulo `rustc_` attributes, and the `"rust-call"` ABI): + + +> [!NOTE] +> We omit some details about the `"rust-call"` calling convention and the fact that the `Args` parameter is enforced to be a tuple. + +```rust +/// An async-aware version of the [`FnOnce`](crate::ops::FnOnce) trait. +/// +/// All `async fn` and functions returning futures implement this trait. +pub trait AsyncFnOnce { + /// Future returned by [`AsyncFnOnce::async_call_once`]. + type CallOnceFuture: Future; + + /// Output type of the called closure's future. + type Output; + + /// Call the [`AsyncFnOnce`], returning a future which may move out of the called closure. + fn async_call_once(self, args: Args) -> Self::CallOnceFuture; +} + +/// An async-aware version of the [`FnMut`](crate::ops::FnMut) trait. +/// +/// All `async fn` and functions returning futures implement this trait. +pub trait AsyncFnMut: AsyncFnOnce { + /// Future returned by [`AsyncFnMut::async_call_mut`] and [`AsyncFn::async_call`]. + type CallRefFuture<'a>: Future + where + Self: 'a; + + /// Call the [`AsyncFnMut`], returning a future which may borrow from the called closure. + fn async_call_mut(&mut self, args: Args) -> Self::CallRefFuture<'_>; +} + +/// An async-aware version of the [`Fn`](crate::ops::Fn) trait. +/// +/// All `async fn` and functions returning futures implement this trait. +pub trait AsyncFn: AsyncFnMut { + /// Call the [`AsyncFn`], returning a future which may borrow from the called closure. + fn async_call(&self, args: Args) -> Self::CallRefFuture<'_>; +} +``` + +### Associated types of `AsyncFn*` traits are not nameable + +Note that, unlike what is true today with the current `Fn*` traits, this RFC reserves as an implementation detail the associates types of the `AsyncFn*` traits, and these will not be nameable as part of the stable interface specified by this RFC. + +### `async` bound modifier on `Fn()` trait bounds + +The `AsyncFn*` traits specified above are nameable via a new `async` bound modifier that is allowed on `Fn` trait bounds. That is, `async Fn*() -> T` desugars to `AsyncFn*() -> T` in bounds, where `Fn*` is one of the three flavors of existing function traits: `Fn`/`FnMut`/`FnOnce`. + +This RFC specifies the modification to the _TraitBound_ nonterminal in the grammar: + +> **Syntax** +> _TraitBound_ : +>     `?`? _ForLifetimes_? `async`? _TypePath_\ +>   | `(` `?`? _ForLifetimes_? `async`? _TypePath_ `)` + +Since the grammar doesn't distinguish parenthesized and angle-bracketed generics in `_TypePath_`, `async` as a trait bound modifier will be **accepted** in all trait bounds at _parsing_ time, but it will be **denied** by the compiler _post-expansion_ if it's not attached to a parenthesized `Fn()` trait bound. + +Users are able to write `async Fn*() -> T` trait bounds in all positions that trait bounds are allowed, for example: + +```rust +fn test(f: F) where F: async Fn() -> i32 {} + +fn apit(f: impl async Fn() -> i32) {} + +trait Tr { + type Callable: async Fn() -> i32; +} + +// Allowed syntactically; not currently object-safe: +let _: Box = todo!(); +``` + +### When is `async Fn*()` implemented? + +All currently-stable callable types (i.e., closures, function items, function pointers, and `dyn Fn*` trait objects) automatically implement `async Fn*() -> T` if they implement `Fn*() -> Fut` for some output type `Fut`, and `Fut` implements `Future`. + +Async closures also implement `async Fn*()`, but their relationship to this trait is detailed later in the RFC. + +Some stable types that would implement `async Fn()` today include, e.g.: + +```rust! +// Async functions: +async fn foo() {} + +// Functions that return a concrete future type: +fn foo() -> Pin>> { Box::pin(async {}) } + +// Closures that return an async block: +let c = || async {}; +``` + +Notably, we can now express higher-ranked async callback bounds: + +```rust +// We could also use APIT: `mut f: impl async FnMut(&str)`. +async fn for_each_city(mut f: F) +where + F: async FnMut(&str), +// ...which is sugar for: +// F: for<'a> async FnMut(&'a str), +{ + for x in ["New York", "London", "Tokyo"] { + f(x).await; + } +} + +async fn increment_city_population_db_query(city_name: &str) { todo!() } + +async fn main() { + // Works for `async fn` that is higher-ranked. + for_each_city(increment_city_population_db_query).await; +} +``` + +### Async closures + +Async closures were first specified in [RFC 2394][]. This RFC doesn't affect them syntactically, but it does lay out new rules for how they interact with `AsyncFn*` traits. + +Like async functions, async closures return futures which execute the code in the body of the closure. Like closures, they are allowed to capture variables from the surrounding environment if they are mentioned in the body of the closure. Each variable is captured in the most permissive capture mode allowed, and this capture analysis generally follows the same rules as closures, including allowing disjoint captures per [RFC 2229](https://rust-lang.github.io/rfcs/2229-capture-disjoint-fields.html). + +#### Async closures allow self-borrows + +However, since async closures return a future instead of executing their bodies directly, the future corresponding to the body must *itself* capture all of the closure's captures. These are captured with the most permissive capture mode allowed, which (unless the captures are being consumed by-value) necessitates borrowing from the closure itself. + +For example: + +```rust +let vec: Vec = vec![]; + +let closure = async || { + vec.push(ready(String::from("")).await); +}; +``` + +The closure captures `vec` with some `&'closure mut Vec` which lives until the closure is dropped. Then every call to the closure reborrows that mutable reference `&'call Vec` which lives until the future is dropped (e.g. `await`ed). + +As another example: + +```rust! +let string: String = "Hello, world".into(); + +let closure = async move || { + ready(&string).await; +}; +``` + +The closure is marked with `move`, which means it takes ownership of the string by *value*. However, if the future also took ownership of the string *from* the closure, then the closure would only be callable once. This is not a problem, since according to the usage of `string` in the closure body, the future only needs take a reference to it to call `ready`. Therefore, the future captures `&'call String` for some lifetime which lives until the future is dropped. + +#### Closure kind analysis + +Similarly to regular closures, async closures always implement `AsyncFnOnce`. They additionally implement `AsyncFnMut` if they do not move any of their captured values, and `AsyncFn` if they additionally do not mutate their captured values. + +Async closures unconditionally implement the (non-async) `FnOnce` trait. They implement `FnMut` and `Fn` if they do not move their captured values, mutate them, or borrow in the future any data from the closure. The future borrows data from the closure if the data being borrowed by the future is owned or if the borrow is mutable. + +For example: + +```rust +let s = String::from("hello, world"); +// Implements `async Fn()` along with `FnMut` and `Fn` +// because it can copy the `&String` that it captures. +let _ = async || { + println!("{s}"); +}; + +let s = String::from("hello, world"); +// Implements `async Fn()` but not `FnMut` or `Fn` because +// it moves and owns a value of type `String`, and therefore +// the future it returns needs to take a pointer to data +// owned by the closure. +let _ = async move || { + println!("{s}"); +}; + +let mut s = String::from("hello, world"); +// Implements `async FnMut()` but not `FnMut` or `Fn` +// because it needs to reborrow a mutable pointer to `s`. +let _ = async move || { + s.push('!'); +}; +``` + +#### Specifics about the `AsyncFnOnce` implementation, interaction with `move` + +If the closure is inferred to be `async Fn` or `async FnMut`, then the compiler will synthesize an `async FnOnce` implementation for the closure which returns a future that doesn't borrow any captured values from the closure, but instead *moves* the captured values into the future. + +For example: + +```rust +let s = String::from("hello, world"); + +let closure = async move || { + ready(&s); +}; +// At this point, `s` is moved out of. However, the +// allocation for `s` is still live. It just lives as a +// captured field in `closure`. + +// Manually call `AsyncFnOnce` -- this isn't stable since +// `AsyncFnOnce` isn't stable, but it's useful for the demo. +let fut = AsyncFnOnce::call_once(closure, ()); +// At this point, `closure` is dropped. However, the +// allocation for `s` is still live. It now lives as a +// captured field in `fut`. + +fut.await; +// After the future is awaited, it's dropped. At that +// point, the allocation for `s` is dropped. +``` + +### Interaction with return-type notation, naming the future returned by calling + +With `async Fn() -> T` trait bounds, we don't know anything about the `Future` returned by calling the async closure other than that it's a `Future` and awaiting that future returns `T`. + +This is not always sufficient, for example, if you want to spawn a future onto another thread: + +```rust +async fn foo(x: impl async Fn(&str)) -> Result<()> { + tokio::spawn(x("hello, world")).await +} +``` + +``` +error[E0277]: cannot be sent between threads safely + --> src/lib.rs + | + | tokio::spawn(x("hello, world")).await + | ------------ ^^^^^^^^^^^^^^^^^ cannot be sent between threads safely + | | + | required by a bound introduced by this call +``` + +When the RTN RFC is accepted, this RFC specifies that users will be allowed to add RTN-like bounds to type parameters that are also bounded by `async Fn()`, like so: + +```rust +async fn foo(x: F) -> Result<()> +where + F: async Fn(&str) -> Result<()>, + // The future from calling `F` is `Send` and `'static`. + F(..): Send + 'static, +{ + tokio::spawn(x("hello, world")).await +} +``` + +This bound is only valid if there is a corresponding `async Fn*()` trait bound. + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +### Why do we need a new set of `AsyncFn*` traits? + +As demonstrated in the motivation section, we need a set of traits that are *lending* in order to represent futures which borrow from the closure's captures. We technically only need to add `LendingFn` and `LendingFnMut` to our lattice of `Fn*` traits, leaving us with a hierarchy of traits like so: + +```mermaid +flowchart LR + +Fn +FnMut +FnOnce +LendingFn +LendingFnMut + +Fn -- isa --> FnMut +FnMut -- isa --> FnOnce + +LendingFn -- isa --> LendingFnMut + +Fn -- isa --> LendingFn +FnMut -- isa --> LendingFnMut +``` + +However, there are some concrete technical implementation details that limit our ability to use `LendingFn` ergonomically in the compiler today. These have to do with: + +- Closure signature inference. +- Limitations around higher-ranked trait bounds. +- Shortcomings with error messages. + +These limitations, plus the fact that the underlying trait should have no effect on the user experience of async closures and async `Fn` trait bounds, leads us to `AsyncFn*` for now. To ensure we can eventually move to these more general traits, we reserved the precise `AsyncFn*` trait definitions (including the associated types) as an implementation detail. + +### Why can't we just use `|| async {}`? + +`async ||` is analogous with `async fn`, and has an intuitive, first-class way to declare the return type of the future: + +```rust! +let c = async || -> i32 { 0 }; +``` + +There isn't currently a way to annotate the future's return type in a closure that returns a future: + +```rust! +let c = || -> /* ??? */ async { 0 }; +``` + +We could reuse `impl Future` to give users the ability to annotate the type of the future returned by the closure this position, but it would require giving yet another subtly different meaning to `impl Trait`, since async closures return a *different* type when being called by-ref or by-move. + +This also would have subtle limitations, e.g.: + +```rust! +// Easy to reanalyze as an async closure. +let _ = || async { do_stuff().await }; + +// Not possible to reanalyze as an async closure without a lot more work. +let _ = || { + let fut = async { do_stuff().await }; + fut +}; +``` + +### Why not `F: AsyncFn() -> T`, naming `AsyncFn*` directly? + +Reusing the `async` keyword allows users to understand what an `async Fn() -> T` trait bound does by analogy, since they already should know that adding `async` to some `fn foo() -> T` makes it return an `impl Future` instead of the type `T`. + +### Why do we even need `AsyncFnOnce`? + +We could desugar `async FnOnce() -> T` directly to `FnOnce<(), Output: Future>`. It seems overly complicated for an implementation detail, since users should never care what's *behind* the `AsyncFnOnce` trait bound. + +# Drawbacks +[drawbacks]: #drawbacks + +### Users might confusedly write `|| async {}` over `async || {}` + +Users may be confused whether to write `|| async {}` or `async || {}`. The fact that `async || {}` has extra "superpowers" with respect to lending may lead to users hitting unnecessary errors if they invert the ordering. + +We should be able to detect when users write `|| async {}` -- and subsequently hit borrow checker issues -- and give a useful error message to move the `async` keyword. We may also lint against `|| async {}` in code that *does* pass, since it's not as expressive. + +### Users might write `F: Fn() -> Fut, Fut: Future` over `F: async Fn() -> T` + +A similar problem could occur if users try to write "old style" trait bounds with two generic parameters `F: Fn() -> Fut` and `Fut: Future`. For example: + +```rust! +async fn for_each_city(cb: F) +where + F: Fn(&str) -> Fut, + Fut: Future, +{ + for x in ["New York", "London", "Tokyo"] { + cb(x).await; + } +} +``` + +This is problematic for two reasons: + +1. Although the `Fn` trait bound may be higher-ranked, the future that is returned cannot be, since we need to infer a single type parameter substitution for the `Future` bound to hold. +2. There's no way for an async closure to be lending in this case, so the expressivity of the closure is limited. + +We can similarly implement a lint to detect cases where users write these two-part bounds and suggest that they instead write a single `async Fn() -> T` bound. This comes with the normal caveats of removing a type parameter from the function signature, e.g. semver incompatibility (since the type parameter may be turbofished). However, when users are designing a new API, they should always reach for `async Fn` trait bounds when they want to be generic over a closure that returns a future. + +### Lack of a desugaring + +It's not possible to directly name the future returned by calling some generic `T: async Fn()`. This means that it's not possible, for example, to convert `futures-rs`'s [`StreamExt::then` combinator](https://docs.rs/futures/0.3.30/futures/stream/trait.StreamExt.html#method.then), since the output future is referenced in the definition of [`Then`](https://docs.rs/futures-util/0.3.30/src/futures_util/stream/stream/then.rs.html#19) returned by the combinator. + +Fixing this is a follow-up goal that we're interested in pursuing in the near future. + +# Prior art +[prior-art]: #prior-art + +[RFC 2394] described async closures at a very high level, and expressed that users would very likely want this feature eventually. This RFC confirms that suspicion. + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +We leave no language questions unresolved in this RFC. + +# Future possibilities +[future-possibilities]: #future-possibilities + +### `gen Fn()`, `async gen Fn()` + +The existence of other coroutine-like modifiers, e.g. `gen` ([RFC 3513](https://rust-lang.github.io/rfcs/3513-gen-blocks.html)) and `async gen`, suggests that we should also think about supporting these in closures and `Fn()` trait bounds. + +This shouldn't be too difficult to support, and we can unify these further by moving on to a general `LendingFn*` trait. This has some implementation concerns, but should be doable in the long term. + +### `async` bound modifier on arbitrary traits + +There has been previous discussion of allowing `async` trait bounds on arbitrary traits, possibly based off a `?async` maybe-async genericity system. + +This RFC neither requires this more general extension to the language to be implemented, nor does it necessarily preclude this being an eventual possibility, since `AsyncFn*` remains perma-unstable to implement. + +### Making `async Fn()` object-safe + +Future work should be done to make `async Fn()` object-safe, so it can be used in `Box`, etc. E.g.: + +```rust +let handlers: HashMap> = todo!(); +``` + +This work will likely take a similar approach to making `async fn` in traits object-safe, since the major problem is how to "erase" the future returned by the async closure or callable, which differs for each implementation of the trait. From 6000a0c2443006f982db657910703be651323763 Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Mon, 1 Jul 2024 17:45:40 -0400 Subject: [PATCH 02/25] update numbers --- text/{0000-async-closure.md => 3668-async-closure.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename text/{0000-async-closure.md => 3668-async-closure.md} (99%) diff --git a/text/0000-async-closure.md b/text/3668-async-closure.md similarity index 99% rename from text/0000-async-closure.md rename to text/3668-async-closure.md index df0f91b3f67..772afed124f 100644 --- a/text/0000-async-closure.md +++ b/text/3668-async-closure.md @@ -2,7 +2,7 @@ [^rework]: This RFC reworks the "async closures" section of [RFC 2394] [^plural]: The original feature name was not pluralized, and though it'd be more correct in the plural, it's probably too late to change at this point. - Start Date: 2024-06-25 -- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- RFC PR: [rust-lang/rfcs#3668](https://github.com/rust-lang/rfcs/pull/3668) - Tracking Issue: [rust-lang/rust#62290](https://github.com/rust-lang/rust/issues/62290) [RFC 2394]: https://rust-lang.github.io/rfcs/2394-async_await.html#async--closures From a514d3b4ac15ada5d9929bd5d31f228b316bed54 Mon Sep 17 00:00:00 2001 From: Travis Cross Date: Tue, 2 Jul 2024 18:24:13 +0000 Subject: [PATCH 03/25] Clarify section about associated types being reserved We specify that the details about the traits, including the associated types, are implementation details. Let's make the section describing this, with respect to the associated types, more clear and give an example. (Thanks to RalfJ and Yosh for raising that this section could be clarified.) --- text/3668-async-closure.md | 29 ++++++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-) diff --git a/text/3668-async-closure.md b/text/3668-async-closure.md index 772afed124f..ae3c5ce7d87 100644 --- a/text/3668-async-closure.md +++ b/text/3668-async-closure.md @@ -193,7 +193,34 @@ pub trait AsyncFn: AsyncFnMut { ### Associated types of `AsyncFn*` traits are not nameable -Note that, unlike what is true today with the current `Fn*` traits, this RFC reserves as an implementation detail the associates types of the `AsyncFn*` traits, and these will not be nameable as part of the stable interface specified by this RFC. +Unlike what is true today with the current `Fn*` traits, this RFC reserves as an implementation detail the associated types of the `AsyncFn*` traits, and these will not be nameable as part of the stable interface specified by this RFC. + +That is, using the existing `FnOnce` trait, we can write this today on stable Rust: + +```rust +fn foo() +where + F: FnOnce() -> T, + F::Output: Send, //~ OK +{ +} +``` + +(We decided to allow this in [#34365](https://github.com/rust-lang/rust/pull/34365).) + +However, this RFC reserves as an implementation detail the associated types of the traits specified above, so this does not work: + +```rust +fn foo() +where + F: async FnOnce() -> T, + F::Output: Send, + //~^ ERROR use of unstable library feature + F::CallOnceFuture: Send, + //~^ ERROR use of unstable library feature +{ +} +``` ### `async` bound modifier on `Fn()` trait bounds From d2446cb34efa6636e994194afccfc05a46c1d6e3 Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Tue, 2 Jul 2024 14:33:59 -0400 Subject: [PATCH 04/25] No perma --- text/3668-async-closure.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/text/3668-async-closure.md b/text/3668-async-closure.md index ae3c5ce7d87..15a9e1d0c3c 100644 --- a/text/3668-async-closure.md +++ b/text/3668-async-closure.md @@ -10,7 +10,7 @@ # Summary [summary]: #summary -This RFC adds an `async` bound modifier to the `Fn` family of trait bounds. The combination desugars to a set of perma-unstable `AsyncFn{,Mut,Once}` traits that parallel the current `Fn{,Mut,Once}` traits. +This RFC adds an `async` bound modifier to the `Fn` family of trait bounds. The combination desugars to a set of unstable `AsyncFn{,Mut,Once}` traits that parallel the current `Fn{,Mut,Once}` traits. These traits give users the ability to express bounds for async callable types that are higher-ranked, and allow async closures to return futures which borrow from the closure's captures. @@ -146,7 +146,7 @@ When called, they return an anonymous future type corresponding to the (not-yet- ### `AsyncFn*` -This RFC introduces a family of `AsyncFn` traits. These traits are intended to remain perma-unstable to name or implement, just like the `Fn` traits. Nonetheless, we'll describe the details of these traits so as to explain the user-facing features enabled by them. +This RFC introduces a family of `AsyncFn` traits. These traits are intended to remain unstable to name or implement, just like the `Fn` traits. Nonetheless, we'll describe the details of these traits so as to explain the user-facing features enabled by them. The definition of the traits is (modulo `rustc_` attributes, and the `"rust-call"` ABI): @@ -555,7 +555,7 @@ This shouldn't be too difficult to support, and we can unify these further by mo There has been previous discussion of allowing `async` trait bounds on arbitrary traits, possibly based off a `?async` maybe-async genericity system. -This RFC neither requires this more general extension to the language to be implemented, nor does it necessarily preclude this being an eventual possibility, since `AsyncFn*` remains perma-unstable to implement. +This RFC neither requires this more general extension to the language to be implemented, nor does it necessarily preclude this being an eventual possibility, since `AsyncFn*` remains unstable to implement. ### Making `async Fn()` object-safe From 29582aa37d78d750746af3d08d1ffc2e4a1f24f5 Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Tue, 2 Jul 2024 14:35:03 -0400 Subject: [PATCH 05/25] More clarifications --- text/3668-async-closure.md | 28 ++++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-) diff --git a/text/3668-async-closure.md b/text/3668-async-closure.md index 15a9e1d0c3c..43757241e0a 100644 --- a/text/3668-async-closure.md +++ b/text/3668-async-closure.md @@ -125,7 +125,7 @@ takes_async_closure(async |i| { }); ``` -We recommend using `async Fn` and `async ||` for async closures. This is more flexible than a closure returning a future for the reasons described elsewhere in this RFC. +We recommend using `async Fn()`/`async FnMut()`/`async FnOnce()` and `async ||` for async closures. This is more flexible than a closure returning a future for the reasons described elsewhere in this RFC. Async closures act similarly to closures, and can have parts of their their signatures specified: @@ -142,6 +142,26 @@ let hr = async |x: &str| { do_something(x).await }; When called, they return an anonymous future type corresponding to the (not-yet-executed) body of the closure. These can be awaited like any other future. +The `async Fn` trait bound syntax can be used anywhere a trait bound is allowed, such as: + +```rust +/// In return-position impl trait: +fn closure() -> impl async Fn() { async || {} } + +/// In trait bounds: +trait Foo: Sized +where + F: async Fn() +{ + fn new(f: F) -> Self; +} + +/// in GATs: +trait Gat { + type AsyncHasher: async Fn(T) -> i32; +} +``` + # Detailed Explanation ### `AsyncFn*` @@ -409,7 +429,7 @@ error[E0277]: cannot be sent between threads safely | required by a bound introduced by this call ``` -When the RTN RFC is accepted, this RFC specifies that users will be allowed to add RTN-like bounds to type parameters that are also bounded by `async Fn()`, like so: +With the acceptance of the RTN (return-type notation) [RFC 3654](https://github.com/rust-lang/rfcs/pull/3654), this RFC specifies that users will be allowed to add RTN-like bounds to type parameters that are also bounded by `async Fn()`. Concretely, this bound expands to bound both `CallOnceFuture` and `CallRefFuture` (if the latter exists): ```rust async fn foo(x: F) -> Result<()> @@ -417,6 +437,10 @@ where F: async Fn(&str) -> Result<()>, // The future from calling `F` is `Send` and `'static`. F(..): Send + 'static, + // Which expands to two bounds: + // `for<'a> ::CallRefFuture<'a>: Send` + // `::CallOnceFuture: Send` + // the latter is only if `F` is bounded with `async Fn` or `async FnMut`. { tokio::spawn(x("hello, world")).await } From 3938f65cf221c7d379a1943f6f38cf03013e7a80 Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Tue, 2 Jul 2024 14:36:17 -0400 Subject: [PATCH 06/25] Add missing generics --- text/3668-async-closure.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/text/3668-async-closure.md b/text/3668-async-closure.md index 43757241e0a..2880c918708 100644 --- a/text/3668-async-closure.md +++ b/text/3668-async-closure.md @@ -34,7 +34,7 @@ Users often employ `Fn()` trait bounds to write more functional code and reduce [^alloc]: Or return a concrete future type, like `F: Fn() -> Pin>>`. ```rust -async fn for_each_city(mut f: F) +async fn for_each_city(mut f: F) where F: for<'c> FnMut(&'c str) -> Fut, Fut: Future, @@ -293,7 +293,7 @@ Notably, we can now express higher-ranked async callback bounds: ```rust // We could also use APIT: `mut f: impl async FnMut(&str)`. -async fn for_each_city(mut f: F) +async fn for_each_city(mut f: F) where F: async FnMut(&str), // ...which is sugar for: @@ -532,7 +532,7 @@ We should be able to detect when users write `|| async {}` -- and subsequently h A similar problem could occur if users try to write "old style" trait bounds with two generic parameters `F: Fn() -> Fut` and `Fut: Future`. For example: ```rust! -async fn for_each_city(cb: F) +async fn for_each_city(cb: F) where F: Fn(&str) -> Fut, Fut: Future, From 86fc14aeab4753561d638663f5766f1dff6e79c1 Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Tue, 2 Jul 2024 14:39:38 -0400 Subject: [PATCH 07/25] Motivate why non-async closures should implement AsyncFn --- text/3668-async-closure.md | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/text/3668-async-closure.md b/text/3668-async-closure.md index 2880c918708..ae19e39a45b 100644 --- a/text/3668-async-closure.md +++ b/text/3668-async-closure.md @@ -274,9 +274,28 @@ let _: Box = todo!(); All currently-stable callable types (i.e., closures, function items, function pointers, and `dyn Fn*` trait objects) automatically implement `async Fn*() -> T` if they implement `Fn*() -> Fut` for some output type `Fut`, and `Fut` implements `Future`. -Async closures also implement `async Fn*()`, but their relationship to this trait is detailed later in the RFC. +This is to make sure that `async Fn*()` trait bounds have maximum compatibility with existing callable types which return futures, such as async function items and closures which return boxed futures. These implementations are built-in, but can conceptually be understood as: -Some stable types that would implement `async Fn()` today include, e.g.: +```rust +impl AsyncFnOnce for F +where + F: FnOnce, + Fut: Future, +{ + type Output = T; + type CallOnceFuture = Fut; + + fn async_call_once(self, args: Args) -> Self::CallOnceFuture { + FnOnce::call_once(self, args) + } +} +``` + +And similarly for `AsyncFnMut` and `AsyncFn`, with the appropriate `FnMut` and `Fn` trait bounds, respectively. + +Async closures also implement `async Fn*()`, but their relationship to this trait is detailed later in the RFC. The reason that all of these bounds (for regular callable types *and* async closures) are built-in is because these blanket impls would overlap with the built-in implementation of `AsyncFn*` for async closures, which must have distinct implementations to support self-borrowing futures. + +Some stable types that implement `async Fn()` today include, e.g.: ```rust! // Async functions: From 90ea425baeea045eb6e47df7ba8d08b709b68f0a Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Tue, 2 Jul 2024 14:39:08 -0400 Subject: [PATCH 08/25] Clarify the ?for<'a> weirdness --- text/3668-async-closure.md | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/text/3668-async-closure.md b/text/3668-async-closure.md index ae19e39a45b..1f73549918a 100644 --- a/text/3668-async-closure.md +++ b/text/3668-async-closure.md @@ -250,8 +250,10 @@ This RFC specifies the modification to the _TraitBound_ nonterminal in the gramm > **Syntax** > _TraitBound_ : ->     `?`? _ForLifetimes_? `async`? _TypePath_\ ->   | `(` `?`? _ForLifetimes_? `async`? _TypePath_ `)` +>     `async`? `?`? _ForLifetimes_? _TypePath_\ +>   | `(` `async`? `?`? _ForLifetimes_? _TypePath_ `)` + +**note**: The grammar specifies that any `for<'a>` higher-ranked lifetimes come *after* the `?` trait polarity. This seems inconsistent, but should be changed independently from this RFC. There's an open question about how to deal with the ordering problem of `?`, `for<'a>`, and `async`, or we want to separate `async` traits into their own production rule. Since the grammar doesn't distinguish parenthesized and angle-bracketed generics in `_TypePath_`, `async` as a trait bound modifier will be **accepted** in all trait bounds at _parsing_ time, but it will be **denied** by the compiler _post-expansion_ if it's not attached to a parenthesized `Fn()` trait bound. @@ -583,7 +585,15 @@ Fixing this is a follow-up goal that we're interested in pursuing in the near fu # Unresolved questions [unresolved-questions]: #unresolved-questions -We leave no language questions unresolved in this RFC. +### `? for<'a>` and its interaction with `async` + +Currently on nightly, we parse the `async` trait bound modifier along with `?` (called polarity) *before* the `for<'a>` lifetime binders. This probably should get fixed so that the binder occurs on the *outside* of the trait, like so: + +``` +where T: for<'a> async ?Trait +``` + +(Which is semantically invalid but syntactically valid.) This is currently proposed in [rust-lang/rust#127054](https://github.com/rust-lang/rust/pull/127054), which should be decided before stabilization, and the stabilization report can re-confirm the correct ordering of `for<'a>` and `async`. # Future possibilities [future-possibilities]: #future-possibilities From febff99d061c34c6dbca2b6e3fda75d20b1c844d Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Tue, 2 Jul 2024 14:44:45 -0400 Subject: [PATCH 09/25] One more caveat about AsyncFn* and built-in impls --- text/3668-async-closure.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/text/3668-async-closure.md b/text/3668-async-closure.md index 1f73549918a..0c202d2718e 100644 --- a/text/3668-async-closure.md +++ b/text/3668-async-closure.md @@ -276,7 +276,9 @@ let _: Box = todo!(); All currently-stable callable types (i.e., closures, function items, function pointers, and `dyn Fn*` trait objects) automatically implement `async Fn*() -> T` if they implement `Fn*() -> Fut` for some output type `Fut`, and `Fut` implements `Future`. -This is to make sure that `async Fn*()` trait bounds have maximum compatibility with existing callable types which return futures, such as async function items and closures which return boxed futures. These implementations are built-in, but can conceptually be understood as: +This is to make sure that `async Fn*()` trait bounds have maximum compatibility with existing callable types which return futures, such as async function items and closures which return boxed futures. Async closures also implement `async Fn*()`, but their relationship to this trait is detailed later in the RFC. + +These implementations are built-in, but can conceptually be understood as: ```rust impl AsyncFnOnce for F @@ -293,9 +295,11 @@ where } ``` -And similarly for `AsyncFnMut` and `AsyncFn`, with the appropriate `FnMut` and `Fn` trait bounds, respectively. +And similarly for `AsyncFnMut` and `AsyncFn`, with the appropriate `FnMut` and `Fn` trait bounds, respectively. + +**NOTE**: This only works currently for *concrete* callable types -- for example, `impl Fn() -> impl Future` does not implement `impl async Fn()`, due to the fact that these blanket impls do not exist in reality. This may be relaxed in the future. Users can work around this by wrapping their type in an async closure and calling it. -Async closures also implement `async Fn*()`, but their relationship to this trait is detailed later in the RFC. The reason that all of these bounds (for regular callable types *and* async closures) are built-in is because these blanket impls would overlap with the built-in implementation of `AsyncFn*` for async closures, which must have distinct implementations to support self-borrowing futures. +The reason that these implementations are built-in is because using blanket impls would cause overlap with the built-in implementation of `AsyncFn*` for async closures, which must have a distinct implementation to support self-borrowing futures. Some stable types that implement `async Fn()` today include, e.g.: From 23b9af83562c342effe9800162ec1ef1c2619f3f Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Tue, 2 Jul 2024 14:46:25 -0400 Subject: [PATCH 10/25] Grammar nit wrt ? and async --- text/3668-async-closure.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3668-async-closure.md b/text/3668-async-closure.md index 0c202d2718e..5d1853a6f4a 100644 --- a/text/3668-async-closure.md +++ b/text/3668-async-closure.md @@ -255,7 +255,7 @@ This RFC specifies the modification to the _TraitBound_ nonterminal in the gramm **note**: The grammar specifies that any `for<'a>` higher-ranked lifetimes come *after* the `?` trait polarity. This seems inconsistent, but should be changed independently from this RFC. There's an open question about how to deal with the ordering problem of `?`, `for<'a>`, and `async`, or we want to separate `async` traits into their own production rule. -Since the grammar doesn't distinguish parenthesized and angle-bracketed generics in `_TypePath_`, `async` as a trait bound modifier will be **accepted** in all trait bounds at _parsing_ time, but it will be **denied** by the compiler _post-expansion_ if it's not attached to a parenthesized `Fn()` trait bound. +Since the grammar doesn't distinguish parenthesized and angle-bracketed generics in `_TypePath_`, `async` as a trait bound modifier will be **accepted** in all trait bounds at _parsing_ time, but it will be **rejected** by the compiler _post-expansion_ if it's not attached to a parenthesized `Fn()` trait bound. Similarly, the combination of `async` and `?` is syntactically valid but semantically invalid, and will be rejected by the compiler post-expansion. Users are able to write `async Fn*() -> T` trait bounds in all positions that trait bounds are allowed, for example: From 26ded6830858ab6baf31ac5eab1a5dade76ae603 Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Tue, 2 Jul 2024 15:00:00 -0400 Subject: [PATCH 11/25] Oops, missing words --- text/3668-async-closure.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3668-async-closure.md b/text/3668-async-closure.md index 5d1853a6f4a..4a569e8b93f 100644 --- a/text/3668-async-closure.md +++ b/text/3668-async-closure.md @@ -253,7 +253,7 @@ This RFC specifies the modification to the _TraitBound_ nonterminal in the gramm >     `async`? `?`? _ForLifetimes_? _TypePath_\ >   | `(` `async`? `?`? _ForLifetimes_? _TypePath_ `)` -**note**: The grammar specifies that any `for<'a>` higher-ranked lifetimes come *after* the `?` trait polarity. This seems inconsistent, but should be changed independently from this RFC. There's an open question about how to deal with the ordering problem of `?`, `for<'a>`, and `async`, or we want to separate `async` traits into their own production rule. +**note**: The grammar specifies that any `for<'a>` higher-ranked lifetimes come *after* the `?` trait polarity. This seems inconsistent, but should be changed independently from this RFC. There's an open question about how to deal with the ordering problem of `?`, `for<'a>`, and `async`, or if we want to separate `async` traits into their own production rule that enforces the right ordering of `for<'a> async`. Since the grammar doesn't distinguish parenthesized and angle-bracketed generics in `_TypePath_`, `async` as a trait bound modifier will be **accepted** in all trait bounds at _parsing_ time, but it will be **rejected** by the compiler _post-expansion_ if it's not attached to a parenthesized `Fn()` trait bound. Similarly, the combination of `async` and `?` is syntactically valid but semantically invalid, and will be rejected by the compiler post-expansion. From b2a8e07d502b1c59a04c0b8a015333110b3dd1c2 Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Wed, 3 Jul 2024 10:14:46 -0400 Subject: [PATCH 12/25] Update text/3668-async-closure.md Co-authored-by: Yosh --- text/3668-async-closure.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3668-async-closure.md b/text/3668-async-closure.md index 4a569e8b93f..ab41a6fddca 100644 --- a/text/3668-async-closure.md +++ b/text/3668-async-closure.md @@ -520,7 +520,7 @@ There isn't currently a way to annotate the future's return type in a closure th let c = || -> /* ??? */ async { 0 }; ``` -We could reuse `impl Future` to give users the ability to annotate the type of the future returned by the closure this position, but it would require giving yet another subtly different meaning to `impl Trait`, since async closures return a *different* type when being called by-ref or by-move. +We could reuse `impl Future` to give users the ability to annotate the type of the future returned by the closure in this position, but it would require giving yet another subtly different meaning to `impl Trait`, since async closures return a *different* type when being called by-ref or by-move. This also would have subtle limitations, e.g.: From 3298481e17a24840cb6abff5b0505b0a67b19365 Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Wed, 3 Jul 2024 12:06:42 -0400 Subject: [PATCH 13/25] Fix hackmd-ism --- text/3668-async-closure.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/text/3668-async-closure.md b/text/3668-async-closure.md index ab41a6fddca..d1b552f463c 100644 --- a/text/3668-async-closure.md +++ b/text/3668-async-closure.md @@ -303,7 +303,7 @@ The reason that these implementations are built-in is because using blanket impl Some stable types that implement `async Fn()` today include, e.g.: -```rust! +```rust // Async functions: async fn foo() {} @@ -361,7 +361,7 @@ The closure captures `vec` with some `&'closure mut Vec` which lives unt As another example: -```rust! +```rust let string: String = "Hello, world".into(); let closure = async move || { @@ -510,13 +510,13 @@ These limitations, plus the fact that the underlying trait should have no effect `async ||` is analogous with `async fn`, and has an intuitive, first-class way to declare the return type of the future: -```rust! +```rust let c = async || -> i32 { 0 }; ``` There isn't currently a way to annotate the future's return type in a closure that returns a future: -```rust! +```rust let c = || -> /* ??? */ async { 0 }; ``` @@ -524,7 +524,7 @@ We could reuse `impl Future` to give users the ability to annotate the type of t This also would have subtle limitations, e.g.: -```rust! +```rust // Easy to reanalyze as an async closure. let _ = || async { do_stuff().await }; @@ -556,7 +556,7 @@ We should be able to detect when users write `|| async {}` -- and subsequently h A similar problem could occur if users try to write "old style" trait bounds with two generic parameters `F: Fn() -> Fut` and `Fut: Future`. For example: -```rust! +```rust async fn for_each_city(cb: F) where F: Fn(&str) -> Fut, From 88faa5bfc0467fb19ada8536a0bfb09d9a2b5ff4 Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Wed, 3 Jul 2024 12:09:47 -0400 Subject: [PATCH 14/25] Recommend for users to write --- text/3668-async-closure.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3668-async-closure.md b/text/3668-async-closure.md index d1b552f463c..2ae52f55eda 100644 --- a/text/3668-async-closure.md +++ b/text/3668-async-closure.md @@ -125,7 +125,7 @@ takes_async_closure(async |i| { }); ``` -We recommend using `async Fn()`/`async FnMut()`/`async FnOnce()` and `async ||` for async closures. This is more flexible than a closure returning a future for the reasons described elsewhere in this RFC. +We recommend for users to write `async Fn()`/`async FnMut()`/`async FnOnce()` and `async ||` for async closures. This is more flexible than a closure returning a future for the reasons described elsewhere in this RFC. Async closures act similarly to closures, and can have parts of their their signatures specified: From 59e0eb228570bb888581f1a989c5704f1885a65d Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Wed, 3 Jul 2024 17:47:30 -0400 Subject: [PATCH 15/25] More writing --- text/3668-async-closure.md | 38 +++++++++++++++++++++++++++++++++++--- 1 file changed, 35 insertions(+), 3 deletions(-) diff --git a/text/3668-async-closure.md b/text/3668-async-closure.md index 2ae52f55eda..ed7139ec193 100644 --- a/text/3668-async-closure.md +++ b/text/3668-async-closure.md @@ -276,7 +276,7 @@ let _: Box = todo!(); All currently-stable callable types (i.e., closures, function items, function pointers, and `dyn Fn*` trait objects) automatically implement `async Fn*() -> T` if they implement `Fn*() -> Fut` for some output type `Fut`, and `Fut` implements `Future`. -This is to make sure that `async Fn*()` trait bounds have maximum compatibility with existing callable types which return futures, such as async function items and closures which return boxed futures. Async closures also implement `async Fn*()`, but their relationship to this trait is detailed later in the RFC. +This is to make sure that `async Fn*()` trait bounds have maximum compatibility with existing callable types which return futures, such as async function items and closures which return boxed futures. Async closures also implement `async Fn*()`, but their relationship to this trait is detailed later in the RFC -- specifically the relationship between the `CallRefFuture` and `CallOnceFuture` associated types. These implementations are built-in, but can conceptually be understood as: @@ -406,7 +406,9 @@ let _ = async move || { #### Specifics about the `AsyncFnOnce` implementation, interaction with `move` -If the closure is inferred to be `async Fn` or `async FnMut`, then the compiler will synthesize an `async FnOnce` implementation for the closure which returns a future that doesn't borrow any captured values from the closure, but instead *moves* the captured values into the future. +If the closure is inferred to be `async Fn` or `async FnMut`, then the compiler will synthesize an `async FnOnce` implementation for the closure which returns a future that doesn't borrow any captured values from the closure, but instead *moves* the captured values into the future. Synthesizing a distinct future that is returned by `async FnOnce` is necessary because the trait *consumes* the closure when it is called (evident from the `self` receiver type in the method signature), meaning that a self-borrowing future would have references to dropped data. This is an interesting problem described in more detail in [compiler-errors' blog post written on async closures][blog post]. + +This is reflected in the fact that `AsyncFnOnce::CallOnceFuture` is a distinct type from `AsyncFnMut::CallRefFuture`. While the latter is a generic-associated-type (GAT) due to supporting self-borrows of the called async closure, the former is not, since it must own all of the captures mentioned in the async closures' body. For example: @@ -432,6 +434,8 @@ fut.await; // point, the allocation for `s` is dropped. ``` +Importantly, although these are distinct futures, they still have the same `Output` type (in other words, their futures await to the same type), and for types that have `async Fn*` implementations, the two future types *execute* identically, since they execute the same future body. They only differ in their captures. + ### Interaction with return-type notation, naming the future returned by calling With `async Fn() -> T` trait bounds, we don't know anything about the `Future` returned by calling the async closure other than that it's a `Future` and awaiting that future returns `T`. @@ -478,7 +482,11 @@ This bound is only valid if there is a corresponding `async Fn*()` trait bound. ### Why do we need a new set of `AsyncFn*` traits? -As demonstrated in the motivation section, we need a set of traits that are *lending* in order to represent futures which borrow from the closure's captures. We technically only need to add `LendingFn` and `LendingFnMut` to our lattice of `Fn*` traits, leaving us with a hierarchy of traits like so: +As demonstrated in the motivation section, we need a set of traits that are *lending* in order to represent futures which borrow from the closure's captures. This is described in more detail in [a blog post written on async closures][blog post]. + +[blog post]: https://hackmd.io/@compiler-errors/async-closures + +We technically only need to add `LendingFn` and `LendingFnMut` to our lattice of `Fn*` traits to support the specifics about async closures' self-borrowing pattern, leaving us with a hierarchy of traits like so: ```mermaid flowchart LR @@ -498,6 +506,16 @@ Fn -- isa --> LendingFn FnMut -- isa --> LendingFnMut ``` +In this case, `async Fn()` would desugar to a `LendingFnMut` trait bound and a `FnOnce` trait bound, like: + +```rust +where F: async Fn() -> i32 + +// is + +where F: for<'s> LendingFn: Future> + FnOnce> +``` + However, there are some concrete technical implementation details that limit our ability to use `LendingFn` ergonomically in the compiler today. These have to do with: - Closure signature inference. @@ -623,3 +641,17 @@ let handlers: HashMap> = todo!(); ``` This work will likely take a similar approach to making `async fn` in traits object-safe, since the major problem is how to "erase" the future returned by the async closure or callable, which differs for each implementation of the trait. + +### Changing the underlying definition to use `LendingFn*` + +As mentioned above, `async Fn*()` trait bounds can be adjusted to desugar to `LendingFn*` + `FnOnce` trait bounds, using associated-type-bounds like: + +```rust +where F: async Fn() -> i32 + +// desugars to + +where F: for<'s> LendingFn: Future> + FnOnce> +``` + +This should be doable in a way that does not affect existing code, but remain blocked on improvements to higher-ranked trait bounds around [GATs](https://blog.rust-lang.org/2022/10/28/gats-stabilization.html#when-gats-go-wrong---a-few-current-bugs-and-limitations). Any changes along these lines remain implementation details unless we decide separately to stabilize more user-observable aspects of the `AsyncFn*` trait, which is not likely to happen soon. From 694d0df4797ca404ce3952aedd4c389f83aeba86 Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Wed, 3 Jul 2024 18:01:37 -0400 Subject: [PATCH 16/25] Why no AsyncFnOnce::Output --- text/3668-async-closure.md | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/text/3668-async-closure.md b/text/3668-async-closure.md index ed7139ec193..6f82c69390c 100644 --- a/text/3668-async-closure.md +++ b/text/3668-async-closure.md @@ -561,6 +561,40 @@ Reusing the `async` keyword allows users to understand what an `async Fn() -> T` We could desugar `async FnOnce() -> T` directly to `FnOnce<(), Output: Future>`. It seems overly complicated for an implementation detail, since users should never care what's *behind* the `AsyncFnOnce` trait bound. +### Why do we recommend the `AsyncFnOnce::Output` type remains unstable, unlike `FnOnce::Output`? + +As mentioned above, `FnOnce::Output` was stabilized in [#34365](https://github.com/rust-lang/rust/pull/34365) as an alternative to break ecosystem code when a bug was fixed to detect usage of unstable associated items in "type-dependent" associated type paths (e.g. `T::Output` that is not qualified with a trait). This allows the following code on stable: + +```rust +fn foo() +where + F: FnOnce() -> T, + F::Output: Send, //~ OK +{ +} +``` + +However, the stabilization of the assoicated type did not actually enable new things to be expressed, and instead `FnOnce::Output` just serves as a type alias for an existing type that may already be named. + +This is because uniquely to `Fn*` trait bounds (compared to the other `std::ops::*` traits that define `Output` associated types, like `Add`), the associated type for `FnOnce::Output` is always constrained by the parenthesized generic syntax. In other words, given `F: Fn*() -> T`, `F::Output` can always be replaced by some type `T`, since `T` is necessary to complete the parenthesized trait bound syntax[^higher]. In that way, naming a type via the `Output` associated type is not more general or flexible than just naming the type itself: + +[^higher]: In fact, the `::Output` syntax doesn't even make it easier to name the return type of a higher-ranked `Fn` trait bound either: https://godbolt.org/z/1rTGhfr9x + +```rust +fn foo() +where + F: FnOnce() -> T, + F::Output: Send, + // Should just be rewritten like: + T: Send, +{ +} +``` + +Exposing the `Output` type for `AsyncFnOnce` complicates eventually moving onto other desugarings for `async Fn*`. For example, if `AsyncFnOnce` is replaced by a trait alias for `FnOnce`, it may change the meaning of `Output` in a way that would require extending the language or adding a hack into the compiler to preserve its meaning. + +Given that expressivity isn't meaningfully impaired by keeping the `Output` associated type as unstable, we do not expect to stabilize this associated type at the same time as async closures, and a stabilization report for the associated type should mention how it affects future possibilities to change the desugaring of `async Fn*`. + # Drawbacks [drawbacks]: #drawbacks From 2e35b78151d165de197ed532615ff660f315f6c7 Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Wed, 10 Jul 2024 11:41:08 -0400 Subject: [PATCH 17/25] Elaborate slightly the section on FnOnce --- text/3668-async-closure.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/text/3668-async-closure.md b/text/3668-async-closure.md index 6f82c69390c..a9c80b1210f 100644 --- a/text/3668-async-closure.md +++ b/text/3668-async-closure.md @@ -404,11 +404,13 @@ let _ = async move || { }; ``` -#### Specifics about the `AsyncFnOnce` implementation, interaction with `move` +#### Specifics about the `AsyncFnOnce` implementation, `CallOnceFuture` vs `CallRefFuture` -If the closure is inferred to be `async Fn` or `async FnMut`, then the compiler will synthesize an `async FnOnce` implementation for the closure which returns a future that doesn't borrow any captured values from the closure, but instead *moves* the captured values into the future. Synthesizing a distinct future that is returned by `async FnOnce` is necessary because the trait *consumes* the closure when it is called (evident from the `self` receiver type in the method signature), meaning that a self-borrowing future would have references to dropped data. This is an interesting problem described in more detail in [compiler-errors' blog post written on async closures][blog post]. +From a user's perspective, it makes sense that if they have an `async FnMut` closure then they should be able to "call it only once" in a way that is uniform with an `async FnOnce` closure. This is because an `FnOnce` is seen as less restrictive to the callee than `FnMut`, and we preserve that distinction with the `async Fn*` trait bounds. -This is reflected in the fact that `AsyncFnOnce::CallOnceFuture` is a distinct type from `AsyncFnMut::CallRefFuture`. While the latter is a generic-associated-type (GAT) due to supporting self-borrows of the called async closure, the former is not, since it must own all of the captures mentioned in the async closures' body. +If the closure is inferred to be `async Fn` or `async FnMut`, then the compiler needs to synthesize an `async FnOnce` implementation for the closure which returns a future that doesn't borrow any captured values from the closure, but instead *moves* those captured values into the future. Synthesizing a distinct future that is returned by `async FnOnce` is necessary because the trait *consumes* the closure when it is called (evident from the `self` receiver type in the method signature), meaning that a self-borrowing future would have references to dropped data. This is an interesting problem described in more detail in [compiler-errors' blog post written on async closures][blog post]. + +This is reflected in the unstable trait implementations by the fact that `AsyncFnOnce::CallOnceFuture` is a distinct type from `AsyncFnMut::CallRefFuture`. While the latter is a generic-associated-type (GAT) due to supporting self-borrows of the called async closure, the former is not, since it must own all of the captures mentioned in the async closures' body. For example: @@ -434,7 +436,7 @@ fut.await; // point, the allocation for `s` is dropped. ``` -Importantly, although these are distinct futures, they still have the same `Output` type (in other words, their futures await to the same type), and for types that have `async Fn*` implementations, the two future types *execute* identically, since they execute the same future body. They only differ in their captures. +For the purposes of the compiler implementation, although these are distinct futures, they still have the same `Output` type (in other words, their futures await to the same type), and for types that have `async Fn*` implementations, the two future types *execute* identically, since they execute the same future body. They only differ in their captures. Given that users usually do not care about the concrete future type itself, but only its final output type, and that both futures are fully anonymous, the fact that a different future is used when calling an `async FnMut` via `async_call_mut` vs `async_call_once` are not noticeable except for pathological examples. ### Interaction with return-type notation, naming the future returned by calling From 662654b36a129107f23da079a356a7dc2067bdc9 Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Wed, 10 Jul 2024 12:03:53 -0400 Subject: [PATCH 18/25] Slightly elaborate drawback with non-nameable RTN --- text/3668-async-closure.md | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/text/3668-async-closure.md b/text/3668-async-closure.md index a9c80b1210f..55c20aad984 100644 --- a/text/3668-async-closure.md +++ b/text/3668-async-closure.md @@ -633,7 +633,27 @@ We can similarly implement a lint to detect cases where users write these two-pa It's not possible to directly name the future returned by calling some generic `T: async Fn()`. This means that it's not possible, for example, to convert `futures-rs`'s [`StreamExt::then` combinator](https://docs.rs/futures/0.3.30/futures/stream/trait.StreamExt.html#method.then), since the output future is referenced in the definition of [`Then`](https://docs.rs/futures-util/0.3.30/src/futures_util/stream/stream/then.rs.html#19) returned by the combinator. -Fixing this is a follow-up goal that we're interested in pursuing in the near future. +For example, consider a `Then` combinator that allows mapping a stream under a future: + +```rust +pub struct Then +where + St: Stream, + F: async FnMut(St::Item) -> Fut::Output, + Fut: Future, +{ + stream: St, + fun: F, + future: Option, + +} +``` + +The first problem here is that the RTN [RFC 3654](https://github.com/rust-lang/rfcs/pull/3654) says that RTN is only allowed in *trait bound* positions, so we can't use it to name the returned future in type position, like in this struct field, without further design work. + +Secondly, even if we could name the `CallRefFuture` type directly, we still need a lifetime to plug into the GAT. Conceptually, the future lives for the transient period of processing a single element in the stream, which isn't representable with a lifetime argument. We would need some sort of `'unsafe` or unsafe binder type. + +Fixing this is a follow-up goal that we're interested in pursuing in the near future. Design work regarding naming the future types in struct position can be done additively on top of what is exposed in this RFC, and ties into the larger question of how to use RTN in struct fields and other non-inference type positions. # Prior art [prior-art]: #prior-art From ee9b0cd5fde30034ed0b055b3c29c098e586aa4f Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Wed, 10 Jul 2024 23:52:45 -0400 Subject: [PATCH 19/25] Update text/3668-async-closure.md whoops forgot a mut Co-authored-by: zachs18 <8355914+zachs18@users.noreply.github.com> --- text/3668-async-closure.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3668-async-closure.md b/text/3668-async-closure.md index 55c20aad984..d885aa3af0c 100644 --- a/text/3668-async-closure.md +++ b/text/3668-async-closure.md @@ -357,7 +357,7 @@ let closure = async || { }; ``` -The closure captures `vec` with some `&'closure mut Vec` which lives until the closure is dropped. Then every call to the closure reborrows that mutable reference `&'call Vec` which lives until the future is dropped (e.g. `await`ed). +The closure captures `vec` with some `&'closure mut Vec` which lives until the closure is dropped. Then every call to the closure reborrows that mutable reference `&'call mut Vec` which lives until the future is dropped (e.g. `await`ed). As another example: From 3681ffd3e9631c1cabedfd4c46b6f380aa681afb Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Tue, 16 Jul 2024 18:58:06 -0400 Subject: [PATCH 20/25] more nits --- text/3668-async-closure.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/text/3668-async-closure.md b/text/3668-async-closure.md index d885aa3af0c..f4a38228909 100644 --- a/text/3668-async-closure.md +++ b/text/3668-async-closure.md @@ -138,6 +138,10 @@ let ret = async || -> Vec { async_iterator.collect().await }; // They can be higher-ranked: let hr = async |x: &str| { do_something(x).await }; + +// They can capture values by move: +let s = String::from("hello, world"); +let c = async || do_something(&s).await }; ``` When called, they return an anonymous future type corresponding to the (not-yet-executed) body of the closure. These can be awaited like any other future. @@ -673,6 +677,12 @@ where T: for<'a> async ?Trait (Which is semantically invalid but syntactically valid.) This is currently proposed in [rust-lang/rust#127054](https://github.com/rust-lang/rust/pull/127054), which should be decided before stabilization, and the stabilization report can re-confirm the correct ordering of `for<'a>` and `async`. +### Where exactly is `async || {}` not backwards with `|| async {}` + +The stabilization report for async closures should thoroughly note any cases where rewriting `|| async {}` into `async || {}` causes errors, as they will be pitfalls for adoption of async closures. + +One predicted shortcoming will likely be due to corner cases of closure signature inference and pre-async-closure trait bounds in a [previous section](#users-might-write-f-fn---fut-fut-futureoutput--t-over-f-async-fn---t). This is not necessarily a blocker, since as the ecosystem migrates to `async Fn()`-style trait bounds, closure signature inference will be restored. + # Future possibilities [future-possibilities]: #future-possibilities From f8c6e9f524bc68d77837f33e10cc2e1c7eace587 Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Tue, 16 Jul 2024 18:59:59 -0400 Subject: [PATCH 21/25] whoops --- text/3668-async-closure.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3668-async-closure.md b/text/3668-async-closure.md index f4a38228909..3b173a004c5 100644 --- a/text/3668-async-closure.md +++ b/text/3668-async-closure.md @@ -141,7 +141,7 @@ let hr = async |x: &str| { do_something(x).await }; // They can capture values by move: let s = String::from("hello, world"); -let c = async || do_something(&s).await }; +let c = async move || do_something(&s).await }; ``` When called, they return an anonymous future type corresponding to the (not-yet-executed) body of the closure. These can be awaited like any other future. From 186ce2086d1df87e7dd17c83db57bbfe01ce5514 Mon Sep 17 00:00:00 2001 From: Travis Cross Date: Sat, 20 Jul 2024 05:27:11 +0000 Subject: [PATCH 22/25] Rename file for RFC 3668 to plural The feature gate for async closures is unfortunately in the singular, `async_closure`, and while that's not a stable commitment, it's probably too much of a pain to change that. But that doesn't mean that we can't name the file in the plural, so let's do that. --- text/{3668-async-closure.md => 3668-async-closures.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename text/{3668-async-closure.md => 3668-async-closures.md} (100%) diff --git a/text/3668-async-closure.md b/text/3668-async-closures.md similarity index 100% rename from text/3668-async-closure.md rename to text/3668-async-closures.md From cb803f0ee99267ed640a4435e54659475738ce19 Mon Sep 17 00:00:00 2001 From: Travis Cross Date: Sat, 20 Jul 2024 05:45:01 +0000 Subject: [PATCH 23/25] Add section about how `AsyncFn` commits us also As it turns out, `AsyncFn()` probably commits our direction on generalized `async` trait bound modifiers to the same degree as `async Fn()`. Let's describe this. CE brought up this excellent point. --- text/3668-async-closures.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/text/3668-async-closures.md b/text/3668-async-closures.md index 3b173a004c5..4032468497f 100644 --- a/text/3668-async-closures.md +++ b/text/3668-async-closures.md @@ -563,6 +563,14 @@ let _ = || { Reusing the `async` keyword allows users to understand what an `async Fn() -> T` trait bound does by analogy, since they already should know that adding `async` to some `fn foo() -> T` makes it return an `impl Future` instead of the type `T`. +### Wouldn't `F: AsyncFn() -> T` save more space for `async` trait bound modifiers? + +Some have argued that using `F: AsyncFn() -> T` rather than `F: async Fn() -> T` would better save space for whatever we might want the semantics to be if we were to later adopt generalized `async` trait bound modifiers. + +We don't think this is correct in a practical sense. If we were give meaning to and stabilize `AsyncFn()` trait bounds, and then later, due to work on generalized `async` trait bound modifiers, we were to give a different meaning to `async Fn()`, that would seem so undesirable that it's hard to imagine we would ever actually do it. People would too strongly expect `AsyncFn()` and `async Fn()` to work in the same way. + +If that's true, then stabilizing either of `AsyncFn()` or `async Fn()` trait bounds would potentially affect our decisions about generalized `async` trait bound modifiers to the same degree. + ### Why do we even need `AsyncFnOnce`? We could desugar `async FnOnce() -> T` directly to `FnOnce<(), Output: Future>`. It seems overly complicated for an implementation detail, since users should never care what's *behind* the `AsyncFnOnce` trait bound. From 86ad4844b9eb7b9e700a9ea105ef22395e1bf7e4 Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Tue, 23 Jul 2024 14:06:53 -0400 Subject: [PATCH 24/25] Reserve some space for async Fn bounds not converging on consensus --- text/3668-async-closures.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/text/3668-async-closures.md b/text/3668-async-closures.md index 4032468497f..451e4056bba 100644 --- a/text/3668-async-closures.md +++ b/text/3668-async-closures.md @@ -10,7 +10,7 @@ # Summary [summary]: #summary -This RFC adds an `async` bound modifier to the `Fn` family of trait bounds. The combination desugars to a set of unstable `AsyncFn{,Mut,Once}` traits that parallel the current `Fn{,Mut,Once}` traits. +This RFC adds an `async` bound modifier to the `Fn` family of trait bounds. The combination currently desugars to a set of unstable `AsyncFn{,Mut,Once}` traits that parallel the current `Fn{,Mut,Once}` traits. These traits give users the ability to express bounds for async callable types that are higher-ranked, and allow async closures to return futures which borrow from the closure's captures. @@ -170,7 +170,7 @@ trait Gat { ### `AsyncFn*` -This RFC introduces a family of `AsyncFn` traits. These traits are intended to remain unstable to name or implement, just like the `Fn` traits. Nonetheless, we'll describe the details of these traits so as to explain the user-facing features enabled by them. +This RFC begins by introducing a family of `AsyncFn` traits for the purposes of demonstrating the lending behavior of async closures. These traits are intended to remain unstable to name or implement, just like the `Fn` traits. Nonetheless, we'll describe the details of these traits so as to explain the user-facing features enabled by them. The definition of the traits is (modulo `rustc_` attributes, and the `"rust-call"` ABI): @@ -248,6 +248,8 @@ where ### `async` bound modifier on `Fn()` trait bounds +(**note**: See the [naming blocking concern](#what-do-we-call-the-trait), which reflects that this remains an open question. Repeating the blocking concern: within this RFC, we generally name the user-facing semantics of async trait bounds as `async Fn*`, and we use the name `AsyncFn*` for the internal details of the trait implementation for the purpose of demonstrating the lending behavior.) + The `AsyncFn*` traits specified above are nameable via a new `async` bound modifier that is allowed on `Fn` trait bounds. That is, `async Fn*() -> T` desugars to `AsyncFn*() -> T` in bounds, where `Fn*` is one of the three flavors of existing function traits: `Fn`/`FnMut`/`FnOnce`. This RFC specifies the modification to the _TraitBound_ nonterminal in the grammar: @@ -561,15 +563,9 @@ let _ = || { ### Why not `F: AsyncFn() -> T`, naming `AsyncFn*` directly? -Reusing the `async` keyword allows users to understand what an `async Fn() -> T` trait bound does by analogy, since they already should know that adding `async` to some `fn foo() -> T` makes it return an `impl Future` instead of the type `T`. - -### Wouldn't `F: AsyncFn() -> T` save more space for `async` trait bound modifiers? - -Some have argued that using `F: AsyncFn() -> T` rather than `F: async Fn() -> T` would better save space for whatever we might want the semantics to be if we were to later adopt generalized `async` trait bound modifiers. +(**note**: See the [naming blocking concern](#what-do-we-call-the-trait), which reflects that this remains an open question.) -We don't think this is correct in a practical sense. If we were give meaning to and stabilize `AsyncFn()` trait bounds, and then later, due to work on generalized `async` trait bound modifiers, we were to give a different meaning to `async Fn()`, that would seem so undesirable that it's hard to imagine we would ever actually do it. People would too strongly expect `AsyncFn()` and `async Fn()` to work in the same way. - -If that's true, then stabilizing either of `AsyncFn()` or `async Fn()` trait bounds would potentially affect our decisions about generalized `async` trait bound modifiers to the same degree. +Reusing the `async` keyword allows users to understand what an `async Fn() -> T` trait bound does by analogy, since they already should know that adding `async` to some `fn foo() -> T` makes it return an `impl Future` instead of the type `T`. ### Why do we even need `AsyncFnOnce`? @@ -675,6 +671,10 @@ Fixing this is a follow-up goal that we're interested in pursuing in the near fu # Unresolved questions [unresolved-questions]: #unresolved-questions +### What do we call the trait? + +There is some discussion about whether to call the bound `T: AsyncFn()` or `T: async Fn()`. As stated above, there is not full consensus about whether `async Fn()` is the syntax we want to commit to name these bounds, but for the purposes of decoupling the fact that `async Fn` is the user-observable trait family, and `AsyncFn` is the traits of the implementation detail, this RFC names them separately. + ### `? for<'a>` and its interaction with `async` Currently on nightly, we parse the `async` trait bound modifier along with `?` (called polarity) *before* the `for<'a>` lifetime binders. This probably should get fixed so that the binder occurs on the *outside* of the trait, like so: From 576b9f41ecb683a8478733a6aef5841ccc1b46e8 Mon Sep 17 00:00:00 2001 From: Michael Goulet Date: Tue, 23 Jul 2024 18:43:21 -0400 Subject: [PATCH 25/25] Update text/3668-async-closures.md add the note in one other place Co-authored-by: Josh Triplett --- text/3668-async-closures.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/3668-async-closures.md b/text/3668-async-closures.md index 451e4056bba..8ebfec41c34 100644 --- a/text/3668-async-closures.md +++ b/text/3668-async-closures.md @@ -111,6 +111,8 @@ In order for this to work, the `FnMut` trait would need to be ["lending"](https: # Guide Level Explanation +(**note**: See the [naming blocking concern](#what-do-we-call-the-trait) about `async Fn*` vs `AsyncFn*` syntax. This RFC uses the `async Fn` syntax for trait bounds to avoid duplicating explanations for two different proposed syntaxes, but the syntax remains an open question.) + Just as you can write functions which accept closures, you can write functions which accept async closures: ```rust