Async Closures With Borrowed Arguments

[cramertj]

Async closures with borrowed arguments are a messy thing. Since the output type of the closure (impl Future + 'a) depends upon the input lifetime of the argument, these closures are always higher-ranked. Eventually I think we'd like to support this through syntax like this:

fn higher_order_fn(
    async_fn: impl for<'a> FnOnce(&'a Foo) -> impl Future<Output = ()> + 'a`
) { ... }

// and someday

fn higher_order_fn(
    async_fn: impl async FnOnce(&Foo),
) { ... }

Unfortunately, the current impl Trait-in-args implementation doesn't support these sorts of higher-ranked use cases. Luckily it's not backwards incompatible to add support for them, since we've currently banned impl Trait inside of Fn syntax.

Until we get one of these nicer solutions, however, we need a way to type these sorts of functions. I made an attempt at one of them that looks like this:

pub trait PinnedFnLt<'a, Data: 'a, Output> {
    type Future: Future<Output = Output> + 'a;
    fn apply(self, data: PinMut<'a, Data>) -> Self::Future;
}

pub trait PinnedFn<Data, Output>: for<'a> PinnedFnLt<'a, Data, Output> + 'static {}
impl<Data, Output, T> PinnedFn<Data, Output> for T
    where T: for<'a> PinnedFnLt<'a, Data, Output> + 'static {}

impl<'a, Data, Output, Fut, T> PinnedFnLt<'a, Data, Output> for T
where
    Data: 'a,
    T: FnOnce(PinMut<'a, Data>) -> Fut,
    Fut: Future<Output = Output> + 'a,
{
    type Future = Fut;
    fn apply(self, data: PinMut<'a, Data>) -> Self::Future {
        (self)(data)
    }
}

pub fn pinned<Data, Output, F>(data: Data, f: F) -> PinnedFut<Data, Output, F>
    where F: PinnedFn<Data, Output>,
{ ... }

Unfortunately, this fails because of rust-lang/rust#51004. Even if this worked, though, this is an awful lot of code to write for each of these functions-- we'd likely want some sort of macro.

Until progress is made on this issue, it's impossible to make a function which accepts an asynchronous closure with an argument borrowed for a non-fixed lifetime (precise named lifetimes like impl FnOnce(&'tcx Foo) -> F are possible).

[withoutboats]

Just to tease out the different threads of this, let me know if anything about this isn't correct:

1. impl Fn(&u8) doesn't do the HRTB elision that F: Fn(&u8) does today.
2. In general, writing the signature of an async function is a pain, made worse by the first point. It would be nice to have some sugar for the signature of an async function, just as we have sugar for declaring one.
3. You have a handwritten solution, which also doesn't work for type system reason (and which you'd want to macro generate).

Am I misunderstanding any of this?

It seems like we should solve 1 promptly (what blockers are there?) and 2 less promptly (since it involves design work). Pursuing the very complicated trait wrapper thing too far seems like a misfocus to me.

[withoutboats]

Nevermind, I think I have a better understanding of the issue from investigating this comment:

Until progress is made on this issue, it's impossible to make a function which accepts an asynchronous closure with an argument borrowed for a non-fixed lifetime

The problem seems to be mainly that you can't write F: FnOnce() -> impl Future, and there's no other way to tie the lifetimes together than to do that, because you want to write for<'a> F: FnOnce() -> Fut, Fut: 'a

[Nemo157]

Is there any reason fully expanding the lifetime and closure return type out to generic type parameters doesn't work?

fn higher_order_fn<'a, T, F>(async_fn: F)
where
    T: Future<Output = ()> + 'a,
    F: FnOnce(&'a Foo) -> T,

This is how I was previously assuming higher order anonymous universal types would be desugared.

[cramertj]

@Nemo157 That gives you a function which can borrow a particular 'a, not an arbitrary 'a. You'd then have to manufacture a reference with that particular lifetime, which is impossible in many cases (often the borrowed value you're passing in would be of an anonymous lifetime defined by a scope of a particular function or lifetime of a particular borrow of a value-- see rust-lang/futures-rs#1023 for an example).

[qnighy]

Coming from this URLO thread. I propose this workaround:

#![feature(async_await)]

use std::future::Future;

macro_rules! impl_async_fn {
    ($(($FnOnce:ident, $FnMut:ident, $Fn:ident, ($($arg:ident: $arg_ty:ident,)*)),)*) => {
        $(
            pub trait $FnOnce<$($arg_ty,)*> {
                type Output;
                type Future: Future<Output = Self::Output> + Send;
                fn call_once(self, $($arg: $arg_ty,)*) -> Self::Future;
            }
            pub trait $FnMut<$($arg_ty,)*>: $FnOnce<$($arg_ty,)*> {
                fn call_mut(&mut self, $($arg: $arg_ty,)*) -> Self::Future;
            }
            pub trait $Fn<$($arg_ty,)*>: $FnMut<$($arg_ty,)*> {
                fn call(&self, $($arg: $arg_ty,)*) -> Self::Future;
            }
            impl<$($arg_ty,)* F, Fut> $FnOnce<$($arg_ty,)*> for F
            where
                F: FnOnce($($arg_ty,)*) -> Fut,
                Fut: Future + Send,
            {
                type Output = Fut::Output;
                type Future = Fut;
                fn call_once(self, $($arg: $arg_ty,)*) -> Self::Future {
                    self($($arg,)*)
                }
            }
            impl<$($arg_ty,)* F, Fut> $FnMut<$($arg_ty,)*> for F
            where
                F: FnMut($($arg_ty,)*) -> Fut,
                Fut: Future + Send,
            {
                fn call_mut(&mut self, $($arg: $arg_ty,)*) -> Self::Future {
                    self($($arg,)*)
                }
            }
            impl<$($arg_ty,)* F, Fut> $Fn<$($arg_ty,)*> for F
            where
                F: Fn($($arg_ty,)*) -> Fut,
                Fut: Future + Send,
            {
                fn call(&self, $($arg: $arg_ty,)*) -> Self::Future {
                    self($($arg,)*)
                }
            }
        )*
    }
}
impl_async_fn! {
    (AsyncFnOnce0, AsyncFnMut0, AsyncFn0, ()),
    (AsyncFnOnce1, AsyncFnMut1, AsyncFn1, (a0:A0, )),
    (AsyncFnOnce2, AsyncFnMut2, AsyncFn2, (a0:A0, a1:A1, )),
    (AsyncFnOnce3, AsyncFnMut3, AsyncFn3, (a0:A0, a1:A1, a2:A2, )),
    (AsyncFnOnce4, AsyncFnMut4, AsyncFn4, (a0:A0, a1:A1, a2:A2, a3:A3, )),
    (AsyncFnOnce5, AsyncFnMut5, AsyncFn5, (a0:A0, a1:A1, a2:A2, a3:A3, a4:A4, )),
    (AsyncFnOnce6, AsyncFnMut6, AsyncFn6, (a0:A0, a1:A1, a2:A2, a3:A3, a4:A4, a5:A5, )),
    (AsyncFnOnce7, AsyncFnMut7, AsyncFn7, (a0:A0, a1:A1, a2:A2, a3:A3, a4:A4, a5:A5, a6:A6, )),
    (AsyncFnOnce8, AsyncFnMut8, AsyncFn8, (a0:A0, a1:A1, a2:A2, a3:A3, a4:A4, a5:A5, a6:A6, a7:A7, )),
    (AsyncFnOnce9, AsyncFnMut9, AsyncFn9, (a0:A0, a1:A1, a2:A2, a3:A3, a4:A4, a5:A5, a6:A6, a7:A7, a8:A8, )),
    (AsyncFnOnce10, AsyncFnMut10, AsyncFn10, (a0:A0, a1:A1, a2:A2, a3:A3, a4:A4, a5:A5, a6:A6, a7:A7, a8:A8, a9:A9, )),
    (AsyncFnOnce11, AsyncFnMut11, AsyncFn11, (a0:A0, a1:A1, a2:A2, a3:A3, a4:A4, a5:A5, a6:A6, a7:A7, a8:A8, a9:A9, a10:A10, )),
    (AsyncFnOnce12, AsyncFnMut12, AsyncFn12, (a0:A0, a1:A1, a2:A2, a3:A3, a4:A4, a5:A5, a6:A6, a7:A7, a8:A8, a9:A9, a10:A10, a11:A11, )),
}

pub async fn call<F>(f: F) -> i32
where
    F: for<'a> AsyncFn1<&'a i32, Output = i32>,
{
    f.call_once(&42).await
}

// #[runtime::test]
async fn test_call() {
    async fn double(x: &i32) -> i32 { *x * 2 }
    assert_eq!(call(double).await, 84);
}

The AsyncFn here philosophically inherits Fn in this way:

pub trait AsyncFnOnce<A>: FnOnce<A, Output = Self::Future> {
    type Output;
    type Future: Future<Output = Self::Output>;
}
impl<F, A> AsyncFnOnce<A> for F
where
    F: FnOnce<A>,
    <F as FnOnce<A>>::Output: Future,
{
    type Output = <<F as FnOnce<A>>::Output as Future>::Output,
    type Future = <F as FnOnce<A>>::Output;
}

But this is rejected due to recursive trait inheritance. Instead, we can say Fn implies AsyncFn by blanket impls but not the other way around (by lack of trait inheritance). Also, we can split them by the number of arguments to bring this trait in stable Rust.

The callee has to write f.call3(a, b, c) instead of f(a, b, c). For the caller, however, this looks like a usual higher-order async fn which receives async fn or async closure.