Problem with GATs, async, and Send-bounds

[JanBeh]

I have experienced problems with Send bounds on GATs. The original code where the problem occurred uses the async-trait crate and looks as follows:

#![feature(generic_associated_types)]

use async_trait::async_trait;
use std::ops::Deref;

async fn some_async_task() {}

#[async_trait]
trait Source {
    type T;
    type Wrapper<'a>: Deref<Target = Self::T> + Send
    where
        Self: 'a;
    async fn retrieve(&mut self) -> Self::Wrapper<'_>;
}

struct S {
    state: i32,
}

impl S {
    fn new() -> Self {
        S { state: 0 }
    }
}

#[async_trait]
impl Source for S {
    type T = i32;
    type Wrapper<'a> = &'a Self::T;
    async fn retrieve(&mut self) -> Self::Wrapper<'_> {
        self.state += 1;
        &self.state
    }
}

#[async_trait]
trait User {
    // After updating nightly rustc, this won't work anymore:
    async fn process<'a, 'b, S>(
        &'a self,
        source: &'b mut S,
    ) -> S::Wrapper<'b>
    where
        S: Source + Send,
    {
        let result = source.retrieve().await;
        some_async_task().await;
        result
    }
}

struct U {}

#[async_trait]
impl User for U {}

#[tokio::main]
async fn main() {
    let mut s = S::new();
    let u = U {};
    let value: &i32 = u.process(&mut s).await;
    println!("Result = {}", value);
}

Using the Rust playground with Nightly version: 1.58.0-nightly (2021-11-07 46b8e74), I get:

   Compiling playground v0.0.1 (/playground)
error[E0311]: the parameter type `S` may not live long enough
  --> src/main.rs:46:5
   |
40 |       async fn process<'a, 'b, S>(
   |                                - help: consider adding an explicit lifetime bound...: `S: 'c`
...
46 | /     {
47 | |         let result = source.retrieve().await;
48 | |         some_async_task().await;
49 | |         result
50 | |     }
   | |_____^ ...so that the type `S` will meet its required lifetime bounds

error: could not compile `playground` due to previous error

If I follow the compiler's advice literally, then of course I'll get:

error[E0261]: use of undeclared lifetime name `'c`

If I declare 'c as a lifetime parameter, then the compiler demands I should add a bound S: 'd, and so on.

This code worked with some earlier version of nightly Rust (without the where Self: 'a bound, and before where Self: 'a was required in the GAT). Unfortunately I do not remember which version that was (a few weeks ago).

However, even before the update of rustc, I had difficulties to remove the Send bound from the GATs definition and to include a bound in another method that uses the GATs. This problem still persists and can be demonstrated with the following code:

#![feature(generic_associated_types)]

use async_trait::async_trait;
use std::ops::Deref;

async fn some_async_task() {}

#[async_trait]
trait Source {
    type T;
    // I removed the `Send` bound here:
    type Wrapper<'a>: Deref<Target = Self::T>
    where
        Self: 'a;
    async fn retrieve(&mut self) -> Self::Wrapper<'_>;
}

struct S {
    state: i32,
}

impl S {
    fn new() -> Self {
        S { state: 0 }
    }
}

#[async_trait]
impl Source for S {
    type T = i32;
    type Wrapper<'a> = &'a Self::T;
    async fn retrieve(&mut self) -> Self::Wrapper<'_> {
        self.state += 1;
        &self.state
    }
}

#[async_trait]
trait User {
    async fn process<'a, 'b, S>(
        &'a self,
        source: &'b mut S,
    ) -> S::Wrapper<'b>
    where
        S: Source + Send + 'static,
        // And added it here:
        <S as Source>::Wrapper<'b>: Send,
    {
        let result = source.retrieve().await;
        some_async_task().await;
        result
    }
}

struct U {}

#[async_trait]
impl User for U {}

#[tokio::main]
async fn main() {
    let mut s = S::new();
    let u = U {};
    let value: &i32 = u.process(&mut s).await;
    println!("Result = {}", value);
}

Using again Rust playground with Nightly version: 1.58.0-nightly (2021-11-07 46b8e74), I get:

   Compiling playground v0.0.1 (/playground)
error: implementation of `Send` is not general enough
  --> src/main.rs:48:5
   |
48 | /     {
49 | |         let result = source.retrieve().await;
50 | |         some_async_task().await;
51 | |         result
52 | |     }
   | |_____^ implementation of `Send` is not general enough
   |
   = note: `<S as Source>::Wrapper<'0>` must implement `Send`, for any lifetime `'0`...
   = note: ...but `Send` is actually implemented for the type `<S as Source>::Wrapper<'b>`

error: could not compile `playground` due to previous error

If I try to use a HRTB (for<'z> <S as Source>::Wrapper<'z>: Send), then I get:

   Compiling playground v0.0.1 (/playground)
error[E0277]: `<_ as Source>::Wrapper<'z>` cannot be sent between threads safely
  --> src/main.rs:64:25
   |
64 |     let value: &i32 = u.process(&mut s).await;
   |                         ^^^^^^^ `<_ as Source>::Wrapper<'z>` cannot be sent between threads safely
   |
   = help: the trait `for<'z> Send` is not implemented for `<_ as Source>::Wrapper<'z>`

For more information about this error, try `rustc --explain E0277`.
error: could not compile `playground` due to previous error

I'm not sure if these problems are related, but there seem to be a link. There has been a discussion on the Rust users forum where another smaller example (without async-trait macros) was created:

#![feature(generic_associated_types)]
use std::{future::Future, marker::PhantomData};

trait Trait {
    type Associated<'a>: Send
    where
        Self: 'a;
}

fn future<'a, S: Trait + 'a, F>(f: F) -> F
where
    F: Future<Output = ()> + Send,
{
    f
}

fn foo<'a, S: Trait + 'a>() {
    future::<'a, S, _>(async move {
        let result: PhantomData<S::Associated<'a>> = PhantomData;
        async {}.await;
    });
}

This results (using the same compiler version) in:

   Compiling playground v0.0.1 (/playground)
error[E0311]: the parameter type `S` may not live long enough
  --> src/lib.rs:18:5
   |
17 | fn foo<'a, S: Trait + 'a>() {
   |            -- help: consider adding an explicit lifetime bound...: `S: 'b +`
18 |     future::<'a, S, _>(async move {
   |     ^^^^^^^^^^^^^^^^^^ ...so that the type `S` will meet its required lifetime bounds...
   |
note: ...that is required by this bound
  --> src/lib.rs:12:30
   |
12 |     F: Future<Output = ()> + Send,
   |                              ^^^^

error: could not compile `playground` due to previous error

If I replace Send with SomeTrait with impl<T> SomeTrait for T {}, the code will compile. We guessed the problem might be auto-trait related?

See also:

• The previously mentioned discussion on the Rust users forum
• Implementation of GATs outlives lint #89970
• GATs: Decide whether to have defaults for where Self: 'a #87479

[JanBeh]

@rustbot label F-generic_associated_types

[steffahn]

@rustbot label A-lifetimes, A-associated-items, A-traits, A-async-await, T-compiler, requires-nightly

[eholk]

This looks like a tricky bug and will take some time to look into. There were some recent closure changes that might be causing this, and this sounds potentially related to #71723, #71671, and #87425.

Bisection would let us narrow down the change that caused the problem.

@rustbot label E-needs-bisection E-needs-mcve AsyncAwait-Triaged

[eholk]

Also potentially related: #64552, #60658.

[eholk]

Also: #70263

[steffahn]

The issue already contains a minimal example,

#![feature(generic_associated_types)]
use std::{future::Future, marker::PhantomData};

trait Trait {
    type Associated<'a>: Send
    where
        Self: 'a;
}

fn future<'a, S: Trait + 'a, F>(f: F) -> F
where
    F: Future<Output = ()> + Send,
{
    f
}

fn foo<'a, S: Trait + 'a>() {
    future::<'a, S, _>(async move {
        let result: PhantomData<S::Associated<'a>> = PhantomData;
        async {}.await;
    });
}

at least for one of the (potentially multiple?) issues described here.

---

There were some recent closure changes that might be causing this

I didn’t consider checking whether this is a regression yet. It's not necessarily related to closures though; it’s an async block, and an async fn would do the same thing.

#![feature(generic_associated_types)]
use std::{future::Future, marker::PhantomData};

trait Trait {
    type Associated<'a>: Send
    where
        Self: 'a;
}

fn future<'a, S: Trait + 'a, F>(f: F) -> F
where
    F: Future<Output = ()> + Send,
{
    f
}

async fn f<'a, S: Trait + 'a>() {
    let result: PhantomData<S::Associated<'a>> = PhantomData;
    async {}.await;
} 

fn foo<'a, S: Trait + 'a>() {
    future::<'a, S, _>(f::<'a, S>());
}

---

Let me do some bisecting... okay, so it is a regression! (But not a very recent one.)

nightly-2020-10-07

warning: the feature `generic_associated_types` is incomplete and may not be safe to use and/or cause compiler crashes
 --> src/main.rs:1:12
  |
1 | #![feature(generic_associated_types)]
  |            ^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(incomplete_features)]` on by default
  = note: see issue #44265 <https://github.com/rust-lang/rust/issues/44265> for more information

error[E0311]: the parameter type `S` may not live long enough
  --> src/main.rs:23:5
   |
22 | fn foo<'a, S: Trait + 'a>() {
   |            -- help: consider adding an explicit lifetime bound...: `S: 'b +`
23 |     future::<'a, S, _>(f::<'a, S>());
   |     ^^^^^^^^^^^^^^^^^^
   |
   = note: the parameter type `S` must be valid for any other region...
   = note: ...so that the type `S` will meet its required lifetime bounds

error: aborting due to previous error; 1 warning emitted

nightly-2020-10-06

warning: the feature `generic_associated_types` is incomplete and may not be safe to use and/or cause compiler crashes
 --> src/main.rs:1:12
  |
1 | #![feature(generic_associated_types)]
  |            ^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(incomplete_features)]` on by default
  = note: see issue #44265 <https://github.com/rust-lang/rust/issues/44265> for more information

warning: unused variable: `result`
  --> src/main.rs:18:9
   |
18 |     let result: PhantomData<S::Associated<'a>> = PhantomData;
   |         ^^^^^^ help: if this is intentional, prefix it with an underscore: `_result`
   |
   = note: `#[warn(unused_variables)]` on by default

warning: function is never used: `future`
  --> src/main.rs:10:4
   |
10 | fn future<'a, S: Trait + 'a, F>(f: F) -> F
   |    ^^^^^^
   |
   = note: `#[warn(dead_code)]` on by default

warning: function is never used: `f`
  --> src/main.rs:17:10
   |
17 | async fn f<'a, S: Trait + 'a>() {
   |          ^

warning: function is never used: `foo`
  --> src/main.rs:22:4
   |
22 | fn foo<'a, S: Trait + 'a>() {
   |    ^^^

warning: unused implementer of `Future` that must be used
  --> src/main.rs:23:5
   |
23 |     future::<'a, S, _>(f::<'a, S>());
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(unused_must_use)]` on by default
   = note: futures do nothing unless you `.await` or poll them

warning: 6 warnings emitted

    Finished dev [unoptimized + debuginfo] target(s) in 0.30s

********************************************************************************
Regression in nightly-2020-10-07
********************************************************************************

fetching https://static.rust-lang.org/dist/2020-10-06/channel-rust-nightly-git-commit-hash.txt
nightly manifest 2020-10-06: 40 B / 40 B [====================================================================] 100.00 % 746.12 KB/s converted 2020-10-06 to a1dfd2490a6cb456b92e469fa550dc217e20ad6d
fetching https://static.rust-lang.org/dist/2020-10-07/channel-rust-nightly-git-commit-hash.txt
nightly manifest 2020-10-07: 40 B / 40 B [====================================================================] 100.00 % 724.12 KB/s converted 2020-10-07 to 98edd1fbf8a68977a2a7c1312eb1ebff80515a92
looking for regression commit between 2020-10-06 and 2020-10-07
fetching (via remote github) commits from max(a1dfd2490a6cb456b92e469fa550dc217e20ad6d, 2020-10-04) to 98edd1fbf8a68977a2a7c1312eb1ebff80515a92
ending github query because we found starting sha: a1dfd2490a6cb456b92e469fa550dc217e20ad6d
get_commits_between returning commits, len: 6
  commit[0] 2020-10-05UTC: Auto merge of #77080 - richkadel:llvm-coverage-counters-2, r=tmandry
  commit[1] 2020-10-06UTC: Auto merge of #77606 - JohnTitor:rollup-7rgahdt, r=JohnTitor
  commit[2] 2020-10-06UTC: Auto merge of #77594 - timvermeulen:chain_advance_by, r=scottmcm
  commit[3] 2020-10-06UTC: Auto merge of #73905 - matthewjasper:projection-bounds-2, r=nikomatsakis
  commit[4] 2020-10-06UTC: Auto merge of #76356 - caass:hooks, r=jyn514
  commit[5] 2020-10-06UTC: Auto merge of #77386 - joshtriplett:static-glibc, r=petrochenkov
ERROR: no CI builds available between a1dfd2490a6cb456b92e469fa550dc217e20ad6d and 98edd1fbf8a68977a2a7c1312eb1ebff80515a92 within last 167 days

@rustbot label -E-needs-bisection -E-needs-mcve

[steffahn]

Regression most likely from

• Separate projection bounds and predicates #73905

[steffahn]

The issue of confusing/unnecessary HRTB-requirements is demonstrated by something like

#![feature(generic_associated_types)]
use std::{future::Future, marker::PhantomData};

trait Trait {
    type Associated<'a>
    where
        Self: 'a;
}

fn future<'a, S: Trait + 'a, F>(f: F) -> F
where
    F: Future<Output = ()> + Send,
{
    f
}

async fn f<'a, S: Trait + 'a>() {
    let result: PhantomData<S::Associated<'a>> = PhantomData;
    async {}.await;
}

fn foo<'a, S: Trait + 'a>()
where
    S::Associated<'a>: Send,
{
    future::<'a, S, _>(f::<'a, S>());
}

(At least from my understanding, the S::Associated<'a>: Send is supposed to be enough.)

error: implementation of `Send` is not general enough
  --> src/main.rs:26:5
   |
26 |     future::<'a, S, _>(f::<'a, S>());
   |     ^^^^^^^^^^^^^^^^^^ implementation of `Send` is not general enough
   |
   = note: `<S as Trait>::Associated<'0>` must implement `Send`, for any lifetime `'0`...
   = note: ...but `Send` is actually implemented for the type `<S as Trait>::Associated<'a>`

which also fails before #73905, e.g. with

warning: the feature `generic_associated_types` is incomplete and may not be safe to use and/or cause compiler crashes
 --> src/main.rs:1:12
  |
1 | #![feature(generic_associated_types)]
  |            ^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(incomplete_features)]` on by default
  = note: see issue #44265 <https://github.com/rust-lang/rust/issues/44265> for more information

error[E0308]: mismatched types
  --> src/main.rs:26:5
   |
26 |     future::<'a, S, _>(f::<'a, S>());
   |     ^^^^^^^^^^^^^^^^^^ one type is more general than the other
   |
   = note: expected type `Send`
              found type `Send`

error: aborting due to previous error; 1 warning emitted

Looking further back... at some point (around nightly-2019-12-23) the error changes to

warning: the feature `generic_associated_types` is incomplete and may cause the compiler to crash
 --> src/main.rs:1:12
  |
1 | #![feature(generic_associated_types)]
  |            ^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(incomplete_features)]` on by default

error[E0109]: lifetime arguments are not allowed for this type
  --> src/main.rs:18:43
   |
18 |     let result: PhantomData<S::Associated<'a>> = PhantomData;
   |                                           ^^ lifetime argument not allowed

error[E0109]: lifetime arguments are not allowed for this type
  --> src/main.rs:24:19
   |
24 |     S::Associated<'a>: Send,
   |                   ^^ lifetime argument not allowed

error: aborting due to 2 previous errors

so there's no regression on this issue – it never worked.

[jackh726]

(At least from my understanding, the S::Associated<'a>: Send is supposed to be enough.)

Changing the where clause to for<'b> S::Associated<'b>: Send makes this code compile.

[jackh726]

GATs issue triage: not blocking. The examples here do use GATs, but the underlying issue seems to have roots in generality and generator capture (from what I can surmise). There also seem to be related issues that don't use GATs.

[ivan770]

(At least from my understanding, the S::Associated<'a>: Send is supposed to be enough.)

Changing the where clause to for<'b> S::Associated<'b>: Send makes this code compile.

I should mention that third approach from this article doesn't suffer from this problem, as I was able to remove those bounds and make code compile.

[QuineDot]

Removing where Self: 'lifetime from the type Gat<'lifetime> works around at least some of these issues.

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=2c1189b0b26ebf06584415ee952a77a4

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=aceda7095f19759d7176ae569b260abf