Confusing type error due to strange inferred type for a closure argument

[jseyfried]

This example:

pub struct Request<'a, 'b: 'a> {
    a: &'a (),
    b: &'b (),
}
pub trait Handler: Send + Sync + 'static {
    fn handle(&self, &mut Request) -> Result<(),()>;
}
impl<F> Handler for F where F: Send + Sync + 'static + Fn(&mut Request) -> Result<(),()> {
    fn handle(&self, _: &mut Request) -> Result<(),()> {
        unimplemented!()
    }
}
fn make_handler(h: &'static Handler) -> Box<Handler> {
    Box::new(move |req| h.handle(req))
}

errors with

error[E0271]: type mismatch resolving `for<'r, 'r, 'r> <[closure@<anon>:14:14: 14:38 h:_] as std::ops::FnOnce<(&'r mut Request<'r, 'r>,)>>::Output == std::result::Result<(), ()>`
  --> <anon>:14:5
   |
14 |     Box::new(move |req| h.handle(req))
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected bound lifetime parameter , found concrete lifetime
   |
   = note: concrete lifetime that was found is lifetime '_#9r
   = note: required because of the requirements on the impl of `Handler` for `[closure@<anon>:14:14: 14:38 h:_]`
   = note: required for the cast to the object type `Handler`

Annotating the closure parameter |req: &mut Response| allow the example to compile.
Interesting, annotating with |req: &&mut Response| produces a similarly-structured error, so I believe we're inferring &&mut here (maybe related to autoderef?).

[jseyfried]

cc @nikomatsakis @eddyb

[eddyb]

This is a known issue IIRC, but I don't know how to actually find the original issue.

[nikomatsakis]

Sorry, I've been putting off this tab for days trying to squeeze out some time to write a more detailed comment. But in short the problem is that we do not infer when to "generalize" the region -- in particular, we will infer a type of &'a mut Handler for some 'a, but your type demands a type of &'a mut Handler for all 'a (i.e., the difference between fn(&'a mut Handler) and for<'a> fn (&'a mut Handler). We do not (yet) infer generalizations except when the expected type provides the hint -- now, in this case, I might expect us to be able to trace things through the expected type. We would I think do so if you were returning Box<FnMut()>, but because the FnMut() requirement is "hidden" (this is a trait that happens to be implemented by closures...) we fail to deduce it. I'm pretty sure there's an open issue on this somewhere.

[gmorenz]

Just ran into this (or at least I'm pretty sure it's this, otherwise feel free to split this off into a new bug)

Smaller test case:

fn main() {
    let suitable = |_| true;
    vec![1,2,3].into_iter().find(suitable);
}

Workarounds: Inline suitable into find, or annotate suitable with a & type, i.e. let suitable = |_: &_| true.

I think all of these issues are older duplicates

• Rust has trouble seeing higher ranked lifetimes through generics #36582
• Failure to determine impls with higher-ranked lifetimes involved #32600 (which has the type annotation workaround)
• Incorrect lifetime inferred for closure #26937 (which has the inlining workaround)
• Incomprehensible error message when inference fails for closure #24680
• Generic fn cannot be converted to for<'a> Fn #22791

[Mark-Simulacrum]

More duplicates, or at least the same underlying issue:

• Incorrect lifetime error with a generic FnMut after annotating the type in a closure. #22557
• Inference failure with mutable reference to closure #34892
• proc with a reference as an argument meets lifetime errors when inferring type #14037
• HRTB bug in streaming iterator #30786 (possibly)

We should decide on a single (or two, one for the diagnostic and one for the underlying issue) canonical issue and deduplicate.

[Mark-Simulacrum]

Leaving the following issues open, since I don't think they're quite duplicates. The rest of the issues linked in the two comments above are about to be closed referencing this issue.

• Rust has trouble seeing higher ranked lifetimes through generics #36582
• Incorrect lifetime error with a generic FnMut after annotating the type in a closure. #22557
• HRTB bug in streaming iterator #30786

[rminderhoud]

I wanted to re-include and extend @shepmaster's example from #26937 as it includes a potential work-around (?).

Namely that including the closure directly into the call of the function vs. binding it avoids the problem entirely and using type annotations can sometimes avoid the problem. Hopefully this is insightful and those more experienced can weigh in more.

https://is.gd/6jjGgX

fn caller<F>(f: F)  where F: Fn(&i32) {
    f(&42);
}

fn caller_ref_return<F>(f: F) where F: Fn(&i32) -> &i32 {
    f(&42);
}

fn main() {
    // Works
    caller(|a| println!("{}", a));
    
    // Works 
    let f = |a: &i32| println!("{}", a);
    caller(f);
    
    // Doesn't work (error: type mismatch)
    let f = |a| println!("{}", a);
    caller(f);
    
    // Works 
    caller_ref_return(|a| a);
    
    // Doesn't work (error: expected bound lifetime parameter, found concrete lifetime)
    let f = |a: &i32| a;
    caller_ref_return(f);
    
    // Doesn't work (error: type mismatch)
    let f = |a| a;
    caller_ref_return(f);
}

[Rufflewind]

Not sure if anyone else pointed this out, but wanted to specifically highlight this part of the error message:

for<'r, 'r, 'r>

This is unreadable − all the lifetime parameters are assigned the same name!

[jonhoo]

Another instance of this was just brought up on #rust-beginners with the regexp crate:

let s = "hello";
let re = regex::Regex::new("[aeiou]").unwrap();
let after = re.replace_all(s, |c| "xxx".to_string());
println!("{} -> {}", s, after);

fails with the supremely unhelpful error:

error[E0271]: type mismatch resolving `for<'r, 's> <[closure@src/main.rs:7:35: 7:56] as std::ops::FnOnce<(&'r regex::Captures<'s>,)>>::Output == std::string::String`
 --> src/main.rs:7:20
  |
7 |     let after = re.replace_all(s, |c| "xxx".to_string());
  |                    ^^^^^^^^^^^ expected bound lifetime parameter, found concrete lifetime
  |
  = note: required because of the requirements on the impl of `regex::Replacer` for `[closure@src/main.rs:7:35: 7:56]`

error[E0631]: type mismatch in closure arguments
 --> src/main.rs:7:20
  |
7 |     let after = re.replace_all(s, |c| "xxx".to_string());
  |                    ^^^^^^^^^^^    --------------------- found signature of `fn(_) -> _`
  |                    |
  |                    expected signature of `for<'r, 's> fn(&'r regex::Captures<'s>) -> _`
  |
  = note: required because of the requirements on the impl of `regex::Replacer` for `[closure@src/main.rs:7:35: 7:56]`

error: aborting due to 2 previous errors

Compiles fine if you use |c: &regex::Captures|.

[quadrupleslap]

I'm dumb, but isn't this a lot like the monomorphization restriction in Haskell, but with lifetimes instead of types?

[nikomatsakis]

Regarding this example that @jonhoo cited (playground link), the cause is that, because the signature for replace_all requires only that R: Replacer, rather than say R: FnMut(&Captures) -> String, we don't have the "expected type" for the closure's arguments at the time that we need them. As a result, we fail to infer that the region must be bound in the closure (sorry, jargon).

I note this because, in addition to improving the error msg, I'd of course prefer if that example just worked -- but resolving that issue (figuring out bound regions more generally) is a bit of an open question. In principle it may be possible, I haven't thought hard about it, but it'll require a lot of refactoring before we get there.

Anyway, we ought to be able to improve the error message in the meantime I suppose.

[nikomatsakis]

(I suppose that we might, in this particular case, be able to get smarter about the expected type. There is even an open issue on this, but I can't find it just now.)

[estebank]

Another repro case:

struct My(i32);

fn call_with_ref<F>(some_closure: F) -> i32
where
    F: for<'a> Fn(&'a My) -> &'a i32,
{
    let value = My(0);
    *some_closure(&value)
}

fn main() {
    
    
    let _f = |arg: &My| &arg.0;
    //This doesn't work
    call_with_ref(_f);
    
    //This is ok.
    //call_with_ref(|arg: &My| &arg.0);
}

[Wizr]

Maybe similar issue playground

fn main() {
    let s = "";
    ((&|_| s) as &dyn Fn(&str) -> &str)(s);
}

But this works

fn main() {
    let s = "";
    let f: &dyn Fn(&str) -> &str = &|_| s;
    f(s);
}

And another one

[apiraino]

Assigning P-medium as discussed as part of the Prioritization Working Group procedure and removing I-prioritize.

Additionally we nominate this longstanding issue to be discussed during the next meeting

[spastorino]

This issue was discussed during today's compiler team meeting

[estebank]

Triage: Current output:

https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=456b5353ab48999eb273c83cfc0ae6f7

error[E0308]: mismatched types
  --> src/lib.rs:14:5
   |
14 |     Box::new(move |req| h.handle(req))
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ one type is more general than the other
   |
   = note: expected type `for<'r, 's, 't0> Fn<(&'r mut Request<'s, 't0>,)>`
              found type `Fn<(&mut Request<'_, '_>,)>`
note: this closure does not fulfill the lifetime requirements
  --> src/lib.rs:14:14
   |
14 |     Box::new(move |req| h.handle(req))
   |              ^^^^^^^^^^^^^^^^^^^^^^^^

https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=8342936d11149e38cafc35da516274cb

error[E0521]: borrowed data escapes outside of closure
 --> src/main.rs:3:28
  |
2 |     let mut fields: Vec<&str> = Vec::new();
  |         ---------- `fields` declared here, outside of the closure body
3 |     let pusher = |a: &str| fields.push(a);
  |                   -        ^^^^^^^^^^^^^^ `a` escapes the closure body here
  |                   |
  |                   `a` is a reference that is only valid in the closure body

https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=28594d6b627049e7a1de6cd590dfcae8

error[E0308]: mismatched types
  --> src/main.rs:30:5
   |
30 |     foo2(|x,y| ()); // Error, but works with one argument
   |     ^^^^ lifetime mismatch
   |
   = note: expected type `for<'r, 's> Fn<(&'r str, &'s str)>`
              found type `Fn<(&str, &str)>`
note: this closure does not fulfill the lifetime requirements
  --> src/main.rs:30:10
   |
30 |     foo2(|x,y| ()); // Error, but works with one argument
   |          ^^^^^^^^
note: the lifetime requirement is introduced here
  --> src/main.rs:15:12
   |
15 | fn foo2<F: Foo>(f: F) {
   |            ^^^

error: implementation of `Fn` is not general enough
  --> src/main.rs:31:10
   |
31 |     foo3(Box::new(&|x,y| ())); // Error, even with one argument
   |          ^^^^^^^^^^^^^^^^^^^ implementation of `Fn` is not general enough
   |
   = note: closure with signature `fn(&'2 str, &str)` must implement `Fn<(&'1 str, &str)>`, for any lifetime `'1`...
   = note: ...but it actually implements `Fn<(&'2 str, &str)>`, for some specific lifetime `'2`

error: implementation of `Fn` is not general enough
  --> src/main.rs:31:10
   |
31 |     foo3(Box::new(&|x,y| ())); // Error, even with one argument
   |          ^^^^^^^^^^^^^^^^^^^ implementation of `Fn` is not general enough
   |
   = note: closure with signature `fn(&str, &'2 str)` must implement `Fn<(&str, &'1 str)>`, for any lifetime `'1`...
   = note: ...but it actually implements `Fn<(&str, &'2 str)>`, for some specific lifetime `'2`

https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=0254e96af9155a0a99880e06b4a75b95

error: implementation of `FnOnce` is not general enough
 --> src/main.rs:3:29
  |
3 |     vec![1,2,3].into_iter().find(suitable);
  |                             ^^^^ implementation of `FnOnce` is not general enough
  |
  = note: closure with signature `fn(&'2 i32) -> bool` must implement `FnOnce<(&'1 i32,)>`, for any lifetime `'1`...
  = note: ...but it actually implements `FnOnce<(&'2 i32,)>`, for some specific lifetime `'2`

[Aaron1011]

Related: #91966

[108anup]

I don't think any of the posts here mention a workaround (until https://rust-lang.github.io/rfcs/3216-closure-lifetime-binder.html gets implemented).

I have been able to use the following workaround (shown for the example posted by @estebank).

struct My(i32);

fn call_with_ref<F>(some_closure: F) -> i32
where
    F: for<'a> Fn(&'a My) -> &'a i32,
{
    let value = My(0);
    *some_closure(&value)
}

fn coerce<F>(closure: F) -> F
where F: for<'a> Fn(&'a My) -> &'a i32
{
    return closure
}

fn fun()
{
    // Does not work
    // let _f = |arg: &My| &arg.0;
    // call_with_ref(_f);

    // Workaround
    let _f = coerce(|arg: &My| &arg.0);
    call_with_ref(_f);

    // This is ok.
    call_with_ref(|arg: &My| &arg.0);
}

While in the above example, the closure can be passed to call_with_ref directly also and the problem seems contrived, this is not the case in the below example. Here the explicit coerce function seems helpful. I am guessing this will also work with Box instead of Option (for the request handling example in the issue).

fn optional_call_with_ref<F>(some_closure: Option<F>) -> i32
where
    F: for<'a> Fn(&'a My) -> &'a i32,
{
    let value = My(0);
    *(some_closure.unwrap())(&value)
}

fn fun2()
{
    // Does not work, have to use explicit coercion.
    // optional_call_with_ref(Some(|arg: &My| &arg.0));

    // Workaround
    let _f = coerce(|arg: &My| &arg.0);
    optional_call_with_ref(Some(_f));
}

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=893482858c3f6e96f464b052e1cf321d

I had three questions about this workaround:

1. Is it possible to give a name to the trait bound for<'a> Fn(&'a My) -> &'a i32 so that multiple instances of this bound can use the name instead of repeating the definition.
2. Is it possible to write a macro to generate the coerce function? Can macros accept a trait bound and a closure. E.g., to replace with let _f = coerce_macro!(for<'a> Fn(&'a My) -> &'a i32, |arg: &My| &arg.0)
3. Is there a better workaround?

Disclaimer: I am fairly new to rust.

[estebank]

@108anup there's no way of being explicit yet in stable Rust, but nightly Rust supports specifying the for lifetime let _f = for<'a> |arg: &'a My| -> &'a i32 { &arg.0 };. Your work-around is what I'd use in stable.

[douglas-raillard-arm]

@108anup: Yes to 1. and 2.:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=0a7258b5644e37aca7a25008d5ebca3d

We can't properly give a name to the bound AFAIK since trait alias are unstable, but we can make a coercion function with a user-chosen name which is basically as good in this case. Then the user can just reuse that coercion function without having to repeat the details.

I hope the for<'a>|&'a ...| syntax gets stabilized soon, nom (or I suppose any closure-heavy lib) is a very special kind of hell without that.

[108anup]

@douglas-raillard-arm Thanks, this is very helpful!!

I wanted to give a name to the trait bound so that I can use the name in (1) the coercion function (when creating the closure) as well as (2) the function (or struct) that consumes the closure. E.g., in your playground link, the function signature is repeated on line 34. Is there a way to avoid repeating the definition (function signature).

My current workaround has been on the following lines (i.e., store reference to trait object). I think there were some cases of closures I wan't able to represent this way. I don't recall exactly what though.

type SetRealValueFn<'a, 'ctx> = &'a dyn Fn(&'ctx z3::Context, RealNumRep) -> z3::ast::Bool<'ctx>;
type GetFeasibleRealValueFn<'a> = &'a dyn Fn(&z3::Solver) -> RealNumRep;
type RealNumInt = i32;
type RealNumRep = Ratio<RealNumInt>;

struct BinarySearchDetails<'a, 'ctx> {
    // https://stackoverflow.com/questions/27831944/how-do-i-store-a-closure-in-a-struct-in-rust
    name: &'a String,
    feasible: Option<RealNumRep>,
    lo: RealNumRep,
    hi: RealNumRep,
    set_value: SetRealValueFn<'a, 'ctx>,
    get_feasible_closure: GetFeasibleRealValueFn<'a>,
    use_int: bool,
}

[douglas-raillard-arm]

In unstable Rust you can with trait_alias feature:
https://doc.rust-lang.org/beta/unstable-book/language-features/trait-alias.html

https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=df05613e03998b6cf6a55824b48c1d89