Tracking issue for future-incompatibility lint late_bound_lifetime_arguments

[petrochenkov]

What is this lint about

In functions not all lifetime parameters are created equal.
For example, if you have a function like

fn f<'a, 'b>(arg: &'a u8) -> &'b u8 { .... }

both 'a and 'b are listed in the same parameter list, but when stripped from the surface syntax the function looks more like

for<'a> fn f<'b>(arg: &'a u8) -> &'b u8 { .... }

where 'b is a "true" ("early-bound") parameter of the function, and 'a is an "existential" ("late-bound") parameter. This means the function is not parameterized by 'a.
To give some more intuition, let's write a type for function pointer to f:

// PtrF is not parameterized by 'a,
type PtrF<'b> = for<'a> fn(&'a u8) -> &'b u8;
// but it has to be parameterized by 'b
type PtrF = for<'a, 'b> fn(&'a u8) -> &'b u8; // ERROR

See more about this distinction in http://smallcultfollowing.com/babysteps/blog/2013/10/29/intermingled-parameter-lists/#early--vs-late-bound-lifetimes

When lifetime arguments are provided to a function explicitly, e.g.

f::<'my_a, 'my_b>

the first argument doesn't make much sense because the function is not parameterized by 'a.
Providing arguments for "late-bound" lifetime parameters in general doesn't make sense, while arguments for "early-bound" lifetime parameters can be provided.

It's not clear how to provide arguments for early-bound lifetime parameters if they are intermixed with late-bound parameters in the same list. For now providing any explicit arguments is prohibited if late-bound parameters are present, so in the future we can adopt any solution without hitting backward compatibility issues.

Note that late-bound lifetime parameters can be introduced implicitly through lifetime elision:

fn f(&u8) {}
// Desugars into
fn f<'a>(&'a u8) {} // 'a is late-bound

The precise rules discerning between early- and late-bound lifetimes can be found here:

rust/src/librustc/middle/resolve_lifetime.rs

Lines 1541 to 1700 in 91aff57

	/// Detects late-bound lifetimes and inserts them into
	/// `map.late_bound`.
	///
	/// A region declared on a fn is **late-bound** if:
	/// - it is constrained by an argument type;
	/// - it does not appear in a where-clause.
	///
	/// "Constrained" basically means that it appears in any type but
	/// not amongst the inputs to a projection. In other words, `<&'a
	/// T as Trait<''b>>::Foo` does not constrain `'a` or `'b`.
	fn insert_late_bound_lifetimes(map: &mut NamedRegionMap,
	fn_def_id: DefId,
	decl: &hir::FnDecl,
	generics: &hir::Generics) {
	debug!("insert_late_bound_lifetimes(decl={:?}, generics={:?})", decl, generics);
	let mut constrained_by_input = ConstrainedCollector { regions: FxHashSet() };
	for arg_ty in &decl.inputs {
	constrained_by_input.visit_ty(arg_ty);
	}
	let mut appears_in_output = AllCollector {
	regions: FxHashSet(),
	impl_trait: false
	};
	intravisit::walk_fn_ret_ty(&mut appears_in_output, &decl.output);
	debug!("insert_late_bound_lifetimes: constrained_by_input={:?}",
	constrained_by_input.regions);
	// Walk the lifetimes that appear in where clauses.
	//
	// Subtle point: because we disallow nested bindings, we can just
	// ignore binders here and scrape up all names we see.
	let mut appears_in_where_clause = AllCollector {
	regions: FxHashSet(),
	impl_trait: false
	};
	for ty_param in generics.ty_params.iter() {
	walk_list!(&mut appears_in_where_clause,
	visit_ty_param_bound,
	&ty_param.bounds);
	}
	walk_list!(&mut appears_in_where_clause,
	visit_where_predicate,
	&generics.where_clause.predicates);
	for lifetime_def in &generics.lifetimes {
	if !lifetime_def.bounds.is_empty() {
	// `'a: 'b` means both `'a` and `'b` are referenced
	appears_in_where_clause.visit_lifetime_def(lifetime_def);
	}
	}
	debug!("insert_late_bound_lifetimes: appears_in_where_clause={:?}",
	appears_in_where_clause.regions);
	// Late bound regions are those that:
	// - appear in the inputs
	// - do not appear in the where-clauses
	// - are not implicitly captured by `impl Trait`
	for lifetime in &generics.lifetimes {
	let name = lifetime.lifetime.name;
	// appears in the where clauses? early-bound.
	if appears_in_where_clause.regions.contains(&name) { continue; }
	// any `impl Trait` in the return type? early-bound.
	if appears_in_output.impl_trait { continue; }
	// does not appear in the inputs, but appears in the return
	// type? eventually this will be early-bound, but for now we
	// just mark it so we can issue warnings.
	let constrained_by_input = constrained_by_input.regions.contains(&name);
	let appears_in_output = appears_in_output.regions.contains(&name);
	if !constrained_by_input && appears_in_output {
	debug!("inserting issue_32330 entry for {:?}, {:?} on {:?}",
	lifetime.lifetime.id,
	name,
	fn_def_id);
	map.issue_32330.insert(
	lifetime.lifetime.id,
	ty::Issue32330 {
	fn_def_id,
	region_name: name,
	});
	continue;
	}
	debug!("insert_late_bound_lifetimes: \
	lifetime {:?} with id {:?} is late-bound",
	lifetime.lifetime.name, lifetime.lifetime.id);
	let inserted = map.late_bound.insert(lifetime.lifetime.id);
	assert!(inserted, "visited lifetime {:?} twice", lifetime.lifetime.id);
	}
	return;
	struct ConstrainedCollector {
	regions: FxHashSet<ast::Name>,
	}
	impl<'v> Visitor<'v> for ConstrainedCollector {
	fn nested_visit_map<'this>(&'this mut self) -> NestedVisitorMap<'this, 'v> {
	NestedVisitorMap::None
	}
	fn visit_ty(&mut self, ty: &'v hir::Ty) {
	match ty.node {
	hir::TyPath(hir::QPath::Resolved(Some(_), _)) |
	hir::TyPath(hir::QPath::TypeRelative(..)) => {
	// ignore lifetimes appearing in associated type
	// projections, as they are not *constrained*
	// (defined above)
	}
	hir::TyPath(hir::QPath::Resolved(None, ref path)) => {
	// consider only the lifetimes on the final
	// segment; I am not sure it's even currently
	// valid to have them elsewhere, but even if it
	// is, those would be potentially inputs to
	// projections
	if let Some(last_segment) = path.segments.last() {
	self.visit_path_segment(path.span, last_segment);
	}
	}
	_ => {
	intravisit::walk_ty(self, ty);
	}
	}
	}
	fn visit_lifetime(&mut self, lifetime_ref: &'v hir::Lifetime) {
	self.regions.insert(lifetime_ref.name);
	}
	}
	struct AllCollector {
	regions: FxHashSet<ast::Name>,
	impl_trait: bool
	}
	impl<'v> Visitor<'v> for AllCollector {
	fn nested_visit_map<'this>(&'this mut self) -> NestedVisitorMap<'this, 'v> {
	NestedVisitorMap::None
	}
	fn visit_lifetime(&mut self, lifetime_ref: &'v hir::Lifetime) {
	self.regions.insert(lifetime_ref.name);
	}
	fn visit_ty(&mut self, ty: &hir::Ty) {
	if let hir::TyImplTrait(_) = ty.node {
	self.impl_trait = true;
	}
	intravisit::walk_ty(self, ty);
	}
	}
	}

How to fix this warning/error

Just removing the lifetime arguments pointed to by the lint should be enough in most cases.

Current status

• Support generic lifetime arguments in method calls #42492 introduces the late_bound_lifetime_arguments lint as warn-by-default
• PR ? makes the late_bound_lifetime_arguments lint deny-by-default
• PR ? makes the late_bound_lifetime_arguments lint a hard error

[SimonSapin]

error: cannot specify lifetime arguments explicitly if late bound lifetime parameters are present
   --> /home/travis/build/servo/servo-with-rust-nightly/servo/components/style/stylesheets/stylesheet.rs:243:27
    |
243 |         self.iter_rules::<'a, 'b, EffectiveRules>(device, guard)
    |                           ^^
    |
note: lint level defined here
   --> /home/travis/build/servo/servo-with-rust-nightly/servo/components/style/lib.rs:26:9
    |
26  | #![deny(warnings)]
    |         ^^^^^^^^
    = note: #[deny(late_bound_lifetime_arguments)] implied by #[deny(warnings)]
    = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
    = note: for more information, see issue #42868 <https://github.com/rust-lang/rust/issues/42868>

What does "late bound" mean in this context? What should we do instead?

[petrochenkov]

@SimonSapin

What does "late bound" mean in this context?

Well, that's a good question, but not something that can be explained in an error message.
See for example http://smallcultfollowing.com/babysteps/blog/2013/10/29/intermingled-parameter-lists/#early--vs-late-bound-lifetimes. I believe nomicon had something about it, but I can't find it right now.

This issue should have a description, and @nikomatsakis is probably the best person to write it, but I can try as well.

What should we do instead?

Remove 'a, 'b, , it never did anything anyway.

[petrochenkov]

The error and lint should probably have an extra note pointing to one of these late bound lifetime parameters.

EDIT: Done in #43343

[SimonSapin]

What should we do instead?

Remove 'a, 'b, it never did anything anyway.

That’s not true in the case of #42508, where it is necessary to make something compile at all.

[petrochenkov]

One possible solution for the problem of intermixed late- and early-bound parameters is to support explicit for<...> clauses on function items in the same way it's done on function pointers:

for<'late> fn f<'early1, 'early2>(a: &'late u8, b: &'early1 u8, c: &/*'elided*/u8) -> &'early2 u8 { ... }

If for is specified then all lifetimes defined in it are considered late-bound, and all lifetimes specified in generic parameters are considered early-bound.
For function pointers the equivalent construction is already supported:

type F<'early1, 'early2> = for<'late> fn(&'late u8, &'early1 u8, &/*'elided*/u8) -> &'early2 u8;

The function could then be called with explicitly specified lifetime arguments, despite the presence of late-bound parameters:

f::<'arg_early1, 'arg_early2>()

[nikomatsakis]

@SimonSapin

I believe there exists a workaround to force early-bound lifetimes at a time when they are appropriate. Are you blocked on this point? In particular, you could do something like where 'static: 'a, which would force 'a to be early bound (because it appears in a where-clause).

The definition of late bound lifetimes, mechanically at least, is that they are those lifetimes that both:

• Lifetimes that appear in argument types
• and do not appear in where clauses.

All other lifetimes bound on a function are early-bound. So if you have a dummy where clause like where 'static: 'a, which has no particular meaning, it does force 'a to be early-bound.

(The reason that the early- vs late-bound distinction exists is something else, of course.)

I do hope we can clean this up in the future -- perhaps in another epoch -- but for now that is how it works. There are some annoying interactions with inference that make this all hard to alter in a totally backwards compatible way.

[SimonSapin]

I am blocked in the sense that I have warnings that I don’t know how to work around, don’t understand in the first place, and that say “it will become a hard error in a future release”.

[arielb1]

I think you could use something like (parse_nested_block: fn(&mut Parser<'i, 't>, _))(...) (using type ascription) instead of passing generic parameters. Ugly, but works.

[arielb1]

Nope, that doesn't work for some reason.

[pnkfelix]

@petrochenkov wrote (in the description):

It's not clear how to provide arguments for early-bound lifetime parameters if they are intermixed with late-bound parameters in the same list. For now providing any explicit arguments is prohibited if late-bound parameters are present, so in the future we can adopt any solution without hitting backward compatibility issues.

Note that late-bound lifetime parameters can be introduced implicitly through lifetime elision
[...]

While each of these paragraphs made sense to me when I read them on my own, I am curious about the conclusion that it leads to.

Consider a case like this:

fn all_explicits_are_early<'a, 'b>(arg: &Foo<'a, 'b>) -> Bar<'a, 'b> where 'a: 'a, 'b: 'b { ... }

I would think that a call expression all_expilcits_are_early::<'x, 'y>(param) would be sensible: we are providing arguments for the explicit parameters, which are all early bound.

But unfortunately, the above function definition actually desugars into something with a hidden late-bound lifetime parameter, due to lifetime elision on the arg: &Foo<'a, 'b>. Which then causes the error here to fire in the case described above.

• (This ends up being one part of why some workarounds to this are not applying to @SimonSapin's example from region inference sometimes fails to recognize implied bound in closure  #42508 )

Would it be possible to loosen the error checking here so that if all the explicitly listed lifetime parameters are early-bound, then one still gets to specify the lifetimes explicitly at the call-site, regardless of any hidden late bound lifetimes introduced due to the elision rules?

[cole-miller]

It seems this shows up as either a warn-level lint or a hard error depending on the code.

Example that triggers a warning (on stable and nightly):

static X: u8 = 0;
fn f<'a, 'b>(_x: &'b u8) -> &'a u8  { &X }
fn main() { let _ = f::<'static>; }

Example that triggers a hard error (what led me to open #80618; again, stable or nightly), with no mention of the tracking issue:

fn f<'a>() {}
fn main() { let _ = f::<'static>; }

Is this expected?

[petrochenkov]

@cole-miller
It's reported as a lint only in cases where making it a hard error would cause breakage back in 2017.
The intent is to produce an error, in general, the lint is only a way to give people some time to fix code.

[cole-miller]

@petrochenkov Thanks, understood. So I can go ahead and assign an error number and --explain text?

[fishrockz]

How do I solve something like

fn lister<'a, 'b, T: LifetimedDisplay<'a>>(
    buffer: &'b mut dyn Write,
    element: project::element::ElementRef,
    project: &'a project::project::Project,
) {
    writeln!(buffer, "{}", T::new(element, project)).unwrap();
}

I get

warning: cannot specify lifetime arguments explicitly if late bound lifetime parameters are present
  --> src/lib.rs:72:18
   |
72 |         lister::<'a, T>(buffer, element.clone(), project);
   |                  ^^
...
78 | fn lister<'a, 'b, T: LifetimedDisplay<'a>>(
   |               -- the late bound lifetime parameter is introduced here
   |
   = note: `#[warn(late_bound_lifetime_arguments)]` on by default
   = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
   = note: for more information, see issue #42868 <https://github.com/rust-lang/rust/issues/42868>

warning: `girderstream` (lib) generated 1 warning

I have spent a few hours on this and am pretty stumpped

[fishrockz]

For anyone else finding things like this you might find this helpful

It was actually solved at the call sight rather than the function

-lister::<'a, T>(buffer, element.clone(), project);
+lister::<T>(buffer, element.clone(), project);

Normally rustc gives the most excellent tips but i think in this case we could do better..

[RalfJung]

I have to say I am quite confused by the issue description here, and I don't understand what the underlying issue is. The core of my confusion seems to be captured by this (repeated) statement

the first argument doesn't make much sense because the function is not parameterized by 'a.

I strongly disagree! The function is parameterized by 'a. Sure, this parameter is a type system fiction (the actual runtime address of the function is the same for all choices of 'a), but that doesn't make 'a any less of a parameter. It can still be valuable to explicitly set that parameter to guide inference. In fact, even if I have a function pointer for<'a> fn(&'a i32) -> &'a i32, I may want to explicitly set the lifetime -- that's a completely meaningful operation and entirely orthogonal to our monomorphization strategy. The issue description seems to presuppose that explicit instantiation is somehow tied to monomorphization, but I don't understand why that would be the case.

Taking a step back: The fact that some lifetime parameters are type system fictions while others are actually relevant for monomorphization is very confusing and surprising. (I have never fully understood why this is the case; I'll need to dig into this some time.) This will clearly incur some friction when creating function pointers since monomorphization-relevant ("early-bound") lifetime parameters need to be chosen the moment the function pointer is created. I can see that we might want to improve that situation. But reading just this issue I really don't understand the motivation for the current change, since there's a big gap between "unfortunately some lifetime parameters actually matter for monomorphization" and "other lifetime parameters (that don't matter for monomorphization) don't make sense and we should disallow people from explicitly stating them". Which problem is being solved here?

[amanlai]

A related Stack Overflow discussion.