check_type_length_limit is broken

[lcnr]

The function was added to prevent hangs during codegen if there are infinite expansions which result in exponential blowup. It does that by computing an estimate of the complexity of the type and checking whether it's above a limit:

rust/compiler/rustc_monomorphize/src/collector.rs

Lines 632 to 641 in 9c8a58f

	fn check_type_length_limit<'tcx>(tcx: TyCtxt<'tcx>, instance: Instance<'tcx>) {
	let type_length = instance
	.args
	.iter()
	.flat_map(|arg| arg.walk())
	.filter(|arg| match arg.unpack() {
	GenericArgKind::Type(_) | GenericArgKind::Const(_) => true,
	GenericArgKind::Lifetime(_) => false,
	})
	.count();

This worked quite well, until we changed the walk to use caching in #72412. It looks like nobody noticed its effect on the type length limit in this PR.

With this the type length estimate grows linearly while the "true" size of the type is still exponential, causing the limit to never be reached in the cases it's intended for. The only remaining test which hits the limit arbitrarily sets it to 8 and uses a linearly recursive type:

rust/tests/ui/type_length_limit.rs

Lines 10 to 30 in 9c8a58f

	#![type_length_limit="8"]
	macro_rules! link {
	($id:ident, $t:ty) => {
	pub type $id = ($t, $t, $t);
	}
	}
	link! { A1, B1 }
	link! { B1, C1 }
	link! { C1, D1 }
	link! { D1, E1 }
	link! { E1, A }
	link! { A, B }
	link! { B, C }
	link! { C, D }
	link! { D, E }
	link! { E, F }
	link! { F, G<Option<i32>, Option<i32>> }
	pub struct G<T, K>(std::marker::PhantomData::<(T, K)>);

[lcnr]

Only checking this for instances in monomorphization seems also insufficient to handle all cases where this causes issues. itertools ends up with enourmous instances during mir inlining, causing a hang in the new solver. These instances do not get detected evenafter fixing this check by using a type visitor

[lcnr]

This regressed in 1.47