typeck hang with recursive type

[jruderman]

This hangs the compiler in typeck:

enum t = @t;
fn main() {
    let x : t;
    x.next = x;
}

[nikomatsakis]

probably my fault.

[nikomatsakis]

Not at all what I expected it to be. Rather, a loop in autoderef. I will fix the loop but I suppose it might also make sense to reject this type as uninstanitable---there should be some enum variant that does not require an instance of the enum itself. But I am not sure this is worth checking in general: after all, the user could never successfully construct an instance of that type, and to check it in general requires expanding and traversing enums and so forth.

[jruderman]

I think rejecting the type makes more sense. If we reject the type immediately, we won't need to change the autoderef loop, right?

[nikomatsakis]

That's the idea. I independently came to the same conclusion. In particular, there are multiple autoderef loops (which ought to be consolidated more, but still) and I don't like the idea of this same kind of bug creeping up in another of them.

[nikomatsakis]

Also, it's just a nicer error message.

[nikomatsakis]

Unfortunately, we still need the loop check in the autoderef loop for typeck, because the actual checking that the enum is instantiable and the checking of the fn body occur in the same pass over the AST. We could change it to have three passes (one to compute types and generate info, one to check for instantiability, and one to check fn bodies and so forth) but that seems annoying.

[jruderman]

Could the pass bail out early after encountering an uninstantiable type?

[nikomatsakis]

@jruderman No, the problem is that the autoderef loop could occur before we get around to checking whether the type is instantiable. As I said, I could restructure the pass to make sure we check all types for instantiability (and possibly other properties) before we delve into function bodies and expressions, but otherwise I think we still need to be careful when autoderef'ing. Such a restructuring might be a good idea though.

[nikomatsakis]

Fixed in 23f92ea

[msullivan]

Should this be closed, then?

[nikomatsakis]

Yes.

[alexcrichton]

The issue-2063.rs test has been xfail'd for some time now, but this issue appears to be cropping up again.

[bblum]

That test case emits an error before hanging: this type cannot be instantiated without an instance of itself

One easy way to fix this would be to make that error emission site do a span_fatal, so that the compilation phase stops running before it gets to the hang. Not sure if this is the right thing to do.

Nominating well-covered.

[nikomatsakis]

It might be. The idea of this error was that code in typeck had to worry about this kind of infinite loop case, but not other code. But maybe it's too much trouble even within typeck.

[graydon]

accepted for production-ready milestone

[catamorphism]

High, but not 1.0

[alexcrichton]

Fixing this bug should also fix this code:

enum A {
    A(B)
}

enum B {
    B1, B2(B)
}

fn main() {}

[isabelmu]

Not sure under what circumstances issue-2063.rs was originally xfailed, but as of now it's missing #[feature(managed_boxes)]; and emits an extra error: "type t does not implement any method in scope named to_str". I enabled managed boxes (and tried adding a &self parameter to the to_str call), but could not get the compiler to hang. It emits the expected error ("this type cannot be instantiated without an instance of itself; consider using Option<t>") and quits.

The recursive-enum stack overflow issue pointed referenced by @alexcrichton is described in #3008, and addressed in pull request #11839. I don't think it's related to this.

[flaper87]

issue-2063.rs is not xfail'd anymore and tests introduced in #3008 already test @alexcrichton example. Look at issue-3008-1.rs

This seems to be fixed, closing.