Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix crash with generic class definition in function #13678

Merged

Commits on Sep 18, 2022

  1. Fix crash with generic class definition in function

    Fixes python#12112.
    
    The reason why mypy was crashing with a "Must not defer during final
    iteration" error in the following snippet:
    
        from typing import TypeVar
    
        def test() -> None:
            T = TypeVar('T', bound='Foo')
    
            class Foo:
                def bar(self, foo: T) -> None:
                    pass
    
    ...was because mypy did not seem to be updating the types of the `bar`
    callable on each pass: the `bind_function_type_variables` method in
    `typeanal.py` always returned the _old_ type variables instead of using
    the new updated ones we found by calling `self.lookup_qualified(...)`.
    
    This in turn prevented us from making any forward progress when mypy
    generated a CallableType containing a placedholder type variable. So,
    we repeated the semanal passes until we hit the limit and crashed.
    
    I opted to fix this by having the function always return the newly-bound
    TypeVarLikeType instead. (Hopefully this is safe -- the way mypy likes
    mutating types always makes it hard to reason about this sort of stuff).
    
    Interestingly, my fix for this bug introduced a regression in one of
    our existing tests:
    
        from typing import NamedTuple, TypeVar
    
        T = TypeVar("T")
        NT = NamedTuple("NT", [("key", int), ("value", T)])
    
        # Test thinks the revealed type should be:
        #     def [T] (key: builtins.int, value: T`-1) -> Tuple[builtins.int, T`-1, fallback=__main__.NT[T`-1]]
        #
        # ...but we started seeing:
        #     def [T, _NT <: Tuple[builtins.int, T`-1]] (key: builtins.int, value: T`-1) -> Tuple[builtins.int, T`-1, fallback=test.WTF[T`-1]]
        reveal_type(NT)
    
    What seems to be happening here is that during the first pass, we add
    two type vars to the `tvar_scope` inside `bind_function_type_variables`:
    `T` with id -1 and `_NT` with id -2.
    
    But in the second pass, we lose track of the `T` typevar definition and/or
    introduce a fresh scope somewhere and infer `_NT` with id -1 instead?
    
    So now mypy thinks there are two type variables associated with this
    NamedTuple, which results in the screwed-up type definition.
    
    I wasn't really sure how to fix this, but I thought it was weird that:
    
    1. We were using negative IDs instead of positive ones. The former is
       meant to be for class definitions, which is what we're using here.
    2. We weren't wrapping this whole thing in a new tvar scope frame, given
       we're nominally synthesizing a new class.
    
    So I did that, and the tests started passing?
    
    I wasn't able to repro this issue for TypedDicts, but opted to introduce
    a new tvar scope frame there as well for consistency.
    Michael0x2a committed Sep 18, 2022
    Configuration menu
    Copy the full SHA
    dd4b8a2 View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2022

  1. Respond to code review

    Michael0x2a committed Sep 25, 2022
    Configuration menu
    Copy the full SHA
    d61d148 View commit details
    Browse the repository at this point in the history