mypy --strict requires circular type parameters when type variables are bound to the generic types that use them

[sg495]

Bug Report

Using mypy --strict requires type parameters to be specified even in circumstances where an untyped generic should be accepted, because the covariant type variable is bound to the generic type using it:

ValueT = TypeVar("ValueT", bound="Value", covariant=True)
# mypy --strict error (Missing type parameters for generic type "Value")

class Value(Generic[ValueT]):
    ...

This creates a circular reference problem, where the type itself must be passed as value to its own type parameter:

ValueT1 = TypeVar("ValueT1", bound="Value", covariant=True)
# mypy --strict error (Missing type parameters for generic type "Value")

ValueT2 = TypeVar("ValueT2", bound="Value[Value]", covariant=True)
# mypy --strict error (Missing type parameters for generic type "Value")

ValueT3 = TypeVar("ValueT3", bound="Value[Value[Value]]", covariant=True)
# mypy --strict error (Missing type parameters for generic type "Value")

# (...turtles all the way down)

I would have expected this first example to not raise any errors, with Value treated as Value[any subtype of Value] by default. This is because the stated bound for ValueT is Value itself, making the top element of the Value type hierarchy type-theoretically well-defined (it should consist of all instances of Value in this case).

On the other hand, mypy --strict allows typing.Any as a type parameter value (regardless of the stated bound). This can be exploited to solve the problem:

from typing import Any, Generic, TypeVar

ValueT = TypeVar("ValueT", bound="GenericValue", covariant=True)

class Value(Generic[ValueT]):
    ...

GenericValue = Value[Any]

At this time, it is impossible to specify the upper bounds for the Value without exploiting this behaviour.

Your Environment

• Mypy version used: mypy 0.920
• Mypy command-line flags: --strict
• Python version used: Python 3.9.7

Related issues

This is, arguably, a well-behaved special case of #4236.

Bonus: A simplified concrete scenario

The following is a (semplified but concrete) scenario involving expressions and value classes (intended to be immutable), which raises two "annoying" errors when mypy --strict is used (and one "desired" error using either mypy or mypy --strict):

from abc import ABC, abstractmethod
from typing import Any, Generic, TypeVar

ExprT = TypeVar("ExprT", bound="Expr[Value]", covariant=True)
# Annoying mypy --strict error (Missing type parameters for generic type "Value")
ValueT = TypeVar("ValueT", bound="Value", covariant=True)
# Annoying mypy --strict error (Missing type parameters for generic type "Value")

class Expr(Generic[ValueT], ABC):

    @abstractmethod
    def eval(self) -> ValueT:
        ...

class Value(Expr[ValueT]):

    def eval(self: ValueT) -> ValueT:
        return self

class ValueOfKind1(Value["ValueOfKind1"]):
    ...

class ValueOfKind2(Value["ValueOfKind2"]):
    ...

x: Expr[ValueOfKind1] = ValueOfKind1() # As desired, no error
y: Expr[ValueOfKind2] = ValueOfKind2() # As desired, no error
z: Expr[ValueOfKind2] = ValueOfKind1() # As desired, mypy error (Incompatible types in assignment)

However, the problem can be solved by defining a GenericValue type alias, setting the type parameter of Value to typing.Any:

from abc import ABC, abstractmethod
from typing import Any, Generic, TypeVar

ExprT = TypeVar("ExprT", bound="Expr[GenericValue]", covariant=True)
ValueT = TypeVar("ValueT", bound="Value[GenericValue]", covariant=True)

class Expr(Generic[ValueT], ABC):

    @abstractmethod
    def eval(self) -> ValueT:
        ...

class Value(Expr[ValueT]):

    def eval(self: ValueT) -> ValueT:
        return self

GenericValue = Value[Any] # mypy does not complain

class ValueOfKind1(Value["ValueOfKind1"]):
    ...

class ValueOfKind2(Value["ValueOfKind2"]):
    ...

x: Expr[ValueOfKind1] = ValueOfKind1() # As desired, no error
y: Expr[ValueOfKind2] = ValueOfKind2() # As desired, no error
z: Expr[ValueOfKind2] = ValueOfKind1() # As desired, mypy error (Incompatible types in assignment)

[A5rocks]

This is unexpected behaviour: Any is not a subtype of Value, so it shouldn't be accepted as a value for the type parameter.

For what it's worth, Any is a bit special; it nearly always works as a type (and where it doesn't work... idk it probably should).

[hauntsaninja]

Yeah, as A5rocks says, Any can behave like a subtype of Value (it can behave both a top type and a bottom type, unlike object which is a top type). So as you noticed you can make your error go away like:

ValueT = TypeVar("ValueT", bound="Value[Any]", covariant=True)

This is Any working as intended.

[sg495]

Thank you, @hauntsaninja and @A5rocks: I have updated my original issue to restrict it to the circular reference problem.

[hauntsaninja]

I'm still a little confused by the update, why doesn't the following work for you (note the explicit Any)?

ValueT = TypeVar("ValueT", bound="Value[Any]", covariant=True)

[sg495]

That works as well, yes. It still exploits the special behaviour of Any, but I guess there isn't much to do about it ATM.