Add basic support for recursive TypeVar defaults (PEP 696)

[cdce8p]

Ref: #14851

[JelleZijlstra]

Shouldn't we .copy_modified() here?

[cdce8p]

That doesn't work, unfortunately. This here is used to link nested TypeVarTypes so if it's changed / evaluated in one location the other stays in sync.

An example from the tests

T1 = TypeVar("T1", default=str)
T2 = TypeVar("T2", default=T1)

class ClassD1(Generic[T1, T2]): ...

k = ClassD1()
reveal_type(k)  # should be `ClassD1[str, str]`

For the first pass the type variables are as follows

T1`1 = str
T2`2 = T1`1 = str

The section here now replaces the default for T2 with the tbd of the default of T1:

T1`1 = str
T2`2 = <result of T1`1>

--
With .copy_modified it would be a copy of the T1'1 thus when it's evaluated it's not used for T2. The test would still return

ClassD1[str, T1`1 = str]

At least that's how I understand it 😅

[cdce8p]

Looked at it more closely. It's basically as said above. In particular, it's called in freshend_function_type_vars. We enter this function with linked TypeVars

callee.variables[0] == callee.variables[1].default  # True

Then create new ones for each and link them again in expand_type, so that

tvs[0] == tvs[1].default  # True

# and by extension
fresh.variables[0] == fresh.variables[1].default  # True

mypy/mypy/expandtype.py

Lines 119 to 130 in 517f5ae

	def freshen_function_type_vars(callee: F) -> F:
	"""Substitute fresh type variables for generic function type variables."""
	if isinstance(callee, CallableType):
	if not callee.is_generic():
	return cast(F, callee)
	tvs = []
	tvmap: dict[TypeVarId, Type] = {}
	for v in callee.variables:
	tv = v.new_unification_variable(v)
	tvs.append(tv)
	tvmap[v.id] = tv
	fresh = expand_type(callee, tvmap).copy_modified(variables=tvs)

[JelleZijlstra]

Unfortunately that's not a safe assumption

[cdce8p]

Do you have an example I could try? Mypy will already complain about this here and reject the second TypeVar call.

T1 = TypeVar("T1", default=int)
T1 = TypeVar("T1", default=str)  # error

error: Cannot redefine "T1" as a type variable
error: Invalid assignment target
error: "int" not callable

[JelleZijlstra]

I think you have to import the other TypeVar from a different module.

[cdce8p]

That results in the same error message. The second TypeVar call would just be ignored. The logic here would still as it only compares the actual TypeVar name type_var_name = fullname.rpartition(".")[2]. Not the complete fullname.

E.g. the comparison is with T1 not __main__.T1.

[JelleZijlstra]

I thought something like this might fail:

[case testTypeVarDefaultsMultipleFiles]

from typing import Generic, TypeVar
from file2 import T as T2

T = TypeVar('T')

class Gen(Generic[T, T2]):
    pass

def func(a: Gen, b: Gen[str], c: Gen[str, str]) -> None:
    reveal_type(a)  # N: Revealed type is "__main__.Gen[Any, builtins.int]"
    reveal_type(b)  # N: Revealed type is "__main__.Gen[builtins.str, builtins.int]"
    reveal_type(c)  # N: Revealed type is "__main__.Gen[builtins.str, builtins.str]"

[file file2.py]
from typing import TypeVar

T = TypeVar('T', default=int)

But this passes as expected, and I couldn't find a variation that fails. So I suppose this is fine.

[JelleZijlstra]

This variation fails:

[case testTypeVarDefaultsMultipleFiles]

from typing import Generic, TypeVar
from file2 import T as T2

T = TypeVar('T', default=T2)

class Gen(Generic[T2, T]):
    pass

def func(a: Gen, b: Gen[str], c: Gen[str, float]) -> None:
    reveal_type(a)  # N: Revealed type is "__main__.Gen[builtins.int, builtins.int]"
    reveal_type(b)  # N: Revealed type is "__main__.Gen[builtins.str, builtins.str]"
    reveal_type(c)  # N: Revealed type is "__main__.Gen[builtins.str, builtins.float]"

[file file2.py]
from typing import TypeVar

T = TypeVar('T', default=int)

Expected:
  main:11: note: Revealed type is "__main__.Gen[builtins.int, builtins.int]" (diff)
  main:12: note: Revealed type is "__main__.Gen[builtins.str, builtins.str]" (diff)
  main:13: note: Revealed type is "__main__.Gen[builtins.str, builtins.float]"
Actual:
  main:11: note: Revealed type is "__main__.Gen[builtins.int, T2?]" (diff)
  main:12: note: Revealed type is "__main__.Gen[builtins.str, T2?]" (diff)
  main:13: note: Revealed type is "__main__.Gen[builtins.str, builtins.float]"

Not code I'd expect anyone to actually write, but it does seem like it should be legal.

[cdce8p]

Thanks for the test, that helped! Tbh I could also have though about this earlier but it's actually quite simple to add a name lookup here 😅

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

steam.py (https://github.com/Gobot1234/steam.py)
+ steam/ext/commands/commands.py:851: error: Overloaded function implementation does not accept all possible arguments of signature 3  [misc]
+ steam/ext/commands/commands.py:851: error: Overloaded function implementation cannot produce return type of signature 3  [misc]