Cannot cast value to TypeVar

[LiraNuna]

This is a bug report however it could be that I'm doing it wrong™. This is for mypy version 0.670

Consider this method:

from typing import Callable, Iterable, Mapping, MutableMapping, TypeVar, List, cast

T = TypeVar('T')
Tk = TypeVar('Tk')
Tv = TypeVar('Tv')

def group_by(
    items: Iterable[T],
    *,
    key: Callable[[T], Tk],
    value: Callable[[T], Tv] = lambda x: cast(Tv, x),
) -> Mapping[Tk, Iterable[Tv]]:
    result: MutableMapping[Tk, List[Tv]] = {}
    for item in items:
        result.setdefault(key(item), []).append(value(item))

    return result

Notice that the default parameter for value is an identity callable that casts x to Tv. This produces an error:

error: Invalid type "Tv"

If I remove the call to cast, I get a different error which makes sense to me and was the original reason I added cast in the first place:

error: Incompatible default for argument "value" (default has type "Callable[[T], T]", argument has type "Callable[[T], Tv]")
error: Incompatible return value type (got "T", expected "Tv")

Given that this method is mostly TypeVar based, what do you think would be the right behavior? I find the second error acceptable (because T != Tv) but I find Tv not being a type very confusing and therefor a bug.

[ethanhs]

Here is a more minimal repro:

from typing import TypeVar, Callable, cast

T = TypeVar('T')

def foo(x: Callable[[], T] = lambda: cast(T, 0)) -> T:
    return x()

This seems inconsistent because this passes type checking:

from typing import TypeVar, Callable, cast, Optional

T = TypeVar('T')

def foo(x: Optional[Callable[[], T]] = None) -> T:
    if x is None:
        x = lambda: cast(T, 0)
    return x()

I'm not sure why the difference in scopes would explain one type checking and the other not...

[ilevkivskyi]

Troubles with specifying defaults for generic arguments is a known issue #3737. And the fact that the cast doesn't work there is unfortunate. A typical workaround is to use an overload. In your case it will actually make the signature more correct:

@overload
def group_by(items: Iterable[T], *, key: Callable[[T], Tk]) -> Mapping[Tk, List[T]]: ...
@overload
def group_by(items: Iterable[T], *,
             key: Callable[[T], Tk], value: Callable[[T], Tv]) -> Mapping[Tk, List[Tv]]: ...

With your signature it is not possible to infer Tv in the return if value is not given in a call.

[hauntsaninja]

Closing as a duplicate of #3737