PEP 647 support mishandles higher-order functions

[gvanrossum]

See #9865 for the initial PEP 647 support (type guards).

I did not manage to get this to work right for higher-order functions, notably combined with overloads.

There's a test that fails: testTypeGuardOverload in check-typeguard.test. The crux is

@overload
def filter(f: Callable[[T], TypeGuard[R]], it: Iterable[T]) -> Iterator[R]: ...
@overload
def filter(f: Callable[[T], bool], it: Iterable[T]) -> Iterator[T]: ...
def filter(*args): pass

[hauntsaninja]

@sobolevn want to take a look at this? The (skipped) test is improved after #11314, but still isn't fully passing.

Possibly related to #1317 and friends. There are some places in applytype / expandtype where we lose track of type variables.

[sobolevn]

Sure! Will do today 👍
Thanks for letting me know.

[sobolevn]

Yeap, it still fails:

_______________________________ testTypeGuardOverload ________________________________
[gw0] darwin -- Python 3.9.1 /Users/sobolev/Desktop/mypy/.venv/bin/python3
data: /Users/sobolev/Desktop/mypy/test-data/unit/check-typeguard.test:250:
/Users/sobolev/Desktop/mypy/mypy/test/testcheck.py:138: in run_case
    self.run_case_once(testcase)
/Users/sobolev/Desktop/mypy/mypy/test/testcheck.py:225: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
E   AssertionError: Unexpected type checker output (/Users/sobolev/Desktop/mypy/test-data/unit/check-typeguard.test, line 250)
-------------------------------- Captured stderr call --------------------------------
Expected:
  main:19: note: Revealed type is "typing.Iterator[Union[builtins.int, None]]" (diff)
  main:22: note: Revealed type is "typing.Iterator[builtins.int*]"
  main:24: note: Revealed type is "typing.Iterator[Union[builtins.int, None]]...
Actual:
  main:19: note: Revealed type is "typing.Iterator[R`10]" (diff)
  main:22: note: Revealed type is "typing.Iterator[builtins.int*]"
  main:24: note: Revealed type is "typing.Iterator[Union[builtins.int, None]]...

Alignment of first line difference:
  E: ...led type is "typing.Iterator[Union[builtins.int, None]]"
  A: ...led type is "typing.Iterator[R`10]"

Looking into it.

[sobolevn]

Minimal reproduction:

from typing import Callable, Iterable, Iterator, List, Optional, TypeVar

T = TypeVar("T")
R = TypeVar("R")

def filter(f: Callable[[T], TypeGuard[R]], it: Iterable[T]) -> Iterator[R]: ...

a: List[Optional[int]]
bb = filter(lambda x: x is not None, a)
reveal_type(bb)  # N: Revealed type is "typing.Iterator[Union[builtins.int, None]]"
# error, actually it is now: Revealed type is "typing.Iterator[R`2]"