Can't use Callable[..., T] | Callable[..., Awaitable[T]] as a function argument

[patrick91]

Code:

I was trying to define a callable that can be async or not and use a TypeVar as the return type, but it seem to be broken, here's the full example:

from typing import Union, Awaitable, Callable, TypeVar, Any

T = TypeVar("T")

_RESOLVER_TYPE = Union[
    Callable[..., T],
    Callable[..., Awaitable[T]],
]

def x() -> int:
    return 1
    
async def a_x() -> int:
    return 1
    
    
def field_ok(
    resolver: _RESOLVER_TYPE,
) -> Any:
    ...
    
def field_broken(
    resolver: _RESOLVER_TYPE[T],
) -> Any:
    ...
    
field_ok(x)
field_ok(a_x)

field_broken(x)
field_broken(a_x)

if I don't pass T it seem to work, no sure what is wrong, the error is:

main.py:31: error: Argument 1 to "field_broken" has 
    incompatible type "Callable[[], Coroutine[Any, Any, int]]"; 
    expected "Union[Callable[..., <nothing>], Callable[..., Awaitable[<nothing>]]]"  [arg-type]

Here's the playground url: https://mypy-play.net/?mypy=latest&python=3.11&gist=6d8ed4a4e55e0d6b2712eff23cd3e3b0

[patrick91]

I don't know if this helps, but while trying to make a test for this I noticed that adding [builtins fixtures/tuple.pyi] makes the problem disappear. here's the test for reference:

[case testCallableAsyncUnion]
[builtins fixtures/tuple.pyi]
from typing import Union, Awaitable, Callable, TypeVar, Any

T = TypeVar("T")

_RESOLVER_TYPE = Union[
    Callable[..., T],
    Callable[..., Awaitable[T]],
]

def x() -> int:
    return 1

async def a_x() -> int:
    return 1

def field_broken(
    resolver: _RESOLVER_TYPE[T],
) -> Any:
    ...

field_broken(x)
field_broken(a_x)

[A5rocks]

For test cases, put the fixtures at the end (that may be why, though it may be a difference in typeshed vs fixtures in which case we should fix that)

[patrick91]

@A5rocks moving it at the end doesn't seem to change anything :)

I can send a PR with the test, if that helps 😊

Also for now I changed my code to look like this:

_RESOLVER_TYPE = Union[
    Callable[..., T],
    Callable[..., Coroutine[T, None, None]],
]

which "fixed" my original problem :)

[fjarri]

Unfortunately it does not fix things if the signature with Awaitable is in another library (currently hitting this problem with trio's Nursery.start_soon() which expects something returning an Awaitable). This worked in mypy==0.991.

[A5rocks]

@patrick91 Sorry for the delay -- I've finally tried out the test... and it seemed to work?

I stuck this at the bottom of my test-data/unit/check-type-aliases.test:

[case testTestTestTest]
from typing import Union, Awaitable, Callable, TypeVar, Any

T = TypeVar("T")

_RESOLVER_TYPE = Union[
    Callable[..., T],
    Callable[..., Awaitable[T]],
]

def x() -> int:
    return 1
    
async def a_x() -> int:
    return 1
    
    
def field_ok(
    resolver: _RESOLVER_TYPE,
) -> Any:
    ...
    
def field_broken(
    resolver: _RESOLVER_TYPE[T],
) -> Any:
    ...
    
field_ok(x)
field_ok(a_x)

field_broken(x)
field_broken(a_x)
[builtins fixtures/tuple.pyi]

And then I ran pytest -k testTestTestTest. pytest output:

(venv) PS C:\Users\A5rocks\Documents\mypy> pytest -k testTestTestTest
================================================= test session starts =================================================
platform win32 -- Python 3.10.9, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: C:\Users\A5rocks\Documents\mypy, configfile: pytest.ini, testpaths: mypy/test, mypyc/test
plugins: cov-2.12.1, forked-1.3.0, xdist-1.34.0
gw0 [1] / gw1 [1] / gw2 [1] / gw3 [1] / gw4 [1] / gw5 [1] / gw6 [1] / gw7 [1] / gw8 [1] / gw9 [1] / gw10 [1] / gw11 [1]
F                                                                                                                [100%]
====================================================== FAILURES =======================================================
__________________________________________________ testTestTestTest ___________________________________________________
[gw0] win32 -- Python 3.10.9 C:\Users\A5rocks\Documents\mypy\venv\Scripts\python.exe
data: C:\Users\A5rocks\Documents\mypy\test-data\unit\check-type-aliases.test:1032:
..\..\..\..\Documents\mypy\mypy\test\testcheck.py:86: in run_case
    self.run_case_once(testcase)
..\..\..\..\Documents\mypy\mypy\test\testcheck.py:181: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
E   AssertionError: Unexpected type checker output (C:\Users\A5rocks\Documents\mypy\test-data\unit\check-type-aliases.test, line 1032)
------------------------------------------------ Captured stderr call -------------------------------------------------
Expected:
Actual:
  main:31: error: Argument 1 to "field_broken" has incompatible type "Callable[[], Coroutine[Any, Any, int]]"; expected "Union[Callable[..., <nothing>], Callable[..., Awaitable[<nothing>]]]" (diff)

=============================================== short test summary info ===============================================
FAILED mypy/test/testcheck.py::TypeCheckSuite::check-type-aliases.test::testTestTestTest
================================================== 1 failed in 7.00s ==================================================

Do you have specifics on what you're doing? (A link to a branch would work, or a PR as you suggested)

[A5rocks]

My best guess is this is another impact of the type alias typevar change (the one which allowed paramspecs as type aliases, #14159). In which case, this is probably just a small missed detail. This hunch is based on the fact mypy seems to be inferring the typevar out and that it worked in 0.991 -- IDK any other disruptive change that impacted type aliases during that time but I may be completely misremembering :P

cc @ilevkivskyi since you made that change (in case you want to correct me or take a closer look at this -- I haven't done so yet! All the above is simply conjecture)

I should have verified before sending this -- this is present in the commit before that! I'll bisect this.

This worked in mypy==0.991.

I can't reproduce this working in mypy 0.991. This breakage doesn't seem recent.

[fjarri]

I can't reproduce this working in mypy 0.991. This breakage doesn't seem recent.

Sorry, just noticed your edit.

This commit: fjarri/nucypher-async@25718f2 passes with mypy 0.991 (ok, there is one fail, but it is unrelated to this issue), but reports a number of problems with Awaitable with mypy 1.3.0:

nucypher_async/utils/__init__.py:30: error: Argument 1 to "start_soon" of "Nursery" has incompatible type "Callable[[Event], Coroutine[Any, Any, None]]"; expected "Callable[[__T1], Awaitable[Any]]"  [arg-type]
nucypher_async/mocks/asgi.py:28: error: Argument 1 to "start_soon" of "Nursery" has incompatible type "Callable[[Union[HTTPScope, WebsocketScope, LifespanScope], Callable[[], Awaitable[Union[HTTPRequestEvent, HTTPDisconnectEvent, WebsocketConnectEvent, WebsocketReceiveEvent, WebsocketDisconnectEvent, LifespanStartupEvent, LifespanShutdownEvent]]], Callable[[Union[HTTPResponseStartEvent, HTTPResponseBodyEvent, HTTPServerPushEvent, HTTPEarlyHintEvent, HTTPDisconnectEvent, WebsocketAcceptEvent, WebsocketSendEvent, WebsocketResponseStartEvent, WebsocketResponseBodyEvent, WebsocketCloseEvent, LifespanStartupCompleteEvent, LifespanStartupFailedEvent, LifespanShutdownCompleteEvent, LifespanShutdownFailedEvent]], Awaitable[None]]], Awaitable[None]]"; expected "Callable[[__T1, __T2, __T3], Awaitable[Any]]"  [arg-type]
nucypher_async/p2p/learner.py:253: error: Argument 1 to "start_soon" of "Nursery" has incompatible type "Callable[[Contact], Coroutine[Any, Any, None]]"; expected "Callable[[__T1], Awaitable[Any]]"  [arg-type]
nucypher_async/p2p/learner.py:255: error: Argument 1 to "start_soon" of "Nursery" has incompatible type "Callable[[Contact], Coroutine[Any, Any, Optional[VerifiedUrsulaInfo]]]"; expected "Callable[[__T1], Awaitable[Any]]"  [arg-type]
nucypher_async/p2p/learner.py:264: error: Argument 1 to "start_soon" of "Nursery" has incompatible type "Callable[[VerifiedUrsulaInfo], Coroutine[Any, Any, None]]"; expected "Callable[[__T1], Awaitable[Any]]"  [arg-type]
nucypher_async/p2p/algorithms.py:132: error: Argument 1 to "start_soon" of "Nursery" has incompatible type "Callable[[Contact], Coroutine[Any, Any, Optional[VerifiedUrsulaInfo]]]"; expected "Callable[[__T1], Awaitable[Any]]"  [arg-type]
nucypher_async/p2p/algorithms.py:144: error: Argument 1 to "start_soon" of "Nursery" has incompatible type "Callable[[Contact], Coroutine[Any, Any, Optional[VerifiedUrsulaInfo]]]"; expected "Callable[[__T1], Awaitable[Any]]"  [arg-type]
nucypher_async/p2p/algorithms.py:218: error: Argument 1 to "start_soon" of "Nursery" has incompatible type "Callable[[Contact], Coroutine[Any, Any, None]]"; expected "Callable[[__T1], Awaitable[Any]]"  [arg-type]
nucypher_async/client/pre.py:154: error: Argument 1 to "start_soon" of "Nursery" has incompatible type "Callable[[Nursery, VerifiedUrsulaInfo], Coroutine[Any, Any, None]]"; expected "Callable[[__T1, __T2], Awaitable[Any]]"  [arg-type]

(Nursery is a type from the trio package)

[patrick91]

@A5rocks sorry for the radio silence, I just tested this again and it seems to work with the latest version of mypy 😊

my final type looks like this:

_RESOLVER_TYPE = Union[
    Callable[..., T],
    Callable[..., Coroutine[T, Any, Any]],
    Callable[..., Awaitable[T]],
    "staticmethod[Any, T]",
    "classmethod[Any, Any, T]",
]

[A5rocks]

Hmm, wacky. I meant to check the repository but completely forgot to. It's good that it's fixed now, nonetheless!

EDIT: I forgot that your original example still fails and has done so since... a while. That should still be fixed, I suppose.

[patrick91]

Hmm, wacky. I meant to check the repository but completely forgot to. It's good that it's fixed now, nonetheless!

EDIT: I forgot that your original example still fails and has done so since... a while. That should still be fixed, I suppose.

Yes, I needed to keep the Callable[..., Coroutine[T, Any, Any]], but also add the awaitable one after removing some parts of our plugin here: https://github.com/strawberry-graphql/strawberry/pull/2852/files#diff-00af9d43c9b59404ba170e72e470939f2a6dd50e2eae24d0e24ef105431e5414L46-L48

I'll leave this issue open then 😊