Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Type Aliases that are generic over ParamSpec don't work #11855

Closed
A5rocks opened this issue Dec 28, 2021 · 17 comments · Fixed by #14159
Closed

Type Aliases that are generic over ParamSpec don't work #11855

A5rocks opened this issue Dec 28, 2021 · 17 comments · Fixed by #14159
Labels
bug mypy got something wrong topic-paramspec PEP 612, ParamSpec, Concatenate topic-type-alias TypeAlias and other type alias issues

Comments

@A5rocks
Copy link
Contributor

A5rocks commented Dec 28, 2021

Bug Report

I expect n = Callable[P, int] to work, or at least n: TypeAlias = Callable[P, int]

To Reproduce

from typing import ParamSpec, Callable, TypeAlias

P = ParamSpec("P")

c: TypeAlias = Callable[P, int]

def f(n: c[P]) -> c[P]:
    ...

Expected Behavior

No errors

Actual Behavior

main.py:5: error: Variable "typing.TypeAlias" is not valid as a type
main.py:5: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
main.py:7: error: Variable "__main__.c" is not valid as a type
main.py:7: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
main.py:7: error: Invalid location for ParamSpec "P"
main.py:7: note: You can use ParamSpec as the first argument to Callable, e.g., 'Callable[P, int]'

Your Environment

mypy playground

@A5rocks A5rocks added the bug mypy got something wrong label Dec 28, 2021
@A5rocks
Copy link
Contributor Author

A5rocks commented Jan 3, 2022

This is a bit annoying to do so far, as we have to return a CallableType as a type alias but... like... we need to mark that it is missing a ParamSpec. This is OK with TypeVars because [<unbound T>] is perfectly fine (representation wise), but there's no way to represent that a ParamSpec is unbound because CallableType expects some arguments which is aaaaaaaa

I think the best way will be to make essentially a sentinel value and check for that, though it's been annoying to do so far.

@cdce8p
Copy link
Collaborator

cdce8p commented Mar 18, 2022

Another example with Concatenate inside a Type Alias. Saw it in the last pyright release. microsoft/pyright@2f5046b

import asyncio
from typing import Coroutine, Any, TypeVar, Concatenate, Callable, ParamSpec

_T = TypeVar("_T")
_P = ParamSpec("_P")

Coro = Coroutine[Any, Any, _T]
CoroFunc = Callable[_P, Coro[_T]]


class ClassA: ...

CheckFunc = CoroFunc[Concatenate[ClassA, _P], bool]


async def my_check_func(obj: ClassA, a: int, b: str) -> bool:
    print(a, b)
    return str(a) == b


async def takes_check_func(
    check_func: CheckFunc[_P], *args: _P.args, **kwargs: _P.kwargs
):
    await check_func(ClassA(), *args, **kwargs)


asyncio.run(takes_check_func(my_check_func, 1, "2"))

# This should generate an error because the signature doesn't match.
asyncio.run(takes_check_func(my_check_func, 1, 2))

@AlexWaygood AlexWaygood added the topic-type-alias TypeAlias and other type alias issues label Apr 4, 2022
@JMMarchant
Copy link

This may also be related?

https://mypy-play.net/?mypy=latest&python=3.10&gist=ae5f16e9b9ca9346f8a28ab60bd9825f

from typing import ParamSpec, Callable, Optional, TypeVar

P = ParamSpec("P")
R = TypeVar("R")

x: Callable[P, R]  # error
y: Optional[Callable[P, R]]  # error
z: Callable[..., R]  # fine
main.py:6: error: The first argument to Callable must be a list of types or "..."
main.py:7: error: The first argument to Callable must be a list of types or "..."
Found 2 errors in 1 file (checked 1 source file)

@JMMarchant
Copy link

Also this:

https://mypy-play.net/?mypy=latest&python=3.10&gist=c4d6f13911151a073aed0d6650e4af3e

from typing import ParamSpec, Callable, Optional, TypeVar

P = ParamSpec("P")
R = TypeVar("R")

def target_func(x: int) -> int:
    return 0

def test(x: Callable[P, R]):
    pass

def test_optional(x: Optional[Callable[P, R]] = None):
    pass

test(target_func)  # fine
test_optional(target_func)  # error
main.py:16: error: Argument 1 to "test_optional" has incompatible type "Callable[[int], int]"; expected "Optional[Callable[[VarArg(<nothing>), KwArg(<nothing>)], <nothing>]]"
Found 1 error in 1 file (checked 1 source file)

@hauntsaninja hauntsaninja changed the title Type Aliases for ParamSpec don't work Type Aliases that are generic over ParamSpec don't work May 23, 2022
@joaoe
Copy link

joaoe commented Jun 20, 2022

In my case I was trying to annotate a decorator, e.g.

P = ParamSpec("P")
R = TypeVar("R")
C = Callable[P, R]
D = Callable[[C], C]

@overload
def my_decorator(arg: logging.Logger, level: str= "debug") -> D:
    ...

@overload
def my_decorator(arg: C, /) -> C:
    ...

def my_decorator(arg: Union[C, logging.Logger], level: str= "debug") -> Union[C, D]:
    ...

instead of

P = ParamSpec("P")
R = TypeVar("R")

def my_decorator(arg: Union[Callable[P, R], logging.Logger], level: str= "debug"
    ) -> Union[Callable[[Callable[P, R]], Callable[P, R]], Callable[P, R]]:

which could be used as

@my_decorator
def my_function():
    pass

or

@my_decorator(logger, level="info")
def my_function():
    pass

Having an alias would make everything much simpler.

PS: Would be cool as well if an unbound Callable had implicit ParamSpec("P") and TypeVar("R") as return value by default, e.g., this could work:


C = Callable
def my_decorator(arg: Union[C, logging.Logger], level: str= "debug") -> Union[Callable[[C], C], C]:
    def wrapper0(fn0: Callable[C.params, C.returns]) -> C.returns:
        ....
    def wrapper1(*args: C.params.args, **kwargs: C.params.kwargs]) -> C.returns:
        ....

But that would require a PEP.

@mttbernardini
Copy link

Another case which also throws another funny error on Concatenate (?)

from typing import ParamSpec, Callable, Concatenate

class Event: ...

P = ParamSpec("P")

EventHandler = Callable[Concatenate[Event, P], None]
main.py:7: error: The first argument to Callable must be a list of types, parameter specification, or "..."
main.py:7: note: See https://mypy.readthedocs.io/en/stable/kinds_of_types.html#callable-types-and-lambdas
main.py:7: error: The last parameter to Concatenate needs to be a ParamSpec
main.py:7: error: ParamSpec "P" is unbound
Found 3 errors in 1 file (checked 1 source file)

mypy playground

Potentially related issues: #13403 and #13476

@GideonBear
Copy link

I'm experiencing this issue too, is there a workaround for this?

@abid-mujtaba
Copy link

Possible MFE (minimum failing example):

from typing import Callable, ParamSpec

P = ParamSpec("P")
FunctionType = Callable[P, None]
main.py:4: error: The first argument to Callable must be a list of types, parameter specification, or "..."
main.py:4: note: See https://mypy.readthedocs.io/en/stable/kinds_of_types.html#callable-types-and-lambdas
Found 1 error in 1 file (checked 1 source file)

mypy playgroud

@gandhis1
Copy link

The issue I reported in #14069 is likely a symptom of this issue (and lack of alias support)

ilevkivskyi added a commit that referenced this issue Nov 24, 2022
Fixes #11855 
Fixes #7084
Fixes #10445
Should fix #4987

After thinking about this for some time, it looks like the best way to
implement this is by switching type aliases from unbound to bound type
variables. Then I can essentially simply share (or copy in one small
place, to avoid cyclic imports) all the logic that currently exists for
`ParamSpec` and `Concatenate` in `expand_type()` etc.

This will also address a big piece of tech debt, and will get some
benefits (almost) for free, such as checking bounds/values for alias
type variables, and much tighter handling of unbound type variables.

Note that in this PR I change logic for emitting some errors, I try to
avoid showing multiple errors for the same location/reason. But this is
not an essential part of this PR (it is just some test cases would
otherwise fail with even more error messages), I can reconsider if there
are objections.
@RRRajput
Copy link

RRRajput commented Dec 8, 2022

Is there any information on when this fix would be available? (When 0.992 is released?)

@AlexWaygood
Copy link
Member

AlexWaygood commented Dec 8, 2022

Is there any information on when this fix would be available? (When 0.992 is released?)

You can track progress on the next release by following #13685 (I think it's unlikely that there'll be a 0.992 release).

@randolf-scholz
Copy link
Contributor

randolf-scholz commented Feb 8, 2023

The following is not working for me in 1.0.0:

from typing import TypeAlias, ParamSpec, Concatenate, Callable

P = ParamSpec("P")
intfun: TypeAlias = Callable[Concatenate[int, P], None]

def foo(i: int) -> None:
    pass

a: intfun = foo  # ✘ Incompatible types in assignment 

@erictraut
Copy link

@randolf-scholz, you have defined a generic type alias that has a single type parameter (P), but when you use the type alias in the last line, you're not specifying a type argument. Mypy should assume ... in this case (just like it assumes Any for a regular type variable). If you explicitly supply the type argument (a: intfun[...] or a: intfun[[]]), mypy succeeds.

So I agree this is a bug in mypy, but your code is also probably not doing what you intended. You might want to file a separate bug since this one has been closed for a while.

@randolf-scholz
Copy link
Contributor

randolf-scholz commented Feb 8, 2023

@erictraut So when I use intfun[...], then mypy reveals the type as def (builtins.int, *Any, **Any). Is there any way to directly specify this exact type hint as a TypeAlias? It seems weird that one has to go this roundabout way, as neither Callable[[int, ...], None], nor Callable[Concatenate[int, ...], None] are supported.

@hauntsaninja
Copy link
Collaborator

I recommend using a callback protocol

@randolf-scholz
Copy link
Contributor

randolf-scholz commented Feb 8, 2023

@hauntsaninja
Copy link
Collaborator

Oh thanks, I see. Basically the Concatenate version of #5876

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug mypy got something wrong topic-paramspec PEP 612, ParamSpec, Concatenate topic-type-alias TypeAlias and other type alias issues
Projects
None yet
Development

Successfully merging a pull request may close this issue.