Am I misunderstanding Concatenate/ParamSpec or is this a bug in MyPy?

[gandhis1]

Here is a contrived code example:

from typing import Any, Callable, Dict
from typing_extensions import Concatenate, ParamSpec
from functools import wraps

P = ParamSpec("P")
PreWrapFunc = Callable[Concatenate[Dict[str, str], P], str]
PostWrapFunc = Callable[P, str]


# Ignore the below for now, as the type aliases above are what is failing

def second_order_decorator(mappings: Dict[str, str]) -> Callable[[PreWrapFunc], PostWrapFunc]:
    def decorator(func: PreWrapFunc) -> PostWrapFunc:
        @wraps(func)
        def wrapped_func(*args: Any, **kwargs: Any):
            return func(mappings, *args, **kwargs)
        return wrapped_func
    return decorator


@second_order_decorator(mappings={"a": "1", "b": "2"})
def decorated_func(mappings: Dict[str, str], a: int, b: str) -> str:
    return "|".join(k + ":" + v for k, v in mappings.items()) + str(a) + b

Here is the output from MyPy:

test_case.py:6: error: The first argument to Callable must be a list of types, parameter specification, or "..."  [valid-type]
test_case.py:6: note: See https://mypy.readthedocs.io/en/stable/kinds_of_types.html#callable-types-and-lambdas
test_case.py:6: error: The last parameter to Concatenate needs to be a ParamSpec  [valid-type]
test_case.py:6: error: ParamSpec "P" is unbound  [valid-type]
test_case.py:7: error: The first argument to Callable must be a list of types, parameter specification, or "..."  [valid-type]
test_case.py:7: note: See https://mypy.readthedocs.io/en/stable/kinds_of_types.html#callable-types-and-lambdas

It's not clear to me what is wrong with my type aliases. The ParamSpec is last, as is required, and I am passing a Concatenate as the first part of the Callable, which also seems normal. I included the second half of the code to illustrate how these type aliases might be used in an actual annotation.

I ran the above code in pyright and it produced no errors. Further, if I manipulate the parameters of decorated_func such that mappings is not the first parameter, then pyright correctly fails the function. This is consistent with my expectation of the proper type checking semantics.

test_case.py:21:2 - error: Argument of type "(c: bool, d: float, mappings: Dict[str, str], a: int, b: str) -> str" cannot be assigned to parameter of type "PreWrapFunc[Unknown]"
    Type "(c: bool, d: float, mappings: Dict[str, str], a: int, b: str) -> str" cannot be assigned to type "PreWrapFunc[Unknown]"
      Parameter 1: type "Dict[str, str]" cannot be assigned to type "bool"
        "Dict[str, str]" is incompatible with "bool" (reportGeneralTypeIssues)
1 error, 0 warnings, 0 informations 

[erictraut]

Yes, this is a bug in mypy. Your code has some issues with the type annotations, but even when I fix those errors, mypy still generates errors. Mypy's support for PEP 612 is pretty new, so it's not surprising that it still has some bugs.

The problem in your code becomes evident if you turn on strict type checking in pyright (i.e. add # pyright: strict anywhere in the file). You have defined two generic type aliases called PreWrapFunc and PostWrapFunc. These type aliases each take a single type argument which must be compatible with a ParamSpec type parameter. But when you use these type aliases further down in the code, you are not providing a type argument. That means a type checker will assume you mean Any (or the moral equivalent of Any which is ... for a ParamSpec).

Here's the code with corrected type annotations. Note that it also uses P.args and P.kwargs for the parameter annotations in wrapped_func.

# pyright: strict

from typing import Callable, Dict
from typing_extensions import Concatenate, ParamSpec
from functools import wraps

P = ParamSpec("P")
PreWrapFunc = Callable[Concatenate[Dict[str, str], P], str]
PostWrapFunc = Callable[P, str]

def second_order_decorator(mappings: Dict[str, str]) -> Callable[[PreWrapFunc[P]], PostWrapFunc[P]]:
    def decorator(func: PreWrapFunc[P]) -> PostWrapFunc[P]:
        @wraps(func)
        def wrapped_func(*args: P.args, **kwargs: P.kwargs):
            return func(mappings, *args, **kwargs)
        return wrapped_func
    return decorator

@second_order_decorator(mappings={"a": "1", "b": "2"})
def decorated_func(mappings: Dict[str, str], a: int, b: str) -> str:
    return "|".join(k + ":" + v for k, v in mappings.items()) + str(a) + b