Feature - Partial Type Specifications For Callable

[rmorshea]

Feature Request

It would be awesome if it were possible to create a partial type spec for Callable. For example, I might want to be able to specify that the first argument of a function must be an integer, but that any other parameters are allowed. This is useful if parameters are being curried to a function as in the decorator cast_first_arg decorator below where the only requirement of func is that it's first argument is an int:

@cast_first_arg(int)  
def func(x: int, y: str) -> Any:
    # First positional argument is always cast to an integer.
    # All other parameters are passed through without modification.
    ...

At the moment, the only alternative to a "partial callable type spec" is to use VarArg and KwArg, however this is problematic since, in the case of cast_first_arg, the function doesn't actually need to accept variadic positional or keyword arguments.

Proposed Syntax

To provide a partial argument specification one should use ... before, between, or after any extended callable types. Using this syntax the type spec for cast_first_arg might look like this:

T = TypeVar("T", bound=Type)
F = Callable[[Var(int), ...], Any]

def cast_first_arg(cls: T) -> Callable[F, Callable[..., Any]]:
    def decorator(func: F) -> Callable[..., Any]:
        def wrapper(first: Any, *args: Any, **kwargs: Any): Any:
            return func(cls(first), *args, **kwargs)
       return wrapper
    return decorator

Ellipsis After

When ... follows some number of argument specifications should indicate that any other function parameters not explicitly indicated therein is allowed.

FirstArgIsInt = Callable[[Arg(int), ...], Any]

def function(a: int, b: str) -> Any: ...    # 'b' is an unspecified, but allowe

f: FirstArgIsInt = function

Ellipsis Before

Similarly if ... is placed before some number of argument specifications then only the last argument will have been specifically defined:

LastArgIsInt = Callable[[..., Arg(int)], Any]

def function(b: str, a: int) -> Any: ...    # 'b' is an unspecified, but allowed

f: LastArgIsInt = function

Ellipsis Between

Lastly ... placed between argument specifications should indicate that any parameters between the two explicitly defined parameters are allowed:

FirstAndLastAreInt = Callable[[Arg(int), ..., Arg(int)], Any]

def function(a: int, b: str, c: int) -> Any: ...    # 'b' is an unspecified, but allowed

f: FirstAndLastAreInt = function

Multiple Ellipsis

This could add complications and edge cases to the feature implementation so perhaps this could be left for a later iteration of partial callable type specifications, however this syntax enables you to describe an even wider variety of functions:

HasArgNamedC = Callable[[..., Arg(Any, "c"), ...], Any]

[msullivan]

The recommended way to do weird stuff with callable types is to use callback protocols: https://mypy.readthedocs.io/en/latest/protocols.html#callback-protocols, which we usually prefer to the experimental extended callables.

That said this doesn't support what you are trying to do here, I think. The best current approach involves writing a lot of overloads.

Would https://www.python.org/dev/peps/pep-0612/ support your use cases?

[rmorshea]

@msullivan unfortunately there is no way to achieve the desired effect using overloads as there's no way I could enumerate all the possibilities, nor does PEP-612 appear to cover this since it doesn't actually apply any restrictions on the callable being passed into the generic function. This feature request might actually dovetail nicely with PEP-612 though if ParameterSpecification were to accept a bound parameter similar to a TypeVar.

[msullivan]

I'm not sure I fully understand your use case here for all of these features

[rmorshea]

@msullivan honestly the main one I'm after is the ability describe a function whose first N parameters are well defined, but where the remainder do not matter. The other features described here simply fall out nicely from the syntax.

A specific use case for this new syntax might be a RequestHandler function for a web server framework that receives a Request object as its first parameter and an unknown series of arguments which are defined by the route URI it is meant to handle (similar to the view functions mentioned in this comment):

def handle_my_request(r: Request, param_1: int, param_2: str) -> str: ...

Now imagine that we want to develop a route decorator that will convert RequestHandlers into Route objects that are understood by the web server framework. How would we write a Callable type such that we can appropriately constrain the kinds of this functions the decorator can be applied to?

Unfortunately there is no way to accomplish this. As a result I'm proposing that ... be used to indicate that the remainder of a functions parameters can be ignored for the purposes of type checking. With this in mind we could the create a route decorator:

RequestHandler = Callable[[Request, ...], Any]  # <-- request the new syntax

def route(uri: str) -> Callable[[RequestHandler], Route]:
    def decorator(function: RequestHandler) -> Route:
        ...
    return decorator

@route("/{param_1}/{param_2}")
def handle_my_request(r: Request, param_1: int, param_2: str) -> str: ...

[JukkaL]

One approach for supporting use cases would be an extension to PEP 612, such as making it possible to concatenate argument lists and ParameterSpecifications. In any case, it seems that supporting this would require an extension to the PEP 484 syntax, and these should be discussed at https://github.com/python/typing and/or typing-sig@. If some approach attracts support there, you can create another issue here (or we can reopen this issue).

[rmorshea]

python/typing#696