Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic support for decorated overloads #15898

Merged
merged 6 commits into from
Aug 18, 2023

Conversation

ilevkivskyi
Copy link
Member

Fixes #15737
Fixes #12844
Fixes #12716

My goal was to fix the ParamSpec issues, but it turns out decorated overloads were not supported at all. Namely:

  • Decorators on overload items were ignored, caller would see original undecorated item types
  • Overload item overlap checks were performed for original types, while arguably we should use decorated types
  • Overload items completeness w.r.t. to implementation was checked with decorated implementation, and undecorated items

Here I add basic support using same logic as for regular decorated functions: initially set type to None and defer callers until definition is type-checked. Note this results in few more Cannot determine type in case of other errors, but I think it is fine.

Note I also add special-casing for "inline" applications of generic functions to overload arguments. This use case was mentioned few times alongside overloads. The general fix would be tricky, and my special-casing should cover typical use cases.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@ilevkivskyi
Copy link
Member Author

Just to confirm: new errors in asynq are correct, they use a decorator like Callable[..., T] -> Callable[..., T] which obviously erased overload item signatures.

Copy link
Collaborator

@hauntsaninja hauntsaninja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@@ -1450,6 +1501,15 @@ def check_call(
"""
callee = get_proper_type(callee)

overloaded = self.is_generic_decorator_overload_call(callee, args)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: check_call is pretty hot, should we move this inside if isinstance(callee, CallableType): on L1513?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, good idea!

@github-actions
Copy link
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

Expression (https://github.com/cognitedata/Expression)
- expression/extra/parser.py:81: error: No overload variant of "starmap" matches argument type "Callable[..., Any]"  [call-overload]
- expression/extra/parser.py:81: note: Possible overload variants:
- expression/extra/parser.py:81: note:     def [_A, _B, _C] starmap(mapper: Callable[[_A, _B], _C], parser: Parser[tuple[_A, _B]]) -> Parser[_C]
- expression/extra/parser.py:81: note:     def [_A, _B, _C, _D] starmap(mapper: Callable[[_A, _B, _C], _D], parser: Parser[tuple[_A, _B, _C]]) -> Parser[_D]
- expression/extra/parser.py:219: error: Overloaded function implementation does not accept all possible arguments of signature 1  [misc]
- expression/extra/parser.py:219: error: Overloaded function implementation does not accept all possible arguments of signature 2  [misc]
+ expression/extra/parser.py:435: error: Cannot infer type argument 3 of "pipe"  [misc]
+ expression/extra/parser.py:440: error: Argument 1 to "starmap" has incompatible type "Callable[[Some[str] | Nothing_[str], Block[str]], int]"; expected "Callable[[_A, _B], _C]"  [arg-type]
- expression/extra/parser.py:440: error: No overload variant of "starmap" matches argument type "Callable[[Some[str] | Nothing_[str], Block[str]], int]"  [call-overload]
- expression/extra/parser.py:440: note: Possible overload variants:
- expression/extra/parser.py:440: note:     def [_A, _B, _C] starmap(mapper: Callable[[_A, _B], _C], parser: Parser[tuple[_A, _B]]) -> Parser[_C]
- expression/extra/parser.py:440: note:     def [_A, _B, _C, _D] starmap(mapper: Callable[[_A, _B, _C], _D], parser: Parser[tuple[_A, _B, _C]]) -> Parser[_D]
+ expression/extra/parser.py:469: error: Cannot infer type argument 2 of "pipe"  [misc]
+ expression/extra/parser.py:480: error: Argument 1 to "starmap" has incompatible type "Callable[[tuple[Some[str] | Nothing_[str], Block[str]], Some[Block[str]] | Nothing_[Block[str]]], float]"; expected "Callable[[_A, _B], _C]"  [arg-type]
- expression/extra/parser.py:480: error: No overload variant of "starmap" matches argument type "Callable[[tuple[Some[str] | Nothing_[str], Block[str]], Some[Block[str]] | Nothing_[Block[str]]], float]"  [call-overload]
- expression/extra/parser.py:480: note: Possible overload variants:
- expression/extra/parser.py:480: note:     def [_A, _B, _C] starmap(mapper: Callable[[_A, _B], _C], parser: Parser[tuple[_A, _B]]) -> Parser[_C]
- expression/extra/parser.py:480: note:     def [_A, _B, _C, _D] starmap(mapper: Callable[[_A, _B, _C], _D], parser: Parser[tuple[_A, _B, _C]]) -> Parser[_D]

asynq (https://github.com/quora/asynq)
+ asynq/tools.pyi:42: error: Overloaded function signature 2 will never be matched: signature 1's parameter type(s) are the same or broader  [misc]
+ asynq/tools.pyi:48: error: Overloaded function signature 2 will never be matched: signature 1's parameter type(s) are the same or broader  [misc]

@ilevkivskyi ilevkivskyi merged commit fa84534 into python:master Aug 18, 2023
17 checks passed
@ilevkivskyi ilevkivskyi deleted the special-overload branch August 18, 2023 13:25
@maffoo
Copy link

maffoo commented Oct 3, 2023

I'm eager to use this feature, since it resolves #15737. Any estimate on when it might be included in a release version of mypy, e.g. a 1.5.2 release?

@AlexWaygood
Copy link
Member

I'm eager to use this feature, since it resolves #15737. Any estimate on when it might be included in a release version of mypy, e.g. a 1.5.2 release?

Follow #15919 for updates on the next feature release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants