-
-
Notifications
You must be signed in to change notification settings - Fork 2.9k
Basic support for ParamSpec type checking #11594
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This comment has been minimized.
This comment has been minimized.
arg_type = self.named_generic_type('builtins.tuple', | ||
[arg_type]) | ||
arg_type = get_proper_type(arg_type) | ||
if not isinstance(arg_type, ParamSpecType): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you should update the proper_plugin to not require this. ParamSpecType
can never by a target of a type alias. (Note TypeVarType
is already excluded).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A good idea! Done. Also removed various now-redundant get_proper_type
calls.
param_spec = t.param_spec() | ||
if param_spec is not None: | ||
repl = get_proper_type(self.variables.get(param_spec.id)) | ||
if isinstance(repl, CallableType): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TBH it is hard to follow the logic here: could you please add a comment on why do we need to have some non-trivial logic for callable?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a comment.
mypy/typeanal.py
Outdated
finally: | ||
if nested: | ||
self.nesting_level -= 1 | ||
# Use type(...) to ignore proper/non-proper type distinction. | ||
if (not allow_param_spec | ||
and type(analyzed) is ParamSpecType |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this really needed? You can update the plugin instead as I suggested above.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Replaced with regular isinstance
.
def __eq__(self, other: object) -> bool: | ||
if not isinstance(other, ParamSpecType): | ||
return NotImplemented | ||
return self.id == other.id and self.flavor == other.flavor |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why upper_bound
is missing here and in various other places like is_same_type()
? Add a comment about this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not user-configurable and always depends on flavor (actually currently it's always object
). Added some comments.
mypy/fixup.py
Outdated
@@ -247,6 +248,9 @@ def visit_type_var(self, tvt: TypeVarType) -> None: | |||
if tvt.upper_bound is not None: | |||
tvt.upper_bound.accept(self) | |||
|
|||
def visit_param_spec(self, p: ParamSpecType) -> None: | |||
pass # Nothing to descend into. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about .upper_bound
? Also why it is ignored in similar fine-grained related logic?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I hadn't updated some places after I added upper_bound
. It's now processed here and fine-grained logic.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ParamSpec! 🎉 🎏
mypy/type_visitor.py
Outdated
@@ -63,6 +63,11 @@ def visit_deleted_type(self, t: DeletedType) -> T: | |||
def visit_type_var(self, t: TypeVarType) -> T: | |||
pass | |||
|
|||
@abstractmethod | |||
def visit_param_spec(self, t: ParamSpecType) -> T: | |||
assert False |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like an implementation artifact. Isn't it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I used this for testing. Removed the assert.
@@ -446,13 +447,63 @@ def deserialize(cls, data: JsonDict) -> 'TypeVarType': | |||
) | |||
|
|||
|
|||
class ParamSpecFlavor: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe Enum
? Like ArgKind
: https://github.com/python/mypy/blob/master/mypy/nodes.py#L1636
Sorry, Enum
s are really my thing 🙂
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tend to avoid enums since mypyc only partially supports them, so it's possible to accidentally write code that it's pretty slow.
def m(self, *args: P.args, **kwargs: P.kwargs) -> int: | ||
return 1 | ||
|
||
c: C[Any] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would any other non-Any
type arguments work? I haven't see any examples of it in tests 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not implemented. I've mentioned it in the commit summary as follow-up work.
Thanks for the feedback! I think that I've addressed all the comments (either by updating the PR or by leaving a reply). |
Diff from mypy_primer, showing the effect of this PR on open source code: spark (https://github.com/apache/spark.git)
+ python/pyspark/shell.py:49: error: Cannot infer type of lambda [misc]
tornado (https://github.com/tornadoweb/tornado.git)
+ tornado/util.py:55: error: Cannot infer type of lambda
|
This is still broken: import atexit
def f() -> None: pass
atexit.register(lambda: f()) # Cannot infer type of lambda I'll fix this in a follow-up PR. |
If ParamSpec is in the context of a lambda, treat it similar to `Callable[..., Any]`. This allows us to infer at least argument counts and kinds. Types can't be inferred since that would require "backwards" type inference, which we don't support. Follow-up to #11594.
If ParamSpec is in the context of a lambda, treat it similar to `Callable[..., Any]`. This allows us to infer at least argument counts and kinds. Types can't be inferred since that would require "backwards" type inference, which we don't support. Follow-up to #11594.
Check that the correct ParamSpec and flavor are used in `*args` and `**kwargs`. Follow-up to #11594.
Check that the correct ParamSpec and flavor are used in `*args` and `**kwargs`. Follow-up to #11594.
Add support for type checking several ParamSpec use cases (PEP 612). @hauntsaninja previously added support for semantic analysis of ParamSpec definitions, and this builds on top that foundation. The implementation has these main things going on: * `ParamSpecType` that is similar to `TypeVarType` but has three "flavors" that correspond to `P`, `P.args` and `P.kwargs` * `CallableType` represents `Callable[P, T]` if the arguments are (`*args: P.args`, `**kwargs: P.kwargs`) -- and more generally, there can also be arbitrary additional prefix arguments * Type variables of functions and classes can now be represented using `ParamSpecType` in addition to `TypeVarType` There are still a bunch of TODOs. Some of these are important to address before the release that includes this. I believe that this is good enough to merge and remaining issues can be fixed in follow-up PRs. Notable missing features include these: * `Concatenate` * Specifying the value of ParamSpec explicitly (e.g. `Z[[int, str, bool]]`) * Various validity checks -- currently only some errors are caught * Special case of decorating a method (python/typeshed#6347) * `atexit.register(lambda: ...)` generates an error
If ParamSpec is in the context of a lambda, treat it similar to `Callable[..., Any]`. This allows us to infer at least argument counts and kinds. Types can't be inferred since that would require "backwards" type inference, which we don't support. Follow-up to python#11594.
Check that the correct ParamSpec and flavor are used in `*args` and `**kwargs`. Follow-up to python#11594.
Add support for type checking several ParamSpec use cases (PEP 612).
@hauntsaninja previously added support for semantic analysis of ParamSpec
definitions, and this builds on top that foundation.
The implementation has these main things going on:
ParamSpecType
that is similar toTypeVarType
but has three "flavors" thatcorrespond to
P
,P.args
andP.kwargs
CallableType
representsCallable[P, T]
if the arguments are(
*args: P.args
,**kwargs: P.kwargs
) -- and more generally, there can alsobe arbitrary additional prefix arguments
ParamSpecType
in addition toTypeVarType
There are still a bunch of TODOs. Some of these are important to address before the
release that includes this. I believe that this is good enough to merge and remaining
issues can be fixed in follow-up PRs.
Notable missing features include these:
Concatenate
Z[[int, str, bool]]
)