-
Notifications
You must be signed in to change notification settings - Fork 1.7k
[pyflakes/pylint] Skip __class__ checks in method definitions (…
#19783
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[pyflakes/pylint] Skip __class__ checks in method definitions (…
#19783
Conversation
|
ntBre
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for working on this!
This looks like it fixes the issue for these two rules. Did you look into my suggestion about synthesizing a __class__ binding for class scopes (#18442 (comment))? I'm still not sure if that's a good idea or not, but it could be nice to resolve this more generally instead of adding special cases to individual rules as needed, and that seemed like one approach.
In grepping around, it looks like we might already have handling for this, if we can call this enclosing resolve_load function:
ruff/crates/ruff_python_semantic/src/model.rs
Lines 412 to 422 in 4d57fcd
| if scope.kind.is_class() { | |
| // Allow usages of `__class__` within methods, e.g.: | |
| // | |
| // ```python | |
| // class Foo: | |
| // def __init__(self): | |
| // print(__class__) | |
| // ``` | |
| if seen_function && matches!(name.id.as_str(), "__class__") { | |
| return ReadResult::ImplicitGlobal; | |
| } |
If neither of these solutions looks fruitful, I'm happy with what you have here too :)
|
|
||
| // Walk up the scope hierarchy to find a class scope, skipping Type scopes | ||
| let mut current_scope_id = binding.scope; | ||
| while let Some(parent_scope_id) = semantic.parent_scope_id(current_scope_id) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We might be able to use something like this here for traversing the parent scopes:
ruff/crates/ruff_linter/src/checkers/ast/mod.rs
Lines 702 to 708 in 4d57fcd
| for scope in self.semantic.current_scopes() { | |
| match scope.kind { | |
| ScopeKind::Class(_) | ScopeKind::Lambda(_) => return false, | |
| ScopeKind::Function(ast::StmtFunctionDef { is_async, .. }) => return *is_async, | |
| ScopeKind::Generator { .. } | ScopeKind::Module | ScopeKind::Type => {} | |
| } | |
| } |
|
@ntBre thanks for the good comments! What would you think about this approach, in which we add a method to the semantic model that would expose implicit global functionality? Something like this: impl<'a> SemanticModel<'a> {
/// Check if a name would resolve as an implicit global from a given scope
/// without actually creating references or modifying state.
pub fn would_resolve_as_implicit_global(&self, name: &str, scope_id: ScopeId) -> bool {
let scope = &self.scopes[scope_id];
match name {
"__class__" => {
if !scope.kind.is_function() {
return false;
}
let mut seen_function = false;
for ancestor_scope_id in self.ancestor_scopes(scope_id) {
let ancestor_scope = &self.scopes[ancestor_scope_id];
if ancestor_scope.kind.is_class() && seen_function {
return true;
}
seen_function |= ancestor_scope.kind.is_function();
}
false
}
"__module__" | "__qualname__" => {
scope.kind.is_class()
}
_ => false,
}
}
}
// RULE
pub(crate) fn unused_variable(checker: &Checker, name: &str, binding: &Binding) {
if binding.is_unpacked_assignment() {
return;
}
if checker
.semantic()
.would_resolve_as_implicit_global(name, binding.scope)
{
return;
}
/*// Skip __class__ in method definitions - it's automatically bound by Python
if name == "__class__" && is_binding_in_method_definition(checker, binding) {
return;
}*/
let mut diagnostic = checker.report_diagnostic(
UnusedVariable {
name: name.to_string(),
},
binding.range(),
);
if let Some(fix) = remove_unused_variable(binding, checker) {
diagnostic.set_fix(fix);
}
}
With this, the implementation of both rules becomes trivial, and we have something more general. I think now it solves this beautifully. |
|
I haven’t reviewed the code but I have three responses to the opening comment. def set_class(self, cls):
__class__ = cls # No longer warns about unused variableThere actually should be an F841 warning because def get_class(self):
nonlocal __class__ # No longer warns about missing binding
return __class__There should be a warning because the Finally, the original issue used F841 and PLE0117 as examples. The fix should probably be at a more fundamental level in the code about bindings and scopes, affecting all rules. |
|
Thanks @dscorbett, it sounds like the fix here is not quite right. @mikeleppane I'm not sure adding this method to the semantic model would be very helpful, at least as used in this snippet. We'd still have to be careful to call it in each affected rule, so it's not much of a step up from the type of special-casing in this PR. That's why I'm still intrigued by the synthetic binding approach. It would be a small special case when creating new scopes, and then all of our other infrastructure should continue working as usual. That may still not work, though, because >>> class C:
... def f():
... x = 1
... nonlocal x
... return x
...
File "<python-input-5>", line 4
nonlocal x
^^^^^^^^^^
SyntaxError: name 'x' is assigned to before nonlocal declarationwhich seems analogous to the I think I've talked myself out of the synthetic binding, so maybe a new semantic model method is called for, but ideally we'd call it within existing semantic model methods so that every rule benefits from it automatically. It may also be worth checking how ty handles this. I didn't turn up anything obviously helpful in a very quick grep, but I suspect that they may have an answer somewhere. We could push a synthetic scope between the class and the method, which seems closest to how this is implemented in CPython, but then we have to skip that scope kind everywhere else, which doesn't seem worth it. |
Summary -- This PR aims to resolve (or help to resolve) #18442 and #19357 by encoding the CPython semantics around the `__class__` cell in our semantic model. Namely, > __class__ is an implicit closure reference created by the compiler if any methods in a class body refer to either __class__ or super. from the Python [docs](https://docs.python.org/3/reference/datamodel.html#creating-the-class-object). As noted in the code comment by @AlexWaygood, we don't fully model this behavior, opting always to create the `__class__` cell binding in a new `ScopeKind::ClassCell` around each method definition, without checking if any method in the class body actually refers to `__class__` or `super`. As such, this PR may only help with #18442 and not #19357. Test Plan -- Existing tests, plus (TODO) tests from #19783 and #19424, which tackled the issues above on a per-rule basis. Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Summary -- This PR aims to resolve (or help to resolve) #18442 and #19357 by encoding the CPython semantics around the `__class__` cell in our semantic model. Namely, > __class__ is an implicit closure reference created by the compiler if any methods in a class body refer to either __class__ or super. from the Python [docs](https://docs.python.org/3/reference/datamodel.html#creating-the-class-object). As noted in the code comment by @AlexWaygood, we don't fully model this behavior, opting always to create the `__class__` cell binding in a new `ScopeKind::ClassCell` around each method definition, without checking if any method in the class body actually refers to `__class__` or `super`. As such, this PR may only help with #18442 and not #19357. Test Plan -- Existing tests, plus (TODO) tests from #19783 and #19424, which tackled the issues above on a per-rule basis. Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Mikko Leppänen <mleppan23@gmail.com>
these don't change any existing test results, and also don't seem to help with the F841 failures in the tests from #19783, but they still seem correct and in line with our other changes
Summary -- This PR aims to resolve (or help to resolve) #18442 and #19357 by encoding the CPython semantics around the `__class__` cell in our semantic model. Namely, > `__class__` is an implicit closure reference created by the compiler if any methods in a class body refer to either `__class__` or super. from the Python [docs](https://docs.python.org/3/reference/datamodel.html#creating-the-class-object). As noted in the variant docs by @AlexWaygood, we don't fully model this behavior, opting always to create the `__class__` cell binding in a new `ScopeKind::DunderClassCell` around each method definition, without checking if any method in the class body actually refers to `__class__` or `super`. As such, this PR fixes #18442 but not #19357. Test Plan -- Existing tests, plus the tests from #19783, which now pass without any rule-specific code. Note that we opted not to alter the behavior of F841 here because flagging `__class__` in these cases still seems helpful. See the discussion in #20048 (comment) and in the test comments for more information. --------- Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Mikko Leppänen <mleppan23@gmail.com>
|
Thanks again for your work here! I think we can close this now in favor of #20048. I pulled in your tests over there and added you as a co-author :) |
Summary -- This PR aims to resolve (or help to resolve) astral-sh#18442 and astral-sh#19357 by encoding the CPython semantics around the `__class__` cell in our semantic model. Namely, > `__class__` is an implicit closure reference created by the compiler if any methods in a class body refer to either `__class__` or super. from the Python [docs](https://docs.python.org/3/reference/datamodel.html#creating-the-class-object). As noted in the variant docs by @AlexWaygood, we don't fully model this behavior, opting always to create the `__class__` cell binding in a new `ScopeKind::DunderClassCell` around each method definition, without checking if any method in the class body actually refers to `__class__` or `super`. As such, this PR fixes astral-sh#18442 but not astral-sh#19357. Test Plan -- Existing tests, plus the tests from astral-sh#19783, which now pass without any rule-specific code. Note that we opted not to alter the behavior of F841 here because flagging `__class__` in these cases still seems helpful. See the discussion in astral-sh#20048 (comment) and in the test comments for more information. --------- Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Mikko Leppänen <mleppan23@gmail.com>
Summary
Problem
Rules F841 (
unused-variable) and PLE0117 (nonlocal-without-binding) incorrectly flagged usage within method definitions, causing false positives.Root Cause: Both rules failed to recognize that it is a special variable automatically available in Python method definitions. In methods, it is implicitly bound by the Python runtime, making assignments to it legitimate and declarations valid.
__class__``__class__``nonlocal __class__Solution
__class____class__nonlocal __class__Test Plan
cargo test