Skip to content

Conversation

@AlexWaygood
Copy link
Member

Summary

This PR adds the necessary machinery to infer the members of a protocol class, and uses that machinery to infer a precise return type for the get_protocol_members introspection function.

What this PR doesn't do is infer what the type of each protocol member is. We'll obviously have to do that in due course to properly implement protocol subtyping and assignability; when we add that functionality, I'm envisaging that ProtocolClassLiteral::members() would return a HashMap mapping from the member to its type rather than a boxed slice of Names. I'm deliberately not doing that right now, however, as implementing all the subtyping rules around protocols is a somewhat large task, which I intend to approach incrementally over several PRs. The first stage will possibly not even consider the types of the protocol members at all.

Test Plan

Existing mdtests updated.

@AlexWaygood AlexWaygood added the ty Multi-file analysis & type inference label Apr 22, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Apr 22, 2025

mypy_primer results

No ecosystem changes detected ✅

@AlexWaygood AlexWaygood force-pushed the alex/get-protocol-members-validation branch from 9255038 to 0d68ffd Compare April 22, 2025 16:44
@AlexWaygood AlexWaygood force-pushed the alex/protocol-member-inference branch 2 times, most recently from a08a191 to afb9ea0 Compare April 22, 2025 16:49
@AlexWaygood AlexWaygood force-pushed the alex/get-protocol-members-validation branch from 0d68ffd to 4ded3fb Compare April 22, 2025 19:02
@AlexWaygood AlexWaygood force-pushed the alex/protocol-member-inference branch from afb9ea0 to 8906bc9 Compare April 22, 2025 19:03
@AlexWaygood AlexWaygood force-pushed the alex/get-protocol-members-validation branch from 4ded3fb to ece0c86 Compare April 22, 2025 19:07
@AlexWaygood AlexWaygood force-pushed the alex/protocol-member-inference branch from 8906bc9 to b4e8e97 Compare April 22, 2025 19:08
@AlexWaygood AlexWaygood force-pushed the alex/get-protocol-members-validation branch from ece0c86 to b3a1538 Compare April 22, 2025 19:35
@AlexWaygood AlexWaygood force-pushed the alex/protocol-member-inference branch from b4e8e97 to 1809727 Compare April 22, 2025 19:36
@AlexWaygood AlexWaygood force-pushed the alex/get-protocol-members-validation branch from b3a1538 to e34384c Compare April 22, 2025 19:44
@AlexWaygood AlexWaygood force-pushed the alex/protocol-member-inference branch from 1809727 to 6aa7491 Compare April 22, 2025 19:44
pub(super) struct ProtocolClassLiteral<'db>(ClassLiteralType<'db>);

impl<'db> ProtocolClassLiteral<'db> {
pub(super) fn members(self, db: &'db dyn Db) -> &'db ordermap::set::Slice<Name> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep in mind (both in terms of method naming, and where you put what functionality) that autocomplete is definitely going to require the ability for ClassLiteralType itself to have a "list all members" method. And if ProtocolClassLiteral will be used inside a future Type::ProtocolInstance, it will probably also need to expose that list-all-members method, without all the Protocol-specific filtering. That could suggest some future refactoring to share some aspects of list-members functionality, but more concretely I think it suggests that we should reserve the method name members for the general "list all known members" that all types will need, and use something like protocol_members to distinguish this protocol-specific method.

Copy link
Member Author

@AlexWaygood AlexWaygood Apr 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

autocomplete

for some reason it took me a moment there to realise that you mean "the autocompletion feature we intend to build into the red-knot LSP for IDE users" rather than "autocompletion in the IDE now for red-knot developers" 😆

That could suggest some future refactoring to share some aspects of list-members functionality, but more concretely I think it suggests that we should reserve the method name members for the general "list all known members" that all types will need, and use something like protocol_members to distinguish this protocol-specific method.

Interesting -- I think they will end up doing nearly the same thing. I suppose the only difference between them really is that the members method I'm adding here (which I'll rename to protocol_members) should not include implicit instance attributes (these are illegal in protocol classes, and do not constitute protocol members). But the autocomplete feature probably should include those members:

class Foo(Protocol):
    x: int

    def __init__(self):
        self.y = 42  # we should emit an error here,
                     # and not infer `Foo` as having a protocol member `y`
                     # (it should only have a single member, `x`)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, makes sense. It seems like also the filtering-out of some "special" names is something we might not want to do for autocomplete?

@AlexWaygood AlexWaygood force-pushed the alex/get-protocol-members-validation branch from e34384c to 0c975b4 Compare April 23, 2025 10:09
Base automatically changed from alex/get-protocol-members-validation to main April 23, 2025 10:13
@AlexWaygood AlexWaygood force-pushed the alex/protocol-member-inference branch from 6aa7491 to a8fba56 Compare April 23, 2025 10:31
Comment on lines 1803 to 1832
members.extend(
use_def_map
.all_public_declarations()
.flat_map(|(symbol_id, declarations)| {
symbol_from_declarations(db, declarations)
.map(|symbol| (symbol_id, symbol))
})
.filter_map(|(symbol_id, symbol)| {
symbol.symbol.ignore_possibly_unbound().map(|_| symbol_id)
})
.chain(use_def_map.all_public_bindings().filter_map(
|(symbol_id, bindings)| {
symbol_from_bindings(db, bindings)
.ignore_possibly_unbound()
.map(|_| symbol_id)
},
))
.map(|symbol_id| symbol_table.symbol(symbol_id).name())
.filter(|name| !excluded_from_proto_members(name))
.cloned(),
);
}
Copy link
Member Author

@AlexWaygood AlexWaygood Apr 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in due course this will need to be converted to a HashMap of some kind where the keys are the names of the protocol members and the values are the types of the protocol members. But I don't want to do that yet because I'm not yet sure whether e.g. an attribute member annotated with Callable should be treated the same as a method member (I need to write some more tests...)

@AlexWaygood AlexWaygood requested review from carljm and sharkdp April 23, 2025 11:54
Copy link
Contributor

@carljm carljm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@AlexWaygood AlexWaygood force-pushed the alex/protocol-member-inference branch from e96e63f to 7ced9ed Compare April 23, 2025 21:31
@AlexWaygood AlexWaygood enabled auto-merge (squash) April 23, 2025 21:32
auto-merge was automatically disabled April 23, 2025 21:32

Pull request was closed

@AlexWaygood AlexWaygood reopened this Apr 23, 2025
@AlexWaygood AlexWaygood enabled auto-merge (squash) April 23, 2025 21:33
@AlexWaygood AlexWaygood merged commit 00e73dc into main Apr 23, 2025
31 checks passed
@AlexWaygood AlexWaygood deleted the alex/protocol-member-inference branch April 23, 2025 21:36
@carljm carljm mentioned this pull request Apr 24, 2025
5 tasks
dcreager added a commit that referenced this pull request Apr 24, 2025
* main:
  [red-knot] fix collapsing literal and its negation to object (#17605)
  [red-knot] Add more tests for protocols (#17603)
  [red-knot] Ban direct instantiations of `Protocol` classes (#17597)
  [`pyupgrade`] Preserve parenthesis when fixing native literals containing newlines (`UP018`) (#17220)
  [`airflow`] fix typos (`AIR302`, `AIR312`) (#17574)
  [red-knot] Special case `@abstractmethod` for function type (#17591)
  [red-knot] Emit diagnostics for isinstance() and issubclass() calls where a non-runtime-checkable protocol is the second argument (#17561)
  [red-knot] Infer the members of a protocol class (#17556)
  [red-knot] Add `FunctionType::to_overloaded` (#17585)
  [red-knot] Add mdtests for `global` statement (#17563)
  [syntax-errors] Make duplicate parameter names a semantic error (#17131)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants