Skip to content

Add a class attribute hook to the plugin system #9881

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
May 7, 2022
4 changes: 4 additions & 0 deletions docs/source/extending_mypy.rst
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,10 @@ fields which already exist on the class. *Exception:* if :py:meth:`__getattr__ <
:py:meth:`__getattribute__ <object.__getattribute__>` is a method on the class, the hook is called for all
fields which do not refer to methods.

**get_class_attribute_hook()** is similar to above, but for attributes on classes rather than instances.
Unlike above, this does not have special casing for :py:meth:`__getattr__ <object.__getattr__>` or
:py:meth:`__getattribute__ <object.__getattribute__>`.

**get_class_decorator_hook()** can be used to update class definition for
given class decorators. For example, you can add some attributes to the class
to match runtime behaviour:
Expand Down
28 changes: 22 additions & 6 deletions mypy/checkmember.py
Original file line number Diff line number Diff line change
Expand Up @@ -703,10 +703,13 @@ def analyze_class_attribute_access(itype: Instance,
if override_info:
info = override_info

fullname = '{}.{}'.format(info.fullname, name)
hook = mx.chk.plugin.get_class_attribute_hook(fullname)

node = info.get(name)
if not node:
if info.fallback_to_any:
return AnyType(TypeOfAny.special_form)
return apply_class_attr_hook(mx, hook, AnyType(TypeOfAny.special_form))
return None

is_decorated = isinstance(node.node, Decorator)
Expand All @@ -731,14 +734,16 @@ def analyze_class_attribute_access(itype: Instance,
if info.is_enum and not (mx.is_lvalue or is_decorated or is_method):
enum_class_attribute_type = analyze_enum_class_attribute_access(itype, name, mx)
if enum_class_attribute_type:
return enum_class_attribute_type
return apply_class_attr_hook(mx, hook, enum_class_attribute_type)

t = node.type
if t:
if isinstance(t, PartialType):
symnode = node.node
assert isinstance(symnode, Var)
return mx.chk.handle_partial_var_type(t, mx.is_lvalue, symnode, mx.context)
return apply_class_attr_hook(mx, hook,
mx.chk.handle_partial_var_type(t, mx.is_lvalue, symnode,
mx.context))

# Find the class where method/variable was defined.
if isinstance(node.node, Decorator):
Expand Down Expand Up @@ -789,7 +794,8 @@ def analyze_class_attribute_access(itype: Instance,
mx.self_type, original_vars=original_vars)
if not mx.is_lvalue:
result = analyze_descriptor_access(result, mx)
return result

return apply_class_attr_hook(mx, hook, result)
elif isinstance(node.node, Var):
mx.not_ready_callback(name, mx.context)
return AnyType(TypeOfAny.special_form)
Expand All @@ -813,7 +819,7 @@ def analyze_class_attribute_access(itype: Instance,
if is_decorated:
assert isinstance(node.node, Decorator)
if node.node.type:
return node.node.type
return apply_class_attr_hook(mx, hook, node.node.type)
else:
mx.not_ready_callback(name, mx.context)
return AnyType(TypeOfAny.from_error)
Expand All @@ -825,7 +831,17 @@ def analyze_class_attribute_access(itype: Instance,
# unannotated implicit class methods we do this here.
if node.node.is_class:
typ = bind_self(typ, is_classmethod=True)
return typ
return apply_class_attr_hook(mx, hook, typ)


def apply_class_attr_hook(mx: MemberContext,
hook: Optional[Callable[[AttributeContext], Type]],
result: Type,
) -> Optional[Type]:
if hook:
result = hook(AttributeContext(get_proper_type(mx.original_type),
result, mx.context, mx.chk))
return result


def analyze_enum_class_attribute_access(itype: Instance,
Expand Down
29 changes: 26 additions & 3 deletions mypy/plugin.py
Original file line number Diff line number Diff line change
Expand Up @@ -632,10 +632,10 @@ def get_method_hook(self, fullname: str

def get_attribute_hook(self, fullname: str
) -> Optional[Callable[[AttributeContext], Type]]:
"""Adjust type of a class attribute.
"""Adjust type of an instance attribute.

This method is called with attribute full name using the class where the attribute was
defined (or Var.info.fullname for generated attributes).
This method is called with attribute full name using the class of the instance where
the attribute was defined (or Var.info.fullname for generated attributes).

For classes without __getattr__ or __getattribute__, this hook is only called for
names of fields/properties (but not methods) that exist in the instance MRO.
Expand All @@ -662,6 +662,25 @@ class Derived(Base):
"""
return None

def get_class_attribute_hook(self, fullname: str
) -> Optional[Callable[[AttributeContext], Type]]:
"""
Adjust type of a class attribute.

This method is called with attribute full name using the class where the attribute was
defined (or Var.info.fullname for generated attributes).

For example:

class Cls:
x: Any

Cls.x

get_class_attribute_hook is called with '__main__.Cls.x' as fullname.
"""
return None

def get_class_decorator_hook(self, fullname: str
) -> Optional[Callable[[ClassDefContext], None]]:
"""Update class definition for given class decorators.
Expand Down Expand Up @@ -783,6 +802,10 @@ def get_attribute_hook(self, fullname: str
) -> Optional[Callable[[AttributeContext], Type]]:
return self._find_hook(lambda plugin: plugin.get_attribute_hook(fullname))

def get_class_attribute_hook(self, fullname: str
) -> Optional[Callable[[AttributeContext], Type]]:
return self._find_hook(lambda plugin: plugin.get_class_attribute_hook(fullname))

def get_class_decorator_hook(self, fullname: str
) -> Optional[Callable[[ClassDefContext], None]]:
return self._find_hook(lambda plugin: plugin.get_class_decorator_hook(fullname))
Expand Down
89 changes: 89 additions & 0 deletions test-data/unit/check-custom-plugin.test
Original file line number Diff line number Diff line change
Expand Up @@ -902,3 +902,92 @@ reveal_type(f()) # N: Revealed type is "builtins.str"
[file mypy.ini]
\[mypy]
plugins=<ROOT>/test-data/unit/plugins/method_in_decorator.py

[case testClassAttrPluginClassVar]
# flags: --config-file tmp/mypy.ini

from typing import Type

class Cls:
attr = 'test'
unchanged = 'test'

reveal_type(Cls().attr) # N: Revealed type is "builtins.str"
reveal_type(Cls.attr) # N: Revealed type is "builtins.int"
reveal_type(Cls.unchanged) # N: Revealed type is "builtins.str"
x: Type[Cls]
reveal_type(x.attr) # N: Revealed type is "builtins.int"
[file mypy.ini]
\[mypy]
plugins=<ROOT>/test-data/unit/plugins/class_attr_hook.py

[case testClassAttrPluginMethod]
# flags: --config-file tmp/mypy.ini

class Cls:
def attr(self) -> None:
pass

reveal_type(Cls.attr) # N: Revealed type is "builtins.int"
[file mypy.ini]
\[mypy]
plugins=<ROOT>/test-data/unit/plugins/class_attr_hook.py

[case testClassAttrPluginEnum]
# flags: --config-file tmp/mypy.ini

import enum

class Cls(enum.Enum):
attr = 'test'

reveal_type(Cls.attr) # N: Revealed type is "builtins.int"
[file mypy.ini]
\[mypy]
plugins=<ROOT>/test-data/unit/plugins/class_attr_hook.py

[case testClassAttrPluginMetaclassAnyBase]
# flags: --config-file tmp/mypy.ini

from typing import Any, Type
class M(type):
attr = 'test'

B: Any
class Cls(B, metaclass=M):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to use one more test example with real B, because subtyping Any is not the same as subtyping a regular class.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added!

pass

reveal_type(Cls.attr) # N: Revealed type is "builtins.int"
[file mypy.ini]
\[mypy]
plugins=<ROOT>/test-data/unit/plugins/class_attr_hook.py

[case testClassAttrPluginMetaclassRegularBase]
# flags: --config-file tmp/mypy.ini

from typing import Any, Type
class M(type):
attr = 'test'

class B:
attr = None

class Cls(B, metaclass=M):
pass

reveal_type(Cls.attr) # N: Revealed type is "builtins.int"
[file mypy.ini]
\[mypy]
plugins=<ROOT>/test-data/unit/plugins/class_attr_hook.py

[case testClassAttrPluginPartialType]
# flags: --config-file tmp/mypy.ini

class Cls:
attr = None
def f(self) -> int:
return Cls.attr + 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about self.__class__.attr? Should we support that?

I think we can also check that c: Type[Cls]; c.attr works correctly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, so it looks like adding a test for __class__ is actually somewhat difficult, but also unclear that it actually adds much new value here. In particular it looks like the __class__ being specified as generic is done in typeshed here: https://github.com/python/typeshed/blob/master/stdlib/builtins.pyi#L87-L88

The way this is done in other tests is via [builtins fixtures/__new__.pyi] but that just defines __class__ = object and isn't generic. But even if we were able to use the real stubs, then really all that __class__ would be testing is that the generics system+stubs work correctly, and less about the class attribute plugin itself. But if you think testing the generic __class__ is important I can add the stub manually within this test!

The other one is already tested above:

x: Type[Cls]
reveal_type(x.attr) # N: Revealed type is "builtins.int"


[file mypy.ini]
\[mypy]
plugins=<ROOT>/test-data/unit/plugins/class_attr_hook.py
20 changes: 20 additions & 0 deletions test-data/unit/plugins/class_attr_hook.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
from typing import Callable, Optional

from mypy.plugin import AttributeContext, Plugin
from mypy.types import Type as MypyType


class ClassAttrPlugin(Plugin):
def get_class_attribute_hook(self, fullname: str
) -> Optional[Callable[[AttributeContext], MypyType]]:
if fullname == '__main__.Cls.attr':
return my_hook
return None


def my_hook(ctx: AttributeContext) -> MypyType:
return ctx.api.named_generic_type('builtins.int', [])


def plugin(_version: str):
return ClassAttrPlugin