Skip to content

Add Class Decorator/Metaclass/Base Class plugin #4328

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Dec 14, 2017
55 changes: 51 additions & 4 deletions mypy/plugin.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
from abc import abstractmethod
from typing import Callable, List, Tuple, Optional, NamedTuple, TypeVar

from mypy.nodes import Expression, StrExpr, IntExpr, UnaryExpr, Context, DictExpr
from mypy.nodes import Expression, StrExpr, IntExpr, UnaryExpr, Context, DictExpr, ClassDef
from mypy.types import (
Type, Instance, CallableType, TypedDictType, UnionType, NoneTyp, FunctionLike, TypeVarType,
AnyType, TypeList, UnboundType, TypeOfAny
Expand All @@ -13,7 +13,7 @@
from mypy.options import Options


class AnalyzerPluginInterface:
class TypeAnalyzerPluginInterface:
"""Interface for accessing semantic analyzer functionality in plugins."""

@abstractmethod
Expand All @@ -40,7 +40,7 @@ def analyze_callable_args(self, arglist: TypeList) -> Optional[Tuple[List[Type],
'AnalyzeTypeContext', [
('type', UnboundType), # Type to analyze
('context', Context),
('api', AnalyzerPluginInterface)])
('api', TypeAnalyzerPluginInterface)])


class CheckerPluginInterface:
Expand All @@ -53,6 +53,23 @@ def named_generic_type(self, name: str, args: List[Type]) -> Instance:
raise NotImplementedError


class SemanticAnalyzerPluginInterface:
"""Interface for accessing semantic analyzer functionality in plugins."""

@abstractmethod
def named_type(self, qualified_name: str, args: Optional[List[Type]] = None) -> Instance:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the only method that would be potentially used by plugins? Do we need to add some more?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My current explorations also need parse_bool and anal_type

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fail could be useful too. Should I add them as needed or add them now?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I add them as needed or add them now?

I am not sure why we actually need this class. Can't we just pass everything? Or am I missing something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm following the pattern of the other plugins. Technically the object that's passed in is the full object. I believe this base class is here to limit what methods the plugins should be calling. i.e. if the method isn't listed here then the mypy checker will complain about it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then we probably need to think what to add here. My expectation is that many methods of SemanticAnalyzer may be useful. You can play more with your implementation for attrs and add here whatever you ever used.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can keep only these methods, and then add more in your PR with the actual plugin (if necessary).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is why we are using an ABC instead exposing the entire semantic analyzer interface:

  • By only giving access to a subset of an interface, we declare the public part of the semantic analyzer interface. This has a few benefits related to modularity and loose coupling:
    • Everything else in the semantic analyzer can be more freely modified without breaking plugins. If we change the public interface, we may break 3rd party plugins. (Currently the plugin interface is still not officially supported so we can change things without worries, but I'd like to make the interface more stable in the future.)
    • It's less likely that plugins will rely on internal implementation details that are likely to change, or on poorly thought-out internal APIs that are hard to use correctly.
    • It's easier to reason about plugins in general, as the plugins are expected to stick to the exposed public interface instead of freely accessing mypy internals.
  • This can avoid a cyclic import dependency (maybe not right now, but it will help untangle cyclic dependencies). They slow down incremental checking.

raise NotImplementedError

@abstractmethod
def parse_bool(self, expr: Expression) -> Optional[bool]:
raise NotImplementedError

@abstractmethod
def fail(self, msg: str, ctx: Context, serious: bool = False, *,
blocker: bool = False) -> None:
raise NotImplementedError


# A context for a function hook that infers the return type of a function with
# a special signature.
#
Expand Down Expand Up @@ -98,6 +115,14 @@ def named_generic_type(self, name: str, args: List[Type]) -> Instance:
('context', Context),
('api', CheckerPluginInterface)])

# A context for a class hook that modifies the class definition.
ClassDefContext = NamedTuple(
'ClassDecoratorContext', [
('cls', ClassDef), # The class definition
('reason', Expression), # The expression being applied (decorator, metaclass, base class)
('api', SemanticAnalyzerPluginInterface)
])


class Plugin:
"""Base class of all type checker plugins.
Expand Down Expand Up @@ -136,7 +161,17 @@ def get_attribute_hook(self, fullname: str
) -> Optional[Callable[[AttributeContext], Type]]:
return None

# TODO: metaclass / class decorator hook
def get_class_decorator_hook(self, fullname: str
) -> Optional[Callable[[ClassDefContext], None]]:
return None

def get_metaclass_hook(self, fullname: str
) -> Optional[Callable[[ClassDefContext], None]]:
return None

def get_base_class_hook(self, fullname: str
) -> Optional[Callable[[ClassDefContext], None]]:
return None


T = TypeVar('T')
Expand Down Expand Up @@ -182,6 +217,18 @@ def get_attribute_hook(self, fullname: str
) -> Optional[Callable[[AttributeContext], Type]]:
return self._find_hook(lambda plugin: plugin.get_attribute_hook(fullname))

def get_class_decorator_hook(self, fullname: str
) -> Optional[Callable[[ClassDefContext], None]]:
return self._find_hook(lambda plugin: plugin.get_class_decorator_hook(fullname))

def get_metaclass_hook(self, fullname: str
) -> Optional[Callable[[ClassDefContext], None]]:
return self._find_hook(lambda plugin: plugin.get_metaclass_hook(fullname))

def get_base_class_hook(self, fullname: str
) -> Optional[Callable[[ClassDefContext], None]]:
return self._find_hook(lambda plugin: plugin.get_base_class_hook(fullname))

def _find_hook(self, lookup: Callable[[Plugin], T]) -> Optional[T]:
for plugin in self._plugins:
hook = lookup(plugin)
Expand Down
45 changes: 42 additions & 3 deletions mypy/semanal.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@
from mypy.sametypes import is_same_type
from mypy.options import Options
from mypy import experiments
from mypy.plugin import Plugin
from mypy.plugin import Plugin, ClassDefContext, SemanticAnalyzerPluginInterface
from mypy import join
from mypy.util import get_prefix

Expand Down Expand Up @@ -172,7 +172,7 @@
}


class SemanticAnalyzerPass2(NodeVisitor[None]):
class SemanticAnalyzerPass2(NodeVisitor[None], SemanticAnalyzerPluginInterface):
"""Semantically analyze parsed mypy files.

The analyzer binds names and does various consistency checks for a
Expand Down Expand Up @@ -719,9 +719,48 @@ def analyze_class_body(self, defn: ClassDef) -> Iterator[bool]:
yield True
self.calculate_abstract_status(defn.info)
self.setup_type_promotion(defn)

self.apply_class_plugin_hooks(defn)
self.leave_class()

def apply_class_plugin_hooks(self, defn: ClassDef) -> None:
"""Apply a plugin hook that may infer a more precise definition for a class."""
def get_fullname(expr: Expression) -> Optional[str]:
if isinstance(expr, CallExpr):
return get_fullname(expr.callee)
elif isinstance(expr, IndexExpr):
return get_fullname(expr.base)
elif isinstance(expr, RefExpr):
if expr.fullname:
return expr.fullname
# If we don't have a fullname look it up. This happens because base classes are
# analyzed in a different manner (see exprtotype.py) and therefore those AST
# nodes will not have full names.
sym = self.lookup_type_node(expr)
if sym:
return sym.fullname
return None

for decorator in defn.decorators:
decorator_name = get_fullname(decorator)
if decorator_name:
hook = self.plugin.get_class_decorator_hook(decorator_name)
if hook:
hook(ClassDefContext(defn, decorator, self))

if defn.metaclass:
metaclass_name = get_fullname(defn.metaclass)
if metaclass_name:
hook = self.plugin.get_metaclass_hook(metaclass_name)
if hook:
hook(ClassDefContext(defn, defn.metaclass, self))

for base_expr in defn.base_type_exprs:
base_name = get_fullname(base_expr)
if base_name:
hook = self.plugin.get_base_class_hook(base_name)
if hook:
hook(ClassDefContext(defn, base_expr, self))

def analyze_class_keywords(self, defn: ClassDef) -> None:
for value in defn.keywords.values():
value.accept(self)
Expand Down
4 changes: 2 additions & 2 deletions mypy/typeanal.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
from mypy.sametypes import is_same_type
from mypy.exprtotype import expr_to_unanalyzed_type, TypeTranslationError
from mypy.subtypes import is_subtype
from mypy.plugin import Plugin, AnalyzerPluginInterface, AnalyzeTypeContext
from mypy.plugin import Plugin, TypeAnalyzerPluginInterface, AnalyzeTypeContext
from mypy import nodes, messages


Expand Down Expand Up @@ -132,7 +132,7 @@ def no_subscript_builtin_alias(name: str, propose_alt: bool = True) -> str:
return msg


class TypeAnalyser(SyntheticTypeVisitor[Type], AnalyzerPluginInterface):
class TypeAnalyser(SyntheticTypeVisitor[Type], TypeAnalyzerPluginInterface):
"""Semantic analyzer for types (semantic analysis pass 2).

Converts unbound types into bound types.
Expand Down