Make Type[X] compatible with metaclass of X #3346

elazarg · 2017-05-09T20:19:54Z

PR #2837 seems completely broken; the checks were all backwards, so this PR reverses them.

I had to special-case EnumMeta since the bound for the self argument can't be lower than the current class. I fail to see how it can work without something like metaclass=M argument for TypeVar.

ilevkivskyi

This mostly looks good. I have an idea about special-casing self-types of the form Type[T].

ilevkivskyi · 2017-05-13T07:18:33Z

mypy/subtypes.py

@@ -155,19 +150,18 @@ def visit_instance(self, left: Instance) -> bool:
 for lefta, righta, tvar in
 zip(t.args, right.args, right.type.defn.type_vars))
 if isinstance(right, TypeType):
- item = right.item
- if isinstance(item, TupleType):
- item = item.fallback


Why did you remove the case of TupleType? A named tuple can appear in Type[].

ilevkivskyi · 2017-05-13T07:20:42Z

mypy/subtypes.py

@@ -263,8 +257,7 @@ def visit_overloaded(self, left: Overloaded) -> bool:
 elif isinstance(right, TypeType):
 # All the items must have the same type object status, so
 # it's sufficient to query only (any) one of them.
- # This is unsound, we don't check the __init__ signature.


I think this comment should stay here, since the __init__ signature should be checked for all items in an overload.

ilevkivskyi · 2017-05-13T07:28:32Z

test-data/unit/check-classes.test

@@ -3066,12 +3066,13 @@ class M(type):
 class A(metaclass=M): pass

 reveal_type(A[M]) # E: Revealed type is 'builtins.int'
+reveal_type(A[M]) # E: Revealed type is 'builtins.int'


This reveal is identical to the previous one. I think you wanted to check something like A[A].

Nah I've probably copied it with the test below somehow :)

ilevkivskyi · 2017-05-13T07:45:40Z

test-data/unit/check-classes.test

+
+class M(type):
+ def g1(cls: 'Type[A]') -> None: pass # E: The erased type of self 'Type[__main__.A]' is not a supertype of its class '__main__.M'
+ def g2(cls: Type[TA]) -> None: pass # E: The erased type of self 'Type[__main__.A]' is not a supertype of its class '__main__.M'


Although Type[A] is lower than M, I think this could be allowed. The special-casing probably should be in checkmember.bind_self (then you will also not need special-casing EnumMeta above). My idea is following:

class M(type): def g2(cls: Type[TA]) -> TA: ... def g4(cls: TM) -> None: ... class A(metaclass=M): ... class A2(A): ... class B(metaclass=M): ... A.g2() # OK, inferred type is 'A' A2.g2() # OK, inferred type is 'A2' B.g2() # Error, g2 should be only called on subclasses of A A.g4(), A2.g4(), B.g4() # All are OK

The situation with one "privileged" class with a given metaclass sometimes appears in frameworks. This will allow users to give more precise types in such situation.

So this is probably a more general decision to be made. Instead of requiring a supertype, we can require a related type - either a supertype or a subtype, in which case there's a check at the point of use, just like any other function.

Although I'm not sure why add g2 in M instead of in A as a classmethod

So this is probably a more general decision to be made. Instead of requiring a supertype, we can require a related type - either a supertype or a subtype, in which case there's a check at the point of use, just like any other function.

I would vote for doing this first only for self types in metaclasses (since this is a situation where we know this is needed).

Although I'm not sure why add g2 in M instead of in A as a classmethod

Yes, but this will not work for dunder methods.

I see.
Adding checks at access seems to have non local effect, which makes this PR significantly more complex. Perhaps it will be better to split it up, and add the check at access only later.

ilevkivskyi · 2017-05-13T07:46:58Z

test-data/unit/check-classes.test

+class A(metaclass=M):
+ def foo(self): pass
+
+# 3 examples of unsoundness - instantiation, classmethod, staticmethod and ClassVar:


It looks here are four examples.

ilevkivskyi · 2017-05-13T07:55:00Z

test-data/unit/check-classes.test

+cv.x
+x3: M = cv
+
+[builtins fixtures/classmethod.pyi]


I would add few reveal_types for methods in these tests, just in case.

I tried, although I'm not sure what information is gained.

Actually I wanted something like reveal_type(A.g1) and reveal_type(A.g1()), in particular, where return types of g2 etc. depend on type variables (like List[TA]). Plus also the situations where name of staticmethod/classmethod in class coincides with a name of a method in metaclass.

Actually I wanted something like reveal_type(A.g1) and reveal_type(A.g1()), in particular, where return types of g2 etc. depend on type variables (like List[TA]). Plus also the situations where name of staticmethod/classmethod in class coincides with a name of a method in metaclass.

@elazarg Have you added these tests? Otherwise I think this PR looks good.
@JukkaL could you please try this PR with the Dropbox internal codebase before it is merged?

Hmm I'm pretty sure I did, but probably forgot to push... :\ I will check.

Ok, I don't have time at this moment, so I tried pushing kinda blindly. I might have mixed up something. This supposed to be the tests only, not the check-on-access you've asked for. I hope I did not mess up too bad; if I did, I can probably try to fix it later this week. In such case, it can still be useful to check earlier commit against the internal repo.

Sorry about the mess.

@ilevkivskyi I can try this with Dropbox internal code tomorrow (need to go now).

No worries, I have tried this with our internal code, there are no new errors this found (either in our codebase or in the PR). I haven't reviewed the PR though so I can't "Approve" it. But Ivan can.

ilevkivskyi

OK, this looks good now. We can discuss whether we need additional errors on access, and whether we can allow cls: Type[A] first argument annotation in metaclass methods (this PR only allows this for EnumMeta) in a separate issue.

* master: Support accessing modules imported in class bodies within methods. (python#3450) Make Type[X] compatible with metaclass of X (python#3346) Sync typeshed (python#3479) Handle flags in pytest passthrough (python#3467)

elazarg added 2 commits May 9, 2017 10:48

Reverse subtype check, add checks for typevar

907f07b

Don't do M <: Type[X]

75d8073

elazarg force-pushed the type-compat-meta branch from 3766205 to 75d8073 Compare May 9, 2017 20:22

ilevkivskyi reviewed May 13, 2017

View reviewed changes

elazarg added 4 commits May 13, 2017 20:56

fix some errors, extend test, don't remove comment

2209639

Merge branch 'master' into type-compat-meta

18edb2d

Merge branch 'master' into type-compat-meta

ae0c77b

add tests, and prepare tests for checking-on-access

34b5817

gvanrossum assigned ilevkivskyi May 30, 2017

ilevkivskyi approved these changes May 31, 2017

View reviewed changes

ilevkivskyi merged commit f60d874 into python:master May 31, 2017

elazarg mentioned this pull request Jun 28, 2017

Metaclass type/instance difficulties #3625

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make Type[X] compatible with metaclass of X #3346

Make Type[X] compatible with metaclass of X #3346

elazarg commented May 9, 2017

ilevkivskyi left a comment

ilevkivskyi May 13, 2017

ilevkivskyi May 13, 2017

ilevkivskyi May 13, 2017

elazarg May 13, 2017

ilevkivskyi May 13, 2017

elazarg May 13, 2017

elazarg May 13, 2017

ilevkivskyi May 14, 2017

elazarg May 16, 2017

ilevkivskyi May 13, 2017

ilevkivskyi May 13, 2017

elazarg May 13, 2017

ilevkivskyi May 14, 2017

ilevkivskyi May 30, 2017

elazarg May 30, 2017

elazarg May 30, 2017

JukkaL May 30, 2017

gvanrossum May 30, 2017

ilevkivskyi left a comment

Make Type[X] compatible with metaclass of X #3346

Make Type[X] compatible with metaclass of X #3346

Conversation

elazarg commented May 9, 2017

ilevkivskyi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ilevkivskyi left a comment

Choose a reason for hiding this comment