stubtest: False-positive errors for type[T] attributes with non-type metaclasses

[AlexWaygood]

Bug Report

Given a stub like so:

from abc import ABCMeta
class Y(metaclass=ABCMeta): ...
class Z:
    foo: type[Y]

And a runtime like so:

from abc import ABCMeta
class Y(metaclass=ABCMeta): ...
class Z:
    foo = Y

Stubtest will issue the following complaint:

error: Z.foo variable differs from runtime type abc.ABCMeta

This error can be reproduced with any custom metaclass; it's not just ABCMeta that triggers the bug:

# STUB
class Meta(type): ...
class Y(metaclass=Meta): ...
class Z:
    foo: type[Y]

# RUNTIME:
class Meta(type): ...
class Y(metaclass=Meta): ...
class Z:
    foo = Y

To Reproduce

1. Check out a local clone of mypy
2. Activate a virtual environment with an editable install of mypy
3. Apply this diff:

diff --git a/mypy/test/teststubtest.py b/mypy/test/teststubtest.py
index 2adbfaac2..60ddebc7f 100644
--- a/mypy/test/teststubtest.py
+++ b/mypy/test/teststubtest.py
@@ -208,6 +208,21 @@ class StubtestUnit(unittest.TestCase):
             """,
             error="X.mistyped_var",
         )
+        yield Case(
+            stub="""
+            class Meta(type): ...
+            class Y(metaclass=Meta): ...
+            class Z:
+                foo: type[Y]
+            """,
+            runtime="""
+            class Meta(type): ...
+            class Y(metaclass=Meta): ...
+            class Z:
+                foo = Y
+            """,
+            error=None
+        )

4. Run pytest ./mypy/test/teststubtest.py

Expected Behavior

No error should be emitted.

Actual Behavior

An error was emitted.

Cc. @hauntsaninja

[AlexWaygood]

The issue seems to be that the call to is_subtype_helper here returns False, whereas for a class without a metaclass, it returns True:

mypy/mypy/stubtest.py

Line 887 in d27bff6

and not is_subtype_helper(runtime_type, stub.type)

[hauntsaninja]

So the question is_subtype_helper gets posed is whether Meta is a subtype of type[Y]. This fails for the same reason that mypy will complain about x: type[Y] = Meta. The fix is probably some change in get_mypy_type_of_runtime_value — we currently always make Instances, but maybe we should have some complicated logic that comes up with TypeType for metaclasses? Might be a little tricky to get heuristics right, but subclass of type is hopefully good enough