State of the Mypy cache makes check results unpredictable

[exarkun]

The result of mypy checking a piece of code depends on both that code and the internal mypy cache. The same piece of code can cause mypy to produce two different results depending on the state of the mypy cache.

(A clear and concise description of what the bug is.)

To Reproduce

Run mypy 3 times. The first run is against version "A" of the demonstration code. The second run is against version "B" of the demonstration code and introduces some mypy errors. The third run is against version "A" of the demonstration code - again. But this time it comes with the same errors version "B" received.

[nix-shell:~/Work/python/tahoe-lafs]$ rm -rf .mypy_cache/
(python38-tahoe-lafs)
[nix-shell:~/Work/python/tahoe-lafs]$ cat example-web.py
from twisted.web.client import HTTPClientFactory
from twisted.web.client import HTTPPageGetter

class Y(HTTPPageGetter):
    pass

class X(HTTPClientFactory):
    protocol = Y
(python38-tahoe-lafs)
[nix-shell:~/Work/python/tahoe-lafs]$ mypy example-web.py
Success: no issues found in 1 source file
(python38-tahoe-lafs)
[nix-shell:~/Work/python/tahoe-lafs]$ cat example-web.py
from twisted.web.client import HTTPClientFactory
from twisted.web.client import HTTPPageGetter

#                       vvvvvv
class Y(HTTPPageGetter, object):
#                       ^^^^^^
    pass

class X(HTTPClientFactory):
    protocol = Y
(python38-tahoe-lafs)
[nix-shell:~/Work/python/tahoe-lafs]$ mypy example-web.py
example-web.py:4: error: Cannot determine consistent method resolution order (MRO) for "Y"
example-web.py:8: error: Incompatible types in assignment (expression has type "Type[Y]", base class "HTTPClientFactory" defined the type as "Type[HTTPPageGetter]")
Found 2 errors in 1 file (checked 1 source file)
(python38-tahoe-lafs)
[nix-shell:~/Work/python/tahoe-lafs]$ cat example-web.py
from twisted.web.client import HTTPClientFactory
from twisted.web.client import HTTPPageGetter

class Y(HTTPPageGetter):
    pass

class X(HTTPClientFactory):
    protocol = Y
(python38-tahoe-lafs)
[nix-shell:~/Work/python/tahoe-lafs]$ mypy example-web.py
example-web.py:4: error: Cannot determine consistent method resolution order (MRO) for "Y"
example-web.py:8: error: Incompatible types in assignment (expression has type "Type[Y]", base class "HTTPClientFactory" defined the type as "Type[HTTPPageGetter]")
Found 2 errors in 1 file (checked 1 source file)
(python38-tahoe-lafs)

Expected Behavior

The two runs of mypy against version "A" of the code should produce the same result.

Actual Behavior

The two runs of mypy against version "A" of the code produce different results.

Your Environment

• Mypy version used: 0.812
• Mypy command-line flags: none
• Mypy configuration options from mypy.ini (and other config files):

[mypy]
ignore_missing_imports = True
plugins=mypy_zope:plugin

• Python version used: 3.8.8
• Operating system and version: NixOS 20.09.3009.8e78c2cfbae

Also required by the demo program above, Twisted 21.2.0.

[JukkaL]

Thanks for the detailed bug report! Mypy should produce the same results independent of the cache state. Any deviations are unexpected (unless the cache is corrupted, but any cache generated by mypy should be fine, including older versions).

[noah-built]

We see similar issues relating to the google.cloud.storage and google.cloud.secretmanager packages.

[phyrwork]

I think I've hit this one - also mypy 0.812. Now clearing the cache before all runs to avoid possible intermittent errors - bit inconvenient as the run time is much longer without a warm cache.

[DMRobertson]

I think I've seen this recently (matrix-org/synapse#13500, where we also use mypy-zope). I could reproduce the example using twisted 21.2.0, mypy 0.981 and mypy-zope 0.3.10. Once you get into the stuck state described, I noticed that rm -r .mypy_cache/3.10/twisted was enough to get mypy into the correct state again.

I wonder if this might be a mypy-zope issue rather than a mypy issue? I'll chuck some print statements into mypy and see if I can find any clues.

[DMRobertson]

Cross-referencing: Shoobx/mypy-zope#76

[DMRobertson]

Shoobx/mypy-zope#80 fixes the problems I was seeing here. If you're also seeing this and are using mypy-zope, it might be worth trying the latest release.

[hauntsaninja]

Nice work!

[DMRobertson]

We see similar issues relating to the google.cloud.storage and google.cloud.secretmanager packages.

For cross-referencing: this might be covered by #10360?