Cache causes mypy to fail every 2nd run for module importing from aws_cdk #9852
Labels
Comments
|
Huh, that's an interesting one. I can repro, including with master. But weirdly, not with my editable install of mypy. |
|
Here's a side by side of |
|
Looks like this was previously reported at #9777 |
|
I suspect it's the same as #7276 as well. |
4 tasks
|
I have a similar issue, any new info on this? |
|
I reported the same bug on pre-commit/pre-commit#1966 as it happened only when I was running mypy from inside pre-commit but this morning I managed to find an workaround for this bug. I reconfigured the hook to force it to avoid use of progressive run. Mainly on hook that is Interesting enough, I am also on MacOS and using py39 from pyenv. Maybe the messed cached issue may be specific to macos. |
|
@ssbarnea I have the same issue on linux. Probably not macos specific. |
|
I am very curious on what is happening as that is a real PITA. It does not happen with all projects but is still common enough. |
|
This a pretty bad bug, non-exhaustive list of reports: #10056 #7276 #9852 #9777 #10906 #10376 #10644 #10906 I found a "fix", but unfortunately, I don't really understand the code. But maybe in posting I can snipe @msullivan into fixing this for real :-) First note that in #10056 there's a pretty minimal repro, scriptified as: I'm also not sure the best way to turn this into a test. I haven't been able to repro without the PEP 561 package. Here's the diff that fixes it: The only test failure this causes is testIncrementalPackageNameOverload, which isn't the worst thing. This code was previously touched in: |
msullivan
added a commit
that referenced
this issue
Aug 4, 2021
As part of doing this I upgraded the pep561 tests to support incremental tests, and removed at least a little of the duplication between different incremental test suites.
msullivan
added a commit
that referenced
this issue
Aug 5, 2021
|
I had a few minutes so I went back and looked at old mypy releases and based on the above repro this started when namespace packages were added all the way back in 0.630. My gut tells me the real issue is modulefinder, but I'll take a closer look this weekend. |
msullivan
added a commit
that referenced
this issue
Aug 6, 2021
Since PEP561-installed namespace packages only show up in FindModuleCache as a side-effect, the submodules need to be listed first in the dependencies for things to work. This was handled already in the Import case but not the ImportFrom case. Also fix the cache handling of namespace packages so it doesn't always get reported stale. Fixes #9852.
msullivan
added a commit
that referenced
this issue
Aug 6, 2021
Since PEP561-installed namespace packages only show up in FindModuleCache as a side-effect, the submodules need to be listed first in the dependencies for things to work. This was handled already in the Import case but not the ImportFrom case. Also fix the cache handling of namespace packages so it doesn't always get reported stale. Fixes #9852.
hauntsaninja
pushed a commit
that referenced
this issue
Aug 6, 2021
Since PEP561-installed namespace packages only show up in FindModuleCache as a side-effect, the submodules need to be listed first in the dependencies for things to work. This was handled already in the Import case but not the ImportFrom case. Also fix the cache handling of namespace packages so it doesn't always get reported stale. Fixes #9852.
bonzini
pushed a commit
to qemu/qemu
that referenced
this issue
Nov 1, 2021
This one is insidious: if you write an import as "from {namespace}
import {subpackage}" as mirror-top-perms (now) does, mypy will fail on
every-other invocation *if* the package being imported is a typed,
installed, namespace-scoped package.
Upsettingly, that's exactly what 'qemu.[aqmp|qmp|machine]' et al are in
the context of Python CI tests.
Now, I could just edit mirror-top-perms to avoid this invocation, but
since I tripped on a landmine, I might as well head it off at the pass
and make sure nobody else trips on that same landmine.
It seems to have something to do with the order in which files are
checked as well, meaning the random order in which set(os.listdir())
produces the list of files to test will cause problems intermittently
and not just strictly "every other run".
This will be fixed in mypy >= 0.920, which is not released yet. The
workaround for now is to disable incremental checking, which avoids the
issue.
Note: This workaround is not applied when running iotest 297 directly,
because the bug does not surface there! Given the nature of CI jobs not
starting with any stale cache to begin with, this really only has a
half-second impact on manual runs of the Python test suite when executed
directly by a developer on their local machine. The workaround may be
removed when the Python package requirements can stipulate mypy 0.920 or
higher, which can happen as soon as it is released. (Barring any
unforseen compatibility issues that 0.920 may bring with it.)
See also:
python/mypy#11010
python/mypy#9852
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211019144918.3159078-14-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
stsquad
pushed a commit
to stsquad/qemu
that referenced
this issue
Nov 2, 2021
This one is insidious: if you write an import as "from {namespace}
import {subpackage}" as mirror-top-perms (now) does, mypy will fail on
every-other invocation *if* the package being imported is a typed,
installed, namespace-scoped package.
Upsettingly, that's exactly what 'qemu.[aqmp|qmp|machine]' et al are in
the context of Python CI tests.
Now, I could just edit mirror-top-perms to avoid this invocation, but
since I tripped on a landmine, I might as well head it off at the pass
and make sure nobody else trips on that same landmine.
It seems to have something to do with the order in which files are
checked as well, meaning the random order in which set(os.listdir())
produces the list of files to test will cause problems intermittently
and not just strictly "every other run".
This will be fixed in mypy >= 0.920, which is not released yet. The
workaround for now is to disable incremental checking, which avoids the
issue.
Note: This workaround is not applied when running iotest 297 directly,
because the bug does not surface there! Given the nature of CI jobs not
starting with any stale cache to begin with, this really only has a
half-second impact on manual runs of the Python test suite when executed
directly by a developer on their local machine. The workaround may be
removed when the Python package requirements can stipulate mypy 0.920 or
higher, which can happen as soon as it is released. (Barring any
unforseen compatibility issues that 0.920 may bring with it.)
See also:
python/mypy#11010
python/mypy#9852
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211019144918.3159078-14-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
|
This issue still persists with mypy 0.910. Every second run fails. This has either not been fixed or isn't fixed in a released version. |
|
I don't think the fix is in 0.910, unfortunately. |
|
I have a similar issue, involving |
|
This issue was fixed. Could you open a new issue with details and preferably a repro? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Bug Report
When
mypyis run multiple times on a Python module containingfrom aws_cdk import coreit will fail every 2nd time due to the error "Skipping analyzing 'aws_cdk': found module but no type hints or library stubs".Disabling the mypy cache with
--no-incrementalwill result inmypypassing every time.To Reproduce
mypyandaws-cdk.core.from aws_cdk import core.mypyon the module and it will pass.mypyon the module again and it will fail. Subsequent runs will cycle between passing and failing.Expected Behavior
Running
mypyon a module containing onlyfrom aws_cdk import coreshould always pass.Actual Behavior
Every second time
mypyis run on the module it will report an error.Your Environment
mypy.ini(and other config files): noneaws-cdk.coreversion used: 1.80.0The problem may be related to how the package that is installed is
aws-cdk.corebut the namespace being imported from isaws_cdk. A workaround is to change:to:
The first form is what is generated by the
cdktool and used in the cdk example code.The text was updated successfully, but these errors were encountered: