-
-
Notifications
You must be signed in to change notification settings - Fork 2.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature Request] Option to Recurse into All Subdirectories with Namespace Packages #6385
Comments
Which of the following would be a better solution: (1) changing the behavior of My vote is for option 1 just to keep things simple by having as few command line args as necessary, and because I find the current behavior of Of course, the flip side is that changing the current behavior of |
Like everything with Python import, stuff here is more complicated than it seems. Mypy has two different ways of finding files: one for the initial set off target files specified on the command line, and a separate set of rules for following imports. The --namespace-packages flag applies to following imports only. But here we’re concerned with the file targets specified on the command line. I think it’s actually a bug that when you specify a directory on the command line it doesn’t really recurse into that directory, but does a shallow directory listing. I propose to fix that bug and replace the shallow directory listing with a proper walking of the directory hierarchy starting at that point. Note that the -p or --packages flag is still different: its argument is not a directory relative to the current directory, but specifies a search in the initial $MYPYPATH, so that’s unrelated. |
I see, thanks for the clarification. I'll work on a PR that properly walks the directory hierarchy for targets specified on the command line. |
+1 I've been hoping for mypy to recursively traverse the given directory for some time. I look forward to this fix! |
I'm investigating what it will take to make this change and found a problem:
How should mypy determine the base directory and top module for namespace packages since there are no |
Hm, looking more at that code it seems that This puts us in a difficult position. The current code seems to be treating the toplevel directory as a member of
and we run And if we wanted to simply extend this to namespace packages (probably only when using But an alternative interpretation might be that So which should it be? I'm not sure. It might be easier to expand the meaning of But this doesn't provide exactly the same functionality: the argument to Oh wait: another difference is that with (In the meantime, a useful diversion might be to implement #7672 first.) |
@JohnHBrock wondering if there any news or any help needed? In the current implementation, mypy pretty much discourages users from going with pep420; I personally had to go and add a bunch of those pesky empty One other use case to note, for namespace packages it's not uncommon to place the source code under This way, you could have something like src/
+-- foo/
+-- bar.py
+-- baz/
+-- __init__.py
+-- yo.py and, ideally, one would expect things to "just work" by running env MYPYPATH=src mypy --namespace-packages -p foo which would then traverse |
@aldanor No updates on my end. It's not clear to me how to proceed until the questions raised by Guido above are resolved. |
We are currently suffering from silently* not type-checking project files when someone forgets** to put in the required A recursive option would solve this issue for us. *"Almost silently" might be more exact; a very observed dev could see that the number of files checked hadn't increased since before they added new modules/files - or just assumed that they must be being ignored because mypy hadn't raised any errors and they are unlikely to code perfectly first time... |
This ensures that mypy will typecheck them. mypy doesn't recursively check directories without an __init__ file. See: python/mypy#6385
This ensures that mypy will typecheck them. mypy doesn't recursively check directories without an __init__ file. See: python/mypy#6385
This ensures that mypy will typecheck them. mypy doesn't recursively check directories without an __init__ file. See: python/mypy#6385
This ensures that mypy will typecheck them. mypy doesn't recursively check directories without an __init__ file. See: python/mypy#6385
Mypy also doesn't seem to type check $ mypy github
Success: no issues found in 95 source files VS $ mypy github/*.py
github/__init__.py:70: error: Type of __all__ must be "Sequence[str]", not "List[object]" |
A workaround (adapted from here): find . -type f -name "*.py" | xargs mypy |
@jamesbraza unfortunately this doesn't work universally. If you have two files with the same name (e.g. Django project):
|
Hi, I just stumbled over the same issue and found this request. Since it's open since more than 1 year I guess I shouldn't expect it to be included any time soon, should I? |
There’s another issue about this that promises to fix this. Can you find it for me? |
I looked through the open pull requests but couldn't spot anything that seemed to be a fix for this. In the open issues I could only find #8548, however it's not clear to me whether it demands to fix the documentation or the code. |
In #8548 (comment) I meant to say that we should fix the code to actually recurse looking for .py files. |
Just to clarify, is the other issue actually the same here?
(perhaps @hauntsaninja has context given these questions) |
There are several overlapping issues here. Recursing is currently "fixed" on master (in theory :-) ), but it might not yet do what you want for namespace packages (see #9632, also mentioned on this thread are issues fixed by #9683). @ehossack To confirm what's going on, could you share the output of |
Sure, see https://gist.github.com/ehossack/5ec1113cc74e03adda3d858e4f7293eb
Sorry it's not open source, and I can't share the code.
So maybe I don't properly understand what command I'm supposed to use if I want to ask mypy to "please check this and all subdirectories for all python files and ensure their typing is compliant" and thus, I apologize, and am probably commenting on the wrong ticket. On my reproduction project (see here), I'm running (in order of various commands to try and understand things):
|
The only foolproof way with mypy 0.790 and current master is to add From your gist, it looks like #9614 is doing what it is intended to do; it's not intended to handle namespace packages (which is why this issue and #5759 are still open). #9632 is the PR that aims to give you a way to do what you want. With that said, I'm somewhat confused by what other parts of mypy are doing in your gist. I'd expect mypy's build to raise a
I could try and figure out what's going on there. |
Thanks @hauntsaninja for the explanation! For the record, I think I suppressed the error according to this thread setting the |
FWIW, I am using this current workaround for Repository structure:
# Global options:
[mypy]
allow_redefinition = True
namespace_packages = True
no_incremental = True Workaround command executed at the root of the repository: find src -type f -name "*.py" | sed 's|/|.|g; s|\.py||g' | xargs -t -I {} mypy -m {} |
I'm glad I found this open issue (and all the linked ones), after (as a brand-new user) being completely unable to get mypy to analyze all of my source files, because I too am using the src-layout and namespace packages (code is targeted at Python 3.6+ and is located two levels deep under a 'company' namespace in which I cannot include an |
Yeah, some of mypy 0.790's behaviour here is pretty bad. But this is fixed in master, specifically by #9742 (and related fixes in other PRs). For the following layout:
Using mypy master, you can get it to check everything with:
Explanation: when passing files (as opposed to packages or modules), mypy will crawl upwards as long as there's an You could also get it to check
Hopefully by the next release we can make |
Feature Request
An option to recursively type-check all
.py
files under a given directory when usingnamespace_packages
(without__init__.py
).Use Case
Given the following directory structure:
Assuming that
hello.py
importshello_service.py
, everything under thehello
namespace will be type checked as expected withmypy ./hello
.However test discovery with
pytest
,nose
,django
et al works differently andhello_test.py
would not usually importhello_service_test.py
. There is currently no way for Mypy to discoverhello_service_test.py
withmypy ./tests
(if not using__init__.py
).Similarly, everything under the
scripts
directory would suffer the same problem.If Mypy supported a
--recursive -r
option (or similar) that would cause it to automatically recurse into subdirectories, this would solve these common use cases.Why not just use
__init__.py
?To quote
iScrE4m
's comment from #1645 (comment)_,Configuration
# setup.cfg [mypy] python_version = 3.7 ignore_missing_imports = True namespace_packages = True
The text was updated successfully, but these errors were encountered: