Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relative imports from within the same package lead to "no type hints or stubs" error #8726

Closed
mschmidt87 opened this issue Apr 26, 2020 · 9 comments · Fixed by #9742
Closed

Comments

@mschmidt87
Copy link

mschmidt87 commented Apr 26, 2020

Hi,

I have a problem with mypy and handling of imports. Basically, I have a Python package with several modules which import other modules of the package. When I run mypy (version 0.770) on the files, I get this error:
error: Skipping analyzing '.file2': found module but no type hints or library stubs

However, I don't understand that error message because (in this example) file2 does contain type hints.
For the purpose of this issue, I have created a minimal reproducer. I have two files file1 and file2 which both define a class. The class in file1 (Example1) has a method which uses the class from file2 (Example2), thus file1 imports file2 with a relative import (from .file2 import Example2). This is the full code:

  • file1.py
from .file2 import Example2

class Example1:
    def __init__(self, value: float) -> None:
        self.value = value

    def add(self, other: Example2) -> None:
        self.value += other.value
  • file2.py
class Example2:
    def __init__(self, value: float) -> None:
        self.value = value

When I run mypy file1.py file2.py, I am receiving the error mentioned above:

file1.py:1: error: Skipping analyzing '.file2': found module but no type hints or library stubs
file1.py:1: note: See https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports

Now I did visit that recommended link in the documentation but I couldn't figure out how to solve my problem.

However, one thing that worked was to turn the relative into a global import, i.e. to write from file2 import Example2. However, I would like to avoid that because in my package I would prefer to use relative imports for ease of readability of the code (amongst others). Also, the error message does not seem to make sense to me because file2.py does contain type annotations.

Apologies for posting this because it seems like a basic thing, but I couldn't figure it out with the documentation or by browsing the issues here.
It would be great if someone could explain how to do this properly or point me to a documentation of the solution.
Thanks a lot.

(Python version if 3.8.2).

@msullivan
Copy link
Collaborator

mypy expects a __init__.py in the directory in order for it to be a package.

@mschmidt87
Copy link
Author

Thank you for your reply. Sorry, I forgot to mention that: I did add an __init__.py in the same folder but it led to the same result. I tried both an empty file or this:

from . import file1
from . import file2

but I got the same error.

@msullivan
Copy link
Collaborator

Oh, hm. Mypy doesn't want you specifying paths to modules in packages without a directory component of the path name, it seems. Running from the parent directory and specifying the paths, works, I think.

@mschmidt87
Copy link
Author

Okay, if I understood you correctly, you want me to move one level up (to the parent dir) and call mypy from there? That actually works with the relative imports.

To be clear, what I did was: The code posted above is in a folder mypy_min_reproducer. Then I do

cd mypy_min_reproducer
cd ..
mypy mypy_min_reproducer/file1.py mypy_min_reproducer/file2.py

and receive Success: no issues found in 2 source files.

Thanks a lot for the tip. However, the error message is not very informative here, could it be improved?

@JukkaL
Copy link
Collaborator

JukkaL commented May 1, 2020

Marked this as a usability issue. The error message isn't very helpful.

@bsmedberg-xometry
Copy link

Is this ticket about fixing the error message, or fixing the mypy behavior not to require an __init__.py file? As of python 3.3, init.py is no longer required for packages.

@hauntsaninja
Copy link
Collaborator

hauntsaninja commented Oct 27, 2020

I think the correct fix here is that we should make find_sources use absolute paths when crawling. I had a diff that did this, but all the tests use relative paths so it broke everything; I'll try again by converting back to relative paths for BuildSource.

Note that this behaviour was documented as a result of #6660, but I traced it all the way back to 0555178 and I don't see a good reason to treat relative paths differently. This has come up in some other issues as well.

Secondarily, we might be able to make some improvements to modulefinder's error messages.

Finally, mypy's handling of __init__.py-less packages isn't ideal (for now, my recommendation is to use mypy -p <package> --namespace-packages), but I've recently made some PRs reworking things, so mypy 0.800 will be better.

hauntsaninja added a commit that referenced this issue Dec 12, 2020
This is the successor to #9632. Things should basically be as discussed in that PR. Since #9616 is merged, this should now resolve #5759.

We leave the Bazel integration with `--package-root` almost entirely untouched, save for a) one change that's a bugfix / doesn't affect the core of what `--package-root` is doing, b) another drive by bugfix that's not related to this PR.
Change a) fixes the package root `__init__.py` hackery when passed absolute paths. Change b) fixes the validation logic for package roots above the current directory; it was broken if you passed `..` as a package root

Since we're leaving `--package-root` alone for now, I named the new flag `--explicit-package-base` to try and avoid confusion. Doing so also matches the language used by BuildSource a little better.

The new logic is summarised in the docstring of `SourceFinder.crawl_up`.

Some commentary:
- I change `find_sources_in_dir ` to call `crawl_up` directly to construct the BuildSource. This helps codify the fact that files discovered will use the same module names as if you passed them directly.
- Doing so keeps things DRY with the more complicated logic and means, for instance, that we now do more sensible things in some cases when we recursively explore directories that have invalid package names.
- Speaking of invalid package names, if we encounter a directory name with an invalid package name, we stop crawling. This is necessary because with namespace packages there's no guarantee that what we're crawling was meant to be a Python package. I add back in a check in the presence of `__init__.py` to preserve current unit tests where we raise InvalidSourceList.
- The changes to modulefinder are purely cosmetic and can be ignored (there's some similar logic between the two files and this just makes sure they mirror each other closely)
- One notable change is we now always use absolute paths to crawl. This makes the behaviour more predictable and addresses a common complaint: fixes #9677, fixes #8726 and others.
- I figured this could use more extensive testing than a couple slow cmdline tests. Hopefully this test setup also helps clarify the behaviour :-)

Co-authored-by: hauntsaninja <>
@victorbartel
Copy link

mypy expects a __init__.py in the directory in order for it to be a package.

It's a shame and it's not obvious, at all. In python >3.5 those empty files are not mandatory, so many people don't bother to use them, why should we do a huge step back?

Instead of saying, hey I need __init__.py to work with your project, it says Skipping analyzing "your.module": found module but no type hints or library stubs . Are you serious?

spezifisch added a commit to spezifisch/divo that referenced this issue Jan 21, 2022
@pythonpro7
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants