modulefinder: make -p actually handle namespace packages correctly

[hauntsaninja]

This is what #9616 should have been. I noticed this as part of #9672.

Note that this has the unfortunate side effect of causing mypy to fail when run on mypy with --namespace-packages, since typeshed is embedded inside mypy. This isn't great, especially in the context of #9636, and maybe indicates a missing "ignore" feature.

[JukkaL]

Note that this has the unfortunate side effect of causing mypy to fail when run on mypy with --namespace-packages, since typeshed is embedded inside mypy.

FWIW, I'd like to have typeshed outside mypy, since being a subdirectory of mypy/ is often awkward.

[JukkaL]

The code and test case update look fine, but I'm not sure what exactly is fixed by this PR. Based on the test case, this seems to related to __init__.py inside a namespace package? It would be good add more detail to the commit message or to create a GitHub issue about the bug that is fixed here.

[hauntsaninja]

Yes, sorry about that! The core of the bugfix is the change from:

if os.path.isdir(abs_path) and \
        (os.path.isfile(os.path.join(abs_path, '__init__.py')) or
        os.path.isfile(os.path.join(abs_path, '__init__.pyi'))):

to

if self.fscache.isdir(path):
    if (self.options and self.options.namespace_packages) or (
        self.fscache.isfile(os.path.join(path, "__init__.py"))
        or self.fscache.isfile(os.path.join(path, "__init__.pyi"))
    ):

That is, once we were in a directory with __init__.py we only recursed into subdirectories if those subdirectories also contained an __init__.py.