Importing a regular package from a namespace package seems to break intermittently

[Cnoor0171]

Bug Report
If you have a library that creates a namespace package (say ns_package) with a regular type-hinted package inside it (say reg_package), and you import the regular package as from ns_package import reg_package, mypy sometimes gives the following error:

script.py:1: error: Skipping analyzing 'ns_package': found module but no type hints or library stubs
script.py:1: note: See https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 1 source file)

The error only shows up intermittently. More details below.

To Reproduce

1. Create the following directory tree:

.
├── lib/
│   ├── ns_package/
│   │   └── reg_package/
│   │       ├── __init__.py
│   │       └── py.typed
│   └── setup.py
└── script.py

3 directories, 4 files

with content

# lib/ns_package/reg_package/__init__.py
SOMETHING: int = 345

# lib/setup.py
from setuptools import setup, find_namespace_packages

setup(
    name="ns_package.reg_package",
    description="",
    version="1.0",
    packages=find_namespace_packages(),
    zip_safe=False,
    package_data={"": ["py.typed"]},
)

# script.py
from ns_package import reg_package

a = reg_package.SOMETHING

print(a)

2. Create venv, activate and install the lib and mypy: python3.8 -m venv .venv; . .venv/bin/activate; python3.8 -m pip install -e ./lib/ mypy
3. Run mypy against script.py with python3.8 -m mypy -p script

Expected Behavior

Mypy should report no errors since the library has a py.typed file and type annotations.

Actual Behavior

Mypy's result changes with every run. Sometimes no error is reported. Sometimes the following error is reported:

script.py:1: error: Skipping analyzing 'ns_package': found module but no type hints or library stubs
script.py:1: note: See https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 1 source file)

From what I can tell, the error is always reported in every other run, consistently. Meaning if I run mypy twice in a row, one of them always fails and one of them always succeeds. This seems to suggest that mypy somehow maintains state from one run to the next?

Couple of points to note:

• If the import statement is changed to import ns_package.reg_package as reg_package or from ns_package.reg_package import SOMETHING, the error is gone.
• Adding the --namespace-packages CLI flag doesn't change the result.
• Whether the lib is installed as an editable package doesnt change the result.
• Whether its installed from local dir or PyPi doesn't change the result.
• If ns_package is just a directory in the cwd instead of an installed package, and the --namespace-packages CLI flag is passed, the error is gone.

Your Environment

• Mypy version used: 0.800
• Mypy command-line flags: -p script
• Mypy configuration options from mypy.ini (and other config files): None
• Python version used: 3.8.6
• Operating system and version: Red Hat Enterprise Linux Server 7.9 (Maipo)

[hauntsaninja]

Thanks for reporting, I think this is #9852 (and the other issues linked there). Pretty unfortunate, but you could use --no-incremental as a workaround.

[Cnoor0171]

@hauntsaninja Ahh yes. That seems to be the same issue. Sorry about the dup. I was searching around for issues relating to "namespace packages", so I didn't find the linked issue, but now I see it happens even if the top package isn't a namespace package. --no-incremental does fix the problem. Unfortunately it slows down mypy quite a bit if running repeatedly, but it's not unreasonable. 👍

[hauntsaninja]

@Cnoor0171 This isn't fixed yet, hopefully soon, but just wanted to say thank you for taking the time to make a minimal repro :-)

[hauntsaninja]

Fixed by #10937