importing google.cloud.logging causes untyped modules in the namespace to be considered missing

[graingert]

Bug Report

To Reproduce

with the mypy config:

[mypy]

[mypy-google.cloud.*]
ignore_missing_imports=true

and the .py file:

from google.cloud import bigquery
from google.cloud import logging

Actual Behavior

I get

foo.py:1: error: Module "google.cloud" has no attribute "bigquery"

however if I comment out:

from google.cloud import bigquery
# from google.cloud import logging

I get

Success: no issues found in 1 source file

Expected Behavior

I don't expect importing a module in a namespace with stubs to affect other modules in that namespace

(Write what happened.)

Your Environment

• Mypy version used: 0.812 and d326952
• Mypy command-line flags: mypy.
• Mypy configuration options from mypy.ini (and other config files): see above
• Python version used: Python 3.8.5 (default, Jan 27 2021, 15:41:15)
• Operating system and version: Ubuntu 20.04.2 LTS

[graingert]

this is also annoying when from google.cloud import logging is imported in a different file, because the has no attribute "bigquery" error isn't present when using incremental

[egilewski]

Still reproducible on current master.

Maybe it will be helpful: mypy check also passes if from google.cloud import logging is replaced with import google.cloud.logging as logging.

[jahas]

Still reproducible on current master.

Maybe it will be helpful: mypy check also passes if from google.cloud import logging is replaced with import google.cloud.logging as logging.

It has helped me. Thank you for your help!

[hauntsaninja]

Summarising the issue (since there are some minor aspects of the original post I can't repro):

λ mypy --no-incremental --show-error-codes -c 'from google.cloud import logging'
<string>:1: error: Skipping analyzing "google.cloud": module is installed, but missing library stubs or py.typed marker  [import]
<string>:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 1 source file)

λ mypy --no-incremental --show-error-codes -c 'from google.cloud import bigquery
from google.cloud import logging'
<string>:2: error: Module "google.cloud" has no attribute "logging"  [attr-defined]
Found 1 error in 1 file (checked 1 source file)

That is, other imports from the same namespace package can affect the error code produced by a faulty import (the import from google-cloud-logging is faulty because it's not py.typed).

egilewski's workaround works well (because with import X form, mypy knows that X has to be a module):

change from google.cloud import logging to import google.cloud.logging as logging

[gmc444-b]

I'm seeing a very similar problem with an in-house project where I run mypy 1.7.1 over a project for both the source package and the tests, with only the --strict option. It succeeds the first time, then fails the second time with an error like the one described above, on a package namespace. Removing .mypy_cache allows it to succeed.

[aelamran]

I'm seeing a very similar problem with an in-house project where I run mypy 1.7.1 over a project for both the source package and the tests, with only the --strict option. It succeeds the first time, then fails the second time with an error like the one described above, on a package namespace. Removing .mypy_cache allows it to succeed.

Having the exact same issue, any workaround or fix?

[gmc444-b]

Having the exact same issue, any workaround or fix?

For me, the workaround was to remove the .mypy_cache before each run.

[asottile-sentry]

so I don't lose it from my duplicate #17210:

I believe this causes the bug -- or at least commenting it out "fixes" my problem --

mypy/mypy/modulefinder.py

Lines 543 to 549 in fb31409

	# Finally, we may be asked to produce an ancestor for an
	# installed package with a py.typed marker that is a
	# subpackage of a namespace package. We only fess up to these
	# if we would otherwise return "not found".
	ancestor = self.ns_ancestors.get(id)
	if ancestor is not None:
	return ancestor

[emmatyping]

Ok I dug into this a bit before a flight. Unfortunately wasn't able to identify the correct fix yet, but I have refreshed myself on how the module finding logic works.

The root of the error is that we currently record that the package is ModuleNotFoundReason.FOUND_WITHOUT_TYPE_HINTS in modulefinder, but by the time we look for the attribute/module google.cloud.X where X is some untyped package, we've thrown out this info, we only know that the module is not in our list of modules, so we assume it is a missing attribute, not a module.