Option for not following certain modules

[rggjan]

Currently, with

# test.py
import tensorflow

Running mypy gives:

mypy test.py
/.../miniconda/envs/foo/lib/python3.6/site-packages/tensorflow/python/ops/gen_spectral_ops.py:1392: error: misplaced type annotation

This file contains:

...
1386   op_def_lib.add_op_list(op_list)
1387   return op_def_lib
1388 # op {
1389 #   name: "BatchFFT"
1390 #   input_arg {
1391 #     name: "input"
1392 #     type: DT_COMPLEX64
1393 #   }
1394 #   output_arg {
1395 #     name: "output"
...

From #5696 I understand that this is an issue in the tensorflow files. However, I don't know how I can work around this without modifying all tensorflow python files and deleting these comments.

import tensorflow # type: ignore

as suggested in #545 doesn't seem to help. And I wouldn't want to set --follow-imports=skip just because of tensorflow, as I have many other imports which are typed correctly and which I'd like to have followed...

[rggjan]

Using

• mypy 0.720
• python 3.6.8
• tensorflow 1.14.1

[JukkaL]

There doesn't seem to be any good way to ignore a file what generates syntax errors. One option would be to allow specifying that some modules should be skipped in the configuration file, similar to ignore_errors. ignore_errors doesn't help since a misplaced type comment is a blocking error that can't be ignored. Alternatively, we could modify ignore_errors to allow ignoring (certain) blocking errors.

[joelgrus]

I have a similar issue related to pytorch. Consider the following program:

import torch

t = torch.tensor([1., 2., 3.])
p = torch.nn.Parameter(t)

this works; however, mypy doesn't like it:

run.py:4: error: Too many arguments for "Parameter"

which is (I think) an artifact of some metaclass voodoo that pytorch is doing.

I could stick a type: ignore on that line, but I have more than 1000 similarly problematic lines in my codebase. Similar to the OP, I tried doing

import torch  # type: ignore

but that didn't fix it.

Also per the documentation, I tried in my mypy.ini doing

[mypy-torch]
ignore_missing_imports = True

but that didn't work (and, given the existence of this issue, I guess is not supposed to work?)

--

even --follow-imports=skip doesn't fix this for me, although --no-site-packages does. So I guess I'm going to have to start running with --no-site-packages? Is there a better solution than that?

[joelgrus]

btw this is with

mypy==0.720
python==3.6.9
torch==1.2.0

[gvanrossum]

I wonder if this would work?

torch: Any = __import__("torch")

[gvanrossum]

Alternatively (perhaps less hacky):

import torch as _torch
torch: Any = torch

[joelgrus]

both of those seem to fix the mypy issue, but they also seem to break the VSCode autosuggest / autocomplete behavior, which is not a great trade-off.

[gvanrossum]

Bummer. Maybe you can rig something with

MYPY = False
if MYPY:
    # do it one way
else:
    import torch

???

[joelgrus]

yes, that seems to work, we'll see how my team feels about it, thanks!

[gvanrossum]

Then I propose to close this -- there's not likely going to be a new feature to make this easier.

[rggjan]

MYPY = False
if MYPY:
    # do it one way
else:
    import torch

When working with a large codebase with hundreds of imports of a certain module, like tensorflow, where many of them are inside the code to make imports of modules faster, this would litter the code with additional hundreds of lines of code just to fix this...

I think this is not really a good solution, and it should be possible to do ignore it globally for mypy without changing the sources. Especially since the "crashing" code is appareantly valid python code, and just not handled well by typed_ast...

[gvanrossum]

Hm, I think it's up to torch to fix this problem somehow. Their distro includes a py.typed marker file (see PEP 561) which means that mypy will load and analyze torch when it sees your code imports it. Without that marker file, mypy would ignore the torch package installed in site-packages and instead issue an error. The error could then be suppressed everywhere by adding this to your mypy.ini file:

[mypy-torch]
ignore_missing_imports = True

But given that that py.typed file exists you should take this up with the PyTorch project.