mypy generates false positive with custom constants defined via the signal module

[ngie-eign]

Summary

• Report Type: Defect

Description

My group added a custom signal to our copy of the signal C extension under python 3.

Unfortunately, mypy 0.770 with --strict set fails to recognize the constant defined as part of the signal module, as seen below with a redacted sample:

# python -m tox -c <TEST-DIR> -e static
static installed: attrs==19.3.0,entrypoints==0.3,flake8==3.7.9,flake8-docstrings==1.5.0,mccabe==0.6.1,more-itertools==8.2.0,mypy==0.770,mypy-extensions==0.4.3,packaging==20.3,pluggy==0.13.1,py==1.8.1,pycodestyle==2.5.0,pydocstyle==5.0.2,
pyflakes==2.1.1,pyparsing==2.4.7,pytest==5.4.1,six==1.14.0,snowballstemmer==2.0.0,typed-ast==1.4.1,typing-extensions==3.7.4.2,wcwidth==0.1.9
static run-test-pre: PYTHONHASHSEED='1899821517'
static run-test: commands[0] | flake8 --config <CONFIG-DIR>/flake8 .
static run-test: commands[1] | mypy --config-file <CONFIG-DIR>/mypy.ini .
test_python_customizations.py:66: error: Module has no attribute "SIGCUSTOM"
Found 1 error in 1 file (checked 1 source file)
ERROR: InvocationError for command <TEST-DIR>/.tox/static/bin/mypy --config-file /var/crash/tools/etc/mypy.ini . (exited with code 1)
__________________________________________________________________________________________________________________ summary __________________________________________________________________________________________________________________
ERROR:   static: commands failed
# cat <CONFIG-DIR>/mypy.ini
[mypy]

[mypy-pytest.*]
# For pytest.mark.parameterize
allow_untyped_decorators=True
ignore_missing_imports=True
follow_imports=False

[mypy-signal.*]
# mypy doesn't recognize constants added to C extensions.
implicit_reexport=True
ignore_missing_imports=True
strict_optional=False

[mypy-test.*]
ignore_missing_imports=True

# python3
...
>>> import os.path, signal
>>> os.path.__all__
['normcase', 'isabs', 'join', 'splitdrive', 'split', 'splitext', 'basename', 'dirname', 'commonprefix', 'getsize', 'getmtime', 'getatime', 'getctime', 'islink', 'exists', 'lexists', 'isdir', 'isfile', 'ismount', 'expanduser', 'expandvars', 'normpath', 'abspath', 'samefile', 'sameopenfile', 'samestat', 'curdir', 'pardir', 'sep', 'pathsep', 'defpath', 'altsep', 'extsep', 'devnull', 'realpath', 'supports_unicode_filenames', 'relpath', 'commonpath']
>>> signal.__all__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'signal' has no attribute '__all__'

Repro steps:

1. Add line 2 to the interpreter copy of signal.py:

1 import _signal
2 _signal.SIGCUSTOM = _signal.SIGINT  # NB: failure instrumentation added.

1. Run the following command in a POSIX-compliant shell, e.g., bash or zsh, to create mypy_doesnt_like_sigcustom.py:

import signal
signal.SIGCUSTOM
EOF

1. Run mypy --strict mypy_doesnt_like_sigcustom.py

Expected result

• mypy shouldn't fail to resolve signal.SIGCUSTOM.

Actual result

• mypy fails similar to the following:

# python3 -m mypy --strict mypy_doesnt_like_sigcustom.py
mypy_doesnt_like_sigcustom.py:2: error: Module has no attribute "SIGCUSTOM"
Found 1 error in 1 file (checked 1 source file)
#

[ngie-eign]

@dmtucker pointed me to the interface file that defines all of the types for signal.py in typeshed. Having to fork/hack typeshed to deal with this is a path I really don't want to have to go down.

[emmatyping]

Having to fork/hack typeshed to deal with this is a path I really don't want to have to go down.

If you are running a non-standard Python build, you will have to do this. I'm not sure what other way you'd want mypy to detect this. Mypy does static analysis, it isn't going to import the C extension.

[ngie-eign]

If you are running a non-standard Python build, you will have to do this. I'm not sure what other way you'd want mypy to detect this. Mypy does static analysis, it isn't going to import the C extension.

sigh. The architecture with mypy & typeshed is broken. I can't believe I'm the first person that's had to fork mypy for this purpose.

I can understand if the stance is that python/typeshed#3953 needs to be fixed first, but the .pyi files need to live with the interfaces they describe -- otherwise this whole architecture will turn into a maintenance hell. After the typeshed issue is resolved, this issue needs to be followed up and resolved.

[hauntsaninja]

You could shadow signal with your own module that just reexports the contents of your modified signal, and provide a stub file for that.

[JukkaL]

You can perhaps use --custom-typeshed-dir. If you modify a stdlib module you'll have to go through some extra trouble.