mypy 0.990 shows no errors when passing source directory as absolute path

[intgr]

Bug Report

I noticed that after upgrading to 0.990, mypy in djangorestframework-stubs CI no longer finds any errors.

It seems this happens when the source folder is passed as a full path, instead of relative path. $PWD is the full path of the current working directory:

% mypy --config-file $PWD/mypy.ini --show-traceback --hide-error-context ./drf_source/tests |tail -n1
Found 569 errors in 48 files (checked 83 source files)
% mypy --config-file $PWD/mypy.ini --show-traceback --hide-error-context $PWD/drf_source/tests |tail -n1
Success: no issues found in 83 source files

Maybe, just maybe, this is related to #13768? EDIT: seems not to be, various --follow-imports= options have no effect.

To Reproduce

1. Check out https://github.com/intgr/djangorestframework-stubs branch update-to-mypy-0.990
2. Install dependencies from requirements.txt in a virtualenv
3. Upgrade to mypy 0.990
4. Run python scripts/typecheck_tests.py, which populates the drf_source directory
5. Try running the commands above

Your Environment

• Mypy version used: 0.990
• Mypy configuration options from mypy.ini (and other config files): https://github.com/typeddjango/djangorestframework-stubs/blob/06a2f2f9f6c05a07f03bc1bd7f1b54ecfd7a3c43/mypy.ini
• Python version used: 3.10.8

[intgr]

I can also reproduce this when I comment out mypy plugins in mypy.ini

[intgr]

Also reproduced this in another project of mine.

[hauntsaninja]

Does it reproduce with --no-namespace-packages?

[intgr]

Yes. Adding --no-namespace-packages does not solve the issue, still getting "no issues found".

[ilevkivskyi]

Also do you have $PWD in your PYTHONPATH maybe? What was the last mypy version that didn't have this issue? Around 0.970 mypy started interpreting everything in PYTHONPATH as third party packages and silencing them. You can try --no-silence-site-packages

[intgr]

This issue does not occur with 0.982

% mypy --version
mypy 0.982 (compiled: yes)
% mypy --config-file $PWD/mypy.ini --show-traceback --hide-error-context --no-namespace-packages $PWD/drf_source/tests |tail -n1
.../djangorestframework-stubs/mypy.ini: [mypy]: disable_error_code: Invalid error code(s): empty-body
Found 575 errors in 49 files (checked 83 source files)

PYTHONPATH is unset in the shell I'm testing this from. The typecheck_tests.py script does indeed add $PWD to PYTHONPATH, but it doesn't seem to affect this issue.

[ilevkivskyi]

Hm, probably the easiest way is to bisect mypy commits. Since the problem looks really bizzare.

[intgr]

Adding --no-silence-site-packages with mypy 0.990 and using $PWD only mostly prints errors from site-packages modules, not the project itself.

Are you unable to reproduce this in any of your own projects?

I can give bisecting a try.

[ilevkivskyi]

Are you unable to reproduce this in any of your own projects?

No, but I also don't have many.

I can give bisecting a try.

That would be great, since I will not have much time today/tomorrow.

[intgr]

Git bisected it to:

00ca6bf is the first bad commit

Which is #13768 as I initially suspected.

[ilevkivskyi]

@JukkaL could you please take a look?

[intgr]

I was wrong before. --no-silence-site-packages does have an effect on the output but it's really weird.

• mypy --no-silence-site-packages $PWD/drf_source/tests 142 errors in 18 files (checked 83 source files) -- most of these are site-pacakges errors, but 6 are from project itself!
• mypy --no-silence-site-packages ./drf_source/tests 142 errors in 18 files (checked 83 source files), same as above
• mypy $PWD/drf_source/tests no issues found in 83 source files
• mypy ./drf_source/tests 569 errors in 48 files (checked 83 source files) -- expected behavior

[intgr]

The behavior of --no-silence-site-packages is also reproducible in mypy 0.982. But the $PWD vs . behavior does not reproduce there.

[trivialfis]

Just to confirm I'm experiencing the same issue with the latest mypy. But the issue is not reproducible in every environment it seems. I run into the same issue as described by @intgr on my local machine, but our CI script can report errors correctly using the same version of mypy.

[JukkaL]

I will investigate. This might be caused by absolute paths treated differently from relative paths, but we'll see.

[JacobHayes]

In case it is useful, -p is also incorrectly omitting errors, matching the absolute path behavior (perhaps -p resolves to an absolute path?). I can confirm relative paths work correctly.

[najtin]

I think this is the cause to microsoft/vscode-python#20182

EDIT: see my next comment, this issue here does not seem to cause the linting problem.

The issue is about mypy not find any error when run by the linter but finding errors when the exact same command is running in a terminal. Notice the relative paths.
Linter inside vscode:

> ./.venv/bin/python ~/.vscode-oss/extensions/ms-python.python-2022.16.1-universal/pythonFiles/linter.py -m mypy --follow-imports=silent --ignore-missing-imports --show-column-numbers --no-pretty ./rootsnakes/app.py
cwd: .
##########Linting Output - mypy##########

Success: no issues found in 1 source file

same code in terminal:

./.venv/bin/python ~/.vscode-oss/extensions/ms-python.python-2022.16.1-universal/pythonFiles/linter.py -m mypy --follow-imports=silent --ignore-missing-imports --show-column-numbers --no-pretty ./lol.py
lol.py:1:1: error: Missing return statement  [return]
Found 1 error in 1 file (checked 1 source file)

See the linked issue for a more detailed description.

[najtin]

also maybe some of the issues reported here are definitely related to #14054

EDIT: through some experimentation i found that the vs-code linting issue is definitely caused by #14054 and not by this issue here. Some package managers automatically add the current working directory to the path, e.g. running poetry install. It seems very likely that this could also break numerous CI setups. @trivialfis @intgr Here i cover in detail how to get reproduce the issue microsoft/vscode-python#20182 (comment) in case anyone get here with the same issue as me.
It seems that both issues are caused by the same changes in #13768

[JukkaL]

The issue should be fixed on master. It would be great if somebody can validate that it works. We'll have a 0.99x poinrt release (probably next week) that will include the fix.

[intgr]

Thanks! I can confirm that this fixes my original use case.