Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--install-types can mask failure details #10768

Open
tk-woven opened this issue Jul 6, 2021 · 7 comments · May be fixed by #17485
Open

--install-types can mask failure details #10768

tk-woven opened this issue Jul 6, 2021 · 7 comments · May be fixed by #17485
Labels
bug mypy got something wrong

Comments

@tk-woven
Copy link

tk-woven commented Jul 6, 2021

Bug Report

When no .mypy_cache folder exists and a user runs mypy --install-types --non-interactive ./, mypy may fail on fundamental issues but suggest that the failure was because of "no mypy cache directory," confusing the user.

To Reproduce

I've created a minimum reproducible example. Please find it here.

I summarize the example below.

Expected Behavior

For the source tree

.
├── a
│   └── lib.py
├── b
│   └── lib.py
└── main.py

where all files are empty except main.py, which might contain

def f(i: int = 0):
    i = "test"  # This is a mypy violation.
    print(i)

if __name__ == "__main__":
    f()

we expect running mypy --install-types --non-interactive ./ at the root of the source tree to either (1) generate the .mypy_cache and then report any typing errors or (2) explain why it cannot generate the cache.

Actual Behavior

mypy fails because b.lib duplicates a.lib by name in the same module. However, mypy does not report the duplicate module error. Instead, it only reports error: --install-types failed (no mypy cache directory).

This behavior is particularly confusing in CI, where our environment has no preexisting .mypy_cache and where an unsuspecting user may accidentally cause the duplicate-module behavior. mypy fails and suggests it's because of the missing cache directory.

(Deeper demonstration is in the reproducible example linked above. In short, if mypy finds an empty .mypy_cache folder, it will report the duplicate module issue. If we place __init__.py in a and b and remove .mypy_cache, mypy will install the cache correctly and report the typing error.)

My Environment

  • Ubuntu 20
  • Python 3.6 (this older version is from the production codebase in which we found this issue)
  • mypy 0.910

Related to #10600 .

@berislav-harbr
Copy link

I'd like to add that I had the same issue when sqlalchemy-stubs was not installed.

AndrewKostousov added a commit to vektonn/vektonn-client-python that referenced this issue Jan 7, 2022
@KotlinIsland
Copy link
Contributor

KotlinIsland commented Mar 31, 2022

Minified example

> mypy --install-types --non-interactive -c " a"
error: --install-types failed (no mypy cache directory)

If you create the cache directory (empty) then it has:

> mypy --install-types --non-interactive -c " a"
<string>:1:2: error: unexpected indent  [syntax]           
Found 1 error in 1 file (errors prevented further checking)

Why does this check even exist?

jmichelp added a commit to jmichelp/scaaml that referenced this issue Feb 6, 2024
ferrarimarco added a commit to super-linter/super-linter that referenced this issue Feb 6, 2024
MyPy expects the cache directory to be there. We now remove it after
each super-linter run to avoid leftovers, so we need to create it before
runniny MyPy.

See python/mypy#10768
and python/mypy#10863
ferrarimarco added a commit to super-linter/super-linter that referenced this issue Feb 6, 2024
MyPy expects the cache directory to be there. We now remove it after
each super-linter run to avoid leftovers, so we need to create it before
running MyPy.

See python/mypy#10768
and python/mypy#10863
ferrarimarco added a commit to super-linter/super-linter that referenced this issue Feb 6, 2024
MyPy expects the cache directory to be there. We now remove it after
each super-linter run to avoid leftovers, so we need to create it before
running MyPy.

See python/mypy#10768
and python/mypy#10863
jmichelp added a commit to jmichelp/scaaml that referenced this issue Feb 8, 2024
@Dobatymo
Copy link

Still an issue with mypy==1.10.0. For me a error: Duplicate module named "xxx" (also at ".\yyy\xxx.py") error was hidden that way.

@amotl
Copy link

amotl commented Jun 4, 2024

Hi there. Thanks for creating this issue ticket. We would like to add that the same problem is also tripping us on a CI/GHA integration.

@jlin880
Copy link

jlin880 commented Jun 11, 2024

Hi there. Do we expect this issue to be resolved at any moment with any mypy releases? Or the only workaround now is to create .mypy_cache/ in our pipelines? Thanks!

@alanbchristie
Copy link

alanbchristie commented Jun 19, 2024

A follow-up to @jlin880 ...

mkdir .mypy_cache helps me get around mypy complaining about it not being there but I shouldn't have to do this.

kravets-levko added a commit to databricks/databricks-sql-python that referenced this issue Jul 2, 2024
* move py.typed to correct places

https://peps.python.org/pep-0561/ says 'For namespace packages (see PEP 420), the py.typed file should be in the submodules of the namespace, to avoid conflicts and for clarity.'. Previously, when I added the py.typed file to this project, #382 , I was unaware this was a namespace package (although, curiously, it seems I had done it right initially and then changed to the wrong way). As PEP 561 warns us, this does create conflicts; other libraries in the databricks namespace package (such as, in my case, databricks-vectorsearch) are then treated as though they are typed, which they are not. This commit moves the py.typed file to the correct places, the submodule folders, fixing that problem.
Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com>

* change target of mypy to src/databricks instead of src.

I think this might fix the CI code-quality checks failure, but unfortunately I can't replicate that failure locally and the error message is unhelpful

Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com>

* Possible workaround for bad error message 'error: --install-types failed (no mypy cache directory)'; see python/mypy#10768 (comment)

Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com>

* fix invalid yaml syntax

Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com>

* Best fix (#3)

Fixes the problem by cding and supplying a flag to mypy (that mypy needs this flag is seemingly fixed/changed in later versions of mypy; but that's another pr altogether...). Also fixes a type error that was somehow in the arguments of the program (?!) (I guess this is because you guys are still using implicit optional)

---------

Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com>

* return the old result_links default (#5)

Return the old result_links default, make the type optional, & I'm pretty sure the original problem is that add_file_links can't take a None, so these statements should be in the body of the if-statement that ensures it is not None

Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com>

* Update src/databricks/sql/utils.py

"self.download_manager is unconditionally used later, so must be created. Looks this part of code is totally not covered with tests 🤔"

Co-authored-by: Levko Kravets <levko.ne@gmail.com>
Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com>

---------

Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com>
Co-authored-by: Levko Kravets <levko.ne@gmail.com>
Iain-S added a commit to alan-turing-institute/rctab-api that referenced this issue Jul 2, 2024
Iain-S added a commit to alan-turing-institute/rctab-api that referenced this issue Jul 3, 2024
wyattscarpenter added a commit to wyattscarpenter/mypy that referenced this issue Jul 5, 2024
It seems that if the mypy cache dir wasn't created, this code would do an exit, preventing the actual errors from being printed. So I've removed the exit.
@wyattscarpenter
Copy link
Contributor

I've created a pull request, #17485, that fixes this issue. Turns out it's as simple as removing an early exit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug mypy got something wrong
Projects
None yet
8 participants