Fail fast if site-packages are in the path #5301

emmatyping · 2018-06-29T21:37:33Z

With my refactoring in #5256, I realized that it would be pretty straightforward to give some nicer warnings if site-packages were in any of the PATHs.

gvanrossum · 2018-06-29T21:45:40Z

mypy/build.py

+    package_path = tuple(_get_site_packages_dirs(options.python_executable))
+    for site in package_path:
+        if site in mypypath:
+            print("site-packages is in the MYPYPATH. Please remove it.", file=sys.stderr)


Since there may be multiple site-packages dirs (else why the outer loop :-) it would be helpful to include the offending site value in the messages.

gvanrossum · 2018-06-29T21:46:55Z

mypy/build.py

+            print("site-packages is in the PYTHONPAT. Please change directory so it is not.",
+                  file=sys.stderr)
+        elif site in lib_path:
+            print("this should never, ever happen", file=sys.stderr)


This needs a capital letter and a trailing period. Also why should this never happen? And if if can't happen perhaps it should be an assert?

It should never happen because the path to the typeshed stubs should never include a site-package directory. I will change it to an assert.

gvanrossum

Sorry for being so nitpicky.

gvanrossum · 2018-06-30T04:54:51Z

mypy/build.py

+        else:
+            # this site not in any of the other paths, we are safe.
+            continue
+        sys.exit(1)
    return SearchPaths(tuple(reversed(python_path)),


I feel there ought to be a blank line here (like you had before the last commit).

gvanrossum · 2018-06-30T04:57:34Z

mypy/build.py

+        else:
+            # this site not in any of the other paths, we are safe.
+            continue
+        sys.exit(1)


I feel it's better to have two sys.exit(1) calls, one after each print(). Then you don't need to else: continue bit which feels a bit odd.

gvanrossum · 2018-06-30T04:59:18Z

mypy/build.py

+    package_path = tuple(_get_site_packages_dirs(options.python_executable))
+    for site_dir in package_path:
+        assert site_dir not in lib_path
+        if site_dir in mypypath:


There are of course many variant spellings of the same path that won't be caught by this check. I don't think it's necessary to worry about that (normalizing the paths feels kind of expensive), but we shouldn't feel too complacent. :-)

Yeah, it won't be perfect, but I think it will catch and help debug some more common cases, as well as avoid crashes in those circumstances.

emmatyping · 2018-06-30T07:36:38Z

Sorry for being so nitpicky.

Not at all, code quality matters. I appreciate constructive feedback, no matter how small :)

gvanrossum

LG now!

sbrunner · 2018-07-16T15:19:27Z

Looks to be related to this pull request, if I run:

MYPYPATH=/usr/local/lib/python3.6/dist-packages mypy --disallow-untyped-defs --strict-optional --follow-imports skip my_package

I get

/usr/local/lib/python3.6/dist-packages is in the MYPYPATH. Please remove it.

And if I run

mypy --disallow-untyped-defs --strict-optional --follow-imports skip commons/c2cgeoportal_commons

I get

commons/c2cgeoportal_commons/config.py:29: error: Cannot find module named 'c2c'

commons/c2cgeoportal_commons/config.py:29: error: Cannot find module named 'c2c.template'

commons/c2cgeoportal_commons/models/__init__.py:33: error: No library stub file for module 'sqlalchemy'

commons/c2cgeoportal_commons/models/__init__.py:33: note: (Stub files are from https://github.com/python/typeshed)

commons/c2cgeoportal_commons/models/__init__.py:33: error: No library stub file for module 'sqlalchemy.orm'

commons/c2cgeoportal_commons/models/__init__.py:34: error: No library stub file for module 'sqlalchemy.ext'

commons/c2cgeoportal_commons/models/__init__.py:34: error: No library stub file for module 'sqlalchemy.ext.declarative'

commons/c2cgeoportal_commons/models/__init__.py:35: error: No library stub file for module 'sqlalchemy.ext.declarative.api'

commons/c2cgeoportal_commons/models/__init__.py:37: error: Cannot find module named 'pyramid.i18n'

commons/c2cgeoportal_commons/__init__.py:4: error: Cannot find module named 'pyramid.config'

commons/c2cgeoportal_commons/__init__.py:4: note: (Perhaps setting MYPYPATH or using the "--ignore-missing-imports" flag would help)

commons/c2cgeoportal_commons/models/sqlalchemy.py:31: error: No library stub file for module 'sqlalchemy.engine'

commons/c2cgeoportal_commons/models/sqlalchemy.py:32: error: No library stub file for module 'sqlalchemy.types'

commons/c2cgeoportal_commons/models/main.py:35: error: Cannot find module named 'papyrus.geo_interface'

commons/c2cgeoportal_commons/models/main.py:36: error: No library stub file for module 'sqlalchemy'

commons/c2cgeoportal_commons/models/main.py:37: error: No library stub file for module 'sqlalchemy.types'

commons/c2cgeoportal_commons/models/main.py:40: error: No library stub file for module 'sqlalchemy.schema'

commons/c2cgeoportal_commons/models/main.py:41: error: No library stub file for module 'sqlalchemy.orm'

commons/c2cgeoportal_commons/models/main.py:42: error: Cannot find module named 'geoalchemy2'

commons/c2cgeoportal_commons/models/main.py:43: error: Cannot find module named 'geoalchemy2.shape'

commons/c2cgeoportal_commons/models/main.py:46: error: Cannot find module named 'zope'

commons/c2cgeoportal_commons/models/main.py:46: error: Cannot find module named 'zope.event'

commons/c2cgeoportal_commons/models/main.py:47: error: Cannot find module named 'colander'

commons/c2cgeoportal_commons/models/main.py:48: error: Cannot find module named 'deform.widget'

commons/c2cgeoportal_commons/models/main.py:49: error: Cannot find module named 'c2cgeoform.ext'

commons/c2cgeoportal_commons/testing/__init__.py:3: error: Cannot find module named 'transaction'

commons/c2cgeoportal_commons/testing/__init__.py:4: error: Cannot find module named 'zope'

commons/c2cgeoportal_commons/testing/__init__.py:4: error: Cannot find module named 'zope.sqlalchemy'

commons/c2cgeoportal_commons/testing/__init__.py:5: error: No library stub file for module 'sqlalchemy'

commons/c2cgeoportal_commons/testing/__init__.py:6: error: No library stub file for module 'sqlalchemy.engine'

commons/c2cgeoportal_commons/testing/__init__.py:7: error: No library stub file for module 'sqlalchemy.orm'

commons/c2cgeoportal_commons/models/static.py:37: error: No library stub file for module 'sqlalchemy'

commons/c2cgeoportal_commons/models/static.py:38: error: No library stub file for module 'sqlalchemy.types'

commons/c2cgeoportal_commons/models/static.py:40: error: Cannot find module named 'colander'

commons/c2cgeoportal_commons/models/static.py:41: error: Cannot find module named 'deform.widget'

commons/c2cgeoportal_commons/models/static.py:42: error: Cannot find module named 'c2cgeoform.ext'

commons/c2cgeoportal_commons/testing/initializedb.py:5: error: Cannot find module named 'transaction'

commons/c2cgeoportal_commons/testing/initializedb.py:8: error: Cannot find module named 'pyramid.paster'

commons/c2cgeoportal_commons/testing/initializedb.py:10: error: Cannot find module named 'pyramid.scripts.common'

commons/c2cgeoportal_commons/testing/initializedb.py:11: error: No library stub file for module 'sqlalchemy.engine'

commons/c2cgeoportal_commons/testing/initializedb.py:12: error: No library stub file for module 'sqlalchemy.orm'

commons/c2cgeoportal_commons/testing/initializedb.py:13: error: No library stub file for module 'alembic'

commons/c2cgeoportal_commons/lib/email_.py:38: error: Cannot find module named 'pyramid.httpexceptions'

In one word he didn't find anything ...

Did I something wrong?

JelleZijlstra · 2018-07-16T15:41:34Z

Try --ignore-missing-imports.

sbrunner · 2018-07-17T05:34:30Z

If I add that It pass but it nearly didn't check anything...

emmatyping · 2018-07-17T05:43:26Z

Hi, unfortunately, unless a package is marked as type safe, we do not inspect it. You may want to read about this in the docs: https://mypy.readthedocs.io/en/latest/installed_packages.html. If you have further questions, feel free to drop by our Gitter chat.

sbrunner · 2018-07-17T06:06:35Z

OK thanks @ethanhs :-)

Fail fast if site-packages are in the path

783faad

gvanrossum requested changes Jun 29, 2018

View reviewed changes

Respond to feedback

01a6f4b

gvanrossum reviewed Jun 30, 2018

View reviewed changes

Make code cleaner

df34473

emmatyping force-pushed the clean-site branch from 90706da to df34473 Compare June 30, 2018 07:34

gvanrossum approved these changes Jun 30, 2018

View reviewed changes

gvanrossum merged commit 047c2c3 into python:master Jun 30, 2018

emmatyping mentioned this pull request Jul 5, 2018

Fix running mypy from editable install directory #5318

Merged

emmatyping deleted the clean-site branch July 17, 2018 05:40

emmatyping mentioned this pull request Aug 21, 2018

Fail fast when attempting to typecheck typing.py #3725

Closed

Uh oh!

Fail fast if site-packages are in the path #5301

Fail fast if site-packages are in the path #5301

Uh oh!

Conversation

emmatyping commented Jun 29, 2018 • edited by gvanrossum Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gvanrossum Jun 29, 2018

Choose a reason for hiding this comment

Uh oh!

gvanrossum Jun 29, 2018

Choose a reason for hiding this comment

Uh oh!

emmatyping Jun 29, 2018

Choose a reason for hiding this comment

Uh oh!

gvanrossum left a comment

Choose a reason for hiding this comment

Uh oh!

gvanrossum Jun 30, 2018

Choose a reason for hiding this comment

Uh oh!

gvanrossum Jun 30, 2018

Choose a reason for hiding this comment

Uh oh!

gvanrossum Jun 30, 2018

Choose a reason for hiding this comment

Uh oh!

emmatyping Jun 30, 2018

Choose a reason for hiding this comment

Uh oh!

emmatyping commented Jun 30, 2018

Uh oh!

gvanrossum left a comment

Choose a reason for hiding this comment

Uh oh!

sbrunner commented Jul 16, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JelleZijlstra commented Jul 16, 2018

Uh oh!

sbrunner commented Jul 17, 2018

Uh oh!

emmatyping commented Jul 17, 2018

Uh oh!

sbrunner commented Jul 17, 2018

Uh oh!

Uh oh!

emmatyping commented Jun 29, 2018 •

edited by gvanrossum

Loading

sbrunner commented Jul 16, 2018 •

edited

Loading