mypy bug with try/except conditional imports

[timabbott]

try:
    import simplejson
except ImportError:
    import json as simplejson

resulst in the error error: Name 'simplejson' already defined

[refi64]

Pretty sure this is a dup, but I can't find the issue.

[refi64]

Related to #649.

[gvanrossum]

Note that #649 was closed; a buch of "leftover" bugs were opened in its
place, but I don't see one that matches this pattern, so I think it's a new
case.

On Mon, Jan 25, 2016 at 3:19 PM, Ryan Gonzalez notifications@github.com
wrote:

Related to #649 #649.

—
Reply to this email directly or view it on GitHub
#1153 (comment).

--Guido van Rossum (python.org/~guido)

[gvanrossum]

Closed by mistake.

[JukkaL]

Another example from #2251 (reported by @RitwikGupta):

try:
    # Python 3
    from urllib.request import urlopen
except ImportError:
    # Python 2
    from urllib2 import urlopen

[gvanrossum]

And another from #2253:

This python code generates an error. It's a common idiom and it would be good if mypy could check for it in some way.

try:
    import cPickle as pickle
except ImportError:
    import pickle

$ mypy --py2 --silent-imports bad.py
bad.py:4: error: Name 'pickle' already defined

[JukkaL]

The current workaround is to add a # type: ignore comment. For example:

try:
    import cPickle as pickle
except ImportError:
    import pickle  # type: ignore     # <<-- add this

[posita]

FYI, this doesn't appear to work with sub-modules:

# This will error
try:
    import foo.bar
except ImportError:
    foo = None  # type:ignore

This seems to work though:

# This will pass
try:
    import doesnt.exist  # type:ignore
except ImportError:
    doesnt = None

# So will this
try:
    import collections.abc  # type:ignore
except ImportError:
    collections = None

assert doesnt is None
assert collections is not None

[quodlibetor]

This is particularly difficult when the import that's happening is a typing import.

For example, this works:

$ mypy --version
mypy 0.530

$ cat no_try.py
from typing import Dict, Any

JSON = Dict[str, Any]

def accept(obj: JSON) -> Any:
    return obj.pop('hello')

$ mypy --ignore-missing-imports no_try.py

$ cat with_try.py
try:
    from typing import Dict, Any
except:
    from backports.typing import Dict, Any  # type: ignore

JSON = Dict[str, Any]

def accept(obj: JSON) -> Any:
    return obj.pop('hello')

$ mypy --ignore-missing-imports with_try.py
with_try.py:10: error: Invalid type "with_try.JSON"
with_try.py:11: error: JSON? has no attribute "pop"

This program should typecheck.

[ilevkivskyi]

@quodlibetor I don't think your particular case is a bug. mypy has special treatment of typing module, so your imports should be written as:

from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from typing import Dict, Any
else:
    try:
        from typing import Dict, Any
    except ImportError:
        from backports.typing import Dict, Any

If your version of typing doesn't have TYPE_CHECKING, then use:

MYPY = False
if MYPY:
    # special imports from typing
else:
    # actual runtime implementation

mypy will understand this.

[quodlibetor]

Hmm, okay, I've verified that that works.

$ cat with_try.py
MYPY = False
if MYPY:
    from typing import Dict, Any
else:
    try:
        from typing import Dict, Any
    except:
        from backports.typing import Dict, Any
JSON = Dict[str, Any]
def accept(obj: JSON) -> Any:
    return obj.pop('hello')

$ mypy with_try.py

I think that I would still be inclined to describe the current behavior as a bug that has a workaround. Would you agree that it is at least as a UX problem?

The if MYPY trick brings the total number of lines of boilerplate to type check a module up to 8, for what would be a single-line import if you were only supporting the most modern python. Even just supporting python 3.5 if you want new items from the typing module means that we jump from 1 to 8 lines of code. It's not even "simple" code. It's also not the obvious first way that I would write that code... obviously.

[JelleZijlstra]

Why do you need backports.typing? The typing module on PyPI supports all Python versions that mypy itself supports.

[quodlibetor]

I did not know about the typing package on pypi. The last time I looked the only option was backports.typing. That that really ought to fix all my problems, or at least cause interesting new problems ;-) Thank you!

[gvanrossum]

We should contact whoever owns backports and ask to remove the typing backport.

…

On Oct 12, 2017 3:01 PM, "Brandon W Maister" ***@***.***> wrote: I did not know about the typing package on pypi. The last time I looked the only option was backports.typing. That that really ought to fix all my problems, or at least cause interesting *new* problems ;-) Thank you! — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#1153 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACwrMu0VaLbDtD_pmgzGlR_Mq7tES4taks5sroxOgaJpZM4HMB58> .

[adam-grant-hendry]

Building on an answer from @cjerdonek, would this be an acceptable workaround instead of # type: ignore?:

try:
    from py3_pkg import module as _module
except:
    from py2_pkg import module
else:
    module = _module

[atrigent]

Is this going to be fixed anytime soon?

[hauntsaninja]

No, it's not, and it's not clear that it should.

A majority of cases I've seen here are to work around Python stdlib changes, especially Python 2 to Python 3 stuff. sys.version_info checks are the preferred way to do this. Also Python 2 is dead.

if sys.version_info >= (3, 8):
    import importlib.metadata as importlib_metadata
else:
    import importlib_metadata

For the remaining cases, it's often not clear that it's sound, especially given nominal typing. Even if the API is structurally the same, nominal isinstance checks could result in surprising behaviour from mypy.

The only thing we should do in this space is #5018 where you can explicitly define exactly the structure you want with a Protocol, and mypy will confirm.

For all other cases, if you want to lie to the typechecker, please, go ahead and lie to the type checker (or type ignore):

if TYPE_CHECKING:
    import json
else:
    try:
        import simplejson as json
    except ImportError:
        import json

[lilydjwg]

FYI "type ignore" doesn't work well because in some cases it will cause Unused "type: ignore" comment errors elsewhere (e.g. GitHub Actions).

[cebtenzzre]

@lilydjwg Issue #8823 was fixed last week and covers what you just described - unused-ignore errors in unreachable code.