Import errors in a try/except ImportError block should be allowed

[bdarnell]

When this pattern is checked in py2 mode (or really any pre-3.5 version)

try:
    from inspect import isawaitable  # py35+
except ImportError:
    from backports_abc import isawaitable

it raises an error tornado/gen.py:118: error: Module has no attribute 'isawaitable'. Instead, mypy should recognize that this is within a try/except block that catches the ImportError and choose the alternate path instead.

This falls under the umbrella of #1297 but I couldn't find an exact match for this issue.

[gvanrossum]

Yup. (Can you shut it up using `# type: ignore`?)

[bdarnell]

Yes, # type: ignore works here (as long as I ignore both branches)

[mig4]

This is also the case with attribute access in try/except block, not only imports, e.g.:

try:
    foo = sys._MEIPASS
except AttributeError:
    foo = '.'

causes mypy to fail with error: Module has no attribute "_MEIPASS", even though it looks like a reasonable approach. Ignoring with # type: ignore comment silences it.

[bhrutledge]

We encountered this in Twine with importlib.metadata.metadata vs importlib_metadata.metadata, but only as of mypy 0.750. There's a fix pending in pypa/twine#551 (which also covers an instance of #1153).

I made a repository to reproduce the errors and try out some fixes: https://github.com/bhrutledge/mypy-importlib-metadata. That's currently passing, but the commit history has more details.

[emmatyping]

The change is likely due to a change in typeshed (adding importlib_metadata I presume). I would recommend using if sys.version_info... checks to tell mypy what Python version is being used in each case so it doesn't misunderstand the code.

[bhrutledge]

@ethanhs Thanks! I didn't know that mypy would use sys.version_info as a hint.

[jaraco]

I would recommend using if sys.version_info...

This approach is less desirable than using the try/except pattern. In particular, one must (a) import sys, (b) parse/compare sys.version_info, and (c) encode availability of module features to their various versions (which may not be a simple >= comparison).

Is the implication of this recommendation that try/except for imports is deprecated/discouraged, or is this recommendation just a temporary workaround for an acknowledged problem?

[JukkaL]

A workaround is needed because of a missing mypy feature. I don't know when the feature might be implemented, though.

[The-Compiler]

Perhaps a naive question: If mypy recognized a "try/except ImportError", could it set the type of the module object to something like Union[Module1, Module2] (or Optional[Module] in case the except does mod = None)?

[emmatyping]

@The-Compiler sadly that probably wouldn't work, as you want to make sure the same module is used throughout. We would probably want something closer to a type variable I believe.

[ghost]

I just ran into the same thing today with this:

try:
    from typing import TypeAlias
except ImportError:
    from typing_extensions import TypeAlias

Since I have this in my pyproject.toml (rather not add an extra dependency when not necessary):

[tool.poetry.dev-dependencies]
# ...
typing-extensions = {version = "^4.1.1", python = "<3.10"}

# type: ignore[attr-defined] works in the meantime.

[adam-grant-hendry]

I've encountered this as well.

From Adam Johnson's blog post, his workaround is to set a boolean flag:

try:
    import markdown

    HAVE_MARKDOWN = True
except ImportError:
    HAVE_MARKDOWN = False


def something():
    ...

    if HAVE_MARKDOWN:
        ...

In fact, @Michael0x2a on this SO post couples the above approach with the --always-true/--always-false options to type check both implementations.

Alternatively, @cjerdonek has developed a nice workaround.

Perhaps a naive question: If mypy recognized a "try/except ImportError", could it set the type of the module object to something like Union[Module1, Module2] (or Optional[Module] in case the except does mod = None)?

@The-Compiler No, I don't think it's naive at all. In fact, I believe @JukkaL hinted at this. However, @ethanhs might be right: I believe (emphasis on "believe") the root of the issue is that mypy is "picky" in the sense that the module/package API must be consistent throughout.

@JukkaL Can you confirm?

Addendum

Below are gists of Adam Johnson's failed attempt and @cjerdonek's working workaround:

Failing Adam Johnson gist
Passing @cjerdonek gist

[hauntsaninja]

I'm going to close this issue. I don't think type checkers should do the fully general thing of ignore type errors in try-except. Modern mypy is a lot happier here, it will allow imports of symbols if the type matches, or if the symbol has a previously declared type (and in particular, modules can now match Protocols). There is also now better support for type ignores in conditionally analysed code. For backports of stdlib features, using checks against sys.version_info also work very well.

See also #1153 (comment) / #8823 (comment)

[AlexWaygood]

I agree that mypy definitely shouldn't support checked exceptions in general, but try/except ImportError is such a common idiom that I honestly think there's a pretty strong argument for special-casing it. It's true that you can switch to sys.version_info checks and mypy will be happy, and that's fine when you get used to it. But that's not an idiom that seems intuitive to anybody who's new to Python typing, in my experience, and the fact that type checkers don't understand the more common way of writing things seems very strange to them.

Even if the decision is that we shouldn't implement any special-casing here, in order to remain consistent about never understanding any exception handling, I still think we could add a better error message to point users towards what they should do instead.

This is one of the most popular feature requests on the issue tracker, and I think that's evidence that this is a real problem that users are facing.

[hauntsaninja]

This issue was opened in 2016 and there's a lot that has changed since then. In particular, I fixed the most common case of this a couple months ago: where a conditional import redefines a symbol with exactly the same structural type. I think in #13969, but some other PRs were involved as well

Like I mention in the comment I link to, the issue is about providing guarantees about the rest of the code. type: ignore is cheap and if you break mypy's ability to reason about code elsewhere, you should have to add one. For instance, if you redefine a symbol with a different type or you have differing nominal types, mypy is right to complain.

I'm happy to leave issues open if we can tweak an error message, but I'm not seeing many additional things to implement that are a good combination of viable and sound, so a broad issue like this isn't super actionable. The main workable thing I can think of is doing something for typing + typing_extensions special forms.

[bdarnell]

This issue was opened in 2016 and there's a lot that has changed since then.

As the original issue author, I agree that a lot has changed. I no longer really need this, because the main case where it came up for me was in code that needed support for Python 2 and 3 simultaneously. I still have a few places with lingering type: ignore comments due to ImportError handling, but they're infrequent enough that I'm comfortable saying that the code should be changed instead of mypy.

[ghost]

Python 2 may be no longer relevant, but typing_extensions and such is still relevant unless you only want to support the absolute latest Python version. It's not a dealbreaker but try/except does feel more Pythonic than checking sys.version_info.

[cebtenzzre]

typing_extensions and such is still relevant

In most cases, guarding your typing imports with if TYPE_CHECKING: and using from __future__ import annotations is sufficient. You can quote types in aliases and use TypeAlias. Inheriting from a type in typing may require some extra logic for runtime. But I haven't found myself using typing_extensions except for trying experimental features.

[TeamSpen210]

That doesn't work for anything examining the types at runtime, and also for types used outside annotations (such as class definitions, decorators, typevarlike definitions, etc).