Skip to content

Internal error with TypeGuard and Literal-based union narrowing using StrEnum #18895

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
frilox042 opened this issue Apr 7, 2025 · 4 comments · Fixed by #18897
Closed

Internal error with TypeGuard and Literal-based union narrowing using StrEnum #18895

frilox042 opened this issue Apr 7, 2025 · 4 comments · Fixed by #18897

Comments

@frilox042
Copy link

Crash Report

When using TypeGuard to narrow a Literal-based union from a StrEnum, mypy raises an internal error.
I found a workaround, but I prefer my first implementation as it is more concise.

Traceback

main.py: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
If this issue continues with mypy master, please report a bug at https://github.com/python/mypy/issues
version: 1.10.0
main.py: : note: please use --show-traceback to print a traceback when reporting a bug

To Reproduce

from typing import TypeGuard, Literal, TypeAlias
from enum import StrEnum

class Model(StrEnum):
    MODEL_A1 = 'model_a1'
    MODEL_A2 = 'model_a2'
    MODEL_B = 'model_b'

MODEL_A: TypeAlias = Literal[Model.MODEL_A1, Model.MODEL_A2]
MODEL_B: TypeAlias = Literal[Model.MODEL_B]

def is_model_a(model: str) -> TypeGuard[MODEL_A]:
    return model in (Model.MODEL_A1, Model.MODEL_A2)

def is_model_b(model: str) -> TypeGuard[MODEL_B]:
    return model == Model.MODEL_B

def process_model(model: MODEL_A | MODEL_B) -> int:
    return 42

# Works
def handle_1(model: Model) -> int:
    if is_model_a(model):
        return process_model(model)
    if is_model_b(model):
        return process_model(model)
    return 0

# Crashes
def handle_2(model: Model) -> int:
    if is_model_a(model) or is_model_b(model):  # ❌ This line causes a crash
        return process_model(model)
    return 0

# Crashes
def handle_3(model: Model) -> int:
    match model:
        case m if is_model_a(m) or is_model_b(m):  # ❌ Also crashes
            return process_model(m)
        case _:
            return 0

Your Environment

  • Mypy version used: 1.15.0
  • Mypy command-line flags:
  • Mypy configuration options from mypy.ini (and other config files):
  • Python version used: 3.12
  • Operating system and version: Linux 6.13.8
@frilox042 frilox042 added the crash label Apr 7, 2025
@brianschubert
Copy link
Collaborator

master traceback:

TESTFILES/SCRATCH.py: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.16.0+dev.749f2584da9425173d68eb220db7e92aa13ad8ea
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/brian/Projects/open-contrib/mypy/mypy/__main__.py", line 37, in <module>
    console_entry()
  File "/home/brian/Projects/open-contrib/mypy/mypy/__main__.py", line 15, in console_entry
    main()
  File "/home/brian/Projects/open-contrib/mypy/mypy/main.py", line 126, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/home/brian/Projects/open-contrib/mypy/mypy/main.py", line 210, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/home/brian/Projects/open-contrib/mypy/mypy/build.py", line 191, in build
    result = _build(
  File "/home/brian/Projects/open-contrib/mypy/mypy/build.py", line 267, in _build
    graph = dispatch(sources, manager, stdout)
  File "/home/brian/Projects/open-contrib/mypy/mypy/build.py", line 2939, in dispatch
    process_graph(graph, manager)
  File "/home/brian/Projects/open-contrib/mypy/mypy/build.py", line 3337, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/brian/Projects/open-contrib/mypy/mypy/build.py", line 3442, in process_stale_scc
    graph[id].finish_passes()
  File "/home/brian/Projects/open-contrib/mypy/mypy/build.py", line 2369, in finish_passes
    with self.wrap_context():
  File "/usr/lib/python3.13/contextlib.py", line 162, in __exit__
    self.gen.throw(value)
  File "/home/brian/Projects/open-contrib/mypy/mypy/build.py", line 2062, in wrap_context
    yield
  File "/home/brian/Projects/open-contrib/mypy/mypy/build.py", line 2391, in finish_passes
    self._patch_indirect_dependencies(self.type_checker().module_refs, all_types)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/brian/Projects/open-contrib/mypy/mypy/build.py", line 2424, in _patch_indirect_dependencies
    encountered = self.manager.indirection_detector.find_modules(types) | module_refs
                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
  File "/home/brian/Projects/open-contrib/mypy/mypy/indirection.py", line 35, in find_modules
    self._visit(typs)
    ~~~~~~~~~~~^^^^^^
  File "/home/brian/Projects/open-contrib/mypy/mypy/indirection.py", line 46, in _visit
    typ.accept(self)
    ~~~~~~~~~~^^^^^^
  File "/home/brian/Projects/open-contrib/mypy/mypy/types.py", line 2978, in accept
    return visitor.visit_union_type(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/brian/Projects/open-contrib/mypy/mypy/indirection.py", line 125, in visit_union_type
    self._visit(t.items)
    ~~~~~~~~~~~^^^^^^^^^
  File "/home/brian/Projects/open-contrib/mypy/mypy/indirection.py", line 46, in _visit
    typ.accept(self)
    ~~~~~~~~~~^^^^^^
  File "/home/brian/Projects/open-contrib/mypy/mypy/types.py", line 276, in accept
    raise RuntimeError("Not implemented", type(self))
RuntimeError: ('Not implemented', <class 'mypy.types.TypeGuardedType'>)

@sterliakov
Copy link
Collaborator

This minimal patch should prevent crashing due to TypeGuardedType leaking from narrow_declared_type. I'm not sure this solution is robust enough, but at least it won't leak short-lived types unexpectedly.

diff --git a/mypy/meet.py b/mypy/meet.py
index b5262f87c..57cc34f78 100644
--- a/mypy/meet.py
+++ b/mypy/meet.py
@@ -142,16 +142,15 @@ def narrow_declared_type(declared: Type, narrowed: Type) -> Type:
                 )
             ]
         )
-    if is_enum_overlapping_union(declared, narrowed):
-        return original_narrowed
-    elif not is_overlapping_types(declared, narrowed, prohibit_none_typevar_overlap=True):
+    if not is_overlapping_types(declared, narrowed, prohibit_none_typevar_overlap=True):
         if state.strict_optional:
             return UninhabitedType()
         else:
             return NoneType()
     elif isinstance(narrowed, UnionType):
         return make_simplified_union(
-            [narrow_declared_type(declared, x) for x in narrowed.relevant_items()]
+            [narrow_declared_type(declared, x) for x in narrowed.relevant_items()],
+            contract_literals=not is_enum_overlapping_union(declared, narrowed)
         )
     elif isinstance(narrowed, AnyType):
         return original_narrowed

Note that is_enum_overlapping_union includes isinstance(narrowed, UnionType). I'll try to dig deeper tomorrow.

@brianschubert
Copy link
Collaborator

Hmm, I notice that if both typeguards use literals (instead of one using a union of literals), then we don't even put a union of TypeGuardTypes in the binder. This branch in make_simplified_union flattens them into non-TypeGuardType-wrapped literals/unions (though I'm not sure that's intentional), which avoids this crash.

Is having a union-of-TypeGuardTypes in the binder useful / meaningful? Or would it make sense to flatten that to a TypeGuardType-of-a-union or just a plain union before putting it in the binder?

@sterliakov
Copy link
Collaborator

Hm, transposing a union-of-guards into guard-of-union sounds reasonable too.

In the linked PR I settled on a slightly different approach, not as in the patch above: I propose to keep narrowing union members even for enum/literal-union overlap - why should meet keep all members or RHS union if any of them are present on the LHS enum type? Unless I'm missing a filter elsewhere, the block I replaced used to return anything from unrelated literals to unrelated types.

cf. playground

from enum import Enum
from typing_extensions import TypeIs, Literal

class Model(str, Enum):
    A = 'a'
    B = 'a'

def is_model_a(model: str) -> TypeIs[Literal[Model.A, "foo"]]:
    return True
def handle(model: Model) -> None:
    if is_model_a(model):
        reveal_type(model)  # N: Revealed type is "Union[Literal[__main__.Model.A], Literal['foo']]"


def is_int_or_list(model: object) -> TypeIs[int | list[int]]:
    return True
def compare(x: int | str) -> None:
    if is_int_or_list(x):
        reveal_type(x)  # N: Revealed type is "builtins.int"

(yes, I can't invent a simpler test to show meet behaviour, shame on me, and TypeIs is just easier to reason about - I hope this example is indeed a raw meet)

On my PR is_model_a reveals Model.A only, without foo added from nowhere.

sobolevn pushed a commit that referenced this issue Apr 11, 2025
…apping with enum (#18897)

Fixes #18895.

The original implementation of that block was introduced as a
performance optimization in #12032. It's in fact incorrect: it produces
overly optimistic meets, assuming that *any* match among union items
makes them *all* relevant. As discussed in #18895, this actually results
in unexpected `meet` behaviour, as demonstrated by

```python
from enum import Enum
from typing_extensions import TypeIs, Literal

class Model(str, Enum):
    A = 'a'
    B = 'a'

def is_model_a(model: str) -> TypeIs[Literal[Model.A, "foo"]]:
    return True
def handle(model: Model) -> None:
    if is_model_a(model):
        reveal_type(model)  # N: Revealed type is "Union[Literal[__main__.Model.A], Literal['foo']]"


def is_int_or_list(model: object) -> TypeIs[int | list[int]]:
    return True
def compare(x: int | str) -> None:
    if is_int_or_list(x):
        reveal_type(x)  # N: Revealed type is "builtins.int"
```

This patch restores filtering of union members, but keeps it running
before the expensive `is_overlapping_types` check involving expansion.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants