Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

1.7 Crash on recursive type #16473

Closed
NMertsch opened this issue Nov 13, 2023 · 2 comments · Fixed by #16483
Closed

1.7 Crash on recursive type #16473

NMertsch opened this issue Nov 13, 2023 · 2 comments · Fixed by #16483
Labels
crash topic-overlap Overlapping equality check topic-recursive-types

Comments

@NMertsch
Copy link

NMertsch commented Nov 13, 2023 •
edited
Loading

Crash Report

I have a recursive type NestedDict = dict[str, "NestedDict"]. Using it in a separate module causes a Segmentation fault with mypy --strict.
This worked with 1.6.1, but failed with 1.7 and also with master.

Traceback
Using the non-compiled 1.7 and master version results in this:

Traceback (most recent call last):
  File "/home/mertschn/.conda/envs/mypy-repro/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/main.py", line 100, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/main.py", line 182, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/build.py", line 191, in build
    result = _build(
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/build.py", line 265, in _build
    graph = dispatch(sources, manager, stdout)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/build.py", line 2943, in dispatch
    process_graph(graph, manager)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/build.py", line 3341, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/build.py", line 3442, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/build.py", line 2311, in type_check_first_pass
    self.type_checker().check_first_pass()
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 481, in check_first_pass
    self.accept(d)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 591, in accept
    stmt.accept(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/nodes.py", line 787, in accept
    return visitor.visit_func_def(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 1000, in visit_func_def
    self._visit_func_def(defn)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 1004, in _visit_func_def
    self.check_func_item(defn, name=defn.name)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 1077, in check_func_item
    self.check_func_def(defn, typ, name, allow_empty)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 1293, in check_func_def
    self.accept(item.body)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 591, in accept
    stmt.accept(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/nodes.py", line 1223, in accept
    return visitor.visit_block(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 2770, in visit_block
    self.accept(s)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 591, in accept
    stmt.accept(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/nodes.py", line 1436, in accept
    return visitor.visit_assert_stmt(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checker.py", line 4565, in visit_assert_stmt
    self.expr_checker.accept(s.expr)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checkexpr.py", line 5716, in accept
    typ = node.accept(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/nodes.py", line 2076, in accept
    return visitor.visit_comparison_expr(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checkexpr.py", line 3573, in visit_comparison_expr
    if self.dangerous_comparison(left_type, right_type):
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checkexpr.py", line 3705, in dangerous_comparison
    ) or self.dangerous_comparison(left.args[1], right.args[1])
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checkexpr.py", line 3705, in dangerous_comparison
    ) or self.dangerous_comparison(left.args[1], right.args[1])
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checkexpr.py", line 3705, in dangerous_comparison
    ) or self.dangerous_comparison(left.args[1], right.args[1])
  [Previous line repeated 16340 more times]
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/checkexpr.py", line 3701, in dangerous_comparison
    left = map_instance_to_supertype(left, abstract_map)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/maptype.py", line 42, in map_instance_to_supertype
    return map_instance_to_supertypes(instance, superclass)[0]
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/maptype.py", line 54, in map_instance_to_supertypes
    a.extend(map_instance_to_direct_supertypes(t, sup))
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/maptype.py", line 96, in map_instance_to_direct_supertypes
    t = expand_type_by_instance(b, instance)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/expandtype.py", line 119, in expand_type_by_instance
    return expand_type(typ, variables)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/expandtype.py", line 71, in expand_type
    return typ.accept(ExpandTypeVisitor(env))
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/types.py", line 1435, in accept
    return visitor.visit_instance(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/expandtype.py", line 214, in visit_instance
    args = self.expand_types_with_unpack(list(t.args))
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/expandtype.py", line 396, in expand_types_with_unpack
    items.append(item.accept(self))
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/types.py", line 635, in accept
    return visitor.visit_type_var(self)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/expandtype.py", line 234, in visit_type_var
    return repl.copy_modified(last_known_value=None)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/types.py", line 1488, in copy_modified
    new = Instance(
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/types.py", line 1373, in __init__
    super().__init__(line, column)
  File "/home/mertschn/.conda/envs/mypy-repro/lib/python3.10/site-packages/mypy/types.py", line 234, in __init__
    super().__init__(line, column)
RecursionError: maximum recursion depth exceeded while calling a Python object

To Reproduce

Code and instructions in this repo: https://github.com/NMertsch/mypy-repro.
tl;dr: I define a recursive type NestedDict = dict[str, "NestedDict"]. Using it in the same file is fine, but in my tests, it causes a crash.

Your Environment

  • Mypy version used: Installed from master, commit efa5dcb (reproducible with 1.7, works with 1.6.1)
  • Mypy command-line flags: --strict
  • Mypy configuration options from mypy.ini (and other config files): None
  • Python version used: 3.10.13
  • Operating system and version: Windows 10, Ubuntu 20.04.5
@NMertsch NMertsch added the crash label Nov 13, 2023
@AlexWaygood AlexWaygood added topic-recursive-types topic-overlap Overlapping equality check labels Nov 13, 2023
@AlexWaygood
Copy link
Member

Simpler repro: rather than running mypy --strict . (which enables many mypy options) on https://github.com/NMertsch/mypy-repro, just run mypy --strict-equality . (which enables only one mypy option) on the repo.

@AlexWaygood
Copy link
Member 

a3af87bf252f0ed0c6e0f977ad4079418b37a70f is the first bad commit
commit a3af87bf252f0ed0c6e0f977ad4079418b37a70f
Author: Ivan Levkivskyi <levkivskyi@gmail.com>
Date:   Sat Oct 21 16:41:10 2023 +0100

    Narrow tuple types using len() (#16237)

    Fixes #1178
    Supersedes #10367

    This is includes implementation for fixed length tuples, homogeneous
    tuples, and variadic tuples (and combinations of those). Generally
    implementation is straightforward. Some notes:
    * Unfortunately, it is necessary to add a new attribute `min_len` to
    `TypeVarTupleType`, which is probably fine, as it doesn't have that many
    attributes so far.
    * Supporting more general use cases (like `>` comparisons for variadic
    tuples) can cause quick proliferation of unions. I added two mechanisms
    to counteract this: not applying the narrowing if the integer literal in
    comparison is itself large, and collapsing unions of tuples into a
    single tuple (if possible) after we are done with the narrowing. This
    looks a bit arbitrary, but I think it is important to have.
    * Main missing feature here is probably not inferring type information
    from indirect comparisons like `len(x) > foo() > 1`. Supporting this
    kind of things in full generality is cumbersome, and we may add cases
    that turn out to be important later.
    * Note I am quite careful with indexing "inside" a `TypeVarTuple`, it is
    not really needed now, but I wanted to make everything future proof, so
    that it will be easy to add support for upper bounds for
    `TypeVarTuple`s, like `Nums = TypeVarTuple("Nums", bound=tuple[float,
    ...])`.
    * I also fix couple existing inconsistencies with `Any` handling in type
    narrowing. It looks like they stem from the old incorrect logic that
    meet of `Any` and `X` should be `X`, while in fact it should be `Any`.
    These fixes are not strictly necessary, but otherwise there may be new
    false positives, because I introduce a bunch of additional type
    narrowing scenarios here.

    cc @hatal175, thanks for the test cases, and for your nice first attempt
    to implement this!
    Co-authored-by: Tal Hayon <talhayon1@gmail.com>

 mypy/binder.py                        |  83 +++++
 mypy/checker.py                       | 359 ++++++++++++++++++++-
 mypy/checkexpr.py                     |  53 +++-
 mypy/meet.py                          |   6 +-
 mypy/operators.py                     |  23 ++
 mypy/options.py                       |   3 +-
 mypy/subtypes.py                      |   2 +-
 mypy/test/testcheck.py                |   2 +-
 mypy/typeops.py                       |   2 +-
 mypy/types.py                         |  27 +-
 mypy_self_check.ini                   |   1 +
 test-data/unit/check-expressions.test |  13 +
 test-data/unit/check-namedtuple.test  |   2 +-
 test-data/unit/check-narrowing.test   | 576 ++++++++++++++++++++++++++++++++++
 test-data/unit/fixtures/len.pyi       |  39 +++
 test-data/unit/lib-stub/typing.pyi    |   1 +
 16 files changed, 1154 insertions(+), 38 deletions(-)
 create mode 100644 test-data/unit/fixtures/len.pyi

Bisects to a3af87b (cc. @ilevkivskyi)

@ilevkivskyi ilevkivskyi mentioned this issue Nov 13, 2023
Merged
ilevkivskyi added a commit that referenced this issue Nov 15, 2023
@ilevkivskyi @AlexWaygood
Fix crash on strict-equality with recursive types (#16483)
0699dde 
Fixes #16473

Potentially we can turn this helper function into a proper visitor, but
I don't think it is worth it as of right now.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
JukkaL pushed a commit that referenced this issue Nov 22, 2023
@ilevkivskyi @AlexWaygood @JukkaL
Fix crash on strict-equality with recursive types (#16483)
661adb7 
Fixes #16473

Potentially we can turn this helper function into a proper visitor, but
I don't think it is worth it as of right now.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
crash topic-overlap Overlapping equality check topic-recursive-types
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix crash on strict-equality with recursive types ilevkivskyi/mypy
2 participants
@NMertsch @AlexWaygood