Generic bounded dataclass crash #10140

xanderbailey · 2021-02-26T14:07:33Z

Crash Report

Ran mypy on my user library - no idea what went wrong.

Traceback

  File "/Users/name/.env/myenv/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/Users/name/.env/myenv/lib/python3.9/site-packages/mypy/__main__.py", line 11, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "/Users/name/.env/myenv/lib/python3.9/site-packages/mypy/main.py", line 98, in main
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/Users/name/.env/myenv/lib/python3.9/site-packages/mypy/build.py", line 179, in build
    result = _build(
  File "/Users/name/.env/myenv/lib/python3.9/site-packages/mypy/build.py", line 253, in _build
    graph = dispatch(sources, manager, stdout)
  File "/Users/name/.env/myenv/lib/python3.9/site-packages/mypy/build.py", line 2688, in dispatch
    process_graph(graph, manager)
  File "/Users/name/.env/myenv/lib/python3.9/site-packages/mypy/build.py", line 3012, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/Users/name/.env/myenv/lib/python3.9/site-packages/mypy/build.py", line 3104, in process_stale_scc
    mypy.semanal_main.semantic_analysis_for_scc(graph, scc, manager.errors)
  File "/Users/name/.env/myenv/lib/python3.9/site-packages/mypy/semanal_main.py", line 79, in semantic_analysis_for_scc
    process_functions(graph, scc, patches)
  File "/Users/name/.env/myenv/lib/python3.9/site-packages/mypy/semanal_main.py", line 233, in process_functions
    process_top_level_function(analyzer,
  File "/Users/name/.env/myenv/lib/python3.9/site-packages/mypy/semanal_main.py", line 274, in process_top_level_function
    deferred, incomplete, progress = semantic_analyze_target(target, state, node, active_type,
  File "/Users/name/.env/myenv/lib/python3.9/site-packages/mypy/semanal_main.py", line 326, in semantic_analyze_target
    analyzer.refresh_partial(refresh_node,
  File "/Users/name/.env/myenv/lib/python3.9/site-packages/mypy/semanal.py", line 397, in refresh_partial
    self.accept(node)
  File "/Users/name/.env/myenv/lib/python3.9/site-packages/mypy/semanal.py", line 4835, in accept
    node.accept(self)
  File "/Users/name/.env/myenv/lib/python3.9/site-packages/mypy/nodes.py", line 687, in accept
    return visitor.visit_func_def(self)
  File "/Users/name/.env/myenv/lib/python3.9/site-packages/mypy/semanal.py", line 573, in visit_func_def
    self.analyze_func_def(defn)
  File "/Users/name/.env/myenv/lib/python3.9/site-packages/mypy/semanal.py", line 605, in analyze_func_def
    self.defer(defn)
  File "/Users/name/.env/myenv/lib/python3.9/site-packages/mypy/semanal.py", line 4554, in defer
    assert not self.final_iteration, 'Must not defer during final iteration'
AssertionError: Must not defer during final iteration

To Reproduce

from dataclasses import dataclass
import dataclasses
from typing import Any, Dict, Generic, List, Type, TypeVar, Union
from abc import ABC, abstractclassmethod

from occupier.utils import EnhancedJSONEncoder

T = TypeVar("T", bound="DictDataClass")


@dataclass(frozen=True)
class DictDataClass(Generic[T]):
    urn: int

    def to_dict(self) -> Dict[str, Any]:
        return dataclasses.asdict(self)

    @classmethod
    def from_dict(cls: Type[T], dict_: Dict[str, Any]) -> T:
        return cls(**dict_)

    @classmethod
    def field_names(cls) -> List[str]:
        return [field.name for field in dataclasses.fields(cls)]


@dataclass(frozen=True)
class JSONDataClass(ABC, DictDataClass):

    _JSON_ENCODER = EnhancedJSONEncoder()

    def to_json(self) -> str:
        return JSONDataClass._JSON_ENCODER.encode(self)

    @classmethod
    @abstractclassmethod
    def from_json(cls: Type[T], json_str: str) -> Union[List[T], T]:
        pass

Your Environment

Mypy version used : 0.820+dev.effd970ad1e8bb09fd2a18634339e1d043a83400
Mypy command-line flags: None
Mypy configuration options from mypy.ini (and other config files):
Python version used: 3.9.1
Operating system and version: MacOS 10.15.7

The text was updated successfully, but these errors were encountered:

ethframe · 2021-03-05T09:24:03Z

[edit by JukkaL: this is tracked in #12112]

I've encountered similar crash:

Traceback (most recent call last):
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\mypy\mypy\__main__.py", line 23, in <module>
    console_entry()
  File "C:\mypy\mypy\__main__.py", line 11, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "C:\mypy\mypy\main.py", line 98, in main
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "C:\mypy\mypy\build.py", line 180, in build
    sources, options, alt_lib_path, flush_errors, fscache, stdout, stderr, extra_plugins
  File "C:\mypy\mypy\build.py", line 253, in _build
    graph = dispatch(sources, manager, stdout)
  File "C:\mypy\mypy\build.py", line 2688, in dispatch
    process_graph(graph, manager)
  File "C:\mypy\mypy\build.py", line 3012, in process_graph
    process_stale_scc(graph, scc, manager)
  File "C:\mypy\mypy\build.py", line 3104, in process_stale_scc
    mypy.semanal_main.semantic_analysis_for_scc(graph, scc, manager.errors)
  File "C:\mypy\mypy\semanal_main.py", line 79, in semantic_analysis_for_scc
    process_functions(graph, scc, patches)
  File "C:\mypy\mypy\semanal_main.py", line 239, in process_functions
    patches)
  File "C:\mypy\mypy\semanal_main.py", line 275, in process_top_level_function
    final_iteration, patches)
  File "C:\mypy\mypy\semanal_main.py", line 331, in semantic_analyze_target
    active_type=active_type)
  File "C:\mypy\mypy\semanal.py", line 397, in refresh_partial
    self.accept(node)
  File "C:\mypy\mypy\semanal.py", line 4835, in accept
    node.accept(self)
  File "C:\mypy\mypy\nodes.py", line 687, in accept
    return visitor.visit_func_def(self)
  File "C:\mypy\mypy\semanal.py", line 573, in visit_func_def
    self.analyze_func_def(defn)
  File "C:\mypy\mypy\semanal.py", line 616, in analyze_func_def
    self.analyze_function_body(defn)
  File "C:\mypy\mypy\semanal.py", line 940, in analyze_function_body
    defn.body.accept(self)
  File "C:\mypy\mypy\nodes.py", line 1015, in accept
    return visitor.visit_block(self)
  File "C:\mypy\mypy\semanal.py", line 3297, in visit_block
    self.accept(s)
  File "C:\mypy\mypy\semanal.py", line 4835, in accept
    node.accept(self)
  File "C:\mypy\mypy\nodes.py", line 950, in accept
    return visitor.visit_class_def(self)
  File "C:\mypy\mypy\semanal.py", line 1048, in visit_class_def
    self.analyze_class(defn)
  File "C:\mypy\mypy\semanal.py", line 1125, in analyze_class
    self.analyze_class_body_common(defn)
  File "C:\mypy\mypy\semanal.py", line 1133, in analyze_class_body_common
    defn.defs.accept(self)
  File "C:\mypy\mypy\nodes.py", line 1015, in accept
    return visitor.visit_block(self)
  File "C:\mypy\mypy\semanal.py", line 3297, in visit_block
    self.accept(s)
  File "C:\mypy\mypy\semanal.py", line 4835, in accept
    node.accept(self)
  File "C:\mypy\mypy\nodes.py", line 687, in accept
    return visitor.visit_func_def(self)
  File "C:\mypy\mypy\semanal.py", line 573, in visit_func_def
    self.analyze_func_def(defn)
  File "C:\mypy\mypy\semanal.py", line 602, in analyze_func_def
    result = analyzer.visit_callable_type(defn.type, nested=False)
  File "C:\mypy\mypy\typeanal.py", line 537, in visit_callable_type
    variables=self.anal_var_defs(variables),
  File "C:\mypy\mypy\typeanal.py", line 978, in anal_var_defs
    return [self.anal_var_def(vd) for vd in var_defs]
  File "C:\mypy\mypy\typeanal.py", line 978, in <listcomp>
    return [self.anal_var_def(vd) for vd in var_defs]
  File "C:\mypy\mypy\typeanal.py", line 970, in anal_var_def
    var_def.upper_bound.accept(self),
  File "C:\mypy\mypy\types.py", line 1964, in accept
    return visitor.visit_placeholder_type(self)
  File "C:\mypy\mypy\typeanal.py", line 663, in visit_placeholder_type
    self.api.defer()  # Still incomplete
  File "C:\mypy\mypy\semanal.py", line 4554, in defer
    assert not self.final_iteration, 'Must not defer during final iteration'
AssertionError: Must not defer during final iteration

Code to reproduce:

from typing import TypeVar

def test() -> None:
    T = TypeVar('T', bound='Foo')

    class Foo:
        def bar(self, foo: T) -> None:
            pass

Mypy version used: 0.820+dev.c8bae06919674b9846e3ff864b0a44592db888eb
Mypy command-line flags: none
Mypy configuration options from mypy.ini (and other config files): none
Python version used: 3.6.8
Operating system and version: Windows 8.1

JukkaL · 2021-03-05T13:15:42Z

Here is a simplified version of the first example:

from dataclasses import dataclass
from typing import Generic, TypeVar

T = TypeVar("T", bound="C")

@dataclass
class C(Generic[T]):
    x: int

dragonpaw · 2021-09-09T01:53:13Z

I think I ran into this with a Django project as well. The following code seems to trigger a crash that seems related.

T = TypeVar("T", bound="MyModel")

class MyModel(Generic[T], models.Model):
    field = models.DateField(db_index=True)

Oddly enough, it's specific to the field type. A CharField won't cause a crash. A DateField will.

/Users/ash/src/ticketing-django/ticketing/utils/models.py:191: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.rtfd.io/en/latest/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.812
Traceback (most recent call last):
  File "/Users/ash/.pyenv/versions/django/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/Users/ash/.pyenv/versions/3.9.4/envs/django/lib/python3.9/site-packages/mypy/__main__.py", line 11, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "/Users/ash/.pyenv/versions/3.9.4/envs/django/lib/python3.9/site-packages/mypy/main.py", line 90, in main
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/Users/ash/.pyenv/versions/3.9.4/envs/django/lib/python3.9/site-packages/mypy/build.py", line 179, in build
    result = _build(
  File "/Users/ash/.pyenv/versions/3.9.4/envs/django/lib/python3.9/site-packages/mypy/build.py", line 253, in _build
    graph = dispatch(sources, manager, stdout)
  File "/Users/ash/.pyenv/versions/3.9.4/envs/django/lib/python3.9/site-packages/mypy/build.py", line 2638, in dispatch
    process_graph(graph, manager)
  File "/Users/ash/.pyenv/versions/3.9.4/envs/django/lib/python3.9/site-packages/mypy/build.py", line 2962, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/Users/ash/.pyenv/versions/3.9.4/envs/django/lib/python3.9/site-packages/mypy/build.py", line 3054, in process_stale_scc
    mypy.semanal_main.semantic_analysis_for_scc(graph, scc, manager.errors)
  File "/Users/ash/.pyenv/versions/3.9.4/envs/django/lib/python3.9/site-packages/mypy/semanal_main.py", line 79, in semantic_analysis_for_scc
    process_functions(graph, scc, patches)
  File "/Users/ash/.pyenv/versions/3.9.4/envs/django/lib/python3.9/site-packages/mypy/semanal_main.py", line 233, in process_functions
    process_top_level_function(analyzer,
  File "/Users/ash/.pyenv/versions/3.9.4/envs/django/lib/python3.9/site-packages/mypy/semanal_main.py", line 274, in process_top_level_function
    deferred, incomplete, progress = semantic_analyze_target(target, state, node, active_type,
  File "/Users/ash/.pyenv/versions/3.9.4/envs/django/lib/python3.9/site-packages/mypy/semanal_main.py", line 326, in semantic_analyze_target
    analyzer.refresh_partial(refresh_node,
  File "/Users/ash/.pyenv/versions/3.9.4/envs/django/lib/python3.9/site-packages/mypy/semanal.py", line 397, in refresh_partial
    self.accept(node)
  File "/Users/ash/.pyenv/versions/3.9.4/envs/django/lib/python3.9/site-packages/mypy/semanal.py", line 4835, in accept
    node.accept(self)
  File "/Users/ash/.pyenv/versions/3.9.4/envs/django/lib/python3.9/site-packages/mypy/nodes.py", line 687, in accept
    return visitor.visit_func_def(self)
  File "/Users/ash/.pyenv/versions/3.9.4/envs/django/lib/python3.9/site-packages/mypy/semanal.py", line 573, in visit_func_def
    self.analyze_func_def(defn)
  File "/Users/ash/.pyenv/versions/3.9.4/envs/django/lib/python3.9/site-packages/mypy/semanal.py", line 605, in analyze_func_def
    self.defer(defn)
  File "/Users/ash/.pyenv/versions/3.9.4/envs/django/lib/python3.9/site-packages/mypy/semanal.py", line 4554, in defer
    assert not self.final_iteration, 'Must not defer during final iteration'
AssertionError: Must not defer during final iteration
/Users/ash/src/myproject-repo/myproject/utils/models.py:191: : note: use --pdb to drop into pdb

AlexWaygood · 2022-03-24T14:42:11Z

Looks like the same assertion that's hit in #12113, #12252, #11728, #10672 and #12112

Gallinaryoso · 2022-05-09T23:10:47Z

Ran into the exact same issue as @dragonpaw - copy/pasting DateField and its parents into the file in which the model is defined resolves the issue, so it looks import-related

JukkaL · 2022-05-11T12:56:12Z

The first example was fixed by #12762.

The issue reported by @ethframe is different (but related). We should probably move it to another issue.

I haven't looked into the issue reported by @dragonpaw and @Gallinaryoso. Are you using the mypy django plugin by any chance?

JukkaL · 2022-05-11T13:00:27Z

@ethframe's issue looks very similar to #12112 -- it's related to classes defined within functions. Let's track it there.

hauntsaninja · 2022-05-23T23:11:32Z

Closing, since the original issue is fixed and we have another issue for ethframe's report. If you have a different issue that is not yet addressed, please open a new issue, since this one is getting confusing.

xanderbailey added the crash label Feb 26, 2021

hauntsaninja mentioned this issue Sep 6, 2021

Crash with TypeVar with bound and @dataclass #10705

Closed

hauntsaninja changed the title ~~Something to do with Generic[T]? Crashed out on: AssertionError: Must not defer during final iteration~~ Generic bounded dataclass crash Sep 6, 2021

AlexWaygood added the topic-dataclasses label Mar 24, 2022

JukkaL added a commit that referenced this issue May 11, 2022

Add another test case from #10140

379b8aa

hauntsaninja closed this as completed May 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generic bounded dataclass crash #10140

Generic bounded dataclass crash #10140

xanderbailey commented Feb 26, 2021

ethframe commented Mar 5, 2021 •

edited by JukkaL

Loading

JukkaL commented Mar 5, 2021

dragonpaw commented Sep 9, 2021

AlexWaygood commented Mar 24, 2022 •

edited

Loading

Gallinaryoso commented May 9, 2022

JukkaL commented May 11, 2022

JukkaL commented May 11, 2022

hauntsaninja commented May 23, 2022

Generic bounded dataclass crash #10140

Generic bounded dataclass crash #10140

Comments

xanderbailey commented Feb 26, 2021

ethframe commented Mar 5, 2021 • edited by JukkaL Loading

JukkaL commented Mar 5, 2021

dragonpaw commented Sep 9, 2021

AlexWaygood commented Mar 24, 2022 • edited Loading

Gallinaryoso commented May 9, 2022

JukkaL commented May 11, 2022

JukkaL commented May 11, 2022

hauntsaninja commented May 23, 2022

ethframe commented Mar 5, 2021 •

edited by JukkaL

Loading

AlexWaygood commented Mar 24, 2022 •

edited

Loading