Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

INTERNAL ERROR in check_argument_types #15495

Closed
LukeHankey opened this issue Jun 22, 2023 · 1 comment · Fixed by #15924
Closed

INTERNAL ERROR in check_argument_types #15495

LukeHankey opened this issue Jun 22, 2023 · 1 comment · Fixed by #15924
Labels
crash topic-pep-646 PEP 646 (TypeVarTuple, Unpack)

Comments

@LukeHankey
Copy link

Crash Report

Ran mypy with experimental features (Unpack, TypeVarTuple)

Traceback

src/features/imputation.py:8: error: Skipping analyzing "missingpy": module is installed, but missing library stubs or py.typed marker  [import]
src/features/imputation.py:8: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
src/features/imputation.py:188: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.5.0+dev.7d031beb017450ac990c97e288d064290e3be55f
Traceback (most recent call last):
  File "/home/luke/oxcan-fs/venv/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/main.py", line 95, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/main.py", line 174, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/build.py", line 196, in build
    result = _build(
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/build.py", line 269, in _build
    graph = dispatch(sources, manager, stdout)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/build.py", line 2926, in dispatch
    process_graph(graph, manager)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/build.py", line 3324, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/build.py", line 3425, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/build.py", line 2311, in type_check_first_pass
    self.type_checker().check_first_pass()
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 472, in check_first_pass
    self.accept(d)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 582, in accept
    stmt.accept(self)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/nodes.py", line 1139, in accept
    return visitor.visit_class_def(self)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 2231, in visit_class_def
    self.accept(defn.defs)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 582, in accept
    stmt.accept(self)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/nodes.py", line 1220, in accept
    return visitor.visit_block(self)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 2684, in visit_block
    self.accept(s)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 582, in accept
    stmt.accept(self)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/nodes.py", line 786, in accept
    return visitor.visit_func_def(self)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 962, in visit_func_def
    self._visit_func_def(defn)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 966, in _visit_func_def
    self.check_func_item(defn, name=defn.name)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 1038, in check_func_item
    self.check_func_def(defn, typ, name, allow_empty)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 1234, in check_func_def
    self.accept(item.body)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 582, in accept
    stmt.accept(self)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/nodes.py", line 1220, in accept
    return visitor.visit_block(self)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 2684, in visit_block
    self.accept(s)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 582, in accept
    stmt.accept(self)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/nodes.py", line 1488, in accept
    return visitor.visit_if_stmt(self)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 4328, in visit_if_stmt
    self.accept(b)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 582, in accept
    stmt.accept(self)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/nodes.py", line 1220, in accept
    return visitor.visit_block(self)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 2684, in visit_block
    self.accept(s)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 582, in accept
    stmt.accept(self)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/nodes.py", line 1307, in accept
    return visitor.visit_assignment_stmt(self)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 2732, in visit_assignment_stmt
    self.check_assignment(s.lvalues[-1], s.rvalue, s.type is None, s.new_syntax)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 2905, in check_assignment
    rvalue_type = self.check_simple_assignment(lvalue_type, rvalue, context=rvalue)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checker.py", line 3955, in check_simple_assignment
    rvalue_type = self.expr_checker.accept(
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checkexpr.py", line 5016, in accept
    typ = node.accept(self)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/nodes.py", line 1901, in accept
    return visitor.visit_call_expr(self)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checkexpr.py", line 438, in visit_call_expr
    return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checkexpr.py", line 558, in visit_call_expr_inner
    ret_type = self.check_call_expr_with_callee_type(
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checkexpr.py", line 1218, in check_call_expr_with_callee_type
    ret_type, callee_type = self.check_call(
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checkexpr.py", line 1301, in check_call
    return self.check_callable_call(
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checkexpr.py", line 1492, in check_callable_call
    self.check_argument_types(
  File "/home/luke/oxcan-fs/venv/lib/python3.10/site-packages/mypy/checkexpr.py", line 2155, in check_argument_types
    assert isinstance(unpacked_type, Instance)
AssertionError: 
src/features/imputation.py:188: : note: use --pdb to drop into pdb

To Reproduce

from typing import Any
from typing import Generic
from typing import TypedDict

import pandas as pd
from missingpy import MissForest
from typing_extensions import TypeVarTuple
from typing_extensions import Unpack


class MissForestParams(TypedDict, total=False):
    """Typing class for MissForest parameters."""

    pass


Ts = TypeVarTuple("Ts")


class Imputer(Generic[Unpack[Ts]]):

    def __init__(self, data: pd.DataFrame) -> None:
        self.data = data

    def __call__(
        self,
        method: str,
        /,
        *args: Unpack[Ts],
        **kwargs: Any,
    ) -> pd.DataFrame:
        imputed_data = self.data
        if method == "miss_forest":
            imputed_data = self._miss_forest(self.data, *args, **kwargs)

        return imputed_data

    def _miss_forest(self, data: pd.DataFrame, *args: Unpack[Ts], **kwargs: Unpack[MissForestParams]) -> pd.DataFrame:
        imputer = MissForest(*args, **kwargs)

        return pd.DataFrame(imputer.fit_transform(data), columns=data.columns)

Your Environment

  • Mypy version used: mypy 1.5.0+dev.7d031beb017450ac990c97e288d064290e3be55f (compiled: no). Also ran on 1.4.0 before checking current GitHub version.
  • Mypy command-line flags:
  • Mypy configuration options from mypy.ini (and other config files):
[tool.mypy]
python_version = "3.10"
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
warn_redundant_casts = true
warn_unreachable = true
show_error_codes = true
disallow_any_generics = true
follow_imports = "silent"
enable_incomplete_feature = ["Unpack", "TypeVarTuple"]
  • Python version used: 3.10.6
  • Operating system and version: Windows 11, Version 22H2
@AlexWaygood AlexWaygood added the topic-pep-646 PEP 646 (TypeVarTuple, Unpack) label Jun 22, 2023
@AlexWaygood
Copy link
Member

Very similar traceback to #15241

ilevkivskyi added a commit that referenced this issue Aug 23, 2023
Fixes #13981
Fixes #15241
Fixes #15495
Fixes #15633
Fixes #15667
Fixes #15897
Fixes #15929

OK, I started following the plan outlined in
#15879. In this PR I focused mostly
on "kinematics". Here are some notes (in random order):
* I decided to normalize `TupleType` and `Instance` items in
`semanal_typeargs.py` (not in the type constructors, like for unions).
It looks like a simpler way to normalize for now. After this, we can
rely on the fact that only non-trivial (more below on what is trivial)
variadic items in a type list is either `*Ts` or `*tuple[X, ...]`. A
single top-level `TupleType` can appear in an unpack only as type of
`*args`.
* Callables turned out to be tricky. There is certain tight coupling
between `FuncDef.type` and `FuncDef.arguments` that makes it hard to
normalize prefix to be expressed as individual arguments _at
definition_. I faced exactly the same problem when I implemented `**`
unpacking for TypedDicts. So we have two choices: either handle prefixes
everywhere, or use normalization helper in relevant code. I propose to
go with the latter (it worked well for `**` unpacking).
* I decided to switch `Unpack` to be disallowed by default in
`typeanal.py`, only very specific potions are allowed now. Although this
required plumbing `allow_unpack` all the way from `semanal.py`,
conceptually it is simple. This is similar to how `ParamSpec` is
handled.
* This PR fixes all currently open crash issues (some intentionally,
some accidentally) plus a bunch of TODOs I found in the tests (but not
all).
* I decided to simplify `TypeAliasExpr` (and made it simple reference to
the `SymbolNode`, like e.g. `TypedDictExpr` and `NamedTupleExpr`). This
is not strictly necessary for this PR, but it makes some parts of it a
bit simpler, and I wanted to do it for long time.

Here is a more detailed plan of what I am leaving for future PRs (in
rough order of priority):
* Close non-crash open issues (it looks like there are only three, and
all seem to be straightforward)
* Handle trivial items in `UnpackType` gracefully. These are `<nothing>`
and `Any` (and also potentially `object`). They can appear e.g. after a
user error. Currently they can cause assert crashes. (Not sure what is
the best way to do this).
* Go over current places where `Unpack` is handled, and verify both
possible variadic items are handled.
* Audit variadic `Instance` constrains and subtyping (the latter is
probably OK, but the former may be broken).
* Audit `Callable` and `Tuple` subtyping for variadic-related edge cases
(constraints seem OK for these).
* Figure out story about `map_instance_to_supertype()` (if no changes
are needed, add tests for subclassing).
* Clear most remaining TODOs.
* Go once more over the large scale picture and check whether we have
some important parts missing (or unhandled interactions between those).
* Verify various "advanced" typing features work well with
`TypeVarTuple`s (and add some support if missing but looks important).
* Enable this feature by default.

I hope to finish these in next few weeks.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
crash topic-pep-646 PEP 646 (TypeVarTuple, Unpack)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants