Skip to content

Crash with --enable-incomplete-feature=Unpack and sliced numpy arrays. #15897

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
gatoniel opened this issue Aug 17, 2023 · 0 comments · Fixed by #15924
Closed

Crash with --enable-incomplete-feature=Unpack and sliced numpy arrays. #15897

gatoniel opened this issue Aug 17, 2023 · 0 comments · Fixed by #15924
Labels
crash topic-pep-646 PEP 646 (TypeVarTuple, Unpack)

Comments

@gatoniel
Copy link

Crash Report
I get an INTERNAL ERROR when I use --enable-incomplete-feature=Unpack with the code described below.

Traceback

mypy_unpack.py:47: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.6.0+dev.76c16a484b8b4fa6ae10abf4e96a286315ed7093
Traceback (most recent call last):
  File "/home/niklas/.pyenv/versions/mypy/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/main.py", line 99, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/main.py", line 178, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/build.py", line 189, in build
    result = _build(
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/build.py", line 262, in _build
    graph = dispatch(sources, manager, stdout)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/build.py", line 2938, in dispatch
    process_graph(graph, manager)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/build.py", line 3336, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/build.py", line 3437, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/build.py", line 2306, in type_check_first_pass
    self.type_checker().check_first_pass()
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checker.py", line 475, in check_first_pass
    self.accept(d)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checker.py", line 585, in accept
    stmt.accept(self)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/nodes.py", line 790, in accept
    return visitor.visit_func_def(self)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checker.py", line 972, in visit_func_def
    self._visit_func_def(defn)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checker.py", line 976, in _visit_func_def
    self.check_func_item(defn, name=defn.name)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checker.py", line 1049, in check_func_item
    self.check_func_def(defn, typ, name, allow_empty)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checker.py", line 1259, in check_func_def
    self.accept(item.body)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checker.py", line 585, in accept
    stmt.accept(self)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/nodes.py", line 1226, in accept
    return visitor.visit_block(self)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checker.py", line 2732, in visit_block
    self.accept(s)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checker.py", line 585, in accept
    stmt.accept(self)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/nodes.py", line 1313, in accept
    return visitor.visit_assignment_stmt(self)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checker.py", line 2780, in visit_assignment_stmt
    self.check_assignment(s.lvalues[-1], s.rvalue, s.type is None, s.new_syntax)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checker.py", line 2987, in check_assignment
    rvalue_type = self.expr_checker.accept(rvalue, type_context=type_context)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checkexpr.py", line 5269, in accept
    typ = node.accept(self)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/nodes.py", line 1907, in accept
    return visitor.visit_call_expr(self)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checkexpr.py", line 452, in visit_call_expr
    return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checkexpr.py", line 581, in visit_call_expr_inner
    ret_type = self.check_call_expr_with_callee_type(
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checkexpr.py", line 1371, in check_call_expr_with_callee_type
    ret_type, callee_type = self.check_call(
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checkexpr.py", line 1454, in check_call
    return self.check_callable_call(
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checkexpr.py", line 1617, in check_callable_call
    callee = self.infer_function_type_arguments(
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checkexpr.py", line 1925, in infer_function_type_arguments
    arg_types = self.infer_arg_types_in_context(
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checkexpr.py", line 1814, in infer_arg_types_in_context
    res[ai] = self.accept(args[ai], arg_type)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checkexpr.py", line 5269, in accept
    typ = node.accept(self)
          ^^^^^^^^^^^^^^^^^
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/nodes.py", line 2316, in accept
    return visitor.visit_generator_expr(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checkexpr.py", line 5036, in visit_generator_expr
    return self.check_generator_or_comprehension(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checkexpr.py", line 5050, in check_generator_or_comprehension
    self.check_for_comp(gen)
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checkexpr.py", line 5122, in check_for_comp
    _, sequence_type = self.chk.analyze_iterable_item_type(sequence)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/checker.py", line 4648, in analyze_iterable_item_type
    joined = join_types(joined, item)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/join.py", line 252, in join_types
    return t.accept(TypeJoinVisitor(s, instance_joiner))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/types.py", line 1058, in accept
    return visitor.visit_unpack_type(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/niklas/.pyenv/versions/3.11.1/envs/mypy/lib/python3.11/site-packages/mypy/join.py", line 315, in visit_unpack_type
    raise NotImplementedError
NotImplementedError:
mypy_unpack.py:47: : note: use --pdb to drop into pdb

To Reproduce

I ran mypy --enable-incomplete-feature=Unpack mypy_unpack.py --show-traceback with the following code:

# mypy_unpack.py
from __future__ import annotations

from typing import TypeVar
from typing import Any
from typing import Unpack

import numpy as np
import numpy.typing as npt


T = TypeVar("T", bound=np.generic)
U = TypeVar("U", bound=np.unsignedinteger[Any])


def sample_patches(
    datas: tuple[npt.NDArray[U], Unpack[tuple[npt.NDArray[T], ...]]],
    patch_size: tuple[int, ...],
    n_samples: int,
    valid_inds: tuple[npt.NDArray[np.uint32], ...] | None = None,
    verbose: bool = False,
) -> tuple[npt.NDArray[U], Unpack[tuple[npt.NDArray[T], ...]]]:
    """Version of stardist.sample_patches.sample_patches for time stacks."""

    if valid_inds is None:
        valid_inds = tuple(
            _s.ravel()
            for _s in np.meshgrid(
                *tuple(
                    np.arange(p // 2, s - p // 2 + 1)
                    for s, p in zip(datas[0].shape[1:], patch_size)
                )
            )
        )

    n_valid = len(valid_inds[0])

    if n_valid == 0:
        raise ValueError("no regions to sample from!")

    idx = list(
        np.random.choice(
            range(n_valid), n_samples, replace=(n_valid < n_samples)
        )
    )
    rand_inds = [v[idx] for v in valid_inds]
    res = tuple(
        np.stack(
            [
                data[
                    (slice(None),)
                    + tuple(
                        slice(_r - (_p // 2), _r + _p - (_p // 2))
                        for _r, _p in zip(r, patch_size)
                    )
                ]
                for r in zip(*rand_inds)
            ]
        )
        for data in datas
    )

    return res

Your Environment

  • Mypy version used: 1.6.0+dev.76c16a484b8b4fa6ae10abf4e96a286315ed7093
  • Mypy command-line flags: --enable-incomplete-feature=Unpack
  • Mypy configuration options from mypy.ini (and other config files): plugins = "numpy.typing.mypy_plugin" is part of [tool.mypy] in the pyproject.toml.
  • Python version used: 3.11.1
  • Operating system and version: Ubuntu 22.04.1 LTS
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
crash topic-pep-646 PEP 646 (TypeVarTuple, Unpack)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Streamline some elements of variadic types support ilevkivskyi/mypy
2 participants
@gatoniel @AlexWaygood