Ignore doesn't work when two errors share a line

[NeilGirdhar]

from typing import Any

from array_api_compat import get_namespace
import numpy as np
import numpy.typing as npt

def create_diagonal_array(m: npt.NDArray[Any]) -> npt.NDArray[Any]:
    """A vectorized version of diagonal.

    Args:
        m: Has shape (*k, n)
    Returns: Array with shape (*k, n, n) and the elements of m on the diagonals.
    """
    xp = get_namespace(m)
    pre = m.shape[:-1]
    n = m.shape[-1]
    s = (*m.shape, n)
    retval = xp.zeros((*pre, n ** 2), dtype=m.dtype)
    for index in np.ndindex(*pre):
        target_index = (*index, slice(None, None, n + 1))
        source_values = m[*index, :]  # type: ignore[arg-type]
        retval[target_index] = source_values
    return xp.reshape(retval, s)

Produces an error without a line number for a line on which the error should be ignored!

[brianschubert]

Simplified reproducer:

from typing import Any
import numpy.typing as npt

m: npt.NDArray[Any]
index: tuple[int, ...]
m[*index, :]

which outputs

main.py: error: Argument 2 to <tuple> has incompatible type "slice[Any, Any, Any]"; expected "ndarray[Any, dtype[integer[Any]]] | ndarray[Any, dtype[bool]]"  [arg-type]
main.py:6: error: Argument 1 to <tuple> has incompatible type "*tuple[int, ...]"; expected "ndarray[Any, dtype[integer[Any]]] | ndarray[Any, dtype[bool]]"  [arg-type]

[brianschubert]

Reproducer without numpy:

class Foo:
    def __getitem__(self, key: tuple[str, ...]) -> None: ...

m: Foo
index: tuple[int, ...]
m[*index, :]

outputs

main.py: error: Argument 2 to <tuple> has incompatible type "slice[Any, Any, Any]"; expected "str"  [arg-type]
main.py:6: error: Argument 1 to <tuple> has incompatible type "*tuple[int, ...]"; expected "str"  [arg-type]

[brianschubert]

Some notes for posterity:

• This issue only occurs when
  1. __getitem__ is typed to accept arbitrary-length tuples (tuple[T, ...]), and
  2. the index includes a star expression (*index), and
  3. the index also includes a slice expression.
• If any of the above conditions aren't satisfied, you'll get a clearer error along the lines Invalid index type "tuple[T1, ...]" for "Foo"; expected type "tuple[T2 ...]" instead of the slew of Argument n to <tuple> has incompatible type errors (ideally, this case should emit the first kind of error too).
• In brief, this issue happens because
  • When a tuple expression contains a star expression (*xyz,), mypy infers the overall type by treating this as a call to a hidden tuple-constructor function with the signature def [T] (*T) -> tuple[T, ...]
  • In order to resolve the type variable in the tuple-constructor, mypy looks at the type context and tries to match it against the return type. Here, the type context is the parameter to __getitem__. If there's a match (as is the case when __getitem__ is typed to accept a variable-length tuple), mypy will resolve the type variable using the parameter type. In the example above, where __getitem__ accepts a tuple[str, ...], this results in the tuple-constructor signature getting resolved to def (*str) -> tuple[str, ...]
  • If mypy resolved the type variable using the type context, then type errors will be emitted for any incompatible arguments to the tuple-constructor. This is where the Argument n to <tuple> has incompatible type errors come from.
  • When an argument comes from a slice expression, the context for the error message will be the corresponding SliceExpr node. Currently, mypy doesn't set line information on these nodes, which is why the error message has no line number. (This is apparently because slice expressions don't contain line information in the raw ast in Python 3.8).
  • Since the overall type of the tuple expression is copied from the type context (__getitem__), the actual call to __getitem__ type checks, hence no Invalid index type X for Y error is emitted

[brianschubert]

Because slice expressions don't contain information in the raw ast before Python 3.9, fixing this while mypy still supports 3.8 is a bit complicated. Fortunately, mypy will be dropping support for 3.8 in the near future. Once that happens, I have a patch ready.

In the meantime, slice(None) can be used in place of : as a workaround.

[NeilGirdhar]

Perfect, thanks for all the research and fix! I'll just wait for MyPy to drop 3.8.