builtins.slice should be generic

[o11c]

It is entirely reasonable that some libraries would have slices with a component type other than int.

[gvanrossum]

(a) Have you seen such code in practice?
(b) If you have, did it look likely they would want to use typing.py in the near future?
(c) We don't want to make builtins generic, but we could add typing.Slice, similar to the way we have typing.List.

[o11c]

(a) Numpy uses this, it has integer-like classes that don't subclass int, and they can be used with slices.
(b) if Numpy itself isn't interested, it has lots of users and some of them will want to use numpy with typing.
(c) For now, sure. Note that there will be irritating errors for people using 3.5's typing instead of 3.6's typing though. In the long run (Python 4), I am not convinced that adding generic info to builtins would be a bad thing.

[shoyer]

Indeed, in the scientific computing world, int and the NumPy integer types are often used interchangeably.

Pandas is another widely used library with objects that support label based slicing using non-integers. For example, you can write x['a':'b'] if x is a pandas.DataFrame. However, most sliceable objects do require integer bounds, and it would be nice to be able to tell these apart.

I think we are unlikely to add type annotations to pandas in the near future, but it does seem likely that some users will be interested in using numpy and pandas with type annotations.

[JukkaL]

Adding typing.Slice that is generic would be a straightforward change. However, if we follow the existing conventions, slice would then become an alias for Slice[Any], which would mean that code that wants to be Python 3.5 compatible or assumes that slices have integers only and uses slice would become imprecisely typed. My guess is that it's a minor issue for almost all programs.

[serhiy-storchaka]

If add generic typing.Slice, how much parameters should it have? slice() takes three arguments: start, stop and step. Do you want to specify all three types separately, e.g.

Slice[Optional[SupportsIndex], Optional[SupportsIndex], Optional[SupportsIndex]]

The type of __getindex__ can be not just a plain index or slice, but a tuple of plain indices and slices. So for NumPy-like arrays it is:

Union[SupportsIndex, slice, Tuple[Union[SupportsIndex, slice, Ellipsis]]]

[gvanrossum]

(I think you meant __getitem__, not __getindex__.)

It seems verbose to have Slice take three parameters. E.g. for list slices we'd get Slice[Optional[int], Optional[int], Optional[int]].

Are you aware of objects that take different types?

[shoyer]

In pandas, start/stop can be a different type than step, e.g., if you have a datetime index and want to select every 10th element between two dates, you could write df.loc[start_date:stop_date:10].

[ilevkivskyi]

@shoyer To support that I would propose to have two generic types (or better a type and an alias). For example in typeshed we can define:

# builtins.pyi
class slice(Generic[T, S, U]):
    ...
# typing.pyi
Slice = slice[T, T, T]

At runtime typing.Slice will be a _GenericAlias to slice, so that the former could be freely used in annotations and in other positions, as one would expect. While in rare cases where different types are needed for precise stubs one can still write slice[str, str, int] (because stubs are never evaluated).

This will deviate from the current common pattern, but this may be the case where practicality beats purity.

@JukkaL I don't think this will deteriorate type safety. First, the type arguments will be inferred, so that slice('start', 'end', 3) will be slice[str, str, int]. Second, they are currently all Any already in typesehed, see https://github.com/python/typeshed/blob/master/stdlib/2and3/builtins.pyi#L844.

[serhiy-storchaka]

Maybe we can assume that the type of start and stop is the same and step is always an integer-like or None.

[lig]

I would like to see Slice in typing. I'm writing some code which uses __getitem__ as part of its API and it will benefit from [a:b:c] argument types specified explicitly.

[gvanrossum]

Maybe we should include slice[T_start_stop, T_step] or slice[T_start, T_stop, T_step] in PEP 585?

[ilevkivskyi]

Maybe we should include slice[T_start_stop, T_step] or slice[T_start, T_stop, T_step] in PEP 585?

I think this makes sense. @ambv

[analog-cbarber]

You cannot assume that step is an integer. It could also be an interval.

[phdowling]

Maybe we should include slice[T_start_stop, T_step] or slice[T_start, T_stop, T_step] in PEP 585?

Anecdotal, but I've come across a use-case for separate T_start and T_stop types. It's a bit niche but here it goes: I was trying to create a utility class for generic function return type annotation, which could optionally associate a name with an output type. I would have liked the convenient syntax of def my_function(...) -> Output["useful_var_name": MyOutputType]: to specify that the function would return an object of type MyOutputType, and would later be be referred to as "varname" (in another context, not really relevant here). Below is how I imagine this would work:

# "useful_var_name" is not relevant in the application-level code, 
# but an underlying framework could use the annotation to do things like generate docs
def my_function(...) -> Output["useful_var_name": MyOutputType]:
    return MyOutputType()  # Output should to be implemented such that the type checker does not complain here


# this should also be allowed and do the same thing as above, minus the var name
def my_function(...) -> Output[MyOutputType]:
    return MyOutputType()  # again, type checker should not complain here

Making Output generic with respect to the the type parameter (specifically MyOutputType) of the slices (or even better, variadically generic for multiple slice objects with different T_stop types) would have been nice, but I couldn't figure out a way to do that without generic slices, and even with those I'd definitely have needed different T_start and T_stop types.

I guess the stub would be something like:

T = TypeVar("T")
class Output(Generic[T]):
    @overload
    def __class_getitem__(cls, item: slice[str, Type[T]]):
        ...
    def __class_getitem__(cls, item: Type[T]):
        ...

Using the new variadic typevars, I imagine one could even make it do something like:

def my_function(...) -> Output["useful_var_name": MyOutputType, "other_var": SecondOutputType]:  # can arbitrarily add more types here
    return MyOutputType(), SecondOutputType()

From what I could tell, this did not make it into PEP 585 - any chance it'll make it into a future version? I'm also not too well-versed in the internals of the typing module, but is there a straightforward way one could already implement something like this without deeper changes to typing? I realize this might be pretty much outside of standard / intended use-cases for slices (and type annotations for that matter).

[harashm]

Hi,
Is there any workaround to silence this error:

error: Slice index must be an integer or None [misc]

?
Adding 'type: ignore[misc]' all over the place is very annoying...
Thanks !

[srittau]

This is an issue for either typeshed and/or the CPython bug tracker, closing it here.