Support variadic generics

[dmontagu]

We eventually want to support variadic generics.

There are already some tests for the desired behavior (thanks @caniko).

I'm creating this issue for tracking purposes as I have closed #5351.

[caniko]

Any news on this front?

[dmontagu]

Working on it now. I think there may be some issues with the existing xfailing tests for this stuff, and it's a bit more complicated than I expected, but making progress..

[dmontagu]

I think we'll need to modify pydantic-core to support a tuple schema with a variable-length item in the "middle"; right now, I believe the tuple-positional schema only supports variable-number of "tail" items, but in https://peps.python.org/pep-0646/ it is clearly supported to have any number of specific parameters at the start and/or end of the tuple.

I would propose that we actually unify the tuple-positional and tuple-variable schemas, with a schema roughly along the lines of:

class TupleSchema(TypedDict, total=False):
    type: Required[Literal['tuple']]
    leading_items_schema: List[CoreSchema]
    trailing_items_schema: List[CoreSchema]
    variadic_schema: CoreSchema
    min_variadic_length: int
    max_variadic_length: int
    strict: bool
    ref: str
    metadata: Any
    serialization: IncExSeqOrElseSerSchema

[JacobHayes]

In case it's helpful, I'm trying out this __class_getitem__ override to merge the separate TypeVarTuple arguments into a single tuple argument:

    @classmethod
    def __class_getitem__(cls, items: type | tuple[type, ...]):
        # NOTE: Pydantic doesn't support variadic generics yet[1], so we merge any TypeVarTuple
        # argument(s) into a single tuple for now.
        # - 1: https://github.com/pydantic/pydantic/issues/5804
        params = cls.__pydantic_generic_metadata__["parameters"]
        # There can be only a single TypeVarTuple parameter.
        tvt_idx = next((i for i, p in enumerate(params) if isinstance(p, TypeVarTuple)), None)
        if tvt_idx is not None and isinstance(items, tuple):
            # Calculate the number of items for TypeVarTuple
            n_before = len(params[:tvt_idx])
            n_after = len(params[tvt_idx + 1 :])
            n_tuple_items = max(0, len(items) - n_before - n_after)
            tuple_items = items[tvt_idx : tvt_idx + n_tuple_items]
            # If the argument isn't still generic, convert the argument(s) to a tuple so pydantic
            # only sees a single element.
            if n_tuple_items == 1 and get_origin(tuple_items[0]) is Unpack:
                middle = (tuple_items[0],)
            else:
                middle = (tuple[*tuple_items],)  # pyright: ignore # noqa: PGH003 # no Pyright code available
            items = (*items[:n_before], *middle, *items[-n_after:])
        return super().__class_getitem__(items)

This fixes the TypeError when subscripting a class containing a TypeVarTuple with more than a single variadic argument, but I'm not actually sure the effect it has (or not) on field (de)serialization (the TypeVarTuples I'm using are just for methods currently, not fields).

[purepani]

Would definitely love support for this! @sydney-runkle you mentioned in a duplicate issue that there was infrastructure in pydantic-core for this already; could you point to how this might be implemented, as it wasn't super obvious to me (this would be my first time contributing so I'm not super familiar with the code base)