Feature: Use PEP 696 defaults for fields

[w0rp]

NOTE: This feature can't be worked on unless PEP 696 is accepted, but given its inclusion in typing_extensions I'd say it's likely it will be.

django-stubs uses set and get value generic types for fields, defined like so:

# __set__ value type
_ST = TypeVar("_ST", contravariant=True)
# __get__ return type
_GT = TypeVar("_GT", covariant=True)

class Field(RegisterLookupMixin, Generic[_ST, _GT]):

Specific field types use special attributes that the mypy plugin for django-stubs reads to fill in the get and set types with.

class CharField(Field[_ST, _GT]):
    _pyi_private_set_type: Union[str, int, Combinable]
    _pyi_private_get_type: str
    
    ...

This works for mypy, but not for other type checkers, where the generic parameters need to be written manually. To better support different type checkers, we could use the default values from PEP 696 instead.

_ST_CHAR = TypeVar("_ST_CHAR", contravariant=True, default=Union[str, int, Combinable])
_GT_CHAR = TypeVar("_GT_CHAR", covariant=True, default=str)

class CharField(Generic[_ST_CHAR, _GT_CHAR], Field[_ST_CHAR, _GT_CHAR]):
    ...

The mypy plugin could read the default values instead of the _pyi_private* attributes from before. For other type checkers, the instance values for __get__ and __set__ will then use the generic type defaults if not specified otherwise.

[intgr]

This feature can't be worked on unless PEP 696 is accepted

I think we don't necessarily have to wait for acceptance of the PEP. As long as it's implemented in typing_extensions, there is at least 1 type checker implementation that already supports it, and its adoption does not cause regressions for other type checkers (particularly mypy), I would be in support for merging this.

Once PEP 696 is implemented in mypy itself (python/mypy#14851), maybe we can get rid of this logic in the mypy plugin? Native type constructs tend to be less fragile and better supported in general.

[w0rp]

Thanks for the comment. Generally I would like django-stubs to be a bit more useful for tools like Pyright by default without making life harder for people who benefit from the mypy plugin.

[christianbundy]

Would this also provide the ability to have generic fields? For example:

class CharField(Field[_ST, _GT]):
    _pyi_private_set_type: _ST
    _pyi_private_get_type: _GT

My use-case: I have a custom field class (a subclass of TextField) that transforms data on its way to/from the database, but it doesn't seem to me that django-stubs supports any way to have generic setter/getter types via _pyi_private_{set,get}_type.

[flaeppe]

My current approach for custom fields is to cast them like e.g.

class SomeModel(models.Model):
    currency = cast(
        "models.CharField[Currency, Currency]",
        CurrencyField(default=None, verbose_name=_("currency")),
    )

It gets quite verbose though, but at least it works.

[Viicos]

This works for mypy, but not for other type checkers, where the generic parameters need to be written manually. To better support different type checkers, we could use the default values from PEP 696 instead.

_ST_CHAR = TypeVar("_ST_CHAR", contravariant=True, default=Union[str, int, Combinable])
_GT_CHAR = TypeVar("_GT_CHAR", covariant=True, default=str)

class CharField(Generic[_ST_CHAR, _GT_CHAR], Field[_ST_CHAR, _GT_CHAR]):
    ...

One issue with this is taking into account null, that when set to True allow _ST and _GT to be None. Instead of using type var defaults, this could be achieved with:

class Field(Generic[_ST, _GT]):
    @overload
    def __new__(
        cls,
        *,
        null: Literal[True],
    ) -> Field[_ST | None, _GT | None]: ...
    @overload
    def __new__(
        cls,
        *,
        null: Literal[False] = ...,
    ) -> Field[_ST, _GT]: ...

Then field subclasses can be written as:

class IntegerField(Field[float | int | str | Combinable, int]):
    ...

Working as expected:

from typing import Generic, TypeVar, Literal, overload


_ST = TypeVar("_ST")
_GT = TypeVar("_GT")


class Field(Generic[_ST, _GT]):
    @overload
    def __new__(
        cls,
        *,
        null: Literal[True],
    ) -> Field[_ST | None, _GT | None]: ...
    @overload
    def __new__(
        cls,
        *,
        null: Literal[False] = ...,
    ) -> Field[_ST, _GT]: ...

    def get_type(self) -> _GT: ...
    def set_type(self) -> _ST: ...

class IntegerField(Field[float | int | str, int]):
    pass


a = IntegerField(null=False)
reveal_type(a.get_type())  # Type of "a.get()" is "int"   + None if null=True
reveal_type(a.set_type())  # Type of "a.set()" is "float | int | str" + None if null=True

However, this requires two overloads each time a field is defined with a new signature (IntegerField is fine here, but this is not the case for all fields available in the stubs).

Edit: see https://discuss.python.org/t/41483, this is in fact not really possible.