diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 542b0bdfb4a..adde7dd8ca9 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -643,6 +643,7 @@ peps/pep-0762.rst @pablogsal @ambv @lysnikolaou @emilyemorehouse peps/pep-0763.rst @dstufft peps/pep-0765.rst @iritkatriel @ncoghlan peps/pep-0766.rst @warsaw +peps/pep-0767.rst @carljm # ... peps/pep-0777.rst @warsaw # ... diff --git a/peps/pep-0767.rst b/peps/pep-0767.rst new file mode 100644 index 00000000000..7adc7cf62ca --- /dev/null +++ b/peps/pep-0767.rst @@ -0,0 +1,650 @@ +PEP: 767 +Title: Annotating Read-Only Attributes +Author: Eneg +Sponsor: Carl Meyer +Discussions-To: https://discuss.python.org/t/expanding-readonly-to-normal-classes-protocols/67359 +Status: Draft +Type: Standards Track +Topic: Typing +Created: 18-Nov-2024 +Python-Version: 3.14 + + +Abstract +======== + +:pep:`705` introduced the :external+py3.13:data:`typing.ReadOnly` type qualifier +to allow defining read-only :class:`typing.TypedDict` items. + +This PEP proposes using ``ReadOnly`` in :term:`annotations ` of class and protocol +:term:`attributes `, as a single concise way to mark them read-only. + +Akin to :pep:`705`, it makes no changes to setting attributes at runtime. Correct +usage of read-only attributes is intended to be enforced only by static type checkers. + + +Motivation +========== + +The Python type system lacks a single concise way to mark an attribute read-only. +This feature is present in other statically and gradually typed languages +(such as `C# `_ +or `TypeScript `_), +and is useful for removing the ability to reassign or ``del``\ ete an attribute +at a type checker level, as well as defining a broad interface for structural subtyping. + +.. _classes: + +Classes +------- + +Today, there are three major ways of achieving read-only attributes, honored by type checkers: + +* annotating the attribute with :data:`typing.Final`:: + + class Foo: + number: Final[int] + + def __init__(self, number: int) -> None: + self.number = number + + + class Bar: + def __init__(self, number: int) -> None: + self.number: Final = number + + - Supported by :mod:`dataclasses` (and type checkers since `typing#1669 `_). + - Overriding ``number`` is not possible - the specification of ``Final`` + imposes that the name cannot be overridden in subclasses. + +* read-only proxy via ``@property``:: + + class Foo: + _number: int + + def __init__(self, number: int) -> None: + self._number = number + + @property + def number(self) -> int: + return self._number + + - Overriding ``number`` is possible. *Type checkers disagree about the specific rules*. [#overriding_property]_ + - Read-only at runtime. [#runtime]_ + - Requires extra boilerplate. + - Supported by :mod:`dataclasses`, but does not compose well - the synthesized + ``__init__`` and ``__repr__`` will use ``_number`` as the parameter/attribute name. + +* using a "freezing" mechanism, such as :func:`dataclasses.dataclass` or :class:`typing.NamedTuple`:: + + @dataclass(frozen=True) + class Foo: + number: int # implicitly read-only + + + class Bar(NamedTuple): + number: int # implicitly read-only + + - Overriding ``number`` is possible in the ``@dataclass`` case. + - Read-only at runtime. [#runtime]_ + - No per-attribute control - these mechanisms apply to the whole class. + - Frozen dataclasses incur some runtime overhead. + - ``NamedTuple`` is still a ``tuple``. Most classes do not need to inherit + indexing, iteration, or concatenation. + +.. _protocols: + +Protocols +--------- + +A read-only attribute ``name: T`` on a :class:`~typing.Protocol` in principle +defines two requirements: + +1. ``hasattr(obj, "name")`` +2. ``isinstance(obj.name, T)`` + +Those requirements are satisfiable at runtime by all of the following: + +* an object with an attribute ``name: T``, +* a class with a class variable ``name: ClassVar[T]``, +* an instance of the class above, +* an object with a ``@property`` ``def name(self) -> T``, +* an object with a custom descriptor, such as :func:`functools.cached_property`. + +The current `typing spec `_ +allows creation of such protocol members using (abstract) properties:: + + class HasName(Protocol): + @property + def name(self) -> T: ... + +This syntax has several drawbacks: + +* It is somewhat verbose. +* It is not obvious that the quality conveyed here is the read-only character of a property. +* It is not composable with :external+typing:term:`type qualifiers `. +* Not all type checkers agree [#property_in_protocol]_ that all of the above five + objects are assignable to this structural type. + +Rationale +========= + +These problems can be resolved by an attribute-level type qualifier. +``ReadOnly`` has been chosen for this role, as its name conveys the intent well, +and the newly proposed changes complement its semantics defined in :pep:`705`. + +A class with a read-only instance attribute can now be defined as:: + + from typing import ReadOnly + + + class Member: + def __init__(self, id: int) -> None: + self.id: ReadOnly[int] = id + +...and the protocol described in :ref:`protocols` is now just:: + + from typing import Protocol, ReadOnly + + + class HasName(Protocol): + name: ReadOnly[str] + + + def greet(obj: HasName, /) -> str: + return f"Hello, {obj.name}!" + +* A subclass of ``Member`` can redefine ``.id`` as a writable attribute or a + :term:`descriptor`. It can also :external+typing:term:`narrow` the type. +* The ``HasName`` protocol has a more succinct definition, and is agnostic + to the writability of the attribute. +* The ``greet`` function can now accept a wide variety of compatible objects, + while being explicit about no modifications being done to the input. + + +Specification +============= + +The :external+py3.13:data:`typing.ReadOnly` :external+typing:term:`type qualifier` +becomes a valid annotation for :term:`attributes ` of classes and protocols. +It can be used at class-level or within ``__init__`` to mark individual attributes read-only:: + + class Book: + id: ReadOnly[int] + + def __init__(self, id: int, name: str) -> None: + self.id = id + self.name: ReadOnly[str] = name + +Type checkers should error on any attempt to reassign or ``del``\ ete an attribute +annotated with ``ReadOnly``. +Type checkers should also error on any attempt to delete an attribute annotated as ``Final``. +(This is not currently specified.) + +Use of ``ReadOnly`` in annotations at other sites where it currently has no meaning +(such as local/global variables or function parameters) is considered out of scope +for this PEP. + +Akin to ``Final`` [#final_mutability]_, ``ReadOnly`` does not influence how +type checkers perceive the mutability of the assigned object. Immutable :term:`ABCs ` +and :mod:`containers ` may be used in combination with ``ReadOnly`` +to forbid mutation of such values at a type checker level: + +.. code-block:: python + + from collections import abc + from dataclasses import dataclass + from typing import Protocol, ReadOnly + + + @dataclass + class Game: + name: str + + + class HasGames[T: abc.Collection[Game]](Protocol): + games: ReadOnly[T] + + + def add_games(shelf: HasGames[list[Game]]) -> None: + shelf.games.append(Game("Half-Life")) # ok: list is mutable + shelf.games[-1].name = "Black Mesa" # ok: "name" is not read-only + shelf.games = [] # error: "games" is read-only + del shelf.games # error: "games" is read-only and cannot be deleted + + + def read_games(shelf: HasGames[abc.Sequence[Game]]) -> None: + shelf.games.append(...) # error: "Sequence" has no attribute "append" + shelf.games[0].name = "Blue Shift" # ok: "name" is not read-only + shelf.games = [] # error: "games" is read-only + + +All instance attributes of frozen dataclasses and ``NamedTuple`` should be +implied to be read-only. Type checkers may inform that annotating such attributes +with ``ReadOnly`` is redundant, but it should not be seen as an error: + +.. code-block:: python + + from dataclasses import dataclass + from typing import NewType, ReadOnly + + + @dataclass(frozen=True) + class Point: + x: int # implicit read-only + y: ReadOnly[int] # ok, redundant + + + uint = NewType("uint", int) + + + @dataclass(frozen=True) + class UnsignedPoint(Point): + x: ReadOnly[uint] # ok, redundant; narrower type + y: Final[uint] # not redundant, Final imposes extra restrictions; narrower type + +.. _init: + +Initialization +-------------- + +Assignment to a read-only attribute can only occur in the class declaring the attribute. +There is no restriction to how many times the attribute can be assigned to. +The assignment must be allowed in the following contexts: + +* In ``__init__``, on the instance received as the first parameter (likely, ``self``). +* In ``__new__``, on instances of the declaring class created via a call + to a super-class' ``__new__`` method. +* At declaration in the body of the class. + +Additionally, a type checker may choose to allow the assignment: + +* In ``__new__``, on instances of the declaring class, without regard + to the origin of the instance. + (This choice trades soundness, as the instance may already be initialized, + for the simplicity of implementation.) +* In ``@classmethod``\ s, on instances of the declaring class created via + a call to the class' or super-class' ``__new__`` method. + +Note that a child class cannot assign to any read-only attributes of a parent class +in any of the aforementioned contexts, unless the attribute is redeclared. + +.. code-block:: python + + from collections import abc + from typing import ReadOnly + + + class Band: + name: str + songs: ReadOnly[list[str]] + + def __init__(self, name: str, songs: abc.Iterable[str] | None = None) -> None: + self.name = name + self.songs = [] + + if songs is not None: + self.songs = list(songs) # multiple assignments are fine + + def clear(self) -> None: + # error: assignment to read-only "songs" outside initialization + self.songs = [] + + + band = Band(name="Bôa", songs=["Duvet"]) + band.name = "Python" # ok: "name" is not read-only + band.songs = [] # error: "songs" is read-only + band.songs.append("Twilight") # ok: list is mutable + + + class SubBand(Band): + def __init__(self) -> None: + self.songs = [] # error: cannot assign to a read-only attribute of a base class + +.. code-block:: python + + # a simplified immutable Fraction class + class Fraction: + numerator: ReadOnly[int] + denominator: ReadOnly[int] + + def __new__( + cls, + numerator: str | int | float | Decimal | Rational = 0, + denominator: int | Rational | None = None + ) -> Self: + self = super().__new__(cls) + + if denominator is None: + if type(numerator) is int: + self.numerator = numerator + self.denominator = 1 + return self + + elif isinstance(numerator, Rational): ... + + else: ... + + @classmethod + def from_float(cls, f: float, /) -> Self: + self = super().__new__(cls) + self.numerator, self.denominator = f.as_integer_ratio() + return self + +When a class-level declaration has an initializing value, it can serve as a `flyweight `_ +default for instances: + +.. code-block:: python + + class Patient: + number: ReadOnly[int] = 0 + + def __init__(self, number: int | None = None) -> None: + if number is not None: + self.number = number + +.. note:: + This feature conflicts with :data:`~object.__slots__`. An attribute with + a class-level value cannot be included in slots, effectively making it a class variable. + +Type checkers may choose to warn on read-only attributes which could be left uninitialized +after an instance is created (except in :external+typing:term:`stubs `, +protocols or ABCs):: + + class Patient: + id: ReadOnly[int] # error: "id" is not initialized on all code paths + name: ReadOnly[str] # error: "name" is never initialized + + def __init__(self) -> None: + if random.random() > 0.5: + self.id = 123 + + + class HasName(Protocol): + name: ReadOnly[str] # ok + +Subtyping +--------- + +Read-only attributes are covariant. This has a few subtyping implications. +Borrowing from :pep:`705#inheritance`: + +* Read-only attributes can be redeclared as writable attributes, descriptors + or class variables:: + + @dataclass + class HasTitle: + title: ReadOnly[str] + + + @dataclass + class Game(HasTitle): + title: str + year: int + + + game = Game(title="DOOM", year=1993) + game.year = 1994 + game.title = "DOOM II" # ok: attribute is not read-only + + + class TitleProxy(HasTitle): + @functools.cached_property + def title(self) -> str: ... + + + class SharedTitle(HasTitle): + title: ClassVar[str] = "Still Grey" + +* If a read-only attribute is not redeclared, it remains read-only:: + + class Game(HasTitle): + year: int + + def __init__(self, title: str, year: int) -> None: + super().__init__(title) + self.title = title # error: cannot assign to a read-only attribute of base class + self.year = year + + + game = Game(title="Robot Wants Kitty", year=2010) + game.title = "Robot Wants Puppy" # error: "title" is read-only + +* Subtypes can :external+typing:term:`narrow` the type of read-only attributes:: + + class GameCollection(Protocol): + games: ReadOnly[abc.Collection[Game]] + + + @dataclass + class GameSeries(GameCollection): + name: str + games: ReadOnly[list[Game]] # ok: list[Game] is assignable to Collection[Game] + +* Nominal subclasses of protocols and ABCs should redeclare read-only attributes + in order to implement them, unless the base class initializes them in some way:: + + class MyBase(abc.ABC): + foo: ReadOnly[int] + bar: ReadOnly[str] = "abc" + baz: ReadOnly[float] + + def __init__(self, baz: float) -> None: + self.baz = baz + + @abstractmethod + def pprint(self) -> None: ... + + + @final + class MySubclass(MyBase): + # error: MySubclass does not override "foo" + + def pprint(self) -> None: + print(self.foo, self.bar, self.baz) + +* In a protocol attribute declaration, ``name: ReadOnly[T]`` indicates that a structural + subtype must support ``.name`` access, and the returned value is assignable to ``T``:: + + class HasName(Protocol): + name: ReadOnly[str] + + + class NamedAttr: + name: str + + class NamedProp: + @property + def name(self) -> str: ... + + class NamedClassVar: + name: ClassVar[str] + + class NamedDescriptor: + @cached_property + def name(self) -> str: ... + + # all of the following are ok + has_name: HasName + has_name = NamedAttr() + has_name = NamedProp() + has_name = NamedClassVar + has_name = NamedClassVar() + has_name = NamedDescriptor() + +Interaction with Other Type Qualifiers +-------------------------------------- + +``ReadOnly`` can be used with ``ClassVar`` and ``Annotated`` in any nesting order: + +.. code-block:: python + + class Foo: + foo: ClassVar[ReadOnly[str]] = "foo" + bar: Annotated[ReadOnly[int], Gt(0)] + +.. code-block:: python + + class Foo: + foo: ReadOnly[ClassVar[str]] = "foo" + bar: ReadOnly[Annotated[int, Gt(0)]] + +This is consistent with the interaction of ``ReadOnly`` and :class:`typing.TypedDict` +defined in :pep:`705`. + +An attribute annotated as both ``ReadOnly`` and ``ClassVar`` can only be assigned to +at declaration in the class body. + +An attribute cannot be annotated as both ``ReadOnly`` and ``Final``, as the two +qualifiers differ in semantics, and ``Final`` is generally more restrictive. +``Final`` remains allowed as an annotation of attributes that are only implied +to be read-only. It can be also used to redeclare a ``ReadOnly`` attribute of a base class. + + +Backwards Compatibility +======================= + +This PEP introduces new contexts where ``ReadOnly`` is valid. Programs inspecting +those places will have to change to support it. This is expected to mainly affect type checkers. + +However, caution is advised while using the backported ``typing_extensions.ReadOnly`` +in older versions of Python. Mechanisms inspecting annotations may behave incorrectly +when encountering ``ReadOnly``; in particular, the ``@dataclass`` decorator +which `looks for `_ +``ClassVar`` may mistakenly treat ``ReadOnly[ClassVar[...]]`` as an instance attribute. + +To avoid issues with introspection, use ``ClassVar[ReadOnly[...]]`` instead of ``ReadOnly[ClassVar[...]]``. + + +Security Implications +===================== + +There are no known security consequences arising from this PEP. + + +How to Teach This +================= + +Suggested changes to the :mod:`typing` module documentation, +following the footsteps of :pep:`705#how-to-teach-this`: + +* Add this PEP to the others listed. +* Link :external+py3.13:data:`typing.ReadOnly` to this PEP. +* Update the description of ``typing.ReadOnly``: + + A special typing construct to mark an attribute of a class or an item of + a ``TypedDict`` as read-only. + +* Add a standalone entry for ``ReadOnly`` under the + `type qualifiers `_ section: + + The ``ReadOnly`` type qualifier in class attribute annotations indicates + that the attribute of the class may be read, but not reassigned or ``del``\ eted. + For usage in ``TypedDict``, see `ReadOnly `_. + + +Rejected Ideas +============== + +Clarifying Interaction of ``@property`` and Protocols +----------------------------------------------------- + +The :ref:`protocols` section mentions an inconsistency between type checkers in +the interpretation of properties in protocols. The problem could be fixed +by amending the typing specification, clarifying what implements the read-only +quality of such properties. + +This PEP makes ``ReadOnly`` a better alternative for defining read-only attributes +in protocols, superseding the use of properties for this purpose. + + +Assignment Only in ``__init__`` and Class Body +---------------------------------------------- + +An earlier version of this PEP proposed that read-only attributes could only be +assigned to in ``__init__`` and the class' body. A later discussion revealed that +this restriction would severely limit the usability of ``ReadOnly`` within +immutable classes, which typically do not define ``__init__``. + +:class:`fractions.Fraction` is one example of an immutable class, where the +initialization of its attributes happens within ``__new__`` and classmethods. +However, unlike in ``__init__``, the assignment in ``__new__`` and classmethods +is potentially unsound, as the instance they work on can be sourced from +an arbitrary place, including an already finalized instance. + +We find it imperative that this type checking feature is useful to the foremost +use site of read-only attributes - immutable classes. Thus, the PEP has changed +since to allow assignment in ``__new__`` and classmethods under a set of rules +described in the :ref:`init` section. + + +Open Issues +=========== + +Extending Initialization +------------------------ + +Mechanisms such as :func:`dataclasses.__post_init__` or attrs' `initialization hooks `_ +augment object creation by providing a set of special hooks which are called +during initialization. + +The current initialization rules defined in this PEP disallow assignment to +read-only attributes in such methods. It is unclear whether the rules could be +satisfyingly shaped in a way that is inclusive of those 3rd party hooks, while +upkeeping the invariants associated with the read-only-ness of those attributes. + +The Python type system has a long and detailed `specification `_ +regarding the behavior of ``__new__`` and ``__init__``. It is rather unfeasible +to expect the same level of detail from 3rd party hooks. + +A potential solution would involve type checkers providing configuration in this +regard, requiring end users to manually specify a set of methods they wish +to allow initialization in. This however could easily result in users mistakenly +or purposefully breaking the aforementioned invariants. It is also a fairly +big ask for a relatively niche feature. + +``ReadOnly[ClassVar[...]]`` and ``__init_subclass__`` +----------------------------------------------------- + +Should read-only class variables be assignable to within the declaring class' +``__init_subclass__``? + +.. code-block:: python + + class URI: + protocol: ReadOnly[ClassVar[str]] = "" + + def __init_subclass__(cls, protocol: str = "") -> None: + cls.foo = protocol + + class File(URI, protocol="file"): ... + + +Footnotes +========= + +.. [#overriding_property] + Pyright in strict mode disallows non-property overrides. + Mypy does not impose this restriction and allows an override with a plain attribute. + `[Pyright playground] `_ + `[mypy playground] `_ + +.. [#runtime] + This PEP focuses solely on the type-checking behavior. Nevertheless, it should + be desirable the name is read-only at runtime. + +.. [#property_in_protocol] + Pyright disallows class variable and non-property descriptor overrides. + `[Pyright] `_ + `[mypy] `_ + `[Pyre] `_ + +.. [#final_mutability] + As noted above the second-to-last code example of https://typing.readthedocs.io/en/latest/spec/qualifiers.html#semantics-and-examples + + +Copyright +========= + +This document is placed in the public domain or under the +CC0-1.0-Universal license, whichever is more permissive.