Skip to content

Commit

Permalink
[doc] bpo-45680: Disambiguate __getitem__ and ``__class_getitem__…
Browse files Browse the repository at this point in the history
…`` in the data model (pythonGH-29389)

The documentation explaining Python's data model does not adequately explain
the differences between ``__getitem__`` and ``__class_getitem__``, nor does it
explain when each is called. There is an attempt at explaining
``__class_getitem__`` in the documentation for ``GenericAlias`` objects, but
this does not give sufficient clarity into how the method works. Moreover, it
is the wrong place for that information to be found; the explanation of
``__class_getitem__`` should be in the documentation explaining the data model.

This PR has been split off from pythonGH-29335.
  • Loading branch information
AlexWaygood authored Nov 18, 2021
1 parent c94664c commit 31b3a70
Show file tree
Hide file tree
Showing 2 changed files with 147 additions and 18 deletions.
1 change: 1 addition & 0 deletions Doc/library/typing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,7 @@ called :class:`TypeVar`.
def first(l: Sequence[T]) -> T: # Generic function
return l[0]

.. _user-defined-generics:

User-defined generic types
==========================
Expand Down
164 changes: 146 additions & 18 deletions Doc/reference/datamodel.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2215,22 +2215,142 @@ case the instance is itself a class.
Emulating generic types
-----------------------

One can implement the generic class syntax as specified by :pep:`484`
(for example ``List[int]``) by defining a special method:
When using :term:`type annotations<annotation>`, it is often useful to
*parameterize* a :term:`generic type` using Python's square-brackets notation.
For example, the annotation ``list[int]`` might be used to signify a
:class:`list` in which all the elements are of type :class:`int`.

.. seealso::

:pep:`484` - Type Hints
Introducing Python's framework for type annotations

:ref:`Generic Alias Types<types-genericalias>`
Documentation for objects representing parameterized generic classes

:ref:`Generics`, :ref:`user-defined generics<user-defined-generics>` and :class:`typing.Generic`
Documentation on how to implement generic classes that can be
parameterized at runtime and understood by static type-checkers.

A class can *generally* only be parameterized if it defines the special
class method ``__class_getitem__()``.

.. classmethod:: object.__class_getitem__(cls, key)

Return an object representing the specialization of a generic class
by type arguments found in *key*.

This method is looked up on the class object itself, and when defined in
the class body, this method is implicitly a class method. Note, this
mechanism is primarily reserved for use with static type hints, other usage
is discouraged.
When defined on a class, ``__class_getitem__()`` is automatically a class
method. As such, there is no need for it to be decorated with
:func:`@classmethod<classmethod>` when it is defined.

.. seealso::

:pep:`560` - Core support for typing module and generic types
The purpose of *__class_getitem__*
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The purpose of :meth:`~object.__class_getitem__` is to allow runtime
parameterization of standard-library generic classes in order to more easily
apply :term:`type hints<type hint>` to these classes.

To implement custom generic classes that can be parameterized at runtime and
understood by static type-checkers, users should either inherit from a standard
library class that already implements :meth:`~object.__class_getitem__`, or
inherit from :class:`typing.Generic`, which has its own implementation of
``__class_getitem__()``.

Custom implementations of :meth:`~object.__class_getitem__` on classes defined
outside of the standard library may not be understood by third-party
type-checkers such as mypy. Using ``__class_getitem__()`` on any class for
purposes other than type hinting is discouraged.


.. _classgetitem-versus-getitem:


*__class_getitem__* versus *__getitem__*
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Usually, the :ref:`subscription<subscriptions>` of an object using square
brackets will call the :meth:`~object.__getitem__` instance method defined on
the object's class. However, if the object being subscribed is itself a class,
the class method :meth:`~object.__class_getitem__` may be called instead.
``__class_getitem__()`` should return a :ref:`GenericAlias<types-genericalias>`
object if it is properly defined.

Presented with the :term:`expression` ``obj[x]``, the Python interpreter
follows something like the following process to decide whether
:meth:`~object.__getitem__` or :meth:`~object.__class_getitem__` should be
called::

from inspect import isclass

def subscribe(obj, x):
"""Return the result of the expression `obj[x]`"""

class_of_obj = type(obj)

# If the class of obj defines __getitem__,
# call class_of_obj.__getitem__(obj, x)
if hasattr(class_of_obj, '__getitem__'):
return class_of_obj.__getitem__(obj, x)

# Else, if obj is a class and defines __class_getitem__,
# call obj.__class_getitem__(x)
elif isclass(obj) and hasattr(obj, '__class_getitem__'):
return obj.__class_getitem__(x)

# Else, raise an exception
else:
raise TypeError(
f"'{class_of_obj.__name__}' object is not subscriptable"
)

In Python, all classes are themselves instances of other classes. The class of
a class is known as that class's :term:`metaclass`, and most classes have the
:class:`type` class as their metaclass. :class:`type` does not define
:meth:`~object.__getitem__`, meaning that expressions such as ``list[int]``,
``dict[str, float]`` and ``tuple[str, bytes]`` all result in
:meth:`~object.__class_getitem__` being called::

>>> # list has class "type" as its metaclass, like most classes:
>>> type(list)
<class 'type'>
>>> type(dict) == type(list) == type(tuple) == type(str) == type(bytes)
True
>>> # "list[int]" calls "list.__class_getitem__(int)"
>>> list[int]
list[int]
>>> # list.__class_getitem__ returns a GenericAlias object:
>>> type(list[int])
<class 'types.GenericAlias'>

However, if a class has a custom metaclass that defines
:meth:`~object.__getitem__`, subscribing the class may result in different
behaviour. An example of this can be found in the :mod:`enum` module::

>>> from enum import Enum
>>> class Menu(Enum):
... """A breakfast menu"""
... SPAM = 'spam'
... BACON = 'bacon'
...
>>> # Enum classes have a custom metaclass:
>>> type(Menu)
<class 'enum.EnumMeta'>
>>> # EnumMeta defines __getitem__,
>>> # so __class_getitem__ is not called,
>>> # and the result is not a GenericAlias object:
>>> Menu['SPAM']
<Menu.SPAM: 'spam'>
>>> type(Menu['SPAM'])
<enum 'Menu'>


.. seealso::
:pep:`560` - Core Support for typing module and generic types
Introducing :meth:`~object.__class_getitem__`, and outlining when a
:ref:`subscription<subscriptions>` results in ``__class_getitem__()``
being called instead of :meth:`~object.__getitem__`


.. _callable-types:
Expand Down Expand Up @@ -2330,19 +2450,27 @@ through the object's keys; for sequences, it should iterate through the values.

.. method:: object.__getitem__(self, key)

Called to implement evaluation of ``self[key]``. For sequence types, the
accepted keys should be integers and slice objects. Note that the special
interpretation of negative indexes (if the class wishes to emulate a sequence
type) is up to the :meth:`__getitem__` method. If *key* is of an inappropriate
type, :exc:`TypeError` may be raised; if of a value outside the set of indexes
for the sequence (after any special interpretation of negative values),
:exc:`IndexError` should be raised. For mapping types, if *key* is missing (not
in the container), :exc:`KeyError` should be raised.
Called to implement evaluation of ``self[key]``. For :term:`sequence` types,
the accepted keys should be integers and slice objects. Note that the
special interpretation of negative indexes (if the class wishes to emulate a
:term:`sequence` type) is up to the :meth:`__getitem__` method. If *key* is
of an inappropriate type, :exc:`TypeError` may be raised; if of a value
outside the set of indexes for the sequence (after any special
interpretation of negative values), :exc:`IndexError` should be raised. For
:term:`mapping` types, if *key* is missing (not in the container),
:exc:`KeyError` should be raised.

.. note::

:keyword:`for` loops expect that an :exc:`IndexError` will be raised for
illegal indexes to allow proper detection of the end of the sequence.

.. note::

:keyword:`for` loops expect that an :exc:`IndexError` will be raised for illegal
indexes to allow proper detection of the end of the sequence.
When :ref:`subscripting<subscriptions>` a *class*, the special
class method :meth:`~object.__class_getitem__` may be called instead of
``__getitem__()``. See :ref:`classgetitem-versus-getitem` for more
details.


.. method:: object.__setitem__(self, key, value)
Expand Down

0 comments on commit 31b3a70

Please sign in to comment.