PEP 747: Fix rules related to UnionType (T1 | T2). Contrast TypeExpr with TypeAlias. Apply other feedback. (#3856)

Author: davidfstr

peps/pep-0747.rst

@@ -254,9 +254,8 @@ A ``TypeExpr`` value represents a :ref:`type expression <typing:type-expression>
 such as ``str | None``, ``dict[str, int]``, or ``MyTypedDict``.
 A ``TypeExpr`` type is written as
 ``TypeExpr[T]`` where ``T`` is a type or a type variable. It can also be
-written without brackets as just ``TypeExpr``, in which case a type
-checker should apply its usual type inference mechanisms to determine
-the type of its argument, possibly ``Any``.
+written without brackets as just ``TypeExpr``, which is treated the same as
+to ``TypeExpr[Any]``.
 
 
 Using TypeExprs
@@ -278,7 +277,6 @@ or a variable type:
 ::
 
    STR_TYPE: TypeExpr = str  # variable type
-   assert_type(STR_TYPE, TypeExpr[str])
 
 Note however that an *unannotated* variable assigned a type expression literal
 will not be inferred to be of ``TypeExpr`` type by type checkers because PEP
@@ -352,7 +350,7 @@ not spell a type are not ``TypeExpr`` values.
 ::
 
    OPTIONAL_INT_TYPE: TypeExpr = TypeExpr[int | None]  # OK
-   assert isassignable(Optional[int], OPTIONAL_INT_TYPE)
+   assert isassignable(int | None, OPTIONAL_INT_TYPE)
 
 .. _non_universal_typeexpr:
 
@@ -442,14 +440,29 @@ so must be disambiguated based on its argument type:
 -  As a value expression, ``Annotated[x, ...]`` has type ``object``
    if ``x`` has a type that is not ``type[C]`` or ``TypeExpr[T]``.
 
-**Union**: The type expression ``T1 | T2`` is ambiguous with the value ``int1 | int2``,
-so must be disambiguated based on its argument type:
+**Union**: The type expression ``T1 | T2`` is ambiguous with 
+the value ``int1 | int2``, ``set1 | set2``, ``dict1 | dict2``, and more,
+so must be disambiguated based on its argument types:
+
+-  As a value expression, ``x | y`` has type equal to the return type of ``type(x).__or__``
+   if ``type(x)`` overrides the ``__or__`` method.
+
+   -  When ``x`` has type ``builtins.type``, ``types.GenericAlias``, or the
+      internal type of a typing special form, ``type(x).__or__`` has a return type
+      in the format ``TypeExpr[T1 | T2]``.
+
+-  As a value expression, ``x | y`` has type equal to the return type of ``type(y).__ror__``
+   if ``type(y)`` overrides the ``__ror__`` method.
+
+   -  When ``y`` has type ``builtins.type``, ``types.GenericAlias``, or the
+      internal type of a typing special form, ``type(y).__ror__`` has a return type
+      in the format ``TypeExpr[T1 | T2]``.
 
--  As a value expression, ``x | y`` has type ``TypeExpr[x | y]``
-   if ``x`` has type ``TypeExpr[t1]`` (or ``type[t1]``)
-   and ``y`` has type ``TypeExpr[t2]`` (or ``type[t2]``).
--  As a value expression, ``x | y`` has type ``int``
-   if ``x`` has type ``int`` and ``y`` has type ``int``
+-  As a value expression, ``x | y`` has type ``UnionType``
+   in all other situations.
+
+   -  This rule is intended to be consistent with the preexisting fallback rule
+      used by static type checkers.
 
 The **stringified type expression** ``"T"`` is ambiguous with both
 the stringified annotation expression ``"T"``
@@ -466,71 +479,24 @@ New kinds of type expressions that are introduced should define how they
 will be recognized in a value expression context.
 
 
-Implicit Annotation Expression Values
-'''''''''''''''''''''''''''''''''''''
-
-Although this PEP is mostly concerned with *type expressions* rather than
-*annotation expressions*, it is straightforward to extend the rules for
-:ref:`recognizing type expressions <implicit_typeexpr_values>`
-to similar rules for recognizing annotation expressions,
-so this PEP takes the opportunity to define those rules as well:
-
-The following **unparameterized annotation expressions** can be recognized unambiguously:
-
--  As a value expression, ``X`` has type ``object``,
-   for each of the following values of X:
-
-   -  ``<TypeAlias>``
-
-The following **parameterized annotation expressions** can be recognized unambiguously:
-
--  As a value expression, ``X`` has type ``object``,
-   for each of the following values of X:
-
-   -  ``<Required> '[' ... ']'``
-   -  ``<NotRequired> '[' ... ']'``
-   -  ``<ReadOnly> '[' ... ']'``
-   -  ``<ClassVar> '[' ... ']'``
-   -  ``<Final> '[' ... ']'``
-   -  ``<InitVar> '[' ... ']'``
-   -  ``<Unpack> '[' ... ']'``
-
-**Annotated**: The annotation expression ``Annotated[...]`` is ambiguous with 
-the type expression ``Annotated[...]``,
-so must be :ref:`disambiguated based on its argument type <recognizing_annotated>`.
-
-The following **syntactic annotation expressions** 
-cannot be recognized in a value expression context at all:
-
--  ``'*' unpackable``
--  ``name '.' 'args'`` (where ``name`` must be an in-scope ParamSpec)
--  ``name '.' 'kwargs'`` (where ``name`` must be an in-scope ParamSpec)
-
-The **stringified annotation expression** ``"T"`` is ambiguous with both
-the stringified type expression ``"T"``
-and the string literal ``"T"``, and
-cannot be recognized in a value expression context at all:
-
--  As a value expression, ``"T"`` continues to have type ``Literal["T"]``.
-
-No other kinds of annotation expressions currently exist.
-
-New kinds of annotation expressions that are introduced should define how they
-will (or will not) be recognized in a value expression context.
-
-
 Literal[] TypeExprs
 '''''''''''''''''''
 
-To simplify static type checking, a ``Literal[...]`` value is *not*
-considered assignable to a ``TypeExpr`` variable even if all of its members
-spell valid types:
+A value of ``Literal[...]`` type is *not* considered assignable to
+a ``TypeExpr`` variable even if all of its members spell valid types because
+dynamic values are not allowed in type expressions:
 
 ::
 
    STRS_TYPE_NAME: Literal['str', 'list[str]'] = 'str'
    STRS_TYPE: TypeExpr = STRS_TYPE_NAME  # ERROR: Literal[] value is not a TypeExpr
 
+However ``Literal[...]`` itself is still a ``TypeExpr``:
+
+::
+
+   DIRECTION_TYPE: TypeExpr[Literal['left', 'right']] = Literal['left', 'right']  # OK
+
 
 Static vs. Runtime Representations of TypeExprs
 '''''''''''''''''''''''''''''''''''''''''''''''
@@ -569,19 +535,35 @@ Subtyping
 Whether a ``TypeExpr`` value can be assigned from one variable to another is
 determined by the following rules:
 
+Relationship with type
+''''''''''''''''''''''
+
 ``TypeExpr[]`` is covariant in its argument type, just like ``type[]``:
 
 -  ``TypeExpr[T1]`` is a subtype of ``TypeExpr[T2]`` iff ``T1`` is a
    subtype of ``T2``.
 -  ``type[C1]`` is a subtype of ``TypeExpr[C2]`` iff ``C1`` is a subtype
    of ``C2``.
 
-A plain ``type`` can be assigned to a plain ``TypeExpr`` but not the
-other way around:
+An unparameterized ``type`` can be assigned to an unparameterized ``TypeExpr``
+but not the other way around:
 
 -  ``type[Any]`` is assignable to ``TypeExpr[Any]``. (But not the
    other way around.)
 
+Relationship with UnionType
+'''''''''''''''''''''''''''
+
+``TypeExpr[U]`` is a subtype of ``UnionType`` iff ``U`` is
+the type expression ``X | Y | ...``:
+
+-  ``TypeExpr[X | Y | ...]`` is a subtype of ``UnionType``.
+
+``UnionType`` is assignable to ``TypeExpr[Any]``.
+
+Relationship with object
+''''''''''''''''''''''''
+
 ``TypeExpr[]`` is a kind of ``object``, just like ``type[]``:
 
 -  ``TypeExpr[T]`` for any ``T`` is a subtype of ``object``.
@@ -623,11 +605,33 @@ Changed signatures
 ''''''''''''''''''
 
 The following signatures related to type expressions introduce
-``TypeExpr`` where previously ``object`` existed:
+``TypeExpr`` where previously ``object`` or ``Any`` existed:
 
 -  ``typing.cast``
 -  ``typing.assert_type``
 
+The following signatures transforming union type expressions introduce
+``TypeExpr`` where previously ``UnionType`` existed so that a more-precise
+``TypeExpr`` type can be inferred:
+
+-  ``builtins.type[T].__or__``
+
+   -  Old: ``def __or__(self, value: Any, /) -> types.UnionType: ...``
+   -  New: ``def __or__[T2](self, value: TypeExpr[T2], /) -> TypeExpr[T | T2]: ...``
+
+-  ``builtins.type[T].__ror__``
+
+   -  Old: ``def __ror__(self, value: Any, /) -> types.UnionType: ...``
+   -  New: ``def __ror__[T1](self, value: TypeExpr[T1], /) -> TypeExpr[T1 | T]: ...``
+
+-  ``types.GenericAlias.{__or__,__ror__}``
+-  «the internal type of a typing special form»``.{__or__,__ror__}``
+
+However the implementations of those methods continue to return ``UnionType``
+instances at runtime so that runtime ``isinstance`` checks like
+``isinstance('42', int | str)`` and ``isinstance(int | str, UnionType)``
+continue to work.
+
 
 Unchanged signatures
 ''''''''''''''''''''
@@ -662,12 +666,32 @@ not propose those changes now:
 
    -  Returns annotation expressions
 
+The following signatures accepting union type expressions continue
+to use ``UnionType``:
+
+-  ``builtins.isinstance``
+-  ``builtins.issubclass``
+-  ``typing.get_origin`` (used in an ``@overload``)
+
+The following signatures transforming union type expressions continue
+to use ``UnionType`` because it is not possible to infer a more-precise
+``TypeExpr`` type:
+
+-  ``types.UnionType.{__or__,__ror__}``
+
 
 Backwards Compatibility
 =======================
 
-Previously the rules for recognizing type expression objects
-in a value expression context were not defined, so static type checkers
+As a value expression, ``X | Y`` previously had type ``UnionType`` (via :pep:`604`)
+but this PEP gives it the more-precise static type ``TypeExpr[X | Y]`` 
+(a subtype of ``UnionType``) while continuing to return a ``UnionType`` instance at runtime.
+Preserving compability with ``UnionType`` is important because ``UnionType``
+supports ``isinstance`` checks, unlike ``TypeExpr``, and existing code relies
+on being able to perform those checks.
+
+The rules for recognizing other kinds of type expression objects
+in a value expression context were not previously defined, so static type checkers
 `varied in what types were assigned <https://discuss.python.org/t/typeform-spelling-for-a-type-annotation-object-at-runtime/51435/34>`_
 to such objects. Existing programs manipulating type expression objects
 were already limited in manipulating them as plain ``object`` values,
@@ -711,12 +735,38 @@ assigned to variables and manipulated like any other data in a program:
 ``TypeExpr[]`` is how you spell the type of a variable containing a
 type annotation object describing a type.
 
-``TypeExpr[]`` is similar to ``type[]``, but ``type[]`` can only used to
+``TypeExpr[]`` is similar to ``type[]``, but ``type[]`` can only
 spell simple **class objects** like ``int``, ``str``, ``list``, or ``MyClass``.
 ``TypeExpr[]`` by contrast can additionally spell more complex types, 
 including those with brackets (like ``list[int]``) or pipes (like ``int | None``),
 and including special types like ``Any``, ``LiteralString``, or ``Never``.
 
+A ``TypeExpr`` variable looks similar to a ``TypeAlias`` definition, but
+can only be used where a dynamic value is expected.
+``TypeAlias`` (and the ``type`` statement) by contrast define a name that can
+be used where a fixed type is expected:
+
+-  Okay, but discouraged in Python 3.12+:
+
+   ::
+
+      MaybeFloat: TypeAlias = float | None
+      def sqrt(n: float) -> MaybeFloat: ...
+
+-  Yes:
+
+   ::
+
+      type MaybeFloat = float | None
+      def sqrt(n: float) -> MaybeFloat: ...
+
+-  No:
+
+   ::
+
+      maybe_float: TypeExpr = float | None
+      def sqrt(n: float) -> maybe_float: ...  # ERROR: Can't use TypeExpr value in a type annotation
+
 It is uncommon for a programmer to define their *own* function which accepts
 a ``TypeExpr`` parameter or returns a ``TypeExpr`` value. Instead it is more common
 for a programmer to pass a literal type expression to an *existing* function
@@ -891,8 +941,9 @@ The following will be true when
 `mypy#9773 <https://github.com/python/mypy/issues/9773>`__ is implemented:
 
     The mypy type checker supports ``TypeExpr`` types.
-    A reference implementation of the runtime component is provided in the
-    ``typing_extensions`` module.
+
+A reference implementation of the runtime component is provided in the
+``typing_extensions`` module.
 
 
 Rejected Ideas