From d2d599f128190acac408faa740b2d1da017c8d90 Mon Sep 17 00:00:00 2001 From: Rebecca Chen Date: Wed, 21 Aug 2024 16:25:48 -0700 Subject: [PATCH] Merge the specification portion of the "Type Stubs" doc into the spec (#1815) * Type Stubs: move intro and syntax to spec. * Moves some content into spec, deletes duplicate spec content. * Adds placeholders for where remaining content will be moved. It will be split between the spec and the writing_stubs doc. * Move "Stub Contents" to writing_stubs. Moves the section on what to include in stubs with editorial changes only - e.g., changing "type stub" to "stub file" to match surrounding terminology. * Moves "Style Guide" to writing_stubs doc. Moves the style guide with a few updates: * Editorial changes like tweaking code examples to use recommended style. * Attribute declarations can use assignments when needed to convey extra information - e.g., final attributes and enum members. * Tweaks a reference to the double-underscore convention for positional-only arguments to note that it's historical. * Change :ref:`stubs` occurrences to :ref:`stub-files`. * Move a few "Supported Constructs" sections into spec. The only substantive change is an update to "Comments" to note that many formats of error suppression comments are allowed. * Migrated "Imports" and "Module Level Attributes" supported constructs. Added `x: Final = ` to module-level attributes section. No other changes. * Migrate "Enums" supported construct section. Replaces outdated info with a link to the enums section of the spec. * Move "Classes" and "Decorators" supported constructs sections. Added an example of an inner class to "Classes." Removed mention of asyncio.coroutine from "Decorators" as it's very deprecated. Added functions decorated with `@dataclass_transform` to list of decorators that type checkers should recognize. * Port "Functions and Methods" supported constructs section. Removed mention of individual supported/unsupported features from this section. Where we state that all typing features from the latest released version are supported, added a caveat with a link to the typeshed feature tracker. * Adds an "Import Conventions" section (deduplication). * Remove missed reference to deleted stubs doc. * Fix broken refs. * Formatting fix to docs/guides/writing_stubs.rst Co-authored-by: Sebastian Rittau * Reword paragraph on argument names to be more concise. Co-authored-by: Sebastian Rittau * Wording clarification in docs/guides/writing_stubs.rst Co-authored-by: Sebastian Rittau * Modernize code example in docs/guides/writing_stubs.rst Co-authored-by: Sebastian Rittau * Typo fix in docs/spec/distributing.rst Co-authored-by: Sebastian Rittau * Typo fix in docs/spec/distributing.rst Co-authored-by: Sebastian Rittau * Grammar fixes * Property deleters should also be understood. * Address future notes. * Fix silly formatting error. * Undelete stubs.rst. * Address review comments. * Reword some sections to focus on type checker behavior. * Move or remove some sections that fit better in the writing_stubs guide or are redundant. * Address more review comments. Cuts redundant sections, moves things to writing_stubs, tries out a new "these should be supported fully" and "these should be supported to at least the described minimum" structure. * Remove no-longer-needed anchor. * And add back an accidentally deleted newline... * Slight rewording of Decorators advice. * Address more reviewer comments. Moves "Library Interface" into spec, further trims redundant text. * Remove extra colon. * Revert changes to wrting_stubs that should be made separately. * Delete unnecessary sentence. Co-authored-by: Carl Meyer * Clarify some language and terminology. * Address reviewer feedback. * Formatting * Drop "annotation contexts." * Add "in annotation expressions" to newer syntax explanation. * Update docs/spec/distributing.rst Co-authored-by: Jelle Zijlstra * Reword syntax section. * Update docs/spec/distributing.rst Co-authored-by: Jelle Zijlstra * [pre-commit.ci] auto fixes from pre-commit.com hooks --------- Co-authored-by: Sebastian Rittau Co-authored-by: Carl Meyer Co-authored-by: Jelle Zijlstra Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> --- docs/guides/libraries.rst | 49 +--- docs/guides/writing_stubs.rst | 2 +- docs/reference/index.rst | 1 - docs/reference/stubs.rst | 455 +--------------------------------- docs/spec/directives.rst | 2 + docs/spec/distributing.rst | 245 ++++++++++++++---- docs/spec/literal.rst | 3 + 7 files changed, 209 insertions(+), 548 deletions(-) diff --git a/docs/guides/libraries.rst b/docs/guides/libraries.rst index 376811ad..4b90da49 100644 --- a/docs/guides/libraries.rst +++ b/docs/guides/libraries.rst @@ -43,7 +43,7 @@ library: Inline type annotations simply refers to the use of annotations within your ``.py`` files. In contrast, with type stub files, type information lives in -separate ``.pyi`` files; see :ref:`stubs` and :ref:`writing_stubs` for more +separate ``.pyi`` files; see :ref:`stub-files` and :ref:`writing_stubs` for more details. We recommend using the inline type annotations approach, since it has the @@ -193,51 +193,8 @@ How much of my library needs types? A "py.typed" library should aim to be type complete so that type checking and inspection can work to their full extent. Here we say that a library is “type complete” if all of the symbols -that comprise its interface have type annotations that refer to types -that are fully known. Private symbols are exempt. - -Library interface (public and private symbols) ----------------------------------------------- - -If a ``py.typed`` module is present, a type checker will treat all modules -within that package (i.e. all files that end in ``.py`` or ``.pyi``) as -importable unless the file name begins with an underscore. These modules -comprise the supported interface for the library. - -Each module exposes a set of symbols. Some of these symbols are -considered "private” — implementation details that are not part of the -library’s interface. Type checkers can use the following rules -to determine which symbols are visible outside of the package. - -- Symbols whose names begin with an underscore (but are not dunder - names) are considered private. -- Imported symbols are considered private by default. If they use the - ``import A as A`` (a redundant module alias), ``from X import A as A`` (a - redundant symbol alias), or ``from . import A`` forms, symbol ``A`` is - not private unless the name begins with an underscore. If a file - ``__init__.py`` uses form ``from .A import X``, symbol ``A`` is treated - likewise. If a wildcard import (of the form ``from X import *``) is - used, all symbols referenced by the wildcard are not private. -- A module can expose an ``__all__`` symbol at the module level that - provides a list of names that are considered part of the interface. - This overrides all other rules above, allowing imported symbols or - symbols whose names begin with an underscore to be included in the - interface. -- Local variables within a function (including nested functions) are - always considered private. - -The following idioms are supported for defining the values contained -within ``__all__``. These restrictions allow type checkers to statically -determine the value of ``__all__``. - -- ``__all__ = ('a', b')`` -- ``__all__ = ['a', b']`` -- ``__all__ += ['a', b']`` -- ``__all__ += submodule.__all__`` -- ``__all__.extend(['a', b'])`` -- ``__all__.extend(submodule.__all__)`` -- ``__all__.append('a')`` -- ``__all__.remove('a')`` +that comprise its :ref:`interface ` have type annotations +that refer to types that are fully known. Private symbols are exempt. Type Completeness ----------------- diff --git a/docs/guides/writing_stubs.rst b/docs/guides/writing_stubs.rst index 703d9749..71d13a71 100644 --- a/docs/guides/writing_stubs.rst +++ b/docs/guides/writing_stubs.rst @@ -5,7 +5,7 @@ Writing and Maintaining Stub Files ********************************** Stub files are a means of providing type information for Python modules. -For a full reference, refer to :ref:`stubs`. +For a full reference, refer to :ref:`stub-files`. Maintaining stubs can be a little cumbersome because they are separated from the implementation. This page lists some tools that make writing and maintaining diff --git a/docs/reference/index.rst b/docs/reference/index.rst index 010f3b49..720e2127 100644 --- a/docs/reference/index.rst +++ b/docs/reference/index.rst @@ -8,7 +8,6 @@ Type System Reference generics protocols - stubs best_practices quality typing Module Documentation diff --git a/docs/reference/stubs.rst b/docs/reference/stubs.rst index aac21f98..b0ce5671 100644 --- a/docs/reference/stubs.rst +++ b/docs/reference/stubs.rst @@ -1,457 +1,8 @@ -.. _stubs: +:orphan: ********** Type Stubs ********** -Introduction -============ - -*type stubs*, also called *stub files*, provide type information for untyped -Python packages and modules. Type stubs serve multiple purposes: - -* They are the only way to add type information to extension modules. -* They can provide type information for packages that do not wish to - add them inline. -* They can be distributed separately from the implementation. - This allows stubs to be developed at a different pace or by different - authors, which is especially useful when adding type annotations to - existing packages. -* They can act as documentation, succinctly explaining the external - API of a package, without including the implementation or private - members. - -This document aims to give guidance to both authors of type stubs and developers -of type checkers and other tools. It describes the constructs that can be used -safely in type stubs and lists constructs that type checkers are expected to -support. - -Type stubs that only use constructs described in this document should work with -all type checkers that also follow this document. -Type stub authors can elect to use additional constructs, but -must be prepared that some type checkers will not parse them as expected. - -A type checker that conforms to this document will parse a type stub that only uses -constructs described here without error and will not interpret any -construct in a contradictory manner. However, type checkers are not -required to implement checks for all these constructs, and -can elect to ignore unsupported ones. Additionally type checkers -can support constructs not described in this document and tool authors are -encouraged to experiment with additional features. - -.. _stub-file-syntax: - -Syntax -====== - -Type stubs are syntactically valid Python 3.8 files with a ``.pyi`` suffix. -The Python syntax used for type stubs is independent from the Python -versions supported by the implementation, and from the Python version the type -checker runs under (if any). Therefore, type stub authors should use the -latest available syntax features in stubs (up to Python 3.8), even if the -implementation supports older, pre-3.8 Python versions. -Type checker authors are encouraged to support syntax features from -post-3.8 Python versions, although type stub authors should not use such -features if they wish to maintain compatibility with all type checkers. - -For example, Python 3.7 added the ``async`` keyword (see :pep:`492`). -Stub authors should use it to mark coroutines, even if the implementation -still uses the ``@coroutine`` decorator. On the other hand, type stubs should -not use the ``type`` soft keyword from :pep:`695`, introduced in -Python 3.12, although type checker authors are encouraged to support it. - -Stubs are treated as if ``from __future__ import annotations`` is enabled. -In particular, built-in generics, pipe union syntax (``X | Y``), and forward -references can be used. - -The :py:mod:`ast` module from the standard library supports -all syntax features required by this document. - -Distribution -============ - -Type stubs can be distributed with or separately from the implementation; -see :ref:`distributing-type` and :ref:`providing-type-annotations` -for more information. - -Supported Constructs -==================== - -This sections lists constructs that type checkers will accept in type stubs. -Type stub authors can safely use these constructs. If a -construct is marked as "unspecified", type checkers may handle it -as they best see fit or report an error. Linters should usually -flag those constructs. Type stub authors should avoid using them to -ensure compatibility across type checkers. - -Unless otherwise mentioned, type stubs support all features from the -``typing`` module of the latest released Python version. If a stub uses -typing features from a later Python version than what the implementation -supports, these features can be imported from ``typing_extensions`` instead -of ``typing``. - -For example, a stub could use ``Literal``, introduced in Python 3.8, -for a library supporting Python 3.7+:: - - from typing_extensions import Literal - - def foo(x: Literal[""]) -> int: ... - -Comments --------- - -Standard Python comments are accepted everywhere Python syntax allows them. - -Two kinds of structured comments are accepted: - -* A ``# type: X`` comment at the end of a line that defines a variable, - declaring that the variable has type ``X``. However, :pep:`526`-style - variable annotations are preferred over type comments. -* A ``# type: ignore`` comment at the end of any line, which suppresses all type - errors in that line. The type checker mypy supports suppressing certain - type errors by using ``# type: ignore[error-type]``. This is not supported - by other type checkers and should not be used in stubs. - -Imports -------- - -Type stubs distinguish between imports that are re-exported and those -that are only used internally. Imports are re-exported if they use one of these -forms (:pep:`484`): - -* ``import X as X`` -* ``from Y import X as X`` -* ``from Y import *`` - -Here are some examples of imports that make names available for internal use in -a stub but do not re-export them:: - - import X - from Y import X - from Y import X as OtherX - -Type aliases can be used to re-export an import under a different name:: - - from foo import bar as _bar - new_bar = _bar # "bar" gets re-exported with the name "new_bar" - -Sub-modules are always exported when they are imported in a module. -For example, consider the following file structure:: - - foo/ - __init__.pyi - bar.pyi - -Then ``foo`` will export ``bar`` when one of the following constructs is used in -``__init__.pyi``:: - - from . import bar - from .bar import Bar - -Stubs support customizing star import semantics by defining a module-level -variable called ``__all__``. In stubs, this must be a string list literal. -Other types are not supported. Neither is the dynamic creation of this -variable (for example by concatenation). - -By default, ``from foo import *`` imports all names in ``foo`` that -do not begin with an underscore. When ``__all__`` is defined, only those names -specified in ``__all__`` are imported:: - - __all__ = ['public_attr', '_private_looking_public_attr'] - - public_attr: int - _private_looking_public_attr: int - private_attr: int - -Type checkers support cyclic imports in stub files. - -Module Level Attributes ------------------------ - -Module level variables and constants can be annotated using either -type comments or variable annotation syntax:: - - x: int # recommended - x: int = 0 - x = 0 # type: int - x = ... # type: int - -The type of a variable is unspecified when the variable is unannotated or -when the annotation -and the assigned value disagree. As an exception, the ellipsis literal can -stand in for any type:: - - x = 0 # type is unspecified - x = ... # type is unspecified - x: int = "" # type is unspecified - x: int = ... # type is int - -Classes -------- - -Class definition syntax follows general Python syntax, but type checkers -are only expected to understand the following constructs in class bodies: - -* The ellipsis literal ``...`` is ignored and used for empty - class bodies. Using ``pass`` in class bodies is undefined. -* Instance attributes follow the same rules as module level attributes - (see above). -* Method definitions (see below) and properties. -* Method aliases. -* Inner class definitions. - -More complex statements don't need to be supported:: - - class Simple: ... - - class Complex(Base): - read_write: int - @property - def read_only(self) -> int: ... - def do_stuff(self, y: str) -> None: ... - doStuff = do_stuff - -The type of generic classes can be narrowed by annotating the ``self`` -argument of the ``__init__`` method:: - - class Foo(Generic[_T]): - @overload - def __init__(self: Foo[str], type: Literal["s"]) -> None: ... - @overload - def __init__(self: Foo[int], type: Literal["i"]) -> None: ... - @overload - def __init__(self, type: str) -> None: ... - -The class must match the class in which it is declared. Using other classes, -including sub or super classes, will not work. In addition, the ``self`` -annotation cannot contain type variables. - -.. _supported-functions: - -Functions and Methods ---------------------- - -Function and method definition syntax follows general Python syntax. -For backwards compatibility, positional-only parameters can also be marked by -prefixing their name with two underscores (but not suffixing it with two -underscores):: - - # x is positional-only - # y can be used positionally or as keyword argument - # z is keyword-only - def foo(x, /, y, *, z): ... # recommended - def foo(__x, y, *, z): ... # backwards compatible syntax - -If an argument or return type is unannotated, per :pep:`484` its -type is assumed to be ``Any``. It is preferred to leave unknown -types unannotated rather than explicitly marking them as ``Any``, as some -type checkers can optionally warn about unannotated arguments. - -If an argument has a literal or constant default value, it must match the implementation -and the type of the argument (if specified) must match the default value. -Alternatively, ``...`` can be used in place of any default value:: - - # The following arguments all have type Any. - def unannotated(a, b=42, c=...): ... - # The following arguments all have type int. - def annotated(a: int, b: int = 42, c: int = ...): ... - # The following default values are invalid and the types are unspecified. - def invalid(a: int = "", b: Foo = Foo()): ... - -For a class ``C``, the type of the first argument to a classmethod is -assumed to be ``type[C]``, if unannotated. For other non-static methods, -its type is assumed to be ``C``:: - - class Foo: - def do_things(self): ... # self has type Foo - @classmethod - def create_it(cls): ... # cls has type Type[Foo] - @staticmethod - def utility(x): ... # x has type Any - -But:: - - _T = TypeVar("_T") - - class Foo: - def do_things(self: _T) -> _T: ... # self has type _T - @classmethod - def create_it(cls: _T) -> _T: ... # cls has type _T - -:pep:`612` parameter specification variables (``ParamSpec``) -are supported in argument and return types:: - - _P = ParamSpec("_P") - _R = TypeVar("_R") - - def foo(cb: Callable[_P, _R], *args: _P.args, **kwargs: _P.kwargs) -> _R: ... - -However, ``Concatenate`` from PEP 612 is not yet supported; nor is using -a ``ParamSpec`` to parameterize a generic class. - -:pep:`647` type guards are supported. - -Using a function or method body other than the ellipsis literal is currently -unspecified. Stub authors may experiment with other bodies, but it is up to -individual type checkers how to interpret them:: - - def foo(): ... # compatible - def bar(): pass # behavior undefined - -Aliases and NewType -------------------- - -Type checkers should accept module-level type aliases, optionally using -``TypeAlias`` (:pep:`613`), e.g.:: - - _IntList = list[int] - _StrList: TypeAlias = list[str] - -Type checkers should also accept regular module-level or class-level aliases, -e.g.:: - - def a() -> None: ... - b = a - - class C: - def f(self) -> int: ... - g = f - -A type alias may contain type variables. As per :pep:`484`, -all type variables must be substituted when the alias is used:: - - _K = TypeVar("_K") - _V = TypeVar("_V") - _MyMap: TypeAlias = dict[str, dict[_K, _V]] - - # either concrete types or other type variables can be substituted - def f(x: _MyMap[str, _V]) -> _V: ... - # explicitly substitute in Any rather than using a bare alias - def g(x: _MyMap[Any, Any]) -> Any: ... - -Otherwise, type variables in aliases follow the same rules as type variables in -generic class definitions. - -``typing.NewType`` is also supported in stubs. - -.. _stub-decorators: - -Decorators ----------- - -Type stubs may only use decorators defined in the ``typing`` module, plus a -fixed set of additional ones: - -* ``classmethod`` -* ``staticmethod`` -* ``property`` (including ``.setter``) -* ``abc.abstractmethod`` -* ``dataclasses.dataclass`` -* ``asyncio.coroutine`` (although ``async`` should be used instead) - -Version and Platform Checks ---------------------------- - -Type stubs for libraries that support multiple Python versions can use version -checks to supply version-specific type hints. Type stubs for different Python -versions should still conform to the most recent supported Python version's -syntax, as explain in the Syntax_ section above. - -Version checks are if-statements that use ``sys.version_info`` to determine the -current Python version. Version checks should only check against the ``major`` and -``minor`` parts of ``sys.version_info``. Type checkers are only required to -support the tuple-based version check syntax:: - - if sys.version_info >= (3,): - # Python 3-specific type hints. This tuple-based syntax is recommended. - else: - # Python 2-specific type hints. - - if sys.version_info >= (3, 5): - # Specific minor version features can be easily checked with tuples. - - if sys.version_info < (3,): - # This is only necessary when a feature has no Python 3 equivalent. - -Type stubs should avoid checking against ``sys.version_info.major`` -directly and should not use comparison operators other than ``<`` and ``>=``. - -No:: - - if sys.version_info.major >= 3: - # Semantically the same as the first tuple check. - - if sys.version_info[0] >= 3: - # This is also the same. - - if sys.version_info <= (2, 7): - # This does not work because e.g. (2, 7, 1) > (2, 7). - -Some type stubs also may need to specify type hints for different platforms. -Platform checks must be equality comparisons between ``sys.platform`` and the name -of a platform as a string literal: - -Yes:: - - if sys.platform == 'win32': - # Windows-specific type hints. - else: - # Posix-specific type hints. - -No:: - - if sys.platform.startswith('linux'): - # Not necessary since Python 3.3. - - if sys.platform in ['linux', 'cygwin', 'darwin']: - # Only '==' or '!=' should be used in platform checks. - -Version and platform comparisons can be chained using the ``and`` and ``or`` -operators:: - - if sys.platform == 'linux' and (sys.version_info < (3,) or sys,version_info >= (3, 7)): ... - -Enums ------ - -Enum classes are supported in stubs, regardless of the Python version targeted by -the stubs. - -Enum members may be specified just like other forms of assignments, for example as -``x: int``, ``x = 0``, or ``x = ...``. The first syntax is preferred because it -allows type checkers to correctly type the ``.value`` attribute of enum members, -without providing unnecessary information like the runtime value of the enum member. - -Additional properties on enum members should be specified with ``@property``, so they -do not get interpreted by type checkers as enum members. - -Yes:: - - from enum import Enum - - class Color(Enum): - RED: int - BLUE: int - @property - def rgb_value(self) -> int: ... - - class Color(Enum): - # discouraged; type checkers will not understand that Color.RED.value is an int - RED = ... - BLUE = ... - @property - def rgb_value(self) -> int: ... - -No:: - - from enum import Enum - - class Color(Enum): - RED: int - BLUE: int - rgb_value: int # no way for type checkers to know that this is not an enum member - -Copyright -========= - -This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. +The contents of this document have been moved to :ref:`stub-files` and +:ref:`writing_stubs`. diff --git a/docs/spec/directives.rst b/docs/spec/directives.rst index 7fca698a..62e1b3c4 100644 --- a/docs/spec/directives.rst +++ b/docs/spec/directives.rst @@ -128,6 +128,8 @@ and return type annotations and treat the function as if it were unannotated. The behavior for the ``no_type_check`` decorator when applied to a class is left undefined by the typing spec at this time. +.. _`version-and-platform-checks`: + Version and platform checking ----------------------------- diff --git a/docs/spec/distributing.rst b/docs/spec/distributing.rst index 9bcd3a6f..57bcc66e 100644 --- a/docs/spec/distributing.rst +++ b/docs/spec/distributing.rst @@ -10,72 +10,164 @@ Stub files (Originally specified in :pep:`484`.) -Stub files are files containing type hints that are only for use by -the type checker, not at runtime. There are several use cases for -stub files: +*Stub files*, also called *type stubs*, provide type information for untyped +Python packages and modules. Stub files serve multiple purposes: + +* They are the only way to add type information to extension modules. +* They can provide type information for packages that do not wish to + add them inline. +* They can be distributed separately from the package or module that they + provide types for. The latter is referred to as the *implementation*. + This allows stubs to be developed at a different pace or by different + authors, which is especially useful when adding type annotations to + existing packages. +* They can act as documentation, succinctly explaining the external + API of a package, without including implementation details or private + members. + +Stub files use a subset of the constructs used in Python source files, as +described in :ref:`stub-file-supported-constructs` below. Type checkers should +parse a stub that uses only such constructs without error and not interpret any +construct in a manner contradictory to this specification. However, type +checkers are not required to implement checks for all of these constructs and +can elect to ignore unsupported ones. Additionally, type checkers can support +constructs not described here. + +If a stub file is found for a module, the type checker should not read the +corresponding "real" module. See :ref:`mro` for more information. + +.. _stub-file-syntax: + +Syntax +^^^^^^ + +Stub files are syntactically valid Python files with a ``.pyi`` suffix. They +should be parseable (e.g., with :py:func:`ast.parse`) in all Python versions +that are supported by the implementation and that are still supported +by the CPython project. For example, defining a type alias using the +``type`` keyword is only accepted by the Python parser in Python 3.12 and later, +so stubs supporting Python 3.11 or earlier versions should not use this syntax. +This allows type checkers implemented in Python to parse stub files using +functionality from the standard library. +Type checkers may choose to support syntactic features from newer Python versions +in stub files, but stubs that rely on such features may not be portable to all +type checkers. Type checkers may also choose to support Python versions that +are no longer supported by CPython; if so, they cannot rely on standard library +functionality to parse stub files. + +Type checkers should evaluate all :term:`annotation expressions ` as if they are quoted. +Consequently, forward references do not need to be quoted, and type system +features that do not depend on Python syntax changes are supported in stubs regardless +of the Python version supported. For example, the use of the ``|`` operator +to create unions (``X | Y``) was introduced in Python 3.10, but may be used +even in stubs that support Python 3.9 and older versions. + +.. _stub-file-supported-constructs: + +Supported Constructs +^^^^^^^^^^^^^^^^^^^^ + +Type checkers should fully support these constructs: + +* All features from the ``typing`` module of the latest released Python version + that use :ref:`supported syntax ` +* Comments, including type declaration (``# type: X``) and error suppression + (``# type: ignore``) comments +* Import statements, including the standard :ref:`import-conventions` and cyclic + imports +* Aliases, including type aliases, at both the module and class level +* :ref:`Simple version and platform checks ` -* Extension modules +The constructs in the following subsections may be supported in a more limited +fashion, as described below. -* Third-party modules whose authors have not yet added type hints +Value Expressions +""""""""""""""""" -* Standard library modules for which type hints have not yet been - written +In locations where value expressions can appear, such as the right-hand side of +assignment statements and function parameter defaults, type checkers should +support the following expressions: -* Modules that must be compatible with Python 2 and 3 +* The ellipsis literal, ``...``, which can stand in for any value +* Any value that is a + :ref:`legal parameter for typing.Literal ` +* Floating point literals, such as ``3.14`` +* Complex literals, such as ``1 + 2j`` -* Modules that use annotations for other purposes +Module Level Attributes +""""""""""""""""""""""" -Stub files have the same syntax as regular Python modules. There is one -feature of the ``typing`` module that is different in stub files: -the ``@overload`` decorator described below. +Type checkers should support module-level variable annotations, with and without +assignments:: -The type checker should only check function signatures in stub files; -It is recommended that function bodies in stub files just be a single -ellipsis (``...``). + x: int + x: int = 0 + x = 0 # type: int + x = ... # type: int -The type checker should have a configurable search path for stub files. -If a stub file is found the type checker should not read the -corresponding "real" module. +The :ref:`Literal shortcut using Final ` should be +supported:: -While stub files are syntactically valid Python modules, they use the -``.pyi`` extension to make it possible to maintain stub files in the -same directory as the corresponding real module. This also reinforces -the notion that no runtime behavior should be expected of stub files. + x: Final = 0 # type is Literal[0] -Additional notes on stub files: +When the type of a variable is omitted or disagrees from the assigned value, +type checker behavior is undefined:: -* Modules and variables imported into the stub are not considered - exported from the stub unless the import uses the ``import ... as - ...`` form or the equivalent ``from ... import ... as ...`` form. - (*UPDATE:* To clarify, the intention here is that only names - imported using the form ``X as X`` will be exported, i.e. the name - before and after ``as`` must be the same.) + x = 0 # behavior undefined + x: Final = ... # behavior undefined + x: int = "" # behavior undefined -* However, as an exception to the previous bullet, all objects - imported into a stub using ``from ... import *`` are considered - exported. (This makes it easier to re-export all objects from a - given module that may vary by Python version.) +Classes +""""""" -* Just like in `normal Python files `_, submodules - automatically become exported attributes of their parent module - when imported. For example, if the ``spam`` package has the - following directory structure:: +Class definition syntax follows general Python syntax, but type checkers +are expected to understand only the following constructs in class bodies: - spam/ - __init__.pyi - ham.pyi +* The ellipsis literal ``...`` is used for empty class bodies. Using ``pass`` in + class bodies is undefined. +* Instance attributes follow the same rules as module level attributes + (see above). +* Method definitions (see below) and properties. +* Aliases. +* Inner class definitions. - where ``__init__.pyi`` contains a line such as ``from . import ham`` - or ``from .ham import Ham``, then ``ham`` is an exported attribute - of ``spam``. +Yes:: -* Stub files may be incomplete. To make type checkers aware of this, the file - can contain the following code:: + class Simple: ... + + class Complex(Base): + read_write: int + @property + def read_only(self) -> int: ... + def do_stuff(self, y: str) -> None: ... + doStuff = do_stuff + IntList: TypeAlias = list[int] + class Inner: ... + +Functions and Methods +""""""""""""""""""""" - def __getattr__(name) -> Any: ... +Function and method definition follows general Python syntax. Using a function +or method body other than the ellipsis literal is undefined:: - Any identifier not defined in the stub is therefore assumed to be of type - ``Any``. + def foo(): ... # compatible + def bar(): pass # behavior undefined + +.. _stub-decorators: + +Decorators +"""""""""" + +Type checkers are expected to understand the effects of all decorators defined +in the ``typing`` module, plus these additional ones: + + * ``classmethod`` + * ``staticmethod`` + * ``property`` (including ``.setter`` and ``.deleter``) + * ``abc.abstractmethod`` + * ``dataclasses.dataclass`` + * ``warnings.deprecated`` + * functions decorated with ``@typing.dataclass_transform`` The Typeshed Project ^^^^^^^^^^^^^^^^^^^^ @@ -266,3 +358,60 @@ of that Python version. This can be queried e.g. ``pythonX.Y -c 'import site; print(site.getsitepackages())'``. It is also recommended that the type checker allow for the user to point to a particular Python binary, in case it is not in the path. + +.. _library-interface: + +Library interface (public and private symbols) +---------------------------------------------- + +If a ``py.typed`` module is present, a type checker will treat all modules +within that package (i.e. all files that end in ``.py`` or ``.pyi``) as +importable unless the file name begins with an underscore. These modules +comprise the supported interface for the library. + +Each module exposes a set of symbols. Some of these symbols are +considered "private” — implementation details that are not part of the +library’s interface. Type checkers can use the following rules +to determine which symbols are visible outside of the package. + +- Symbols whose names begin with an underscore (but are not dunder + names) are considered private. +- Imported symbols are considered private by default. A fixed set of + :ref:`import forms ` re-export imported symbols. +- A module can expose an ``__all__`` symbol at the module level that + provides a list of names that are considered part of the interface. + This overrides all other rules above, allowing imported symbols or + symbols whose names begin with an underscore to be included in the + interface. +- Local variables within a function (including nested functions) are + always considered private. + +The following idioms are supported for defining the values contained +within ``__all__``. These restrictions allow type checkers to statically +determine the value of ``__all__``. + +- ``__all__ = ('a', b')`` +- ``__all__ = ['a', b']`` +- ``__all__ += ['a', b']`` +- ``__all__ += submodule.__all__`` +- ``__all__.extend(['a', b'])`` +- ``__all__.extend(submodule.__all__)`` +- ``__all__.append('a')`` +- ``__all__.remove('a')`` + +.. _import-conventions: + +Import Conventions +------------------ + +By convention, certain import forms indicate to type checkers that an imported +symbol is re-exported and should be considered part of the importing module's +public interface. All other imported symbols are considered private by default. + +The following import forms re-export symbols: + +* ``import X as X`` (a redundant module alias): re-exports ``X``. +* ``from Y import X as X`` (a redundant symbol alias): re-exports ``X``. +* ``from Y import *``: if ``Y`` defines a module-level ``__all__`` list, + re-exports all names in ``__all__``; otherwise, re-exports all public symbols + in ``Y``'s global scope. diff --git a/docs/spec/literal.rst b/docs/spec/literal.rst index 5cd50a44..564ddaa4 100644 --- a/docs/spec/literal.rst +++ b/docs/spec/literal.rst @@ -93,6 +93,7 @@ what values may and may not be used as parameters. In short, a ``Literal[...]`` type may be parameterized by one or more literal expressions, and nothing else. +.. _literal-legal-parameters: Legal parameters for ``Literal`` at type check time """"""""""""""""""""""""""""""""""""""""""""""""""" @@ -505,6 +506,8 @@ involving Literal bools. For example, we can combine ``Literal[True]``, else: scalar += "foo" # Type checks: type of 'scalar' is narrowed to 'str' +.. _literal-final-interactions: + Interactions with Final """""""""""""""""""""""