From 0dbc0be3ae5ffcf0d3ca770c5cf3a5e1236ecd7b Mon Sep 17 00:00:00 2001 From: Ethan Smith Date: Sat, 9 Sep 2017 15:25:57 -0700 Subject: [PATCH 1/8] start Distributing Type Information PEP --- pep-9999.rst | 92 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 92 insertions(+) create mode 100644 pep-9999.rst diff --git a/pep-9999.rst b/pep-9999.rst new file mode 100644 index 00000000000..2613606bb54 --- /dev/null +++ b/pep-9999.rst @@ -0,0 +1,92 @@ +PEP: 9999 +Title: Distributing and Packaging Type Information +Author: +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 09-Sep-2017 +Python-Version: 3.7 +Post-History: + + +Abstract +======== + +PEP 484 introduced type hints to Python, with goals of making typing +gradual and easy to adopt. This PEP provides a standardized means to +package and distribute type information and how type checkers should +collect this information for type checking using existing packaging +architecture. + +With a complete distribution solution there are many complexities to +consider such as package version, Python version, source of the type +information. For example, third parties may wish to ship their own stubs +if the package maintainer does not wish to do so. Furthermore, the type +checker may not be running on the same version of Python as the version +of Python it is checking. + +There are several motivations and methods of supporting typing in a package. This PEP recognizes four (4) types of packages that may be created: 1. The package maintainer would like to add type information inline. 2. The package maintainer would like to add type information via stubs. 3. The package maintainer would like to add type information via a mix of 1 & 2. 4. A third party would like to share stub files for a package, but the maintainer does not want to include them in the source of the package. + +This PEP aims to support these scenarios and make them simple to add to +packaging and deploy. + +### TODO: Summarize below + + +Rationale +========= + +PEP 484 introduced type hints to Python, with goals of making typing +gradual and easy to adopt. PEP 484 introduces a suggestion [1]_ of using +``shared/typehints/pythonX.Y/`` for shipping stub files. However type +annotations can also live within user code. Manually adding a path to +stub files for each third party library does not scale. Additionally, +adding the path to ``site-packages`` will cause type checkers to fail +on packages that are highly dynamic (e.g. sqlalchemy and Django). + +Furthermore, package authors are wishing to distribute code that has +inline type information, and there currently is no standard method to +distribute packages with inline type annotations or syntax that can +simultaneously be used at runtime and in type checking. + +Specification +============= + +The two major parts of this specification are the packaging specifications +and the resolution order for resolving module type information. + +Packaging Type Information +-------------------------- + +Packages must opt into supporting typing. [HOW?] Options include a new +trove classifier, a data file, a special keyword, a pattern for package +name or a requirement on the typing package. Ideally this would be universal +accross all types of packages. + + + + + +References +========== + +.. [1] PEP 484, Storing and Distributing Stub Files + (https://www.python.org/dev/peps/pep-0484/#storing-and-distributing-stub-files) + + + +Copyright +========= + +This document has been placed in the public domain. + + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: From 581feb155d8a0a5c2cb8003cb6022502ac9b36e1 Mon Sep 17 00:00:00 2001 From: Ethan Smith Date: Sat, 9 Sep 2017 22:48:32 -0700 Subject: [PATCH 2/8] Finish packaging, start resolution order --- pep-9999.rst | 99 ++++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 89 insertions(+), 10 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 2613606bb54..12ecca05f6e 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -13,7 +13,8 @@ Abstract ======== PEP 484 introduced type hints to Python, with goals of making typing -gradual and easy to adopt. This PEP provides a standardized means to +gradual and easy to adopt. Currently, distribution of typing information +must be distributed manually. This PEP provides a standardized means to package and distribute type information and how type checkers should collect this information for type checking using existing packaging architecture. @@ -25,13 +26,23 @@ if the package maintainer does not wish to do so. Furthermore, the type checker may not be running on the same version of Python as the version of Python it is checking. -There are several motivations and methods of supporting typing in a package. This PEP recognizes four (4) types of packages that may be created: 1. The package maintainer would like to add type information inline. 2. The package maintainer would like to add type information via stubs. 3. The package maintainer would like to add type information via a mix of 1 & 2. 4. A third party would like to share stub files for a package, but the maintainer does not want to include them in the source of the package. +There are several motivations and methods of supporting typing in a package. +This PEP recognizes four (4) types of packages that may be created: + +1. The package maintainer would like to add type information inline. + +2. The package maintainer would like to add type information via stubs. + +3. The package maintainer would like to add type information via a mix +of 1 & 2. + +4. A third party would like to share stub files for a package, but the +maintainer does not want to include them in the source of the package. This PEP aims to support these scenarios and make them simple to add to packaging and deploy. -### TODO: Summarize below - +### TODO: Summarize below once the method is agreed upon. Rationale ========= @@ -53,18 +64,85 @@ Specification ============= The two major parts of this specification are the packaging specifications -and the resolution order for resolving module type information. +and the resolution order for resolving module type information. This spec +is meant to replace the ``shared/typehints/pythonX.Y/`` spec of PEP 484. Packaging Type Information -------------------------- -Packages must opt into supporting typing. [HOW?] Options include a new -trove classifier, a data file, a special keyword, a pattern for package -name or a requirement on the typing package. Ideally this would be universal -accross all types of packages. +Packages must opt into supporting typing. This will be done though a distutils +extension [2]_, providing a ``typed`` keyword argument to the distutils +``setup()`` command. The argument value will depend on the kind of type +information the package provides. The distutils extension will be added to the +``typing`` package. Therefore a package maintainer may write + +:: + + setup( + ... + setup_requires=["typing"], + typed="inline", + ... + ) + +For package maintainers wishing to ship stub files containing all of their +type information, it is prefered that the ``*.pyi`` stubs are alongside the +corresponding ``*.py`` files. However, the stubs may be put in a sub-folder +of the Python sources, th the same name the ``*.py`` files are in. For +example, the ``flyingcircus`` package would have its stubs in the folder +``flyingcircus/flyingcircus/``. This path is chosen so that if stubs are +not found in ``flyingcircus/`` the type checker may treat the subdirectory as +a normal package. The normal resolution order of checking ``*.pyi`` before +``*.py`` will be maintained. The value of the ``typed`` argument to +``setup()`` is ``"stubs"`` for this type of distribution. + +Packages that have inline type annotations simply have to pass the value +``"inline"`` to the ``typed`` argument in ``setup()``. + +Third parties seeking to distribute stub files are encouraged to contact the +maintainer of the package about distribution alongside the package. If the +maintainer does not wish to maintain or package stub files or type information +inline, then a "third party stub package" should be created. The structure is +similar, but slightly different from that of stub only packages. If the stubs +are for the library ``flyingcircus`` then the package should be named +``flyingcircus-stubs`` and the stub files should be put in a sub-directory +named ``flyingcircus``. This allows the stubs to be checked as if they were in +a regular package. These packages should also pass ``"stubs"`` as the value +of ``typed`` argument in ``setup()``. +Packages wishing to distribute a mix of inline and stub files should put the +stub files alongside their Python sources, and provide the value ``"mixed"`` +to the ``typed`` argument in ``setup()``. +Type Checker Module Resolution Order +------------------------------------ + +The following is the order that type checkers supporting this PEP should +resolve modules containing type information: + +1. User code - the files the type checker is running on. + +2. Stubs or Python source in ``PYTHONPATH``. This is to allow the user +complete control of which stubs to use, and patch broken stubs/inline +types from packages. + +3. Third party stub packages - these packages can supersede the installed +untyped packages. They can be found at ``pkg-stubs`` for package ``pkg``, +however it is encouraged to check their metadata to confirm that they opt +into type checking. + +4. Typeshed (if used) - Provides the stdlib types and several third party +libraries + +5. Inline packages - finally, if there is nothing overriding the installed +package, and it opts into type checking. + + +To check if a package has opted into type checking, type checkers are +recommended to use the ``pkg_resources`` module to query the package +metadata. If the ``typed`` package metadata has ``None`` as its value, the +package has not opted into type checking. References @@ -73,7 +151,8 @@ References .. [1] PEP 484, Storing and Distributing Stub Files (https://www.python.org/dev/peps/pep-0484/#storing-and-distributing-stub-files) - +.. [2] Distutils Extensions, Adding setup() arguments + (http://setuptools.readthedocs.io/en/latest/setuptools.html#adding-setup-arguments) Copyright ========= From cc9dc0b5188d07c5666e0427e8e6a3d661d0b99d Mon Sep 17 00:00:00 2001 From: Ethan Smith Date: Sun, 10 Sep 2017 11:21:47 -0700 Subject: [PATCH 3/8] add versioning, other minor edits --- pep-9999.rst | 89 ++++++++++++++++++++++++++-------------------------- 1 file changed, 44 insertions(+), 45 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 12ecca05f6e..d4b23d9d939 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -13,59 +13,47 @@ Abstract ======== PEP 484 introduced type hints to Python, with goals of making typing -gradual and easy to adopt. Currently, distribution of typing information -must be distributed manually. This PEP provides a standardized means to -package and distribute type information and how type checkers should +gradual and easy to adopt. Currently, typing information must be distributed +manually. This PEP provides a standardized means to package and distribute +type information and an ordering for type checkers to resolve modules and collect this information for type checking using existing packaging architecture. -With a complete distribution solution there are many complexities to -consider such as package version, Python version, source of the type -information. For example, third parties may wish to ship their own stubs -if the package maintainer does not wish to do so. Furthermore, the type -checker may not be running on the same version of Python as the version -of Python it is checking. - -There are several motivations and methods of supporting typing in a package. -This PEP recognizes four (4) types of packages that may be created: - -1. The package maintainer would like to add type information inline. - -2. The package maintainer would like to add type information via stubs. - -3. The package maintainer would like to add type information via a mix -of 1 & 2. - -4. A third party would like to share stub files for a package, but the -maintainer does not want to include them in the source of the package. - -This PEP aims to support these scenarios and make them simple to add to -packaging and deploy. - -### TODO: Summarize below once the method is agreed upon. Rationale ========= -PEP 484 introduced type hints to Python, with goals of making typing -gradual and easy to adopt. PEP 484 introduces a suggestion [1]_ of using -``shared/typehints/pythonX.Y/`` for shipping stub files. However type -annotations can also live within user code. Manually adding a path to -stub files for each third party library does not scale. Additionally, -adding the path to ``site-packages`` will cause type checkers to fail -on packages that are highly dynamic (e.g. sqlalchemy and Django). +PEP 484 has a brief section on distributing typing information. In this +section [1]_ the PEP recommends using ``shared/typehints/pythonX.Y/`` for +shipping stub files. However, manually adding a path to stub files for each +third party library does not scale. The simplest approach people have taken +is to add ``site-packages`` to their ``PYTHONPATH``, but this causes type +checkers to fail on packages that are highly dynamic (e.g. sqlalchemy +and Django). Furthermore, package authors are wishing to distribute code that has inline type information, and there currently is no standard method to distribute packages with inline type annotations or syntax that can simultaneously be used at runtime and in type checking. + Specification ============= +There are several motivations and methods of supporting typing in a package. This PEP recognizes three (3) types of packages that may be created: + +1. The package maintainer would like to add type information inline. + +2. The package maintainer would like to add type information via stubs. + +3. A third party would like to share stub files for a package, but the + maintainer does not want to include them in the source of the package. + +This PEP aims to support these scenarios and make them simple to add to packaging and deploy. + The two major parts of this specification are the packaging specifications and the resolution order for resolving module type information. This spec -is meant to replace the ``shared/typehints/pythonX.Y/`` spec of PEP 484. +is meant to replace the ``shared/typehints/pythonX.Y/`` spec of PEP 484 [1]_. Packaging Type Information -------------------------- @@ -85,6 +73,9 @@ information the package provides. The distutils extension will be added to the ... ) +Stub Only Packages +'''''''''''''''''' + For package maintainers wishing to ship stub files containing all of their type information, it is prefered that the ``*.pyi`` stubs are alongside the corresponding ``*.py`` files. However, the stubs may be put in a sub-folder @@ -94,11 +85,19 @@ example, the ``flyingcircus`` package would have its stubs in the folder not found in ``flyingcircus/`` the type checker may treat the subdirectory as a normal package. The normal resolution order of checking ``*.pyi`` before ``*.py`` will be maintained. The value of the ``typed`` argument to -``setup()`` is ``"stubs"`` for this type of distribution. +``setup()`` is ``"stubs"`` for this type of distribution. The author of the +package is suggested to use ``package_data`` to assure the stub files are +installed alongside the runtime Python code. + +Inline Typed Packages +'''''''''''''''''''' Packages that have inline type annotations simply have to pass the value ``"inline"`` to the ``typed`` argument in ``setup()``. +Third Party Stub Packages +''''''''''''''''''''''''' + Third parties seeking to distribute stub files are encouraged to contact the maintainer of the package about distribution alongside the package. If the maintainer does not wish to maintain or package stub files or type information @@ -110,10 +109,8 @@ named ``flyingcircus``. This allows the stubs to be checked as if they were in a regular package. These packages should also pass ``"stubs"`` as the value of ``typed`` argument in ``setup()``. -Packages wishing to distribute a mix of inline and stub files should put the -stub files alongside their Python sources, and provide the value ``"mixed"`` -to the ``typed`` argument in ``setup()``. - +The version of the ``flyingcircus-stubs`` package should match the version of +the ``flyingcircus`` package it is providing types for. Type Checker Module Resolution Order ------------------------------------ @@ -132,17 +129,19 @@ untyped packages. They can be found at ``pkg-stubs`` for package ``pkg``, however it is encouraged to check their metadata to confirm that they opt into type checking. -4. Typeshed (if used) - Provides the stdlib types and several third party -libraries - -5. Inline packages - finally, if there is nothing overriding the installed +4. Inline packages - finally, if there is nothing overriding the installed package, and it opts into type checking. +5. Typeshed (if used) - Provides the stdlib types and several third party libraries To check if a package has opted into type checking, type checkers are recommended to use the ``pkg_resources`` module to query the package metadata. If the ``typed`` package metadata has ``None`` as its value, the -package has not opted into type checking. +package has not opted into type checking, and the type checker should skip that +package. + +When resolving step (3) type checkers should assure the version of the stubs +match the installed runtime package. References From b206a797675c22c7f0b6600df1db1c011d1ce259 Mon Sep 17 00:00:00 2001 From: Ethan Smith Date: Sun, 10 Sep 2017 11:37:16 -0700 Subject: [PATCH 4/8] formatting fixes, clarify Python versioning --- pep-9999.rst | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index d4b23d9d939..81fd3f96068 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -121,28 +121,34 @@ resolve modules containing type information: 1. User code - the files the type checker is running on. 2. Stubs or Python source in ``PYTHONPATH``. This is to allow the user -complete control of which stubs to use, and patch broken stubs/inline -types from packages. + complete control of which stubs to use, and patch broken stubs/inline + types from packages. 3. Third party stub packages - these packages can supersede the installed -untyped packages. They can be found at ``pkg-stubs`` for package ``pkg``, -however it is encouraged to check their metadata to confirm that they opt -into type checking. + untyped packages. They can be found at ``pkg-stubs`` for package ``pkg``, + however it is encouraged to check their metadata to confirm that they opt + into type checking. 4. Inline packages - finally, if there is nothing overriding the installed -package, and it opts into type checking. + package, and it opts into type checking. 5. Typeshed (if used) - Provides the stdlib types and several third party libraries +When resolving step (3) type checkers should assure the version of the stubs +match the installed runtime package. + +Type checkers that check a different Python version must find the type +information in the ``site-packages`` of that Python version. This can be +queried e.g. ``pythonX.Y -c 'import sys; print(sys.exec_prefix)'``. It is +also recommended that the type checker allow for the user to point to a +particular Python binary, in case it is not in the path. + To check if a package has opted into type checking, type checkers are recommended to use the ``pkg_resources`` module to query the package metadata. If the ``typed`` package metadata has ``None`` as its value, the package has not opted into type checking, and the type checker should skip that package. -When resolving step (3) type checkers should assure the version of the stubs -match the installed runtime package. - References ========== From 9678c4faf4b54594b28e7c87680d40c0955df394 Mon Sep 17 00:00:00 2001 From: Ethan Smith Date: Sun, 10 Sep 2017 12:23:53 -0700 Subject: [PATCH 5/8] Minor edits --- pep-9999.rst | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/pep-9999.rst b/pep-9999.rst index 81fd3f96068..51bb7e39095 100644 --- a/pep-9999.rst +++ b/pep-9999.rst @@ -1,6 +1,6 @@ PEP: 9999 Title: Distributing and Packaging Type Information -Author: +Author: Ethan Smith Status: Draft Type: Standards Track Content-Type: text/x-rst @@ -107,7 +107,8 @@ are for the library ``flyingcircus`` then the package should be named ``flyingcircus-stubs`` and the stub files should be put in a sub-directory named ``flyingcircus``. This allows the stubs to be checked as if they were in a regular package. These packages should also pass ``"stubs"`` as the value -of ``typed`` argument in ``setup()``. +of ``typed`` argument in ``setup()``. These packages are suggested to use +``package_data`` to package stub files. The version of the ``flyingcircus-stubs`` package should match the version of the ``flyingcircus`` package it is providing types for. @@ -137,11 +138,12 @@ resolve modules containing type information: When resolving step (3) type checkers should assure the version of the stubs match the installed runtime package. -Type checkers that check a different Python version must find the type -information in the ``site-packages`` of that Python version. This can be -queried e.g. ``pythonX.Y -c 'import sys; print(sys.exec_prefix)'``. It is -also recommended that the type checker allow for the user to point to a -particular Python binary, in case it is not in the path. +Type checkers that check a different Python version than the version they run +on must find the type information in the ``site-packages``/``dist-packages`` +of that Python version. This can be queried e.g. +``pythonX.Y -c 'import sys; print(sys.exec_prefix)'``. It is also recommended +that the type checker allow for the user to point to a particular Python +binary, in case it is not in the path. To check if a package has opted into type checking, type checkers are recommended to use the ``pkg_resources`` module to query the package From b906b53b8d9762fe55937fc82ac01e68072c7e6d Mon Sep 17 00:00:00 2001 From: Ethan Smith Date: Sun, 10 Sep 2017 13:07:23 -0700 Subject: [PATCH 6/8] reserve PEP 0561, fix formatting --- pep-0561.rst | 178 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 178 insertions(+) create mode 100644 pep-0561.rst diff --git a/pep-0561.rst b/pep-0561.rst new file mode 100644 index 00000000000..3a5e0999776 --- /dev/null +++ b/pep-0561.rst @@ -0,0 +1,178 @@ +PEP: 561 +Title: Distributing and Packaging Type Information +Author: Ethan Smith +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 09-Sep-2017 +Python-Version: 3.7 +Post-History: + + +Abstract +======== + +PEP 484 introduced type hints to Python, with goals of making typing +gradual and easy to adopt. Currently, typing information must be distributed +manually. This PEP provides a standardized means to package and distribute +type information and an ordering for type checkers to resolve modules and +collect this information for type checking using existing packaging +architecture. + + +Rationale +========= + +PEP 484 has a brief section on distributing typing information. In this +section [1]_ the PEP recommends using ``shared/typehints/pythonX.Y/`` for +shipping stub files. However, manually adding a path to stub files for each +third party library does not scale. The simplest approach people have taken +is to add ``site-packages`` to their ``PYTHONPATH``, but this causes type +checkers to fail on packages that are highly dynamic (e.g. sqlalchemy +and Django). + +Furthermore, package authors are wishing to distribute code that has +inline type information, and there currently is no standard method to +distribute packages with inline type annotations or syntax that can +simultaneously be used at runtime and in type checking. + + +Specification +============= + +There are several motivations and methods of supporting typing in a package. This PEP recognizes three (3) types of packages that may be created: + +1. The package maintainer would like to add type information inline. + +2. The package maintainer would like to add type information via stubs. + +3. A third party would like to share stub files for a package, but the + maintainer does not want to include them in the source of the package. + +This PEP aims to support these scenarios and make them simple to add to packaging and deploy. + +The two major parts of this specification are the packaging specifications +and the resolution order for resolving module type information. This spec +is meant to replace the ``shared/typehints/pythonX.Y/`` spec of PEP 484 [1]_. + +Packaging Type Information +-------------------------- + +Packages must opt into supporting typing. This will be done though a distutils +extension [2]_, providing a ``typed`` keyword argument to the distutils +``setup()`` command. The argument value will depend on the kind of type +information the package provides. The distutils extension will be added to the +``typing`` package. Therefore a package maintainer may write + +:: + + setup( + ... + setup_requires=["typing"], + typed="inline", + ... + ) + +Stub Only Packages +'''''''''''''''''' + +For package maintainers wishing to ship stub files containing all of their +type information, it is prefered that the ``*.pyi`` stubs are alongside the +corresponding ``*.py`` files. However, the stubs may be put in a sub-folder +of the Python sources, th the same name the ``*.py`` files are in. For +example, the ``flyingcircus`` package would have its stubs in the folder +``flyingcircus/flyingcircus/``. This path is chosen so that if stubs are +not found in ``flyingcircus/`` the type checker may treat the subdirectory as +a normal package. The normal resolution order of checking ``*.pyi`` before +``*.py`` will be maintained. The value of the ``typed`` argument to +``setup()`` is ``"stubs"`` for this type of distribution. The author of the +package is suggested to use ``package_data`` to assure the stub files are +installed alongside the runtime Python code. + +Inline Typed Packages +''''''''''''''''''''' + +Packages that have inline type annotations simply have to pass the value +``"inline"`` to the ``typed`` argument in ``setup()``. + +Third Party Stub Packages +''''''''''''''''''''''''' + +Third parties seeking to distribute stub files are encouraged to contact the +maintainer of the package about distribution alongside the package. If the +maintainer does not wish to maintain or package stub files or type information +inline, then a "third party stub package" should be created. The structure is +similar, but slightly different from that of stub only packages. If the stubs +are for the library ``flyingcircus`` then the package should be named +``flyingcircus-stubs`` and the stub files should be put in a sub-directory +named ``flyingcircus``. This allows the stubs to be checked as if they were in +a regular package. These packages should also pass ``"stubs"`` as the value +of ``typed`` argument in ``setup()``. These packages are suggested to use +``package_data`` to package stub files. + +The version of the ``flyingcircus-stubs`` package should match the version of +the ``flyingcircus`` package it is providing types for. + +Type Checker Module Resolution Order +------------------------------------ + +The following is the order that type checkers supporting this PEP should +resolve modules containing type information: + +1. User code - the files the type checker is running on. + +2. Stubs or Python source in ``PYTHONPATH``. This is to allow the user + complete control of which stubs to use, and patch broken stubs/inline + types from packages. + +3. Third party stub packages - these packages can supersede the installed + untyped packages. They can be found at ``pkg-stubs`` for package ``pkg``, + however it is encouraged to check their metadata to confirm that they opt + into type checking. + +4. Inline packages - finally, if there is nothing overriding the installed + package, and it opts into type checking. + +5. Typeshed (if used) - Provides the stdlib types and several third party libraries + +When resolving step (3) type checkers should assure the version of the stubs +match the installed runtime package. + +Type checkers that check a different Python version than the version they run +on must find the type information in the ``site-packages``/``dist-packages`` +of that Python version. This can be queried e.g. +``pythonX.Y -c 'import sys; print(sys.exec_prefix)'``. It is also recommended +that the type checker allow for the user to point to a particular Python +binary, in case it is not in the path. + +To check if a package has opted into type checking, type checkers are +recommended to use the ``pkg_resources`` module to query the package +metadata. If the ``typed`` package metadata has ``None`` as its value, the +package has not opted into type checking, and the type checker should skip that +package. + + +References +========== + +.. [1] PEP 484, Storing and Distributing Stub Files + (https://www.python.org/dev/peps/pep-0484/#storing-and-distributing-stub-files) + +.. [2] Distutils Extensions, Adding setup() arguments + (http://setuptools.readthedocs.io/en/latest/setuptools.html#adding-setup-arguments) + +Copyright +========= + +This document has been placed in the public domain. + + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: From 334f64aa7413f0ff53987232199dd513894c5304 Mon Sep 17 00:00:00 2001 From: Ethan Smith Date: Sun, 10 Sep 2017 13:11:25 -0700 Subject: [PATCH 7/8] remove old PEP --- pep-9999.rst | 178 --------------------------------------------------- 1 file changed, 178 deletions(-) delete mode 100644 pep-9999.rst diff --git a/pep-9999.rst b/pep-9999.rst deleted file mode 100644 index 51bb7e39095..00000000000 --- a/pep-9999.rst +++ /dev/null @@ -1,178 +0,0 @@ -PEP: 9999 -Title: Distributing and Packaging Type Information -Author: Ethan Smith -Status: Draft -Type: Standards Track -Content-Type: text/x-rst -Created: 09-Sep-2017 -Python-Version: 3.7 -Post-History: - - -Abstract -======== - -PEP 484 introduced type hints to Python, with goals of making typing -gradual and easy to adopt. Currently, typing information must be distributed -manually. This PEP provides a standardized means to package and distribute -type information and an ordering for type checkers to resolve modules and -collect this information for type checking using existing packaging -architecture. - - -Rationale -========= - -PEP 484 has a brief section on distributing typing information. In this -section [1]_ the PEP recommends using ``shared/typehints/pythonX.Y/`` for -shipping stub files. However, manually adding a path to stub files for each -third party library does not scale. The simplest approach people have taken -is to add ``site-packages`` to their ``PYTHONPATH``, but this causes type -checkers to fail on packages that are highly dynamic (e.g. sqlalchemy -and Django). - -Furthermore, package authors are wishing to distribute code that has -inline type information, and there currently is no standard method to -distribute packages with inline type annotations or syntax that can -simultaneously be used at runtime and in type checking. - - -Specification -============= - -There are several motivations and methods of supporting typing in a package. This PEP recognizes three (3) types of packages that may be created: - -1. The package maintainer would like to add type information inline. - -2. The package maintainer would like to add type information via stubs. - -3. A third party would like to share stub files for a package, but the - maintainer does not want to include them in the source of the package. - -This PEP aims to support these scenarios and make them simple to add to packaging and deploy. - -The two major parts of this specification are the packaging specifications -and the resolution order for resolving module type information. This spec -is meant to replace the ``shared/typehints/pythonX.Y/`` spec of PEP 484 [1]_. - -Packaging Type Information --------------------------- - -Packages must opt into supporting typing. This will be done though a distutils -extension [2]_, providing a ``typed`` keyword argument to the distutils -``setup()`` command. The argument value will depend on the kind of type -information the package provides. The distutils extension will be added to the -``typing`` package. Therefore a package maintainer may write - -:: - - setup( - ... - setup_requires=["typing"], - typed="inline", - ... - ) - -Stub Only Packages -'''''''''''''''''' - -For package maintainers wishing to ship stub files containing all of their -type information, it is prefered that the ``*.pyi`` stubs are alongside the -corresponding ``*.py`` files. However, the stubs may be put in a sub-folder -of the Python sources, th the same name the ``*.py`` files are in. For -example, the ``flyingcircus`` package would have its stubs in the folder -``flyingcircus/flyingcircus/``. This path is chosen so that if stubs are -not found in ``flyingcircus/`` the type checker may treat the subdirectory as -a normal package. The normal resolution order of checking ``*.pyi`` before -``*.py`` will be maintained. The value of the ``typed`` argument to -``setup()`` is ``"stubs"`` for this type of distribution. The author of the -package is suggested to use ``package_data`` to assure the stub files are -installed alongside the runtime Python code. - -Inline Typed Packages -'''''''''''''''''''' - -Packages that have inline type annotations simply have to pass the value -``"inline"`` to the ``typed`` argument in ``setup()``. - -Third Party Stub Packages -''''''''''''''''''''''''' - -Third parties seeking to distribute stub files are encouraged to contact the -maintainer of the package about distribution alongside the package. If the -maintainer does not wish to maintain or package stub files or type information -inline, then a "third party stub package" should be created. The structure is -similar, but slightly different from that of stub only packages. If the stubs -are for the library ``flyingcircus`` then the package should be named -``flyingcircus-stubs`` and the stub files should be put in a sub-directory -named ``flyingcircus``. This allows the stubs to be checked as if they were in -a regular package. These packages should also pass ``"stubs"`` as the value -of ``typed`` argument in ``setup()``. These packages are suggested to use -``package_data`` to package stub files. - -The version of the ``flyingcircus-stubs`` package should match the version of -the ``flyingcircus`` package it is providing types for. - -Type Checker Module Resolution Order ------------------------------------- - -The following is the order that type checkers supporting this PEP should -resolve modules containing type information: - -1. User code - the files the type checker is running on. - -2. Stubs or Python source in ``PYTHONPATH``. This is to allow the user - complete control of which stubs to use, and patch broken stubs/inline - types from packages. - -3. Third party stub packages - these packages can supersede the installed - untyped packages. They can be found at ``pkg-stubs`` for package ``pkg``, - however it is encouraged to check their metadata to confirm that they opt - into type checking. - -4. Inline packages - finally, if there is nothing overriding the installed - package, and it opts into type checking. - -5. Typeshed (if used) - Provides the stdlib types and several third party libraries - -When resolving step (3) type checkers should assure the version of the stubs -match the installed runtime package. - -Type checkers that check a different Python version than the version they run -on must find the type information in the ``site-packages``/``dist-packages`` -of that Python version. This can be queried e.g. -``pythonX.Y -c 'import sys; print(sys.exec_prefix)'``. It is also recommended -that the type checker allow for the user to point to a particular Python -binary, in case it is not in the path. - -To check if a package has opted into type checking, type checkers are -recommended to use the ``pkg_resources`` module to query the package -metadata. If the ``typed`` package metadata has ``None`` as its value, the -package has not opted into type checking, and the type checker should skip that -package. - - -References -========== - -.. [1] PEP 484, Storing and Distributing Stub Files - (https://www.python.org/dev/peps/pep-0484/#storing-and-distributing-stub-files) - -.. [2] Distutils Extensions, Adding setup() arguments - (http://setuptools.readthedocs.io/en/latest/setuptools.html#adding-setup-arguments) - -Copyright -========= - -This document has been placed in the public domain. - - - -.. - Local Variables: - mode: indented-text - indent-tabs-mode: nil - sentence-end-double-space: t - fill-column: 70 - coding: utf-8 - End: From 48817228e847654764a210a50b037052c7e2d8b7 Mon Sep 17 00:00:00 2001 From: Ethan Smith Date: Sun, 10 Sep 2017 14:17:12 -0700 Subject: [PATCH 8/8] reorder Packaging and minor fixes --- pep-0561.rst | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/pep-0561.rst b/pep-0561.rst index 3a5e0999776..ebc8158cfa4 100644 --- a/pep-0561.rst +++ b/pep-0561.rst @@ -49,7 +49,8 @@ There are several motivations and methods of supporting typing in a package. 3. A third party would like to share stub files for a package, but the maintainer does not want to include them in the source of the package. -This PEP aims to support these scenarios and make them simple to add to packaging and deploy. +This PEP aims to support these scenarios and make them simple to add to +packaging and deployment. The two major parts of this specification are the packaging specifications and the resolution order for resolving module type information. This spec @@ -73,13 +74,19 @@ information the package provides. The distutils extension will be added to the ... ) +Inline Typed Packages +''''''''''''''''''''' + +Packages that have inline type annotations simply have to pass the value +``"inline"`` to the ``typed`` argument in ``setup()``. + Stub Only Packages '''''''''''''''''' For package maintainers wishing to ship stub files containing all of their type information, it is prefered that the ``*.pyi`` stubs are alongside the corresponding ``*.py`` files. However, the stubs may be put in a sub-folder -of the Python sources, th the same name the ``*.py`` files are in. For +of the Python sources, with the same name the ``*.py`` files are in. For example, the ``flyingcircus`` package would have its stubs in the folder ``flyingcircus/flyingcircus/``. This path is chosen so that if stubs are not found in ``flyingcircus/`` the type checker may treat the subdirectory as @@ -89,12 +96,6 @@ a normal package. The normal resolution order of checking ``*.pyi`` before package is suggested to use ``package_data`` to assure the stub files are installed alongside the runtime Python code. -Inline Typed Packages -''''''''''''''''''''' - -Packages that have inline type annotations simply have to pass the value -``"inline"`` to the ``typed`` argument in ``setup()``. - Third Party Stub Packages '''''''''''''''''''''''''