Skip to content

Commit

Permalink
[3.11] pythongh-98706: Sync with importlib_metadata 4.13.0.
Browse files Browse the repository at this point in the history
  • Loading branch information
jaraco committed Oct 30, 2022
1 parent a55bd6f commit 2c36820
Show file tree
Hide file tree
Showing 6 changed files with 208 additions and 92 deletions.
128 changes: 95 additions & 33 deletions Doc/library/importlib.metadata.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,29 +13,61 @@

**Source code:** :source:`Lib/importlib/metadata/__init__.py`

``importlib.metadata`` is a library that provides for access to installed
package metadata. Built in part on Python's import system, this library
``importlib_metadata`` is a library that provides access to
the metadata of an installed :term:`packaging:Distribution Package`,
such as its entry points
or its top-level names (:term:`packaging:Import Package`s, modules, if any).
Built in part on Python's import system, this library
intends to replace similar functionality in the `entry point
API`_ and `metadata API`_ of ``pkg_resources``. Along with
:mod:`importlib.resources` (with new features backported to the
`importlib_resources`_ package), this can eliminate the need to use the older
and less efficient
:mod:`importlib.resources`,
this package can eliminate the need to use the older and less efficient
``pkg_resources`` package.

By "installed package" we generally mean a third-party package installed into
Python's ``site-packages`` directory via tools such as `pip
<https://pypi.org/project/pip/>`_. Specifically,
it means a package with either a discoverable ``dist-info`` or ``egg-info``
directory, and metadata defined by :pep:`566` or its older specifications.
By default, package metadata can live on the file system or in zip archives on
``importlib_metadata`` operates on third-party *distribution packages*
installed into Python's ``site-packages`` directory via tools such as
`pip <https://pypi.org/project/pip/>`_.
Specifically, it works with distributions with discoverable
``dist-info`` or ``egg-info`` directories,
and metadata defined by the :ref:`packaging:core-metadata`.

.. important::

These are *not* necessarily equivalent to or correspond 1:1 with
the top-level *import package* names
that can be imported inside Python code.
One *distribution package* can contain multiple *import packages*
(and single modules),
and one top-level *import package*
may map to multiple *distribution packages*
if it is a namespace package.
You can use :ref:`package_distributions() <package-distributions>`
to get a mapping between them.

By default, distribution metadata can live on the file system
or in zip archives on
:data:`sys.path`. Through an extension mechanism, the metadata can live almost
anywhere.


.. seealso::

https://importlib-metadata.readthedocs.io/
The documentation for ``importlib_metadata``, which supplies a
backport of ``importlib.metadata``.
This includes an `API reference
<https://importlib-metadata.readthedocs.io/en/latest/api.html>`__
for this module's classes and functions,
as well as a `migration guide
<https://importlib-metadata.readthedocs.io/en/latest/migration.html>`__
for existing users of ``pkg_resources``.


Overview
========

Let's say you wanted to get the version string for a package you've installed
Let's say you wanted to get the version string for a
:term:`packaging:Distribution Package` you've installed
using ``pip``. We start by creating a virtual environment and installing
something into it:

Expand All @@ -54,9 +86,9 @@ You can get the version string for ``wheel`` by running the following:
>>> version('wheel') # doctest: +SKIP
'0.32.3'
You can also get the set of entry points keyed by group, such as
You can also get a collection of entry points selectable by properties of the EntryPoint (typically 'group' or 'name'), such as
``console_scripts``, ``distutils.commands`` and others. Each group contains a
sequence of :ref:`EntryPoint <entry-points>` objects.
collection of :ref:`EntryPoint <entry-points>` objects.

You can get the :ref:`metadata for a distribution <metadata>`::

Expand Down Expand Up @@ -91,7 +123,7 @@ Query all entry points::
>>> eps = entry_points() # doctest: +SKIP

The ``entry_points()`` function returns an ``EntryPoints`` object,
a sequence of all ``EntryPoint`` objects with ``names`` and ``groups``
a collection of all ``EntryPoint`` objects with ``names`` and ``groups``
attributes for convenience::

>>> sorted(eps.groups) # doctest: +SKIP
Expand Down Expand Up @@ -156,7 +188,8 @@ interface to retrieve entry points by group.
Distribution metadata
---------------------

Every distribution includes some metadata, which you can extract using the
Every :term:`packaging:Distribution Package` includes some metadata,
which you can extract using the
``metadata()`` function::

>>> wheel_metadata = metadata('wheel') # doctest: +SKIP
Expand All @@ -174,6 +207,13 @@ all the metadata in a JSON-compatible form per :PEP:`566`::
>>> wheel_metadata.json['requires_python']
'>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*'

.. note::

The actual type of the object returned by ``metadata()`` is an
implementation detail and should be accessed only through the interface
described by the
`PackageMetadata protocol <https://importlib-metadata.readthedocs.io/en/latest/api.html#importlib_metadata.PackageMetadata>`.

.. versionchanged:: 3.10
The ``Description`` is now included in the metadata when presented
through the payload. Line continuation characters have been removed.
Expand All @@ -187,7 +227,8 @@ all the metadata in a JSON-compatible form per :PEP:`566`::
Distribution versions
---------------------

The ``version()`` function is the quickest way to get a distribution's version
The ``version()`` function is the quickest way to get a
:term:`packaging:Distribution Package`'s version
number, as a string::

>>> version('wheel') # doctest: +SKIP
Expand All @@ -200,7 +241,8 @@ Distribution files
------------------

You can also get the full set of files contained within a distribution. The
``files()`` function takes a distribution package name and returns all of the
``files()`` function takes a :term:`packaging:Distribution Package` name
and returns all of the
files installed by this distribution. Each file object returned is a
``PackagePath``, a :class:`pathlib.PurePath` derived object with additional ``dist``,
``size``, and ``hash`` properties as indicated by the metadata. For example::
Expand Down Expand Up @@ -245,19 +287,24 @@ distribution is not known to have the metadata present.
Distribution requirements
-------------------------

To get the full set of requirements for a distribution, use the ``requires()``
To get the full set of requirements for a :term:`packaging:Distribution Package`,
use the ``requires()``
function::

>>> requires('wheel') # doctest: +SKIP
["pytest (>=3.0.0) ; extra == 'test'", "pytest-cov ; extra == 'test'"]


Package distributions
---------------------
.. _package-distributions:
.. _import-distribution-package-mapping:

A convenience method to resolve the distribution or
distributions (in the case of a namespace package) for top-level
Python packages or modules::
Mapping import to distribution packages
---------------------------------------

A convenience method to resolve the :term:`packaging:Distribution Package`
name (or names, in the case of a namespace package)
that provide each importable top-level
Python module or :term:`packaging:Import Package`::

>>> packages_distributions()
{'importlib_metadata': ['importlib-metadata'], 'yaml': ['PyYAML'], 'jaraco': ['jaraco.classes', 'jaraco.functools'], ...}
Expand All @@ -271,7 +318,8 @@ Distributions

While the above API is the most common and convenient usage, you can get all
of that information from the ``Distribution`` class. A ``Distribution`` is an
abstract object that represents the metadata for a Python package. You can
abstract object that represents the metadata for
a Python :term:`packaging:Distribution Package`. You can
get the ``Distribution`` instance::

>>> from importlib.metadata import distribution # doctest: +SKIP
Expand All @@ -291,22 +339,36 @@ instance::
>>> dist.metadata['License'] # doctest: +SKIP
'MIT'

The full set of available metadata is not described here. See :pep:`566`
for additional details.
The full set of available metadata is not described here.
See the :ref:`packaging:core-metadata` for additional details.


Distribution Discovery
======================

By default, this package provides built-in support for discovery of metadata
for file system and zip file :term:`packaging:Distribution Package`\s.
This metadata finder search defaults to ``sys.path``, but varies slightly in how it interprets those values from how other import machinery does. In particular:

- ``importlib.metadata`` does not honor :class:`bytes` objects on ``sys.path``.
- ``importlib.metadata`` will incidentally honor :py:class:`pathlib.Path` objects on ``sys.path`` even though such values will be ignored for imports.


Extending the search algorithm
==============================

Because package metadata is not available through :data:`sys.path` searches, or
package loaders directly, the metadata for a package is found through import
system :ref:`finders <finders-and-loaders>`. To find a distribution package's metadata,
Because :term:`packaging:Distribution Package` metadata
is not available through :data:`sys.path` searches, or
package loaders directly,
the metadata for a distribution is found through import
system `finders`_. To find a distribution package's metadata,
``importlib.metadata`` queries the list of :term:`meta path finders <meta path finder>` on
:data:`sys.meta_path`.

The default ``PathFinder`` for Python includes a hook that calls into
``importlib.metadata.MetadataPathFinder`` for finding distributions
loaded from typical file-system-based paths.
By default ``importlib_metadata`` installs a finder for distribution packages
found on the file system.
This finder doesn't actually find any *distributions*,
but it can find their metadata.

The abstract class :py:class:`importlib.abc.MetaPathFinder` defines the
interface expected of finders by Python's import system.
Expand Down
52 changes: 38 additions & 14 deletions Lib/importlib/metadata/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,10 @@ class EntryPoint(DeprecatedTuple):
following the attr, and following any extras.
"""

name: str
value: str
group: str

dist: Optional['Distribution'] = None

def __init__(self, name, value, group):
Expand Down Expand Up @@ -543,21 +547,21 @@ def locate_file(self, path):
"""

@classmethod
def from_name(cls, name):
def from_name(cls, name: str):
"""Return the Distribution for the given package name.
:param name: The name of the distribution package to search for.
:return: The Distribution instance (or subclass thereof) for the named
package, if found.
:raises PackageNotFoundError: When the named package's distribution
metadata cannot be found.
:raises ValueError: When an invalid value is supplied for name.
"""
for resolver in cls._discover_resolvers():
dists = resolver(DistributionFinder.Context(name=name))
dist = next(iter(dists), None)
if dist is not None:
return dist
else:
if not name:
raise ValueError("A distribution name is required.")
try:
return next(cls.discover(name=name))
except StopIteration:
raise PackageNotFoundError(name)

@classmethod
Expand Down Expand Up @@ -945,13 +949,26 @@ def _normalized_name(self):
normalized name from the file system path.
"""
stem = os.path.basename(str(self._path))
return self._name_from_stem(stem) or super()._normalized_name
return (
pass_none(Prepared.normalize)(self._name_from_stem(stem))
or super()._normalized_name
)

def _name_from_stem(self, stem):
name, ext = os.path.splitext(stem)
@staticmethod
def _name_from_stem(stem):
"""
>>> PathDistribution._name_from_stem('foo-3.0.egg-info')
'foo'
>>> PathDistribution._name_from_stem('CherryPy-3.0.dist-info')
'CherryPy'
>>> PathDistribution._name_from_stem('face.egg-info')
'face'
>>> PathDistribution._name_from_stem('foo.bar')
"""
filename, ext = os.path.splitext(stem)
if ext not in ('.dist-info', '.egg-info'):
return
name, sep, rest = stem.partition('-')
name, sep, rest = filename.partition('-')
return name


Expand Down Expand Up @@ -991,6 +1008,15 @@ def version(distribution_name):
return distribution(distribution_name).version


_unique = functools.partial(
unique_everseen,
key=operator.attrgetter('_normalized_name'),
)
"""
Wrapper for ``distributions`` to return unique distributions by name.
"""


def entry_points(**params) -> Union[EntryPoints, SelectableGroups]:
"""Return EntryPoint objects for all installed packages.
Expand All @@ -1008,10 +1034,8 @@ def entry_points(**params) -> Union[EntryPoints, SelectableGroups]:
:return: EntryPoints or SelectableGroups for all installed packages.
"""
norm_name = operator.attrgetter('_normalized_name')
unique = functools.partial(unique_everseen, key=norm_name)
eps = itertools.chain.from_iterable(
dist.entry_points for dist in unique(distributions())
dist.entry_points for dist in _unique(distributions())
)
return SelectableGroups.load(eps).select(**params)

Expand Down
16 changes: 16 additions & 0 deletions Lib/test/test_importlib/fixtures.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
import pathlib
import tempfile
import textwrap
import functools
import contextlib

from test.support.os_helper import FS_NONASCII
Expand Down Expand Up @@ -296,3 +297,18 @@ def setUp(self):
# Add self.zip_name to the front of sys.path.
self.resources = contextlib.ExitStack()
self.addCleanup(self.resources.close)


def parameterize(*args_set):
"""Run test method with a series of parameters."""

def wrapper(func):
@functools.wraps(func)
def _inner(self):
for args in args_set:
with self.subTest(**args):
func(self, **args)

return _inner

return wrapper
Loading

0 comments on commit 2c36820

Please sign in to comment.