Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace pkg_resources with importlib #731

Merged
merged 4 commits into from
Oct 19, 2021
Merged

Replace pkg_resources with importlib #731

merged 4 commits into from
Oct 19, 2021

Conversation

ubaumann
Copy link
Contributor

I was using nornir in a jupyter notebook and realized the plugin auto register did not work.

The following snippet gives an empty list in the newest jupyter notebook and works fine in REPL.

import pkg_resources
pkg_resources.iter_entry_points("nornir.plugins.connections")

After some investigation I ended up with this snippet working in both environments:

from importlib import metadata
metadata.entry_points()["nornir.plugins.connections"]

Setuptools is recommending using importlib.metadata in the documentation: https://setuptools.pypa.io/en/latest/userguide/entry_point.html#advertising-behavior

If I can change or add anything please let me know

@dbarrosop
Copy link
Contributor

dbarrosop commented Oct 19, 2021

Nice work, just one comment, the docs you linked mentions the new metadata functionality was introduced in 3.8, which I can confirm with docker:

$ docker run --rm -it python:3.8
Python 3.8.12 (default, Oct 16 2021, 10:26:20) 
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from importlib import metadata
>>> metadata.entry_points()
{'console_scripts': (EntryPoint(name='wheel',...}

But in your PR you set the dependency and the import for <3.10. What am I missing here?

@dbarrosop
Copy link
Contributor

This is the exact line I am referring to;

For a project wishing to solicit entry points, Setuptools recommends the importlib.metadata module (part of stdlib since Python 3.8) or its backport, importlib_metadata.

From https://setuptools.pypa.io/en/latest/userguide/entry_point.html#advertising-behavior

@ubaumann
Copy link
Contributor Author

Yes I was really confused in the beginning. I had everything running in 3.8 but mypy didn't like me in py3.9 so I looked deeper and saw a fundamental API change between 3.8 and 3.10.

The document calls importlib.metadata "no longer provisional" in 3.10.

https://docs.python.org/3/library/importlib.metadata.html

The API in 3.10 and the backport lib is now the same and works well as I think.

@ubaumann
Copy link
Contributor Author

In the readme of the backport lib https://pypi.org/project/importlib-metadata/ you can see from version 4.4 it is compatible with Python 3.10

@ubaumann
Copy link
Contributor Author

3.8

Python 3.8.12 (default, Oct 13 2021, 09:15:35)
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from importlib import metadata
>>> type(metadata.entry_points())
<class 'dict'>

3.9

Python 3.9.7 (default, Sep  3 2021, 20:10:26)
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from importlib import metadata
>>> type(metadata.entry_points())
<class 'dict'>

3.10

Python 3.10.0 (default, Oct  5 2021, 23:39:58) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from importlib import metadata
>>> type(metadata.entry_points())
<class 'importlib.metadata.SelectableGroups'>

3.9 with pip install importlib-metadata

Python 3.9.7 (default, Sep  3 2021, 20:10:26)
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import importlib_metadata as metadata
>>> type(metadata.entry_points())
<class 'importlib_metadata.SelectableGroups'>

@dbarrosop
Copy link
Contributor

I figured there was something, would you mind adding a note in the pyproject.toml for future reference? Otherwise I am sure someone will be doing this same spelunking again.

Thanks!

@ubaumann
Copy link
Contributor Author

Sure. I added a short note.

Thanks for your feedback

@dbarrosop
Copy link
Contributor

Awesome, thanks to you for the PR!

@dbarrosop dbarrosop merged commit e63492c into nornir-automation:develop Oct 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants