5.0.0 fails with flake8

[xqt]

The new import_meta_data 5.0.0 fails with hacking flake8 extension:

12:58:37 hacking-py37 cannot reuse: no previous config /src/.tox/hacking-py37/.tox-config1
12:58:37 hacking-py37 create: /src/.tox/hacking-py37
12:58:37 [302] /src/.tox$ /usr/bin/python3 -m virtualenv --no-download --python /usr/bin/python3 hacking-py37 >hacking-py37/log/hacking-py37-0.log
12:58:37 hacking-py37 installdeps: .[hacking]
12:58:37 [309] /src$ /src/.tox/hacking-py37/bin/python -m pip install '.[hacking]' >.tox/hacking-py37/log/hacking-py37-1.log
12:58:42 hacking-py37 develop-inst: /src
12:58:42 write config to /src/.tox/hacking-py37/.tox-config1 as 'a333bea1ff24374b5ce3a229b583f709ed538d6822235f40ac210c70df65a22b /usr/bin/python3\n3.21.4 0 1 0\n00000000000000000000000000000000 .[hacking]'
12:58:42 [321] /src$ /src/.tox/hacking-py37/bin/python -m pip install --exists-action w -e . >.tox/hacking-py37/log/hacking-py37-2.log
12:58:44 [332] /src$ /src/.tox/hacking-py37/bin/python -m pip freeze >.tox/hacking-py37/log/hacking-py37-3.log
12:58:44 hacking-py37 installed: certifi==2022.9.24,charset-normalizer==2.1.1,flake8==3.8.4,hacking==4.1.0,idna==3.4,importlib-metadata==5.0.0,mccabe==0.6.1,pycodestyle==2.6.0,pyflakes==2.2.0,# Editable Git install with no remote (pywikibot==8.0.0.dev0),-e /src,requests==2.28.1,typing_extensions==4.3.0,urllib3==1.26.12,zipp==3.8.1
12:58:44 hacking-py37 run-test-pre: PYTHONHASHSEED='2157981050'
12:58:44 hacking-py37 run-test: commands[0] | flake8 --version
12:58:44 [335] /src$ /src/.tox/hacking-py37/bin/flake8 --version
12:58:44 Traceback (most recent call last):
12:58:44   File "/src/.tox/hacking-py37/bin/flake8", line 8, in <module>
12:58:44     sys.exit(main())
12:58:44   File "/src/.tox/hacking-py37/lib/python3.7/site-packages/flake8/main/cli.py", line 22, in main
12:58:44     app.run(argv)
12:58:44   File "/src/.tox/hacking-py37/lib/python3.7/site-packages/flake8/main/application.py", line 363, in run
12:58:44     self._run(argv)
12:58:44   File "/src/.tox/hacking-py37/lib/python3.7/site-packages/flake8/main/application.py", line 350, in _run
12:58:44     self.initialize(argv)
12:58:44   File "/src/.tox/hacking-py37/lib/python3.7/site-packages/flake8/main/application.py", line 330, in initialize
12:58:44     self.find_plugins(config_finder)
12:58:44   File "/src/.tox/hacking-py37/lib/python3.7/site-packages/flake8/main/application.py", line 153, in find_plugins
12:58:44     self.check_plugins = plugin_manager.Checkers(local_plugins.extension)
12:58:44   File "/src/.tox/hacking-py37/lib/python3.7/site-packages/flake8/plugins/manager.py", line 357, in __init__
12:58:44     self.namespace, local_plugins=local_plugins
12:58:44   File "/src/.tox/hacking-py37/lib/python3.7/site-packages/flake8/plugins/manager.py", line 238, in __init__
12:58:44     self._load_entrypoint_plugins()
12:58:44   File "/src/.tox/hacking-py37/lib/python3.7/site-packages/flake8/plugins/manager.py", line 254, in _load_entrypoint_plugins
12:58:44     eps = importlib_metadata.entry_points().get(self.namespace, ())
12:58:44 AttributeError: 'EntryPoints' object has no attribute 'get'
12:58:44 ERROR: InvocationError for command /src/.tox/hacking-py37/bin/flake8 --version (exited with code 1)

See CI test here

It works with the previous release 4.13.0:

• https://integration.wikimedia.org/ci/job/pywikibot-core-tox-docker/20921/console
• https://gerrit.wikimedia.org/r/c/pywikibot/core/+/837218

[jaraco]

It's a known issue that flake8 relies on the deprecated interfaces removed in importlib_metadata 5. I've attempted to engage with the maintainers of that project, but they refuse to engage and dismiss my proposals without suggesting alternatives. Flake8 does declare a dependency on importlib_metadata<4.3, so the fact that an incompatible version of importlib_metadata is being installed is probably an issue with the environment setup.

I do see that I can replicate the issue with pip install hacking (via pip-run):

 draft $ docker run -it jaraco/multipy-tox py -3.7 -m pip-run hacking -- -c pass
Collecting hacking
  Downloading hacking-4.1.0-py3-none-any.whl (42 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.2/42.2 kB 1.6 MB/s eta 0:00:00
Collecting flake8<3.9.0,>=3.8.0
  Downloading flake8-3.8.4-py2.py3-none-any.whl (72 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 72.9/72.9 kB 8.9 MB/s eta 0:00:00
Collecting mccabe<0.7.0,>=0.6.0
  Downloading mccabe-0.6.1-py2.py3-none-any.whl (8.6 kB)
Collecting pycodestyle<2.7.0,>=2.6.0a1
  Downloading pycodestyle-2.6.0-py2.py3-none-any.whl (41 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.4/41.4 kB 12.4 MB/s eta 0:00:00
Collecting importlib-metadata
  Downloading importlib_metadata-5.0.0-py3-none-any.whl (21 kB)
Collecting pyflakes<2.3.0,>=2.2.0
  Downloading pyflakes-2.2.0-py2.py3-none-any.whl (66 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67.0/67.0 kB 16.6 MB/s eta 0:00:00
Collecting typing-extensions>=3.6.4
  Using cached typing_extensions-4.3.0-py3-none-any.whl (25 kB)
Collecting zipp>=0.5
  Downloading zipp-3.8.1-py3-none-any.whl (5.6 kB)
Installing collected packages: mccabe, zipp, typing-extensions, pyflakes, pycodestyle, importlib-metadata, flake8, hacking
Successfully installed flake8-3.8.4 hacking-4.1.0 importlib-metadata-5.0.0 mccabe-0.6.1 pycodestyle-2.6.0 pyflakes-2.2.0 typing-extensions-4.3.0 zipp-3.8.1

Even though flake8 pins importlib_metadata on Python 3.7, pip installs importlib_metadata 5.

Interestingly, if I install only flake8~=4.0.1 (the sole dependency of hacking) using the same command, the declared version of importlib_metadata is pulled:

 draft $ docker run -it jaraco/multipy-tox py -3.7 -m pip-run 'flake8~=4.0.1' -- -c pass
Collecting flake8~=4.0.1
  Downloading flake8-4.0.1-py2.py3-none-any.whl (64 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.1/64.1 kB 2.1 MB/s eta 0:00:00
Collecting mccabe<0.7.0,>=0.6.0
  Downloading mccabe-0.6.1-py2.py3-none-any.whl (8.6 kB)
Collecting pycodestyle<2.9.0,>=2.8.0
  Downloading pycodestyle-2.8.0-py2.py3-none-any.whl (42 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.1/42.1 kB 9.7 MB/s eta 0:00:00
Collecting pyflakes<2.5.0,>=2.4.0
  Downloading pyflakes-2.4.0-py2.py3-none-any.whl (69 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 69.7/69.7 kB 17.2 MB/s eta 0:00:00
Collecting importlib-metadata<4.3
  Downloading importlib_metadata-4.2.0-py3-none-any.whl (16 kB)
Collecting zipp>=0.5
  Downloading zipp-3.8.1-py3-none-any.whl (5.6 kB)
Collecting typing-extensions>=3.6.4
  Using cached typing_extensions-4.3.0-py3-none-any.whl (25 kB)
Installing collected packages: mccabe, zipp, typing-extensions, pyflakes, pycodestyle, importlib-metadata, flake8
Successfully installed flake8-4.0.1 importlib-metadata-4.2.0 mccabe-0.6.1 pycodestyle-2.8.0 pyflakes-2.4.0 typing-extensions-4.3.0 zipp-3.8.1

I guess there may be an issue with how pip resolves dependencies of dependencies.

Oh, I see a difference. When installing hacking from a wheel, it gets 4.1.0, which depends on flake8 3.8.x and flake8 3.8.x doesn't pin importlib_metadata.

[jaraco]

There's nothing that can be done here from the perspective of importlib_metadata. The deprecation has been in place since Feb 2021 and other projects have been able to adapt to the transition. These issues were anticipated and socialized and this project has worked actively with affected projects to provide multiple avenues to navigate the transition. I'm still willing to help flake8 support the preferred interfaces. What's not going to happen is to undo the deprecation or restore support for these deprecated interfaces indefinitely (or for many years).

As a user of flake8, your best option is probably to preemptively install importlib_metadata<5 until flake8 can adapt.

[tony]

@jaraco @xqt I filed an issue on the flake8 side here: PyCQA/flake8#1701

Workaround is as @jaraco says above, pin importlib_metadata to <5 for now.