📦 Enable support for PEP 639 metadata

[webknjaz]

This is achieved by upgrading Twine to v6.1.0. Prior to this version, Twine was unable to pick up and publish licensing information declared in the new License-Expression core packaging metadata [1] [2]. And now it does that.

Resolves #325.
Closes #326.

[dnicolodi]

For metadata 2.4 support, you also need to update packaging to 24.2, see the twine 6.1 release notes.

[webknjaz]

@dnicolodi thanks for the pointer! I also need to figure out the CI failure, just haven't had a chance to debug it yet. Quick googling implies that upgrading pip is needed, but it's happening in pypa/build meaning that pip it uses might be a little more difficult to control.

[dnicolodi]

I had a quick look, and I have the impression that the problem is caused by the old version of setuptools (or better, the version of pkg_resources vendored into setuptools) not supporting the Python version used in the tests. setuptools 65.6.3 uses pkgutil.ImpImporter which is gone in Python 3.12.

[webknjaz]

That's my impression too. But I'm unsure in what place exactly.

[webknjaz]

Just confirmed that the CI failure is unrelated to the patch in this PR. See: #328.

[webknjaz]

The last successful CI run was a month ago. And it has a slightly different set of transitive deps: https://github.com/pypa/gh-action-pypi-publish/actions/runs/12459567492/job/34776560934#step:4:102. Too bad we're not pinning those in CI.

The successful workflow run pulls in pkginfo-1.12.0 while these failing ones do not.

- Successfully installed Pygments-2.18.0 SecretStorage-3.3.3 backports.tarfile-1.2.0 build-1.2.2.post1 certifi-2024.2.2 cffi-1.16.0 charset-normalizer-3.3.2 cryptography-42.0.7 docutils-0.21.2 idna-3.7 importlib-metadata-8.5.0 jaraco.classes-3.4.0 jaraco.context-5.3.0 jaraco.functools-4.0.1 jeepney-0.8.0 keyring-25.2.1 markdown-it-py-3.0.0 mdurl-0.1.2 more-itertools-10.2.0 nh3-0.2.17 packaging-24.1 pkginfo-1.12.0 pycparser-2.22 pyproject_hooks-1.2.0 readme-renderer-43.0 requests-2.32.3 requests-toolbelt-1.0.0 rfc3986-2.0.0 rich-13.7.1 twine-6.0.1 urllib3-2.2.1 zipp-3.21.0
+ Successfully installed Pygments-2.18.0 SecretStorage-3.3.3 build-1.2.2.post1 certifi-2024.2.2 cffi-1.16.0 charset-normalizer-3.3.2 cryptography-42.0.7 docutils-0.21.2 idna-3.7 jaraco.classes-3.4.0 jaraco.context-5.3.0 jaraco.functools-4.0.1 jeepney-0.8.0 keyring-25.2.1 more-itertools-10.2.0 nh3-0.2.17 packaging-24.1 pkginfo-1.12.0 pycparser-2.22 pyproject_hooks-1.2.0 readme-renderer-43.0 requests-2.32.3 requests-toolbelt-1.0.0 rfc3986-2.0.0 twine-6.0.1 urllib3-2.2.1

[webknjaz]

Ah, I was wrong. Both install pkginfo but there's other differences.

[webknjaz]

Pinning all of those to the same version didn't help. There's something else influencing this.

[webknjaz]

The failing jobs use Ubuntu 24.04.1 while the successful ones run 22.04.5. This would explain the difference in what python3 is pointing to. The successful jobs used to run Python 3.9 while the failing ones pick up 3.12.

[webknjaz]

The successful jobs used to run Python 3.9 while the failing ones pick up 3.12.

Yep! That was it: I suppose because Python 3.9 bundles a certain version of setuptools in stdlib (as a part of ensurepip) and 3.12 dropped it, it no longer shadows some other version that's also available somewhere in the env.

The ephemeral PEP 517 build envs are supposed to be isolated, but there were cases where the isolation was imperfect, causing importables from other envs to leak into the “isolated” env. This must be it!

[kytta]

Can it be that it's the old setuptools being used as build-system?

The CI is using hard-pinned 65.6.3. In 66.1.0, something around pkgutil was fixed (see also pypa/setuptools#3685). The newest version is 75.x.x.

[dnicolodi]

I should have been more explicit in my comment: the problem is caused by the Python update from 3.9 to 3.12. setuptools being pinned to a version that does not support Python 3.12 (because of pkgutil.ImpImporter being removed in Python 3.12) stops working.

I don't think python shipping setuptools or imperfect build isolation are related to the issue: setuptools gets installed in the pinned version in the build environment (as per the build logs) and pkgutil is a standard library package. The only other moving part in all this is pkg_resources, but that these days is distributed with setuptools, thus what matters is the setuptools version installed.

Anyhow, I agree with the sentiment that updating to a more recent Python would be better than keeping using Python 3.9. In any case, for reproducibility, especially if you are pinning all other dependencies, you should be pinning the Python version too 🙂

[webknjaz]

Yeah, I missed that place. Will see what I can pin in a separate PR.

[webknjaz]

https://bsky.app/profile/webknjaz.me/post/3lghhtwey522z