[FR] Follow semantic versioning and make setuptools more stable

[GergelyKalmar]

What's the problem this feature will solve?

It seems that there are deprecations happening in minor releases (e.g. https://setuptools.pypa.io/en/latest/history.html#v67-3-0, see it causing googleapis/api-common-protos#121 and streamlink/streamlink#5167 among many others), which often breaks workflows even if the major version is pinned.

Additionally, setuptools seems to have a relatively high amount of breaking changes and changes in general for a library that is extremely broadly used (two major version increases already this year, five last year), the changelog is quite daunting to keep up with.

Describe the solution you'd like

It would be awesome if the amount of changes would be lower on setuptools, especially when it comes to backwards-incompatible or breaking changes. Introducing deprecation warnings should be considered a breaking change because it can break workflows that mark Python warnings as errors (often the case in test suites or in CI/CD).

Alternative Solutions

No response

Additional context

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct

[abravalheri]

Hi @GergelyKalmar, thank you very much for opening the discussion.

If I understood correctly the SemVer spec, deprecations are meant to be introduced in minor version bumps, but the final removal should happen in a major version bump. The change you pointed out seems to be inline with that policy.

Also note that built-in support for "modern-style" namespaces was introduced since Python 3.3. "Old-style" namespace packages have been discouraged by PyPA docs for years. They were formally described as deprecated in setuptools docs around 3 years ago. What the change did was merely add the deprecation warning to improve communication with the end users.

Introducing deprecation warnings should be considered a breaking change because it can break workflows that mark Python warnings as errors (often the case in test suites or in CI/CD)

I disagree with you in this point. Warnings are the only tool developers have to effectively communicate with the user base. In the example that you mentioned, the deprecation has been documented for a while, but it only started to make a difference and compel users to change, now that we introduced the warning.

Users are encouraged to opt into a more strict testing routine (using -Werror or -Xdev), so they are not surprised last minute when the breaking change actually happens. Implementing modifications / handling the deprecation immediately is optional. The users can acknowledge that the warning exist and add a warning filter to temporarily ignore it until they have the time to properly handle it.

Additionally, setuptools seems to have a relatively high amount of breaking changes and changes in general for a library that is extremely broadly used (two major version increases already this year, five last year), the changelog is quite daunting to keep up with.

Please note that Python packaging ecosystem is a very dynamic environment and setuptools has to keep up with it. Moreover setuptools has an long-lasting codebase that survived many versions of Python, and maintenance is required.

Not all major changes will break everyone's environment (actually not breaking most of people setup is more likely).

In general, setuptools tries to announce and discuss breaking changes months or even years before final implementation, and we plan to keep doing that using deprecation warnings as a communication tool.

If you are interest in the setuptools release cadence, you can have a look on this comment from the lead developer.

Considering the size of the pool of maintainers in setuptools (and the time we have available for it), I think that is the best we can do.

[GergelyKalmar]

You are totally right insofar as deprecations can be introduced in minor versions, but posting a DeprecationWarning in the Python world is a breaking change in the literal sense of the world (for anyone using PYTHONWARNINGS=error anyways). I've posted multiple cases where a setuptools minor version change broke working systems, so it is hard to argue that this was a backwards-compatible change.

For the reason that warnings can directly alter the flow of the user's program by raising an exception, they should be considered to be part of the public API of a Python package, hence my argument that new warnings should not be added in minor versions. For this reason a slightly better approach would be to add a DeprecationWarning in a major version, and only remove the affected feature in the next major version afterwards (or even a few major versions later when possible). I personally don't see the value of DeprecationWarnings – if there is a major version change that breaks something, users can always decide to defer updating that package until they made the necessary changes to be compatible with it.

I understand that setuptools is in a tough position with the mess that is the Python packaging ecosystem, nonetheless, having a longer change process that bundles multiple strategic changes in less-often released major versions would be much better for most users than the current cadence. This is because many of the benefits that semver provides (e.g. receiving new features/patches/improvements automatically) are only realized when the public API is stable and when major versions change infrequently.

[abravalheri]

DeprecationWarning in the Python world is a breaking change in the literal sense of the world (for anyone using PYTHONWARNINGS=error anyways).

We probably have different (and incompatible) "philosophical" views on this subject, but I respect your opinion¹.

That said, although I disagree with the approach, it would be possible to change the versioning policy to only add deprecation warnings in major versions. However, even if we do that, I don't think the setuptools project is in a position where we can reduce the release cadence and combine multiple changes in less-often released major versions.

@jaraco, would you have any thoughts on this?

Footnotes

1. For me DeprecationWarnings are not breaking changes because they will not break products or systems used in production. They will introduce some tasks in staging environments that are (appropriately) testing with PYTHONWARNINGS=error. I view these tasks as part of the maintenance of any healthy CI workflow. In my mind, noticing the warning and adding the appropriate warning filter are the "software world equivalent" to a RSVP for the deprecation event... ↩

[GergelyKalmar]

Actually, upon thinking about this further, I'm not sure if any of this will help much. I'm not even sure what version of setuptools venv installs (I guess it will be the latest for some if they use --upgrade-deps in 3.9+), and for packages following PEP 517 most won't pin the build backend version either, so in those cases I assume the latest setuptools version will be used during installation/building too. Should we be actually pinning the setuptools version when following PEP 517, even though the documentation doesn't do that? I'm wondering what kind of issues does it bring to always rely on the latest version during building, for example, if a package used one of the current beta or non-beta features that later got deprecated/changed, will that package fail to build and then install in the future from source? Is the installed setuptools version only relevant for older packages that don't follow PEP 517 anyways? Some libraries seem to use features of setuptools (like the one that got now the deprecation warning), even though they don't even list it as a direct dependency – do package authors just rely on setuptools being available on all systems where pip is available? That seems like a bad idea, but it seems it happens anyways.

For some reason setuptools tends to frustrate me the most as a Python developer. I find it hard to understand what is exactly going on (which is perhaps my own fault), but it is also breaking my development workflows in unexpected ways and at random times (which is perhaps not my own fault).

[GergelyKalmar]

Another breakage caused by a deprecation warning introduced in a minor patch: https://github.com/pypa/setuptools/blob/main/CHANGES.rst#v6750 causing googleapis/python-crc32c#168.

My work is interrupted again by a random new setuptools deprecation warning coming from an indirect dependency. Should I really be forced to care about every indirect dependency's own deprecation warnings now? It's madness.

[jaraco]

I agree it’s a high bar to error on DeprecationWarnings. The default expected behavior for DeprecationWarnings is they’re silent for users and visible but not breaking in tests. If a project has configured a more aggressive behavior, the onus is on that project to bear the maintenance burden.

Should we be actually pinning the setuptools version?

My instinct is no, at least not at the build-requirements level. It's perfectly acceptable, even recommended, to do so at the end environment level in order to ensure stability and reproducibility. The challenge, as you've pointed out, is that PEP 517 isolated builds don't give the end environment (installer) any control over the build.

if a package used one of the current beta or non-beta features that later got deprecated/changed, will that package fail to build and then install in the future from source?

Yes, but Setuptools goes to great lengths to maintain compatibility, taking multi-year deprecation periods to remove behaviors depended on by more than a handful of projects.

Some libraries seem to use features of setuptools (like the one that got now the deprecation warning), even though they don't even list it as a direct dependency – do package authors just rely on setuptools being available on all systems where pip is available?

This reliance is a bug. Projects should not be relying on pkg_resources just because it happens to be installed in some environments. If I encounter a project without the declared dependency, I report it. Setuptools is working toward being a build backend and nothing more, which is why pkg_resources is deprecated and why we've put so much effort into creating replacements for it. The CPython and PyPA teams are working to limit places where Setuptools is available implicitly such that it will need to be declared explicitly where used.

I actually wasn't planning on deprecating pkg_resources, except that I encountered projects that were resistant to migrating away from pkg_resources until it was formally deprecated (even though someone had done the work to plumb in the replacements).

---

I agree with Anderson's assessment.

Deprecations are not meant to be breaking changes, aren't breaking changes for most users, and so shouldn't be advertised as such. I've not seen any projects where DeprecationWarnings were treated as breaking changes.

If there were substantial consensus in the community that DeprecationWarnings should be treated as breaking changes, I would surely adopt that across the projects I maintain, but until then, Setuptools will continue to follow the current common best practice.

Thanks for understanding.

[GergelyKalmar]

Thank you for the explanations! What both of you are saying makes sense, and of course, it is entirely your call, I'm merely providing end user feedback.

It seems to me that the whole warnings mechanism is just not an adequate approach to dealing with deprecations, after all, I really should not have to care about deprecations in indirect dependencies, regardless if they are merely polluting my test output or actually breaking them.

As the Python warning mechanism doesn't seem to allow us to restrict the warnings to direct dependencies, therefore breaking encapsulation and separation of concerns, the only sane approach seems to be filtering out deprecation warnings entirely. Of course, that kind of kills the point of having deprecation warnings in the first place 🤷, but I guess semver is a much better solution to this problem anyways.