Remove usage of setup.py in packaging configurations

[freakboy3742]

What is the problem or limitation you are having?

The various Toga subprojects all use setup.py; this configuration method has been deprecated, and is now causing complications with other tooling. We should complete the removal of setup.py from packaging configurations.

Describe the solution you'd like

The use of setup.py should be replaced in favor of a "pure" setup.cfg configuration.

The complication is that the dependencies for platforms have a version requirement that is dynamic - toga-cocoa vX depends on toga-core vX - and I'm not aware of any way to define dynamic version dependencies.

We could drop this as a requirement, and just rely on the end-user pinning versions appropriately. This would have the benefit of:

1. Silencing the errors we see in Briefcase when the venv has one version of Toga installed, but the pyproject.toml specifies a different version. This is particularly common when the local version is a development version.
2. Deliberately allowing for toga-core to be updated when a toga-backend isn't. It's unclear if this would be a good idea, but it would at least be possible.

Alternatively, we could introduce a runtime check to validate that the toga-core version is equivalent to the toga-backend version at the time the factory is constructed.

Describe alternatives you've considered

We could move to a pure pyproject.toml configuration, removing the need for setup.cfg as well. This would be a bigger migration. The use of pyproject.toml for setuptools and setuptools_scm configuration has only recently moved out of beta status, so it's not clear if we'll hit problems in the migration; and some of our tools tools aren't compatible with pyproject.toml configuration (most notably, flake8).

Alternatively, we could move to a completely different build tool (hatch would seem the most likely candidate).

Additional context

No response

[mhsmith]

For dependabot this only affects the dev requirements of toga-core. Since toga-core doesn't have the issue with matching versions, its setup.py could be removed right now, assuming setuptools-scm can be migrated.

Silencing the errors we see in Briefcase when the venv has one version of Toga installed, but the pyproject.toml specifies a different version.

Just linking to the relevant Briefcase issue:

• False alarms from pip's dependency resolver caused by conflicts with local packages briefcase#768

[freakboy3742]

Via Mastodon: this suggestion provides a fairly simple and elegant way to fix the dynamic dependency problem.

[ofek]

Maintainer of Hatch and Hatchling here! Let me know if I can be of assistance 🙂

[freakboy3742]

@ofek Thanks, as the closed ticket indicates, we've already been able to resolve this issue.

However, if Hatch has a built-in solution for the dynamic dependency problem, it might be worth exploring porting from setuptools.

[ofek]

Yes indeed we do! In fact that is one of the core value propositions of the Hatch ecosystem.

What you're looking for is to implement a metadata hook. You can implement one in a reusable package as you have done for your setuptools plugin, however if that functionality is just for a single project you can instead have a custom, local hook similar to this example from Twisted: https://github.com/twisted/twisted/blob/trunk/hatch_build.py

[freakboy3742]

So... unless I'm missing something, what you're saying isn't "yes", but "the thing you've done with setuptools can also be done with Hatch". It isn't a built-in feature - it's a feature you can add with a custom plugin. Which is great... but isn't, in and of itself, a compelling reason to spend the effort to switch to Hatch.

And - to be brutally frank, that's pretty much my reaction every time Hatch comes up - why does this exist?

Based on the Hatch homepage, the benefits of Hatch are that it is:

• "Modern". This isn't a compelling argument to me - I like old and boring, because that usually also means "well understood and widely tested"; and I've been around in software long enough to see "modern" things become legacy things that are replaced by the next "modern" thing several times over.
• "Extensible". This would be a compelling argument... except that setuptools is also extensible (or, at least, sufficiently extensible for my purposes)
• "Reproducible builds" and "environment management". These are both features of build AFAICT.
• "Easy publication". I'm not sure what is hard about twine upload.
• "Configurable project generation" might be compelling if this were a new project... but it isn't.
• "2-3 times faster". That might be true, but I can't recall ever being concerned by the speed of setuptools.

So - from the perspective of this (and related) projects, I genuinely don't see what we gain by spending the effort to switch to Hatch.

Don't get me wrong - I won't argue that setuptools is perfect. It has a bunch of annoying legacy and some dead bodies buried in the codebase... but that's the nature of any old software project. And, with the adoption of PEP517, PEP621 et al, it's now possible to almost entirely avoid most of the gnarly edges that historically existed. I'm not a die-hard setuptools fan who will use it to my dying day... but why should I abandon a tool that I know and understand for a new tool? What will be able to do with Hatch that we can't currently do because we're using setuptools? Or, what reason do I have to believe that Setuptools is dying, and Hatch is the life raft?

I really don't like laying into someone else's project like this - many of us are volunteers, and I honestly do believe that people start new projects because they see value or potential for improvement. (Also - I'll point out that you came to us, commenting on a ticket that was closed over a week ago, fixed to the satisfaction of the project with no hacky workarounds or off-label usage).

What I'm drawing to your attention is that the Hatch project has completely failed (IMHO) to communicate Hatch's value proposition.

The only compelling argument I'm aware of in favor of Hatch at present is that the PyPA packaging tutorial defaults to Hatch. I'm on record about being perplexed about this decision. I also note that in your own response to that thread, you indicate that I shouldn't read anything into Hatch being a default... but to me, that massively undersells the significance of defaults.

In your response, you also reference an upcoming document detailing why I have no idea if that document ever emerged, but if it did, didn't hit my personal radar. If it does exist, I'd suggest it should be prominently linked from the Hatch homepage, because for me, "why hatch and not setuptools" is a big missing piece of the puzzle.

[ofek]

I'll point out that you came to us, commenting on a ticket that was closed over a week ago, fixed to the satisfaction of the project with no hacky workarounds or off-label usage

I did come here but as a result of your comment in the OP: "we could move to a completely different build tool (hatch would seem the most likely candidate)"

I try to search for keywords on issues every few weeks to see if I can help folks.

In your response, you also reference an upcoming document detailing why I have no idea if that document ever emerged, but if it did, didn't hit my personal radar. If it does exist, I'd suggest it should be prominently linked from the Hatch homepage, because for me, "why hatch and not setuptools" is a big missing piece of the puzzle.

Thank you for your feedback! I de-prioritized that page due to time constraints (I was focusing on features) but I will do that tonight before I cut the next release.

"Reproducible builds" and "environment management". These are both features of build AFAICT.

This is very far off actually. The tool you mention is for executing builds which then triggers the logic of a build backend which is the thing that actually produces the archive(s). Hatchling produces reproducible builds for wheels and source distributions whereas setuptools just does the former (although I don't know that it guarantees reproducibility for wheels for sure).

By your environment management comment, I think I understand the ultimate confusion here and that is the fact that Hatch is a CLI similar to Cargo or NPM while Hatchling is the sister project only in charge of builds and is comparable to setuptools and flit.

You can mix and match features because the defaults are all based on standards e.g. using just the build backend.

Environment management refers to tox-like functionality that is built-in to Hatch.

The main benefit is that, if a user wishes, they can get rid of the majority of their tooling (tox/nox, twine, bumpversion/versioneer, pyenv, etc.) by switching to Hatch.

[freakboy3742]

I'll point out that you came to us, commenting on a ticket that was closed over a week ago, fixed to the satisfaction of the project with no hacky workarounds or off-label usage

I did come here but as a result of your comment in the OP: "we could move to a completely different build tool (hatch would seem the most likely candidate)"

I try to search for keywords on issues every few weeks to see if I can help folks.

Ah - that at least explains why you've joined us :-)

FWIW: The "maybe Hatch?" comment was presented for completeness as one possible option, if we couldn't do what we needed with setuptools. As it turns out, we could do what we needed to do, so the move to Hatch wasn't required.

"Reproducible builds" and "environment management". These are both features of build AFAICT.

This is very far off actually. The tool you mention is for executing builds which then triggers the logic of a build backend which is the thing that actually produces the archive(s). Hatchling produces reproducible builds for wheels and source distributions whereas setuptools just does the former (although I don't know that it guarantees reproducibility for wheels for sure).

I'm not sure I understand how a source distribution can't be reproducible... it's a tarball of files. What can't be reliably reproduced?

By your environment management comment, I think I understand the ultimate confusion here and that is the fact that Hatch is a CLI similar to Cargo or NPM while Hatchling is the sister project only in charge of builds and is comparable to setuptools and flit.

That is firmly in the category of "things that aren't even remotely explained on the homepage". :-) The PyPI page for hatchling points at the same landing page as hatch, and the only mention of hatchling I can find is the "history" mention, which gives the release history of 2 independent tools, but doesn't explain how they relate. My read of the information available was that hatchling was the the tool used to package hatch, because of some bootstrapping limitation.

Environment management refers to tox-like functionality that is built-in to Hatch.

The main benefit is that, if a user wishes, they can get rid of the majority of their tooling (tox/nox, twine, bumpversion/versioneer, pyenv, etc.) by switching to Hatch.

Hrm. I'll have to take a closer look then. The "replace all your tooling" aspect definitely missed my radar.

[ofek]

I'm not sure I understand how a source distribution can't be reproducible... it's a tarball of files. What can't be reliably reproduced?

Bits that represent file permissions and timestamps, normalization of new line characters for metadata, etc. there is a lot that goes into it tbh

[ofek]

The first part of the value proposition page is complete https://hatch.pypa.io/dev/why/

I'm going to add other sections to that page tonight.

[ofek]

I added another section about environment management. I'll add more at a later date because I really need to dedicate my free time to preparing this next release. Thank you again for your feedback!