Chore: use Hatch build backend

[agoose77]

Description

Setuptools as a build backend is getting fairly long in the tooth. A few consequences of this are:

• deprecated APIs
• fragmentation of configuration (setup.py, setup.cfg, MANIFEST.in)
• additional configuration files (besides pyproject.toml)

The new selection of PEP517 builders means that we can simplify our build backend, whilst also making things future-proof!

TODO:

• ensure wheel contents match (waiting on Allow multiple inclusion of symlinks pypa/hatch#276)

Checklist Before Requesting Reviewer

• Tests are passing
• "WIP" removed from the title of the pull request
• Selected an Assignee for the PR to be responsible for the log summary

Before Merging

For the PR Assignees:

• Summarize commit messages into a comprehensive review of the PR

[matthewfeickert]

I'm not sure I have any interest in using hatch for library development. For applications sure, but I'm not sold on switching over to it for library work yet.

[matthewfeickert]

I'm going to close this as I don't think this is PR ready before a discussion is had.

[agoose77]

OK.

My thought process here was to create a draft PR that moves the metadata into the standardised PEP 621 format. Hatch is only one of several build backends that support this new format (e.g. flit, pdm), so one isn't limited to Hatch.

For posterity, this really just changes what happens when you run pip install or pyproject-build. The developer consequences are (intentionally) minimal.

[ofek]

@matthewfeickert Hey, Hatch maintainer here!

I'm not sure I have any interest in using hatch for library development. For applications sure, but I'm not sold on switching over to it for library work yet.

That's very interesting; what makes you say that? Everyone else says the opposite due to not supporting lock files currently.

[matthewfeickert]

That's very interesting; what makes you say that? Everyone else says the opposite due to not supporting lock files currently.

@ofek sorry — low sleep and reading fast when I sent that — I mentally got hatch and pdm confused. 🙁

Congrats on having hatch be the default example backend for the PyPA's Packaging Python Projects tutorial!

I'll follow pypa/hatch#276 and once that is revised @agoose77 can open a GitHub Issue in pyhf and we can take a look at switching things over after we discuss some things (I also need to look at what is possible in pyproject.toml RE: dealing with extras that bundle extras without a bunch of .bloat)

[ofek]

This will fix builds for you https://hatch.pypa.io/latest/config/build/#explicit-selection

[matthewfeickert]

@agoose77 can I charge you with reminding me to look into reopening this next week?

[matthewfeickert]

Actually, a follow up question: After our Slack discussion today I'm trying to poke around a bit myself (branch build/migrate-to-hatch which is a debug branch before we go back to here) to understand some of the issues that I had before better.

I notice after I build a sdist and wheel that the sdist has a .gitignore file in it for some reason.

$ rm -rf dist && python -m build . && python -m tarfile --list dist/pyhf*.tar.gz | grep .gitignore
* Creating virtualenv isolated environment...
* Installing packages in isolated environment... (hatch-vcs, hatchling)
* Getting build dependencies for sdist...
* Building sdist...
* Building wheel from sdist
* Creating virtualenv isolated environment...
* Installing packages in isolated environment... (hatch-vcs, hatchling)
* Getting build dependencies for wheel...
* Building wheel...
Successfully built pyhf-0.7.1.dev46+g174da491.d20230118.tar.gz and pyhf-0.7.1.dev46+g174da491.d20230118-py3-none-any.whl
pyhf-0.7.1.dev46+g174da491.d20230118/.gitignore

the sdist is not a Git repository, so why is this file included here if I'm explictily excluding it in pyproject.toml?

# ...

[tool.hatch]
version.source = "vcs"
build.hooks.vcs.version-file = "src/pyhf/_version.py"

[tool.hatch.build.targets.sdist]
exclude = [
    "/.gitignore",
    "/src/conftest.py",
]
include = [
    "/src",
    "/LICENSE",
    "/README.rst",
    "/pyproject.toml",
    "/AUTHORS",
    "/CITATION.cff"
]
[tool.hatch.build.targets.wheel]
packages = ["src/pyhf"]

# ...

[ofek]

https://hatch.pypa.io/latest/plugins/builder/sdist/#default-file-selection

It is so builds are reproducible

[matthewfeickert]

https://hatch.pypa.io/latest/plugins/builder/sdist/#default-file-selection

It is so builds are reproducible

Hm okay thanks. I'll not ask you to explain why including this makes things reproducible (or rather why excluding it leads to unreproducible builds) as I will assume that is covered somewhere on GitHub if I dig enough. I'm still coming from the setuptools MANIFEST.in mindset where the sdist contains exactly the files listed and not additional things I haven't specified.

[ofek]

Oh sorry yes I should explain. What I mean is that builds can occur in different contexts and Hatchling tries its best to ensure the artifacts are consistent. So for example, if you build from a Git checkout you would get one set of files because the ignore file is available but when building from a source distribution you would likely get a different set of files without the extra exclusion rules.

[matthewfeickert]

Oh sorry yes I should explain.

Thanks that's quite generous of your time — I wasn't trying to be sarcastic, I just didn't think it was worth your time as I know you've put in huge amounts of effort to have good reason here.

So for example, if you build from a Git checkout you would get one set of files because the ignore file is available but when building from a source distribution you would likely get a different set of files without the extra exclusion rules.

Ah thanks very much for this nice example. I now understand that this really does mean a 1:1 matching of the VCS to the sdist in terms of build output then. I hadn't thought of that before, but that's slick. 🚀

[matthewfeickert]

@agoose77 I can't reopen this PR as master has been deleted when we switched to main. I'm going to open a new PR and we'll give you co-author credit on it (and probably ping you there to review it too. 🙂).