Use setuptools_scm for dynamic versioning

[hugovk]

Following on from #7616.

I had a go setting it up on the weekend, I'm going to pause it for now, but opening a draft to show how it looks. We don't necessarily need to finish this off, and can do it another way instead, if at all.

For non-tagged builds, setuptools_scm would give us a different version, with a .devX number appended based on the number of commits from the last tag. This allows us to upload intra-tag builds to TestPyPI, as [Test]PyPI doesn't allow the same sdist/wheel filenames to be re-uploaded.

[webknjaz]

@hugovk how about enabling versioning of Git archives for arbitrary commits? This would require adding 2 small files: https://setuptools-scm.readthedocs.io/en/latest/usage/#git-archives.

[hugovk]

What are Git archives, and what is their use case?

Does it help with publishing, or is it a nice-to-have extra when using setuptools_scm?

[webknjaz]

What are Git archives, and what is their use case?

It allows doing stuff like pip install https://github.com/python-pillow/Pillow/archive/refs/heads/main.tar.gz and getting the version generated correctly (instead of something like 0.0.1), when that commit is not tagged. Otherwise, setuptools-scm won't know where to get those tags from.

See also: https://docs.github.com/en/repositories/working-with-files/using-files/downloading-source-code-archives

Does it help with publishing, or is it a nice-to-have extra when using setuptools_scm?

It's a very nice to have, but is not required for publishing. FWIW it's like adding 5 lines of text to the Git repo — so it's a boilerplate I just have everywhere.

[webknjaz]

@hugovk so in the case of this PR, pip install https://github.com/hugovk/Pillow/archive/refs/heads/setuptools_scm.tar.gz doesn't work well, but with this, it would (provided that you also have the upstream tags in your fork, of course).

[webknjaz]

@hugovk I recommend adding a src/PIL/_version.pyi file so that the type checkers wouldn't be confused with a missing import. Example: https://github.com/sphinx-contrib/sphinxcontrib-towncrier/blob/750485e/src/sphinxcontrib/towncrier/_scm_version.pyi.

[radarhere]

The discussion to add that file began with sphinx-contrib/sphinxcontrib-towncrier#33 (comment), where you quoted an error. I'm guessing that error came from mypy?

[webknjaz]

Yes.

[radarhere]

Minor comment considering this is still in progress: I don't see why it is necessary to remove PILLOW_VERSION as a Python variable - we're still defining PILLOW_VERSION as the same value for C, so I don't see the change as creating any distinction in meaning.

[hugovk]

When doing it globally we get aFileNotFoundError: [Errno 2] No such file or directory: 'src/PIL/_version.py', I presume because when opening setup.py to begin the build, the build hasn't had a chance to generate the file yet.

Details

❯ pip install -e .
Found existing alias for "pip install -e .". You should use: "pie"
Obtaining file:///Users/hugo/github/Pillow
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... error
  error: subprocess-exited-with-error

  × Getting requirements to build editable did not run successfully.
  │ exit code: 1
  ╰─> [22 lines of output]
      Traceback (most recent call last):
        File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 132, in get_requires_for_build_editable
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/p6/lf2s1s5d4kb335g2n1td8z8c0000gn/T/pip-build-env-5qxfgm5i/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 441, in get_requires_for_build_editable
          return self.get_requires_for_build_wheel(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/p6/lf2s1s5d4kb335g2n1td8z8c0000gn/T/pip-build-env-5qxfgm5i/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/p6/lf2s1s5d4kb335g2n1td8z8c0000gn/T/pip-build-env-5qxfgm5i/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "/private/var/folders/p6/lf2s1s5d4kb335g2n1td8z8c0000gn/T/pip-build-env-5qxfgm5i/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 31, in <module>
        File "<string>", line 26, in get_version
      FileNotFoundError: [Errno 2] No such file or directory: 'src/PIL/_version.py'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build editable did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[hugovk]

This PR was a demo, I'm not planning on continuing with it, so closing :)

[aclark4life]

FWIW I did this without setuptools_scm recently or so it appears from the syntax. I was hoping that adding dynamic = ["version"] to the [project] would be enough to enable the feature and that setuptools would default to the project package and search for a set of version files in that package, but no it does not, hence the 🤦 in the commit message.