Migrate to pyproject.toml #369
Description
Following PEP 518 and PEP 621, we should migrate the older setup.cfg and setup.py build system to one making use of pyproject.toml. Furthermore, the use of setup.py is strongly discouraged and many aspects will be discontinued:
Setuptools offers first class support for
setup.pyfiles as a configuration mechanism.It is important to remember, however, that running this file as a script (e.g.
python setup.py sdist) is strongly discouraged, and that the majority of the command line interfaces are (or will be) deprecated (e.g.python setup.py install,python setup.py bdist_wininst, …).We also recommend users to expose as much as possible configuration in a more declarative way via the pyproject.toml or setup.cfg, and keep the
setup.pyminimal with only the dynamic parts (or even omit it completely if applicable).See Why you shouldn’t invoke setup.py directly for more background.
The existing setup.py build script also makes use of distutils which has been deprecated since 3.10 and is scheduled to removal in 3.12.
As part of this change, we also have an opportunity to change the underlying build system that handles the packaging of the Python package into a distributable bundle. The main ones to consider are:
An excellent write up by Jay Qi is available in their blog post The Basic of Python Packaging in Early 2023.
As pact-python will eventually make use of the Pact Rust core, any build system considered must be able to support so-called 'extensions' (which are typically used to build C extensions).
Deep Dive
Setuptools
This by far the most popular build system and is the current build system being used. Extensions are supported through setup.py.
Note that even if we keep this, closing this issue will require an extensive rework of setup.py due to the aforementioned deprecations.
A good example I have come across for this implementation would be following playwright-python's setup.py.
Poetry
Poetry is a very popular Python tool that predates PEP 621. Unfortunately, it does not yet support PEP 621 (though it is on the roadmap) and instead uses its own custom metadata section in pyproject.toml.
More concerning is the unstable support of extensions (python-poetry/poetry#2740) which effectively rules out Poetry as the build tool.
Poetry may still be useful to manage dependencies, as it does an excellent job of ensuring all dependencies installed within the virtual environment are mutually compatible. Furthermore, it integrates with dependabot. This of course depends on whether the build tool can work with Poetry for dependency management.
Maturin
Maturin is a project by the PyO3 team which maintains an ecosystem of tools that provide Rust binding for Python.
This may be an excellent option given that pact-python will eventually need to integrate with the Rust core and may be easier to use than Setuptools.
There is a concern that integrating too much with Rust might dissuade contributions from Python developers, and we must also check whether we can still rely on Ruby while migrating to the Rust core.
Hatch
Hatch is build tool and front-end. It appears to have extensive support for customisations through build hooks. These hooks are likely able to support both the Ruby and Rust implementations simultaneously, but this would need to be investigated properly.
Flit
Flit is a lightweight build manager, but unfortunately does not support build extensions and therefore is not appropriate here.
PDM
PDM describes itself as a "modern Python package and dependency manager supporting the latest PEP standards". It was created as an implementation of PEP 582 which provides an venv-less way of managing dependencies in a similar way to npm. While PEP 582 was rejected, PDM still support this.
PDM appears to have excellent build and publishing capabilities, and the PDM backend allows for customisations through hooks pdm_build.py.
Metadata
Metadata
Assignees
Type
Projects
Status