[Proposal] Add Python packaging support for SDFormat

[sauk2]

Desired behavior

SDFormat currently provides Python bindings but no standard way to build and distribute them as Python wheels on PyPI. The proposal is to enable creating a Python wheel using standard packaging tools to allow users to install SDFormat's Python bindings easily through a pip install command.

Alternatives considered

Manual local builds and installs of Python bindings through CMake.

Implementation suggestion

• Add a pyproject.toml and specify scikit-build as build backend to automatically produce Python wheels by calling pip install .
• Add a minimal setup.py to provide metadata and to ensure backwards compatibility.
• Modify CMakeLists.txt in python directory if necessary to expose proper targets for scikit-build to use
• Test and validate against multiple Python versions

Additional context

• This enhancement will prepare the foundation for automating PyPI publishing though the existing Gazebo release pipeline (to be handled in a separate issue [Proposal] Add PyPI publishing support to the Gazebo release pipeline for libraries with Python bindings gazebo-tooling/release-tools#1318)
• Testing can be initially done on TestPyPI before publishing official releases on PyPI.

[traversaro]

fyi @flferretti @CarlottaSartore

[traversaro]

Superinteresting, thanks!

I think one problem to solve is how to handle the non-Python dependencies of sdformat, so I guess:

• urdfdom
• urdfdom-headers
• console-bridge
• gz-utils
• gz-cmake
• gz-math

Did you already considered how to handle this?

[flferretti]

fyi @xela-95

[sauk2]

Did you already considered how to handle this?

@traversaro Thanks for bringing this up!

There are two approaches to handling the non-Python dependencies. As a starting point, we could build the wheel assuming users already have the required C++ libraries installed on their system. This would keep the initial setup simple and would help validate the packaging workflow.

The next step would be to bundle all dependencies into the wheel using auditwheel. This would make the wheel fully self-contained and easier to install but would require more effort to set up and maintain.

Let me know what you think of this idea!

[traversaro]

As a starting point, we could build the wheel assuming users already have the required C++ libraries installed on their system. This would keep the initial setup simple and would help validate the packaging workflow.

This make sense, but personally I would avoid to upload such wheels to PyPI, as they would be broken for most users without the pip solver being aware of them being broken (see https://pypackaging-native.github.io/key-issues/native-dependencies/), so they would not be a good user experience (and you only get one chance of giving a first impression).

The next step would be to bundle all dependencies into the wheel using auditwheel. This would make the wheel fully self-contained and easier to install but would require more effort to set up and maintain.

This make sense! However, my question was how you plan to provide the C++ dependencies used in building the wheels and that are then repackaged by auditwheel. Assuming we use apt binaries, if I am not wrong the apt binaries for gz-* dependencies are available only on Ubuntu 24.04, I guess the resulting wheels will be only be compatible as manylinux_2_39, so they could be installed only on recent Linux distribution, is that correct right?

[sauk2]

This make sense, but personally I would avoid to upload such wheels to PyPI, as they would be broken for most users without the pip solver being aware of them being broken (see https://pypackaging-native.github.io/key-issues/native-dependencies/), so they would not be a good user experience (and you only get one chance of giving a first impression).

Thank for raising this, definitely want to avoid broken first impression. To refine my previous point, the first step would be to only adjust the current approach to building python bindings by supporting pip install ., assuming all the required C++ dependencies are already available on the system. Keeping the scope limited to local creation and internal testing and not publishing wheels to PyPI. Once the packaging and build process is stable, we can focus on bundling dependencies and distribution.

This make sense! However, my question was how you plan to provide the C++ dependencies used in building the wheels and that are then repackaged by auditwheel. Assuming we use apt binaries, if I am not wrong the apt binaries for gz-* dependencies are available only on Ubuntu 24.04, I guess the resulting wheels will be only be compatible as manylinux_2_39, so they could be installed only on recent Linux distribution, is that correct right?

You bring up a good point. However one thing to mention would be that the resulting wheel will also depend on the Gazebo collection target. Apt binaries are available for both Ubuntu 22.04 and 24.04. If we are aiming to support the latest version of SDFormat, building the wheel in 24.04 is a reasonable starting point and would produce a wheel tagged manylinux_2_39. The said, we can also try to support Ubuntu 22.04, in which case the wheel would be tagged as manylinux_2_35.

Hope I answered your questions!

[talregev]

I am in favor to do only the second step without the first. because on the first step no one will use it.
For second step to provide c++ libs for python to store inside the wheels files and load them from there, I would suggest vcpkg.
I already put gz-sim ports and all it dependencies, and it will compile in linux, osx and windows, vcpkg will be more easy then handle lib installation manually. vcpkg is ready to use and you can have the c++ libraries very quickly and we can try to build the wheels files for SDFormat and put it in the CI.
@traversaro Do you think it will work? You also familiar with vcpkg and gz-sim port.

[talregev]

@sauk2 if you want to explore:
https://github.com/microsoft/vcpkg
https://vcpkg.io/en/packages?query=sdformat