Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add pybind11 example #2324

Merged
merged 8 commits into from
Oct 22, 2024
Merged

Conversation

alvgaona
Copy link
Contributor

This is an example of using Pixi to compile pybind11 modules and make them available to any Python interpreter.

  1. It uses CMake to compile the C++ code.
  2. scikit-build-core is the tool used in the example to generate the .whl files.
  3. It leverages the build PyPI package to run the build task.

@alvgaona alvgaona marked this pull request as draft October 21, 2024 00:26
@alvgaona alvgaona marked this pull request as ready for review October 21, 2024 01:24
@alvgaona alvgaona changed the title Add pybind11 example docs: add pybind11 example Oct 21, 2024
Copy link
Contributor

@Hofer-Julian Hofer-Julian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your PR!

The build took 3 minutes on my machine, which feels a bit long for 10 lines of C++ code. I haven't used pybind11 before, so I don't know if that is normal.

examples/pybind11/pyproject.toml Outdated Show resolved Hide resolved
@alvgaona
Copy link
Contributor Author

alvgaona commented Oct 21, 2024

Thank you for your PR!

The build took 3 minutes on my machine, which feels a bit long for 10 lines of C++ code. I haven't used pybind11 before, so I don't know if that is normal.

@Hofer-Julian I'm not sure why that is. I think it's mainly due to scikit-build-core doing things 🫠 Happens on my Intel-based PC too.

@Hofer-Julian
Copy link
Contributor

Maybe someone with more C++ experience like @baszalmstra or @wolfv could check it out as well

@alvgaona
Copy link
Contributor Author

alvgaona commented Oct 21, 2024

The composed command took 6.604s when I built it with CMake directly. So I'm pretty sure it's scikit-build-core.

Screenshot 2024-10-21T15 42 44@2x

@alvgaona
Copy link
Contributor Author

alvgaona commented Oct 21, 2024

2m 33.07s when running the full task on osx-64.

Screenshot 2024-10-21T15 46 43@2x

@alvgaona
Copy link
Contributor Author

alvgaona commented Oct 21, 2024

~17s with the --wheel flag. The task that takes too long is when it is ran with with the --sdist flag, which I think it's default.

By default, a source distribution (sdist) is built from {srcdir}
and a binary distribution (wheel) is built from the sdist.
This is recommended as it will ensure the sdist can be used
to build wheels.

Pass -s/--sdist and/or -w/--wheel to build a specific distribution.
If you do this, the default behavior will be disabled, and all
artifacts will be built from {srcdir} (even if you combine
-w/--wheel with -s/--sdist, the wheel will be built from {srcdir}).

Screenshot 2024-10-21T15 49 14@2x

@Hofer-Julian
Copy link
Contributor

~17s with the --wheel flag. The task that takes too long is when it is ran with with the --sdist flag, which I think it's default.

Sounds to me like we should pass the --wheel flag then, since that's the interesting output in my opinion :)

@alvgaona
Copy link
Contributor Author

~17s with the --wheel flag. The task that takes too long is when it is ran with with the --sdist flag, which I think it's default.

Sounds to me like we should pass the --wheel flag then, since that's the interesting output in my opinion :)

Yeah, generally you'd like --sdist to share it via repositories. You can do whatever in this case.

@alvgaona alvgaona requested a review from Hofer-Julian October 21, 2024 21:45
@alvgaona
Copy link
Contributor Author

alvgaona commented Oct 22, 2024

@Hofer-Julian fixed it. The sdist was including .pixi in the tarballed package which was causing to take minutes to finish, and by excluding it, it takes ~34s.

Screenshot 2024-10-22T03 08 01@2x

Copy link
Contributor

@Hofer-Julian Hofer-Julian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Let's bring it in then!

@Hofer-Julian Hofer-Julian merged commit 9eb6e59 into prefix-dev:main Oct 22, 2024
17 checks passed
@alvgaona alvgaona deleted the pybind11-example branch October 22, 2024 13:58
@beenje
Copy link
Contributor

beenje commented Oct 23, 2024

Hi! Just saw this PR. It's a nice example.

I wanted to mention that we also use scikit-build-core in one of our project (with boost).
I created an install task that runs pip to build and install the package in editable mode as defined here. It's still marked as experimental support but has been working well for us.

pixi run install builds and install the package.
It allows to work on the pure python part without re-installing. If you touch cpp files, you have to re-run the command.

You might find that useful.
I think it's more convenient during development than pixi run build + pip install ... that you have to re-run every time.
After it might depend on what you want and if your project is pure cpp or include python as well.

See https://gitlab.com/tango-controls/pytango/-/blob/develop/pixi.toml?ref_type=heads#L18

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants