Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PGO] Provide Pre-Generated Profiles #127425

Open
ptr1337 opened this issue Nov 29, 2024 · 1 comment
Open

[PGO] Provide Pre-Generated Profiles #127425

ptr1337 opened this issue Nov 29, 2024 · 1 comment
Labels
build The build process and cross-build type-feature A feature request or enhancement

Comments

@ptr1337
Copy link

ptr1337 commented Nov 29, 2024

Feature or enhancement

PGO (Profile-Guided Optimization) offers significant performance benefits for Python. However, the current approach has a drawback: it makes the builds non-reproducible.

This issue could be resolved if upstream provided pre-generated profiles (perhaps via CI pipelines). These profiles could then be included in the distributed tarball, similar to how Chromium handles it. By doing so, distributions wouldn't need to:

  1. Compile the source code twice.
  2. Perform profiling themselves.
    In addition to streamlining the process for maintainers, this would also help maintain package reproducibility—a key consideration for many distributions.

I am not sure, about the integration with GCC, since in my testing the compilation failed with a preset profile (but I didnt dig deeper into this yet.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

@ptr1337 ptr1337 added the type-feature A feature request or enhancement label Nov 29, 2024
@picnixz picnixz added the build The build process and cross-build label Nov 29, 2024
@ptr1337
Copy link
Author

ptr1337 commented Nov 29, 2024

I have tested reusing the profile from a clang build, and this worked well.
I did not benchmark it yet - there were some profile mismatches, but I think this should be generally not that heavy.

Distributions like Fedora/Archlinux would be very interested in having this feature properly implemented.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build The build process and cross-build type-feature A feature request or enhancement
Projects
None yet
Development

No branches or pull requests

2 participants