Using Profile-Guided Optimization (PGO) for C/C++/Rust/Fortran-based packages

[zamazan4ik]

Profile-Guided Optimization (PGO) is a compiler optimization technique to improve compiled binaries performance based on how these binaries are executed in runtime (which code parts are executed, how frequently, etc.). This information allows a compiler to make better optimization decisions during the compilation process (one of the most important - much better inlining decisions). All major C/C++/Rust/Fortran compilers support PGO.

I suggest using PGO to increase the performance of native-based Python packages.

Several existing PGO examples for Python packages:

• Pydantic: add build-pgo make target pydantic/pydantic-core#741
• Polars: Build Python polars wheels with PGO pola-rs/polars#9702
• Python-libipld: Test PGO on x86_64 ubuntu MarshalX/python-libipld#21

If you want to read more about PGO, I can suggest my repo: https://github.com/zamazan4ik/awesome-pgo . There I collect various benchmarks about PGO, and share my thoughts and guides about it. You also could be interested in the article about PGO and Post-Link Optimization (PLO) (unfortunately, unfinished yet but still should be helpful): https://github.com/zamazan4ik/awesome-pgo/blob/main/article/article.md