You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I investigate PGO and PLO effects on different kinds of software - all my current results are available at https://github.com/zamazan4ik/awesome-pgo. According to the results, enabling PGO and PLO can help with achieving better overall performance in many cases. I think expanding PGO and PLO usage for the Conan packages would be a good idea since it could help to achieve better software performance for Conan users.
Profile-Guided Optimization (PGO)
PGO is already a well-known technique. All currently known PGO effects on performance can be found at https://github.com/zamazan4ik/awesome-pgo#pgo-showcases . Several OS distros and package manager already enabled PGO for packages like GCC, Clang, Rustc, and many others (it depends on each package manager, of course). For some projects PGO is already enabled in Conan recipes - like CPython.
I think we can start by enabling PGO at least for the following projects (these packages are already PGO-optimized in many OS distros):
So at least for the projects above LLVM BOLT effects are tested and some preparations are already done in the upstream projects. In this case, it should be easier to enable BOLT for these packages.
For some projects right now there is ongoing work on integrating LLVM BOLT into the build scripts:
Some OS already using LLVM BOLT in their build scripts - check Clang recipe in Solus.
Usually, PGO and PLO are mostly used for executable packages, not for libraries. However, PGO and PLO can be used for distributed libraries too - e.g. check pydantic-corecase. We can think about PGO and PLO integration approaches into Conan as well. Imagine the case when a user uses libraries from Conan and wants to optimize them all according to their profile with PGO. How can we implement this case right now with Conan? Do we need an additional integration in Conan like it's done in Bazel?
First, I want to discuss these optimization approaches with Conan maintainers. What do you think about the ideas? If these ideas would be valuable for Conan, then we can think about what to do next: create per-package corresponding PGO/PLO issues, create meta-tracking issues, etc.
Thank you for your attention.
The text was updated successfully, but these errors were encountered:
But regarding LTO or PGO and conancenter, this idea is to build portable binaries by default (binaries built with LTO are not portable for example, at least for msvc) in a consistent way in the all dependency graph, and allow user to build binaries with LTO (no idea how it would work with PGO, is there even a standard way to enable externally this thing in CMake for example without modifying build files?) if they want by providing appropriate flags in their profile.
It depends on how we interpret the issue. Because I am talking about enabling PGO for Conan packages. Since PGO cannot be enabled easily for all of them - it probably needs to be done in a per-package way, so some package recipe changes are required.
and allow user to build binaries with LTO (no idea how it would work with PGO, is there even a standard way to enable externally this thing in CMake for example without modifying build files?) if they want by providing appropriate flags in their profile.
If we are talking about enabling PGO via a semi-standard build option, as far as I know only Bazel and Meson have such options. For CMake, there is only a request.
As an example of how PGO is enabled right now in various projects:
What is your question?
Hi!
I investigate PGO and PLO effects on different kinds of software - all my current results are available at https://github.com/zamazan4ik/awesome-pgo. According to the results, enabling PGO and PLO can help with achieving better overall performance in many cases. I think expanding PGO and PLO usage for the Conan packages would be a good idea since it could help to achieve better software performance for Conan users.
Profile-Guided Optimization (PGO)
PGO is already a well-known technique. All currently known PGO effects on performance can be found at https://github.com/zamazan4ik/awesome-pgo#pgo-showcases . Several OS distros and package manager already enabled PGO for packages like GCC, Clang, Rustc, and many others (it depends on each package manager, of course). For some projects PGO is already enabled in Conan recipes - like CPython.
I think we can start by enabling PGO at least for the following projects (these packages are already PGO-optimized in many OS distros):
Probably there are more PGO-suitable packages in the Conan repository. All found right now PGO results (with performance numbers) can be checked here - https://github.com/zamazan4ik/awesome-pgo#pgo-showcases
Post Link Optimization (PLO)
Regarding Post Link Optimization (PLO), right now there are two main tools - LLVM BOLT and Google Propeller.
According to the Facebook Research Paper (https://research.facebook.com/publications/bolt-a-practical-binary-optimizer-for-data-centers-and-beyond/), LLVM BOLT (https://github.com/llvm/llvm-project/blob/main/bolt/README.md) helps with achieving better performance for various packages like compilers and interpreters. I think it would be a good idea to enable LLVM BOLT for some packages to deliver faster binaries for users (since Propeller is less stable right now in my opinion).
Here I got some examples of how LLVM BOLT is already integrated into other projects:
So at least for the projects above LLVM BOLT effects are tested and some preparations are already done in the upstream projects. In this case, it should be easier to enable BOLT for these packages.
For some projects right now there is ongoing work on integrating LLVM BOLT into the build scripts:
More about LLVM BOLT performance results for other projects can be found in:
librustc_driver.so
with BOLT rust-lang/rust#116352Some OS already using LLVM BOLT in their build scripts - check Clang recipe in Solus.
Usually, PGO and PLO are mostly used for executable packages, not for libraries. However, PGO and PLO can be used for distributed libraries too - e.g. check
pydantic-core
case. We can think about PGO and PLO integration approaches into Conan as well. Imagine the case when a user uses libraries from Conan and wants to optimize them all according to their profile with PGO. How can we implement this case right now with Conan? Do we need an additional integration in Conan like it's done in Bazel?First, I want to discuss these optimization approaches with Conan maintainers. What do you think about the ideas? If these ideas would be valuable for Conan, then we can think about what to do next: create per-package corresponding PGO/PLO issues, create meta-tracking issues, etc.
Thank you for your attention.
The text was updated successfully, but these errors were encountered: