Profile-Guided Optimization (PGO) benchmark report

[zamazan4ik]

Hi!

As I have done many times before, I decided to test the Profile-Guided Optimization (PGO) technique to optimize the library performance. For reference, results for other projects are available at https://github.com/zamazan4ik/awesome-pgo . Since PGO helped a lot for many libraries, I decided to apply it to cosmic-text to see if the performance win (or loss) can be achieved. Here are my benchmark results.

Test environment

• Fedora 40
• Linux kernel 6.10.12
• AMD Ryzen 9 5900x
• 48 Gib RAM
• SSD Samsung 980 Pro 2 Tib
• Compiler - Rustc 1.81.0
• cosmic-text version: main branch on commit 4fe90bb6126c22f589b46768d7754d65ae300c5e
• Disabled Turbo boost

Benchmark

For benchmark purposes, I use built-in into the project benchmarks. For PGO optimization I use cargo-pgo tool. Release bench results I got with taskset -c 0 cargo bench command. The PGO training phase is done with taskset -c 0 cargo pgo bench, PGO optimization phase - with taskset -c 0 cargo pgo optimize bench.

taskset -c 0 is used to reduce the OS scheduler's influence on the results. All measurements are done on the same machine, with the same background "noise" (as much as I can guarantee).

Results

I got the following results:

• Release: https://gist.github.com/zamazan4ik/3f188dbd2ddd51ca12716a0fe8c557d9
• PGO optimized compared to Release: https://gist.github.com/zamazan4ik/1a893ee4cc24c6ce9f9b61af0ca4a513
• (just for reference) PGO instrumented compared to Release: https://gist.github.com/zamazan4ik/c03e81a8fa4b1db532d68abf8bf59939

According to the results, we see constant improvements in the library's performance.

Further steps

At the very least, the library's users can find this performance report and decide to enable PGO for their applications if they care about cosmic-text performance in their workloads (maybe other Cosmic apps?). Maybe a small note somewhere in the documentation (the README file?) will be enough to raise awareness about this work. Another way - try to figure out the root cause of performance differences between PGO and non-PGO library versions, and, probably, try to tweak the library sources a bit more - however this way also requires some time to analyze the resulting LLVM IR/assembly differences between them.

Also, Post-Link Optimization (PLO) can be tested after PGO. It can be done by applying tools like LLVM BOLT to applications with' cosmic-text' apps. However, it's a much less mature optimization technique compared to PGO.

Thank you.