enable LTO for chromium #127
Comments
|
I've been building successfully with USE custom-cflags, which should disable the flag stripping, although I haven't verified that LTO is being enabled |
|
Anyone willing to do a PR? I don't use Chromium myself and I remember it took forever to compile. |
|
@InBetweenNames is there a way to confirm a binary has been LTO'd or do you look at the build logs? |
|
I just look at the build logs, however if there are any static libraries, you can The |
|
but if they're unstripped and you're lucky you can find LTO function fragments. |
alternatively, simply:
|
|
OK, I finally built it successfully. this does work. note that it significantly increases CPU, RAM, and disk usage during build. it (very very approximately) doubles each of these. |
|
@Hello71, were you able to just use |
|
I didn't use custom-cflags, I used the gn options. I assume this is better supported upstream. |
|
Ye I opened a bug report about that sever months ago because I noticed that in benchmark Chromium compiled in gentoo was slower than binary Chrome or Arch linux Chromium. I also tried to compile Chromium with GCC using fedora's patches but it was even slower in benchmarks. |
|
@funghetto How much slower, and what was slower, JS? I only compile Chromium with GCC and have had no issue with performance with custom cflags. |
|
I've used mainly Speedometer (https://browserbench.org/Speedometer2.0/) and there was like a 15%-20% performance drop there. |
|
USE=custom-cflags compilation works for me by changing the compiler to clang, linker to lld and changing -flto=n (which isn't a valid option in clang) to -flto. Is that something to make a PR for, or do we avoid replacing gcc with clang (since I haven't seen this as something suggested anywhere nor noticed such workarounds)? @InBetweenNames |
|
While it is not about Chromium but Firefox, there is a great blog post of a GCC developer with some interesting data points on the Clang vs. GCC debate: http://hubicka.blogspot.com/2018/12/firefox-64-built-with-gcc-and-clang.html |
For Firefox at this point I've given up and switched to clang+lto+pgo mostly because it's more tested (being default). Firefox source code has a decent amount of ambiguous code that easily breaks with many optimizations from one version to the next and it's often subtle issues rather than a segfault, and most are long-standing known issues that aren't getting fixed anytime soon, only gcc 6 is officially supported too. Shame the ebuild doesn't have a flag for pgo right now though, but it's just about setting Even if gcc+lto+pgo performs better, I'm not sure I want to deal with this anymore unless it gets some official support effort. |
|
In the short term, you can use his binary instead. :)
|
|
I've managed to builf Chromium with LTO |
|
with clang? or gcc ? @perfect7gentleman |
|
GCC. |
|
good thanks @perfect7gentleman for the info |
|
I've used some Dedian and OpenSUSE patches |
|
|
On the other hand, I can't compile chromium with gcc 9 and lto: |
|
the problem was actually -fipa-pta so I disabled and tried to compile without it. |
|
Hmm... That's interesting. I have never built Chrome(ium) though, since LibreOffice and QtWebengine take long enough already and my primary browser is Firefox Nightly. |
|
Thing is tests are more difficult due to the big nature of this package. A single test can take 1-2 days if you compile with a low powered machine. Since this error does not give any clue on what is going wrong, I'm now trying to compile with USE="-custom-cflags". |
I could test it for you, it takes hour or so to build but I gave up on compiling with gcc. Chromium uses clang specific optimizations/patches which result in significant performance differences [ 20%~ ] |
Can you demonstrate that? If that's true, we don't we force chromium to use clang as compiler? |
Chrome/ium uses clangs -fwhole-program-vtables for performance boost amongst other things, As to why it isn't used? Nobody cares about compiling chromium from source with the exception of niche Gentoo user. Gentoo isn't particularly bleeding edge at its core nor it cares about chromium, clang that much. There are talks but it moves at snail's pace I've seen some ebuilds in overlays with clang as an option but most of them disappeared recently, chromium is a fast moving target and stuff around it adapts slowly Firefox upstream is also compiled with clang, just FYI, and clang version had double rendering performance for some time [ which was fixed/patched ] |
After reading about this I decided to give clang another try. I am able to build chromium fine with GCC 9.2 using LTO, but when I attempt to use clang, it fails very early in the build process, and I haven't been able to figure out why. Here is the error I get: I tried enabling The changes I made to the portage environment since successfully building with gcc are as follows:
|
|
Sorry to be a bother, but this isn't proving simple to debug, given Chromium takes several (seeeeeeveral) hours to build. Does anyone else have this situation, or am I doing something unusual? |
|
i can no longer build with USE=custom-cflags due to a failure with the bundled ANGLE third-party library. without I'm building/running chromium successfully, tho it's not my main browser. Hulu wasn't working with the previous release, but seems to have corrected itself with the latest. |
|
I've reproduced this failure with Chromium 83.0.4103.61. I'm beginning to suspect the problem is caused by a dependency of Chromium, but I'm not sure which. |
|
Building with stock gentooLTO settings and custom-flags fails for me as well, but I have successfully built chromium-85.0.4183.83 with custom-cflags and an overrides file in /etc/portage/env using the template provided by @perfect7gentleman #127 (comment) I plan on putting together an ebuild to apply those changes to all my gentoo boxes, and I'd like to see if anyone else has success with them. @InBetweenNames, if we can get a consensus on what the bare minimum is for LTO to work and get it tested widely enough to be reliable, would a PR to update sys-config/ltoize with them be welcomed? |
|
@dallenwilson If you could link me to a file that I can drop into /etc/portage/ somewhere, I'd be happy to test for you. Note, however, that I'm still using the most recently stabilized GCC, which is 9.3 as of today. I don't use unstable compilers. |
|
@jonesmz Sorry for the delayed response. I'll make it available once I'm satisfied that it is likely to work for someone else. I have three Gentoo boxes to work with, two different generations of AMD and one Intel, all with somewhat different USE flags and such. Once I get it the various options pruned down to a common set that build on all of them, I'll share it. |
|
@dallenwilson Unexpectedly, chromium now launches... but of course there have to be problems. Clipboard doesn't work :-P |
|
I ended up testing quite a few permutations of gentooLTO's default CFLAGS. I finally got chromium to build with custom-cflags set on all my machines without modifying anything at all, except for setting The error I'd been getting ( |
|
Always dies here for me when I try what @dallenwilson suggested (using gcc): What's also weird is that it doesn't compile with clang anymore for me either, so something's changed in chromium-86.0. |
you cropped out the important line of this error, but you might fix it by disabling --as-needed. |
|
Mhh I never got the lto build to work without errors, so I went with default flags now, it seems like the requirements for a standard build is higher now than before. I ran out of memory with -j4 on a a system with 8Gb of RAM, So I suspect an lto build will require even more. |
|
I was able to get chrome to build once with gcc and this repository's lto configuration. All other attempts ended with errors like @shelterx and @dallenwilson. I don't know how I got it to build, I think it was by sheer dumb luck. However, that build was extremely buggy. It would pass Browser Speedometer 2.0 tests at an average of 12 runs per minute. I was able to get www-client/chromium-87.0.4280.40::gentoo to build with the following env set up. /etc/portage/package.env/chromium.conf: This build is greatly out preforming the stock gcc builds I had which averaged 75-80 runs/min on Browser Speedometer 2.0. Build 87.0.4280.40 is currently posting 125+ runs/min consistently. |
@dallenwilson Do you have any update on this? I'm facing the same problem.
Are you sure about the fail reason? I've tried to increase number of allowed user processes via |
|
I have to build with -j2 if I use gcc, I gave up on clang but I might try the settings @fcsager posted. It's weird tho', jumbo builds when they were supported used quite a lot of more memory but I never ran OOM. |
Is it just |
chromium.conf env file: package.use: I also use ccache for chromium but it doesn't seem to help much. |
|
@fufler I went to |
I had to remove chrome_pgo_phase=2 but it seems to be compiling now, running -j4 and no signs of OOM or other errors yet. EDIT: |
|
Just a reminder, emerge --info will not show you what real flags were used. |
Not sure what I've done apart from using the above settings but Chromium compiles with -march=native for me. (I don't use EXTRA_GN="x64_arch="native""`) |
|
@shelterx How do you know? emerge --info told you? |
|
@barolo i see the -march=native being used flag on the source files when i compile chromium... |
|
Is there a consensus on how to properly build chromium with lto-overlay in place? What appears to be working correctly (albeit build is still in-progress) is the following with no changes to default lto-overlay flags and without enabling custom-cflags: |
|
Note the -j3 is due to only <6 GB ram with about the same amount of swap. |
I have 16 GB and I'm tired of compiling chromium, it is till swapping... |
|
Actually that ^^ did work, but i had to set gcc 10.x for the right cxx libs. |
|
Updated ungoogled to latest, it took 8h~ to compile, entirelly in RAM on 4 core 4Ghz CPU with clang and full set of optimizations. |
|
Also, chrome-bin is quite a bit faster than my chromium build with all optimizations turned on, at least in rendering benches, with identical gpu info in both. 640 points for chrome and 500 for chromium in Motionmark 1.2 for example. |
Maybe it's because chrome-bin uses PGO? |
|
Yes it does, but as I said, my build had all optimizations—including PGO—turned on. |
and others
it is enabled for official builds upstream since 2016, but Gentoo does not use it, even if -flto is specified.
The text was updated successfully, but these errors were encountered: