Skip to content

How LTOize works

Shane Peelar edited this page May 21, 2019 · 1 revision

ltoize relies heavily on the package.cflags functionality from the app-portage/portage-bashrc-mv package. This extends the package.env functionality in Portage with a Bash-like syntax which is critical to making this work properly. Originally, we were using package.env overrides, but it turns out that the flag-o-matic.eclass used in ebuilds does not "see" flags the same way GCC does--the functions contained inside simply check for the presence of a particular string or pattern inside your *FLAGS variables and determines whether the flag is active based on that. However, in GCC, later flags override previous flags, and flags can also toggle other flags not listed. For example, CFLAGS=-O3 toggles -ftree-loop-distribution on GCC 8, but is-flagq -ftree-loop-distribution would return false as -ftree-loop-distribution is not listed in CFLAGS directly. Another example: if LDFLAGS=-flto -fno-lto, then is-ldflagq "-flto*" would return true despite that GCC would have -flto unset due to the later argument overriding it. The only real way to know what flags are active would be to pass in *FLAGS to GCC itself and then ask it what flags are active. Unfortunately, there are probably many packages that depend on the existing flag-o-matic.eclass behaviour, and so changing this is probably not an option. To try to work around this, we mandate that our *FLAGS variables contain no "redundant" flags. If the effect of a particular flag would be "undone" by a following flag, then that flag is considered "redundant". This doesn't solve the -O3 problem as listed above, but it should at least allow is-flagq to work in the cases we need it to (which is mainly for overriding -flto).

The actual /etc/portage modifications are in sys-config/ltoize/files. This is a stripped down version of my own Portage configuration which ltoize uses to install into your own /etc/portage. ltoize uses symlinks to accomplish this task so that when you do an emerge --sync or equivalent, you will automatically pull in the latest set of overrides. An eselect news entry will be made when a change is made to the default recommended LTO settings in make.conf.lto. That could be including some new compiler flags, or perhaps revising how LTO is done. Any such a change will require manual intervention if you are not using the default configuration. We'll do our best to ensure breaking changes are opt-in, rather than opt-out.

Not all packages build cleanly. Environment overrides are used to allow packages to build that have trouble with O3, Graphite, and LTO. These can be found in package.cflags/ltoworkarounds.conf. I have tried to categorize the overrides based on the kind of failure were being exhibited, but in some cases this was difficult. All optimization flag overrides are included in that file as well, but they won't affect you if you are not using those compiler flags, as potentially in the case of using a custom configuration.