-
-
Notifications
You must be signed in to change notification settings - Fork 187
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Parallel make doesn't work on a clean checkout #935
Comments
I'm currently running into this trying to build one of the x230 pull requests. -j isn't propagated to the subdirectories, and setting MAKE_JOBS breaks bootstrapping of the toolchain. Without anything set everything runs as -j1, though, which takes ages. At least some of that seems to have been introduced in the past 6 months - I remember having some initial issues to make it use parallel builds back in summer, but once I had that working it took about 5 minutes for a full build (including bootstrapping the tool chain). So far I've been sitting on this now for about 1 hour, and still didn't get past the tool chain bootstrapping. |
@bwachter documentation has been updated here, rendered here, to state that I'm still a bit confused here, since the main Makefile of Heads is taking the ouptut of nproc if not specified and populates CPUS variable which is then passed along if not defined on make initial call. That hack was implemented, because some modules won't like to have -j forced, where others play along. The idea here was to pass CPUS down in other makefiles (modules/*) where they play fair, and go single threaded for when modules don't play well. |
The issue with
versus
|
@osresearch The problem with -j24 is that there is no locking between interdependent tasks. Here, I launch the same build with CPUS=24(functional) and -j24(failing badly with files missing) over CircleCI. The line So the next task picks up created logs and outputs them on CI with delimitators: The resulting logs are concatenated here with ==> and <== separators. |
@osresearch : i'm redoing a build without decompressing your cache file, which failed on CircleCI for your latest commit, since host binaries were not found on fed config.status (sed, gawk and other host binaries were not found on provided paths). A CI build is happening here |
@osresearch seems like the missing culprit was in commenting MAKE_JOBS under global Mafefile on PR #984 (and deleting the configure cache which doesn't work under CircleCI). Then a weird race condition happens only under newt module build, which can be hacked by forcing that module to be built only with one job. Will do a replacement PR once working around new CircleCI limitations pass. |
@osresearch @bwachter : I was successful into fixing partly this issue in that commit: 5e4309c Where MAKE_JOBS was commented out into the main Makefile. |
…impact on the now working build system prior of mergin linuxboot#1035 for tracibility of linuxboot#935 and linuxboot#984
Tag me to reopen. CircleCI builds in parallel with 36 cores now under 45 minutes on clean checkout. |
There are some components that do not correctly depend on the cross compiler, so a make for the clean checkout fails. Easy fixes are the various tools in
$(COREBOOT_UTIL_DIR)
likecbmem
,superiotool
andinteltool
, which need to wait until the cross compiler is available.The text was updated successfully, but these errors were encountered: