Speedup CMake Configuration

[mostafaelhoushi]

🚀 Feature

Speedup CMake Configuration Step

Motivation

Everytime we update a CMakeLists.txt file, cmake build/configuration takes a really long time. Probably, it has to re-run the entire configuration, and that means rebuilding LLVM and a bunch of stuff from source.

Pitch

N/A

Alternatives

N/A

Additional context

Some projects have rolled their own ad-hoc way to prevent rebuilds, like https://github.com/facebookresearch/CompilerGym/blob/development/external/protobuf/build_protobuf.cmake#L8-L9

so we could use a similar approach

Cc @ChrisCummins

[ChrisCummins]

Thanks for filing this.

I think the goal here will be to either:

1. build less stuff from source, by using system-wide deps or depending on binary releases
2. figure out how to smartly prevent rebuilding deps when the config changes

The (1) option would be useful for the CI as well, as atm the build-linux-cmake step takes 5 hours, which is a productivity drain. The real time-sink is building LLVM from source, so the fix for this might be related to #568.

Cheers,
Chris

[mostafaelhoushi]

Also I am wondering, if a quick temporary speedup of the LLVM build would be to add -j32 to its build command.

[ChrisCummins]

Also I am wondering, if a quick temporary speedup of the LLVM build would be to add -j32 to its build command.

As far as I understand, the LLVM build is hardcoded to use Ninja, so should already be achieving high levels of parallelism:
https://github.com/facebookresearch/CompilerGym/blob/development/external/llvm/CMakeLists.txt#L43-L44