Distributed building of projects

[mmatyas]

Building big projects, where many additional crates are used (eg. Servo) can be fairly slow, so I would like to set up a distributed build system. I'm mainly thinking about using distcc/icecc, where you can set up a toolchain package to use on the build slaves, but I don't know if Rust/Cargo could operate in such way.
Is this technically possible? If not, is it because of a design decision, or some other limitations?

[brson]

It's techncally possible to create a distributed build system for Rust, but it will likely amount to an alternate implementation of Cargo. There has been no groundwork toward this at all. If you want to get started then I'd suggest writing a distributed cargo from scratch.

Here's a related Cargo bug.

internals would be a good place to discuss this. I'm going to close this issue - if you'd like to have an issue open for it cargo is a good place to put it.

[bergercookie]

@mmatyas you can probably use sccache nowadays. It supports both caching of already built artifacts as well as distributed compilation like icecream:

sccache also provides icecream-style distributed compilation (automatic packaging of local toolchains) for all supported compilers (including Rust). The distributed compilation system includes several security features that icecream lacks such as authentication, transport layer encryption, and sandboxed compiler execution on build servers. See the distributed quickstart guide for more information.

[gxtaillon]

@mmatyas you can probably use sccache nowadays. It supports both caching of already built artifacts as well as distributed compilation like icecream:

sccache also provides icecream-style distributed compilation (automatic packaging of local toolchains) for all supported compilers (including Rust). The distributed compilation system includes several security features that icecream lacks such as authentication, transport layer encryption, and sandboxed compiler execution on build servers. See the distributed quickstart guide for more information.

Somewhat tangentially related, but sscache does not support ARM at this time (see mozilla/sccache#656), so it would not be possible to use it on a raspberry pi farm to speed up iterative development.

[Goosegit11]

@gxtaillon Why would you use Raspberry to compile anyway? Isn't it slow?
well, I can expect the farm to not be that slow, but why not just buy a computer?

[satmandu]

The Raspberry Pi 5 is actually decently fast, with NVMe storage and up to 8Gb of ram. A compile on a current RPI5 device can be ~ 75% of the speed of a compile on an older skylake i3 machine, while using much less power...

[Goosegit11]

but Pi 5 is expensive

[satmandu]

but Pi 5 is expensive

$80 US... And well supported...