ci: Move dist-aarch64-linux to an aarch64 runner

[mrkajetanp]

Move the dist-aarch64-linux CI job to an aarch64 runner instead of cross-compiling it from an x86 one. This will make it possible to perform optimisations such as LTO, PGO and BOLT later on.

r? @Kobzol

try-job: dist-aarch64-linux
try-job: dist-x86_64-linux
try-job: dist-i686-linux

[Kobzol]

First let's see if it works, and what's the time impact :)

@bors try

[bors]

⌛ Trying commit 4122380 with merge 6b8022e...

[bors]

☀️ Try build successful - checks-actions
Build commit: 6b8022e (6b8022ef4cbfc68ca42d9ec189f62d4c129e8685)

[Kobzol]

@bors try

[bors]

⌛ Trying commit 4122380 with merge 3521911...

[bors]

☀️ Try build successful - checks-actions
Build commit: 3521911 (352191141fb3c2d284583d558f204d3299a4f7f6)

[Kobzol]

The cached duration now seems to be ~1h 20m. Before, it was approximately ~1h 50m, although on a free runner.

[mrkajetanp]

Before meaning when it was running on the x86 runner? That would make sense given that it had to build the whole cross-compiling toolchain first I suppose.

[Kobzol]

Yeah, before meaning when it was cross-compiling, and also running on a 4-core machine (vs now running on a 8-core machine). The costs of the ARM machine is quite different to us than the previous machine though, so we'll need to discuss this in the infra team.

[mrkajetanp]

Fair enough, best of luck! :)

[nikic]

Our documented glibc baseline for aarch64 is 2.17, while this is switching to ubuntu 22.04, which is glibc 2.35.

[Kobzol]

Ah, good point! To clarify, before crosstool was used to explicitly configure glibc 2.17 (https://github.com/rust-lang/rust/pull/133809/files#diff-08277b07e8cb8ff6abaa4638229d2a8eb8d14a58590e3ebaafa6d13b3da07081L9), while now we just use a native build on Ubuntu 22.04 (where configuring glibc 2.17 is very painful, at least as far as I'm aware). So we might need to switch e.g. to CentOS, same as we do for the x64 Linux dist build.

[mrkajetanp]

e.g. to CentOS, same as we do for the x64 Linux dist build.

I can switch the build to CentOS, though are we sure that's a good idea given that it's now past its end of life? EOL for the CentOS 7 used in the x86 dist build was June 2024. Afaict all distros' versions that ship 2.17 are now past their EOL.

[Kobzol]

That is indeed an issue, although orthogonal to ARM, I suppose.

@nikic CentOS 7 is indeed EOL since June 2024 (https://www.redhat.com/en/topics/linux/centos-linux-eol). Should we do https://blog.rust-lang.org/2022/08/01/Increasing-glibc-kernel-requirements.html again? Our HPC center has updated to glibc 2.28 last year, which I personally use as a litmus test to see where the current baseline is =D

[mrkajetanp]

Debian Buster has glibc 2.28 and Extended LTS until 2029, how does trying that one sound?
There is also Debian Stretch with 2.24 until 2027 and Debian Jessie with 2.19, though that one is only alive until June.

[Kobzol]

The extended LTS is a commercial offering, if I understand correctly, we don't currently make use of any such extended variants.

Created https://rust-lang.zulipchat.com/#narrow/channel/242791-t-infra/topic/Increasing.20glibc.20baseline/near/486974115 to discuss this bump.

[nikic]

I don't think we really care whether the base image we use for building is EOL. Lack of security updates shouldn't have a material effect on our artifacts -- I don't think we're linking anything statically from the host that we aren't building ourselves? Like, stuff like openssl in cargo comes from a vendored build, and we even build our own zstd on these.

The last time we bumped the glibc requirement it was because it was not possible to both target old glibc and build new LLVM with crosstool-ng. I expect we're still a bit off from hitting that problem again (maybe next time LLVM bumps requirements...)

cc @cuviper who probably has opinions on both glibc baseline and our use of centos:7 base images :)

[mrkajetanp]

The extended LTS is a commercial offering

From what I understand it's commercial in that a 3rd party gets paid to backport security fixes and such, it's still free to use. Besides it's effectively an opt-in repo so we just use an older official Debian docker image, then if a security fix is needed it's just a matter of adding in the repo and running an update.
https://www.freexian.com/lts/extended/docs/how-to-use-extended-lts/

Whether this is relevant or not is a different question - I imagine not - but it's slightly better than just using a completely dead version I suppose. Especially compared to CentOS where relying on an archive of old builds of a dead distro seems pretty dodgy if we can avoid it.

[mrkajetanp]

To clarify, the only concern I have here is that CentOS has been discontinued so I'm wondering whether it's still okay to rely on the repos being up. It's probably good to have at least a migration plan in case they get taken down. If we're confident that they'll stay available for the foreseeable future then there's no problem at all imo.

[Kobzol]

Based on https://rust-lang.zulipchat.com/#narrow/channel/242791-t-infra/topic/Increasing.20glibc.20baseline, we're currently OK with using the EOL runner on CI, as we're not really using anything from it statically, we just use it to build the compiler and the stdlib.

[mrkajetanp]

Fine by me, let's try the CentOS variant then :)

[Kobzol]

@bors try

[Kobzol]

The job had uncached Docker image, and apparently that times out now :/ I'll probably have to increase the timeout again.

@bors try

[bors]

⌛ Trying commit f1ed637 with merge 2935440...

[bors]

☀️ Try build successful - checks-actions
Build commit: 2935440 (29354404c35bae84a0559c1b1c1b4d917cd6ec87)

[davidtwco]

@mrkajetanp is away for the holidays so I've rebased and fixed the conflict

[Kobzol]

@bors try

[bors]

⌛ Trying commit 3afda7e with merge f868ea3...

[bors]

💥 Test timed out

[Kobzol]

@bors try

[bors]

⌛ Trying commit 3afda7e with merge 9c3de39...

[bors]

☀️ Try build successful - checks-actions
Build commit: 9c3de39 (9c3de39e85ac30def4927018e30074c0331f8252)

[Kobzol]

The duplication is not ideal, but at least it was balanced a bit by the small CI script cleanup :)

@bors r+ rollup=never

[bors]

📌 Commit 3afda7e has been approved by Kobzol

It is now in the queue for this repository.

[bors]

⌛ Testing commit 3afda7e with merge 023521e...

[bors]

☀️ Test successful - checks-actions
Approved by: Kobzol
Pushing 023521e to master...