Use rust's docker images for mips*, powerpc*, and s390x

[cuviper]

When building for linux-gnu targets, we need to be careful about which
versioned GLIBC symbols we'll link, as this sets the minimum supportable
system version in ABI. The images in rust-lang/rust/src/ci/docker/ have
already taken this into account, and since they're uploaded to S3, we
can just use those exact images to get the same ABI compatibility.

The builds for mips*, powerpc*, and s390x were identified in #1681 as
having newer glibc symbols than rustc, so those are now changed to use
rust's docker images. In the future, we might want to do this for other
architectures too, so all toolchains are in sync.

[kinnison]

This looks plausible to me but I'm still very new to docker and travis when it comes down to it. @alexcrichton can you confirm if this looks sane?

[mati865]

For how long images like that are retained?

[alexcrichton]

I think using the exact same images as rust-lang/rust is a great idea! I think though this may need to be tweaked slightly becaues the hashes are all stored on s3 which is deleted every 90 days or so. That means we'll have to manually update the hashes here :(

I wonder if we could tweak our CI to maybe upload a file describing each target and the container it was built in? For example /rustc-builds/$target would always contain the sha512 and sha256 to do what this script does (or something like that) and while still not guaranteed to work it's "much more likely to work"

[cuviper]

I think though this may need to be tweaked slightly because the hashes are all stored on s3 which is deleted every 90 days or so.

I didn't realize that -- does this mean rust CI is forced to rebuild and reupload the images periodically? I haven't witnessed this, but I certainly don't watch every PR go by. :)

I wonder if we could tweak our CI to maybe upload a file describing each target and the container it was built in? For example /rustc-builds/$target would always contain the sha512 and sha256 to do what this script does (or something like that) and while still not guaranteed to work it's "much more likely to work"

OK, I can try adding something like that.

I think we'd only want to update that when building for nightly / the master branch, or else include the branch in that path name. I'm not sure how to tell though -- src/ci/run.sh sets the channel, but src/ci/docker/run.sh doesn't know that. I guess I could grep it or something...

[alexcrichton]

In practice we tend to modify each image at least once every 90 days so the rebuilds happen on that PR vs others, or we just have tons of timeouts randomly as everything is rebuilt and we don't really see it much.

One alternative could which may avoid grepping around is that we upload a well-known filename into the s3 bucket /rustc-builds/${commit_sha}-${image_name} which contains the sha512 and sha256 here in this PR. That way this PR would look up the current commit on the master branch and then use that plus a shared name of the cross builders to look-up the sha512 and such perhaps?

[cuviper]

Ah, deployed artifacts already use s3://rust-lang-ci2/rustc-builds/$TRAVIS_COMMIT/, so I guess we can drop image shas in there too, and on the rustup side use git ls-remote to find master.

[alexcrichton]

Sounds reasonable to me!

[cuviper]

OK, I've added image info in rust-lang/rust#59420, and updated this PR to use it -- though of course that won't work until we've merged and deployed the new info. It seems to work locally, at least.

[kinnison]

Thanks for the review @alexcrichton -- We'll look to merge this later tonight unless one of the other wg-rustup members has concerns.

[tesuji]

I think this PR is blocked on rust-lang/rust#59420.

[cuviper]

Right, it won't work without the Rust PR. It's not clear to me when the extra CI jobs actually run for this repo though -- is that only for the stable branch? Maybe I should temporarily toggle that branch filter in .travis.yml when we're ready to try this?

[kinnison]

Oh right, I was under the impression it was approved because things were ready, we'll hold off then 👍

[kinnison]

And yes, the non-default builds only happen on the stable branch in order to reduce the load on the appveyor and other travis workers.

[cuviper]

OK, the Rust PR has merged and deployed, and it works for me to read through that and fetch images. Should be ready to go!

[kinnison]

Let's give it a spin and see how badly things go :D Thank for your efforts.