Add a disabled builder for aarch64 emulated tests #43226

alexcrichton · 2017-07-14T00:48:34Z

This commit adds a disabled builder which will run all tests for the standard
library for aarch64 in a QEMU instance. Once we get enough capacity to run this
on Travis this can be used to boost our platform coverage of AArch64

rust-highfive · 2017-07-14T00:48:37Z

r? @aturon

(rust_highfive has picked a reviewer for you, use r? to override)

kennytm · 2017-07-14T02:30:11Z

src/ci/shared.sh

@@ -15,6 +15,12 @@

 # See http://unix.stackexchange.com/questions/82598
 function retry {
+  # Only do retries on CI, locally they shouldn't really be necessary
+  if [ -z "$TRAVIS$APPVEYOR" ]; then


Travis complains about AppVeyor here 😄

[00:00:00] /home/travis/build/rust-lang/rust/src/ci/shared.sh: line 19: APPVEYOR: unbound variable

You should be able to just check $CI, this is defined on both Travis and AppVeyor. This might need to be manually passed to the Docker instances though.

An alternate to avoid unbound variable issues: "${TRAVIS:-}${APPVEYOR:-}", though it's possible using $CI would be cleaner.

kennytm · 2017-07-14T06:29:55Z

src/tools/remote-test-client/src/main.rs

+               .arg("-kernel").arg("/tmp/zImage")
+               .arg("-initrd").arg(&rootfs_img)
+               .arg("-dtb").arg("/tmp/vexpress-v2p-ca15-tc1.dtb")
+               .arg("-append").arg("console=ttyAMA0 root=/dev/ram rdinit=/sbin/init init=/sbin/init")


tidy check (x86_64-unknown-linux-gnu) [00:03:20] tidy error: /checkout/src/tools/remote-test-client/src/main.rs:157: line longer than 100 chars [00:03:20] tidy error: /checkout/src/tools/remote-test-client/src/main.rs:169: line longer than 100 chars [00:03:21] some tidy checks failed

aidanhs · 2017-07-18T22:14:25Z

src/ci/shared.sh

+      "$@"
+      return
+  fi
+


As far as I can see, the $CI environment variable is not exported inside docker (https://github.com/rust-lang/rust/blob/4bd27fc/src/ci/docker/run.sh#L70-L87) so this will disable retries for any docker builds.

That aside, I'm reluctant to give users a different experience to what we work with in CI.

Just to expand on my reluctance - one reason we do CI is so we know other people can build it correctly. I can imagine myself getting frustrated by an intermittent failure, just to realise that rust uses a magic variable to dodge it.

Is there a particular reason for changing it as part of this PR?

Oops sorry I meant to respond to this earlier! I changed this here because it's a pretty bad experience when working on the docker files. Every failure turns into a "retry" so if you've got a normal failure in the docker image it just gives it to you 5 times which often makes it hard to find the error.

This seems contentious though so I'll just back it out.

bors · 2017-07-19T00:38:08Z

☔ The latest upstream changes (presumably #43316) made this pull request unmergeable. Please resolve the merge conflicts.

aturon · 2017-07-19T00:40:00Z

cc @rust-lang/infra, anybody up for giving this an official review?

aidanhs · 2017-07-19T23:50:35Z

I'll go through it this weekend (unless someone jumps in first).

aidanhs

Since this is disabled I don't particularly mind r+ing this mostly as-is, but the CI variable stuff needs tweaking in some way (either passing it down to docker, or removing the retry disabling) or I think we're going to start seeing spurious failures again.

aidanhs · 2017-07-24T13:09:46Z

src/ci/docker/disabled/aarch64-gnu/Dockerfile

+      cd /build/linux-4.4.42 && \
+      cp /build/.config . && \
+      make -j$(nproc) all && \
+      cp arch/arm64/boot/Image /tmp && \


Looks like we use zImage for armhf, it'd probably be nice if we did the same here just for consistency.

I would yeah, but the normal build process didn't create that, so we didn't have it to use it :(

I should mention that I know nothing about kernel build processes, I just found that this particular configuration didn't create a zImage

aidanhs · 2017-07-24T13:11:26Z

src/ci/docker/disabled/aarch64-gnu/Dockerfile

+RUN chmod +x rootfs/etc/init.d/rcS
+
+# Helper to quickly fill the entropy pool in the kernel.
+COPY armhf-gnu/addentropy.c /tmp/


This and rcS above are being pulled out of the armhf-gnu folder - maybe a qemu-common folder?

aidanhs · 2017-07-24T13:19:38Z

src/ci/shared.sh

+      "$@"
+      return
+  fi
+


Just to expand on my reluctance - one reason we do CI is so we know other people can build it correctly. I can imagine myself getting frustrated by an intermittent failure, just to realise that rust uses a magic variable to dodge it.

Is there a particular reason for changing it as part of this PR?

alexcrichton · 2017-07-24T21:31:41Z

Updated!

aidanhs · 2017-07-24T22:17:56Z

src/ci/docker/disabled/aarch64-gnu/Dockerfile

+RUN chmod +x rootfs/etc/init.d/rcS
+
+# Helper to quickly fill the entropy pool in the kernel.
+COPY armhf-gnu/addentropy.c /tmp/


Another one for scripts/?

aidanhs · 2017-07-24T22:21:18Z

src/bootstrap/config.rs

@@ -636,6 +636,12 @@ impl Config {
                    let target = self.target_config.entry(target).or_insert(Target::default());
                    target.qemu_rootfs = Some(parse_configure_path(value));
                }
+                "CFG_QEMU_AARCH64_ROOTFS" if value.len() > 0 => {
+                    let target = "aarch64-unknown-linux-gnu".to_string();


This needs to use the interner in the new rustbuild, see the ARMHF line above. Looks like it changed underneath you!

aidanhs · 2017-07-24T22:22:22Z

r=me with the interning fix to get the PR build passing and the move of addentropy.c

This commit adds a disabled builder which will run all tests for the standard library for aarch64 in a QEMU instance. Once we get enough capacity to run this on Travis this can be used to boost our platform coverage of AArch64

alexcrichton · 2017-07-24T23:25:29Z

@bors: r=aidanhs

bors · 2017-07-24T23:25:30Z

📌 Commit 8ef3f69 has been approved by aidanhs

bors · 2017-07-25T00:10:19Z

⌛ Testing commit 8ef3f69 with merge 1399c0d92e42e5510d784e3015e87c3c2eef7481...

bors · 2017-07-25T00:48:09Z

💔 Test failed - status-travis

Mark-Simulacrum · 2017-07-25T00:55:58Z

LLVM just stopped building... presumably sccache related? Not sure. cc @alexcrichton

@bors retry

possibly Tracking issue for spurious network failures on bots #40474 (network related failure)

[00:04:14] [ 18%] Building CXX object lib/DebugInfo/PDB/CMakeFiles/LLVMDebugInfoPDB.dir/Raw/RawError.cpp.o
[00:04:14] [ 18%] Building CXX object lib/DebugInfo/PDB/CMakeFiles/LLVMDebugInfoPDB.dir/Raw/RawSession.cpp.o
[00:04:14] [ 18%] Building CXX object lib/DebugInfo/PDB/CMakeFiles/LLVMDebugInfoPDB.dir/Raw/SymbolStream.cpp.o
[00:04:14] [ 18%] Building CXX object lib/DebugInfo/PDB/CMakeFiles/LLVMDebugInfoPDB.dir/Raw/TpiHashing.cpp.o
[00:04:14] [ 18%] Building CXX object lib/DebugInfo/PDB/CMakeFiles/LLVMDebugInfoPDB.dir/Raw/TpiStream.cpp.o
[00:04:14] [ 19%] Building CXX object lib/DebugInfo/PDB/CMakeFiles/LLVMDebugInfoPDB.dir/Raw/TpiStreamBuilder.cpp.o
[00:04:15] [ 19%] Linking CXX static library ../../libLLVMDebugInfoPDB.a
[00:04:15] [ 19%] Built target LLVMDebugInfoPDB


No output has been received in the last 30m0s, this potentially indicates a stalled build or something wrong with the build itself.
Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received

The build has been terminated

bors · 2017-07-25T03:22:44Z

⌛ Testing commit 8ef3f69 with merge e0d7d4672073054e89eeebfc2287427305cb6301...

bors · 2017-07-25T04:00:16Z

💔 Test failed - status-travis

alexcrichton · 2017-07-25T04:40:47Z

@bors: retry

Let's see if it's #40474

bors · 2017-07-25T07:26:20Z

⌛ Testing commit 8ef3f69 with merge e68b13ae6fab2d475a3d831257d85932867be3fe...

bors · 2017-07-25T08:04:12Z

💔 Test failed - status-travis

kennytm · 2017-07-25T08:42:48Z

Same error, should be legit.

aidanhs · 2017-07-25T13:17:12Z

Third time in a row for that hang while building LLVM, but I can't figure out what's causing it given that virtually nothing has changed about armhf. My best guess is something to do with the docker build not being cached any more so some steps are re-run, but the only one that looks like it could change is the acquisition of vexpress-v2p-ca15-tc1.dtb...but a) I don't think that gets used during llvm compilation and b) I doubt there's a "loop forever" option in it.

@alexcrichton any thoughts?

alexcrichton · 2017-07-25T21:40:55Z

I have no idea how this PR could be related to that error. I'm running tests locally to try to replicate, I doubt I will be successful.

alexcrichton · 2017-07-25T22:10:36Z

@bors: retry

Continue to be no failures locally, and I can't fathom how this is related to that...

bors · 2017-07-26T03:53:52Z

⌛ Testing commit 8ef3f69 with merge bad58f2...

Add a disabled builder for aarch64 emulated tests This commit adds a disabled builder which will run all tests for the standard library for aarch64 in a QEMU instance. Once we get enough capacity to run this on Travis this can be used to boost our platform coverage of AArch64

bors · 2017-07-26T06:23:13Z

☀️ Test successful - status-appveyor, status-travis
Approved by: aidanhs
Pushing bad58f2 to master...

rust-highfive assigned aturon Jul 14, 2017

kennytm reviewed Jul 14, 2017

View reviewed changes

alexcrichton force-pushed the aarch64-ci branch from ba2afcd to 4864e91 Compare July 14, 2017 03:25

kennytm reviewed Jul 14, 2017

View reviewed changes

alexcrichton force-pushed the aarch64-ci branch from 4864e91 to 7aa073c Compare July 14, 2017 13:01

shepmaster added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 14, 2017

aidanhs reviewed Jul 18, 2017

View reviewed changes

alexcrichton force-pushed the aarch64-ci branch 2 times, most recently from 0f43c8c to 0282dfe Compare July 19, 2017 14:16

aidanhs assigned aidanhs and unassigned aturon Jul 19, 2017

aidanhs suggested changes Jul 24, 2017

View reviewed changes

alexcrichton force-pushed the aarch64-ci branch 2 times, most recently from 1fe1437 to eb6f996 Compare July 24, 2017 21:31

aidanhs reviewed Jul 24, 2017

View reviewed changes

aidanhs approved these changes Jul 24, 2017

View reviewed changes

Add a disabled builder for aarch64 emulated tests

8ef3f69

This commit adds a disabled builder which will run all tests for the standard library for aarch64 in a QEMU instance. Once we get enough capacity to run this on Travis this can be used to boost our platform coverage of AArch64

alexcrichton force-pushed the aarch64-ci branch from eb6f996 to 8ef3f69 Compare July 24, 2017 23:25

bors merged commit 8ef3f69 into rust-lang:master Jul 26, 2017

alexcrichton deleted the aarch64-ci branch August 22, 2017 19:43

Add a disabled builder for aarch64 emulated tests #43226

Add a disabled builder for aarch64 emulated tests #43226

Conversation

alexcrichton commented Jul 14, 2017

rust-highfive commented Jul 14, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bors commented Jul 19, 2017

aturon commented Jul 19, 2017

aidanhs commented Jul 19, 2017

aidanhs left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexcrichton commented Jul 24, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aidanhs commented Jul 24, 2017

alexcrichton commented Jul 24, 2017

bors commented Jul 24, 2017

bors commented Jul 25, 2017

bors commented Jul 25, 2017

Mark-Simulacrum commented Jul 25, 2017

bors commented Jul 25, 2017

bors commented Jul 25, 2017

alexcrichton commented Jul 25, 2017

bors commented Jul 25, 2017

bors commented Jul 25, 2017

kennytm commented Jul 25, 2017

aidanhs commented Jul 25, 2017

alexcrichton commented Jul 25, 2017

alexcrichton commented Jul 25, 2017

bors commented Jul 26, 2017

bors commented Jul 26, 2017