LLVM 4.0 Upgrade #40123

TimNN · 2017-02-27T15:09:35Z

Since nobody has done this yet, I decided to get things started:

Todo:

push the relevant commits to rust-lang/llvm and rust-lang/compiler-rt
cleanup .gitmodules
Verify if there are any other commits from rust-lang/llvm which need backporting
Investigate / fix debuginfo ("<optimized out>") failures
Use correct emscripten version in docker image

Closes #37609.

Test results:

Everything is green 🎉

rust-highfive · 2017-02-27T15:09:47Z

r? @brson

(rust_highfive has picked a reviewer for you, use r? to override)

alexcrichton · 2017-02-27T16:46:27Z

@TimNN you can probably work faster than going through @bors by selectively adding ALLOW_PR=1 to various entries in .travis.yml, that'll run the tests on the PR itself before we hit @bors.

I'd recommend doing that for a couple of the cross targets at least and then probably some of the other builders as well (such as emscripten)

My guess is that AppVeyor will be one of the most difficult pieces to update as part of this upgrade. Unfortunately we don't have any extra capacity there for running tests so it may be difficult to do so on the PR before bors :(

TimNN · 2017-02-27T17:14:28Z

The debuginfo failures all occur because static muts are optimized out. (I verified that no optimisation flags were passed to rustc).

I'm unsure what the best fix would be here (~~something like the #[used] attribute from #39987 would probably work~~).

hanna-kruppe · 2017-02-27T18:54:04Z

static muts getting optimized out sounds like an issue with the IR/debug info we generate. Perhaps one of the debug info-related changes was incomplete and we don't emit everything we need to emit for globals? (#39528 looks very related.) To clarify, does this:

static muts are optimized out

mean that gdb outputs <optimized out>, or did you check the object files and the symbols for the globals were missing?

In any case, requiring #[used] on static muts to enable debugging would be a serious regression, not to mention that #[used] doesn't exist yet and I'm not even sure it would fix this (see above).

petrochenkov · 2017-02-27T19:19:55Z

I vaguely remember about statics being optimized away the last summer already, when I wrote debuginfo tests for unions (one of the failing tests in this PR).
So I had to add at least an assignment for the static to be kept.
Maybe the LLVM optimizer become better and now sees that the optimization is still possible?

TimNN · 2017-02-27T19:20:12Z

To clarify, does this:

static muts are optimized out

mean that gdb outputs <optimized out>, or did you check the object files and the symbols for the globals were missing?

It means that gdb prints <optimized out>

Apparently I made some assumptions that weren't quite correct.

The symbols exist (output of nm from the simple-struct test:

0000000000201008 d _ZN13simple_struct13NO_PADDING_1617h8f8e1027ea816fe7E
000000000020100c d _ZN13simple_struct13NO_PADDING_3217h9e367c36ef03eabcE
0000000000201018 d _ZN13simple_struct13NO_PADDING_6417h3c4e19994ee55915E
0000000000201050 d _ZN13simple_struct14PADDING_AT_END17h210f8219dd75f271E
0000000000201040 d _ZN13simple_struct16INTERNAL_PADDING17he308dba4866e981bE
0000000000201030 d _ZN13simple_struct17NO_PADDING_16326417h136d93d95c4cf5b5E

Since this is apparently indeed debuginfo related I guess cc @michaelwoerister, @dylanmckay

TimNN · 2017-02-27T19:23:01Z

The generated IR, in case that helps anyone: https://gist.github.com/TimNN/92152a2f8062909805657d1bb4131998

TimNN · 2017-02-27T19:39:35Z

The failed build of the IMAGE=cross seems to be qemu related, one of the failed tests:

---- [run-pass] run-pass/alignment-gep-tup-like-1.rs stdout ----
	
error: test run failed!
status: exit code: 101
command: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools/x86_64-unknown-linux-gnu/release/qemu-test-client run /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass/alignment-gep-tup-like-1.stage2-arm-unknown-linux-gnueabihf
stdout:
------------------------------------------
uploaded "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass/alignment-gep-tup-like-1.stage2-arm-unknown-linux-gnueabihf", waiting for result
a=22 b=44

------------------------------------------
stderr:
------------------------------------------
thread 'main' panicked at 'client.read_exact(&mut header) failed with failed to fill whole buffer', /checkout/src/tools/qemu-test-client/src/main.rs:174
note: Run with `RUST_BACKTRACE=1` for a backtrace.

------------------------------------------

thread '[run-pass] run-pass/alignment-gep-tup-like-1.rs' panicked at 'explicit panic', /checkout/src/tools/compiletest/src/runtest.rs:2637
note: Run with `RUST_BACKTRACE=1` for a backtrace.

I saw the following panics:

     61 thread 'main' panicked at 'client.read_exact(&mut header) failed with Connection reset by peer (os error 104)', /checkout/src/tools/qemu-test-client/src/main.rs:174
      3 thread 'main' panicked at 'client.read_exact(&mut header) failed with failed to fill whole buffer', /checkout/src/tools/qemu-test-client/src/main.rs:174
      1 thread 'main' panicked at 'io::copy(&mut file, dst) failed I/O failure during tests: Error { repr: Os { code: 11, message: "Resource temporarily unavailable" } }
     96 thread 'main' panicked at 'io::copy(&mut file, dst) failed with Broken pipe (os error 32)', /checkout/src/tools/qemu-test-client/src/main.rs:220
      1 thread 'main' panicked at 'io::copy(&mut file, dst) failed with Connection reset by peer (os error 104)', /checkout/src/tools/qemu-test-client/src/main.rs:220

Do they ring a bell for anyone?

TimNN · 2017-02-27T19:59:14Z

The emscripten failure are mainly of the kind Invalid record (Producer: 'LLVM4.0.0' Reader: 'LLVM 3.9.0'), although there are some assertion failures as well. (I would fix the llvm version mismatch first, maybe that fixes the assertion failures as well).

TimNN · 2017-02-27T22:57:58Z

The android image fails with an LLVM assertion:

rustc: /checkout/src/llvm/lib/Target/ARM/ARMConstantIslandPass.cpp:492: void {anonymous}::ARMConstantIslands::doInitialConstPlacement(std::vector<llvm::MachineInstr*>&): Assertion `Size >= 4 && "Too small constant pool entry"' failed.
Build failed, waiting for other jobs to finish...
rustc: /checkout/src/llvm/lib/Target/ARM/ARMConstantIslandPass.cpp:492: void {anonymous}::ARMConstantIslands::doInitialConstPlacement(std::vector<llvm::MachineInstr*>&): Assertion `Size >= 4 && "Too small constant pool entry"' failed.
error: Could not compile `core`.

There are also some warnings (see below for an example), of which I am unsure how relevant / important they are.

warning: ../compiler-rt/lib/builtins/mulsc3.c:21:1: warning: conflicting types for built-in function '__mulsc3'
warning:  __mulsc3(float __a, float __b, float __c, float __d)
warning:  ^

alexcrichton · 2017-02-27T23:35:14Z

@TimNN the former may be a bug in LLVM? (or just something we've never exposed ourselves before). The latter is normal, I believe it's happening on builds today.

alexcrichton · 2017-02-27T23:35:55Z

Oh we've also got ~10 extra capacity on Travis so feel free to test more than one row at a time if you'd like :)

TimNN · 2017-02-28T07:43:30Z

@alexcrichton: Have you ever seen something like the qemu failures in #40123 (comment) before?

Oh we've also got ~10 extra capacity on Travis so feel free to test more than one row at a time if you'd like :)

Ah, ok. I've been doing 3 at a time (when not debugging emscripten), but I guess I can run a few more :)

TimNN · 2017-02-28T10:54:05Z

The dist-s390x-linux-netbsd build fails while cross compiling llvm due to missing std::thread support:

In file included from /checkout/src/llvm/include/llvm/Support/ThreadPool.h:17:0,
                 from /checkout/src/llvm/lib/Support/ThreadPool.cpp:14:
/checkout/src/llvm/include/llvm/Support/thread.h:41:14: error: 'thread' in namespace 'std' does not name a type
 typedef std::thread thread;

I guess the fix here is to backport (part of?) rust-lang/llvm@58731be as well.

TimNN · 2017-02-28T12:18:00Z

I've been investigating the emscripten failures, see below for the findings. The one thing that both test have in common is that they use fixed sized arrays ([constexpr; len]) although if that is related / the problem, I don't know.

run-pass/packed-struct-vec.rs IR:

The #[repr(packed)] seems to be just broken, printing instead of asserting for equality gives the following results:

Foo { bar: 2, baz: 2 }, Foo { bar: 1, baz: 33649522 }
Foo { bar: 0, baz: 2 }, Foo { bar: 1, baz: 33649522 }
Foo { bar: 0, baz: 2 }, Foo { bar: 1, baz: 33649522 }
Foo { bar: 1, baz: 2 }, Foo { bar: 1, baz: 33649522 }
Foo { bar: 2, baz: 2 }, Foo { bar: 1, baz: 33649522 }
Foo { bar: 0, baz: 2 }, Foo { bar: 1, baz: 33649522 }
Foo { bar: 0, baz: 2 }, Foo { bar: 1, baz: 33649522 }
Foo { bar: 1, baz: 2 }, Foo { bar: 1, baz: 33649522 }
Foo { bar: 2, baz: 2 }, Foo { bar: 1, baz: 33649522 }
Foo { bar: 0, baz: 2 }, Foo { bar: 1, baz: 33649522 }
Foo { bar: 0, baz: 144115188075855872 }, Foo { bar: 1, baz: 33649522 }
Foo { bar: 0, baz: 562949953421312 }, Foo { bar: 1, baz: 33649522 }
Foo { bar: 0, baz: 0 }, Foo { bar: 1, baz: 33649522 }
Foo { bar: 2, baz: 2 }, Foo { bar: 1, baz: 33649522 }
Foo { bar: 0, baz: 144115188075855872 }, Foo { bar: 1, baz: 33649522 }
Foo { bar: 0, baz: 562949953421312 }, Foo { bar: 1, baz: 33649522 }
Foo { bar: 0, baz: 0 }, Foo { bar: 1, baz: 33649522 }
Foo { bar: 2, baz: 2 }, Foo { bar: 1, baz: 33649522 }
Foo { bar: 0, baz: 144115188075855872 }, Foo { bar: 1, baz: 33649522 }
Foo { bar: 0, baz: 0 }, Foo { bar: 1, baz: 33649522 }

(with this code:)

use std::mem;

#[repr(packed)]
#[derive(Copy, Clone, PartialEq, Debug)]
struct Foo {
    bar: u8,
    baz: u64
}

pub fn main() {
    let foos = [Foo { bar: 1, baz: 2 }; 10];

    assert_eq!(mem::size_of::<[Foo; 10]>(), 90);

    for i in 0..10 {
        println!("{:?}, {:?}", foos[i], Foo { bar: 1, baz: 2});
    }

    for &foo in &foos {
        println!("{:?}, {:?}", foo, Foo { bar: 1, baz: 2 });
    }

    assert!(false);
}

run-pass/issue-29663.rs IR:

The write_volatile in the following snippet is apparently not executed correctly:

        let mut x = E([0; 32]);
        write_volatile(&mut x, E([1; 32]));
        assert_eq!(read_volatile(&x), E([1; 32])); // line 61
        assert_eq!(x, E([1; 32]));                 // line 62

The output:

thread 'main' panicked at 'assertion failed: `(left == right)` (left: `E([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])`, right: `E([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])`)', /checkout/src/test/run-pass/issue-29663.rs:61

If line 61 is commented:

thread 'main' panicked at 'assertion failed: `(left == right)` (left: `E([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1])`, right: `E([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])`)', /checkout/src/test/run-pass/issue-29663.rs:62

Note that other write_volatile / read_volatile pairs work correctly.

alexcrichton · 2017-02-28T15:05:41Z

@TimNN

The QEMU failures don't look familiar but are perhaps indicative of the program segfaulting or otherwise exiting un-cleanly. Do you have the full logs I could help take a look at?

The missing std::thread business may be related to how we compile compilers. It may be that one of our compilers is too old or something like that. I know that MinGW C++ compilers, at least, do not have std::thread (at least if I'm remembering correctly). Historically we've "dealt" with this by just deleting code that uses std::thread, but it's clearly getting harder over time! This also isn't scalable into the future really. Unfortunately I don't know of a great solution here.

TimNN · 2017-02-28T15:08:41Z

The QEMU failures don't look familiar but are perhaps indicative of the program segfaulting or otherwise exiting un-cleanly. Do you have the full logs I could help take a look at?

Ah, sorry, I linked the logs only from the original post, here they are: https://travis-ci.org/rust-lang/rust/jobs/205860539

alexcrichton · 2017-02-28T15:27:45Z

Fascinating! Unfortunately I may not be of much help. Also thanks for the links, I should have looked around for them! Of the failures so far:

armhf - this is really suspicious. In theory if qemu-test-server panics in any way we'd see it in the logs (as it doesn't daemonize that much or anything), but I'm not seeing anything. The errors mean that the server is prematurely closing the connection, but I'm not sure how that's possible without otherwise printing error information! It may be a bug in LLVM or related to the assertion failures, but it may also require more debugging the qemu instance itself :(
emscripten - looks like you're on top of these (maybe fastcomp regressions? maybe llvm regression? unsure myself...)
android - yeah looks like an LLVM bug? Although I wouldn't rule out invalid codegen on our side just yet.
s390x-netbsd - Ok so this specifically failed to compiled LLVM for NetBSD. This is our script to compile NetBSD gcc and I don't see anything immediately wrong that should compile the wrong C++. I wonder if the gcc options accidentally disable std::thread? Or maybe it's disabled somewhere else on NetBSD by default? Not sure why that happened :(

Some other tips I'd have is:

You can run docker images locally via ./src/ci/docker/run.sh $image_name which can help with debugging (e.g. avoiding going through Travis). That should in theory use the precise same environment as Travis modulo kernel and hardware.
For suspected LLVM bugs the best way (although certainly not the fastest way) that I know to work with them is to (a) get it to reproduce with a manual rustc invocation then (b) get it to reproduce with an opt invocation by using rustc to generate LLVM IR then switching to LLVM's tools and then (c) minimizing that IR as much as possible. If it's small enough then reporting a bug on LLVM's issue tracker is usually good for getting the issue fixed in a timely fashion.

TimNN · 2017-02-28T15:56:16Z

You can run docker images locally via ./src/ci/docker/run.sh $image_name which can help with debugging (e.g. avoiding going through Travis). That should in theory use the precise same environment as Travis modulo kernel and hardware.

Yeah, I'm doing that right now :)

s390x-netbsd

Alright, so the problem is, I think, that the following ./configure check fails when compiling: checking for gthreads library... no

alexcrichton · 2017-02-28T16:11:39Z

@TimNN heh yeah that'd do it! I wonder if some more headers need to be copied from the NetBSD base system or something like that? Unfortunately I forget now at this point where I got those instructions from to build a NetBSD cross-compiler...

mattico · 2017-02-28T19:43:08Z

Those warnings aren't new. I forget the cause.
Edit: meaning the compiler-rt function signature warnings which seem to have disappeared...

TimNN · 2017-02-28T22:44:11Z

Some notes on the armhf-gnu image:

I (once) got the same LLVM assertion as on android, this went away when retrying the build. I'll try to get this to reliably reproduce.
The qemu connection errors happen non deterministically, as far as I can tell, example: run-pass ran successfully, incremental failed afterwards, after a retry all ofrun-pass failed.
Now I got a segfault... and no idea what actually segfaulted...

alexcrichton · 2017-02-28T23:36:17Z

Odd! I wouldn't entirely rule out a bug in qemu-test-{client,server} FWIW

TimNN · 2017-04-24T11:57:04Z

Yay, looks like all the builds timed out again, so things seem to be good to go. (I didn't verify all the logs this time).

alexcrichton · 2017-04-24T15:40:41Z

@TimNN looks great to me!

I hope to branch beta later today, so want to pull out the fast-fail? I'll r+ this after the beta is branched.

I'd also like to reiterate that you're at least my own personal "Rust Hero of the last N Months" where N is two and counting. If we delay this for 3 more days then it'll be a 2+ month PR!

TimNN · 2017-04-24T15:50:42Z

@alexcrichton: I removed the always fail commit.

I'd also like to reiterate that you're at least my own personal "Rust Hero of the last N Months" where N is two and counting. If we delay this for 3 more days then it'll be a 2+ month PR!

Thanks a lot! All the positive encouragement and feedback has helped a lot in keeping me motivated to work on the upgrade :).

Kixunil · 2017-04-24T17:02:06Z

I'd also like to reiterate that you're at least my own personal "Rust Hero of the last N Months"

Mine too! :)

alexcrichton · 2017-04-24T19:31:21Z

@bors: r+

Beta's branched, let's do this!

bors · 2017-04-24T19:31:21Z

📌 Commit 8994277 has been approved by alexcrichton

bors · 2017-04-24T22:18:22Z

⌛ Testing commit 8994277 with merge 0777c75...

LLVM 4.0 Upgrade Since nobody has done this yet, I decided to get things started: **Todo:** * [x] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt` * [x] cleanup `.gitmodules` * [x] Verify if there are any other commits from `rust-lang/llvm` which need backporting * [x] Investigate / fix debuginfo ("`<optimized out>`") failures * [x] Use correct emscripten version in docker image --- Closes #37609. --- **Test results:** Everything is green 🎉

bors · 2017-04-25T01:21:30Z

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing 0777c75 to master...

brson · 2017-04-25T01:29:19Z

Thanks for slogging through this @TimNN.

BatmanAoD · 2017-04-25T01:41:14Z

Congratulations @TimNN!

DemiMarie · 2017-04-25T04:55:00Z

Thank you @TimNN!

michaelwoerister · 2017-04-25T08:46:27Z

🎉

According to rust-lang/rust#40123 rust now supports LLVM 4

pkphilip · 2017-05-06T12:47:14Z

Wow! Thanks a lot @TimNN! That is some effort!

Kixunil · 2017-05-06T17:36:15Z

I've just noticed that README mentions clang 3.x. Shouldn't this be updated?

hanna-kruppe · 2017-05-06T17:53:27Z

@Kixunil That part of the readme is about the C and C++ compiler used for compiling C and C++ dependencies during the build, not about the LLVM version.

Kixunil · 2017-05-06T18:13:00Z

@rkruppe I guess I'm too hungry and tired. Thank you for pointing that out! :)

…g rustc See: rust-lang/rust#40123 (comment)

rust-highfive assigned brson Feb 27, 2017

TimNN force-pushed the llvm40 branch from 7bf22e0 to f6f33f2 Compare February 27, 2017 20:01

amboar mentioned this pull request Feb 28, 2017

1.14.0 powerpc64le test failures: smoke_dtor, test_typed_arena_drop_small_count #39015

Closed

TimNN force-pushed the llvm40 branch 2 times, most recently from a387dec to 03412b9 Compare February 28, 2017 16:11

TimNN force-pushed the llvm40 branch 2 times, most recently from a2f620f to 934aa51 Compare March 1, 2017 12:36

TimNN force-pushed the llvm40 branch from 516d3a0 to 8994277 Compare April 24, 2017 15:44

bors merged commit 8994277 into rust-lang:master Apr 25, 2017

bors mentioned this pull request Apr 25, 2017

appveyor: Upgrade to gcc for mingw 6.3.0 #41420

Merged

brson added the relnotes Marks issues that should be documented in the release notes of the next release. label Apr 25, 2017

TimNN deleted the llvm40 branch April 25, 2017 05:12

cnd added a commit to gentoo/gentoo-rust that referenced this pull request Apr 25, 2017

Merge pull request #254 from TyanNN/master

5e0c740

According to rust-lang/rust#40123 rust now supports LLVM 4

TimNN mentioned this pull request Apr 27, 2017

can't build ole32-sys with nightly-gnu #41589

Closed

TimNN mentioned this pull request Jun 7, 2017

Updated releases notes for 1.19 #42503

Closed

est31 mentioned this pull request Jun 26, 2017

rg crashes with -C target-cpu=native on Xeon E5-2670 #36677

Closed

arielb1 pushed a commit to rust-lang/llvm that referenced this pull request Jun 27, 2017

[JSBackend] don't use dllexport since it causes problems when buildin…

a62cd42

…g rustc See: rust-lang/rust#40123 (comment)

arielb1 mentioned this pull request Jul 13, 2017

Firefox compile time regression #43211

Closed

vivo75 pushed a commit to vivo75/vivovl that referenced this pull request Sep 6, 2017

According to rust-lang/rust#40123 rust now supports LLVM 4

0acc41e

kennytm mentioned this pull request Jan 22, 2018

Performance regressions of compiled code over the last year #47561

Open

LLVM 4.0 Upgrade #40123

LLVM 4.0 Upgrade #40123

Conversation

TimNN commented Feb 27, 2017 • edited Loading

rust-highfive commented Feb 27, 2017

alexcrichton commented Feb 27, 2017

TimNN commented Feb 27, 2017 • edited Loading

hanna-kruppe commented Feb 27, 2017 • edited Loading

petrochenkov commented Feb 27, 2017

TimNN commented Feb 27, 2017

TimNN commented Feb 27, 2017

TimNN commented Feb 27, 2017

TimNN commented Feb 27, 2017

TimNN commented Feb 27, 2017

alexcrichton commented Feb 27, 2017

alexcrichton commented Feb 27, 2017

TimNN commented Feb 28, 2017

TimNN commented Feb 28, 2017 • edited Loading

TimNN commented Feb 28, 2017 • edited Loading

alexcrichton commented Feb 28, 2017

TimNN commented Feb 28, 2017

alexcrichton commented Feb 28, 2017

TimNN commented Feb 28, 2017

alexcrichton commented Feb 28, 2017

mattico commented Feb 28, 2017 • edited Loading

TimNN commented Feb 28, 2017 • edited Loading

alexcrichton commented Feb 28, 2017

TimNN commented Apr 24, 2017

alexcrichton commented Apr 24, 2017

TimNN commented Apr 24, 2017

Kixunil commented Apr 24, 2017

alexcrichton commented Apr 24, 2017

bors commented Apr 24, 2017

bors commented Apr 24, 2017

bors commented Apr 25, 2017

brson commented Apr 25, 2017

BatmanAoD commented Apr 25, 2017

DemiMarie commented Apr 25, 2017

michaelwoerister commented Apr 25, 2017

pkphilip commented May 6, 2017

Kixunil commented May 6, 2017

hanna-kruppe commented May 6, 2017

Kixunil commented May 6, 2017

TimNN commented Feb 27, 2017 •

edited

Loading

TimNN commented Feb 27, 2017 •

edited

Loading

hanna-kruppe commented Feb 27, 2017 •

edited

Loading

TimNN commented Feb 28, 2017 •

edited

Loading

TimNN commented Feb 28, 2017 •

edited

Loading

mattico commented Feb 28, 2017 •

edited

Loading

TimNN commented Feb 28, 2017 •

edited

Loading