LLVM 5.0 upgrade

[alexcrichton]

Hello! With LLVM 5.0 branched relatively recently we started the LLVM 5.0 upgrade in rust-lang/rust a week or so ago and it turned out that we didn't have many blockers on our end! Much of the various bugs here and there have been fixed now and one of the last remaining pieces would be the LLVM 5.0 upgrade for emscripten.

We've got a "small" local commit which adds the asmjs target to the LLVM fork we've got, so we don't actually need to merge the fastcomp backend into our own fork. All of the LLVM tools that emscripten uses, however, currently choke because we'd otherwise be producing LLVM 5.0 bitcode (tagged as such) and emscriptens current LLVM version (4.0 I believe) rejects any bitcode generated by a future LLVM (rightfully so!)

I'm just opening this to track progress to upgrading to LLVM 5.0, I'm not actually sure what's involved in doing that on this site! It's also worth pointing out that this isn't super urgent on our end. AFAIK there's not a huge amount of demand to upgrade rustc to LLVM 5.0, it's just generally good to stay updated. I wanted to start early this time to shake out any regressions but that ended up not taking as long as expected!

[kripken]

The work for this is happening in

• Merge llvm 401 #191
• Merge clang 401 emscripten-fastcomp-clang#18
• merge_llvm_401 emscripten#5400

(Those say 4.0.1, but I think it's 5.0, not sure why the name.)

@juj did the initial work, looks like there are some breaking tests that need fixing. Not sure if anyone is currently working on it, help welcome.

[alexcrichton]

Oh! Sorry I misinterpreted! Feel free to close this in that case, I can just follow along to this threads!

[kripken]

No worries, actually let's leave this open, we didn't have an issue tracking all of that. (Also there is a chance I misunderstood what @juj was doing and it is 4 and not 5.)

[juj]

For the merge, I wanted to pick the latest shipped stable version and not e.g. the tip, which would be a untagged git commit that has no guarantees of being stable. That is why I went for version number 4.0.1 instead of 5.0, as 4.0.1 is the latest stable release at https://llvm.org

There is an issue with LLVM Git repositories that either their SVN->Git mirroring does not pick up minor commit numbers x.y.z, but only major x.y, or their SVN->Git mirroring of tags is a manual process that they did not do for 4.0.1. As a result, neither the Github mirror of LLVM at https://github.com/llvm-mirror/llvm/branches nor the LLVM Git repository at https://git.llvm.org/git/llvm has a tag or a branch named 4.0.1.

To get to 4.0.1, I manually took the Git changeset from the Git repository that corresponds to their announced tag of 4.0.1 release at http://llvm.org/viewvc/llvm-project/llvm/tags/RELEASE_401/final/ and http://llvm.org/viewvc/llvm-project/cfe/tags/RELEASE_401/final/. These are SVN changesets http://llvm.org/viewvc/llvm-project?view=revision&revision=305825 and http://llvm.org/viewvc/llvm-project?view=revision&revision=305826 respectively. Each Git commit has a git-svn-id, which I used to find the Git commits that match to SVN commits 305825 and 305826.

This should not be LLVM 5, though oddly that commit hash does have the source code configured to report 5.0 as version number, even though it should say 4.0.1. When I asked about this at #llvm irc, I understood that after 4.0.0 shipped, the version number was bumped to 5.0 immediately in trunk, and for the 4.0.1 release in SVN it was apparently changed from 5.0 back to 4.0.1. Likely due to SVN->Git mirroring missing tag 4.0.1, this is not reflected in the Git repository.

Just got back from PTO, and I am looking to continue fixing the remaining test issues with the merge, though there are some other more urgent items before this, so that will be sometime next week or the week after that the earliest.

[est31]

@tlively given rust-lang/rust#44006 and a switch to llvm's wasm backend, is this LLVM 5.0 upgrade still required?

[tlively]

@est31 Yes, because as long as the asmjs and wasm32 Rust targets emit LLVM bitcode, Rust's LLVM cannot be newer than Emscripten's LLVM. Otherwise the Emscripten targets refuse to work, as @alexcrichton pointed out. If the asm.js and wasm32 targets emitted object files instead of LLVM bitcode then this would be solved (I should add that as an item to rust-lang/rust#44006), but in order for that to happen Rust needs LLVM 5 anyway, and therefore requires this upgrade.

[est31]

@tlively thanks for clarification!

[dylanmckay]

Have kicked this off for the LLVM 5 branch

• WIP: Update to LLVM 5.0 #199
• WIP: Update to LLVM 5.0 emscripten-fastcomp-clang#19

[gnzlbg]

Update and heads up: we have now bumped the minimum LLVM version that Rust supports to LLVM 5.0, and as soon as the distros that are shipping Rust with a system LLVM using version 5.0.x upgrade their system LLVM to version 6.y, which might happen some time this year (maybe after LLVM 7 is released in August), we might further bump the minimum required LLVM version to 6.0.

Currently, emscripten Rust targets (wasm32 and asmjs) are set up to use its own specific LLVM 4.0 backend inside the Rust toolchain, and we have kept the workarounds for LLVM 4.0 around so that Rust code that used to successfully compile to emscripten targets continues to compile. However, Rust code that uses Rust newer languages or library features that depend on features only available in LLVM 5, 6, or 7 does not compile successfully already to the emscripten targets (and hasn't for a while).

[kripken]

@gnzlbg Thanks for the update!

For emscripten, we support 2 versions of LLVM:

• The asm.js/asm2wasm path, which has been updated to LLVM 6.0.1 (in 1.38.3, late May).
• The LLVM wasm backend path, using pure LLVM upstream, tested on LLVM svn (7.0).

So from emscripten's perspective there is no need to support LLVM 4 or 5 at this point, if you use a recent enough version.

Going forward, it may also be easier for Rust to focus on emscripten with the LLVM wasm backend, which uses very latest pure upstream LLVM. That is expected to become the default in emscripten too.