Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to LLVM 4.0 #37609

Closed
22 of 23 tasks
shepmaster opened this issue Nov 5, 2016 · 49 comments · Fixed by #40123
Closed
22 of 23 tasks

Upgrade to LLVM 4.0 #37609

shepmaster opened this issue Nov 5, 2016 · 49 comments · Fixed by #40123
Labels
A-LLVM Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues. C-enhancement Category: An issue proposing an enhancement or a PR with one. E-hard Call for participation: Hard difficulty. Experience needed to fix: A lot. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@shepmaster
Copy link
Member

shepmaster commented Nov 5, 2016

The AVR-Rust post continues to track AVR-LLVM, which is tracking upstream LLVM. That means we are running into changes from the future LLVM 4.0. I figured we could open up this issue now to track the changes required to stay compatible.

This is the current list of upgrade hazards. Note that some of the text may be nonsensical as I don't truly grok all the internals of LLVM so some changes are wild stabs in the dark. It's also possible that some failed fix for earlier problems actually caused later problems:


A WIP PR that performs the upgrade! #40123


Some issues whose resolution is believed to be blocked on LLVM 4.0:

@brson
Copy link
Contributor

brson commented Nov 9, 2016

Next time we do this upgrade we'll need emscripten to do it to. It would be good if we could pull in a working llvm wasm backend in the same upgrade, but that may not be ready before the end of the year.

@hanna-kruppe
Copy link
Contributor

hanna-kruppe commented Nov 10, 2016

For the record, I am also working with a fork of LLVM that is tracking upstream (not AVR-LLVM but similar in spirit) and am starting to work on this now. I'm looking through the AVR-Rust commits and I'd love to coordinate --- feel free to ping me on IRC (rkruppe) or via mail!

@TimNN
Copy link
Contributor

TimNN commented Nov 16, 2016

Note that there is a regression reported against llvm 4.0 regarding the powi intrinsic on armhf targets:

https://llvm.org/bugs/show_bug.cgi?id=30543

I'm explicitly mentioning this here, because there were previously problems with it, see #37559 and related issues. Also cc rust-lang/compiler-builtins#116, an issue about the same problem in the compiler-rt rust port.

@japaric japaric mentioned this issue Nov 16, 2016
8 tasks
@alexcrichton
Copy link
Member

To clarify what I think the @TimNN is referring to (correct me if I'm wrong)

  • In LLVM 3.9 and before, the calling convention LLVM used with the powidf2 and powisf2 intrinsics was basically the C calling convention
  • In LLVM 4.0, the backend was corrected to expect these intrinsics to have the aapcs calling convention
  • One person claims that aapcs is expected for powis2f and powid2f, while another cites a different thread claiming that the normal C calling convention should be used.

So it seems that if the LLVM change is not reverted (expected powisf2 to be aapcs) then we need to update our definition.

This is going to be a huge PITA for supporting multiple LLVM versions as well. We need to select the calling convention depending on the LLVM version as well...

@shepmaster
Copy link
Member Author

and I'd love to coordinate

@rkruppe Any objections to doing that coordination here on GitHub, or on one of our forks?

@hanna-kruppe
Copy link
Contributor

I don't have strong preferences for the communication channel.

@hanna-kruppe
Copy link
Contributor

#37831 should address what is listed as "AttrBuilder API changes" above (the core of the change is in how attributes are represented).

@hanna-kruppe
Copy link
Contributor

@shepmaster I would prefer to upstream LLVM 4.0 compatibility patches sooner rather than later. That includes my own work obviously, but I'm also looking at the commits I cherry-picked from AVR-Rust. In particular, avr-rust@3e1e4a4 looks like a reasonable simplification even independent of LLVM 4.0 compatibility, and avr-rust@c32f9c1 looks like a nice little self-contained fix. Could you create PRs for those? (If you don't have the time but are otherwise fine with upstreaming, I'd be happy to manage that, giving proper credit of course.)

@shepmaster
Copy link
Member Author

@rkruppe Absolutely! I wasn't really sure how much we could expect LLVM 4 to change as it goes towards stable / how much churn the Rust maintainers would want to put up with. I'll get those pulled out and submitted soon. I think I'm also starting to understand some of the silly error handling that the new LLVM does.

@hanna-kruppe
Copy link
Contributor

@shepmaster Awesome, thanks in advance!

@hanna-kruppe
Copy link
Contributor

BTW the PR reference for "llvm::AttrBuilder changed API" is wrong, it points at this issue instead of #37831

bors added a commit that referenced this issue Nov 19, 2016
[LLVM 4.0] Use llvm::Attribute APIs instead of "raw value" APIs

The latter will be removed in LLVM 4.0 (see llvm-mirror/llvm@4a6fc8b).

The librustc_llvm API remains mostly unchanged, except that llvm::Attribute is no longer a bitflag but represents only a *single* attribute.
The ability to store many attributes in a small number of bits and modify them without interacting with LLVM is only used in rustc_trans::abi and closely related modules, and only attributes for function arguments are considered there.
Thus rustc_trans::abi now has its own bit-packed representation of argument attributes, which are translated to rustc_llvm::Attribute when applying the attributes.

cc #37609
@sanxiyn sanxiyn added the A-LLVM Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues. label Nov 22, 2016
@brson brson added E-hard Call for participation: Hard difficulty. Experience needed to fix: A lot. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-tools C-enhancement Category: An issue proposing an enhancement or a PR with one. labels Nov 23, 2016
bors added a commit that referenced this issue Nov 25, 2016
[LLVM 4.0] OptimizationDiagnostic FFI forward compatibility

- getMsg() changed to return std::string by-value. Fix: copy the data to a rust String during unpacking.
- getPassName() changed to return StringRef

cc #37609
bors added a commit that referenced this issue Nov 25, 2016
…hton

[LLVM 4.0] Pass new argument ExportSymbol to DIBuilder::createNameSpace

cc #37609
@Kixunil
Copy link
Contributor

Kixunil commented Nov 26, 2016

Does anyone have ETA? I'd like to know when I can use AVR. :)

bors added a commit that referenced this issue Nov 27, 2016
…hton

[LLVM 4.0] LLVMRustPrintPasses

Adapt `LLVMRustPrintPasses` to LLVM 4.0 preferring `StringRef` over `char *`

cc #37609
arielb1 pushed a commit to arielb1/rust that referenced this issue Mar 10, 2017
alexcrichton added a commit to alexcrichton/rust that referenced this issue Mar 11, 2017
arielb1 pushed a commit to arielb1/rust that referenced this issue Mar 11, 2017
arielb1 pushed a commit to arielb1/rust that referenced this issue Mar 11, 2017
bors added a commit that referenced this issue Apr 8, 2017
[WIP] LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [ ] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [ ] cleanup `.gitmodules`
* [ ] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

* `FAIL`: `IMAGE=armhf-gnu` [travis](https://travis-ci.org/rust-lang/rust/jobs/205860539) (looks to be qemu related, or rather the client / server tools)
* `FAIL`: `IMAGE=emscripten` [travis](https://travis-ci.org/rust-lang/rust/jobs/205978867) (with an updated emcc: one bad emcc optimization: emscripten-core/emscripten-fastcomp#176) ~~[travis](https://travis-ci.org/rust-lang/rust/jobs/205860559) (mainly (only?) due to emcc / rust llvm version mismatch)~~
* `FAIL`: `IMAGE=dist-powerpc64-linux` [travis](https://travis-ci.org/rust-lang/rust/jobs/206514546) (llvm unreachable when compiling core: `Invalid PPC CTR loop!
UNREACHABLE executed at .../PPCCTRLoops.cpp:722!`, reported as [upstream bug #32485](https://bugs.llvm.org//show_bug.cgi?id=32485))
* `SUCCESS/TIMEOUT`: `IMAGE=dist-arm-linux` [travis](https://travis-ci.org/rust-lang/rust/jobs/206514540) (passes locally)
* `SUCCESS/TIMEOUT`: `IMAGE=dist-armv7-aarch64-linux` [travis](https://travis-ci.org/rust-lang/rust/jobs/206514541) (passes locally)
* `SUCCESS`: `IMAGE=arm-android` (fixed)
* `SUCCESS`: `IMAGE=cross`
* `SUCCESS`: `IMAGE=linux-tested-targets`
* `SUCCESS`: `IMAGE=dist-android`
* `SUCCESS`: `IMAGE=dist-freebsd` (fixed)
* `SUCCESS`: `IMAGE=dist-fuchsia`
* `SUCCESS`: `IMAGE=dist-mips-linux`
* `SUCCESS`: `IMAGE=dist-mips64-linux`
* `SUCCESS`: `IMAGE=dist-powerpc-linux`
* `SUCCESS`: `IMAGE=dist-s390x-linux-netbsd` (fixed)
* `SUCCESS`: `IMAGE=dist-x86-linux` (fixed)
* `SUCCESS`: `IMAGE=i686-gnu`
* `SUCCESS`: `IMAGE=i686-gnu-nopt`
* `SUCCESS`: `IMAGE=x86_64-gnu-llvm-3.7`
* `SUCCESS`: `IMAGE=x86_64-gnu`
* `SUCCESS`: `IMAGE=x86_64-gnu-aux`
* `SUCCESS`: `IMAGE=x86_64-gnu-debug`
* `SUCCESS`: `IMAGE=x86_64-gnu-nopt`
* `SUCCESS`: `IMAGE=x86_64-gnu-full-bootstrap`
* `SUCCESS`: `IMAGE=x86_64-gnu-distcheck`
* `SUCCESS`: `IMAGE=x86_64-gnu-incremental`
bors added a commit that referenced this issue Apr 14, 2017
LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [ ] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [ ] cleanup `.gitmodules`
* [ ] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

* `FAIL`: `IMAGE=armhf-gnu` [travis](https://travis-ci.org/rust-lang/rust/jobs/205860539) (looks to be qemu related, or rather the client / server tools)
* `FAIL`: `IMAGE=emscripten` [travis](https://travis-ci.org/rust-lang/rust/jobs/205978867) (with an updated emcc: one bad emcc optimization: emscripten-core/emscripten-fastcomp#176) ~~[travis](https://travis-ci.org/rust-lang/rust/jobs/205860559) (mainly (only?) due to emcc / rust llvm version mismatch)~~
* `FAIL`: `IMAGE=dist-powerpc64-linux` [travis](https://travis-ci.org/rust-lang/rust/jobs/206514546) (llvm unreachable when compiling core: `Invalid PPC CTR loop!
UNREACHABLE executed at .../PPCCTRLoops.cpp:722!`, reported as [upstream bug #32485](https://bugs.llvm.org//show_bug.cgi?id=32485))
* `SUCCESS/TIMEOUT`: `IMAGE=dist-arm-linux` [travis](https://travis-ci.org/rust-lang/rust/jobs/206514540) (passes locally)
* `SUCCESS/TIMEOUT`: `IMAGE=dist-armv7-aarch64-linux` [travis](https://travis-ci.org/rust-lang/rust/jobs/206514541) (passes locally)
* `SUCCESS`: `IMAGE=arm-android` (fixed)
* `SUCCESS`: `IMAGE=cross`
* `SUCCESS`: `IMAGE=linux-tested-targets`
* `SUCCESS`: `IMAGE=dist-android`
* `SUCCESS`: `IMAGE=dist-freebsd` (fixed)
* `SUCCESS`: `IMAGE=dist-fuchsia`
* `SUCCESS`: `IMAGE=dist-mips-linux`
* `SUCCESS`: `IMAGE=dist-mips64-linux`
* `SUCCESS`: `IMAGE=dist-powerpc-linux`
* `SUCCESS`: `IMAGE=dist-s390x-linux-netbsd` (fixed)
* `SUCCESS`: `IMAGE=dist-x86-linux` (fixed)
* `SUCCESS`: `IMAGE=i686-gnu`
* `SUCCESS`: `IMAGE=i686-gnu-nopt`
* `SUCCESS`: `IMAGE=x86_64-gnu-llvm-3.7`
* `SUCCESS`: `IMAGE=x86_64-gnu`
* `SUCCESS`: `IMAGE=x86_64-gnu-aux`
* `SUCCESS`: `IMAGE=x86_64-gnu-debug`
* `SUCCESS`: `IMAGE=x86_64-gnu-nopt`
* `SUCCESS`: `IMAGE=x86_64-gnu-full-bootstrap`
* `SUCCESS`: `IMAGE=x86_64-gnu-distcheck`
* `SUCCESS`: `IMAGE=x86_64-gnu-incremental`
bors added a commit that referenced this issue Apr 17, 2017
LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [x] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [x] cleanup `.gitmodules`
* [ ] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

* Travis (full build) is green
* Local Windows build was green
* Appveyor Windows build segfaults while running `llvm-tablegen`, investigating.
bors added a commit that referenced this issue Apr 18, 2017
LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [x] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [x] cleanup `.gitmodules`
* [ ] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

* Travis (full build) is green
* Local Windows build was green
* Appveyor Windows build segfaults while running `llvm-tablegen`, investigating.
bors added a commit that referenced this issue Apr 24, 2017
LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [x] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [x] cleanup `.gitmodules`
* [ ] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

* Travis (full build) is green
* Local Windows build was green
* Appveyor Windows build segfaults while running `llvm-tablegen`, investigating.
bors added a commit that referenced this issue Apr 24, 2017
LLVM 4.0 Upgrade

Since nobody has done this yet, I decided to get things started:

**Todo:**

* [x] push the relevant commits to `rust-lang/llvm` and `rust-lang/compiler-rt`
* [x] cleanup `.gitmodules`
* [x] Verify if there are any other commits from `rust-lang/llvm` which need backporting
* [x] Investigate / fix debuginfo ("`<optimized out>`") failures
* [x] Use correct emscripten version in docker image

---

Closes #37609.

---

**Test results:**

Everything is green 🎉
@dsvensson
Copy link

dsvensson commented Apr 25, 2017

@shepmaster @dylanmckay So whats the next issue(s) to subscribe to in the race-to-AVR? :)

@dylanmckay
Copy link
Contributor

As @shepmaster says on reddit, we can integrate the latest Rust master into the avr-rust compiler. After that, we can spend some time getting libcore compiling for AVR.

Once that is working, we could then start looking into upstreaming the AVR target to Rust.

@shepmaster
Copy link
Member Author

So whats the next issue(s) to subscribe to in the race-to-AVR?

AVR-specific issues can be raised / tracked in https://github.com/avr-rust/rust

For example, there are two that I'm aware of right now:

@BatmanAoD
Copy link
Member

#39782 has been closed, so that can be checked off in the top post.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-LLVM Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues. C-enhancement Category: An issue proposing an enhancement or a PR with one. E-hard Call for participation: Hard difficulty. Experience needed to fix: A lot. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging a pull request may close this issue.