Musl host toolchain #58575
Merged
Musl host toolchain #58575
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
(rust_highfive has picked a reviewer for you, use r? to override) |
rust-highfive
added
the
S-waiting-on-review
|
I'm gonna unsubscribe myself from this for now (sorry I don't have time to help debug this), but you can cc me back in when it's ready for review. |
|
No problem, could you ping somebody who you think could help with the tests? |
mati865
force-pushed
the
musl_toolchain
branch
from
65943a5 to
e676f99
Compare
|
The job Click to expand the log.I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
|
Retriggering Travis, hopefully the cache will kick in |
|
The job Click to expand the log.I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
|
Okay caching is not going to work because CI configured like that doesn't have permission to upload the image. I made clean build with latests bunch of hacks on my machine and after 1 hour all the tests passed. @alexcrichton I'm not quite happy with this amount of hacks but that's what I had to do to make it pass the tests. IMO keeping I'm open to anything the team decides. |
mati865
commented
Mar 1, 2019
|
@bors: delegate+ try Let's try running this through (for now please refrain from r+'ing, I'm doing
I agree! I'm certainly not the only one with opinions here, however. |
|
✌️ @mati865 can now approve this pull request |
bors
added a commit
that referenced
this pull request
Mar 1, 2019
WIP Musl host toolchain Based on #55163 and #57359 Depends on #55566 CC #57439 ### How it works Tested compiler made by `dist` on glibc and musl based distributions and verified binaries it produces: * Ubuntu (glibc) - installed it as a target for host toolchain and observed no regressions for static (default) linking, dynamic linking apparently requires musl build libgcc so I didn't test it. * Alpine (musl) - installed as the host toolchain, by default it links statically (executables are portable and work on glibc distributions) but with `-C target-feature=-crt-static` Rust flag it links dynamically (executables require musl built libraries). ### What's debatable It should be decided whether this toolchain should link dynamically or statically when using it on musl distribution. I believe the distributions would prefer dynamic linking but it'd be misleading because `$ARCH-unknown-linux-musl` target links statically on the other hosts. Another problem is using `RUSTFLAGS='-C target-feature=-crt-static'` for dynamic builds which is really uncomfortable. To address both issues I suggest leaving `$ARCH-unknown-linux-musl` static for both host and cross target and introducing "alias triple" `$ARCH-unknown-linux-dynmusl`. It'd be the same as `$ARCH-unknown-linux-musl` (and use the same libraries to avoid duplication) but it'd link dynamically. ### Why it's still WIP (help wanted) I'm having a hard time getting all tests to pass and I'd appreciate help. Non-verbose error: <details> ``` Testing proc_macro stage1 (x86_64-unknown-linux-musl -> x86_64-unknown-linux-musl) Compiling proc_macro v0.0.0 (/checkout/src/libproc_macro) error[E0463]: can't find crate for `std` error[E0463]: can't find crate for `std` error: aborting due to previous error For more information about this error, try `rustc --explain E0463`. error: aborting due to previous error For more information about this error, try `rustc --explain E0463`. [RUSTC-TIMING] proc_macro test:true 0.529 [RUSTC-TIMING] proc_macro test:false 0.530 error: Could not compile `proc_macro`. warning: build failed, waiting for other jobs to finish... error: Could not compile `proc_macro`. To learn more, run the command again with --verbose. command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "test" "--target" "x86_64-unknown-linux-musl" "-j" "16" "--release" "--locked" "--manifest-path" "/checkout/src/libtest/Cargo.toml" "-p" "proc_macro" "--" expected success, got: exit code: 101 failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test --host x86_64-unknown-linux-musl --target x86_64-unknown-linux-musl ``` </details> Verbose error: <details> ``` Testing proc_macro stage1 (x86_64-unknown-linux-musl -> x86_64-unknown-linux-musl) Compiling proc_macro v0.0.0 (/checkout/src/libproc_macro) Running `/checkout/obj/build/bootstrap/debug/rustc --edition=2018 --crate-name proc_macro src/libproc_macro/lib.rs --color never --crate-type lib --emit=dep-info,link -C opt-level=2 -C metadata=09ddd3ecc930ab63 -C extra-filename=-09ddd3ecc930ab63 --out-dir /checkout/obj/build/x86_64-unknown-linux-musl/stage1-test/x86_64-unknown-linux-musl/release/deps --target x86_64-unknown-linux-musl -L dependency=/checkout/obj/build/x86_64-unknown-linux-musl/stage1-test/x86_64-unknown-linux-musl/release/deps -L dependency=/checkout/obj/build/x86_64-unknown-linux-musl/stage1-test/release/deps -C target-feature=-crt-static` Running `/checkout/obj/build/bootstrap/debug/rustc --edition=2018 --crate-name proc_macro src/libproc_macro/lib.rs --color never --emit=dep-info,link -C opt-level=2 --test -C metadata=a564d363930469c8 -C extra-filename=-a564d363930469c8 --out-dir /checkout/obj/build/x86_64-unknown-linux-musl/stage1-test/x86_64-unknown-linux-musl/release/deps --target x86_64-unknown-linux-musl -L dependency=/checkout/obj/build/x86_64-unknown-linux-musl/stage1-test/x86_64-unknown-linux-musl/release/deps -L dependency=/checkout/obj/build/x86_64-unknown-linux-musl/stage1-test/release/deps -C target-feature=-crt-static` error[E0463]: can't find crate for `std` error[E0463]: can't find crate for `std` error: aborting due to previous error For more information about this error, try `rustc --explain E0463`. error: aborting due to previous error For more information about this error, try `rustc --explain E0463`. [RUSTC-TIMING] proc_macro test:false 0.248 error: Could not compile `proc_macro`. Caused by: process didn't exit successfully: `/checkout/obj/build/bootstrap/debug/rustc --edition=2018 --crate-name proc_macro src/libproc_macro/lib.rs --color never --crate-type lib --emit=dep-info,link -C opt-level=2 -C metadata=09ddd3ecc930ab63 -C extra-filename=-09ddd3ecc930ab63 --out-dir /checkout/obj/build/x86_64-unknown-linux-musl/stage1-test/x86_64-unknown-linux-musl/release/deps --target x86_64-unknown-linux-musl -L dependency=/checkout/obj/build/x86_64-unknown-linux-musl/stage1-test/x86_64-unknown-linux-musl/release/deps -L dependency=/checkout/obj/build/x86_64-unknown-linux-musl/stage1-test/release/deps -C target-feature=-crt-static` (exit code: 1) warning: build failed, waiting for other jobs to finish... [RUSTC-TIMING] proc_macro test:true 0.248 error: Could not compile `proc_macro`. Caused by: process didn't exit successfully: `/checkout/obj/build/bootstrap/debug/rustc --edition=2018 --crate-name proc_macro src/libproc_macro/lib.rs --color never --emit=dep-info,link -C opt-level=2 --test -C metadata=a564d363930469c8 -C extra-filename=-a564d363930469c8 --out-dir /checkout/obj/build/x86_64-unknown-linux-musl/stage1-test/x86_64-unknown-linux-musl/release/deps --target x86_64-unknown-linux-musl -L dependency=/checkout/obj/build/x86_64-unknown-linux-musl/stage1-test/x86_64-unknown-linux-musl/release/deps -L dependency=/checkout/obj/build/x86_64-unknown-linux-musl/stage1-test/release/deps -C target-feature=-crt-static` (exit code: 1) command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "test" "--target" "x86_64-unknown-linux-musl" "-j" "16" "--release" "--locked" "--manifest-path" "/checkout/src/libtest/Cargo.toml" "--verbose" "-p" "proc_macro" "--" expected success, got: exit code: 101 failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test --host x86_64-unknown-linux-musl --target x86_64-unknown-linux-musl ``` </details> Whole tests non-verbose output: [rust-tests.log](https://github.com/rust-lang/rust/files/2879945/rust-tests.log) I think the error is because build system (correctly?) tries to use `obj/build/x86_64-unknown-linux-musl/stage1-test/x86_64-unknown-linux-musl/release/deps` which is empty but `obj/build/x86_64-unknown-linux-gnu/stage1-test/x86_64-unknown-linux-musl/release/deps` contains required libs.
alexcrichton
reviewed
Mar 1, 2019
I have already tested artifacts from my local dist builds both as the dynamic host on Alpine
Sure and thanks. |
|
💔 Test failed - checks-travis |
bors
removed
the
S-waiting-on-review
|
The job Click to expand the log.I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
bors
added
the
S-waiting-on-author
|
Let's see if it can finish in time with the cache. @bors try |
|
@bors try It didn't notice previous comment? |
I think bors only responds to commands like that from authorized users.
|
|
@ids1024 I don't quite understand, shouldn't |
|
Ah right. Perhaps it should be |
|
@bors retry It shouldn't matter and bors should reply anything. Let's see if it works now. |
|
@malbarbo oh I didn't know about this, thanks for letting know. @alexcrichton sorry for troubling you again but adding it to the manifest is the only thing left to make it available via |
|
I'm not sure if that's all that's needed, but it's at least required! |
|
Okay, is there any way to test it locally? |
|
There is, yeah, but it's not really documented or easy to set up. It should be fine to send a PR with an update to the manifest builder and then we can test it on the manifests once merged. |
This was referenced Mar 18, 2019
This was referenced Mar 27, 2019
|
@XAMPPRocky I think this should be mentioned in 1.35.0 Release Notes, many users were waiting years for it. |
|
@mati865 Can you explain how this change affects stable users? |
|
@XAMPPRocky before this PR there was no way to use official Rust packages on musl based distributions (like Alpine Linux, Void Linux musl variant and others). The only way to use Rust there was to install package via distribution package manager (which often had issues because of older unpatched LLVM). This change is now it beta and 1.35.0 will be the first native stable version to work on those distributions. It also makes porting Rust to other musl distros easier because they can use native compiler for bootstrapping instead of cross compiling. |
Closed
|
Thanks for the work! Unfortunately, it's not possible to build Any hint why this isn't supported would be very helpful. |
|
@jnbr Because the |
|
Thank you @smaeul. I wasn't aware that |
|
Can't something be done about this default? It should at least be configurable at compile time I guess? |
|
@Keruspe default static linking cannot be changed because of backward compatibility. You can use environment variable posted above by I have few ideas how to make it more convenient but I haven't got yet to test or discuss them with maintainers. |
netbsd-srcmastr
pushed a commit
to NetBSD/pkgsrc
that referenced
this pull request
May 31, 2019
Version 1.35.0 (2019-05-23)
==========================
Language
--------
- [`FnOnce`, `FnMut`, and the `Fn` traits are now implemented for `Box<FnOnce>`,
`Box<FnMut>`, and `Box<Fn>` respectively.][59500]
- [You can now coerce closures into unsafe function pointers.][59580] e.g.
```rust
unsafe fn call_unsafe(func: unsafe fn()) {
func()
}
pub fn main() {
unsafe { call_unsafe(|| {}); }
}
```
Compiler
--------
- [Added the `armv6-unknown-freebsd-gnueabihf` and
`armv7-unknown-freebsd-gnueabihf` targets.][58080]
- [Added the `wasm32-unknown-wasi` target.][59464]
Libraries
---------
- [`Thread` will now show its ID in `Debug` output.][59460]
- [`StdinLock`, `StdoutLock`, and `StderrLock` now implement `AsRawFd`.][59512]
- [`alloc::System` now implements `Default`.][59451]
- [Expanded `Debug` output (`{:#?}`) for structs now has a trailing comma on the
last field.][59076]
- [`char::{ToLowercase, ToUppercase}` now
implement `ExactSizeIterator`.][58778]
- [All `NonZero` numeric types now implement `FromStr`.][58717]
- [Removed the `Read` trait bounds
on the `BufReader::{get_ref, get_mut, into_inner}` methods.][58423]
- [You can now call the `dbg!` macro without any parameters to print the file
and line where it is called.][57847]
- [In place ASCII case conversions are now up to 4× faster.][59283]
e.g. `str::make_ascii_lowercase`
- [`hash_map::{OccupiedEntry, VacantEntry}` now implement `Sync`
and `Send`.][58369]
Stabilized APIs
---------------
- [`f32::copysign`]
- [`f64::copysign`]
- [`RefCell::replace_with`]
- [`RefCell::map_split`]
- [`ptr::hash`]
- [`Range::contains`]
- [`RangeFrom::contains`]
- [`RangeTo::contains`]
- [`RangeInclusive::contains`]
- [`RangeToInclusive::contains`]
- [`Option::copied`]
Cargo
-----
- [You can now set `cargo:rustc-cdylib-link-arg` at build time to pass custom
linker arguments when building a `cdylib`.][cargo/6298] Its usage is highly
platform specific.
Misc
----
- [The Rust toolchain is now available natively for musl based distros.][58575]
[59460]: rust-lang/rust#59460
[59464]: rust-lang/rust#59464
[59500]: rust-lang/rust#59500
[59512]: rust-lang/rust#59512
[59580]: rust-lang/rust#59580
[59283]: rust-lang/rust#59283
[59451]: rust-lang/rust#59451
[59076]: rust-lang/rust#59076
[58778]: rust-lang/rust#58778
[58717]: rust-lang/rust#58717
[58369]: rust-lang/rust#58369
[58423]: rust-lang/rust#58423
[58080]: rust-lang/rust#58080
[57847]: rust-lang/rust#57847
[58575]: rust-lang/rust#58575
[cargo/6298]: rust-lang/cargo#6298
[`f32::copysign`]: https://doc.rust-lang.org/stable/std/primitive.f32.html#method.copysign
[`f64::copysign`]: https://doc.rust-lang.org/stable/std/primitive.f64.html#method.copysign
[`RefCell::replace_with`]: https://doc.rust-lang.org/stable/std/cell/struct.RefCell.html#method.replace_with
[`RefCell::map_split`]: https://doc.rust-lang.org/stable/std/cell/struct.RefCell.html#method.map_split
[`ptr::hash`]: https://doc.rust-lang.org/stable/std/ptr/fn.hash.html
[`Range::contains`]: https://doc.rust-lang.org/std/ops/struct.Range.html#method.contains
[`RangeFrom::contains`]: https://doc.rust-lang.org/std/ops/struct.RangeFrom.html#method.contains
[`RangeTo::contains`]: https://doc.rust-lang.org/std/ops/struct.RangeTo.html#method.contains
[`RangeInclusive::contains`]: https://doc.rust-lang.org/std/ops/struct.RangeInclusive.html#method.contains
[`RangeToInclusive::contains`]: https://doc.rust-lang.org/std/ops/struct.RangeToInclusive.html#method.contains
[`Option::copied`]: https://doc.rust-lang.org/std/option/enum.Option.html#method.copied
Could rust add a newly-named target that uses musl normally? This would save downstream users a lot of of headaches, for example Alpine has to patch it, adding their own target names with Having a general normal musl target would save a lot of work overall, and reduce the chance of bugs and need for more specific documentation due to custom target names. |
2 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Based on #55163 and #57359
Depends on #55566
CC #57439
How it works
Tested compiler made by
diston glibc and musl based distributions and verified binaries it produces:-C target-feature=-crt-staticRust flag it links dynamically (executables require musl built libraries).What's debatable
It should be decided whether this toolchain should link dynamically or statically when using it on musl distribution. I believe the distributions would prefer dynamic linking but it'd be misleading because
$ARCH-unknown-linux-musltarget links statically on the other hosts.Another problem is using
RUSTFLAGS='-C target-feature=-crt-static'for dynamic builds which is really uncomfortable.To address both issues I suggest leaving
$ARCH-unknown-linux-muslstatic for both host and cross target and introducing "alias triple"$ARCH-unknown-linux-dynmusl. It'd be the same as$ARCH-unknown-linux-musl(and use the same libraries to avoid duplication) but it'd link dynamically.### Why it's still WIP (help wanted)
I'm having a hard time getting all tests to pass and I'd appreciate help.
Non-verbose error:
Verbose error:
Whole tests non-verbose output: rust-tests.log
I think the error is because build system (correctly?) tries to use
obj/build/x86_64-unknown-linux-musl/stage1-test/x86_64-unknown-linux-musl/release/depswhich is empty butobj/build/x86_64-unknown-linux-gnu/stage1-test/x86_64-unknown-linux-musl/release/depscontains required libs.