Skip to content

Commit d69b248

Browse files
committed
rust: Import LLD for linking wasm objects
This commit imports the LLD project from LLVM to serve as the default linker for the `wasm32-unknown-unknown` target. The `binaryen` submoule is consequently removed along with "binaryen linker" support in rustc. Moving to LLD brings with it a number of benefits for wasm code: * LLD is itself an actual linker, so there's no need to compile all wasm code with LTO any more. As a result builds should be *much* speedier as LTO is no longer forcibly enabled for all builds of the wasm target. * LLD is quickly becoming an "official solution" for linking wasm code together. This, I believe at least, is intended to be the main supported linker for native code and wasm moving forward. Picking up support early on should help ensure that we can help LLD identify bugs and otherwise prove that it works great for all our use cases! * Improvements to the wasm toolchain are currently primarily focused around LLVM and LLD (from what I can tell at least), so it's in general much better to be on this bandwagon for bugfixes and new features. * Historical "hacks" like `wasm-gc` will soon no longer be necessary, LLD will [natively implement][gc] `--gc-sections` (better than `wasm-gc`!) which means a postprocessor is no longer needed to show off Rust's "small wasm binary size". LLD is added in a pretty standard way to rustc right now. A new rustbuild target was defined for building LLD, and this is executed when a compiler's sysroot is being assembled. LLD is compiled against the LLVM that we've got in tree, which means we're currently on the `release_60` branch, but this may get upgraded in the near future! LLD is placed into rustc's sysroot in a `bin` directory. This is similar to where `gcc.exe` can be found on Windows. This directory is automatically added to `PATH` whenever rustc executes the linker, allowing us to define a `WasmLd` linker which implements the interface that `wasm-ld`, LLD's frontend, expects. Like Emscripten the LLD target is currently only enabled for Tier 1 platforms, notably OSX/Windows/Linux, and will need to be installed manually for compiling to wasm on other platforms. LLD is by default turned off in rustbuild, and requires a `config.toml` option to be enabled to turn it on. Finally the unstable `#![wasm_import_memory]` attribute was also removed as LLD has a native option for controlling this. [gc]: https://reviews.llvm.org/D42511
1 parent 0be38e1 commit d69b248

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

41 files changed

+408
-745
lines changed

.gitmodules

+3-3
Original file line numberDiff line numberDiff line change
@@ -41,9 +41,6 @@
4141
[submodule "src/dlmalloc"]
4242
path = src/dlmalloc
4343
url = https://github.com/alexcrichton/dlmalloc-rs.git
44-
[submodule "src/binaryen"]
45-
path = src/binaryen
46-
url = https://github.com/alexcrichton/binaryen.git
4744
[submodule "src/doc/rust-by-example"]
4845
path = src/doc/rust-by-example
4946
url = https://github.com/rust-lang/rust-by-example
@@ -53,3 +50,6 @@
5350
[submodule "src/stdsimd"]
5451
path = src/stdsimd
5552
url = https://github.com/rust-lang-nursery/stdsimd
53+
[submodule "src/tools/lld"]
54+
path = src/tools/lld
55+
url = https://github.com/rust-lang/lld.git

.travis.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ matrix:
8181
# OSX 10.7 and `xcode7` is the latest Xcode able to compile LLVM for 10.7.
8282
- env: >
8383
RUST_CHECK_TARGET=dist
84-
RUST_CONFIGURE_ARGS="--build=i686-apple-darwin --enable-extended --enable-profiler --enable-emscripten"
84+
RUST_CONFIGURE_ARGS="--build=i686-apple-darwin --enable-full-tools --enable-profiler"
8585
SRC=.
8686
DEPLOY=1
8787
RUSTC_RETRY_LINKER_ON_SEGFAULT=1
@@ -95,7 +95,7 @@ matrix:
9595
9696
- env: >
9797
RUST_CHECK_TARGET=dist
98-
RUST_CONFIGURE_ARGS="--target=aarch64-apple-ios,armv7-apple-ios,armv7s-apple-ios,i386-apple-ios,x86_64-apple-ios --enable-extended --enable-sanitizers --enable-profiler --enable-emscripten"
98+
RUST_CONFIGURE_ARGS="--target=aarch64-apple-ios,armv7-apple-ios,armv7s-apple-ios,i386-apple-ios,x86_64-apple-ios --enable-full-tools --enable-sanitizers --enable-profiler"
9999
SRC=.
100100
DEPLOY=1
101101
RUSTC_RETRY_LINKER_ON_SEGFAULT=1

README.md

-3
Original file line numberDiff line numberDiff line change
@@ -129,9 +129,6 @@ CALL "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64\vcvars64.
129129
python x.py build
130130
```
131131

132-
If you are seeing build failure when compiling `rustc_binaryen`, make sure the path
133-
length of the rust folder is not longer than 22 characters.
134-
135132
#### Specifying an ABI
136133
[specifying-an-abi]: #specifying-an-abi
137134

appveyor.yml

+4-6
Original file line numberDiff line numberDiff line change
@@ -67,29 +67,27 @@ environment:
6767
# 32/64 bit MSVC and GNU deployment
6868
- RUST_CONFIGURE_ARGS: >
6969
--build=x86_64-pc-windows-msvc
70-
--enable-extended
70+
--enable-full-tools
7171
--enable-profiler
72-
--enable-emscripten
7372
SCRIPT: python x.py dist
7473
DEPLOY: 1
7574
- RUST_CONFIGURE_ARGS: >
7675
--build=i686-pc-windows-msvc
7776
--target=i586-pc-windows-msvc
78-
--enable-extended
77+
--enable-full-tools
7978
--enable-profiler
80-
--enable-emscripten
8179
SCRIPT: python x.py dist
8280
DEPLOY: 1
8381
- MSYS_BITS: 32
84-
RUST_CONFIGURE_ARGS: --build=i686-pc-windows-gnu --enable-extended --enable-emscripten
82+
RUST_CONFIGURE_ARGS: --build=i686-pc-windows-gnu --enable-full-tools
8583
SCRIPT: python x.py dist
8684
MINGW_URL: https://s3-us-west-1.amazonaws.com/rust-lang-ci2/rust-ci-mirror
8785
MINGW_ARCHIVE: i686-6.3.0-release-posix-dwarf-rt_v5-rev2.7z
8886
MINGW_DIR: mingw32
8987
DEPLOY: 1
9088
- MSYS_BITS: 64
9189
SCRIPT: python x.py dist
92-
RUST_CONFIGURE_ARGS: --build=x86_64-pc-windows-gnu --enable-extended --enable-emscripten
90+
RUST_CONFIGURE_ARGS: --build=x86_64-pc-windows-gnu --enable-full-tools
9391
MINGW_URL: https://s3-us-west-1.amazonaws.com/rust-lang-ci2/rust-ci-mirror
9492
MINGW_ARCHIVE: x86_64-6.3.0-release-posix-seh-rt_v5-rev2.7z
9593
MINGW_DIR: mingw64

config.toml.example

+4
Original file line numberDiff line numberDiff line change
@@ -329,6 +329,10 @@
329329
# target, as without this option the test output will not be captured.
330330
#wasm-syscall = false
331331

332+
# Indicates whether LLD will be compiled and made available in the sysroot for
333+
# rustc to execute.
334+
#lld = false
335+
332336
# =============================================================================
333337
# Options for specific targets
334338
#

src/Cargo.lock

-10
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

src/binaryen

-1
This file was deleted.

src/bootstrap/bin/rustc.rs

+1-1
Original file line numberDiff line numberDiff line change
@@ -224,7 +224,7 @@ fn main() {
224224
// flesh out rpath support more fully in the future.
225225
cmd.arg("-Z").arg("osx-rpath-install-name");
226226
Some("-Wl,-rpath,@loader_path/../lib")
227-
} else if !target.contains("windows") {
227+
} else if !target.contains("windows") && !target.contains("wasm32") {
228228
Some("-Wl,-rpath,$ORIGIN/../lib")
229229
} else {
230230
None

src/bootstrap/bootstrap.py

+4
Original file line numberDiff line numberDiff line change
@@ -641,6 +641,10 @@ def update_submodules(self):
641641
continue
642642
if self.get_toml('jemalloc'):
643643
continue
644+
if module.endswith("lld"):
645+
config = self.get_toml('lld')
646+
if config is None or config == 'false':
647+
continue
644648
filtered_submodules.append(module)
645649
run(["git", "submodule", "update",
646650
"--init", "--recursive"] + filtered_submodules,

src/bootstrap/builder.rs

+1-1
Original file line numberDiff line numberDiff line change
@@ -316,7 +316,7 @@ impl<'a> Builder<'a> {
316316
tool::UnstableBookGen, tool::Tidy, tool::Linkchecker, tool::CargoTest,
317317
tool::Compiletest, tool::RemoteTestServer, tool::RemoteTestClient,
318318
tool::RustInstaller, tool::Cargo, tool::Rls, tool::Rustdoc, tool::Clippy,
319-
native::Llvm, tool::Rustfmt, tool::Miri),
319+
native::Llvm, tool::Rustfmt, tool::Miri, native::Lld),
320320
Kind::Check => describe!(check::Std, check::Test, check::Rustc),
321321
Kind::Test => describe!(test::Tidy, test::Bootstrap, test::Ui, test::RunPass,
322322
test::CompileFail, test::ParseFail, test::RunFail, test::RunPassValgrind,

src/bootstrap/cc_detect.rs

+3
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,9 @@ pub fn find(build: &mut Build) {
7979
let mut cfg = cc::Build::new();
8080
cfg.cargo_metadata(false).opt_level(0).warnings(false).debug(false)
8181
.target(&target).host(&build.build);
82+
if target.contains("msvc") {
83+
cfg.static_crt(true);
84+
}
8285

8386
let config = build.config.target_config.get(&target);
8487
if let Some(cc) = config.and_then(|c| c.cc.as_ref()) {

src/bootstrap/compile.rs

+26
Original file line numberDiff line numberDiff line change
@@ -747,6 +747,21 @@ fn copy_codegen_backends_to_sysroot(builder: &Builder,
747747
}
748748
}
749749

750+
fn copy_lld_to_sysroot(builder: &Builder,
751+
target_compiler: Compiler,
752+
lld_install_root: &Path) {
753+
let target = target_compiler.host;
754+
755+
let dst = builder.sysroot_libdir(target_compiler, target)
756+
.parent()
757+
.unwrap()
758+
.join("bin");
759+
t!(fs::create_dir_all(&dst));
760+
761+
let exe = exe("lld", &target);
762+
copy(&lld_install_root.join("bin").join(&exe), &dst.join(&exe));
763+
}
764+
750765
/// Cargo's output path for the standard library in a given stage, compiled
751766
/// by a particular compiler for the specified target.
752767
pub fn libstd_stamp(build: &Build, compiler: Compiler, target: Interned<String>) -> PathBuf {
@@ -896,6 +911,14 @@ impl Step for Assemble {
896911
}
897912
}
898913

914+
let lld_install = if build.config.lld_enabled && target_compiler.stage > 0 {
915+
Some(builder.ensure(native::Lld {
916+
target: target_compiler.host,
917+
}))
918+
} else {
919+
None
920+
};
921+
899922
let stage = target_compiler.stage;
900923
let host = target_compiler.host;
901924
println!("Assembling stage{} compiler ({})", stage, host);
@@ -915,6 +938,9 @@ impl Step for Assemble {
915938
copy_codegen_backends_to_sysroot(builder,
916939
build_compiler,
917940
target_compiler);
941+
if let Some(lld_install) = lld_install {
942+
copy_lld_to_sysroot(builder, target_compiler, &lld_install);
943+
}
918944

919945
// Link the compiler binary itself into place
920946
let out_dir = build.cargo_out(build_compiler, Mode::Librustc, host);

src/bootstrap/config.rs

+4
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,8 @@ pub struct Config {
8181
pub llvm_experimental_targets: String,
8282
pub llvm_link_jobs: Option<u32>,
8383

84+
pub lld_enabled: bool,
85+
8486
// rust codegen options
8587
pub rust_optimize: bool,
8688
pub rust_codegen_units: Option<u32>,
@@ -292,6 +294,7 @@ struct Rust {
292294
codegen_backends: Option<Vec<String>>,
293295
codegen_backends_dir: Option<String>,
294296
wasm_syscall: Option<bool>,
297+
lld: Option<bool>,
295298
}
296299

297300
/// TOML representation of how each build target is configured.
@@ -480,6 +483,7 @@ impl Config {
480483
set(&mut config.quiet_tests, rust.quiet_tests);
481484
set(&mut config.test_miri, rust.test_miri);
482485
set(&mut config.wasm_syscall, rust.wasm_syscall);
486+
set(&mut config.lld_enabled, rust.lld);
483487
config.rustc_parallel_queries = rust.experimental_parallel_queries.unwrap_or(false);
484488
config.rustc_default_linker = rust.default_linker.clone();
485489
config.musl_root = rust.musl_root.clone().map(PathBuf::from);

src/bootstrap/configure.py

+5
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,7 @@ def v(*args):
6666
o("cargo-openssl-static", "build.openssl-static", "static openssl in cargo")
6767
o("profiler", "build.profiler", "build the profiler runtime")
6868
o("emscripten", None, "compile the emscripten backend as well as LLVM")
69+
o("full-tools", None, "enable all tools")
6970

7071
# Optimization and debugging options. These may be overridden by the release
7172
# channel, etc.
@@ -326,6 +327,10 @@ def set(key, value):
326327
set('build.target', value.split(','))
327328
elif option.name == 'emscripten':
328329
set('rust.codegen-backends', ['llvm', 'emscripten'])
330+
elif option.name == 'full-tools':
331+
set('rust.codegen-backends', ['llvm', 'emscripten'])
332+
set('rust.lld', True)
333+
set('build.extended', True)
329334
elif option.name == 'option-checking':
330335
# this was handled above
331336
pass

src/bootstrap/dist.rs

+17-1
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ use build_helper::output;
2828

2929
use {Build, Compiler, Mode};
3030
use channel;
31-
use util::{cp_r, libdir, is_dylib, cp_filtered, copy, replace_in_file};
31+
use util::{cp_r, libdir, is_dylib, cp_filtered, copy, replace_in_file, exe};
3232
use builder::{Builder, RunConfig, ShouldRun, Step};
3333
use compile;
3434
use native;
@@ -443,6 +443,22 @@ impl Step for Rustc {
443443
t!(fs::create_dir_all(&backends_dst));
444444
cp_r(&backends_src, &backends_dst);
445445

446+
// Copy over lld if it's there
447+
if builder.config.lld_enabled {
448+
let exe = exe("lld", &compiler.host);
449+
let src = builder.sysroot_libdir(compiler, host)
450+
.parent()
451+
.unwrap()
452+
.join("bin")
453+
.join(&exe);
454+
let dst = image.join("lib/rustlib")
455+
.join(&*host)
456+
.join("bin")
457+
.join(&exe);
458+
t!(fs::create_dir_all(&dst.parent().unwrap()));
459+
copy(&src, &dst);
460+
}
461+
446462
// Man pages
447463
t!(fs::create_dir_all(image.join("share/man/man1")));
448464
let man_src = build.src.join("src/doc/man");

src/bootstrap/lib.rs

+7-1
Original file line numberDiff line numberDiff line change
@@ -501,6 +501,10 @@ impl Build {
501501
self.out.join(&*target).join("llvm-emscripten")
502502
}
503503

504+
fn lld_out(&self, target: Interned<String>) -> PathBuf {
505+
self.out.join(&*target).join("lld")
506+
}
507+
504508
/// Output directory for all documentation for a target
505509
fn doc_out(&self, target: Interned<String>) -> PathBuf {
506510
self.out.join(&*target).join("doc")
@@ -685,7 +689,9 @@ impl Build {
685689
.and_then(|c| c.linker.as_ref()) {
686690
Some(linker)
687691
} else if target != self.config.build &&
688-
!target.contains("msvc") && !target.contains("emscripten") {
692+
!target.contains("msvc") &&
693+
!target.contains("emscripten") &&
694+
!target.contains("wasm32") {
689695
Some(self.cc(target))
690696
} else {
691697
None

0 commit comments

Comments
 (0)