assertion failed: !are_upstream_rust_objects_already_included(sess) when building rustc_private with monolithic lto

[matthiaskrgr]

EDIT: repo for reproducing: https://github.com/matthiaskrgr/rustc_crashtest_lto , run cargo build --release

rustc --version #rustc 1.23.0-nightly (8b22e70b2 2017-10-31)

git clone https://github.com/rust-lang-nursery/rustfmt
cd rustfmt
git checkout 0af8825eb104e6c7b9444693d583b5fa0bd55ceb

echo "
[profile.release]
opt-level = 3
lto = true
" >> Cargo.toml

RUST_BACKTRACE=full cargo build --release --verbose

crashes rustc:

       Fresh quote v0.3.15
       Fresh utf8-ranges v1.0.0
       Fresh num-traits v0.1.40
       Fresh unicode-xid v0.0.4
       Fresh getopts v0.2.15
       Fresh serde v1.0.16
       Fresh itoa v0.3.4
       Fresh void v1.0.2
       Fresh dtoa v0.4.2
       Fresh diff v0.1.10
       Fresh term v0.4.6
       Fresh regex-syntax v0.4.1
       Fresh unicode-segmentation v1.2.0
       Fresh log v0.3.8
       Fresh lazy_static v0.2.9
       Fresh libc v0.2.32
       Fresh synom v0.11.3
       Fresh toml v0.4.5
       Fresh unreachable v1.0.0
       Fresh serde_json v1.0.4
       Fresh strings v0.1.0
       Fresh memchr v1.0.2
       Fresh syn v0.11.11
       Fresh thread_local v0.3.4
       Fresh aho-corasick v0.6.3
       Fresh serde_derive_internals v0.16.0
       Fresh regex v0.2.2
       Fresh serde_derive v1.0.16
       Fresh env_logger v0.4.3
   Compiling rustfmt-nightly v0.2.13 (file:///home/matthias/vcs/github/rustfmt)
     Running `rustc --crate-name rustfmt src/bin/rustfmt.rs --crate-type bin --emit=dep-info,link -C opt-level=3 -C lto --cfg 'feature="cargo-fmt"' --cfg 'feature="default"' --cfg 'feature="rustfmt-format-diff"' -C metadata=8286bf522b4875a9 -C extra-filename=-8286bf522b4875a9 --out-dir /home/matthias/vcs/github/rustfmt/target/release/deps -L dependency=/home/matthias/vcs/github/rustfmt/target/release/deps --extern serde_derive=/home/matthias/vcs/github/rustfmt/target/release/deps/libserde_derive-2b4ee28cf16ac2a4.so --extern term=/home/matthias/vcs/github/rustfmt/target/release/deps/libterm-752362bbc8237001.rlib --extern serde=/home/matthias/vcs/github/rustfmt/target/release/deps/libserde-45127027bf81d438.rlib --extern log=/home/matthias/vcs/github/rustfmt/target/release/deps/liblog-d09fa7f67c1f577c.rlib --extern diff=/home/matthias/vcs/github/rustfmt/target/release/deps/libdiff-6cc97c0e6df9495d.rlib --extern getopts=/home/matthias/vcs/github/rustfmt/target/release/deps/libgetopts-8ff6434fa2a5d019.rlib --extern unicode_segmentation=/home/matthias/vcs/github/rustfmt/target/release/deps/libunicode_segmentation-6bb2cdd83d97a0ec.rlib --extern serde_json=/home/matthias/vcs/github/rustfmt/target/release/deps/libserde_json-53e4f5d05eed2957.rlib --extern strings=/home/matthias/vcs/github/rustfmt/target/release/deps/libstrings-04c4ec84130f6565.rlib --extern regex=/home/matthias/vcs/github/rustfmt/target/release/deps/libregex-48d942f70d747749.rlib --extern toml=/home/matthias/vcs/github/rustfmt/target/release/deps/libtoml-65d6559cb921e7a7.rlib --extern env_logger=/home/matthias/vcs/github/rustfmt/target/release/deps/libenv_logger-10d3b6fcb2fa4ecb.rlib --extern libc=/home/matthias/vcs/github/rustfmt/target/release/deps/liblibc-2029413d0fb43b31.rlib --extern rustfmt_nightly=/home/matthias/vcs/github/rustfmt/target/release/deps/librustfmt_nightly-13335655e8960ac0.rlib -C target-cpu=native`
error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.23.0-nightly (8b22e70b2 2017-10-31) running on x86_64-unknown-linux-gnu

note: run with `RUST_BACKTRACE=1` for a backtrace

thread 'rustc' panicked at 'assertion failed: !sess.lto()', /checkout/src/librustc_trans/back/link.rs:1287:8
stack backtrace:
   0:     0x7fc18198a153 - std::sys::imp::backtrace::tracing::imp::unwind_backtrace::hf409d569470ae30b
                               at /checkout/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1:     0x7fc1819847f0 - std::sys_common::backtrace::_print::h9f8ff77762968e1c
                               at /checkout/src/libstd/sys_common/backtrace.rs:69
   2:     0x7fc181997473 - std::panicking::default_hook::{{closure}}::h233cc40af697cbfb
                               at /checkout/src/libstd/sys_common/backtrace.rs:58
                               at /checkout/src/libstd/panicking.rs:381
   3:     0x7fc18199717d - std::panicking::default_hook::hefff18022ca24d92
                               at /checkout/src/libstd/panicking.rs:391
   4:     0x7fc181997907 - std::panicking::rust_panic_with_hook::hd94a4492e4561dca
                               at /checkout/src/libstd/panicking.rs:577
   5:     0x7fc17f79a6c1 - std::panicking::begin_panic::h1d4a7052e8a95c5a
   6:     0x7fc17f75d24d - _ZN11rustc_trans4back4link13link_natively17hfbc8890611f67b24E.llvm.C0978D50
   7:     0x7fc17f75879d - rustc_trans::back::link::link_binary::ha0632ae2f8eab4a2
   8:     0x7fc17f770135 - <rustc_trans::LlvmTransCrate as rustc_trans_utils::trans_crate::TransCrate>::link_binary::hef3d77a5e1caaee2
   9:     0x7fc181d5d03b - rustc_driver::driver::compile_input::h6d65afe4a82d280a
  10:     0x7fc181da41a0 - rustc_driver::run_compiler::h6a01af2106f7c680
  11:     0x7fc181d330f2 - _ZN3std10sys_common9backtrace28__rust_begin_short_backtrace17h5665586c1dd72980E.llvm.B78FDE68
  12:     0x7fc1819e0a0e - __rust_maybe_catch_panic
                               at /checkout/src/libpanic_unwind/lib.rs:99
  13:     0x7fc181d4e7a2 - _ZN50_$LT$F$u20$as$u20$alloc..boxed..FnBox$LT$A$GT$$GT$8call_box17hda2c4e140d408872E.llvm.B78FDE68
  14:     0x7fc18199634b - std::sys::imp::thread::Thread::new::thread_start::h024eb26cf106639b
                               at /checkout/src/liballoc/boxed.rs:772
                               at /checkout/src/libstd/sys_common/thread.rs:24
                               at /checkout/src/libstd/sys/unix/thread.rs:90
  15:     0x7fc17bd51739 - start_thread
  16:     0x7fc18165ce7e - clone
  17:                0x0 - <unknown>

error: Could not compile `rustfmt-nightly`.

Caused by:
  process didn't exit successfully: `rustc --crate-name rustfmt src/bin/rustfmt.rs --crate-type bin --emit=dep-info,link -C opt-level=3 -C lto --cfg feature="cargo-fmt" --cfg feature="default" --cfg feature="rustfmt-format-diff" -C metadata=8286bf522b4875a9 -C extra-filename=-8286bf522b4875a9 --out-dir /home/matthias/vcs/github/rustfmt/target/release/deps -L dependency=/home/matthias/vcs/github/rustfmt/target/release/deps --extern serde_derive=/home/matthias/vcs/github/rustfmt/target/release/deps/libserde_derive-2b4ee28cf16ac2a4.so --extern term=/home/matthias/vcs/github/rustfmt/target/release/deps/libterm-752362bbc8237001.rlib --extern serde=/home/matthias/vcs/github/rustfmt/target/release/deps/libserde-45127027bf81d438.rlib --extern log=/home/matthias/vcs/github/rustfmt/target/release/deps/liblog-d09fa7f67c1f577c.rlib --extern diff=/home/matthias/vcs/github/rustfmt/target/release/deps/libdiff-6cc97c0e6df9495d.rlib --extern getopts=/home/matthias/vcs/github/rustfmt/target/release/deps/libgetopts-8ff6434fa2a5d019.rlib --extern unicode_segmentation=/home/matthias/vcs/github/rustfmt/target/release/deps/libunicode_segmentation-6bb2cdd83d97a0ec.rlib --extern serde_json=/home/matthias/vcs/github/rustfmt/target/release/deps/libserde_json-53e4f5d05eed2957.rlib --extern strings=/home/matthias/vcs/github/rustfmt/target/release/deps/libstrings-04c4ec84130f6565.rlib --extern regex=/home/matthias/vcs/github/rustfmt/target/release/deps/libregex-48d942f70d747749.rlib --extern toml=/home/matthias/vcs/github/rustfmt/target/release/deps/libtoml-65d6559cb921e7a7.rlib --extern env_logger=/home/matthias/vcs/github/rustfmt/target/release/deps/libenv_logger-10d3b6fcb2fa4ecb.rlib --extern libc=/home/matthias/vcs/github/rustfmt/target/release/deps/liblibc-2029413d0fb43b31.rlib --extern rustfmt_nightly=/home/matthias/vcs/github/rustfmt/target/release/deps/librustfmt_nightly-13335655e8960ac0.rlib -C target-cpu=native` (exit code: 101)

[matthiaskrgr]

Still crashing with 1.23.0-nightly (aabfed5 2017-11-17) and rustfmt @ abe6eec

[matthiaskrgr]

Alright, the reproduce was shorter than I expected:

cargo new --bin crashtest
cd crashtest
echo "
[profile.release]
lto = true
" >> Cargo.toml

make src/main.rs this:

#![feature(rustc_private)]
extern crate rustc_errors as errors;
fn main() {  println!("Hello, world!"); }

then
cargo build --release
=> crash

cargo 0.24.0-nightly (6529d418d 2017-11-29)
rustc 1.24.0-nightly (bb42071f6 2017-12-01)

[matthiaskrgr]

For convenience I made this a repo: https://github.com/matthiaskrgr/rustc_crashtest_lto (run cargo build --release on this)

[matthiaskrgr]

Still crashing with

rustc 1.25.0-nightly (6828cf901 2018-01-06)
cargo 0.25.0-nightly (a88fbace4 2017-12-29)

[matthiaskrgr]

Had a look at this again.
Still crashing with

rustc 1.25.0-nightly (27a046e93 2018-02-18)
binary: rustc
commit-hash: 27a046e9338fb0455c33b13e8fe28da78212dedc
commit-date: 2018-02-18
host: x86_64-unknown-linux-gnu
release: 1.25.0-nightly
LLVM version: 6.0

This is the code causing the assert

   // Same thing as above, but for dynamic crates instead of static crates.
    fn add_dynamic_crate(cmd: &mut Linker, sess: &Session, cratepath: &Path) {
        // If we're performing LTO, then it should have been previously required
        // that all upstream rust dependencies were available in an rlib format.
        assert!(!is_full_lto_enabled(sess));

        // Just need to tell the linker about where the library lives and
        // what its name is
        let parent = cratepath.parent();
        if let Some(dir) = parent {
            cmd.include_path(&fix_windows_verbatim_for_gcc(dir));
        }
        let filestem = cratepath.file_stem().unwrap().to_str().unwrap();
        cmd.link_rust_dylib(&unlib(&sess.target, filestem),
                            parent.unwrap_or(Path::new("")));
}

The problem might be that by trying to import parts of rustc it is required that we have these parts of rustc compiled as rlib and prepared for the lto which however might not be the case.
Im still curious though why this is not a problem when using thinlto.

[matthiaskrgr]

rustc_private tracking issue: #27812

[matthiaskrgr]

rustc 1.33.0-nightly (9eac38634 2018-12-31)

The ICE is still happening but the backtrace and assert message slightly changed

  Compiling crashtest v0.1.0 (/tmp/crashtest)
thread 'rustc' panicked at 'assertion failed: !are_upstream_rust_objects_already_included(sess)', src/librustc_codegen_llvm/back/link.rs:1402:9
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:70
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:58
             at src/libstd/panicking.rs:200
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:215
   4: rustc::util::common::panic_hook
   5: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:482
   6: std::panicking::begin_panic
   7: rustc_codegen_llvm::back::link::link_natively
   8: <rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_utils::codegen_backend::CodegenBackend>::join_codegen_and_link::{{closure}}
   9: rustc::util::common::time
  10: <rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_utils::codegen_backend::CodegenBackend>::join_codegen_and_link
  11: rustc_driver::driver::compile_input
  12: rustc_driver::run_compiler_with_pool
  13: <scoped_tls::ScopedKey<T>>::set
  14: rustc_driver::run_compiler
  15: <scoped_tls::ScopedKey<T>>::set
  16: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
  17: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:92
  18: <F as alloc::boxed::FnBox<A>>::call_box
  19: std::sys::unix::thread::Thread::new::thread_start
             at /rustc/9eac386342c601b14311b435f2b6d314fc817bb5/src/liballoc/boxed.rs:734
             at src/libstd/sys_common/thread.rs:14
             at src/libstd/sys/unix/thread.rs:81
  20: start_thread
  21: clone
query stack during panic:
end of query stack

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.33.0-nightly (9eac38634 2018-12-31) running on x86_64-unknown-linux-gnu

note: compiler flags: -C opt-level=3 -C lto -C target-cpu=native --crate-type bin

note: some of the compiler flags provided by cargo are hidden

error: Could not compile `crashtest`.

To learn more, run the command again with --verbose.

[Centril]

@matthiaskrgr Does this still reproduce on a current nightly?

[matthiaskrgr]

The minimal example works.
I was able to build clippy with lto=true.
rls got linker errors with lto=true (do we want a ticket about that?)
rustfmt builds
miri gets linker errors as well