Long compilation time due to LLVM creating excessive landing pads

[not-fl3]

Long compilation time (almost infinity on real project) with every nightly versions after 2017-03-18.

I tried this:

git clone https://github.com/not-fl3/incremental_bug
CARGO_INCREMENTAL=1 cargo build --release

I expected to see this happen:
On 2017-03-18 it compiles for less than a second (in 0.59 secs on my machine)

Instead, this happened:

With every version newer than 2017-03-18 it compiles for 389+ seconds on my machine.

Meta

rustc --version --verbose:

rustc 1.19.0-nightly (06fb4d256 2017-04-30)
binary: rustc
commit-hash: 06fb4d25642a3f223db1441972dd5962085cfba1
commit-date: 2017-04-30
host: x86_64-unknown-linux-gnu
release: 1.19.0-nightly
LLVM version: 4.0

Also same behaviour on this versions:
nightly-2017-03-28-x86_64-unknown-linux-gnu
nightly-2017-04-22-x86_64-unknown-linux-gnu
nightly-2017-04-30-x86_64-unknown-linux-gnu

Backtrace:

log of -Z time-asses: https://gist.github.com/not-fl3/8c4ee411fe6c23d4b4abb9a3b6a263d0

[kennytm]

Not reproducible for me on macOS.

$ time RUST_INCREMENTAL=1 cargo build --release
   Compiling incremental_bug v0.0.1 (file:///Users/kennytm/Downloads/incremental_bug)
warning: field is never used: `name`
 --> src/main.rs:9:5
  |
9 |     name : String,
  |     ^^^^^^^^^^^^^
  |
  = note: #[warn(dead_code)] on by default

warning: field is never used: `next`
  --> src/main.rs:11:5
   |
11 |     next : U
   |     ^^^^^^^^
   |
   = note: #[warn(dead_code)] on by default

    Finished release [optimized] target(s) in 1.12 secs

real	0m1.445s
user	0m0.417s
sys	0m0.297s

$ rustc -vV
rustc 1.19.0-nightly (777ee2079 2017-05-01)
binary: rustc
commit-hash: 777ee20796e80a31d4b7c985dd68eda2941460d6
commit-date: 2017-05-01
host: x86_64-apple-darwin
release: 1.19.0-nightly
LLVM version: 4.0

Are you sure nightly-2017-03-18 is fast while nightly-2017-03-19 is slow?

[not-fl3]

@kennytm versions I tried: 2017-03-28, 2017-04-22 and 2017-04-30.

$ cargo build --release
   Compiling incremental_bug v0.0.1 (file:///home/fl3/fun/outrage_test/incremental_bug)
warning: field is never used: `name`
 --> src/main.rs:9:5
  |
9 |     name : String,
  |     ^^^^^^^^^^^^^
  |
  = note: #[warn(dead_code)] on by default

warning: field is never used: `next`
  --> src/main.rs:11:5
   |
11 |     next : U
   |     ^^^^^^^^
   |
   = note: #[warn(dead_code)] on by default

    Finished release [optimized] target(s) in 443.30 secs

rustc -vV
rustc 1.19.0-nightly (06fb4d256 2017-04-30)
binary: rustc
commit-hash: 06fb4d25642a3f223db1441972dd5962085cfba1
commit-date: 2017-04-30
host: x86_64-unknown-linux-gnu
release: 1.19.0-nightly
LLVM version: 4.0

Also tried on different linux boxes - same problem.

[kennytm]

@not-fl3 Could you specifically test with 2017-03-18 and 2017-03-19? You may install it from rustup using rustup default nightly-2017-03-18. Just wanna confirm if it is really 2017-03-19 causing the problem for you.

No repro on my Debian VM either.

$ cargo clean

$ time RUST_INCREMENTAL=1 cargo build --release
   Compiling incremental_bug v0.0.1 (file:///root/incremental_bug)
warning: field is never used: `name`
 --> src/main.rs:9:5
  |
9 |     name : String,
  |     ^^^^^^^^^^^^^
  |
  = note: #[warn(dead_code)] on by default

warning: field is never used: `next`
  --> src/main.rs:11:5
   |
11 |     next : U,
   |     ^^^^^^^^
   |
   = note: #[warn(dead_code)] on by default

    Finished release [optimized] target(s) in 0.51 secs

real	0m0.613s
user	0m0.532s
sys	0m0.052s

$ rustc -vV
rustc 1.19.0-nightly (06fb4d256 2017-04-30)
binary: rustc
commit-hash: 06fb4d25642a3f223db1441972dd5962085cfba1
commit-date: 2017-04-30
host: x86_64-unknown-linux-gnu
release: 1.19.0-nightly
LLVM version: 4.0

[not-fl3]

@kennytm just tried continuously:

rustup override set nightly-2017-03-19
rustup override set nightly-2017-03-20
rustup override set nightly-2017-03-21

And first broken version is 2017-03-21.

rustc 1.17.0-nightly (134c4a0f0 2017-03-20)
binary: rustc
commit-hash: 134c4a0f08a3d1f55ea8968fbe728fa935c71698
commit-date: 2017-03-20
host: x86_64-unknown-linux-gnu
release: 1.17.0-nightly
LLVM version: 3.9

[not-fl3]

@kennytm hmm it seems that on that 03-21 it hangs forever. It works for 20+ minutes and keep working slowly increasing consumed ram.

[kennytm]

The commit for 2017-03-21 is 1.17.0-nightly (134c4a0 2017-03-20)
The commit for 2017-03-20 is 1.17.0-nightly (6eb9960 2017-03-19)

PRs between the two are

$ git log --oneline --merges 6eb9960d3...134c4a0f0
134c4a0f08 Auto merge of #39628 - arielb1:shimmir, r=eddyb
244f893ed7 Auto merge of #40659 - frewsxcv:rollup, r=frewsxcv
6738cd4d47 Auto merge of #40281 - jimmycuadra:try-from-from-str, r=aturon
d49f86901c Rollup merge of #40587 - GuillaumeGomez:rustdoc-const-display, r=frewsxcv
de724ba9a6 Rollup merge of #40581 - TimNN:di-global-40, r=alexcrichton
4e9033124b Rollup merge of #40566 - clarcharr:never_error, r=sfackler
dedf9d3593 Rollup merge of #40554 - nrc:rls-data, r=alexcrichton
7471d9793c Rollup merge of #40532 - jseyfried:improve_tokenstream_quoter, r=nrc
1d1543d566 Rollup merge of #40521 - TimNN:panic-free-shift, r=alexcrichton
6480a00222 Rollup merge of #40398 - eddyb:struct-hint, r=nikomatsakis
573e906594 Rollup merge of #40281 - jimmycuadra:try-from-from-str, r=aturon
e5221f9397 Rollup merge of #40241 - Sawyer47:fix-39997, r=alexcrichton

[nagisa]

Could only reproduce with

cargo +nightly-2017-03-21 rustc --release -- -Zincremental=.

The cause, is of course, the incremental compilation.

[not-fl3]

@kennytm
except RUSTFLAGS typed in this gist I had CARGO_INCREMENTAL=1 in my env.

https://gist.github.com/not-fl3/832b85ec9ff08537e479bb42ed4e2402

[not-fl3]

@kennytm checked your log from debian and found that in original post I used RUST_INCREMENTAL instead of CARGO_INCREMENTAL=1.
My bad, fixed original post now.

[kennytm]

Reduced test case:

// rustc -Copt-level=2 -Zincremental=. main.rs

struct DS<U> {
    name: String,
    next: U,
}

fn add<U>(ds: DS<U>, name: String) -> DS<DS<U>> {
    DS {
        name: "?".to_owned(),
        next: ds,
    }
}

fn main() {
    let deserializers = DS { name: "?".to_owned(), next: () };
    let deserializers = add(deserializers, "?".to_owned());
    let deserializers = add(deserializers, "?".to_owned());
    let deserializers = add(deserializers, "?".to_owned());
    let deserializers = add(deserializers, "?".to_owned());
    let deserializers = add(deserializers, "?".to_owned());
    let deserializers = add(deserializers, "?".to_owned());
    let deserializers = add(deserializers, "?".to_owned()); // 0.7s
    let deserializers = add(deserializers, "?".to_owned()); // 1.3s
    let deserializers = add(deserializers, "?".to_owned()); // 2.4s
    let deserializers = add(deserializers, "?".to_owned()); // 6.7s
    // let deserializers = add(deserializers, "?".to_owned()); // 26.0s
    // let deserializers = add(deserializers, "?".to_owned()); // 114.0s
}

Looks like some exponential blow-up.

---

In the 26s case, most time (51%) is spent on the RegisterCoalescer pass.

[kgv]

Looks like the same problem:

fn main() {
        let t: std::result::Result<(), ()> = Ok(());
        let f = t
            .into_future()
            .and_then(|_| future::ok(()))
            .and_then(|_| future::ok(()))
            .and_then(|_| future::ok(()))
            .and_then(|_| future::ok(()))
            .and_then(|_| future::ok(())) // 0.69 secs
            .and_then(|_| future::ok(()))
            .and_then(|_| future::ok(()))
            .and_then(|_| future::ok(()))
            .and_then(|_| future::ok(())) // 1.94 secs
            .and_then(|_| future::ok(()))
            .and_then(|_| future::ok(()))
            .and_then(|_| future::ok(()))
            .and_then(|_| future::ok(())) // 20.68 secs
            .and_then(|_| future::ok(())) // 40.58 secs
            .and_then(|_| future::ok(())) // 80.70 secs
            .and_then(|_| future::ok(())) // 160.59 secs
            .and_then(|_| future::ok(())) // 321.88 secs
            .and_then(|_| future::ok(())); // 642.42 secs 
        f.wait();
}

run: cargo run --bin=test_bin
(future - from https://crates.io/crates/futures)

rustc -vV
rustc 1.18.0-nightly (94e884b63 2017-04-27)
binary: rustc
commit-hash: 94e884b6307a59f1e6e64aa7ebc1996b651a7629
commit-date: 2017-04-27
host: x86_64-pc-windows-msvc
release: 1.18.0-nightly
LLVM version: 4.0

[kennytm]

@kgv Not really the same issue. OP's code is slow to compile due to an LLVM pass, your code is slow to compile due to MIR optimization (specifically the "translation item collection" pass).