Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[draft] Forward port "Soft disable incremental" to 1.53 #86301

Closed

Conversation

pnkfelix
Copy link
Member

This disables incremental (i.e., -Cincremental) taking effect unless an environment variable, RUSTC_FORCE_INCREMENTAL, is set to 1 in the environment of the running rustc. Currently incremental causes errors for many users, and we do not have an expectation of being able to quickly fix these errors in a backportable way - so, for now, disable incremental entirely. The user can still opt-in, but this way the majority of users merely get slower builds, not broken builds.

(Cherry-pick of 4e62503 )

pietroalbini and others added 30 commits May 4, 2021 16:22
Revert "Add missing brace"

This reverts commit 85ad773.

Revert "Simplify base_expr"

This reverts commit 899aae4.

Revert "Warn write-only fields"

This reverts commit d3c69a4.
That PR caused multiple test failures when Rust's channel is changed
from nightly to anything else. The commit will have to be landed again
after the test suite is fixed.
…crum

[beta] Prepare beta 1.53.0

Along with updating the channel and updating the bootstrap compiler, this also includes:

* 692454d - Fixes the diagnostics difference we had for some cycles already
* 9157430 - Disables the known broken RLS builder on AArch64 Windows
* rust-lang#84183 - Update RELEASES.md for 1.52.0
* rust-lang#84950 - Revert PR 83866
* rust-lang#84886 - Update RLS and Rustfmt

Changes we backported to 1.52.0 that we also need to include in 1.53.0 to avoid regressing:

* rust-lang#84867 - rustdoc: revert deref recur to resume inclusion of impl ExtTrait<Local> for ExtType
* rust-lang#84759 - [beta] remove assert_matches
* rust-lang#83171 - Revert PR 81473 to resolve (on beta) issues 81626 and 81658.

r? `@ghost`
…ulacrum

[beta] backports

First-ish round of beta backports:

*  [beta] backport for rust-lang#84769 rust-lang#84969
*  [beta] Bump stage0 to production 1.52.0 rust-lang#84994
*  Deduplicate ParamCandidates with the same value except for bound vars rust-lang#84559
Backport of rust-lang#7170 to beta

r? `@ghost`

changelog: none (fixes stack overflow, but this was introduced in this release cycle)

Now actually opened towards the `beta` branch.
The unsoundness is not in Peekable per se, it rather is due to the
interaction between Peekable being able to hold an extra item
and vec::IntoIter's clone implementation shortening the allocation.

An alternative solution would be to change IntoIter's clone implementation
to keep enough spare capacity available.
This also checks the contents and not only the capacity in case IntoIter's clone implementation is changed to add capacity at the end. Extra capacity at the beginning would be needed to make InPlaceIterable work.

Co-authored-by: Giacomo Stevanato <giaco.stevanato@gmail.com>
When encountering a path that can't have generics, do not call
`generics_of`. This would happen when writing something like
`path::this_is_a_mod<const_val>`.

Fix rust-lang#84831.
using allow_internal_unstable (as recommended)

Fixes: rust-lang#84836

```shell
$ ./build/x86_64-unknown-linux-gnu/stage1/bin/rustc     src/test/run-make-fulldeps/coverage/no_cov_crate.rs
error[E0554]: `#![feature]` may not be used on the dev release channel
 --> src/test/run-make-fulldeps/coverage/no_cov_crate.rs:2:1
  |
2 | #![feature(no_coverage)]
  | ^^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to previous error

For more information about this error, try `rustc --explain E0554`.
```
pnkfelix and others added 17 commits June 10, 2021 13:58
…pping`.

Test was added in PR rust-lang#84404.

The intent here is: The `copy`/`copy_overlapping` intrinsics are going through
some flip-flopping now of "are they intrinsics or not". We can achieve the same
effect that the test intended by using `likely`/`unlikely`.
…r=dtolnay

Beta targetted Make copy/copy_nonoverlapping fn's again

beta backport of PR rust-lang#86003

to address issue rust-lang#84297
This addresses a codegen-issue that needs to be fixed upstream in LLVM.
While we wait for the fix, we can disable it.

Verified manually that the outliner is no longer run when
`-Copt-level=z` is specified, and also that you can override this with
`-Cllvm-args=-enable-machine-outliner` if you need it anyway.

A regression test is not really feasible in this instance, given that we
do not have any minimal reproducers.

Fixes rust-lang#85351
Fixes rust-lang#85432

When processing a `#[derive]` or `#[cfg_eval]` attribute, we need to
re-parse our attribute target, which requires flattenting all
`Nonterminals`. However, this caused us to incorrectly gate on a
(flattented) nonterminal in a key-value attribute, which is supposed to
be allowed.

Since we already perform this gating during the initial parse, we
suppress it in `capture_cfg` mode.
Fixes rust-lang#85197

We already preserved the `SyntaxContext` for invalid/dummy spans in the
incremental cache, but we weren't doing the same for crate metadata.
If an invalid (lo/hi from different files) span is written to the
incremental cache, we will decode it with a 'dummy' location, but keep
the original `SyntaxContext`. Since the crate metadata encoder was only
checking for `DUMMY_SP` (dummy location + root `SyntaxContext`),
the metadata encoder would treat it as a normal span, encoding the
`SyntaxContext`. As a result, the final span encoded to the metadata
would change across sessions, even if the crate itself was unchanged.

This PR updates our encoding of spans in the crate metadata to mirror
the encoding of spans into the incremental cache. We now always encode a
`SyntaxContext`, and encode location information for spans with a
non-dummy location.
Removes the implementations that depend on the user-definable trait `Copy`.

Beta backport: Does not modify `vec::IntoIter`.
…ulacrum

[beta] backports

* Disable the machine outliner by default rust-lang#86020
* Fix incorrect gating of nonterminals in key-value attributes rust-lang#85445
* Build crtbegin.o/crtend.o from source code rust-lang#85395
* Bring back x86_64-sun-solaris target to rustup rust-lang#85252
* Preserve SyntaxContext for invalid/dummy spans in crate metadata rust-lang#85211
* [beta] backport: Remove unsound TrustedRandomAccess implementations rust-lang#86222

r? `@Mark-Simulacrum`
…lacrum

Integrate attributes as part of the crate hash

Backport of rust-lang#83901

r? `@Mark-Simulacrum`
This disables incremental (i.e., -Cincremental) taking effect unless an
environment variable, RUSTC_FORCE_INCREMENTAL, is set to 1 in the environment of
the running rustc. Currently incremental causes errors for many users, and we do
not have an expectation of being able to quickly fix these errors in a
backportable way - so, for now, disable incremental entirely. The user can still
opt-in, but this way the majority of users merely get slower builds, not broken
builds.
@rust-highfive
Copy link
Contributor

r? @Mark-Simulacrum

(rust-highfive has picked a reviewer for you, use r? to override)

@rust-highfive
Copy link
Contributor

⚠️ Warning ⚠️

  • Pull requests are usually filed against the master branch for this repo, but this one is against beta. Please double check that you specified the right target!

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 14, 2021
@pnkfelix pnkfelix changed the title Forward port "Soft disable incremental" to 1.53 [draft] Forward port "Soft disable incremental" to 1.53 Jun 14, 2021
@pnkfelix
Copy link
Member Author

pnkfelix commented Jun 14, 2021

(I'm just putting this up so @Mark-Simulacrum has a trivial thing to grab as part of the beta-to-stable promotional rollup they land.)

@cuviper
Copy link
Member

cuviper commented Jun 14, 2021

Should we also include the updated error messaging from #85097? i.e. 572d14d and a0190b5, which I do see in stable (1.52.1) and master but not beta yet.

@rust-log-analyzer
Copy link
Collaborator

The job x86_64-gnu-llvm-10 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
 finished in 0.153 seconds
Check compiletest suite=incremental mode=incremental (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)

running 131 tests
............................................F......F..................................F............. 100/131
...............F...............
Some tests failed in compiletest suite=incremental mode=incremental host=x86_64-unknown-linux-gnu target=x86_64-unknown-linux-gnu

---- [incremental] incremental/hygiene/load_cached_hygiene.rs stdout ----


error in revision `rpass1`: auxiliary build of "/checkout/src/test/incremental/hygiene/auxiliary/cached_hygiene.rs" failed to compile: 
status: exit status: 101
command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/incremental/hygiene/auxiliary/cached_hygiene.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--cfg" "rpass1" "-C" "incremental=/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hygiene/load_cached_hygiene/load_cached_hygiene.inc" "-Z" "incremental-verify-ich" "--error-format" "json" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hygiene/load_cached_hygiene/auxiliary" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "query-dep-graph" "--crate-type" "dylib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/hygiene/load_cached_hygiene/auxiliary"
------------------------------------------

------------------------------------------
stderr:
stderr:
------------------------------------------
thread 'rustc' panicked at 'called `Option::unwrap()` on a `None` value', /checkout/compiler/rustc_query_system/src/dep_graph/graph.rs:422:50

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.
note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.53.0-beta.13 (8dedee855 2021-06-14) running on x86_64-unknown-linux-gnu

note: compiler flags: -Z threads=1 -Z incremental-verify-ich -Z ui-testing -Z deduplicate-diagnostics=no -Z unstable-options -Z query-dep-graph -C incremental -C prefer-dynamic -C rpath -C debuginfo=0 --crate-type dylib
query stack during panic:
end of query stack

------------------------------------------
------------------------------------------


---- [incremental] incremental/issue-39828/issue-39828.rs stdout ----

error in revision `rpass1`: auxiliary build of "/checkout/src/test/incremental/issue-39828/auxiliary/generic.rs" failed to compile: 
status: exit status: 101
command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/incremental/issue-39828/auxiliary/generic.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--cfg" "rpass1" "-C" "incremental=/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/issue-39828/issue-39828/issue-39828.inc" "-Z" "incremental-verify-ich" "--error-format" "json" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/issue-39828/issue-39828/auxiliary" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "query-dep-graph" "--crate-type" "dylib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/issue-39828/issue-39828/auxiliary"
------------------------------------------

------------------------------------------
stderr:
stderr:
------------------------------------------
thread 'rustc' panicked at 'called `Option::unwrap()` on a `None` value', /checkout/compiler/rustc_query_system/src/dep_graph/graph.rs:422:50

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.
note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.53.0-beta.13 (8dedee855 2021-06-14) running on x86_64-unknown-linux-gnu

note: compiler flags: -Z threads=1 -Z incremental-verify-ich -Z ui-testing -Z deduplicate-diagnostics=no -Z unstable-options -Z query-dep-graph -C incremental -C prefer-dynamic -C rpath -C debuginfo=0 --crate-type dylib
query stack during panic:
end of query stack

------------------------------------------
------------------------------------------


---- [incremental] incremental/rlib_cross_crate/b.rs stdout ----

error in revision `rpass1`: auxiliary build of "/checkout/src/test/incremental/rlib_cross_crate/auxiliary/a.rs" failed to compile: 
status: exit status: 101
command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/incremental/rlib_cross_crate/auxiliary/a.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--cfg" "rpass1" "-C" "incremental=/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/rlib_cross_crate/b/b.inc" "-Z" "incremental-verify-ich" "--error-format" "json" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/rlib_cross_crate/b/auxiliary" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "query-dep-graph" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/rlib_cross_crate/b/auxiliary"
------------------------------------------

------------------------------------------
stderr:
stderr:
------------------------------------------
thread 'rustc' panicked at 'called `Option::unwrap()` on a `None` value', /checkout/compiler/rustc_query_system/src/dep_graph/graph.rs:422:50

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.
note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.53.0-beta.13 (8dedee855 2021-06-14) running on x86_64-unknown-linux-gnu

note: compiler flags: -Z threads=1 -Z incremental-verify-ich -Z ui-testing -Z deduplicate-diagnostics=no -Z unstable-options -Z query-dep-graph -C incremental -C rpath -C debuginfo=0
query stack during panic:
end of query stack

------------------------------------------
------------------------------------------


---- [incremental] incremental/type_alias_cross_crate/b.rs stdout ----

error in revision `rpass1`: auxiliary build of "/checkout/src/test/incremental/type_alias_cross_crate/auxiliary/a.rs" failed to compile: 
status: exit status: 101
command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/incremental/type_alias_cross_crate/auxiliary/a.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--cfg" "rpass1" "-C" "incremental=/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/type_alias_cross_crate/b/b.inc" "-Z" "incremental-verify-ich" "--error-format" "json" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/type_alias_cross_crate/b/auxiliary" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "query-dep-graph" "--crate-type" "dylib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental/type_alias_cross_crate/b/auxiliary"
------------------------------------------

------------------------------------------
stderr:
stderr:
------------------------------------------
thread 'rustc' panicked at 'called `Option::unwrap()` on a `None` value', /checkout/compiler/rustc_query_system/src/dep_graph/graph.rs:422:50

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.
note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.53.0-beta.13 (8dedee855 2021-06-14) running on x86_64-unknown-linux-gnu

note: compiler flags: -Z threads=1 -Z incremental-verify-ich -Z ui-testing -Z deduplicate-diagnostics=no -Z unstable-options -Z query-dep-graph -C incremental -C prefer-dynamic -C rpath -C debuginfo=0 --crate-type dylib
query stack during panic:
end of query stack

------------------------------------------
---
test result: FAILED. 127 passed; 4 failed; 0 ignored; 0 measured; 0 filtered out; finished in 9.65s



command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/incremental" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/incremental" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--suite" "incremental" "--mode" "incremental" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-10/bin/FileCheck" "--nodejs" "/usr/bin/node" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python3" "--lldb-python" "/usr/bin/python3" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "10.0.0" "--llvm-components" "aarch64 aarch64asmparser aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils aggressiveinstcombine all all-targets amdgpu amdgpuasmparser amdgpucodegen amdgpudesc amdgpudisassembler amdgpuinfo amdgpuutils analysis arm armasmparser armcodegen armdesc armdisassembler arminfo armutils asmparser asmprinter avr avrasmparser avrcodegen avrdesc avrdisassembler avrinfo binaryformat bitreader bitstreamreader bitwriter bpf bpfasmparser bpfcodegen bpfdesc bpfdisassembler bpfinfo cfguard codegen core coroutines coverage debuginfocodeview debuginfodwarf debuginfogsym debuginfomsf debuginfopdb demangle dlltooldriver dwarflinker engine executionengine frontendopenmp fuzzmutate globalisel hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo instcombine instrumentation interpreter ipo irreader jitlink lanai lanaiasmparser lanaicodegen lanaidesc lanaidisassembler lanaiinfo libdriver lineeditor linker lto mc mca mcdisassembler mcjit mcparser mips mipsasmparser mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmparser msp430codegen msp430desc msp430disassembler msp430info native nativecodegen nvptx nvptxcodegen nvptxdesc nvptxinfo objcarcopts object objectyaml option orcerror orcjit passes perfjitevents powerpc powerpcasmparser powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata remarks riscv riscvasmparser riscvcodegen riscvdesc riscvdisassembler riscvinfo riscvutils runtimedyld scalaropts selectiondag sparc sparcasmparser sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target textapi transformutils vectorize webassembly webassemblyasmparser webassemblycodegen webassemblydesc webassemblydisassembler webassemblyinfo windowsmanifest x86 x86asmparser x86codegen x86desc x86disassembler x86info x86utils xcore xcorecodegen xcoredesc xcoredisassembler xcoreinfo xray" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"


failed to run: /checkout/obj/build/bootstrap/debug/bootstrap --stage 2 test --exclude src/tools/tidy
Build completed unsuccessfully in 0:11:20

@Mark-Simulacrum
Copy link
Member

Should we also include the updated error messaging from #85097? i.e. 572d14d and a0190b5, which I do see in stable (1.52.1) and master but not beta yet.

I think this would be good.

It also looks like CI is (still) broken here.

@Mark-Simulacrum
Copy link
Member

Going to go ahead and close this as I'm manually packaging up the commits.

@bors
Copy link
Collaborator

bors commented Jun 17, 2021

☔ The latest upstream changes (presumably #86226) made this pull request unmergeable. Please resolve the merge conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties.
Projects
None yet
Development

Successfully merging this pull request may close these issues.