-
Notifications
You must be signed in to change notification settings - Fork 12.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
compiletest: Introduce // {check,build,run}-pass
pass modes
#61778
Conversation
(rust_highfive has picked a reviewer for you, use r? to override) |
r? @Mark-Simulacrum @Centril |
I would eventually like to have mandatory |
It feels like in order for #61755 to work well we'd want to actually require that all tests are annotated with all three "modes" (check, build, run), since presumably you could have a test that checks successfully, builds successfully, but should fail when being run. There could be some heuristic that run-pass implies check-pass and build-pass (and build-pass implies check-pass). Otherwise we'd not be able to effectively switch all tests to Other than these concerns r=me on the actual implementation though. |
It's nice to have tests without extra noise. |
Such tests will need to be moved to the (Tests failing with |
Ah, I thought we had already merged the run-fail suite in, okay. |
I suppose it's a question wrt. noise/signal. I think it's more robust wrt. change to do I think |
Would be good to land this so we can unblock |
@bors r=Mark-Simulacrum |
📌 Commit 0886bc4 has been approved by |
compiletest: Introduce `// {check,build,run}-pass` pass modes Pass UI tests now have three modes ``` // check-pass // build-pass // run-pass ``` mirroring equivalent well-known `cargo` commands. `// check-pass` will compile the test skipping codegen (which is expensive and isn't supposed to fail in most cases). `// build-pass` will compile and link the test without running it. `// run-pass` will compile, link and run the test. Tests without a "pass" annotation are still considered "fail" tests. Most UI tests would probably want to switch to `check-pass`. Tests validating codegen would probably want to run the generated code as well and use `run-pass`. `build-pass` should probably be rare (linking tests?). rust-lang#61755 will provide a way to run the tests with any mode, e.g. bump `check-pass` tests to `run-pass` to satisfy especially suspicious people, and be able to make sure that codegen doesn't breaks in some entirely unexpected way. Tests marked with any mode are expected to pass with any other mode, if that's not the case for some legitimate reason, then the test should be made a "fail" test rather than a "pass" test. Perhaps some secondary CI can verify this invariant, but that's not super urgent. `// compile-pass` still works and is equivalent to `build-pass`. Why is `// compile-pass` bad - 1) it gives an impression that the test is only compiled, but not linked, 2) it doesn't mirror a cargo command. It can be removed some time in the future in a separate PR. cc rust-lang#61712
Rollup of 5 pull requests Successful merges: - #61778 (compiletest: Introduce `// {check,build,run}-pass` pass modes) - #62037 (Speed up tidy) - #62052 (submodules: Update clippy from 5a11ed7 to c5d1ecd) - #62070 (Run rustfmt on some libsyntax files) - #62075 (Remove `ast::Guard`) Failed merges: r? @ghost
compiletest: Introduce `// {check,build,run}-pass` pass modes Pass UI tests now have three modes ``` // check-pass // build-pass // run-pass ``` mirroring equivalent well-known `cargo` commands. `// check-pass` will compile the test skipping codegen (which is expensive and isn't supposed to fail in most cases). `// build-pass` will compile and link the test without running it. `// run-pass` will compile, link and run the test. Tests without a "pass" annotation are still considered "fail" tests. Most UI tests would probably want to switch to `check-pass`. Tests validating codegen would probably want to run the generated code as well and use `run-pass`. `build-pass` should probably be rare (linking tests?). #61755 will provide a way to run the tests with any mode, e.g. bump `check-pass` tests to `run-pass` to satisfy especially suspicious people, and be able to make sure that codegen doesn't breaks in some entirely unexpected way. Tests marked with any mode are expected to pass with any other mode, if that's not the case for some legitimate reason, then the test should be made a "fail" test rather than a "pass" test. Perhaps some secondary CI can verify this invariant, but that's not super urgent. `// compile-pass` still works and is equivalent to `build-pass`. Why is `// compile-pass` bad - 1) it gives an impression that the test is only compiled, but not linked, 2) it doesn't mirror a cargo command. It can be removed some time in the future in a separate PR. cc #61712
☀️ Test successful - checks-travis, status-appveyor |
…petrochenkov Add `--pass $mode` to compiletest through `./x.py` Adds a flag `--pass $mode` to compiletest, which is exposed through `./x.py`. When `--pass $mode` is passed, `{check,build,compile,run}-pass` tests will be forced to run under the given `$mode` unless the directive `// ignore-pass` exists in the test file. The modes are explained in rust-lang#61778: - `check` has the same effect as `cargo check` - `build` or `compile` have the same effect as `cargo build` - `run` has the same effect as `cargo run` On my machine, `./x.py -i test src/test/run-pass --stage 1 --pass check` takes 38 seconds whereas it takes 2 min 7 seconds without `--pass check`. cc rust-lang#61712 r? @petrochenkov
…petrochenkov Add `--pass $mode` to compiletest through `./x.py` Adds a flag `--pass $mode` to compiletest, which is exposed through `./x.py`. When `--pass $mode` is passed, `{check,build,compile,run}-pass` tests will be forced to run under the given `$mode` unless the directive `// ignore-pass` exists in the test file. The modes are explained in rust-lang#61778: - `check` has the same effect as `cargo check` - `build` or `compile` have the same effect as `cargo build` - `run` has the same effect as `cargo run` On my machine, `./x.py -i test src/test/run-pass --stage 1 --pass check` takes 38 seconds whereas it takes 2 min 7 seconds without `--pass check`. cc rust-lang#61712 r? @petrochenkov
…petrochenkov Add `--pass $mode` to compiletest through `./x.py` Adds a flag `--pass $mode` to compiletest, which is exposed through `./x.py`. When `--pass $mode` is passed, `{check,build,compile,run}-pass` tests will be forced to run under the given `$mode` unless the directive `// ignore-pass` exists in the test file. The modes are explained in rust-lang#61778: - `check` has the same effect as `cargo check` - `build` or `compile` have the same effect as `cargo build` - `run` has the same effect as `cargo run` On my machine, `./x.py -i test src/test/run-pass --stage 1 --pass check` takes 38 seconds whereas it takes 2 min 7 seconds without `--pass check`. cc rust-lang#61712 r? @petrochenkov
…petrochenkov Add `--pass $mode` to compiletest through `./x.py` Adds a flag `--pass $mode` to compiletest, which is exposed through `./x.py`. When `--pass $mode` is passed, `{check,build,compile,run}-pass` tests will be forced to run under the given `$mode` unless the directive `// ignore-pass` exists in the test file. The modes are explained in rust-lang#61778: - `check` has the same effect as `cargo check` - `build` or `compile` have the same effect as `cargo build` - `run` has the same effect as `cargo run` On my machine, `./x.py -i test src/test/run-pass --stage 1 --pass check` takes 38 seconds whereas it takes 2 min 7 seconds without `--pass check`. cc rust-lang#61712 r? @petrochenkov
Is there an issue tracking deprecation/removal of |
Not to my knowledge but it should be a rather trivial find/replace with |
@Centril I'd figure most examples would want to be |
@cramertj Yes, but it's a risky move to blanket replace |
yeah, just pointing out that we might not want to blanket search-and-replace to |
@cramertj This seems fine; want to file a C-cleanup issue? |
Opened #62277. |
Pass UI tests now have three modes
mirroring equivalent well-known
cargo
commands.// check-pass
will compile the test skipping codegen (which is expensive and isn't supposed to fail in most cases).// build-pass
will compile and link the test without running it.// run-pass
will compile, link and run the test.Tests without a "pass" annotation are still considered "fail" tests.
Most UI tests would probably want to switch to
check-pass
.Tests validating codegen would probably want to run the generated code as well and use
run-pass
.build-pass
should probably be rare (linking tests?).#61755 will provide a way to run the tests with any mode, e.g. bump
check-pass
tests torun-pass
to satisfy especially suspicious people, and be able to make sure that codegen doesn't breaks in some entirely unexpected way.Tests marked with any mode are expected to pass with any other mode, if that's not the case for some legitimate reason, then the test should be made a "fail" test rather than a "pass" test.
Perhaps some secondary CI can verify this invariant, but that's not super urgent.
// compile-pass
still works and is equivalent tobuild-pass
.Why is
// compile-pass
bad - 1) it gives an impression that the test is only compiled, but not linked, 2) it doesn't mirror a cargo command.It can be removed some time in the future in a separate PR.
cc #61712