libtest: add a --skip flag to the test runner #36604
Conversation
This flag takes a FILTER argument and instructs the test runner to skip the tests whose names contain the word FILTER. --skip can be used several times.
|
Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @alexcrichton (or someone else) soon. If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes. Please see the contribution instructions for more information. |
|
Looks good to me! Perhaps a test could be added as well? cc @rust-lang/tools thoughts on semantics, naming, and/or just the feature itself? |
|
LGTM! |
|
sgtm. I have other cases where this would be useful, similar to japaric's: someday we'll have std factored so that it can be implemented out of tree, and with its test suite in another crate, and those out-of-tree consumers would need a way to ignore tests for their platforms. |
brson
added
the
relnotes
|
@bors: r+ |
|
📌 Commit adf85d3 has been approved by |
|
⌛ Testing commit adf85d3 with merge 4eab0e4... |
|
💔 Test failed - auto-win-gnu-32-opt-rustbuild |
|
@bors r=alexcrichton |
|
📌 Commit 45916ec has been approved by |
libtest: add a --skip flag to the test runner This flag takes a FILTER argument and instructs the test runner to skip the tests whose names contain the word FILTER. --skip can be used several times. --- My motivation for submitting this is that while using [smoke] to run `std` unit tests for cross targets I found that a few of the tests always fail due to limitations in QEMU (it can't handle too many threads) and I'd like to skip these problematic tests from the command line to be able to run the rest of the unit tests. [smoke]: https://github.com/japaric/smoke I know there is another mechanism to skip tests: `#[ignore]` but this doesn't work in my use case because I can't (easily) modify the source of the standard libraries to `#[ignore]` some tests. And even if I could, the change would involve conditionally ignoring some tests for some targets but that's not a perfect solution because those tests should pass if executed on real hardware so they should not be `#[ignored]` in that scenario. r? @alexcrichton cc @brson
This flag takes a FILTER argument and instructs the test runner to skip
the tests whose names contain the word FILTER. --skip can be used
several times.
My motivation for submitting this is that while using smoke to run
stdunit tests for crosstargets I found that a few of the tests always fail due to limitations in QEMU (it can't handle too
many threads) and I'd like to skip these problematic tests from the command line to be able to run
the rest of the unit tests.
I know there is another mechanism to skip tests:
#[ignore]but this doesn't work in my use casebecause I can't (easily) modify the source of the standard libraries to
#[ignore]some tests. Andeven if I could, the change would involve conditionally ignoring some tests for some targets but
that's not a perfect solution because those tests should pass if executed on real hardware so they
should not be
#[ignored]in that scenario.r? @alexcrichton
cc @brson