Rewrite cts_exe.py to handle match logic #2199
Merged
+169
−67
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
RossBrunton
force-pushed
the
ross/fulltest
branch
from
dc44744
to
e032380
Compare
RossBrunton
force-pushed
the
ross/fulltest
branch
12 times, most recently
from
97756a8
to
49fdac0
Compare
RossBrunton
changed the title
EwanC
approved these changes
Oct 15, 2024
RossBrunton
force-pushed
the
ross/fulltest
branch
from
49fdac0
to
d8e6dd9
Compare
|
Added specific logic to handle |
cts_exe.py has been rewritten to include the logic for handling
.match files. Specifically, it will divide the test space into
two halves; tests that are expected to pass, and those that are
expected to fail.
For the tests expected to pass, it will run them all in the same
gtest invocation with the assumption that it succeeds. For the
tests expected to fail, they will each be ran with individual
gtest invocations. This allows them to freely segfault or abort
without hurting other tests.
In this commit, the match files are (mostly) unchanged, and the
passing and failing tests should be the same. The match file is
treated as a list of failing tests with a few tokens that are
replaced:
* `{{NONDETERMINISTIC}}` ignored, required for compatibility with the
match checker.
* `{{OPT}}` this test may or may not fail. It's still ran seperately,
but doesn't report an error on failure.
* `{{.*}}` replaced with `*`; converts "match" wildcard matches to
"gtest" test name matches.
* `#` and empty lines are ignored and treated as a comment.
* `{{Segmentation` for compatibility, this will cause a failure in
the "excepted success" execution to not count as an error. This
matches the behaviour of the prior match test logic.
Some .match files have been fixed and empty ones have been removed.
If GTEST_OUTPUT is specified, we assume that we are being run in
ctest_parser.py and don't do anything fancy.
RossBrunton
force-pushed
the
ross/fulltest
branch
from
d8e6dd9
to
ff06878
Compare
kbenzie
approved these changes
Oct 17, 2024
|
Doesn't affect intel/llvm, merging without associated commit. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
cts_exe.py has been rewritten to include the logic for handling .match files. Specifically, it will divide the test space into two halves; tests that are expected to pass, and those that are expected to fail.
For the tests expected to pass, it will run them all in the same gtest invocation with the assumption that it succeeds. For the tests expected to fail, they will each be ran with individual gtest invocations. This allows them to freely segfault or abort without hurting other tests.
In this commit, the match files are (mostly) unchanged, and the passing and failing tests should be the same. The match file is treated as a list of failing tests with a few tokens that are replaced:
{{NONDETERMINISTIC}}ignored, required for compatibility with the match checker.{{OPT}}this test may or may not fail. It's still ran seperately, but doesn't report an error on failure.{{.*}}replaced with*; converts "match" wildcard matches to "gtest" test name matches.#and empty lines are ignored and treated as a comment.{{Segmentationfor compatibility, this will cause a failure in the "excepted success" execution to not count as an error. This matches the behaviour of the prior match test logic.Some .match files have been fixed and empty ones have been removed.