GD-220: Add support for flaky test handling and retrying their execution #558
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
MikeSchulze
force-pushed
the
GD-220
branch
3 times, most recently
from
2c63f6e to
6cbd756
Compare
# Why We want to rerun failed test by a configured retry count, because of possible random failing tests (flaky) # What introduced new settings - `flaky_check_enable` "Rerun tests on failure and mark them as FLAKY" - `flaky_max_retries` "Sets the number of retries for rerunning a flaky test" added new flaky state to test execution statistics visualize flaky state in the inspector and console
MikeSchulze
commented
Aug 31, 2024
MikeSchulze
commented
Aug 31, 2024
MikeSchulze
force-pushed
the
GD-220
branch
2 times, most recently
from
ec1763e to
4fae803
Compare
MikeSchulze
force-pushed
the
GD-220
branch
5 times, most recently
from
f34cd1f to
20da3e5
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why
We want to rerun failed test by a configured retry count, because of possible random failing tests (flaky)
What
introduced new settings
flaky_check_enable"Rerun tests on failure and mark them as FLAKY"flaky_max_retries"Sets the number of retries for rerunning a flaky test"rework on execution context to introduce test retry behavior
Added flaky status and navigation to the inspector
Added flaky state visualization to the console
Bug fixes
fuzzerinner class injection, there was an issue with using the right object typeadded new flaky state to test execution statistics
visualize flaky state in the inspector and console