Repeat tests until it passes once, temporary solution to flaky tests #4575
Unanswered
s-dakshin
asked this question in
Community Help
Replies: 1 comment
-
I'd welcome such feature also |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Goal: I am trying to solve flakiness in UI tests in our custom test framework using Google Test. Such that I can re-run tests until they pass at least once. Like have them repeat 3 times and if they pass even once, mark the test as passed. The opposite of this is possible with the
::testing::GTEST_FLAG(repeat) = 3;
What I am trying to achieve is based on a blog post - https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html. Where the suggestions to flaky tests are -
Based on this, it seems like what I am trying to achieve should be possible. But based on the things I tried, here are my findings -
testing::FLAGS_gtest_repeat
- seems to be used to run tests until it fails to help with debugging. I almost want the inverse of this.testing::TestPartResult
- seems to have a way to find if fatal or non-fatal errors occurred in your test. But there is no way to clear these flags or handle it inside our code. More precisely,::testing::Test::HasFailure()
is set to failure even if a test fails once, I want something liketesting::TestResult::ClearTestPartResults()
, but that is a private function.Can someone please help/suggest a way to achieve my goal using existing functions/flags that Google Test framework offers?
Beta Was this translation helpful? Give feedback.
All reactions