-
-
Notifications
You must be signed in to change notification settings - Fork 4.8k
Add test retry logic for flaky tests #8654
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Thanks for opening this issue!
|
@mtrezza I was just looking into doing something like this. Using the test name as an identifier would be the easiest way to go. |
There may be ways; I've looked into this quite extensively with jasmine but I couldn't find a way given its architecture. Hence I published #8655 to document my findings, closed it and kept this issue open, in case someone finds a way in the future. I think I've spent more than a day on it. The challenge is to build something on top of, or integrated into jasmine, so we have the test stats, etc. It may not be difficult to re-run a test, but I couldn't find a way to do that so that jasmine understands what's happening. Another approach that came to my mind was to maybe use the GitHub CI to automatically create a test run per spec file. So that instead of re-running all tests when one fails, only the spec file of the failing test needs to re-run. But this'd need to consider coverage reporting, not spamming our CI dashboard, etc. |
🎉 This change has been released in version 7.3.0-alpha.7 |
🎉 This change has been released in version 7.3.0-beta.1 |
🎉 This change has been released in version 7.3.0 |
New Feature / Enhancement Checklist
Current Limitation
There are still some flaky tests;
ci: failed
labels on READMEFeature / Enhancement Description
Add a test-retry logic on jasmine-level (not GitHub workflow level) specifically for flaky tests.
parse-server/spec/helper.js
Line 422 in 1850be4
Alternatives / Workarounds
Manually re-run tests (status quo)
References
The text was updated successfully, but these errors were encountered: