-
Notifications
You must be signed in to change notification settings - Fork 122
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Build Cop: Flaky Tests are hard to track #281
Comments
Idea: If the bot sees that the issue has been opened and closed 2 or more times, add a Then, if the test fails after someone other than the bot closed it, reopen it and say it's up to you to close this. |
Yea, something that reverts to a human having to come in and fix would be nifty. I had 3 the other day that I created a PR to fix and before it was merged 2 of them automatically closed. |
Sorry about that! I'll move forward with the idea. |
If the bot reopens an issue, it marks it as flaky then stops commenting on it and will not close it. If a flaky test is closed (by a human) then fails again, it will be reopened. Fixes #281.
If the bot reopens an issue, it marks it as flaky then stops commenting on it and will not close it. If a flaky test is closed (by a human) then fails again, it will be reopened. Fixes #281.
If the bot reopens an issue, it marks it as flaky then stops commenting on it and will not close it. If a flaky test is closed (by a human) then fails again, it will be reopened. Fixes #281.
If the bot reopens an issue, it marks it as flaky then stops commenting on it and will not close it. If a flaky test is closed (by a human) then fails again, it will be reopened. Fixes #281.
This didn't work for GoogleCloudPlatform/python-docs-samples#2898. |
Another example of this not working: GoogleCloudPlatform/python-docs-samples#2875 |
4dd6255 commit 4dd6255 Author: Tyler Bui-Palsulich <26876514+tbpg@users.noreply.github.com> Date: Fri Feb 14 19:37:55 2020 -0500 feat(Build Cop): handle flaky tests (#285) If the bot reopens an issue, it marks it as flaky then stops commenting on it and will not close it. If a flaky test is closed (by a human) then fails again, it will be reopened. Fixes #281.
Flaky tests seem to get opened, closed, opened, closed, opened, closed by the bot, which makes it relatively difficult to properly track what I'm trying to fix.
One example: GoogleCloudPlatform/python-docs-samples#2910
The text was updated successfully, but these errors were encountered: