-
Notifications
You must be signed in to change notification settings - Fork 29.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
doc,meta: add issue template for flaky tests #24450
Conversation
@refack sadly an error occured when I tried to trigger a build :( |
|
ac7a496
to
2d5284d
Compare
BTW: this is also an option, with the long URIs out of line (👍 / 👎 your opinion)
|
/cc @Trott who opens most of those issues |
I'm -0 on this.
Still, just a -0, not blocking or anything. If you put it in, I'll try to use it. If it causes significant problems (unlikely), we can remove it. If it helps, great. |
The name/description might make people think that it's about unreliable tests they experience on their local machines (and maybe that still qualifies?) but the information requested seems to assume CI. Maybe that's OK, but maybe it's not, so I'm just pointing it out.... |
``` | ||
CI console output for the failing test. | ||
Or content of JUnit test details page. | ||
Note: should be indented with 2 spaces to align with * bullet point. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd remove this. Having to indent output by two spaces can be annoying/tedious. It's OK if it's not indented (or does that cause a big problem--seems mostly aesthetics)?
Yeah, that might be a good thing.
I'm wired to like structure, but if we don't have consensus ¯_(ツ)_/¯ |
New template for reporting flaky tests. Lists the data point relevant to later investigate the issue (i.e. CI-job, CI-worker, link to test code, console output).
Examples:
#24449
#24403
#24397
Checklist
make -j4 test
(UNIX), orvcbuild test
(Windows) passes