|
1 | 1 | # Test Triage at AdoptOpenJDK
|
2 | 2 |
|
| 3 | +## TRSS Parser for Release Builds |
| 4 | + |
| 5 | +We've created a NodeJS module to parse TRSS output, convert it to Markdown so folks can paste that summary into a GitHub issue. |
| 6 | + |
| 7 | +```bash |
| 8 | +npm install -g trss-parser |
| 9 | +trss-parser <parent id of TRSS pipeline run> |
| 10 | +``` |
| 11 | + |
3 | 12 | ## Triage Guidance
|
4 | 13 |
|
5 |
| -There are many different test jobs running at the AdoptOpenJDK project. No matter which test jobs are being triaged, there is a straight-forward pattern to follow: |
6 |
| -#### Categorize the test failure (based on test output) into 1 of 3 general types: |
| 14 | +There are many different test jobs running at the AdoptOpenJDK project. No matter which test jobs are being triaged, there is a straight-forward pattern to follow. |
| 15 | + |
| 16 | +### Categorize the test failure (based on test output) into 1 of 3 general types |
| 17 | + |
7 | 18 | - infra problem (machine/network)
|
8 | 19 | - test problem
|
9 |
| -- product problem |
10 |
| -#### Check for an existing issue |
11 |
| -- if issue already exists for the failure, annotate with additional information if needed. For example, if you are investigating failures in openjdk tests, look to see if its already reported in JBS, as per instructions in [Guidance for creating OpenJDK bugs](https://github.com/AdoptOpenJDK/openjdk-tests/wiki/Guidance-for-Creating-OpenJDK-Test-Defects). |
12 |
| -#### Raise an issue if no issue exists |
| 20 | +- product problem |
| 21 | + |
| 22 | +### Check for an existing issue |
| 23 | + |
| 24 | +- if issue already exists for the failure, annotate with additional information if needed. For example, if you are investigating failures in openjdk tests, look to see if its already reported in JBS, as per instructions in [Guidance for creating OpenJDK bugs](https://github.com/AdoptOpenJDK/openjdk-tests/wiki/Guidance-for-Creating-OpenJDK-Test-Defects). |
| 25 | + |
| 26 | +### Raise an issue if no issue exists |
| 27 | + |
13 | 28 | - infra issue - raise an issue in [openjdk-infrastructure](https://github.com/AdoptOpenJDK/openjdk-infrastructure/issues)
|
14 |
| -- test issue - ideally, there are enough details to determine which test repo to raise an issue in ond of the test repos from which test material is pulled (OpenJDK, [openjdk-systemtest](https://github.com/AdoptOpenJDK/openjdk-systemtest/issues), [openjdk-tests](https://github.com/AdoptOpenJDK/openjdk-tests/issues) or any of the various 3rd party application suites). If in doubt, ask some questions in the [#testing channel](https://adoptopenjdk.slack.com/messages/C5219G28G) and/or raise in [openjdk-tests](https://github.com/AdoptOpenJDK/openjdk-tests/issues) where it will get routed to proper repo |
15 |
| - - OpenJDK test issues - see [Guidance for creating OpenJDK bugs](https://github.com/AdoptOpenJDK/openjdk-tests/wiki/Guidance-for-Creating-OpenJDK-Test-Defects) |
| 29 | +- test issue - ideally, there are enough details to determine which test repo to raise an issue in ond of the test repositories from which test material is pulled (OpenJDK, [openjdk-systemtest](https://github.com/AdoptOpenJDK/openjdk-systemtest/issues), [openjdk-tests](https://github.com/AdoptOpenJDK/openjdk-tests/issues) or any of the various 3rd party application suites). If in doubt, ask some questions in the [#testing channel](https://adoptopenjdk.slack.com/messages/C5219G28G) and/or raise in [openjdk-tests](https://github.com/AdoptOpenJDK/openjdk-tests/issues) where it will get routed to proper repo |
| 30 | + - OpenJDK test issues - see [Guidance for creating OpenJDK bugs](https://github.com/AdoptOpenJDK/openjdk-tests/wiki/Guidance-for-Creating-OpenJDK-Test-Defects) |
16 | 31 | - Additional guidance for external tests - [Triage Rules for Application tests](https://github.com/AdoptOpenJDK/openjdk-tests/tree/master/external#triage-rules)
|
17 | 32 | - product issue - additional steps may be necessary, before raising an issue
|
18 | 33 | - rerun the test - locally or using a Grinder: see [How to Run a Grinder wiki](https://github.com/AdoptOpenJDK/openjdk-tests/wiki/How-to-Run-a-Grinder-Build-on-Jenkins)
|
19 | 34 | - determine if the problem is occurs in other jdk versions, implementations and on other platforms
|
20 |
| - - if only observed against 1 implementation, raise an issue against that implementation in the correct upstream repo (typically either OpenJDK or [Eclipse OpenJ9](https://github.com/eclipse/openj9/issues) projects). |
21 |
| -#### Exclude the test |
| 35 | + - if only observed against 1 implementation, raise an issue against that implementation in the correct upstream repo (typically either OpenJDK or [Eclipse OpenJ9](https://github.com/eclipse/openj9/issues) projects). |
| 36 | + |
| 37 | +### Exclude the test |
| 38 | + |
22 | 39 | - exclude in the most minimal way possible, if failing only on 1 platform, version or implementation, only exclude for that instance
|
23 | 40 | - put the full link to the associated open issue into the exclude file
|
24 | 41 | - exclude files vary depending on what test group you are triaging, refer to the README files in the openjdk-tests subdirectories for more details
|
|
0 commit comments