-
-
Notifications
You must be signed in to change notification settings - Fork 328
Description
Currently, for new jdk version, a new exclusion file ProblemList_openjdkversion.txt is required to be added or the default one ( ProblemList_openjdk11.txt) will be used to avoid breaking the test job. Normally we will copy the most recent previous version (e.g, if the newest is 25, we copy the version 24) and add it as the newest one.
The may cause inaccurate exclusion. The old exclusion file may contain test cases that have been fixed or even enhanced in the latest version. Keeping them excluded unnecessarily reduces test coverage and prevent immediate detection of new failures. (Consider the openjdk developing process, backport always takes time or won't happen at all).
Instead of requiring a manually copied exclusion file for each new version we could modify the process to run without local exclusion file to allow all test cases to execute ( sync with upstream openjdk ) and generate the exclusion file based on actual triage failures. Note generating a new exclusion file may take extra time if many tests fail.
This can be limited to impl=hotspot as openj9 may prefer different option #5890
Metadata
Metadata
Assignees
Labels
Type
Projects
Status