Skip to content

Improve openjdk exclusion file handling for new versions #5932

@sophia-guo

Description

@sophia-guo

Currently, for new jdk version, a new exclusion file ProblemList_openjdkversion.txt is required to be added or the default one ( ProblemList_openjdk11.txt) will be used to avoid breaking the test job. Normally we will copy the most recent previous version (e.g, if the newest is 25, we copy the version 24) and add it as the newest one.

The may cause inaccurate exclusion. The old exclusion file may contain test cases that have been fixed or even enhanced in the latest version. Keeping them excluded unnecessarily reduces test coverage and prevent immediate detection of new failures. (Consider the openjdk developing process, backport always takes time or won't happen at all).

Instead of requiring a manually copied exclusion file for each new version we could modify the process to run without local exclusion file to allow all test cases to execute ( sync with upstream openjdk ) and generate the exclusion file based on actual triage failures. Note generating a new exclusion file may take extra time if many tests fail.

This can be limited to impl=hotspot as openj9 may prefer different option #5890

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Status

In Progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions