Patch release 4.13.1 seems to break maven surefire plugin

[honnix]

Not super sure of this but I can describe the issue.

After bumping up to 4.13.1. our CI reports failing to parse "4.13.1:

Oct 16, 2020 10:04:34 AM org.junit.platform.launcher.core.DefaultLauncher handleThrowable
WARNING: TestEngine with ID 'junit-vintage' failed to discover tests
org.junit.platform.commons.JUnitException: Failed to parse version of junit:junit: 4.13.1
	at org.junit.vintage.engine.JUnit4VersionCheck.parseVersion(JUnit4VersionCheck.java:54)
	at org.junit.vintage.engine.JUnit4VersionCheck.checkSupported(JUnit4VersionCheck.java:37)
	at org.junit.vintage.engine.JUnit4VersionCheck.checkSupported(JUnit4VersionCheck.java:32)
	at org.junit.vintage.engine.VintageTestEngine.discover(VintageTestEngine.java:61)
	at org.junit.platform.launcher.core.DefaultLauncher.discoverEngineRoot(DefaultLauncher.java:168)
	at org.junit.platform.launcher.core.DefaultLauncher.discoverRoot(DefaultLauncher.java:155)
	at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:120)
	at org.apache.maven.surefire.junitplatform.TestPlanScannerFilter.accept(TestPlanScannerFilter.java:56)
	at org.apache.maven.surefire.util.DefaultScanResult.applyFilter(DefaultScanResult.java:102)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.scanClasspath(JUnitPlatformProvider.java:143)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.getSuites(JUnitPlatformProvider.java:101)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:169)
	at org.apache.maven.surefire.util.ReflectionUtils.invokeGetter(ReflectionUtils.java:76)
	at org.apache.maven.surefire.util.ReflectionUtils.invokeGetter(ReflectionUtils.java:70)
	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.getSuites(ProviderFactory.java:145)
	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.getSuitesIterator(ForkStarter.java:698)
	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.runSuitesForkPerTestSet(ForkStarter.java:418)
	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:299)
	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:247)
	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1161)
	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1002)
	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:848)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:956)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:192)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)

After looking into JUnit4VersionCheck, it seems it recently starts to support better versioning in this junit-team/junit5@f30c96a commit that went to junit-vintage-engine:5.7.0`.

However the latest maven surefire plugins depends on junit-platform-launcher:1.3.2 that eventually depends on junit-vintage-engine:5.3.2 that doesn't support 4.13.1 as versioning scheme.

Forcing an upgrade:

        <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>3.0.0-M5</version>
          <dependencies>
            <dependency>
              <groupId>org.junit.platform</groupId>
              <artifactId>junit-platform-launcher</artifactId>
              <version>1.7.0</version>
            </dependency>
          </dependencies>
        </plugin>

breaks due to incompatible change over years:

org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M5:test failed: java.lang.NoClassDefFoundError: org/junit/platform/commons/util/ClassNamePatternFilterUtils: org.junit.platform.commons.util.ClassNamePatternFilterUtil

Maybe this issue fits better into maven surefire plugin repo?

[sbrannen]

Maybe this issue fits better into maven surefire plugin repo?

Yes, I agree with that.

@sormuras, do you have any further input?

[marcphilipp]

This was indeed fixed in junit-team/junit5#2198 for JUnit Vintage Engine 5.7.

Can you upgrade to a 3.x version of Surefire?

[honnix]

@marcphilipp I couldn't find any new version than 3.0.0-M5 though.

[marcphilipp]

Ah, sorry I overlooked that part in the issue description. Since 4.13.1 works fine in our sample project I suspect you may have mixed versions of JUnit artifacts on the classpath. Could you please provide a sample project that reproduces the issue?

[honnix]

@marcphilipp I think that is possible I will try to reproduce it using a simple project. However the error says explicitly that it cannot parse 4.13.1 as a version. Anyway please give me some time.

[honnix]

@marcphilipp You are right about mixing JUnit 5 and JUnit 4. It seems that triggers maven-surefire-plugin to work in a very different way. https://github.com/junit-team/junit5-samples/pull/137/checks?check_run_id=1265044819 reproduced it.

I'm thinking about possible solutions I can take:

• Pinning junit-jupiter-api to 5.7.0
• Or exclude JUnit 5 introduced as transitive deps.

[honnix]

Still I think this is somewhat annoying. A patch version bump requires quite some changes in the pom.

[marcphilipp]

Closing since upgrading to 5.7.0 of the JUnit Vintage Engine is a valid fix. Sorry for any inconvenience caused.