PITest suddenly breaks #340

Ledmington · 2023-06-24T14:49:43Z

Hello, I am using PITest in my project along with many parameterized tests and I noticed that when launching the tests with gradlew.bat test everything passes, but when I run gradlew.bat pitest I get Pitest needs a green suite. I tried upgrading it to the following configuration and now it doesn't even start.

plugins {
    id 'java-library'
    id 'info.solidsoft.pitest' version '1.9.11'
}

...

pitest {
    pitestVersion = '1.14.2'
    junit5PluginVersion = '1.2.0'
    //threads = 4
    outputFormats.set(['XML', 'HTML'])
    verbose = true
    timestampedReports = false
}

This is the error I receive

> Task :json:pitest FAILED
16:42:39 PIT >> INFO : Project base directory is null
16:42:39 PIT >> INFO : ---------------------------------------------------------------------------
16:42:39 PIT >> INFO : Enabled (+) and disabled (-) features.
16:42:39 PIT >> INFO : -----------------------------------------
16:42:39 PIT >> INFO : +fann           Filters mutations in classes and methods with matching annotations of class or runtime retention
16:42:39 PIT >> INFO :   [annotation]    Annotation to avoid (full package name not required)
16:42:39 PIT >> INFO : +fassert        Filters mutations in compiler generated code for assertions
16:42:39 PIT >> INFO : +fenum          Filters junk mutations in enums
16:42:39 PIT >> INFO : +feswitch       Filters mutations in switch statements on enums
16:42:39 PIT >> INFO : +ffblock        Filters mutations in code duplicated by finally block inlining
16:42:39 PIT >> INFO : +ffeach         Filters mutations in compiler generated code that implements for each loops
16:42:39 PIT >> INFO : +ffloop         Filters any mutations to increments in for loops as they may cause timeouts
16:42:39 PIT >> INFO : +fgroovy        Filters out junk mutations in groovy code
16:42:39 PIT >> INFO : +finfinc        Filters mutations to increments that may cause infinite loops
16:42:39 PIT >> INFO : +finfit         Filters mutations that may cause infinite loops by removing calls to iterator.next
16:42:39 PIT >> INFO : +finull         Filters mutations in compiler generated code that checks for null by calling getClass
16:42:39 PIT >> INFO : +fkotlin        Filters out junk mutations in bytecode created by compiler for kotlin language features
16:42:39 PIT >> INFO : +flogcall       Filters mutations in code that makes calls to logging frameworks
16:42:39 PIT >> INFO : +fmrnull        Filters mutations in compiler generated code that inserts Objects.requireNonNull for method references
16:42:39 PIT >> INFO : +frecord        Filters mutations in compiler generated record code
16:42:39 PIT >> INFO : +fretequiv      Filters return vals mutants with bytecode equivalent to the unmutated class
16:42:39 PIT >> INFO : +fsequivdiv     Filters equivalent mutations of the form x * -1 -> x / -1
16:42:39 PIT >> INFO : +fsequivequals  Filters equivalent mutations that affect only performance in short cutting equals methods
16:42:39 PIT >> INFO : +fstati         Filters mutations in static initializers and code called only from them
16:42:39 PIT >> INFO : +fstrswitch     Filters mutations in compiler generated code for string switch statements
16:42:39 PIT >> INFO : +ftrywr         Filters mutations in code generated for try with resources statements
16:42:39 PIT >> INFO : +macos_focus    Auto add java.awt.headless=true to keep keyboard focus on Mac OS
16:42:39 PIT >> INFO : -auto_threads   Auto set number of threads based on machine
16:42:39 PIT >> INFO : -classlimit     Limits the maximum number of mutations per class
16:42:39 PIT >> INFO :   [limit]         Integer value for maximum mutations to create per class
16:42:39 PIT >> INFO : -export         Exports mutants bytecode and other details to disk
16:42:39 PIT >> INFO : -nofirstline    Filters mutants with line number <= 1
16:42:39 PIT >> INFO : ---------------------------------------------------------------------------
16:42:39 PIT >> FINE : Running report with ReportOptions[targetClasses=[com.ledmington.json.*], excludedMethods=[], excludedClasses=[], excludedTestClasses=[], codePaths=[C:\Users\Filippo\Desktop\ppa\backend\json\build\classes\java\main], reportDir='C:\Users\Filippo\Desktop\ppa\backend\json\build\reports\pitest', historyInputLocation=null, historyOutputLocation=null, sourceDirs=[C:\Users\Filippo\Desktop\ppa\backend\json\src\main\resources, C:\Users\Filippo\Desktop\ppa\backend\json\src\main\java], classPathElements=[C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.pitest\pitest-junit5-plugin\1.2.0\791a7bda11dc68761002ec8574d9132394f3292\pitest-junit5-plugin-1.2.0.jar, C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.pitest\pitest\1.14.2\46ffa937b08248c7a34f4e5f8c19473b32a8bed1\pitest-1.14.2.jar, C:\Users\Filippo\Desktop\ppa\backend\json\build\classes\java\test, C:\Users\Filippo\Desktop\ppa\backend\json\build\resources\test, C:\Users\Filippo\Desktop\ppa\backend\json\build\classes\java\main, C:\Users\Filippo\Desktop\ppa\backend\json\build\resources\main, C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\com.github.javaparser\javaparser-symbol-solver-core\3.25.3\38c6143b14bdfd2e1e20b3a47cded11262bd014a\javaparser-symbol-solver-core-3.25.3.jar, C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\com.google.guava\guava\32.0.1-jre\6e5d51a72d142f2d40a57dfb897188b36a95b489\guava-32.0.1-jre.jar, C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.junit.jupiter\junit-jupiter-engine\5.9.3\355322b03bf39306a183162cd06626c206f0286b\junit-jupiter-engine-5.9.3.jar, C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.junit.platform\junit-platform-engine\1.9.3\8616734a190f8d307376aeb7353dba0a2c037a09\junit-platform-engine-1.9.3.jar, C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.junit.platform\junit-platform-commons\1.9.3\36b2e26a90c41603be7f0094bee80e3f8a2cd4d4\junit-platform-commons-1.9.3.jar, C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.junit.jupiter\junit-jupiter-params\5.9.3\9e2a4bf6016a1975f408a73523392875cff7c26f\junit-jupiter-params-5.9.3.jar, C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.junit.jupiter\junit-jupiter-api\5.9.3\815818ad6ffcc8d320d8fbdf3d748c753cf83201\junit-jupiter-api-5.9.3.jar, C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\com.google.guava\failureaccess\1.0.1\1dcf1de382a0bf95a3d8b0849546c88bac1292c9\failureaccess-1.0.1.jar, C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\com.google.guava\listenablefuture\9999.0-empty-to-avoid-conflict-with-guava\b421526c5f297295adef1c886e5246c39d4ac629\listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar, C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\com.google.code.findbugs\jsr305\3.0.2\25ea2e8b0c338a877313bd4672d3fe056ea78f0d\jsr305-3.0.2.jar, C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.checkerframework\checker-qual\3.33.0\de2b60b62da487644fc11f734e73c8b0b431238f\checker-qual-3.33.0.jar, C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\com.google.errorprone\error_prone_annotations\2.18.0\89b684257096f548fa39a7df9fdaa409d4d4df91\error_prone_annotations-2.18.0.jar, C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\com.google.j2objc\j2objc-annotations\2.8\c85270e307e7b822f1086b93689124b89768e273\j2objc-annotations-2.8.jar, C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.opentest4j\opentest4j\1.2.0\28c11eb91f9b6d8e200631d46e20a7f407f2a046\opentest4j-1.2.0.jar, C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\com.github.javaparser\javaparser-core\3.25.3\55a960eea36e9ae20e48c500c3dd356b33331f1f\javaparser-core-3.25.3.jar, C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.javassist\javassist\3.29.2-GA\6c32028609e5dd4a1b78e10fbcd122b09b3928b1\javassist-3.29.2-GA.jar], mutators=[], features=[], jvmArgs=[-Djava.awt.headless=true], argLine=null, numberOfThreads=1, timeoutFactor=1.25, timeoutConstant=4000, targetTests=[^com\.ledmington\.json\..*$], loggingClasses=[], verbosity=VERBOSE, failWhenNoMutations=true, skipFailingTests=false, outputs=[XML, HTML], groupConfig=TestGroupConfig [excludedGroups=[], includedGroups=[]], fullMutationMatrix=false, mutationUnitSize=0, shouldCreateTimestampedReports=false, detectInlinedCode=true, exportLineCoverage=false, mutationThreshold=0, coverageThreshold=0, testStrengthThreshold=0, mutationEngine='gregor', javaExecutable='null', includeLaunchClasspath=false, properties={}, maxSurvivors=-1, excludedRunners=[], includedTestMethods=[], testPlugin='', useClasspathJar=false, projectBase=null, inputEncoding=windows-1252, outputEncoding=windows-1252]
16:42:39 PIT >> FINE : System class path is C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.pitest\pitest-command-line\1.14.2\de9a3a9ade263be77d1ded68a6602b822247a69d\pitest-command-line-1.14.2.jar;C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.pitest\pitest-junit5-plugin\1.2.0\791a7bda11dc68761002ec8574d9132394f3292\pitest-junit5-plugin-1.2.0.jar;C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.pitest\pitest-entry\1.14.2\2d51f8fa8b79f391d589c5b2e8cab7c2e24248de\pitest-entry-1.14.2.jar;C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.junit.platform\junit-platform-engine\1.9.2\40aeef2be7b04f96bb91e8b054affc28b7c7c935\junit-platform-engine-1.9.2.jar;C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.junit.platform\junit-platform-commons\1.9.2\6f9f8621d8230cd38aa42e58ccbc0c00569131ce\junit-platform-commons-1.9.2.jar;C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.junit.platform\junit-platform-launcher\1.9.2\38e0bfad8c57d4cd1a8f27926c25ffe9543068d6\junit-platform-launcher-1.9.2.jar;C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.pitest\pitest\1.14.2\46ffa937b08248c7a34f4e5f8c19473b32a8bed1\pitest-1.14.2.jar;C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.apache.commons\commons-text\1.10.0\3363381aef8cef2dbc1023b3e3a9433b08b64e01\commons-text-1.10.0.jar;C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.apache.commons\commons-lang3\3.12.0\c6842c86792ff03b9f1d1fe2aab8dc23aa6c6f0e\commons-lang3-3.12.0.jar;C:\Users\Filippo\.gradle\caches\modules-2\files-2.1\org.opentest4j\opentest4j\1.2.0\28c11eb91f9b6d8e200631d46e20a7f407f2a046\opentest4j-1.2.0.jar
16:42:39 PIT >> FINE : Maximum available memory is 2018 mb
16:42:40 PIT >> FINE : Incremental analysis set 243 mutations to a status of NOT_STARTED
16:42:40 PIT >> INFO : Incremental analysis reduced number of mutations by 0
16:42:40 PIT >> INFO : Created 9 mutation test units in pre scan
16:42:40 PIT >> FINE : MINION : Installing PIT agent
16:42:40 PIT >> INFO : Sending 17 test classes to minion
16:42:40 PIT >> INFO : Sent tests to minion
16:42:40 PIT >> FINE : MINION : java.lang.NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory
16:42:40 PIT >> FINE : MINION :         at org.pitest.junit5.JUnit5TestUnitFinder.<init>(JUnit5TestUnitFinder.java:58)
16:42:40 PIT >> FINE : MINION :         at org.pitest.junit5.JUnit5Configuration.testUnitFinder(JUnit5Configuration.java:43)
16:42:40 PIT >> FINE : MINION :         at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
16:42:40 PIT >> FINE : MINION :         at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
16:42:40 PIT >> FINE : MINION :         at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
16:42:40 PIT >> FINE : MINION :         at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
16:42:40 PIT >> FINE : MINION :         at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
16:42:40 PIT >> FINE : MINION :         at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
16:42:40 PIT >> FINE : MINION :         at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
16:42:40 PIT >> FINE : MINION :         at org.pitest.mutationtest.config.PrioritisingTestConfiguration.makeFinder(PrioritisingTestConfiguration.java:61)
16:42:40 PIT >> FINE : MINION :         at org.pitest.mutationtest.config.PrioritisingTestConfiguration.<init>(PrioritisingTestConfiguration.java:20)
16:42:40 PIT >> FINE : MINION :         at org.pitest.mutationtest.config.MinionSettings.getTestFrameworkPlugin(MinionSettings.java:57)
16:42:40 PIT >> FINE : MINION :         at org.pitest.coverage.execute.CoverageMinion.createTestPlugin(CoverageMinion.java:188)
16:42:40 PIT >> FINE : MINION :         at org.pitest.coverage.execute.CoverageMinion.getTestsFromParent(CoverageMinion.java:160)
16:42:40 PIT >> FINE : MINION :         at org.pitest.coverage.execute.CoverageMinion.main(CoverageMinion.java:87)
16:42:40 PIT >> FINE : MINION : Caused by: java.lang.ClassNotFoundException: org.junit.platform.launcher.core.LauncherFactory
16:42:40 PIT >> FINE : MINION :         at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
16:42:40 PIT >> FINE : MINION :         at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
16:42:40 PIT >> FINE : MINION :         at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
16:42:40 PIT >> FINE : MINION :         ... 15 more
16:42:40 PIT >> INFO : MINION : 16:42:40 PIT >> FINE : Expecting 17 tests classes from parent
16:42:40 PIT >> SEVERE : Coverage generator Minion exited abnormally due to UNKNOWN_ERROR
16:42:40 PIT >> INFO : MINION : 16:42:40 PIT >> FINE : Tests classes received
16:42:40 PIT >> INFO : MINION : 16:42:40 PIT >> SEVERE : Error calculating coverage. Process will exit.
16:42:40 PIT >> INFO : MINION : java.lang.NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory
16:42:40 PIT >> INFO : MINION :         at org.pitest.junit5.JUnit5TestUnitFinder.<init>(JUnit5TestUnitFinder.java:58)
16:42:40 PIT >> INFO : MINION :         at org.pitest.junit5.JUnit5Configuration.testUnitFinder(JUnit5Configuration.java:43)
16:42:40 PIT >> INFO : MINION :         at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
16:42:40 PIT >> INFO : MINION :         at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
16:42:40 PIT >> INFO : MINION :         at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
16:42:40 PIT >> INFO : MINION :         at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
16:42:40 PIT >> INFO : MINION :         at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
16:42:40 PIT >> INFO : MINION :         at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
16:42:40 PIT >> INFO : MINION :         at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
16:42:40 PIT >> INFO : MINION :         at org.pitest.mutationtest.config.PrioritisingTestConfiguration.makeFinder(PrioritisingTestConfiguration.java:61)
16:42:40 PIT >> INFO : MINION :         at org.pitest.mutationtest.config.PrioritisingTestConfiguration.<init>(PrioritisingTestConfiguration.java:20)
16:42:40 PIT >> INFO : MINION :         at org.pitest.mutationtest.config.MinionSettings.getTestFrameworkPlugin(MinionSettings.java:57)
16:42:40 PIT >> INFO : MINION :         at org.pitest.coverage.execute.CoverageMinion.createTestPlugin(CoverageMinion.java:188)
16:42:40 PIT >> INFO : MINION :         at org.pitest.coverage.execute.CoverageMinion.getTestsFromParent(CoverageMinion.java:160)
16:42:40 PIT >> INFO : MINION :         at org.pitest.coverage.execute.CoverageMinion.main(CoverageMinion.java:87)
16:42:40 PIT >> INFO : MINION : Caused by: java.lang.ClassNotFoundException: org.junit.platform.launcher.core.LauncherFactory
16:42:40 PIT >> INFO : MINION :         at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
16:42:40 PIT >> INFO : MINION :         at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
16:42:40 PIT >> INFO : MINION :         at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
16:42:40 PIT >> INFO : MINION :         ... 15 more
Exception in thread "main" org.pitest.util.PitError: Coverage generation minion exited abnormally! (UNKNOWN_ERROR)

Please copy and paste the information and the complete stacktrace below when reporting an issue
VM : OpenJDK 64-Bit Server VM
Vendor : Oracle Corporation
Version : 17+35-2724
Uptime : 1296
Input ->
 1 : -Dfile.encoding=windows-1252
 2 : -Duser.country=IT
 3 : -Duser.language=it
 4 : -Duser.variant
BootClassPathSupported : false


Please copy and paste the information and the complete stacktrace below when reporting an issue
VM : OpenJDK 64-Bit Server VM
Vendor : Oracle Corporation
Version : 17+35-2724
Uptime : 1300
Input ->
 1 : -Dfile.encoding=windows-1252
 2 : -Duser.country=IT
 3 : -Duser.language=it
 4 : -Duser.variant
BootClassPathSupported : false

        at org.pitest.util.Unchecked.translateCheckedException(Unchecked.java:20)
        at org.pitest.coverage.execute.DefaultCoverageGenerator.calculateCoverage(DefaultCoverageGenerator.java:112)
        at org.pitest.coverage.execute.DefaultCoverageGenerator.calculateCoverage(DefaultCoverageGenerator.java:54)
        at org.pitest.mutationtest.tooling.MutationCoverage.runAnalysis(MutationCoverage.java:154)
        at org.pitest.mutationtest.tooling.MutationCoverage.runReport(MutationCoverage.java:144)
        at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:130)
        at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:58)
        at org.pitest.mutationtest.commandline.MutationCoverageReport.runReport(MutationCoverageReport.java:98)
        at org.pitest.mutationtest.commandline.MutationCoverageReport.main(MutationCoverageReport.java:45)
Caused by: org.pitest.util.PitError: Coverage generation minion exited abnormally! (UNKNOWN_ERROR)

Please copy and paste the information and the complete stacktrace below when reporting an issue
VM : OpenJDK 64-Bit Server VM
Vendor : Oracle Corporation
Version : 17+35-2724
Uptime : 1296
Input ->
 1 : -Dfile.encoding=windows-1252
 2 : -Duser.country=IT
 3 : -Duser.language=it
 4 : -Duser.variant
BootClassPathSupported : false

        at org.pitest.coverage.execute.DefaultCoverageGenerator.gatherCoverageData(DefaultCoverageGenerator.java:151)
        at org.pitest.coverage.execute.DefaultCoverageGenerator.calculateCoverage(DefaultCoverageGenerator.java:96)
        ... 7 more

The text was updated successfully, but these errors were encountered:

hcoles · 2023-06-24T15:48:14Z

When running from gradle, 1.2.0 of pitest-junit5-plugin may need the junit-platform-launcher to be added to pitest's classpath. See

https://github.com/pitest/pitest-junit5-plugin

Your original "pitest needs a green suite" message means one or more tests are failing when run via pitest. To understand why, the first step is to enable pitest's verbose logging and identify which tests have failed.

szpak · 2023-06-25T11:54:07Z

@Ledmington As Henry said, that should help. The feature to automatically add junit-platform-launcher is already tracked.

Ledmington · 2023-08-30T21:17:26Z

Hello, sorry for the late answer... I honestly forgot about this issue.
I solved it by adding

testRuntimeOnly("org.junit.platform:junit-platform-launcher") {
    because("required for pitest")
}

in my dependencies.

szpak · 2023-09-25T21:57:34Z

@Ledmington I've released the SNAPSHOT version which automatically adds that dependency 1.14.0 (see #353). You can see how to use the SNAPSHOT version in that PR: https://github.com/rahulsom/pitest-junit-compat/pull/2/files

Could you check, if it works in your project? The implementation for Gradle is somehow tricky and there might be some corner cases :-/.

szpak closed this as completed Aug 31, 2023

szpak mentioned this issue Sep 5, 2023

Compatibility with JUnit 5.10.0 is broken #352

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PITest suddenly breaks #340

PITest suddenly breaks #340

Ledmington commented Jun 24, 2023

hcoles commented Jun 24, 2023

szpak commented Jun 25, 2023

Ledmington commented Aug 30, 2023

szpak commented Sep 25, 2023 •

edited

Loading

PITest suddenly breaks #340

PITest suddenly breaks #340

Comments

Ledmington commented Jun 24, 2023

hcoles commented Jun 24, 2023

szpak commented Jun 25, 2023

Ledmington commented Aug 30, 2023

szpak commented Sep 25, 2023 • edited Loading

szpak commented Sep 25, 2023 •

edited

Loading