Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pitest fails to execute on projects that don't have test classes #314

Closed
vmassol opened this issue Dec 2, 2016 · 14 comments
Closed

pitest fails to execute on projects that don't have test classes #314

vmassol opened this issue Dec 2, 2016 · 14 comments

Comments

@vmassol
Copy link

vmassol commented Dec 2, 2016

For example running pitest on xwiki-commons-core (https://github.com/xwiki/xwiki-commons/tree/master/xwiki-commons-core) with the following configuration:

      <plugin>
        <groupId>org.pitest</groupId>
        <artifactId>pitest-maven</artifactId>
        <version>1.1.10</version>
        <configuration>
          <targetClasses>
            <param>org.xwiki*</param>
          </targetClasses>
          <targetTests>
            <param>org.xwiki*</param>
          </targetTests>
        </configuration>
      </plugin>

Leads to:

[ERROR] Failed to execute goal org.pitest:pitest-maven:1.1.10:mutationCoverage (default-cli) on project xwiki-commons-stability: Execution default-cli of goal org.pitest:pitest-maven:1.1.10:mutationCoverage failed: No test library found on classpath. PIT requires either JUnit 4 (but can run JUnit 3 tests) or TestNG
[ERROR] See http://pitest.org for more details.
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.pitest:pitest-maven:1.1.10:mutationCoverage (default-cli) on project xwiki-commons-stability: Execution default-cli of goal org.pitest:pitest-maven:1.1.10:mutationCoverage failed: No test library found on classpath. PIT requires either JUnit 4 (but can run JUnit 3 tests) or TestNG
See http://pitest.org for more details.
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution default-cli of goal org.pitest:pitest-maven:1.1.10:mutationCoverage failed: No test library found on classpath. PIT requires either JUnit 4 (but can run JUnit 3 tests) or TestNG
See http://pitest.org for more details.
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
	... 20 more
Caused by: org.pitest.help.PitHelpError: No test library found on classpath. PIT requires either JUnit 4 (but can run JUnit 3 tests) or TestNG
See http://pitest.org for more details.
	at org.pitest.mutationtest.config.ConfigurationFactory.createConfiguration(ConfigurationFactory.java:56)
	at org.pitest.mutationtest.config.LegacyTestFrameworkPlugin.createTestFrameworkConfiguration(LegacyTestFrameworkPlugin.java:36)
	at org.pitest.mutationtest.config.SettingsFactory.getTestFrameworkPlugin(SettingsFactory.java:132)
	at org.pitest.mutationtest.config.SettingsFactory.createCoverageOptions(SettingsFactory.java:140)
	at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:80)
	at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:45)
	at org.pitest.maven.RunPitStrategy.execute(RunPitStrategy.java:35)
	at org.pitest.maven.AbstractPitMojo.analyse(AbstractPitMojo.java:398)
	at org.pitest.maven.AbstractPitMojo.execute(AbstractPitMojo.java:342)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
	... 21 more
[ERROR] 

Note: The xwiki-commons-stability maven project doesn't have any test (and thus doesn't depend on junit).

@hcoles
Copy link
Owner

hcoles commented Dec 2, 2016

Pitest includes a skip option so modules such as this can be configured to not break the build. It would arguably be an improvement to auto skip when no tests are found, however the most common case where tests aren't found is when pitest has been misconfigured - in this scenario a hard failure is useful.

@vmassol
Copy link
Author

vmassol commented Dec 2, 2016

Hi. Thanks for the answer. However if I'm correct this will mean override the plugin configuration in each module that doesn't have tests and the xwiki codebase that would be a lot of modules :) At this stage I only wanted to quickly test pitest and see what I could get out of it (without committing anything in the XWiki project yet).

So indeed this would be an improvement but possibly a nice one :) (and should be pretty easy to do).

Thanks!

@samisalonen
Copy link

Would something like this work for you?

  <properties>
    <pitest.skip>false</pitest.skip>
  </properties>

  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>org.pitest</groupId>
          <artifactId>pitest-maven</artifactId>
          <version>1.1.10</version>
          <configuration>
            <skip>${pitest.skip}</skip>
          </configuration>
        </plugin>
      </plugins>
    </pluginManagement>
  </build>

  <profiles>
    <profile>
      <id>skip-pitest-if-no-main-dir</id>
      <activation>
        <file>
          <missing>src/main/java</missing>
        </file>
      </activation>
      <properties>
        <pitest.skip>true</pitest.skip>
      </properties>
    </profile>
    <profile>
      <id>skip-pitest-if-no-test-dir</id>
      <activation>
        <file>
          <missing>src/test/java</missing>
        </file>
      </activation>
      <properties>
        <pitest.skip>true</pitest.skip>
      </properties>
    </profile>
  </profiles>

@vmassol
Copy link
Author

vmassol commented Dec 4, 2016

Thanks that's very creative :)

I think it worked but it's failing elsewhere now I think:

vmassol ~/dev/xwiki/xwiki-commons/xwiki-commons-core/xwiki-commons-text $ mvn org.pitest:pitest-maven:mutationCoverage -e
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=196m; support was removed in 8.0
[INFO] Error stacktraces are turned on.
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building XWiki Commons - Text 9.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- pitest-maven:1.1.10:mutationCoverage (default-cli) @ xwiki-commons-text ---
[INFO] Found plugin : Default csv report plugin
[INFO] Found plugin : Default xml report plugin
[INFO] Found plugin : Default html report plugin
[INFO] Found plugin : Default limit mutations plugin
[INFO] Found plugin : Kotlin junk mutations filter
[INFO] Found shared classpath plugin : Default mutation engine
[INFO] Adding org.pitest:pitest to SUT classpath
[INFO] Mutating from /Users/vmassol/dev/xwiki/xwiki-commons/xwiki-commons-core/xwiki-commons-text/target/classes
6:57:14 PM PIT >> INFO : Verbose logging is disabled. If you encounter an problem please enable it before reporting an issue.
6:57:14 PM PIT >> INFO : MINION : objc[14642]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre/bin/java (0x106b774c0) and /Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre/lib/libinstrument.dylib (0x108bf
6:57:14 PM PIT >> INFO : MINION : 04e0). One of the two will be used. Which one is undefined.

6:57:15 PM PIT >> INFO : Sending 3 test classes to minion
6:57:15 PM PIT >> INFO : Sent tests to minion
6:57:15 PM PIT >> INFO : MINION : 6:57:15 PM PIT >> INFO : Checking environment

6:57:15 PM PIT >> INFO : MINION : 6:57:15 PM PIT >> INFO : Found  2 tests

6:57:15 PM PIT >> INFO : MINION : 6:57:15 PM PIT >> INFO : Dependency analysis reduced number of potential tests by 0

6:57:15 PM PIT >> INFO : MINION : 6:57:15 PM PIT >> INFO : 2 tests received

-6:57:15 PM PIT >> INFO : Calculated coverage in 0 seconds.
6:57:15 PM PIT >> INFO : Created  0 mutation test units
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.929 s
[INFO] Finished at: 2016-12-04T18:57:15+01:00
[INFO] Final Memory: 14M/303M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.pitest:pitest-maven:1.1.10:mutationCoverage (default-cli) on project xwiki-commons-text: Execution default-cli of goal org.pitest:pitest-maven:1.1.10:mutationCoverage failed: No mutations found. This probably means there is an issue with either the supplied classpath or filters.
[ERROR] See http://pitest.org for more details.
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.pitest:pitest-maven:1.1.10:mutationCoverage (default-cli) on project xwiki-commons-text: Execution default-cli of goal org.pitest:pitest-maven:1.1.10:mutationCoverage failed: No mutations found. This probably means there is an issue with either the supplied classpath or filters.
See http://pitest.org for more details.
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution default-cli of goal org.pitest:pitest-maven:1.1.10:mutationCoverage failed: No mutations found. This probably means there is an issue with either the supplied classpath or filters.
See http://pitest.org for more details.
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
	... 20 more
Caused by: org.pitest.help.PitHelpError: No mutations found. This probably means there is an issue with either the supplied classpath or filters.
See http://pitest.org for more details.
	at org.pitest.mutationtest.tooling.MutationCoverage.checkMutationsFound(MutationCoverage.java:272)
	at org.pitest.mutationtest.tooling.MutationCoverage.runReport(MutationCoverage.java:136)
	at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:103)
	at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:45)
	at org.pitest.maven.RunPitStrategy.execute(RunPitStrategy.java:35)
	at org.pitest.maven.AbstractPitMojo.analyse(AbstractPitMojo.java:398)
	at org.pitest.maven.AbstractPitMojo.execute(AbstractPitMojo.java:342)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
	... 21 more
[ERROR] 

I've used this config:

        <plugin>
          <groupId>org.pitest</groupId>
          <artifactId>pitest-maven</artifactId>
          <version>1.1.10</version>
          <configuration>
            <targetClasses>
              <param>org.xwikki*</param>
            </targetClasses>
            <targetTests>
              <param>org.xwiki*</param>
            </targetTests>
            <skip>${pitest.skip}</skip>
          </configuration>
        </plugin>

Any idea? Thanks again

@monperrus
Copy link

word! (to get updated about this issue)

hcoles pushed a commit to ncredinburgh/pitest that referenced this issue Dec 9, 2016
@hcoles
Copy link
Owner

hcoles commented Dec 11, 2016

I got chance to make a few changes that I'll merge I'll merge in shortly.

The maven plugin now checks for that and both src and test folders are configured for each module and that those folders actually exist. If they do not the project is auto skipped.

With no configuration set things seemed to work for xwiki but I haven't left it to complete a run so there may be issues I've not hit yet.

@vmassol
Copy link
Author

vmassol commented Dec 11, 2016

Seems awesome @hcoles :) Waiting for it.

@hcoles
Copy link
Owner

hcoles commented Dec 12, 2016

That's merged in now and a 1.1.12 snapshot has been published.

@vmassol
Copy link
Author

vmassol commented Dec 13, 2016

Seems to be working much better, thanks!

I get a lot of the following, I don't know if that's "normal" though:

5:48:20 PM PIT >> WARNING : Minion exited abnormally due to TIMED_OUT

What would be awesome now would be to get an aggregated report, i.e. #41 :)

Thanks again for your support, that's great.

@vmassol vmassol closed this as completed Dec 13, 2016
@hcoles
Copy link
Owner

hcoles commented Dec 13, 2016

Some time outs are normal and expected (they happen when a mutation causes an infinite loop).

A large number of them might indicate a problem. See http://pitest.org/faq/

@vmassol
Copy link
Author

vmassol commented Dec 13, 2016

Thanks. I need to do more reading (including the FAQ!) and see how I could use pitest now that I know it's working ;)

@khanhassan
Copy link

khanhassan commented Jan 9, 2017

When are you going to publish 1.1.12 on maven central?

@hcoles
Copy link
Owner

hcoles commented Mar 28, 2017

@khanhassan Yes, the next release is long overdue.

Will publish when #334 has been addressed.

@hcoles
Copy link
Owner

hcoles commented Mar 31, 2017

Released in 1.2.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants