Coverage generation fails when using pitmp for multi-module project

[sushantchoudhary]

Characteristics

• Issue Type: [test report]
• Reproducibility: [always]
• Severity: [block]
• Tool/Service/Component: [pitmp, junit]
• Execution Environment: [OSX]
• Reporter: [Sushant, sushant.choudhary@hotmail.com]

Description

One of the modules fails with following error,
Error : SEVERE : Error generating coverage. Please check that your classpath contains JUnit 4.6 or above.

JUnit version on my local machine is 4.12.

[ERROR] Please copy and paste the information and the complete stacktrace below when reporting an issue
[ERROR] VM : Java HotSpot(TM) 64-Bit Server VM
[ERROR] Vendor : Oracle Corporation
[ERROR] Version : 25.171-b11
[ERROR] Uptime : 69769
[ERROR] Input ->
[ERROR] 1 : -Xmx2048m
[ERROR] 2 : -Xms512m
[ERROR] 3 : -Dbulk.download.url=https://maven-bulk-downloader.prod.atl-paas.net
[ERROR] 4 : -XX:+TieredCompilation
[ERROR] 5 : -XX:TieredStopAtLevel=1
[ERROR] 6 : -Dclassworlds.conf=/Users/xyz/.mvnvm/apache-maven-3.5.0/bin/m2.conf
[ERROR] 7 : -Dmaven.home=/Users/xyz/.mvnvm/apache-maven-3.5.0
[ERROR] 8 : -Dmaven.multiModuleProjectDirectory=/Users/xyz/Development/abc/xyz
[ERROR] BootClassPathSupported : true

Steps to reproduce

• Add plugin to root pom.xml

<plugin>
                <groupId>eu.stamp</groupId>
                <artifactId>pitmp-maven-plugin</artifactId>
                <version>1.3.4</version>
                <configuration>
                    <targetModules>
                        <param>xyz-rest-plugin</param>
                    </targetModules>
                </configuration>
            </plugin>

• Run pitmp using mvn
  mvn eu.stamp-project:pitmp-maven-plugin:run

Reference : https://github.com/hcoles/pitest/blob/master/pitest-entry/src/main/java/org/pitest/coverage/execute/DefaultCoverageGenerator.java#L136

[sushantchoudhary]

Referring to Running report with ReportOptions [*] section in logs, for some reason the failing module doesn't have junit jar on its classpath however its present when running report for other modules . Not sure what would cause that?

Update: Looks like pitmp is bailing when the module doesn't have test classes. I tried to skip the failing module using <configuration>property but for some reason it doesn't honor the config and still tries to mutate that module.

            <plugin>
                <groupId>eu.stamp</groupId>
                <artifactId>pitmp-maven-plugin</artifactId>
                <version>1.3.4</version>
                <configuration>
                    <targetModules>
                        <param>xyz-rest-plugin</param>
                    </targetModules>
                    <skippedModules>
                        <param>customer-account-directory-service-client</param>
                    </skippedModules>
                </configuration>
            </plugin>

Also , pitest seem to have a fix for this problem where they just skip if both source and test are not found, hcoles/pitest#314 . Should it work out of box in pitmp?

[nrainer]

I also noticed that skippedModules might not be working at the moment. The problem is indeed caused by a module without tests. A solution would be to additionally declare the JUnit dependency in the parent pom (note that root and parent pom may be different files). Make sure to declare the dependency in the root dependencies tag and not in the dependencies tag within the buildManagement tag.

[sushantchoudhary]

Thanks @nrainer , I will try that approach, however I have lots of modules(~20) without test classes so adding JUnit dependency to parent poms might be tricky .

[sushantchoudhary]

Do you think you would get some time to resolve this issue in the plugin as a clean fix sooner? I can try to look at the code and create a PR if its trivial fix.

[nrainer]

Skipping modules will be fixed with #22.

[sushantchoudhary]

Hi @nrainer , do you have plan to bump the version to 1.3.5 in maven central sooner? I have verified the fix on a snapshot version and it works fine. Also, is there a fix for skipping modules without test classes as well?

[nrainer]

Hi @sushantchoudhary, I just provided the patch. I am not in charge of and do not have the repository rights to release a new version. You need to contact @CaelInria.
Modules without test classes should already be skipped with the current master version. :)

[sushantchoudhary]

Cool, thanks a lot @nrainer . As for the modules without test classes , it doesn't seem to work with latest master since I am still getting this error

[INFO] Mutating from /Users/schoudhary/Development/app-cloud/app/app-components/app-core/target/classes
10:04:10 AM PIT >> INFO : Verbose logging is disabled. If you encounter an problem please enable it before reporting an issue.
10:04:23 AM PIT >> INFO : Sending 6829 test classes to minion
10:04:23 AM PIT >> INFO : Sent tests to minion
10:04:23 AM PIT >> SEVERE : Error generating coverage. Please check that your classpath contains JUnit 4.6 or above.

[ERROR] Failed to execute goal eu.stamp-project:pitmp-maven-plugin:1.3.5-SNAPSHOT:run (default-cli) on project app-core: Execution default-cli of goal eu.stamp-project:pitmp-maven-plugin:1.3.5-SNAPSHOT:run failed: Coverage generation minion exited abnormally. Please check the classpath.

Hi @CaelInria , could you share schedule for bumping version to 1.3.5 .

[Cael35]

I have several modifications to include in the next release, I hope I can release a new version by next Friday.
Sorry for the delay, I'm trying to catch up... :-)

[sushantchoudhary]

Hi @CaelInria , any update on the 1..3.5 release?

[Cael35]

Yes ! :-)
version 1.3.5 will be released on Monday (waiting for the next release of Descartes)
But if you can't wait to use it, 1.3.5-SNAPSHOT is up to date and is tested with PIT 1.4.2 and Descartes 1.2.4

[Cael35]

v1.3.6 is available.