Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Increment Analysis Issue #763

Closed
Yuvarajahbk opened this issue May 5, 2020 · 3 comments · Fixed by #765
Closed

Increment Analysis Issue #763

Yuvarajahbk opened this issue May 5, 2020 · 3 comments · Fixed by #765

Comments

@Yuvarajahbk
Copy link

Increment analysis is not working correctly.

			<plugin>
			<groupId>org.pitest</groupId>
			<artifactId>pitest-maven</artifactId>
			<version>1.4.11</version>
			<configuration>
				<testPlugin>testng</testPlugin>
				<verbose>true</verbose>
				<features>
					<value>+EXPORT</value>
				</features>
				<targetClasses>
					<param>com.dt.appservice.*</param>
				</targetClasses>
				<historyInputFile>${project.basedir}\\test-output\\PitestHistory\\history.txt</historyInputFile>
				<historyOutputFile>${project.basedir}\\test-output\\PitestHistory\\history.txt</historyOutputFile>
				<targetTests>
					<param>com.abc.dtc.service.*</param>
				</targetTests>
				
					</configuration>
				<plugin>

It is creating history file but not referring for next run.. When I am running same set of files without any modifications, It is taking same time. No performance improvement.

could you help me on this?
@StefanPenndorf
Copy link
Contributor

I could reproduce this issue with a minimal example project using maven and testng. The problem is the combination of different granularity of the coverage calculation which triggers on a "by test class" basis, but reports covered tests "by test method" (for history calculation) on the one hand and the history check which in turn acts on a "by test class" basis. Those different definitions of a TestUnit seem to collide for testng.

I'll try to provide a fix for this.

StefanPenndorf added a commit to StefanPenndorf/pitest that referenced this issue May 6, 2020
@StefanPenndorf
made incremental analysis more tolerant of coverage changes
0d515cf 
fixes hcoles#763

Incremental analysis honors all killing tests now instead of limiting
the scope of analysis to the first killing test.
This is often identical if there is no matrix mutation in place but
TestNG plugin calculates test units in terms of test classes thus
executes all test methods of a test class and records historic data
with method level granularity and class level granularity. We need
to take all killing tests into account to match history data and
executable test units in the analyzer. This will also help if
different test classes cover a certain production class and one of
those test classes is deleted or renamed.

Improved logging to help spotting analyzer misbehaviour.
@StefanPenndorf StefanPenndorf mentioned this issue May 6, 2020
Merged
hcoles pushed a commit that referenced this issue May 10, 2020
@StefanPenndorf
Allowed more tests to be considered killing a mutant when using histo…
503e939 
…ric data (fixes #763) (#765)

* made incremental analysis more tolerant of coverage changes

fixes #763

Incremental analysis honors all killing tests now instead of limiting
the scope of analysis to the first killing test.
This is often identical if there is no matrix mutation in place but
TestNG plugin calculates test units in terms of test classes thus
executes all test methods of a test class and records historic data
with method level granularity and class level granularity. We need
to take all killing tests into account to match history data and
executable test units in the analyzer. This will also help if
different test classes cover a certain production class and one of
those test classes is deleted or renamed.

Improved logging to help spotting analyzer misbehaviour.

* remove log catcher after tests

Changing the configuration of the Logger is a global setting per JVM.
When running tests with surefire the JVM will be reused among tests.
Permanently changing the logging configuration might interfere with
other tests.
@Yuvarajahbk
Copy link
Author

Yuvarajahbk commented May 15, 2020 via email

@StefanPenndorf
Copy link
Contributor

StefanPenndorf commented May 23, 2020 •
edited
Loading

Hi Yuvaraja,

did you build the project using mvn clean install ? This should install the current snapshot version into your lokal .m2 directory. Thus no need for maven to query your company's nexus respository.

EDIT: Version 1.5.2 with this issue fixed has already been released. Just drop the -SNAPSHOT suffix and maven should download the latest version from central.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@StefanPenndorf @Yuvarajahbk