Tests Reporting Failed Without Mutation Using Pitest >= 1.6.2

[scampbell5]

Hello,

We have recently upgraded to pitest 1.6.4 and started experiencing issues where pitest says tests did not pass without mutation. If I change the version back from 1.6.4 to 1.6.1, the issue goes away and pitest runs successfully. It looks like the issue may have been introduced in version 1.6.2 of the plugin. I created a very minimalistic project outlining our issue:

https://github.com/scampbell5/pitest-example

Version 1.6.4 failing:
https://github.com/scampbell5/pitest-example/runs/2126518135?check_suite_focus=true#step:4:7300

Version 1.6.1 passing:
https://github.com/scampbell5/pitest-example/runs/2126520186?check_suite_focus=true#step:4:7320

[hcoles]

Hi @scampbell5

Thanks for the report and the minimal project, always a huge help.

The issue looks to be the change in modifiers for the coverage probe field. Previously this was always marked as final, but this change

#850

required that it be made non final in order to deal with a nasty initialisation order issue.

Is seems that the assertion library you are using is expecting a getter and setter to exist for all fields in the class, including those introduced by pitest.

I think this is actually a bug in that assertion library. The probe field is marked as synthetic, so should be ignored by all well behaved tooling. It seems the library previously ignored the probe field due to the final modifier.

It may be possible to work around it in pitest by reducing the scenarios in which the field is made non final, perhaps keeping the final modifier if the instrumented class decends directly from object. The issue would still remain in more complex scenarios however, so I think it really needs to be fixed in the assertion library. The same problem may also occur with other coverge tools.

[scampbell5]

Hi @hcoles,

Thank you for such a well written and detailed reply on this issue. I agree that it sounds like an issue that should be fixed in the openpojo library. I've opened up an issue on their repository along with a PR to fix the issue. Please feel free to close this issue. Thanks again for your assistance with this!

[hcoles]

Closing as this looks to be resolved in an openpojo release.