-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Provide a finer grain control on mutators to enable #114
Comments
I would like to work on that topic, but would like to add this in the run config to be more flexible. |
That's great! But please, keep in mind that you should also write a few UI tests for such new features. |
I have looked into it, I just realized there are 150 mutators. Would you think to have full control over them would be nice or should we just present some of them? |
Hey there! Just so you know I've already worked on this feature in the The branch adds a table to the Pit tab in the Run Configuration view that allows to select the mutators to run (the user can either selects pre-defined groups or selects a custom subset of mutators): I think I added all the mutators that were described in Pitest's website at that time; they are all defined in the I think that I was almost done with the UI part but didn't start to actually use it to configure the Pitest run. So the Notice that with my implementation the user can also select a pre-defined subset of mutators (DEFAULTS, STRONGER or ALL) which corresponds to a group defined by Pitest — looks like there is now an OLD group as well. Groups are defined in the |
Thanks for the quick reply. I will take a look at your code and would like to use the mutators from pitest itself from the class "Mutator" in the package "org.pitest.mutationtest.engine.gregor.config". That would bring the benefit, if a new mutator gets added to pitest, the mutator would be selectable without any new implementation in pitclipse. Any thoughts on that idea? |
A different idea I had was that you could use the defined groups of the "Mutator" class to have different sections of the selection. |
Indeed. And Pitclipse wouldn't break if Pitest renamed or removed some mutators. However since Pitest provides neither a human-readable name nor a description of its mutators then we'd still have to implement them anyway (which shouldn't be a big deal, just saying that although it could make upgrading Pitest safer it doesn't remove all the work). Moreoveor it looks like raw mutators and groups are stored in the same Do you know if this class is API? If it's likely to change then we may want to avoid using it.
It looks like there is about a dozen groups defined in this class and I don't know if it's really relevant to provide so many of them (I'm not a regular user of Pitest). If it does make sense to provide them all then we the UI should has a combo box rather than the list group I implemented. Besides that it has about the same pros & cons than querying the mutators right from Pitest's |
Not yet, but I plan to look at this today or tomorrow.
I don't know if it is Api. How would I know if it is? Besides asking in the repo of pitest. |
For now I will use your implementation and get it working. |
I think it is crucial to keep this completely maintainable and completely self-updating when updating to a new version of PIT. For this reason, IMHO, we cannot afford to hardcode anything and take what's available from PIT itself. Maybe @hcoles can confirm that there's some kind of API to retrieve Mutators descriptors? I would stick with IDs, which, if I remember correctly should be quite clear to understand and skip human-readable descriptions. After all, if anyone wants to have such a power on the configuration, they might have to be already aware of Mutators identifiers? |
How about a third option: We use the mutators given by pit to build the table for selection and use the hard coded descriptions for all mutators which are currently implemented. If new ones get added, the description could say: "No description yet, look at pitest.org for more info". |
That would indeed be ideal.
Some IDs are not intuitive at all: UOI, AOR, CRCR. And often not enough to figure out the whole effect of the mutator:
It may not be a big deal if the user is already used to Pitest, but...
I disagree. Pitclipse aims at easing mutation testing of Java projects within an Eclipse environment and as such I believe that we should expose as little internal details as possible. Users may start to use Pitclipse without prior knowledge of Pitest and I'd rather provide them all the info they need right in the IDE rather than asking them to look at some doc on the internet. Even Pitest users may be unsure about the meaning of an ID and may have to take a look at the doc, which feels inconvenient. Providing a comprehensible table, with a readable name and a clear description, may require a bit more work from our side but IMHO it also greatly enhances the user experience.
Yes, that's what I expect if we decide to gather the mutators from Pitest; ideally we would provide the description at the same time we update Pitest though. The "no description" text would be a placeholder in case we fail to notice the addition of a new mutator.
@JKutscha I'm not sure I understand you. What I envision is that:
Hence when upgrading Pitest we'd just have to make sure the label providers are up-to-date. Everything else woud be automatic. Does your "third option" differ from that?
@LorenzoBettini I agree but as I said above I also think that providing a clear table would come in handy for users. With the solution I proposed Pitclipse would be automatically updated, we'd just have to update the label adapter when the mutators provided by Pitest change (assuming that there's no existing API to get a mutator description). IMO that's acceptable. What do you think about that? |
I was talking about this behavior, but wasn't able to describe it good enough. |
@echebbi as long as any of you commit to keeping it up-to-date I'm perfectly fine ;) O:) |
I now have looked into it a little more. The problem I see currently there is no way to get the keys of the mutators, which can be given to the cli of pit to use the mutator. Also it seems to there is no actual naming theme of the keys, which would allow to generate them from their names. |
@LorenzoBettini Ahah, fine, I'll take care of that.
@JKutscha I didn't notice that Pitest's Would you mind creating an issue in the Pitest repository to explain our situation and ask for a way to programmatically query all the IDs of mutators handled by Pitest? If they have no API yet we might want to propose to contribute by adding a new method that does the job in Mutator.java, something like |
Great! :)
For the time being we could do a "dirty" thing: access the map via reflection?
Of course, asking for an API is the best choice! |
@echebbi I would create the issue and a corresponding PR. This shouldn't be much effort to provide that. Seeing all the open issues and PRs, my hope of getting it integrated is quite low. At least for the next few days or weeks. |
Thank you very much @JKutscha!
If it's just a matter of days then it's not a big deal. If the PR takes too much time to be integrated or if it's refused then we'll consider another solution. As @LorenzoBettini suggested, reflection could be a quick & dirty fix. |
@echebbi I implemented it with reflection in the draft and it works. I marked it to be replaced, if the PR is merged and we can use it. |
Now that we use PIT 1.6.8, you can access them without reflection |
Done, forgot to push that change. |
Sorry if this isn't the right place, but it seems although this has been merged, it hasn't been released yet (at least it's not part of version 2.1.2 which is the one available on Eclipse). If so, how could I try it (is there any guide on how to build the latest state of this repo and use it in Eclipse)? There is a bunch of mutators I'd like to enable without going for the "all mutators" option. Thanks in advance! |
@contivero you're right: it hasn't been released yet, and I had always forgotten about that. @echebbi do you think it's time we make a new release with the new features? I can do that in the next few days. |
Motivation
As suggested by @LorenzoBettini in #77 (here)
Proposed Solution
Enhance the preferences page with a table displaying the full list of options (we should be able to support all the options offered by Pitest) which a checkbox to enable/disable each.
The text was updated successfully, but these errors were encountered: