-
Notifications
You must be signed in to change notification settings - Fork 250
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Continuous Integration compatibility #220
Comments
Thanks for reporting this issue. I agree that this is a missing feature at the moment. What would you like the config to look like? We could combine this feature with a way to configure a high and low threshold. Those thresholds could be used by reporters. The For example: thresholds: {
high: 80,
low: 50,
break: 40
} Everything above 80% will be green, above 50% yellow, below 50 red, below 40 would set the Also, i think the @simondel, @CheshireSwift do you agree with this approach? Do you agree with the naming? Will the name |
All sounds good, though I'd personally like to be able to choose which mutation score breaks the build (I agree the full code base one is probably the sensible default). |
Here's the code I'm using. It creates stryker config based on which files were changed in the last commit (so only files in the src directory that were changed get mutated), runs Stryker and returns a non zero exit code if the mutant kill score was not within a specified threshold. I've used it in my CI system and it works well.
|
@jeznag Nice! I had the following scenario in mind:
Because I just don't want to rely on percentages and/or thresholds. |
Update: I created this baseline reporter. There are only two minor problems:
On the plus side, it is neat to be told when you introduced new mutants since the previous test run. |
…tion score * Add `thresholds` to the `Config` class and `StrykerOptions`. With this, you can configure the `high`, `low` and `break` values. * Move validation logic from `Stryker` class to a new `ConfigValidator`. * Add validaton for configured threshold values * Add a mechanism to exit with exit code 1 if the mutation score comes below the `thresholds.break` value. * Update the `ConsoleReporter` and `HtmlReporter` to make use of the threshold values for coloring. * Refactor stryker unit tests to now mock away functionality more cleanly * Update test command in package.json to allow for html coverage reporters Fixes #220
…#355) * Add `thresholds` to the `Config` class and `StrykerOptions`. With this, you can configure the `high`, `low` and `break` values. * Move validation logic from `Stryker` class to a new `ConfigValidator`. * Add validaton for configured threshold values * Add a mechanism to exit with exit code 1 if the mutation score comes below the `thresholds.break` value. * Update the `ConsoleReporter` and `HtmlReporter` to make use of the threshold values for coloring. * Refactor stryker unit tests to now mock away functionality more cleanly * Update test command in package.json to allow for html coverage reporters Fixes #220
Currently Stryker always returns exit code 0, which makes it hard to integrate with CI servers and similar tooling.
It would be good to have some way of reporting "failure". Ideally this would be via exit code and "failure" would probably be a configurable threshold of mutation score, though it could be a fixed value like "mutation score on covered code was less than 100%".
The text was updated successfully, but these errors were encountered: