Add Infection #29
Add Infection #29
Conversation
|
I love the idea of incorporating mutation testing in this project. I want tests to test my tests! 💡🥳 |
…ation false positive and the computational cost of unnecessary casting
…avoiding muation false positives and the computational cost of unnecessary castings
…riables such as $_SERVER, $_POST and $_GET, etc... are restored in each test
It is not being used anywhere outside the class
|
You might want to take a look at this: https://infection.github.io/guide/mutation-badge.html |
|
I am using macos, and the Using I would suggest to use 8 threads 😄 |
| run: composer install --no-interaction --no-ansi --no-progress | ||
|
|
||
| - name: Run mutation tests | ||
| run: ./vendor/bin/infection --show-mutations --threads=$(nproc) --min-msi=100 --min-covered-msi=100 |
There was a problem hiding this comment.
You can use --threads=max that will automatically get the number of cores on any OS
There was a problem hiding this comment.
Thank you very much @maks-rafalko! The truth is that we discovered it yesterday while we were adding it to Gacela too.
There was a problem hiding this comment.
Thanks, @maks-rafalko. I already changed it here 84df2cc
Chemaclass
added
pure testing
📚 Description
Out of curiosity, I have installed and configured Infection in the project. It is a library that allows applying the mutation testing methodology in PHP. This methodology makes small modifications to the production code to try to detect whether the test suite detects these changes or not. Each of these small changes should always be detected by one of our tests. If not, the library reports it so that we can act accordingly.
This library thus makes our tests better, helping us achieve 100% coverage and ensuring that we test all edge cases. However, like everything else, being an automated process, it can also produce false positives. Therefore, it is up to our discretion to decide whether to fix or ignore these failures.
I launched the command to take a look and the truth is that we got a pretty good results. However, it still detected that one mutation had no coverage, 14 were not detected by any test, and 8 timed out. Nonetheless, we have some enviable metrics!
I'm creating this draft to work on it, so you can take a look and decide whether to incorporate this methodology (perhaps to apply it to the other Gacela modules) and to check the errors it has detected. We can also add it as a step in the CI pipeline.
🔖 Changes
🤔 Considerations
Since my computer has 16 cores, the script's execution time is exactly 10 seconds, which is the default timeout amount. Currently, we have 8 timeouts, so on computers with less than 8 cores, the execution time will be between 20 and 80 seconds.
We need to study whether these timeouts are something we can fix (or ignore), so that we ensure they don't potentially add 10 seconds each on slower computers (those with a single core). However, since our test suite is so fast, another option we have is to drastically reduce the timeout amount. Nevertheless, as I mentioned earlier, we need to consider what to do in this situation.