-
Notifications
You must be signed in to change notification settings - Fork 112
Setup mutation testing #686
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
8a4e7cd
65db2c8
d3a6e60
69b69ca
baab8a3
b44bb9c
d223186
843fbfe
7a2ece5
788a450
7517bac
0cf3164
811f79d
cf457bb
547569a
ba0c6b8
bcc667f
6075749
71c74c4
c180a99
ae2e5ff
e680440
1398077
d1637c4
80978eb
054bb5e
2ae9a03
fb3db22
c37fc3f
871cd46
7e0fd4a
946fc28
e509b9a
2eecbb9
704bce2
19706cb
2caea8d
9f9de8f
c4d823e
93d3d97
01a9197
58e765a
378e518
b3b8b99
67cf382
c0e5252
d954a45
13ad641
3bddff7
91d8b14
60f5abd
34a70de
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -145,6 +145,55 @@ jobs: | |
| - name: "Tests" | ||
| run: "make tests" | ||
|
|
||
| mutation-testing: | ||
| name: "Mutation Testing" | ||
| runs-on: "ubuntu-latest" | ||
| needs: ["tests", "static-analysis"] | ||
|
|
||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| php-version: | ||
| - "8.2" | ||
| - "8.3" | ||
| - "8.4" | ||
|
|
||
| steps: | ||
| - name: "Checkout" | ||
| uses: actions/checkout@v5 | ||
|
|
||
| - name: "Install PHP" | ||
| uses: "shivammathur/setup-php@v2" | ||
| with: | ||
| coverage: "pcov" | ||
| php-version: "${{ matrix.php-version }}" | ||
| ini-file: development | ||
| extensions: pdo, mysqli, pgsql, pdo_mysql, pdo_pgsql, pdo_sqlite, mongodb | ||
| tools: infection:0.31.4 | ||
|
|
||
| - name: "Allow installing on PHP 8.4" | ||
| if: matrix.php-version == '8.4' | ||
| run: "composer config platform.php 8.3.99" | ||
|
|
||
| - name: "Install dependencies" | ||
| run: "composer install --no-interaction --no-progress" | ||
|
|
||
| - uses: "actions/download-artifact@v4" | ||
| with: | ||
| name: "result-cache-${{ matrix.php-version }}" | ||
| path: "tmp/" | ||
|
|
||
| - name: "Run infection" | ||
| run: | | ||
| git fetch --depth=1 origin $GITHUB_BASE_REF | ||
| infection --git-diff-base=origin/$GITHUB_BASE_REF --git-diff-lines --ignore-msi-with-no-mutations --min-msi=100 --min-covered-msi=100 --log-verbosity=all --debug | ||
|
|
||
| - uses: "actions/upload-artifact@v4" | ||
| if: always() | ||
| with: | ||
| name: "infection-log-${{ matrix.php-version }}" | ||
| path: "tmp/infection.log" | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should always upload this, not only when Infection passes
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. fixed There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hi there 👋 I'm coming from CuyZ/Valinor#721 @ondrejmirtes could you explain the reason why the artifacts should always be uploaded? Also, I'm not familiar with the artifact feature: when it's uploaded we can have access to the files directly?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Because I want to see the logs regardless of whether Infection suceeds or fails.
Yes, on job summary. Artifacts are useful to pass files between jobs, but also for letting users download them. Example https://github.com/phpstan/phpstan-src/actions/runs/18351063855 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Great! Thanks for the info. Have a nice day! 😊
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There is far more information in infection.log than what's in the linked screenshot. The log shows the JSON output PHPStan printed when it finished.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I had no idea it's possible to have the same thing on stdout that's currently uploaded in infection.log. If it's possible then I'd probably welcome a PR 😊 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 👀 waiting for an answer, to modify CuyZ/Valinor#721 before merging 👀 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If you really want to always see the most detailed information available for each build, then
So, the final command would be: Here, Similar approach is used e.g. by BetterReflection, they always print the whole list of escaped mutations to But you will have even more details thatnks to config needs to be updated: - "text": "tmp/infection.log",
+ "text": "php://stdout",There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. FYI @ondrejmirtes I just submitted infection/infection#2438 — thought you could be interested. |
||
|
|
||
| static-analysis: | ||
| name: "PHPStan" | ||
| runs-on: "ubuntu-latest" | ||
|
|
@@ -189,3 +238,9 @@ jobs: | |
|
|
||
| - name: "PHPStan" | ||
| run: "make phpstan" | ||
|
|
||
| - uses: "actions/upload-artifact@v4" | ||
| with: | ||
| # "update-packages" is not relevant for the download-artifact counterpart, but we need it here to get unique artifact names across all jobs | ||
| name: "result-cache-${{ matrix.php-version }}${{ matrix.update-packages && '-packages-updated' || '' }}" | ||
| path: "tmp/resultCache.php" | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| { | ||
| "$schema": "vendor/infection/infection/resources/schema.json", | ||
| "timeout": 30, | ||
| "source": { | ||
| "directories": [ | ||
| "src" | ||
| ] | ||
| }, | ||
| "staticAnalysisTool": "phpstan", | ||
| "logs": { | ||
| "text": "tmp/infection.log" | ||
| }, | ||
| "mutators": { | ||
| "@default": false, | ||
| "PHPStan\\Infection\\TrinaryLogicMutator": true | ||
| } | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| <?php declare(strict_types = 1); | ||
|
|
||
| namespace PHPStan\Infection; | ||
|
|
||
| use Infection\Mutator\Definition; | ||
| use Infection\Mutator\Mutator; | ||
| use Infection\Mutator\MutatorCategory; | ||
| use LogicException; | ||
| use PhpParser\Node; | ||
| use function in_array; | ||
|
|
||
| /** | ||
| * @implements Mutator<Node\Expr\MethodCall> | ||
| */ | ||
| final class TrinaryLogicMutator implements Mutator | ||
| { | ||
|
|
||
| public static function getDefinition(): Definition | ||
| { | ||
| return new Definition( | ||
| <<<'TXT' | ||
| Replaces TrinaryLogic->yes() with !TrinaryLogic->no() and vice versa. | ||
| TXT | ||
| , | ||
| MutatorCategory::ORTHOGONAL_REPLACEMENT, | ||
| null, | ||
| <<<'DIFF' | ||
| - $type->isBoolean()->yes(); | ||
| + !$type->isBoolean()->no(); | ||
| DIFF, | ||
| ); | ||
| } | ||
|
|
||
| public function getName(): string | ||
| { | ||
| return 'TrinaryLogicMutator'; | ||
| } | ||
|
|
||
| public function canMutate(Node $node): bool | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. atm this class replaces any call to a method named
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
| { | ||
| if (!$node instanceof Node\Expr\MethodCall) { | ||
| return false; | ||
| } | ||
|
|
||
| if (!$node->name instanceof Node\Identifier) { | ||
| return false; | ||
| } | ||
|
|
||
| if (!in_array($node->name->name, ['yes', 'no'], true)) { | ||
| return false; | ||
| } | ||
|
|
||
| return true; | ||
| } | ||
|
|
||
| public function mutate(Node $node): iterable | ||
| { | ||
| if (!$node->name instanceof Node\Identifier) { | ||
| throw new LogicException(); | ||
| } | ||
|
|
||
| if ($node->name->name === 'yes') { | ||
| yield new Node\Expr\BooleanNot(new Node\Expr\MethodCall($node->var, 'no')); | ||
| } else { | ||
| yield new Node\Expr\BooleanNot(new Node\Expr\MethodCall($node->var, 'yes')); | ||
| } | ||
| } | ||
|
|
||
| } | ||

There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Phpunit killer requires a green test run
Phpstan killer requires a green phpstan run