Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filter option can break tests dependency tree #110

Open
OndraM opened this issue Feb 14, 2017 · 1 comment
Open

Filter option can break tests dependency tree #110

OndraM opened this issue Feb 14, 2017 · 1 comment
Labels

Comments

@OndraM
Copy link
Member

OndraM commented Feb 14, 2017

When option --filter is used and it filter out all but one testcase, which has dependency on some previous one (which hasn't been run), the testcase is run despite of this (and may fail on some precondition).

For example, when run in

$ vendor/bin/steward run production firefox -vvv --filter DelayedExample
...
My\DelayedExampleTest> There was 1 error:
My\DelayedExampleTest> 
My\DelayedExampleTest> 1) My\DelayedExampleTest::testShouldReturnSearchResults
My\DelayedExampleTest> Lmc\Steward\Component\LegacyException: Cannot read Legacy file /.../steward-example/selenium-tests/logs/seed-data.legacy
My\DelayedExampleTest> 
My\DelayedExampleTest> /.../steward-example/selenium-tests/vendor/lmc/steward/src/Component/Legacy.php:200
My\DelayedExampleTest> /.../steward-example/selenium-tests/tests/DelayedExampleTest.php:32

Maybe tests with "No tests executed" should be marked as failed / or skipped? Or the --filter option could only be allowed for test methods, not for classes (because for classes the --pattern option should be used).

@OndraM OndraM added the bug label Feb 14, 2017
@OndraM
Copy link
Member Author

OndraM commented Feb 14, 2017

Maybe the value passed to PHPUnit --filter option could always be prefixed with something like ::, to make sure the filtering is only applied to test methods, not test classes. See https://phpunit.de/manual/5.7/en/textui.html#textui.examples.filter-patterns

However, this is only first part of the solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant