Cannot run a single test case with PHPStorm since 8.4.1 #3904
Comments
codekandis
added
type/backward-compatibility
sebastianbergmann
removed
the
type/backward-compatibility
|
This issue tracker is for the Open Source project that provides the PHPUnit testing framework as well as the CLI tool for executing tests. Please report issues with regards to IDE integration to the vendor of your IDE. That being said, please provide a minimal, self-contained, reproducing test case that shows the problem you are reporting and that does not involve PhpStorm, Vagrant, VirtualBox, etc. Without such a minimal, self-contained, reproducing test case I will not be able to investigate this issue. |
sebastianbergmann
added
the
status/waiting-for-feedback
|
Attached test scenario: phpunit-841-issue-testcase.zip First I have to correct the version constraint: The error occurs since PHPUnit 8.4.0. PHPStorm runs /usr/bin/php /vagrant/vendor/phpunit/phpunit/phpunit --configuration /vagrant/phpunit.xml Vendor\\ProjectNamespace\\UnitTests\\AnotherTest /vagrant/tests/UnitTests/AnotherTest.php --teamcityThis would result in a Vagrant SSH call vagrant ssh --command "/usr/bin/php /vagrant/vendor/phpunit/phpunit/phpunit --configuration /vagrant/phpunit.xml Vendor\\ProjectNamespace\\UnitTests\\AnotherTest /vagrant/tests/UnitTests/AnotherTest.php --teamcity"After some testing I focused on the escaped backslash namespace separator and I had minimized the tests to the following two IDE and Vagrant independent scenarios. single backslash namespace separators /usr/bin/php vendor/phpunit/phpunit/phpunit --configuration phpunit.xml Vendor\ProjectNamespace\UnitTests\AnotherTest tests/UnitTests/AnotherTest.php --teamcity
double backslash namespace separators /usr/bin/php vendor/phpunit/phpunit/phpunit --configuration phpunit.xml Vendor\\ProjectNamespace\\UnitTests\\AnotherTest tests/UnitTests/AnotherTest.php --teamcity
Conclusion With the tests above it's obvious now since PHPUnit 8.4.0 the esacped backslash namespace separators are interpreted differently. To reproduce my tests change the PHPUnit version in the |
sebastianbergmann
removed
the
status/waiting-for-feedback
|
@flow-control Is this yet another side effect of (y)our cleanup? |
|
@sebastianbergmann could totally be. I will check this weekend if I can find the cause to this regression. There are so much special cases, time to get #3859 done 😉 |
|
@codekandis thanks for reporting the issue. Could you do me a favour and try the call without the class name and namespace? /usr/bin/php vendor/phpunit/phpunit/phpunit --configuration phpunit.xml tests/UnitTests/AnotherTest.php --teamcity |
|
@flow-control I confirm it works. |
|
Hey @codekandis, i tried to break this down to simple example, but could not reproduce.
<?php declare(strict_types=1);
/*
* This file is part of PHPUnit.
*
* (c) Sebastian Bergmann <sebastian@phpunit.de>
*
* For the full copyright and license information, please view the LICENSE
* file that was distributed with this source code.
*/
namespace Test\Foo\Bar;
use PHPUnit\Framework\TestCase;
final class Test extends TestCase
{
public function testOne(): void
{
$this->assertTrue(true);
}
}and then run $ ./phpunit Test\\Foo\\Bar\\Test Test.phpgives me PHPUnit 8.4.0 by Sebastian Bergmann and contributors.
Runtime: PHP 7.3.10 with Xdebug 2.7.2
Configuration: /home/bowman/Work/flow-control/phpunit/phpunit.xml
. 1 / 1 (100%)
Time: 22 ms, Memory: 6.00 MB
OK (1 test, 1 assertion)same with PHPUnit The only thing i could find is that if i misspell the namespace or class name it will not work in Could you double check that the namespace and class name is correct? Or could you create and paste here a minimal test file that reproduces the behavior you have experienced? Kind regards |
|
Hey @flow-control , I reproduced this bug in my project. You have to use 4 times backslash as separator in cli. Clone my project https://github.com/dadrych/phpunit_8_4_1_bug, run |
|
Hey @dadrych, you should use two backslashes, otherwise PHPUnit will get the string $ php vendor/phpunit/phpunit/phpunit --configuration=phpunit.xml --filter "/(::testFoo)( .*)?$/" App\\Tests\\FooTest tests/FooTest.php
PHPUnit 8.4.1 by Sebastian Bergmann and contributors.
Runtime: PHP 7.3.10 with Xdebug 2.7.2
Configuration: /home/bowman/Work/phpunit_8_4_1_bug/phpunit.xml
. 1 / 1 (100%)
Time: 28 ms, Memory: 4.00 MB
OK (1 test, 1 assertion)
This is a feature that in This #3889 (comment) explains it pretty well /Flo |
|
I think it only happen with PHPStorm, I have same issue when I run my test from PHPStorm but when I copy the command to terminal it works fine. |
|
Related to PHPStorm bug https://youtrack.jetbrains.com/issue/WI-49193 |
Please take a look at the both scenarios I posted above. The are two commands I've run on the CLI of my standard The focus has been on IDE independent and VM independent scenarios. So it's definitely not PHPStorm related. |
|
@flow-control Try without using the filter. |
|
Hey @codekandis, i tried without the filter: [bowman@localhost phpunit_8_4_1_bug]$ php vendor/phpunit/phpunit/phpunit --configuration=phpunit.xml App\\Tests\\FooTest tests/FooTest.php
PHPUnit 8.4.1 by Sebastian Bergmann and contributors.
Runtime: PHP 7.3.10 with Xdebug 2.7.2
Configuration: /tmp/phpunit_8_4_1_bug/phpunit.xml
. 1 / 1 (100%)
Time: 17 ms, Memory: 4.00 MB
OK (1 test, 1 assertion)Works as expceted Kind regards |
|
Ok, so however. It's definitely not PHPStorm related. |
|
@codekandis Can you please try the current state of |
|
@sebastianbergmann I can confirm it's working in |
|
@codekandis Thank you. PHPUnit 8.4.2 will be released later today. |
|
@sebastianbergmann I updated PHPUnit and |
|
We have the oposite side-effect, where it doesn't work in 8.4.2 but works in <=8.4.1 8.4.2 reverting to 8.4.1 As I see it, the only diff is the added brackets on the filter option. Do I need to open a new ticket for this @flow-control? |
|
Hey @danijelk, this looks like another problem to me, because the test files seems to be found, but no tests in that file are matching the /Flo |
|
Without brackets gives us some useful debug info I narrowed down the cause. I have 2 files, 1 test class (GraphExportEmailTest), and one class that other test classes extend (MailTracking) which extends TestCase. So the 8.4.2 seems to just scan the parent MailTracking class and not the actual test class itself as it did in 8.4.1. Hope this helps. Else let me know and I'll try to make a self contained example if you cannot replicate with an extended class. |
|
@flow-control I think we have reached a point where we should at least consider to revert all changes we made here for PHPUnit 8.4 and then take the time needed to figure out the effects of these changes. Then we can decide on a case by case basis whether we want that effect (and (possibly) break BC) or not. |
|
@sebastianbergmann i think so, too. I will prepare a pull request /usr/local/bin/php /Volumes/c/development/pr/vendor/phpunit/phpunit/phpunit --configuration /Volumes/c/development/pr/phpunit.xml --filter "/(::make_sure_branding_works)( .*)?$/" Tests\\UnitTest\\Reports\\GraphExport\\GraphExportEmailTest /Volumes/c/development/pr/tests/UnitTests/Reports/GraphExport/GraphExportEmailTest.php --teamcityI just only changed to using two backslashes (one is for escaping the other). To me it looks like there is another fallback somewhere after the |
Same issue with your command. Here is the repo for the failed test https://github.com/danijelk/phpunit_8_4_2_bug Let me know if there is anything more I can do. |
|
Thanks very much, i found out why this is not working in One hint i have found while debugging this: on the Thanks again, i will add that example to the test pipeline, while reverting the cleanup. |
|
I have issue with PHPStorm and WSL: PHPStrom execute this command: After that I have result: But when I remove --filter everything work |
|
Hey there, could you provide that test file? Kind regards |
|
This is simple Laravel's example test: I've used And this commad works properly. |
|
AFAIK this is fixed in the latest PhpStorm EAP version. I'm running it and it works, it didn't with the latest 2019.3* version before. |
|
@mfn Yes, I've just downloaded EAP version and it works perfectly. |
|
I've had the same issue. It's working with version PhpStorm |
|
Also experiencing a similar issue with PHPStorm 2019.3.3 and PHPUnit 9.0.1. Running a single method/class test appears to run this command Which fails saying But manually running it as Seems to pass. I have dropped back down to PHPUnit 8.5.2 and PHPStorm can run single method and class tests fine, and it still runs the same command. |
|
@oniice same issue here with with PHPStorm 2019.3.3 and PHPUnit 9.0.1. |
|
Hey @iranianpep, @oniice, @backstageel, this is a problem with how PHPStorm executes PHPUnit, could you open a bug report with PHPStorm? If you do so, it would be nice if you post the link to the ticket here. The problem is as @oniice spotted correct, that the class argument /Flo |
|
Thanks @flow-control! Glad it's confirmed as a PHPStorm issue - they always seem a few steps behind world libs 🤦🏻♂️ Will get on that :) |
|
Hi there, I found a ticket for this in their issue tracker. Looks like the upcoming /Flo |
Summary
I'm using the mentioned PHP / PHPUnit setup in a Vagrant VirtualBox VM. Since PHPUnit 8.4.1 I cannot run a single test case from the context menu of PHPStorm without a fatal error.
Previous Behavior
Until 8.4.0 it was possible to run single test cases.
Current Behavior
Running all test cases from a folder context menu or running a single test from inside a test case are not affected. They still run as expected.
How to reproduce
Install PHPUnit 8.4.1 and select
Run 'SomeClassTest (PHPUnit)'.The text was updated successfully, but these errors were encountered: