[PHPUnit bridge] Support for PHPUnit 11.1+

[Jean85]

Description

PHPUnit 10 is around the corner (possibly released next month), and it contains a complete overhaul of its internals, particularly the new event system: sebastianbergmann/phpunit#4676

The new system also provides an easy way too hook into those events: https://github.com/sebastianbergmann/phpunit/blob/2a89c3c62c4d94fa92b72cb3693ca4fa057e4517/src/Event/Facade.php#L40
Among the events, there are even multiple dedicated to deprecations:

• triggered by E_USER_DEPRECATED: https://github.com/sebastianbergmann/phpunit/blob/main/src/Event/Events/Test/Issue/DeprecationTriggered.php
• triggered by E_DEPRECATED: https://github.com/sebastianbergmann/phpunit/blob/main/src/Event/Events/Test/Issue/PhpDeprecationTriggered.php
• triggered manually by PHPUnit itself: https://github.com/sebastianbergmann/phpunit/blob/main/src/Event/Events/Test/Issue/PhpunitDeprecationTriggered.php

This maybe warrants a discussion here on how to tackle this new release, and how to leverage all this new features.

Example

No response

[Jean85]

I should add that those events are triggered through an error handler, and that having the bridge installed actively breaks it.

Maybe we should disable (and later disband) that feature under PHPUnit 10+?

[stof]

Well, I don't think PHPUnit 10 contains the existing deprecation-related features of the bridge (at least not all of them).

However, we should look at rebuilding them on top of the PHPUnit events instead of replacing the error handler with our own for PHPUnit 10+.

[sebastianbergmann]

I discussed PHPUnit 10's event system with @nicolas-grekas at SymfonyCon in November. I think the way forward for Symfony is to no longer register its own error handler for the bridge, let PHPUnit's own error handle emit events for deprecations etc., and then have the bridge subscribe to the events it is interested in. If you want the bridge to produce different output for deprecations than what PHPUnit offers out of the box.

[dozer111]

Do u plan to fix it in 6.3?

[arderyp]

now that PHPUnit 10 is out, I'm giving this a shot. I experience the same error noted here: symfony/recipes#1173

@nicolas-grekas notes here that this should resolve the problem.

It looks like that was merged, so I'm wondering what the next step is here.

[Jean85]

Next step would be adding the PHPUnit 10 support to the deprecation handler; this probably means a nearly total rewrite of it, to be done side-by-side to the old one, since the versioning of the bridge is tied to the framework.

[arderyp]

Sounds pretty labor intensive.

Would you say this is a priority of the team, or consumers shouldn't expect to see 10 support for quite some time?

[stof]

Well, this indeed requires work. Whether this will be done in time for 6.3 depends of when someone works on it (however, it would be great to have it in 6.3 rather than waiting for 6.4 if possible, given that PHPUnit 10 is stable).
Help is welcome on that topic (but that's indeed not an easy task).

[Jean85]

Can we at least consider a quick patch were we completely disable the error handler if we detect PHPUnit 10? As of now, the simple presence of the symfony/phpunit-bridge is messing up my CI on facile-it/paraunit#192, see:

• green build on main branch: https://github.com/facile-it/paraunit/actions/runs/4134171677
• red on PR: https://github.com/facile-it/paraunit/actions/runs/4207911964/jobs/7304460539
• commit to remove the bridge: facile-it/paraunit@94874a6
• then (nearly) all failures disappear, remaining are due to missing ClockMock class: https://github.com/facile-it/paraunit/actions/runs/4212928270/jobs/7312296419

I'm even setting SYMFONY_DEPRECATIONS_HELPER to disabled but it seems to not solve the issue. I'll try to set it in real env variables now, but IMO we should disable the whole flow under PHPUnit 10.

[EDIT] Using a real env var fixed it: https://github.com/facile-it/paraunit/actions/runs/4213058997/jobs/7312518640
So it's clear that we need that patch.

[arderyp]

@Jean85 I like your idea! I'd rather be able to stay on a neutered phpunit-bridge and let it stay silent until its ready rather than switching my project over to raw phpunit/phpunit temporarily and then remembering to switch back

[arderyp]

@Jean85 are you saying this can be worked around temporarily by setting SYMFONY_DEPRECATIONS_HELPER=false at the core os/linux level, but setting the same var in .env or phpuni.xml won't work?

[Jean85]

Yes, exactly.

The bootstrap of the bridge is loaded through the autoloader, so long before DotEnv or PHPUnit config can kick in. They can still influence the bridge behavior, but the error handler is already registered (and causing the issue) at that point.

[arderyp]

@Jean85 I am unable to reproduce your workaround.

I removed phpunit/phpunit (I was explicitly including ^10.0). Then I did composer require --dev symfony/test-pack. It installed bin/phpunit and it also installed added phpunit/phpunit:^9 to my composer.json. So then I had to require phpunit/phpunit:^10.0 --with-all-dependencies to get phpunit back up to 10.

when I run ./bin/phpunit

$ ./bin/phpunit --version
PHP Fatal error:  Uncaught Error: Class "PHPUnit\TextUI\Command" not found in /home/arderyp/app/bin/phpunit:11
Stack trace:
#0 {main}
  thrown in /home/arderyp/app/bin/phpunit on line 1

Similar issues when I run ./vendor/bin/phpunit I get errors.

PHP Fatal error:  Uncaught Error: Class "PHPUnit\TextUI\Command" not found in /home/arderyp/app/vendor/symfony/phpunit-bridge/Legacy/CommandForV9.php:25
Stack trace:
#0 /home/arderyp/app/vendor/bin/.phpunit/phpunit-10.0-0/vendor/composer/ClassLoader.php(571): include()
#1 /home/arderyp/app/vendor/bin/.phpunit/phpunit-10.0-0/vendor/composer/ClassLoader.php(428): Composer\Autoload\includeFile()
#2 [internal function]: Composer\Autoload\ClassLoader->loadClass()
#3 /home/arderyp/app/vendor/symfony/phpunit-bridge/TextUI/Command.php(17): class_alias()
#4 /home/arderyp/app/vendor/bin/.phpunit/phpunit-10.0-0/vendor/composer/ClassLoader.php(571): include('...')
#5 /home/arderyp/app/vendor/bin/.phpunit/phpunit-10.0-0/vendor/composer/ClassLoader.php(428): Composer\Autoload\includeFile()
#6 /home/arderyp/app/vendor/bin/.phpunit/phpunit-10.0-0/phpunit(22): Composer\Autoload\ClassLoader->loadClass()
#7 /home/arderyp/app/vendor/symfony/phpunit-bridge/bin/simple-phpunit.php(441): include('...')
#8 /home/arderyp/app/vendor/symfony/phpunit-bridge/bin/simple-phpunit(13): require('...')
#9 /home/arderyp/app/vendor/bin/phpunit(120): include('...')
#10 {main}
  thrown in /home/arderyp/app/vendor/symfony/phpunit-bridge/Legacy/CommandForV9.php on line 25

Defining hard env var SYMFONY_DEPRECATIONS_HELPER=disabled doesn't change the ouput. The error is occuring because both executables are looking for this PHPUnit 9 class/method, which doesn't exist in PHPUnit 10.

I'm wondering how you managed to get phpunit 10 working with symfony/test-pack at all. Or are you just explicitly calling /vendor/phpunit/phpunit/phpunit to run your tests?

$ ./vendor/phpunit/phpunit/phpunit --version
PHPUnit 10.0.11 by Sebastian Bergmann and contributors.

[Jean85]

Yeah I'm not using the test pack, just plain PHPUnit + the bridge. I was using it just for the deprecation helper (and the clock mock).

[arderyp]

hmmm, okay. Maybe I will try that. I was using phpunit-bridge previously, but it looks like the symfony official documentation has been updated to instruct us to install symfony/test-pack instead of symfony/phpunit-bridge, which is the only reason I was attempting to install that. test-pack does install phpunit-bridge

[arderyp]

@stof @nicolas-grekas @dunglas i can work on this and submit a PR if you all are interested and have time to review my work. I've done so for symfony panther, though the work still isn't merged: symfony/panther#589

The change log there could be reduced if we get this package phpunit 10 compliant first, as the work there currently requires a workaround the lack of compliance here.

Are you all interested?

[nicolas-grekas]

There are different tasks on the topic. One of them is ensuring test suites can run on phpunit 10. We've already merged some changes on this topic from @OskarStark and @alexandre-daubois mostly. The first step is running deprecation-free on 9.6.
Then, there is figuring out what to do with the bridge itself? I must admit I'm 🙈 for now but sure, any help would be appreciated!

[arderyp]

Thanks for the info @nicolas-grekas. Would it be helpful for me to start work or hold off until those other items are addressed first?

Part of the question is whether we are merely adding support for 10, or also dropping support for some older versions(s) too

[alexandre-daubois]

I think we're almost done with deprecations brought by PHPUnit 9.6 on the Symfony code base 🙂 Once that's done, I'll have a look at the bridge if I can also do something to help. @arderyp I think that if you have ideas on how to bring support of PHPUnit 10, you should definitely go ahead! 😄

[arderyp]

Thanks for the insight @alexandre-daubois. Is there a specific branch I should be working off of?

[arderyp]

@alexandre-daubois @nicolas-grekas will/should 5.4 support phpunit 10? If I work on this, should I be working off the 5.4 branch? I'd certainly like 5.4 to support it.

[vinceAmstoutz]

An official post would be most welcome. I'm confused about the bridge, I'd probably be happy just using PHPUnit itself, but so much of the testing documentation for Symfony uses the bridge that it's hard to separate. Thanks for the link to the slideshow!

Yes, it's clear that it's not always very clear what role bridge, the test pack and the library play in relation to each other.

[mondrake]

Hi, thanks for the update.

In latest PHPUnit 11 I still do not find an alternative to the deprecation ignore file. We use it a lot in Drupal. I saw that the @group legacy annotations can be replaced by the #[IgnoreDeprecations] attribute, but nothing on the file.

Is that planned if you know?

FTR, I will close now the MR #50371

[stof]

PHPUnit also has a concept of baseline (in 10.4+): https://docs.phpunit.de/en/11.0/error-handling.html#ignoring-previously-reported-issues

[mondrake]

Yes I saw that, @stof, thanks. I was inquiring on the ignore file since in Drupal we did not implement the baseline, and it will be rather hard to do in our testing framework.

[stof]

The plan for the new version of the bridge is to drop all features related to the deprecations, by using the features of PHPUnit. If the current PHPUnit baseline feature is not usable for you, I would suggest discussing that with the PHPUnit team.

[soyuka]

I guess phpunit 11.2 now supports direct/indirect (https://docs.phpunit.de/en/11.2/error-handling.html#limiting-issues-to-your-code) and a baseline therefore we don't need the phpunit bridge anymore this should probably be closed in favor of #53812.

[arderyp]

So will the test-pack drop the bridge component? That would be nice

[upyx]

So will the test-pack drop the bridge component? That would be nice

Bridge does many things, and PHPUnit isn't the only part of TestPack. If you'd like to install PHPUnit only, you can do it. You do not have to install the whole TestPack.

If I remember right, the original main goal of Bridge was compatibility between tests written atop different PHPUnit versions. Imagine you have a lot of legacy tests in PHPunit (6.x, 7.x, etc.). It would take a while to update them, and there is no guarantee that a bug wouldn't be added. So we need a way to run old tests alongside new ones.

[derrabus]

That is correct. You don't have to install PHPUnit through the bridge if you don't want to. Use PHPUnit 11 and be happy with it. Symfony's own test suite is stuck on PHPUnit 9.6 at the moment, but don't let this stop you.

[stof]

Also note that the PHPUnit bridge is not only about the deprecation reporting layer. Once we work on using the PHPUnit native deprecation reporting system on 11.2+, we should provide a version of the bridge supporting its other features on PHPunit 11

[soyuka]

Also note that the PHPUnit bridge is not only about the deprecation reporting layer. Once we work on using the PHPUnit native deprecation reporting system on 11.2+, we should provide a version of the bridge supporting its other features on PHPunit 11

Let's see what can be done! Mocks should already work with phpunit 11, is there a repository where we could move these and avoid the phpunit 9 dependency lock?

For other features I think that most are not useful to Symfony developers but let's see each documented feature.

1. Sets by default a consistent locale (C) for your tests (if you create locale-sensitive tests, use PHPUnit's setLocale() method);

setLocale is deprecated but anyways is not useful, just use the native setlocale if you really need this (sebastianbergmann/phpunit#5200 (comment))

2. Auto-register class_exists to load Doctrine annotations (when used);

Not needed anymore, use attributes.

3. It displays the whole list of deprecated features used in the application;

--display-deprecations

4. Displays the stack trace of a deprecation on-demand;

--display-deprecations outputs the stack trace

5. Provides a ClockMock, DnsMock and ClassExistsMock classes for tests sensitive to time, network or class existence;

Any idea where to move these?

6. Provides a modified version of PHPUnit that allows:
   a. separating the dependencies of your app from those of phpunit to prevent any unwanted constraints to apply;

Not sure why this is needed by Symfony users this should probably stay internal.

b. running tests in parallel when a test suite is split in several phpunit.xml files;

If you need this you will probably prefer ParaTest or --process-isolation

c. recording and replaying skipped tests;

I never needed this but there's no equivalent right now as far as I know.

7. It allows to create tests that are compatible with multiple PHPUnit versions (because it provides polyfills for missing methods, namespaced aliases for non-namespaced classes, etc.).

Same as above I think that this is mostly for Symfony tests and it should probably stay internal.

[nikophil]

I don't think that --display-deprecations outputs the stack trace

[derrabus]

is there a repository where we could move these and avoid the phpunit 9 dependency lock?

There is no dependency lock. You can install the PhpUnitBridge together with PHPUnit 11. Or am I missing something?

[AnjaLiebermann]

Is there any hope of symfony/phpunit-bridge supporting the newer phpunit versions?
ClockMock is a fantastic feature, but with any of my projects, which doesn't need it I threw the bridge away and use phpunit 11.3 directly.

[derrabus]

ClockMock is a fantastic feature

If you want to work on making ClockMock compatible with PHPUnit 11, please do so. It's not actively being worked on, as far as I know.