Execution time increases dramatically with process isolation

[v1nc3n4]

Hi,

I'm facing an important performance issue with tests execution when 'process isolation' is enabled.
The context is a web application (Symfony 2.5), and automated tests are executed with phpunit 4.1.1 in a continuous integration server (Jenkins up-to-date), on an Ubuntu 14.04 LTS 64 bits workstation.

When I enable process isolation in my phpunit XML configuration, I can see execution time dramatically increases. My test suite contains 12 tests and 148 assertions.

Without process isolation, the test suite is executed in 4 min 30 s.
With process isolation, the test suite is executed in 28 min 30 s.

I've repeated the tests with different PHPunit PHAR archives:
With PHPunit 4.1.1 and PHPunit 4.1.3, the same issue occurs.
With PHPunit 3.7.30, no issue, the test suite is executed "normally", without significant differences.

It seems to me the execution time increases at the end of each test, just like if it was very hard with process isolation to clean everything, and terminate the test's process. However, I can't tell exactly what's happening.
Moreover, on my own workstation (Windows 7), with the command line, there is no particular issue with process isolation.

Thanks in advance for your clarifications.
BR

[sun]

Related #950

[sun]

context is a web application (Symfony 2.5) […]
execution time increases at the end of each test, just like if it was very hard with process isolation to clean everything, and terminate the test's process

That sounds fairly related to #1351, which has been merged into master (4.3).

Could you try whether you're still experiencing this issue when running your tests with phpunit master?

[v1nc3n4]

Unfortunately, I can't build phpunit on Windows 7.
I seems the build process in the Ant file is not portable on Windows O/S.
If someone can build a PHAR of the master branch, and send it to me, sure I'll do the same test with this archive.

[sebastianbergmann]

Please read https://github.com/sebastianbergmann/phpunit/wiki/Release-Process. Nightly builds from the master branch are available at https://phar.phpunit.de/phpunit-alpha.phar

[v1nc3n4]

Ah ok, thanks Sebastian.

[sebastianbergmann]

Please note that both https://phar.phpunit.de/phpunit-alpha.phar and https://phar.phpunit.de/phpunit-beta.phar do not seem to work atm. I am trying to investigate what's going on but am pretty busy with other stuff today.

[sebastianbergmann]

Okay, the alpha and beta PHARs work fine on the server they are built on but not on my local machine. No clue what's going on here but I have no time to investigate right now.

[whatthejeff]

@sebastianbergmann Could it be from f220079? I don't think SCRIPT_NAME is guaranteed to have the full path like __FILE__.

[whatthejeff]

I think you'd want something like:

if(__FILE__ === realpath($GLOBALS['_SERVER']['SCRIPT_NAME'])) {
//...
}

[sebastianbergmann]

Would that explain the fact that the PHARs work fine on phar.phpunit.de but not on my localhost?

[whatthejeff]

I think it depends on how you execute it. For instance:

$ php phpunit-alpha.phar --version
$ php `pwd`/phpunit-alpha.phar --version
PHPUnit alpha-2014-08-01 by Sebastian Bergmann.

[v1nc3n4]

Hi,
Sorry for the late answer, I wasn't available the days before.
I ran a test with the beta version I grabbed from the website.

The test suite with process isolation now executes in about 12 minutes.
The result is clearly better than the ones with PHPunit 4.1.1 and PHPunit 4.1.3 : 28.5 minutes.
However, the result is still significantly bad than the one with PHPunit 3.7.30 : 4.5 minutes.

[sebastianbergmann]

Dear contributor,

let me start by apologizing for not commenting and/or working on the issue you have reported or merging the pull request you have sent sooner.

PHPUnit 5.0 was released today. And today I am closing all open bug reports and pull requests for PHPUnit and its dependencies that I maintain. Please do not interpret the closing of this ticket as an insult or a lack of interest in your problem. I am sorry for any inconvenience this may cause.

If the topic of this ticket is still relevant then please open a new ticket or send a new pull request. If your ticket or pull request is about a defect then please check whether the issue still exists in PHPUnit 4.8 (which will received bug fixes until August 2016). If your ticket or pull request is about a new feature then please port your patch PHPUnit 5.0 before sending a new pull request.

I hope that today's extreme backlog grooming will allow me to respond to bug reports and pull requests in a more timely manner in the future.

Thank you for your understanding,
Sebastian