Tests not executed and marked as correct

[dertin]

Q	A
PHPUnit version	7.3.5
PHP version	7.2.1-dev
Installation Method	Composer

All tests marked as correct with a time of 0ms. I checked and no test was executed.

The results are a false positive affected by session_set_save_handler().

Could this be possible ?

phpunit.xml

<phpunit bootstrap="bootstrap.php"
colors="true"
processIsolation="true"
verbose="false"
printerClass="Sempro\PHPUnitPrettyPrinter\PrettyPrinter">
  <testsuites>
    <testsuite name="Addons">
      <directory>./tests/addons</directory>
    </testsuite>
    <testsuite name="Controllers">
      <directory>./tests/controllers</directory>
    </testsuite>
  </testsuites>
</phpunit>

bootstrap.php

<?php

// Bootstrap for PHPUnit

// Constants configuration
require_once '../../../config.php';
configFramework();

// Autoload
require_once PARTICLE_PATH_CORE.'autoload'.DS.'autoload.php';

// init Session
$session = Particle\Core\Session::singleton();

Session.php

<?php
namespace Particle\Core;
use Particle\Core;

final class Session extends \SessionHandler
{
    protected $name;
    protected $cookie;
    private static $instancia;

    // método singleton
    public static function singleton()
    {
        if (!isset(self::$instancia)) {
            $miclase = __CLASS__;
            self::$instancia = new $miclase('sid');
            session_set_save_handler(self::$instancia, true); // CAUSES STRANGE BEHAVIOR
            self::$instancia->start();
        }
        return self::$instancia;
    }
...

[sebastianbergmann]

Thank you for your report.

Please provide a minimal, self-contained, reproducing test case that shows the problem you are reporting.

Without such a minimal, self-contained, reproducing test case I will not be able to investigate this issue.

Please disable custom test listeners such as Sempro\PHPUnitPrettyPrinter\PrettyPrinter.

[dertin]

I have updated PHP and the problem was not solved.

Simply run any test with the next bootstrap.php:

<?php
class MySessionHandler implements SessionHandlerInterface
{
    function open($savePath, $sessionName)
    {
          return true;
    }

    function close()
    {
        return true;
    }

    function read($id)
    {
        return '';
    }

    function write($id, $data)
    {
        return true;
    }

    function destroy($id)
    {
        return true;
    }

    function gc($maxlifetime)
    {
        return true;
    }
}
$handler = new MySessionHandler();
session_set_save_handler($handler, true); // CAUSES STRANGE BEHAVIOR
session_start();

myTest.php

<?php

namespace MyTest;

use PHPUnit\Framework\TestCase;
use PHPUnit\Framework\Constraint;

final class myTestTest extends TestCase
{
    public function testSession()
    {
        sleep(100); // 1.66 minutes
        $this->assertTrue(true);
    }
}

phpunit.xml

<phpunit bootstrap="bootstrap.php"
colors="true"
processIsolation="true"
verbose="false">
  <testsuites>
    <testsuite name="Addons">
      <directory>./tests/error</directory>
    </testsuite>
  </testsuites>
</phpunit>

The test is marked as correct, but never executed:
Time: 41 ms, Memory: 4.00MB
OK (1 test, 0 assertions)

The expected result is the following:
Time: 1.66 minutes, Memory: 4.00MB
OK (1 test, 1 assertion)

[dertin]

If I set
processIsolation="false"
last problem does not happen.

[dertin]

Now I have the following problem #1416, since my tests use cookie and session
And if the sessions work for me, the cookies stop working for me.
If I set processIsolation="false"

[dertin]

Solved:
processIsolation="false" and --stderr CLI option.

[yya]

Since php 7.2 it is not possible to set 'user' save handler by ini_set().
s. https://github.com/php/php-src/blob/php-7.2.14/ext/session/session.c#L559

When running tests with processIsolation=true and preserveGlobalState=true then PHPUnit tries to backup all the php ini-settings with ini_get_all (s. PHPUnit\Util\Globalstate.php):

public static function getIniSettingsAsString(): string
    {
        $result      = '';
        $iniSettings = \ini_get_all(null, false);

        foreach ($iniSettings as $key => $value) {
            $result .= \sprintf(
                '@ini_set(%s, %s);' . "\n",
                self::exportVariable($key),
                self::exportVariable($value)
            );
        }

        return $result;
    }

In the isolated process PHPUnit sets all php-ini-settings again within a long list of @ini_set-calls (.s Util/PHP/Template/TestCaseMethod.tpl.dist)

This long list contains also the following call @ini_set('session.save_handler', 'user'); which prevents the isolated process to run.

I think PHPUnit have to exclude all non-editable ini-settings

[sebastianbergmann]

@yya Please open a new ticket for this.

[mattsoftware]

Was a new ticket every created for this? I have just spent the day debugging to find this is exactly why all of our @runInSeparateProcess tests are failing. Simply removing the session.save_handler ini setting is enough to get the tests running again.