Performance issue

[alexislefebvre]

Preconditions

Performance issue with version 1.0 of this bundle: liip/LiipFunctionalTestBundle#516 (comment) compared to LiipFunctionalTestBundle 2.0.0-alpha16

Steps to reproduce

.

Expected result

.

Actual result

Tests are about 2x slower.

[alexislefebvre]

@Simperfit found that createSchema is taking much more time.

[alexislefebvre]

TODO : use Blackfire to compare performance with https://github.com/liip/LiipFunctionalTestBundle/releases/tag/2.0.0-alpha16 and LiipFunctionalTestBundle 3.0 and LiipTestFixturesBundle 1.0.

[alexislefebvre]

On a personal project, tests are 4 times slower. 🤔

[alexislefebvre]

TODO: compare when using dependency injection as in #18.

[magnetik]

Could this be because kernel & containers are not shared between the two bundle so when using both loadFixtures() and makeClient() two kernels are being built?

[alexislefebvre]

@magnetik to my knowledge, event if some code is shared between the 2 bundles, there will be only one instance of the test class so there will be only one kernel built?

[magnetik]

When using both bundles, the two kernel built are from :

• \Liip\FunctionalTestBundle\Test\WebTestCase::makeClient through \Symfony\Bundle\FrameworkBundle\Test\WebTestCase::createClient
• \Liip\TestFixturesBundle\Test\FixturesTrait::loadFixtures (via \Liip\TestFixturesBundle\Test\FixturesTrait::getContainer)

Am I missing something?

[biozshock]

I found out why in our project we had a slowdown of about 3x.

According to the

LiipTestFixturesBundle/src/DependencyInjection/Configuration.php

Line 49 in 2e6e867

->booleanNode('keep_database_and_schema')->defaultFalse()->end()

setting you always recreate schema for the database

LiipTestFixturesBundle/src/Services/DatabaseTools/ORMDatabaseTool.php

Line 129 in f97ccdf

if (false === $this->getKeepDatabaseAndSchemaParameter()) {

.

In LiipFunctionalTestBundle v1.x you had that only when sqlite database was defined for tests https://github.com/liip/LiipFunctionalTestBundle/blob/1.x/Test/WebTestCase.php#L449 .

In our case with the tests with MySQL (mysql, percona and mariadb) we had tests passing faster than it was with LiipFunctionalTestBundle 1.x

[alexislefebvre]

Thanks for the investigation @biozshock

The first change to this code was in this PR: https://github.com/liip/LiipFunctionalTestBundle/pull/398/files#diff-3216cb8554ab63f3b7b8531ebcee5156L387

I'll continue to look at changes later.

[alexislefebvre]

We could add a note in the documentation that there are several options to create database and schema:

If the schema doesn't change between tests:

1. define keep_database_and_schema: true
2. create database with bin/console --env=test doctrine:database:create:
3. create schema with php app/console --env=test doctrine:schema:update --force or php app/console --env=test doctrine:migrations:migrate --no-interaction
4. run PHPUnit

If the schema change between tests:

1. don't define keep_database_and_schema: true
2. run PHPUnit and database and schema will be created automatically, please note that this method is slower

[alexislefebvre]

We could add a note in the documentation that there are several options to create database and schema:

Fixed in #120.

The performance issue should be fixed in the 2.x release.

[Dekrikh]

Performance with LiipTestFixturesBundle v2.6.0

I am experiencing significant performance issues with LiipTestFixturesBundle version 2.6.0 when using the loadFixtures method with zero insertions. Even with an empty set of fixtures as shown below, tests take 7 to 9 seconds to run, which seems abnormally high. And without loadFixtures() 260 ms

$this->databaseTool->loadFixtures([ ExempleFixtures::class]);

 <?php

namespace App\DataFixtures;

use Doctrine\Bundle\FixturesBundle\Fixture;
use Doctrine\Persistence\ObjectManager;

class ExempleFixtures extends Fixture
{
    public function load(ObjectManager $manager)
    {

    }
}