!!! TASK: Update Doctrine Migrations to 3.0

[kdambekalns]

This updated the required version of doctrine/migrations from 1.8 to 3.0.

While there are new features in Doctrine Migrations, the reason for us to do
an upgrade is to move forward – the previously used version will not be
maintained forever… This post also gives some background on that:
https://www.doctrine-project.org/2020/04/10/doctrine-migrations-3.0.html

For a Flow user the commands remain unchanged, so far no multi-namespace
migrations are supported and the features to the "official" CLI do not matter,
since we embed the functionality in our own commands.

Breaking changes

There are three things that make this upgrade a breaking change:

• Doctrine\DBAL\Migrations moved to Doctrine\Migrations
• AbstractMigration changed method signatures (type delcarations added)

To adjust your PHP code (the migration files), a core migration is provided that
should fix the vast majority of existing migrations. (That core migration is in Flow
and named Version20201109224100.)

• The "version" is the FQCN of the migration class (existing entries in the migrations table will be automatically updated)

The needed changes to the DB table where the migration status is stored are done
the first time a command that accesses that table is used. Make sure to have a current
backup and then run ./flow doctrine:migrationstatus --show-migrations. If all
went well, the migrations should all be listed as a fully-qualified class name, no
longer just a date/time string. If any errors occurred during the command, restore the
backup (the migrations table is sufficient), fix the errors and try again.

See https://github.com/doctrine/migrations/blob/3.0.x/UPGRADE.md#code-bc-breaks
and https://github.com/doctrine/migrations/blob/3.0.x/UPGRADE.md#upgrade-to-20
for a full list of other changes. Most of those are wrapped in Flow code and need no
adjustments in userland code.

Resolves #2122

[sorenmalling]

@kdambekalns Would you like me to go over this for doctrine/orm 2.2.* ?
Did you already do any research and know of any blockers?

[kdambekalns]

Oh, yes, I did do things… here are three branches that complement each other:

• https://github.com/kdambekalns/neos-development-distribution/tree/task/doctrine-migrations-update
• https://github.com/kdambekalns/flow-development-collection/tree/task/update-doctrine-migrations
• https://github.com/kdambekalns/neos-development-collection/tree/task/update-doctrine-migrations

[albe]

D'oh, now we have Doctrine Migrations 3.x (https://www.doctrine-project.org/2020/04/10/doctrine-migrations-3.0.html) - see #2026. We might also need to update doctrine/common with this #2021, though it's still supposed to be phased out. So maybe we can even just get rid of it?

[kdambekalns]

I'll look into this (again), now that we have 7.0 coming it seems worth it! 💪

[albe]

Can we still get this in for 7.0?

[kdambekalns]

Good question… sorry, I didn't get around to it. @jonnitto Is the RM to ask, I guess. What are your thoughts, worth working on it today?

[kdambekalns]

These command are available and need to be working (again):

• doctrine:validate Validate the class/table mappings
• doctrine:create Create the database schema
• doctrine:update Update the database schema
• doctrine:entitystatus Show the current status of entities
• doctrine:dql Run arbitrary DQL and display
• doctrine:migrationstatus Show the current migration status
• doctrine:migrate Migrate the database schema
• doctrine:migrationexecute Execute a single migration
• doctrine:migrationversion Mark/unmark migrations as migrated
• doctrine:migrationgenerate Generate a new migration

[kdambekalns]

Woah, I'm at it… hold your horses. 😎

[albe]

Just fixing the obtrusive github thingies :D sorry! I'll hold back now

[kdambekalns]

One caveat I know of: when a property has been removed from a model, cache:flush needs to be run to avoid an error:

$ ./flow doctrine:migrationgenerate
Property Acme\Com\Domain\Model\SomeThing::$someProperty does not exist

  Type: ReflectionException
  File: Packages/Libraries/doctrine/persistence/lib/Doctrine/Persistence/Mapping/Ru
        ntimeReflectionService.php
  Line: 75

[albe]

Regarding the psalm failures left now - guess we need to update the docblocks in our Doctrine\Query. We'll need to baseline the QueryInterface->Doctrine\Query discrepancy though.
Most other look okay, but there's also

Docblock-defined class or interface Neos\Flow\Persistence\Doctrine\Migrations\MigrationException does not exist
Docblock-defined class or interface Doctrine\DBAL\Migrations\AbortMigrationException does not exist

[kdambekalns]

Let's see what is left now…

We'll need to baseline the QueryInterface->Doctrine\Query discrepancy though.

Or we adjust that along with #2231 and relax it a bit… 🤷‍♂️

[albe]

Yeah, we could do this. I mean at this point it's just a cosmetic change, which can happen any time.

[bwaidelich]

Can someone give me an update what needs to be done here?
I'd say we should try to get it in by tomorrow or postpone it to some later release

[albe]

AFAIS only a psalm-update-baseline (and then actually fixing things with #2231)

[kdambekalns]

The tests should pass with a psalm baseline update (I'll do that now).

Then we have two items from #1880 (comment) and finally #1880 (comment) - these are issues that could be addressed in a follow-up PR, if you agree…

Oh, and of course we need to adjust the DB migrations in Neos and our other packages…

[kdambekalns]

Well… that seems to be a mismatch betweeen Psalm versions. @albe Can you check this and maybe do the baseline update for now?

[albe]

<files psalm-version="3.18.2@19aa905f7c3c7350569999a93c40ae91ae4e1626"> yep - you need to install psalm ~3.10.0, or maybe we should finally add that as requirement to the dev-distribution

[kdambekalns]

Thanks for updating the baseline, @albe!

yep - you need to install psalm ~3.10.0

Hm… what about something new? #2205 suggests that. Let's track this as #2243

[kdambekalns]

I created a new issue for the known open tasks - so this could go in fast(er) for easier testing. 😬

[albe]

Hm… what about something new?

Yeah, I have that on my radar, but I think we should then try to keep versions across our branches as close as possible (optimally in sync), because upmerging the baseline might otherwise become a hassle.