Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EntityManager::find() calls UnitOfWork::registerManaged() twice and that causes Change Detection to fail #10880

Closed
eXsio opened this issue Aug 4, 2023 · 10 comments · Fixed by #10884

Comments

@eXsio
Copy link

eXsio commented Aug 4, 2023

BC Break Report

Q A
BC Break yes
Version 2.16.0

Summary

I have a Symfony 5.4 based Application that uses Framework Extra Bundle and its Param Converter feature. After upgrading from 2.15.3 to 2.16.0 Doctrine has stopped detecting Entity Changes in Entities loaded by the ParamConverter, due to the new logic introduced by this commit: 01a1432

After some snooping I've found out, that the UnitOfWork::registerManaged() is actually called twice:

The second call rewrites previously loaded/hydrated Entity and sets the UnitOfWork::$originalEntityData property to []. This causes the change detection to fail, because the UoW is not able to correctly compare the Actual Data to the Original Data.

The Framework Extra Bundle uses the EntityManager::find() method under the hood, so I wouldn't blame the lib for the failure. The fact is that Doctrine has altered it's behavior significantly because of that change.

Previous behavior

when an Entity wired by the ParamConverter was changed, Doctrine has correctly detected changes made in the Entity.

Current behavior

Doctrine doesn't detect changes in the Entities wired by the ParamConverter.

How to reproduce

  1. Create a Symfony 5.4 Application with Framework Extra Bundle
  2. Autowire an Entity to a Controller
  3. change the Entitie's state and save
  4. change detection fails

I am not sure if this is the case for every such configuration, but I am positive that the UnitOfWork::registerManaged method can be called twice for a single createEntity method call.

Suggested Fix

Any tampering with not calling the UnitOfWork::registerManaged() a second time has caused the ORM to blow up in some other place. However, modifying the UnitOfWork::registerManaged() method itself to not allow to overwrite the Entity seems to have helped:

   public function registerManaged($entity, array $id, array $data)
    {
        $oid = spl_object_id($entity);
        //don't allow to overwrite Entity once it has been registered
        if(array_key_exists($oid, $this->entityIdentifiers)) {
            return;
        }
        ....

P.S.

This is not the only regression caused by this change. I can also see multiple tickets that point to this change as a culprit. Don't mean to tell you how to do things, but this seems like a very impactful and drastic change for a minor release.

@pich
Copy link

pich commented Aug 4, 2023

+1

@mpdude
Copy link
Contributor

mpdude commented Aug 4, 2023

Can you provide the two code paths that ultimately lead to UnitOfWork::registerManaged() being called twice?

Do both calls come from the same createEntity invocation?

Also, are the two links correct that refer to the UoW?

@mpdude
Copy link
Contributor

mpdude commented Aug 4, 2023

I have tried your reproduce steps but cannot see a problem.

Please provide stack traces for both invocations of registerManaged(), and also share the version of FrameworkBundle for completeness.

@mpdude
Copy link
Contributor

mpdude commented Aug 4, 2023

The second invocation you mentioned can only come into play when there is something with associations going on. Does the entity loaded through the ParamConverter reference back to itself?

@eXsio
Copy link
Author

eXsio commented Aug 4, 2023

  1. Yes, both calls were originated by the same createEntity() method invocation
  2. Framework Extra Bundle version is v6.2.10
  3. There are 3 Entities at play:
  • Process
  • ProcessStage
  • ProcessOwner
  1. The relations between the Entities are:
  • The ProcessOwner has a FK to the Process (ManyToOne)
  • The Process has a FK to the ProcessOwner (OneToOne)
  • The ProcessStage has a FK to the Process (ManyToOne) - single Process can have multiple Stages. The relation is bidirectional.
  • The Process has a FK to ProcessStage (OneToOne) - the FK to a Current Process Stage ($currentStage field)

The Entity that is found by the ParamConverter is the ProcessOwner. The logic is about advancing the Process to the next Stage. The Controller logic, after applying some business validations, sets the new $currentStage to reflect that a Process has been moved forward.

The re-assignment of the Process::$currentStage, alhough completed in PHP, is not detected by the Change Detector, because
upon loading the Process via the createEntity() method the registerManaged() is called twice - once correctly, and once with an empty array for $data

I realize that the Entity hierarchy is not the greatest, but this is a legacy code riddled with inheritance as decisions that seemed to be logical at the time.

Having that said this was working flawlessly (the Project is 7 years old) up until 2.16.0.

@mpdude
Copy link
Contributor

mpdude commented Aug 5, 2023

When is the Process being loaded and createEntity being called - does that happen from within the ParamConverter?

Did I get it right that the ParamConverter is fetching a ProcessOwner? Does that have the Process as a proxy (lazy load)?

Are there any extra configuration settings like fetch=EAGER or so?

@mpdude
Copy link
Contributor

mpdude commented Aug 5, 2023

@eXsio I've been trying to add a reproduce test in #10884, but still unsuccessful – the test passes.

Feel free to open a PR against https://github.com/mpdude/doctrine2/tree/reproducer-10880 to show where it fails. For now, I'd assume the ParamConverter and Symfony or the FrameworkBundle don't have anything to do with it.

We need a test that passes on 2.15.x and fails on 2.16.0 to proceed.

@eXsio
Copy link
Author

eXsio commented Aug 5, 2023

OK, I will try to recreate this during the following week and submit a PR, thanks for trying :)

@eXsio
Copy link
Author

eXsio commented Aug 5, 2023

Hello

I've created a PR with a working reproduction:
mpdude#3

My Branch forked from your repo proves that it doesn't work:
https://github.com/eXsio/doctrine2/tree/gh-10880-reproduction-2-16-0
https://github.com/eXsio/doctrine2/commit/7ea8474736e8325a142b0c0c7b80eb8d74aab673

I've created a second branch to make sure that the exact same thing worked on 2.15.3 and, in fact, I was right:
https://github.com/eXsio/doctrine2/tree/gh-10880-reproduction-2-15-3
https://github.com/eXsio/doctrine2/commit/1f9aa8e847a3fb463ae5805d9c082fd0e4b38716

The whole thing seems to be about inheritance. If change the GH10880BaseProcess definition to

/**
 * @ORM\MappedSuperClass
 * @ORM\Table(name="processes")
 * @ORM\DiscriminatorMap({
 *  "process" = "GH10880Process"
 * })
 */
class GH10880BaseProcess

It magically starts to work on 2.16.0. The thing is, that in my Application I have multiple inheritors of the GH10880BaseProcess so I need to have the Single Table Inheritance working, and it was working on 2.15.3.

@eXsio
Copy link
Author

eXsio commented Aug 5, 2023

P.S.

One Clarification: the registerManaged() calls do not originate from the same createEntity() call after all.

After adding some var_dump calls to the UnitOfWork I can see the following call sequence:

string(71) "CREATE ENTITY: Doctrine\Tests\ORM\Functional\Ticket\GH10880ProcessOwner"
string(66) "CREATE ENTITY: Doctrine\Tests\ORM\Functional\Ticket\GH10880Process"
string(70) "REGISTER MANAGED - Doctrine\Tests\ORM\Functional\Ticket\GH10880Process"
string(22) "$data parameter value:"
array(4) {
  ["id"]=>
  int(1)
  ["parentObject"]=>
  string(1) "1"
  ["current_stage"]=>
  int(2)
  ["parent_object_id"]=>
  int(1)
}
string(71) "CREATE ENTITY: Doctrine\Tests\ORM\Functional\Ticket\GH10880ProcessStage"
string(71) "CREATE ENTITY: Doctrine\Tests\ORM\Functional\Ticket\GH10880ProcessStage"
string(71) "CREATE ENTITY: Doctrine\Tests\ORM\Functional\Ticket\GH10880ProcessStage"
string(70) "REGISTER MANAGED - Doctrine\Tests\ORM\Functional\Ticket\GH10880Process"
string(22) "$data parameter value:"
array(0) {
}

Hope this helps.

mpdude added a commit to mpdude/doctrine2 that referenced this issue Aug 5, 2023
GH10880 reproduction on Doctrine 2.16.0 - proof that it doesn't work anymore
mpdude added a commit to mpdude/doctrine2 that referenced this issue Aug 6, 2023
mpdude added a commit to mpdude/doctrine2 that referenced this issue Aug 6, 2023
mpdude added a commit to mpdude/doctrine2 that referenced this issue Aug 6, 2023
mpdude added a commit to mpdude/doctrine2 that referenced this issue Aug 6, 2023
mpdude added a commit to mpdude/doctrine2 that referenced this issue Aug 6, 2023
…ies loaded through fetch=EAGER + using inheritance

 doctrine#10880 reports a case where the changes from doctrine#10785 cause entity updates to be missed.

Upon closer inspection, this change seems to be causing it:

https://github.com/doctrine/orm/pull/10785/files#diff-55a900494fc8033ab498c53929716caf0aa39d6bdd7058e7d256787a24412ee4L2990-L3003

The code was changed to use `registerManaged()` instead, which basically does the same things, but (since doctrine#10785) also includes an additional check against duplicate entity instances.

But, one detail slipped through tests and reviews: `registerManaged()` also updates `\Doctrine\ORM\UnitOfWork::$originalEntityData`, which is used to compute entity changesets. An empty array `[]` was passed for $data here.

This will make the changeset computation assume that a partial object was loaded and effectively ignore all field updates here:

https://github.com/doctrine/orm/blob/a616914887ea160db4158d2c67752e99624f7c8a/lib/Doctrine/ORM/UnitOfWork.php#L762-L764

I think that, effectively, it is sufficient to call `registerManaged()` only in the two cases where a proxy was created.

Calling `registerManaged()` with `[]` as data for a proxy object is consistent with e. g. `\Doctrine\ORM\EntityManager::getReference()`.

In the case that a full entity has to be loaded, we need not call `registerManaged()` at all, since that will already happen inside `EntityManager::find()` (or, more specifically, `UnitOfWork::createEntity()` called inside it).

Note that the test case has to make some provisions so that we actually reach this case:
* Load an entity that uses `fetch="EAGER"` on a to-one association
* That association being against a class that uses inheritance (why's that?)
mpdude added a commit to mpdude/doctrine2 that referenced this issue Aug 6, 2023
…ies loaded through fetch=EAGER + using inheritance

 doctrine#10880 reports a case where the changes from doctrine#10785 cause entity updates to be missed.

Upon closer inspection, this change seems to be causing it:

https://github.com/doctrine/orm/pull/10785/files#diff-55a900494fc8033ab498c53929716caf0aa39d6bdd7058e7d256787a24412ee4L2990-L3003

The code was changed to use `registerManaged()` instead, which basically does the same things, but (since doctrine#10785) also includes an additional check against duplicate entity instances.

But, one detail slipped through tests and reviews: `registerManaged()` also updates `\Doctrine\ORM\UnitOfWork::$originalEntityData`, which is used to compute entity changesets. An empty array `[]` was passed for $data here.

This will make the changeset computation assume that a partial object was loaded and effectively ignore all field updates here:

https://github.com/doctrine/orm/blob/a616914887ea160db4158d2c67752e99624f7c8a/lib/Doctrine/ORM/UnitOfWork.php#L762-L764

I think that, effectively, it is sufficient to call `registerManaged()` only in the two cases where a proxy was created.

Calling `registerManaged()` with `[]` as data for a proxy object is consistent with e. g. `\Doctrine\ORM\EntityManager::getReference()`.

In the case that a full entity has to be loaded, we need not call `registerManaged()` at all, since that will already happen inside `EntityManager::find()` (or, more specifically, `UnitOfWork::createEntity()` called inside it).

Note that the test case has to make some provisions so that we actually reach this case:
* Load an entity that uses `fetch="EAGER"` on a to-one association
* That association being against a class that uses inheritance (why's that?)
derrabus pushed a commit to mpdude/doctrine2 that referenced this issue Aug 9, 2023
…ies loaded through fetch=EAGER + using inheritance

 doctrine#10880 reports a case where the changes from doctrine#10785 cause entity updates to be missed.

Upon closer inspection, this change seems to be causing it:

https://github.com/doctrine/orm/pull/10785/files#diff-55a900494fc8033ab498c53929716caf0aa39d6bdd7058e7d256787a24412ee4L2990-L3003

The code was changed to use `registerManaged()` instead, which basically does the same things, but (since doctrine#10785) also includes an additional check against duplicate entity instances.

But, one detail slipped through tests and reviews: `registerManaged()` also updates `\Doctrine\ORM\UnitOfWork::$originalEntityData`, which is used to compute entity changesets. An empty array `[]` was passed for $data here.

This will make the changeset computation assume that a partial object was loaded and effectively ignore all field updates here:

https://github.com/doctrine/orm/blob/a616914887ea160db4158d2c67752e99624f7c8a/lib/Doctrine/ORM/UnitOfWork.php#L762-L764

I think that, effectively, it is sufficient to call `registerManaged()` only in the two cases where a proxy was created.

Calling `registerManaged()` with `[]` as data for a proxy object is consistent with e. g. `\Doctrine\ORM\EntityManager::getReference()`.

In the case that a full entity has to be loaded, we need not call `registerManaged()` at all, since that will already happen inside `EntityManager::find()` (or, more specifically, `UnitOfWork::createEntity()` called inside it).

Note that the test case has to make some provisions so that we actually reach this case:
* Load an entity that uses `fetch="EAGER"` on a to-one association
* That association being against a class that uses inheritance (why's that?)
derrabus pushed a commit that referenced this issue Aug 9, 2023
…GER + using inheritance (#10884)

#10880 reports a case where the changes from #10785 cause entity updates to be missed.

Upon closer inspection, this change seems to be causing it:

https://github.com/doctrine/orm/pull/10785/files#diff-55a900494fc8033ab498c53929716caf0aa39d6bdd7058e7d256787a24412ee4L2990-L3003

The code was changed to use `registerManaged()` instead, which basically does the same things, but (since #10785) also includes an additional check against duplicate entity instances.

But, one detail slipped through tests and reviews: `registerManaged()` also updates `\Doctrine\ORM\UnitOfWork::$originalEntityData`, which is used to compute entity changesets. An empty array `[]` was passed for $data here.

This will make the changeset computation assume that a partial object was loaded and effectively ignore all field updates here:

https://github.com/doctrine/orm/blob/a616914887ea160db4158d2c67752e99624f7c8a/lib/Doctrine/ORM/UnitOfWork.php#L762-L764

I think that, effectively, it is sufficient to call `registerManaged()` only in the two cases where a proxy was created.

Calling `registerManaged()` with `[]` as data for a proxy object is consistent with e. g. `\Doctrine\ORM\EntityManager::getReference()`.

In the case that a full entity has to be loaded, we need not call `registerManaged()` at all, since that will already happen inside `EntityManager::find()` (or, more specifically, `UnitOfWork::createEntity()` called inside it).

Note that the test case has to make some provisions so that we actually reach this case:
* Load an entity that uses `fetch="EAGER"` on a to-one association
* That association being against a class that uses inheritance (why's that?)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants