magento/module-catalog/Setup/Patch/Data/UpdateMultiselectAttributesBackendTypes.php breaks upgrade in migrated sites from 1.x

[ioweb-gr]

Preconditions and environment

• 2.4.6-p2

Steps to reproduce

To replicate this you need

1. Magento 2 (e.g. 2.4.1) that was migrated from 1.x and the 1.x instance had multiselect attributes with data (stored in _text table)
2. Use data migration tool to transfer the data which will copy the values from _text table to _varchar table during migration
3. Upgrade to 2.4.6-p2

The system will try to copy data from the _varchar table back to the _text table which was already migrated with the data migration tool to _text and to _varchar table. In the process you'll get this error about duplicate values

Expected result

Upgrade occurs properly

Actual result

Duplicated values throw exceptions

Additional information

There is only one real solution here, the patch needs to first clear the invalid data from _text table before migrating from _varchar to _text

e.g.

delete from catalog_product_entity_text where catalog_product_entity_text.attribute_id in (select eav_attribute.attribute_id from eav_attribute where eav_attribute.backend_type = 'varchar' and eav_attribute.entity_type_id = 4)

Release note

No response

Triage and priority

• Severity: S0 - Affects critical data or functionality and leaves users without workaround.
• Severity: S1 - Affects critical data or functionality and forces users to employ a workaround.
• Severity: S2 - Affects non-critical data or functionality and forces users to employ a workaround.
• Severity: S3 - Affects non-critical data or functionality and does not force users to employ a workaround.
• Severity: S4 - Affects aesthetics, professional look and feel, “quality” or “usability”.

[m2-assistant]

Hi @ioweb-gr. Thank you for your report.
To speed up processing of this issue, make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, Add a comment to the issue:

• @magento give me 2.4-develop instance - upcoming 2.4.x release
• For more details, review the Magento Contributor Assistant documentation.
• Add a comment to assign the issue: @magento I am working on this
• To learn more about issue processing workflow, refer to the Code Contributions.

---

Join Magento Community Engineering Slack and ask your questions in #github channel.
⚠️ According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.
🕙 You can find the schedule on the Magento Community Calendar page.
📞 The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, join the Community Contributions Triage session to discuss the appropriate ticket.

[hostep]

Use data migration tool to transfer the data which will copy the values from _text table to _varchar table during migration

Hmm, isn't the bug then in the data migration tool itself? Instead of copying the values, it should move them, so no duplicated values remain in the database?
In my opinion it's not a bug in the patch, but in the migration tool itself.

Or am I misunderstanding the issue?

[ioweb-gr]

True but it's also a fact that people who upgraded M2 from M1 won't be able to upgrade to newer versions of Magento due to this patch so even if the data migration tool needs to be fixed, this patch should also guarantee a seemless upgrade experience as well

[hostep]

Hmm maybe...

I recently ran into the Magento 2 EAV Cleaner Console Command module (which we needed for the eav:attributes:restore-use-default-value command after a recent mistake in the mailchimp module).

But I have the suspicion that its eav:clean:attributes-and-values-without-parent command might be useful in your case to clean out the database...

If you're interested in trying it out, make sure to not execute it on production immediately and have backups ready in case it does stuff you don't want it to do.

[m2-assistant]

Hi @engcom-Bravo. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue.
• 3. Add Area: XXXXX label to the ticket, indicating the functional areas it may be related to.
• 4. Verify that the issue is reproducible on 2.4-develop branch
  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!
• 5. Add label Issue: Confirmed once verification is complete.
• 6. Make sure that automatic system confirms that report has been added to the backlog.

[engcom-Bravo]

Hello @ioweb-gr ,

Thanks for the report and collaboration!

For data-migration-related issues we are using the below board:

https://github.com/magento/data-migration-tool/issues

For your convenience, we have created this issue there. You can refer to the below link for the same:

magento/data-migration-tool#919

We are closing this issue from here. Please follow the above URL for any updates on this.

Thanks

[ioweb-gr]

@engcom-Bravo This is not an issue for the data migration tool though. No code you change there will help users who already migrated solve this. The code that needs to be altered is the patch I mentioned

[ioweb-gr]

Unless @engcom-Bravo your official stance is that people who upgraded in the past, should never go to latest magento versions and they should redo the migration even if they are active in Magento 2 for a few years, I would suggest you reopen it.