REST products update category_ids cannot be removed

[siliconalchemy]

** Note: This is a copy of issue #14038, which was assigned to someone who has no idea how to reproduce or fix it and seems to be permanently stuck **

Preconditions and Steps to Reproduce

For example, if a product currently has custom_attribute { 'category_ids': '['10127', '10048', '10123', '10126'] }, and I want to remove two of the categories from the product by setting this to { 'category_ids': '['10127', '10048'] }.

I then update the product with /rest/all/V1/products/ with the updated data. I get a success returned, and any other product data is successfully updated, and categories can be added to, but trying to remove categories such as above fail silently. The category_ids remain with the original 4 categories, as in the example above.

Magento 2.2.3
Php 7.1, Mysql 5.7 (Percona Xtradb Cluster)
Expected Result

I expect to be able to remove categories. This worked until 2.2 upgrade.
Actual result

See description. Categories are not updated, but the update command reports success.

[magento-engcom-team]

Hi @siliconalchemy. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento-engcom-team give me $VERSION instance

where $VERSION is version tags (starting from 2.2.0+) or develop branches (for example: 2.3-develop).
For more details, please, review the Magento Contributor Assistant documentation.

@siliconalchemy do you confirm that you was able to reproduce the issue on vanilla Magento instance following steps to reproduce?

• yes
• no

[magento-engcom-team]

Hi @engcom-backlog-nazar. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 4. Verify that the issue is reproducible on 2.3-develop branch

  Details

  - Add the comment @magento-engcom-team give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 5. Verify that the issue is reproducible on 2.2-develop branch.

  Details

  - Add the comment @magento-engcom-team give me 2.2-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.2-develop branch, please add the label Reproduced on 2.2.x

• 6. Add label Issue: Confirmed once verification is complete.

• 7. Make sure that automatic system confirms that report has been added to the backlog.

[magento-engcom-team]

@engcom-backlog-nazar Thank you for verifying the issue. Based on the provided information internal tickets MAGETWO-97880, MAGETWO-97881 were created

[magento-engcom-team]

Hi @vpodorozh. Thank you for working on this issue.
Looks like this issue is already verified and confirmed. But if your want to validate it one more time, please, go though the following instruction:

• 1. Add/Edit Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 2. Verify that the issue is reproducible on 2.3-develop branch

  Details

  - Add the comment @magento-engcom-team give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 3. Verify that the issue is reproducible on 2.2-develop branch.

  Details

  - Add the comment @magento-engcom-team give me 2.2-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.2-develop branch, please add the label Reproduced on 2.2.x

• 4. If the issue is not relevant or is not reproducible any more, feel free to close it.

[magento-engcom-team]

Hi @ygyryn. Thank you for working on this issue.
Looks like this issue is already verified and confirmed. But if your want to validate it one more time, please, go though the following instruction:

• 1. Add/Edit Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 2. Verify that the issue is reproducible on 2.3-develop branch

  Details

  - Add the comment @magento-engcom-team give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 3. Verify that the issue is reproducible on 2.2-develop branch.

  Details

  - Add the comment @magento-engcom-team give me 2.2-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.2-develop branch, please add the label Reproduced on 2.2.x

• 4. If the issue is not relevant or is not reproducible any more, feel free to close it.

[dmytro-ch]

@magento-engcom-team give me 2.3-develop instance

[magento-engcom-team]

Hi @dmytro-ch. Thank you for your request. I'm working on Magento 2.3-develop instance for you

[magento-engcom-team]

Hi @dmytro-ch, here is your Magento instance.
Admin access: https://i-20481-2-3-develop.instances.magento-community.engineering/admin
Login: admin Password: 123123q
Instance will be terminated in up to 3 hours.

[magento-engcom-team]

Hi @siliconalchemy. Thank you for your report.
The issue has been fixed in #20842 by @ygyryn in 2.3-develop branch
Related commit(s):

• 9dc3a19d2cb37f4d0a4cddbab7334d0c579c45b9
• b6020677c4fd3e99d59c67b953bfa53d70bed165
• d1b7097bc38e87669607cfac094d312d76ce7ab3
• 814947905ba35e2bbaca4f4124d1e0ac19225ce5
• bd46d50da3ece6f19522532909fff3cc23a88a93
• 96b6feb5feacb130c7c0e181db79b5718fb8b98b

The fix will be available with the upcoming 2.3.2 release.

[siliconalchemy]

As of Magento 2.3.3, this problem is NOT fixed. Assigning a reduced list of category_ids does not correctly remove remaining category_ids that should no longer be there.

[sgriffiths-ly]

In which version is this fixed?
I'm experiencing this in Magento 2.3.3

[simonworkhouse]

@magento-engcom-team Still a problem in 2.3.4.

@siliconalchemy It looks like the issue resides in the mergeCategoryLinks($newCategoryPositions, $oldCategoryPositions) function in Magento\Catalog\Model\Category\Link\SaveHandler. This function will remove the categories iff you supply an empty array, otherwise it will always merge the old with the new.

Something odd to take note of is that this is also used when saving products in the admin panel, but it appears that the order of parameters are reversed where $newCategoryPositions contains the old categories list and $oldCategoryPositions contains the new ones.

I'll need to do a bit more digging, but it really looks like this function is just nonsense. Maybe it was an attempt to preserve the category order? If so, it could have been done in a sensible way.

[siliconalchemy]

@simonworkhouse - Magento don't care. See #25646. I just hack my fix into place for every point release, a long with a bunch of other bugs Magento don't care about fixing.

Edit: actually I made this into a module (unzip into app/code/SiliconAlchemy to use) in case it's useful for anyone:
FixDeleteCategory.zip
It lives in there along with an ever increasing other bunch of hacks I got fed up making every release.

[simonworkhouse]

@siliconalchemy Ah, I didn't see that one, but thank you. It looks like I found the same source of the issue as you did.

It astonishes me that Magento was sold for ~$1.7b, I honestly didn't know piles of manure could be so expensive.

[cfarnleitner]

some ruff climate here...

Anyways, a shout out to leonardo_pinhei:

https://community.magento.com/t5/Magento-2-x-Programming/Magento-2-REST-API-Change-product-category/td-p/96956

the trick seems to be setting (the default custom attribute) category_ids to null/empty in addition to the category_links in extension_attributes, like:

"custom_attribute": [
    {
        "attribute_code": "category_ids",
        "value": null
    }
]

I actually like the fact that you can control whether or not to keep existing categories using this attribute now that I know about it... ;)

[siliconalchemy]

@cfarnleitner - You think that an obscure bug (or at least obscure undocumented handling) in an API is a good way to fix a major bug in an API?
The reason it's a 'ruff climate' in here is that it's business as usual for Magento - the people who man github seem to get paid on the numbers of issues they close, not on the quality of the product. It took 3 github issues and 3 years to fix a simple but significant API bug.

[gdvisser]

As long as this is not fixed in Magento you can take this approach: https://stackoverflow.com/questions/68622618/magento-2-rest-api-cant-remove-category-ids/68623022#answer-68623022

[mackieee]

I've encountered this on Magento 2.4.2 today. Shocked to find I'll be needing to unset this everytime if need be!