-
Notifications
You must be signed in to change notification settings - Fork 3
Home
We need to send lots of data to Magento via the web API, particularly for products and stock which have thousands of updates each day.
Some requests are also very slow, like updating an attribute that has hundreds or thousands of options, and this creates timeouts.
Magento produces deadlocks on various tables at any more than 1 concurrent request for product updates. The higher number of requests, the more deadlocks are created. This creates a lot of failed requests which must be retried continuously before the update is saved by Magento.
Most commonly the deadlock is a gap lock against one of the EAV tables for a product which is required for the database to be replication-safe.
Optimising the existing interfaces is ideal as this doesn’t require any change in logic for existing API clients.
As a simple example: when an external system is trying to import products into Magento via the REST API and sends all "POST /products" requests one after another without any queue, then some of those calls are failed to execute. The Problem appears on the Magento side when Magento tries to write several entries in one database table at the same time. The first request locks the table and the next one tries to write data to the same table, but the table is already locked. This causes results like "Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction" or similar errors but all because of the same reason.
Most of the time, the source system which holds the product data, cannot export only those products which have changed since the last import, so with every product import, Magento gets all the products, and not only those which as changed, because eg. a price was updated.