Skip to content
This repository has been archived by the owner on May 29, 2019. It is now read-only.

Home

Jump to bottom
Eugene Tulika edited this page Oct 30, 2017 · 8 revisions

Introduction

We need to send lots of data to Magento via the web API, particularly for products and stock which have thousands of updates each day.

Some requests are also very slow, like updating an attribute that has hundreds or thousands of options, and this creates timeouts.

Magento produces deadlocks on various tables at any more than 1 concurrent request for product updates. The higher number of requests, the more deadlocks are created. This creates a lot of failed requests which must be retried continuously before the update is saved by Magento.

Problems

Deadlocks

Most commonly the deadlock is a gap lock against one of the EAV tables for a product which is required for the database to be replication-safe.

Optimising the existing interfaces is ideal as this doesn’t require any change in logic for existing API clients.

As a simple example: when an external system is trying to import products into Magento via the REST API and sends all "POST /products" requests one after another without any queue, then some of those calls are failed to execute. The Problem appears on the Magento side when Magento tries to write several entries in one database table at the same time. The first request locks the table and the next one tries to write data to the same table, but the table is already locked. This causes results like "Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction" or similar errors but all because of the same reason.

Incremental Import

Most of the time, the source system which holds the product data, cannot export only those products which have changed since the last import, so with every product import, Magento gets all the products, and not only those which as changed, because eg. a price was updated.

Use Cases

Use cases Product

  • mass create
  • update inventory
  • update price (include all price fields, including EAV and extension attributes)
  • update attribute value (any attribute type)
  • update media

Customer

  • mass create
  • update attribute value
  • personal info update
  • address info update

Order

  • mass create
  • update attribute value
  • order status update

Shipment

  • mass create
  • shipping/tracking info update
Clone this wiki locally