[CHG] review product aggregation by backend to improve performance #9
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
guewen
reviewed
Jul 26, 2017
| for backend in self.mapped('backend_id'): | ||
| self._recompute_magento_qty_backend( | ||
| backend, self.filtered( | ||
| lambda x: x.backend_id.id == backend.id)) |
There was a problem hiding this comment.
But then if you have more than 1 backend, each binding will be iterated N times + 1 where N is the number of backend (1 loop for filtered on each backend + 1 loop for mapped to obtain the backends). In the best case when you have 1 backend, you still loop 2 times on the records.
If the recordset's |= is a concern (I didn't know it was particularly slow), then I would use a simple set in a dictionary and rebrowse afterwards:
backends = defaultdict(set)
for product in self:
backends[product.backend_id].add(product.id)
for backend, product_ids in backends.iteritems():
self._recompute_magento_qty_backend(backend, self.browse(product_ids))There was a problem hiding this comment.
Yes but itering on a recordset is quicker than creating a new big one.
I have more than 26.000 products binded. The original code took more than 15 minutes to complete the dictionnary.
This version takes only 6 secondes ...
There was a problem hiding this comment.
Ok, good to know that updating a recordset is so slow...it's What about the version I propose?
There was a problem hiding this comment.
The version you propose is good, just a few second to proceed. :-)
Cedric-Pigeon
force-pushed
the
10.0-components
branch
from
800372e
to
8989e07
Compare
|
@guewen I updated my PR with your suggestion. |
|
Thanks! |
gurneyalex
reviewed
Jul 27, 2017
| @@ -108,12 +108,13 @@ def recompute_magento_qty(self): | |||
| informations for each product. | |||
| """ | |||
| # group products by backend | |||
| backends = defaultdict(self.browse) | |||
| backends = defaultdict(set) | |||
There was a problem hiding this comment.
@guewen If you want to squeeze some more perfs out of this, make this defaultdict(list) and:
- on line 113 use append instead of add
- on line 117 below use
self.browse(set(product_ids))
guewen
added a commit
to guewen/connector-prestashop
that referenced
this pull request
Jul 27, 2017
The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
guewen
added a commit
to guewen/connector-prestashop
that referenced
this pull request
Jul 27, 2017
The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
guewen
added a commit
to guewen/connector-prestashop
that referenced
this pull request
Aug 21, 2017
The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
sergio-teruel
pushed a commit
to Tecnativa/connector-prestashop
that referenced
this pull request
Sep 23, 2017
The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9 (cherry picked from commit 5d617fd)
sergio-teruel
pushed a commit
to Tecnativa/connector-prestashop
that referenced
this pull request
Sep 23, 2017
The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9 (cherry picked from commit 5d617fd)
pedrobaeza
pushed a commit
to Tecnativa/connector-prestashop
that referenced
this pull request
Aug 21, 2018
* [FIX] Fix export when no location has flag prestashop_synchronized When there is no location with the flag prestashop_synchronized in the location tree, Current behavior: The `location` key in the context is empty then the qty_available is computed for *every* locations/warehouses. Expected behavior: If there is at least one location flagged, use it, otherwise, use all the locations of the tree, starting from the selected location or selected warehouse location. When no location can be found (for instance because user selected a location with a usage different than 'internal'), then it should just fail and not return the stock of all warehouses. Being able to synchronize without activating 'prestashop_synchronized' is important: when used, this feature will trigger an export every time a quant is changed or created, on a large catalog, we might prefer only using the cron to have less frequent updates and less jobs. A second fix is that when a stock_location_id is set on the backend, it should be used, it was only used for the import of stock. * [FIX] Exclude children from qty computation We already pass the children in 'location', if we don't disable 'compute_child', the qty computation will try to get the children of each child and generates query of 1mio chars. * [IMP] Group computation of qty_available Stats with 6700 products: Before: 570s After: 28s * [FIX] Apply filter on locations for variants quantities It was only applied on export of template quantities. * [IMP] Improve performance on prestashop qty recompute The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
pedrobaeza
pushed a commit
to Tecnativa/connector-prestashop
that referenced
this pull request
Aug 21, 2018
* [FIX] Fix export when no location has flag prestashop_synchronized When there is no location with the flag prestashop_synchronized in the location tree, Current behavior: The `location` key in the context is empty then the qty_available is computed for *every* locations/warehouses. Expected behavior: If there is at least one location flagged, use it, otherwise, use all the locations of the tree, starting from the selected location or selected warehouse location. When no location can be found (for instance because user selected a location with a usage different than 'internal'), then it should just fail and not return the stock of all warehouses. Being able to synchronize without activating 'prestashop_synchronized' is important: when used, this feature will trigger an export every time a quant is changed or created, on a large catalog, we might prefer only using the cron to have less frequent updates and less jobs. A second fix is that when a stock_location_id is set on the backend, it should be used, it was only used for the import of stock. * [FIX] Exclude children from qty computation We already pass the children in 'location', if we don't disable 'compute_child', the qty computation will try to get the children of each child and generates query of 1mio chars. * [IMP] Group computation of qty_available Stats with 6700 products: Before: 570s After: 28s * [FIX] Apply filter on locations for variants quantities It was only applied on export of template quantities. * [IMP] Improve performance on prestashop qty recompute The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
pedrobaeza
pushed a commit
to Tecnativa/connector-prestashop
that referenced
this pull request
Mar 22, 2019
* [FIX] Fix export when no location has flag prestashop_synchronized When there is no location with the flag prestashop_synchronized in the location tree, Current behavior: The `location` key in the context is empty then the qty_available is computed for *every* locations/warehouses. Expected behavior: If there is at least one location flagged, use it, otherwise, use all the locations of the tree, starting from the selected location or selected warehouse location. When no location can be found (for instance because user selected a location with a usage different than 'internal'), then it should just fail and not return the stock of all warehouses. Being able to synchronize without activating 'prestashop_synchronized' is important: when used, this feature will trigger an export every time a quant is changed or created, on a large catalog, we might prefer only using the cron to have less frequent updates and less jobs. A second fix is that when a stock_location_id is set on the backend, it should be used, it was only used for the import of stock. * [FIX] Exclude children from qty computation We already pass the children in 'location', if we don't disable 'compute_child', the qty computation will try to get the children of each child and generates query of 1mio chars. * [IMP] Group computation of qty_available Stats with 6700 products: Before: 570s After: 28s * [FIX] Apply filter on locations for variants quantities It was only applied on export of template quantities. * [IMP] Improve performance on prestashop qty recompute The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
pedrobaeza
pushed a commit
to Tecnativa/connector-prestashop
that referenced
this pull request
Mar 22, 2019
* [FIX] Fix export when no location has flag prestashop_synchronized When there is no location with the flag prestashop_synchronized in the location tree, Current behavior: The `location` key in the context is empty then the qty_available is computed for *every* locations/warehouses. Expected behavior: If there is at least one location flagged, use it, otherwise, use all the locations of the tree, starting from the selected location or selected warehouse location. When no location can be found (for instance because user selected a location with a usage different than 'internal'), then it should just fail and not return the stock of all warehouses. Being able to synchronize without activating 'prestashop_synchronized' is important: when used, this feature will trigger an export every time a quant is changed or created, on a large catalog, we might prefer only using the cron to have less frequent updates and less jobs. A second fix is that when a stock_location_id is set on the backend, it should be used, it was only used for the import of stock. * [FIX] Exclude children from qty computation We already pass the children in 'location', if we don't disable 'compute_child', the qty computation will try to get the children of each child and generates query of 1mio chars. * [IMP] Group computation of qty_available Stats with 6700 products: Before: 570s After: 28s * [FIX] Apply filter on locations for variants quantities It was only applied on export of template quantities. * [IMP] Improve performance on prestashop qty recompute The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
florian-dacosta
pushed a commit
to florian-dacosta/connector-prestashop
that referenced
this pull request
Dec 23, 2020
* [FIX] Fix export when no location has flag prestashop_synchronized When there is no location with the flag prestashop_synchronized in the location tree, Current behavior: The `location` key in the context is empty then the qty_available is computed for *every* locations/warehouses. Expected behavior: If there is at least one location flagged, use it, otherwise, use all the locations of the tree, starting from the selected location or selected warehouse location. When no location can be found (for instance because user selected a location with a usage different than 'internal'), then it should just fail and not return the stock of all warehouses. Being able to synchronize without activating 'prestashop_synchronized' is important: when used, this feature will trigger an export every time a quant is changed or created, on a large catalog, we might prefer only using the cron to have less frequent updates and less jobs. A second fix is that when a stock_location_id is set on the backend, it should be used, it was only used for the import of stock. * [FIX] Exclude children from qty computation We already pass the children in 'location', if we don't disable 'compute_child', the qty computation will try to get the children of each child and generates query of 1mio chars. * [IMP] Group computation of qty_available Stats with 6700 products: Before: 570s After: 28s * [FIX] Apply filter on locations for variants quantities It was only applied on export of template quantities. * [IMP] Improve performance on prestashop qty recompute The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
hailangvn
pushed a commit
to hailangvn/connector-prestashop
that referenced
this pull request
Jun 25, 2021
* [FIX] Fix export when no location has flag prestashop_synchronized When there is no location with the flag prestashop_synchronized in the location tree, Current behavior: The `location` key in the context is empty then the qty_available is computed for *every* locations/warehouses. Expected behavior: If there is at least one location flagged, use it, otherwise, use all the locations of the tree, starting from the selected location or selected warehouse location. When no location can be found (for instance because user selected a location with a usage different than 'internal'), then it should just fail and not return the stock of all warehouses. Being able to synchronize without activating 'prestashop_synchronized' is important: when used, this feature will trigger an export every time a quant is changed or created, on a large catalog, we might prefer only using the cron to have less frequent updates and less jobs. A second fix is that when a stock_location_id is set on the backend, it should be used, it was only used for the import of stock. * [FIX] Exclude children from qty computation We already pass the children in 'location', if we don't disable 'compute_child', the qty computation will try to get the children of each child and generates query of 1mio chars. * [IMP] Group computation of qty_available Stats with 6700 products: Before: 570s After: 28s * [FIX] Apply filter on locations for variants quantities It was only applied on export of template quantities. * [IMP] Improve performance on prestashop qty recompute The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
MPoch-PlanetaTIC
pushed a commit
to PlanetaTIC/connector-prestashop
that referenced
this pull request
Feb 16, 2022
* [FIX] Fix export when no location has flag prestashop_synchronized When there is no location with the flag prestashop_synchronized in the location tree, Current behavior: The `location` key in the context is empty then the qty_available is computed for *every* locations/warehouses. Expected behavior: If there is at least one location flagged, use it, otherwise, use all the locations of the tree, starting from the selected location or selected warehouse location. When no location can be found (for instance because user selected a location with a usage different than 'internal'), then it should just fail and not return the stock of all warehouses. Being able to synchronize without activating 'prestashop_synchronized' is important: when used, this feature will trigger an export every time a quant is changed or created, on a large catalog, we might prefer only using the cron to have less frequent updates and less jobs. A second fix is that when a stock_location_id is set on the backend, it should be used, it was only used for the import of stock. * [FIX] Exclude children from qty computation We already pass the children in 'location', if we don't disable 'compute_child', the qty computation will try to get the children of each child and generates query of 1mio chars. * [IMP] Group computation of qty_available Stats with 6700 products: Before: 570s After: 28s * [FIX] Apply filter on locations for variants quantities It was only applied on export of template quantities. * [IMP] Improve performance on prestashop qty recompute The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
hailangvn
pushed a commit
to hailangvn/connector-prestashop
that referenced
this pull request
May 4, 2022
* [FIX] Fix export when no location has flag prestashop_synchronized When there is no location with the flag prestashop_synchronized in the location tree, Current behavior: The `location` key in the context is empty then the qty_available is computed for *every* locations/warehouses. Expected behavior: If there is at least one location flagged, use it, otherwise, use all the locations of the tree, starting from the selected location or selected warehouse location. When no location can be found (for instance because user selected a location with a usage different than 'internal'), then it should just fail and not return the stock of all warehouses. Being able to synchronize without activating 'prestashop_synchronized' is important: when used, this feature will trigger an export every time a quant is changed or created, on a large catalog, we might prefer only using the cron to have less frequent updates and less jobs. A second fix is that when a stock_location_id is set on the backend, it should be used, it was only used for the import of stock. * [FIX] Exclude children from qty computation We already pass the children in 'location', if we don't disable 'compute_child', the qty computation will try to get the children of each child and generates query of 1mio chars. * [IMP] Group computation of qty_available Stats with 6700 products: Before: 570s After: 28s * [FIX] Apply filter on locations for variants quantities It was only applied on export of template quantities. * [IMP] Improve performance on prestashop qty recompute The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
flachica
pushed a commit
to flachica/connector-prestashop
that referenced
this pull request
May 6, 2022
* [FIX] Fix export when no location has flag prestashop_synchronized When there is no location with the flag prestashop_synchronized in the location tree, Current behavior: The `location` key in the context is empty then the qty_available is computed for *every* locations/warehouses. Expected behavior: If there is at least one location flagged, use it, otherwise, use all the locations of the tree, starting from the selected location or selected warehouse location. When no location can be found (for instance because user selected a location with a usage different than 'internal'), then it should just fail and not return the stock of all warehouses. Being able to synchronize without activating 'prestashop_synchronized' is important: when used, this feature will trigger an export every time a quant is changed or created, on a large catalog, we might prefer only using the cron to have less frequent updates and less jobs. A second fix is that when a stock_location_id is set on the backend, it should be used, it was only used for the import of stock. * [FIX] Exclude children from qty computation We already pass the children in 'location', if we don't disable 'compute_child', the qty computation will try to get the children of each child and generates query of 1mio chars. * [IMP] Group computation of qty_available Stats with 6700 products: Before: 570s After: 28s * [FIX] Apply filter on locations for variants quantities It was only applied on export of template quantities. * [IMP] Improve performance on prestashop qty recompute The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
flachica
pushed a commit
to flachica/connector-prestashop
that referenced
this pull request
May 25, 2022
* [FIX] Fix export when no location has flag prestashop_synchronized When there is no location with the flag prestashop_synchronized in the location tree, Current behavior: The `location` key in the context is empty then the qty_available is computed for *every* locations/warehouses. Expected behavior: If there is at least one location flagged, use it, otherwise, use all the locations of the tree, starting from the selected location or selected warehouse location. When no location can be found (for instance because user selected a location with a usage different than 'internal'), then it should just fail and not return the stock of all warehouses. Being able to synchronize without activating 'prestashop_synchronized' is important: when used, this feature will trigger an export every time a quant is changed or created, on a large catalog, we might prefer only using the cron to have less frequent updates and less jobs. A second fix is that when a stock_location_id is set on the backend, it should be used, it was only used for the import of stock. * [FIX] Exclude children from qty computation We already pass the children in 'location', if we don't disable 'compute_child', the qty computation will try to get the children of each child and generates query of 1mio chars. * [IMP] Group computation of qty_available Stats with 6700 products: Before: 570s After: 28s * [FIX] Apply filter on locations for variants quantities It was only applied on export of template quantities. * [IMP] Improve performance on prestashop qty recompute The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
flachica
pushed a commit
to flachica/connector-prestashop
that referenced
this pull request
May 25, 2022
* [FIX] Fix export when no location has flag prestashop_synchronized When there is no location with the flag prestashop_synchronized in the location tree, Current behavior: The `location` key in the context is empty then the qty_available is computed for *every* locations/warehouses. Expected behavior: If there is at least one location flagged, use it, otherwise, use all the locations of the tree, starting from the selected location or selected warehouse location. When no location can be found (for instance because user selected a location with a usage different than 'internal'), then it should just fail and not return the stock of all warehouses. Being able to synchronize without activating 'prestashop_synchronized' is important: when used, this feature will trigger an export every time a quant is changed or created, on a large catalog, we might prefer only using the cron to have less frequent updates and less jobs. A second fix is that when a stock_location_id is set on the backend, it should be used, it was only used for the import of stock. * [FIX] Exclude children from qty computation We already pass the children in 'location', if we don't disable 'compute_child', the qty computation will try to get the children of each child and generates query of 1mio chars. * [IMP] Group computation of qty_available Stats with 6700 products: Before: 570s After: 28s * [FIX] Apply filter on locations for variants quantities It was only applied on export of template quantities. * [IMP] Improve performance on prestashop qty recompute The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
cubells
pushed a commit
to cubells/connector-prestashop
that referenced
this pull request
Jul 10, 2022
* [FIX] Fix export when no location has flag prestashop_synchronized When there is no location with the flag prestashop_synchronized in the location tree, Current behavior: The `location` key in the context is empty then the qty_available is computed for *every* locations/warehouses. Expected behavior: If there is at least one location flagged, use it, otherwise, use all the locations of the tree, starting from the selected location or selected warehouse location. When no location can be found (for instance because user selected a location with a usage different than 'internal'), then it should just fail and not return the stock of all warehouses. Being able to synchronize without activating 'prestashop_synchronized' is important: when used, this feature will trigger an export every time a quant is changed or created, on a large catalog, we might prefer only using the cron to have less frequent updates and less jobs. A second fix is that when a stock_location_id is set on the backend, it should be used, it was only used for the import of stock. * [FIX] Exclude children from qty computation We already pass the children in 'location', if we don't disable 'compute_child', the qty computation will try to get the children of each child and generates query of 1mio chars. * [IMP] Group computation of qty_available Stats with 6700 products: Before: 570s After: 28s * [FIX] Apply filter on locations for variants quantities It was only applied on export of template quantities. * [IMP] Improve performance on prestashop qty recompute The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
flachica
pushed a commit
to flachica/connector-prestashop
that referenced
this pull request
Aug 17, 2022
* [FIX] Fix export when no location has flag prestashop_synchronized When there is no location with the flag prestashop_synchronized in the location tree, Current behavior: The `location` key in the context is empty then the qty_available is computed for *every* locations/warehouses. Expected behavior: If there is at least one location flagged, use it, otherwise, use all the locations of the tree, starting from the selected location or selected warehouse location. When no location can be found (for instance because user selected a location with a usage different than 'internal'), then it should just fail and not return the stock of all warehouses. Being able to synchronize without activating 'prestashop_synchronized' is important: when used, this feature will trigger an export every time a quant is changed or created, on a large catalog, we might prefer only using the cron to have less frequent updates and less jobs. A second fix is that when a stock_location_id is set on the backend, it should be used, it was only used for the import of stock. * [FIX] Exclude children from qty computation We already pass the children in 'location', if we don't disable 'compute_child', the qty computation will try to get the children of each child and generates query of 1mio chars. * [IMP] Group computation of qty_available Stats with 6700 products: Before: 570s After: 28s * [FIX] Apply filter on locations for variants quantities It was only applied on export of template quantities. * [IMP] Improve performance on prestashop qty recompute The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
flachica
pushed a commit
to flachica/connector-prestashop
that referenced
this pull request
Oct 20, 2022
* [FIX] Fix export when no location has flag prestashop_synchronized When there is no location with the flag prestashop_synchronized in the location tree, Current behavior: The `location` key in the context is empty then the qty_available is computed for *every* locations/warehouses. Expected behavior: If there is at least one location flagged, use it, otherwise, use all the locations of the tree, starting from the selected location or selected warehouse location. When no location can be found (for instance because user selected a location with a usage different than 'internal'), then it should just fail and not return the stock of all warehouses. Being able to synchronize without activating 'prestashop_synchronized' is important: when used, this feature will trigger an export every time a quant is changed or created, on a large catalog, we might prefer only using the cron to have less frequent updates and less jobs. A second fix is that when a stock_location_id is set on the backend, it should be used, it was only used for the import of stock. * [FIX] Exclude children from qty computation We already pass the children in 'location', if we don't disable 'compute_child', the qty computation will try to get the children of each child and generates query of 1mio chars. * [IMP] Group computation of qty_available Stats with 6700 products: Before: 570s After: 28s * [FIX] Apply filter on locations for variants quantities It was only applied on export of template quantities. * [IMP] Improve performance on prestashop qty recompute The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
flachica
pushed a commit
to flachica/connector-prestashop
that referenced
this pull request
Oct 20, 2022
* [FIX] Fix export when no location has flag prestashop_synchronized When there is no location with the flag prestashop_synchronized in the location tree, Current behavior: The `location` key in the context is empty then the qty_available is computed for *every* locations/warehouses. Expected behavior: If there is at least one location flagged, use it, otherwise, use all the locations of the tree, starting from the selected location or selected warehouse location. When no location can be found (for instance because user selected a location with a usage different than 'internal'), then it should just fail and not return the stock of all warehouses. Being able to synchronize without activating 'prestashop_synchronized' is important: when used, this feature will trigger an export every time a quant is changed or created, on a large catalog, we might prefer only using the cron to have less frequent updates and less jobs. A second fix is that when a stock_location_id is set on the backend, it should be used, it was only used for the import of stock. * [FIX] Exclude children from qty computation We already pass the children in 'location', if we don't disable 'compute_child', the qty computation will try to get the children of each child and generates query of 1mio chars. * [IMP] Group computation of qty_available Stats with 6700 products: Before: 570s After: 28s * [FIX] Apply filter on locations for variants quantities It was only applied on export of template quantities. * [IMP] Improve performance on prestashop qty recompute The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
florian-dacosta
pushed a commit
to akretion/connector-prestashop
that referenced
this pull request
Jun 15, 2023
* [FIX] Fix export when no location has flag prestashop_synchronized When there is no location with the flag prestashop_synchronized in the location tree, Current behavior: The `location` key in the context is empty then the qty_available is computed for *every* locations/warehouses. Expected behavior: If there is at least one location flagged, use it, otherwise, use all the locations of the tree, starting from the selected location or selected warehouse location. When no location can be found (for instance because user selected a location with a usage different than 'internal'), then it should just fail and not return the stock of all warehouses. Being able to synchronize without activating 'prestashop_synchronized' is important: when used, this feature will trigger an export every time a quant is changed or created, on a large catalog, we might prefer only using the cron to have less frequent updates and less jobs. A second fix is that when a stock_location_id is set on the backend, it should be used, it was only used for the import of stock. * [FIX] Exclude children from qty computation We already pass the children in 'location', if we don't disable 'compute_child', the qty computation will try to get the children of each child and generates query of 1mio chars. * [IMP] Group computation of qty_available Stats with 6700 products: Before: 570s After: 28s * [FIX] Apply filter on locations for variants quantities It was only applied on export of template quantities. * [IMP] Improve performance on prestashop qty recompute The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
Neitherkx
pushed a commit
to admingrupoisonores/connector-prestashop
that referenced
this pull request
Jan 24, 2024
* [FIX] Fix export when no location has flag prestashop_synchronized When there is no location with the flag prestashop_synchronized in the location tree, Current behavior: The `location` key in the context is empty then the qty_available is computed for *every* locations/warehouses. Expected behavior: If there is at least one location flagged, use it, otherwise, use all the locations of the tree, starting from the selected location or selected warehouse location. When no location can be found (for instance because user selected a location with a usage different than 'internal'), then it should just fail and not return the stock of all warehouses. Being able to synchronize without activating 'prestashop_synchronized' is important: when used, this feature will trigger an export every time a quant is changed or created, on a large catalog, we might prefer only using the cron to have less frequent updates and less jobs. A second fix is that when a stock_location_id is set on the backend, it should be used, it was only used for the import of stock. * [FIX] Exclude children from qty computation We already pass the children in 'location', if we don't disable 'compute_child', the qty computation will try to get the children of each child and generates query of 1mio chars. * [IMP] Group computation of qty_available Stats with 6700 products: Before: 570s After: 28s * [FIX] Apply filter on locations for variants quantities It was only applied on export of template quantities. * [IMP] Improve performance on prestashop qty recompute The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
Neitherkx
pushed a commit
to admingrupoisonores/connector-prestashop
that referenced
this pull request
Aug 19, 2024
* [FIX] Fix export when no location has flag prestashop_synchronized When there is no location with the flag prestashop_synchronized in the location tree, Current behavior: The `location` key in the context is empty then the qty_available is computed for *every* locations/warehouses. Expected behavior: If there is at least one location flagged, use it, otherwise, use all the locations of the tree, starting from the selected location or selected warehouse location. When no location can be found (for instance because user selected a location with a usage different than 'internal'), then it should just fail and not return the stock of all warehouses. Being able to synchronize without activating 'prestashop_synchronized' is important: when used, this feature will trigger an export every time a quant is changed or created, on a large catalog, we might prefer only using the cron to have less frequent updates and less jobs. A second fix is that when a stock_location_id is set on the backend, it should be used, it was only used for the import of stock. * [FIX] Exclude children from qty computation We already pass the children in 'location', if we don't disable 'compute_child', the qty computation will try to get the children of each child and generates query of 1mio chars. * [IMP] Group computation of qty_available Stats with 6700 products: Before: 570s After: 28s * [FIX] Apply filter on locations for variants quantities It was only applied on export of template quantities. * [IMP] Improve performance on prestashop qty recompute The add/union operation between recordsets is slow, aggregate the ids in a set and call the browse only once on the complete set is dramatically faster. See guewen/connector-magento#9
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I skipped the usage of a dictionnary because the |= on a big recordset is really slow and not very usefull.
What do you htink ?