-
Notifications
You must be signed in to change notification settings - Fork 9.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Elastic Search 5 Indexing Performance Issue #20966
Comments
Hi @DarthFly. Thank you for your report.
Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:
For more details, please, review the Magento Contributor Assistant documentation. @DarthFly do you confirm that you was able to reproduce the issue on vanilla Magento instance following steps to reproduce?
|
Hi @engcom-backlog-nazar. Thank you for working on this issue.
|
Hi @DarthFly thank you for your report, which version of elastic you use ? |
@engcom-backlog-nazar Elastic 5 latest and the one installed on cloud. It doesn't really matter as problem is inside magento code, not when data is transferred to and from Elastic. |
@DarthFly in your case seems like not) but really which is version 5.2.x? |
✅ Confirmed by @engcom-backlog-nazar Issue Available: @engcom-backlog-nazar, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself. |
It any case it was |
@DarthFly thanks, now clear ) |
Hi @GovindaSharma. Thank you for working on this issue.
|
This is a HUGE issue for us as well. We have 2 store views with 480,xxx skus. Indexing of catalogsearch_fulltext for elasticsearch 5.2.2 went from approximately 60 minutes on 2.2.5 (enterprise edition) to 89+ hours on 2.3. We cannot go to production with this. How could this have been released (especially for enterprise customers) with this large an issue in it? |
Hi @DarthFly. Thank you for your report.
The fix will be available with the upcoming 2.4.0 release. |
Preconditions (*)
Steps to reproduce (*)
brands
. We had around 300 values.bin/magento indexer:reindex catalogsearch_fulltext
(or trigger it in a way to be able to debug)Expected result (*)
Actual result (*)
8h
Issue comes from this method.
For each product magento run this code providing an array of attribute ids here. This is used for both multiple and single (select) attributes, so
$attributeValues
may look like[123, 456]
(simplified, value looks a little differently), but forbrands
it mostly contain one value.$attribute->getOptions()
return an array with 300+ values, each is compared to the$attributeValues
array to be able to retrieve a label value for this specific attribute.Even if one search takes milliseconds (well, mostly hundreds of milliseconds) multiply this code by amount of products and you will see drop down of performance here by 5 hours.
There are 2 ways to improve this
foreach
fast - cache attribute values by id somewhere and validate if each value inside$attributeValues
exists as key or cache values that are returned after function (will require joining array). Class is used for every product, so pre-cached value will be used correctly without recalculating it on each product.The text was updated successfully, but these errors were encountered: