Results limit?

[chrisvidal]

Is there an easy to implement a limitation in the number of results returned?
I am facing a memory allocation error on the prod. server.

[tobias-kuendig]

What provider is causing the memory issue? If it is your own search provider, you can return as many results as you want. If it is one of the built in providers, we would have to implement something.

You can also require a query to be at least a certain length so people don't run queries that generate lots of hits (like a query for a) or something.

If you find out where exactly the the memory issue is, I can help you resolve it.

[chrisvidal]

not much trace log with a memory error.

it is doing a search in the MALL products, so I guess it is using the ProductSearchProvider

the search input is limited to 3 char., and it still gives the error, I upgraded to 4 char. minimum.

what I can find is:

ymfony\Component\Debug\Exception\FatalErrorException: Allowed memory size of 536870912 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Concerns/HasAttributes.php:775

and

Symfony\Component\Debug\Exception\FatalErrorException: Allowed memory size of 536870912 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/vendor/october/rain/src/Extension/ExtendableTrait.php:142

Symfony\Component\Debug\Exception\FatalErrorException: Allowed memory size of 536870912 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/plugins/voilaah/marketplace/classes/plugin/BootExtensions.php:290

the code for the last one is an extension of MALL Variant model

protected function extendOFFLINEMallVariant()
    {
        VariantModel::extend(function ($model) {
            $model->addFillable(['sku']);
            $model->addJsonable(['main_images_set']);

=>line 290
          $model->addDynamicMethod( 'pickup_addresses', function () use ($model) {
                return $model->product->pickup_addresses;
            });
            $model->addDynamicMethod( 'getAllShippingMethodsAttribute', function () use ($model) {
                $collection = $model->shipping_methods;
                $collection = ($collection) ? $collection->merge(ShippingMethod::where('is_marketplace_level', true)->get())
                : ShippingMethod::where('is_marketplace_level', true)->get();
                return $collection;
            });

[tobias-kuendig]

How many Shipping methods do you have? Is it just a few entries or are there thousands?

[chrisvidal]

2 entries only

[tobias-kuendig]

It is very hard to help you here. Out of memory exceptions are nasty in general, since you often don't see the exact cause in the log/exception. Try to find the line or method that triggers the exception by disabling providers or skipping certain queries, for example by simply returning empty Collections.

[chrisvidal]

it is the ProductSearchProvider who is raising the exception.

we have 100k of products/variants in the MALL index.

searching with 1, 2 or 3 characters raised the exception.

4 characters or more are ok.

[tobias-kuendig]

I can imagine that this line (or something close) is the culprit:

https://github.com/OFFLINE-GmbH/oc-mall-plugin/blob/develop/classes/search/ProductsSearchProvider.php#L27

The way the SearchProvider is pretty naive, as it simply queries both model types and then merges the results together. This works great with a few products but obviously falls apart in your case.

A better solution is to query the offline_mall_index table, this will be very tricky to implement using translations though!

[chrisvidal]

thanks for your answer @tobias-kuendig

could set a limit property in the component be easier to solve this kind of issue?

[tobias-kuendig]

🤔 good question. If a search returns let's say more than 100 results, maybe we can safely take the first 100 and present them to the user. Any higher number of results is not that helpful to begin with. The question is how we decie what the first 100 results are without querying all of them first.

I guess limiting the number of results is a good first solution to this problem as I'd rather have less results than none at all because of the out of memory problem.

But we will need a proper solution in the long run, for bigger installations.

[damsfx]

I sometimes encounter similar problems when the plugins RainLab.DebugBar or Offline.Clockwork are active.

[tobias-kuendig]

Good point @damsfx, the debugbar and clockwork packages are very ressource hungry. Might this be the problem here @chrisvidal? Also, do you have XDebug enabled? This increases memory usage as well.

[chrisvidal]

that was not my case, i wasn't using any XDebug nor debugbar tools

[tasaduqh]

Is anyone working on this considering this is open? I am facing an out of memory issue even though I have barely added include in site search component to some pages.