Optimise database queries

[thenameiwantedwastaken]

Every time I try to access my recipes page I get a 504 timeout. If I refresh the page it works okay. All other pages of my nextcloud implementation work okay, so it seems there is something about the recipe app that is simply too slow. I am running nextcloud via docker / nginx. I don't think it's a problem with my specific setup of nextcloud as all other aspects of nextcloud appear quickly.

Navigating within the recipes is fine once the recipe page has shown... but just the initial view is too slow.

[mrzapp]

I am using the same setup as you, Docker with NGINX, and I've never observed this behaviour. You're also the first to open an issue like this, so I assume there is something else going on.

I can only imagine a timeout would occur if the filesystem is being slow for some reason. Are your recipes stored on a network mount that might not respond at times?

Also, if you can find some relevant log messages, it would help shed some light on the situation.

[thenameiwantedwastaken]

Recipes are stored in the same folder structure as all the other nextcloud files. I can browse the recipe images and json files without delay via the nextcloud folder interface. However, if I actually try to open a json file to preview it in the nextcloud interface, I just get a spinning circle. This happens to all json files, not just recipes.

[christianlupus]

Do you mazbe have a plugin installed in NC that tries to start up and open the JSON?

[mrzapp]

@thenameiwantedwastaken are you perhaps using an older NC version than 18?

[dietrichmd]

I too have this issue. When I load cookbook, I get 504 error..upon reload, it loads fine.

Looking at the logs, it looks like a php memory error

[proxy_fcgi:error] [pid 17264] [client 172.69.71.97:63198] AH01071: Got error 'PHP message: PHP Fatal error: Allowed memory size of 2097152 bytes exhausted (tried to allocate 4096 bytes) in /home/drive.braeswood.pl/public_html/3rdparty/composer/autoload_static.php on line 446\n', referer: https://drive.braeswood.tk/index.php/apps/cookbook/

Using the latest and gratest Nextcloud (updated 2d ago) -- Virtual Mem is set per nextcloud instructions and this instance is allocated 512M for PHP-FPM.

Included apps in nextcloud: https://i.imgur.com/fuIsXW7.png

In addition to the 504, occasionally I'll click a category and cookbook will just sit there like the link doesn't even exist. A reload will fix this behavior as well.

Thanks

[stenosis]

Just wanted to mention that I can confirm the same behavior described here, receiving a 504 on the first load.

[mrzapp]

Is this still the case for v0.7.0?

[stenosis]

I just updated the Cookbook app to version 0.7.1 on the latest Nextcloud version 19.0.0 and unfortunately the problem still exists for me.

For the info: I use a Nextcloud docker installation which gets automatically updated to the latest version. In addition it use a PostgreSQL 12 database.

There is plenty of resources available and every other app in Nextcloud just works fine for me so this problem only occurs with the Cookbook app. And a look at the Nextcloud log file shows no recording related to this.

[mrzapp]

@stenosis @dietrichmd @thenameiwantedwastaken do you have any interesting setups related to file storage? For instance, are your recipes stored on a remote volume?

In any case, I think the best way to debug this would be to back up your recipe folder and then add recipes back one by one until you hit the problem again. It could be content related, seeing as you're the only ones dealing with this issue.

[dietrichmd]

@mrzapp nothing weird or interesting here -- everything is local. On the recipe side, I have 20 in there, so definitely not a content-size issue. Ill try to remove and readd over the next few days.

[dietrichmd]

@mrzapp I will say, after yesterday's update, things are snappier. The initial load of the app sometimes hangs and I have to completely reload nextcloud (web) for it to come up. And after swapping between recipes for a few minutes, it seems to get sluggish, but not near as slow as before and no timeout errors (for now).

[thenameiwantedwastaken]

Here's what I found after adding recipes one-by-one:

1. The more recipes you have, the longer the recipes page takes to display. A lot of recipes leads to a time out. For me, it's a little over one second per recipe.
2. The update-interval affects the above point. The delay only occurs if the time between viewing the recipes page is longer than the update interval.

A temporary fix is to increase the update interval to a large number, and manually use the Rescan Library button under the settings options after you add a recipe. If you set the update interval to 1440, the delay mentioned in point 1 above will only occur once a day.

Could it be that rescanning the library actually occurs when you view the recipes page if the update interval has passed since the last view. If this is the case, it's not a great design. The rescanning would be better to occur somehow in the background.

I hope this helps.

[mrzapp]

@thenameiwantedwastaken it does indeed work like that, because cron jobs need to be set up manually by users. This is what I have found on the topic: https://docs.nextcloud.com/server/stable/admin_manual/configuration_server/background_jobs_configuration.html?highlight=background%20jobs

Relying on this would mean no auto rescans until users perform this configuration, so I don't think it's feasible, seeing as PHP is a per-request runtime, and NextCloud doesn't enable any workarounds for that by default.

We could batch the recipes, though, that would certainly speed things up a bit. How many recipes are in your library, and how long does a rescan typically take?