Fixing performance issues programme

[oparoz]

This is a meta ticket which tracks all the various work being done to improve the performance of the app

Please support and comment on individual tickets if you want to help as it's difficult to track what's important to backers otherwise.

Loading the app

• Do not map the whole cloud into memory, load one folder at a time Initialisation process of the app takes too long and is too memory and CPU hungry as it tries to map all media files found in the system #17 . Potential side effects are discussed here How to deal with empty parent folders? #24
• Cache albums as we go along so that we don't have to reload the list of pictures when we're in an album we've visited before Cache albums as we go along #41

Getting thumbnails

• Only preload the number of thumbnails we were told to preload Only preload the number of thumbnails we were told to preload #60
• Switch to an ID based mapping of pictures and Thumbnails to fix URL used to retrieve thumbnails can generate "URI too long" error messages [$15 awarded] #27
• Shorten the paths of pictures located in sub-albums. Requires URL used to retrieve thumbnails can generate "URI too long" error messages [$15 awarded] #27
• Show thumbnails as soon as they're received (Instant thumbnails showing #511) instead of waiting for the whole row to be loaded Show thumbnails as soon as they're received #29
• Generate thumbnails on demand (Generate thumbnails on demand #22), so that users don't have to wait too long before being able to see something. The cron/background task solution won't be implemented here, but is a core solution Migrate preview generation to async job execution core#11685

For issues in core, see this meta ticket: owncloud/core#13609
Things should be much better on oC 8.1+, once core can generate thumbnails in seconds rather than minutes for large media files.

Feel free to add your thoughts and to raise new issues to address specific problems.

There is a $60 open bounty on this issue. Add to the bounty at Bountysource.

[cbxk1xg]

I would be pleased to help you guys with the code. As I wrote earlier, I developed similar stuff before. And I guess it's more an user experience thing than a performance issue. But I do also have some ideas to improve the performance.

[oparoz]

That's great news! :)
From my point of view, what's most needed is fixes in core regarding thumbnail generation:

• Having preview categories and returning a predefined thumbnail. Asked size is 1920x1080, which is larger than 1024x1024, but smaller than 2048x2048. Return 1024 or 2048 and let the browser shrink it, etc.
• Async thumbnail generation, triggered at upload time Migrate preview generation to async job execution core#11685 (comment) , but we're told to wait for a solution. Also, bear in mind that in 8.1, we should have the max preview which is used to create all the smaller sizes.

In the app itself, we could have this:

• Generate thumbnails on demand, so that users don't have to wait too long before being able to see something Generate thumbnails on demand #22 . This cannot be done offline because of encryption.

But I also would like to hear what you thought of :)
Make sure you check with me before touching files as some classes may already be in a different state internally, so that we can work together.

[libasys]

A new idea to fix performance issue, i have changed the gallery searching mechanism. In OC core you have a function called searchRaw. With this function you can search explicite for a file. My idea, I place or rename an image to "cover.jpg" in a folder. Now with the search function above, we only search for folders with "cover.jpg". With this mechanism you have a simple structure and a big performance improvement. The 2. part, i use the cover.jpg as thumbnail for the folder and as info, so if you add iptc information to this file like title, than the folder gets the title of the cover.jpg! So you need no startpath and no scanning for all files. You have the control what is shown up in the gallery! No need to walk from folder to subfolder and soon!

[oparoz]

Could I ask you to open a separate issue for this so that people can voice their opinion?

[oparoz]

occ command to generate thumbnails: #387

[oparoz]

There is now a PR to show thumbnails before the row is completed: #511

[oparoz]

This issue was moved to nextcloud/gallery#18