Background thumbnail generation [$25]

[oparoz]

From @ted42 on February 11, 2015 18:53

Feature request

User type: any

User level: any

Description

@TheDD
Since the app is extremely slow when opening a folder for the first time I think some actions like thumbnail generation should be moved to a background/cron job.

(I didn't review the code and have no idea if there's already some background work happening. If there is, I'd still like to discuss performance improvements)

@astroleo
I run owncloud 8.2.2 on a Raspberry Pi which works just fine (and thank you all for this!), but isn't terribly fast. When I share a link to a gallery it takes quite some time to browse through the gallery for the first visitor. opening the gallery, clicking on an image, then clicking the overlaid "next" arrow in the screen-sized image. I understand that opening one screen-sized image triggers the creation of the next one, but (for the Raspy) this isn't fast enough and the first user has to wait each time she clicks the "next" button.

My suggestion would be to add an option to create, right after uploading the album, the various required image sizes so that they are served quickly from the cache already for the first visitor.

Benefit / value

For OC installations on low-power computers (e.g. Raspberry Pi) the first visitor would already be able to smoothly scroll through an album.

Risk / caveats

If uploading a large amount of pictures, creating the cached versions may create quite some server load. The pre-caching task should therefore be given a low priority.

Sponsorship

@astroleo
I'm afraid I cannot sponsor anything yet, but I'd be happy to help developing / testing this new feature if an expert could provide me with some background on how caching works in OC.

Are you a developer willing to implement this feature?: @astroleo: yes

Can you sponsor the development of this feature or do you know somewone who can?: no

There is a $25 open bounty on this issue. Add to the bounty at Bountysource.

Copied from original issue: owncloud/gallery#18

[oparoz]

This will happen in 8.1, see this issue in core: owncloud/core#11685

[oparoz]

Moved to 8.2...

[oparoz]

owncloud/core#11685 has been moved to 9.0 :(

[oparoz]

Depending on the timeline for the delivery of owncloud/gallery#221, I could add a method to the service to generate the thumbnail at the same time the information is collected.

[oparoz]

@astroleo - I invite you to read owncloud/core#11685 where quite a few ideas are thrown around.

In short:

• At upload time is the best solution as the server would have the encryption key
• There may be a timing issue if it takes 5 minutes to generate a thumbnail. We can't make sync requests and we can't request the encryption key if files are encrypted
• Create an app to manage the meta data stored in media files owncloud/gallery#550 is an app which manages images meta data. Maybe it could be modified to generate the thumbnail after analysis
• There is already code to generate thumbnails offline. The idea is to test this on the WD Pi-drive to be able to tune it into an acceptable solution

[friesoft]

Any news on this? I tried to read through all open issues but all issues just crosslink each other with latest infos all from owncloud. I couldn't find the core bug for nextcloud. All bountys for nextcloud also still only mention the owncloud issues..

[oparoz]

Nextcloud 11 includes a new command which can be used from a cron to generate thumbnails. That's a start :)

[Ramalama2]

Im on Nextcloud 11 RC2 there is no new command...

Still waiting, this is the most needed feature for me :-)

Cheers :-)

[Huster-at-github]

@oparoz

Nextcloud 11 includes a new command which can be used from a cron to generate thumbnails.

Cannot find anything! :-(

[oparoz]

Use this app:
https://apps.nextcloud.com/apps/previewgenerator

[elisman]

As well as my tests show, the app generates preview images for the slideshow and the list view. No thumbnails will be generated by the app. Unfortunately.

[techadrian]

Please stop suggesting previewgenerator as a thumbnail generation solution as this app does not generate thumbnails but previews in file manager. In a desktop browser session these are too small to see and distinguish the contents of a picture.

This issue is on the table since 2013, and I've been waiting it ever since. I have 4000 (4-7 MB each) pictures stored in a raid 0 ssd with 16 GB of memory and 4 high freq cores, and thumbnail generation time is crap, 5-6 seconds per row of several picture ......all the community is asking for this feature as we use nextcloud for picture storage also.
If you don't care about this issue, just let us know straight forward so we can try and move on from nextcloud.......I saw posts from nextcloud devs about this being implemented in nextcloud 9 ....we are at 11 now and thumbnail problem is still here ....what gives....?

[rullzer]

@techadrian the issue here was that the gallery shipped with Nextcloud 11 was not updated to use the new endpoint (it used a hack to get around server limitations). If you use gallery from git. Or use the Nextcloud 12 beta the previews are shared and thus properly generated with the preview generator app.

[Huster-at-github]

It works in NC 12?

[oparoz]

Yes

[Huster-at-github]

Yes, its running ..... but also stops when detecting a damaged JPG as this:

[ghost]

So what's the status on this? Bug is still open and I don't see any commands in occ related to thumbnails. Nextcloud 13.0.0.

[wUFr]

personally dont understand why is generating of previews so slow. I used same technique on my websites, where it would generate previews from original images on first page load and it did not have a problem handling 10+ fullhd images under 1 second.... Nextcloud is like 1 image per second, even on PHP 7, which should give that extra performance boost

hope this gets fixed asap, pain to browse photos on nextcloud like this :(

[TMaddox]

please add this asap, I am really looking forward to this feature!

[azlux]

You can install php-imagick and imagemagick-6-common

Nextcloud will use it (it's native) and become quite faster. And with previewgenerator, you can precreate it.

Az

[wUFr]

btw putting nextcloud on SSD "resolved" this issue with bruteforce, it still should run way faster, considering its on SSD

@azlux will check imagick

[nebulade]

This is also an issue for me, especially when using a slower external storage volume. Is there maybe an option somewhere, which would generate the thumbnails on file upload instead of a cron job?

[juliusknorr]

@oparoz @rullzer This should already work fine right now, as the gallery app uses the server preview manager, and therefore the previewgenerator app can do that, right?