Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Limit the parallel processing on upload #913

Closed
miahi opened this issue Feb 11, 2021 · 4 comments
Closed

[Enhancement] Limit the parallel processing on upload #913

miahi opened this issue Feb 11, 2021 · 4 comments
Labels
enhancement New feature or request

Comments

@miahi
Copy link

miahi commented Feb 11, 2021

I have a memory and CPU limited server (shared hosting) and when I try to upload a large number of photos it seems that each of them creates a process for resizing (so if I drag and drop 100 photos, 100 threads will start). This hits the CPU/memory limit and the processes crash, so I get a lot of fails in the status, or the upload crashes completely because the upload process hit the limit and the whole page crashes. The limit varies with the photo dimensions, probably because the high resolution photos need more CPU time and memory to resize.

It would be great if only a limited number of processing threads can be set to start in parallel, and all the other uploads wait until there is a processing slot for them.

@kamil4 kamil4 added enhancement New feature or request JS - Lychee-Front labels Feb 11, 2021
@kamil4
Copy link
Contributor

kamil4 commented Feb 11, 2021

Agreed. We discussed some of this recently in #903 as well, although the fix for that particular bug will be different.

@kamil4
Copy link
Contributor

kamil4 commented Feb 11, 2021

As a workaround for now, assuming that you have file access to that server, you can upload the files to the server using ftp/ssh/whatever and then use Lychee's Import from Server feature. Importing is then done one-by-one (sequentially).

@miahi
Copy link
Author

miahi commented Feb 12, 2021

Good to know that Import from server is sequential (and yes, I hit the #903 also). Thank you!

@ildyria
Copy link
Member

ildyria commented Feb 13, 2021

should be fixed now with #919

@ildyria ildyria closed this as completed Feb 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants