Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collectstatic to S3 is slow #3290

Closed
DeD1rk opened this issue Aug 19, 2023 · 4 comments · Fixed by #3479
Closed

Collectstatic to S3 is slow #3290

DeD1rk opened this issue Aug 19, 2023 · 4 comments · Fixed by #3479
Assignees
Labels
bug Something that should be fixed optimization Issues regarding slowness priority: high Must be dealt with before the next release is deployed. server Server related issues
Milestone

Comments

@DeD1rk
Copy link
Member

DeD1rk commented Aug 19, 2023

Describe the bug

On every boot, we run collectstatic, which uploads static files to S3. This is very slow as it appears that all/most files are uploaded every time. It would be nice if we can check whether a file has already been uploaded before doing that, to speed up our deployments/reboots.

How to reproduce

Reboot stating, and observe that booting takes quite a long time.

Expected behaviour

If few/no static files have changes, collectstatic is quick.

@DeD1rk DeD1rk added priority: low Should be dealt with when nothing else remains. bug Something that should be fixed server Server related issues optimization Issues regarding slowness labels Aug 19, 2023
@DeD1rk
Copy link
Member Author

DeD1rk commented Nov 9, 2023

While it's definitely not easy to make collectstatic become efficient, we can probably do something simple like setting a flag file in a docker container after running collectstatic once, so that we don't rerun collectstatic on rebooting, but only whenever a new container is created (i.e. after a release or if for some reason a container has to be recreated).

@DeD1rk
Copy link
Member Author

DeD1rk commented Nov 9, 2023

We need something like that in order to make #3477 practical.

@DeD1rk DeD1rk added priority: medium A new feature or a bugfix that is non-critical. priority: high Must be dealt with before the next release is deployed. and removed priority: medium A new feature or a bugfix that is non-critical. priority: low Should be dealt with when nothing else remains. labels Nov 9, 2023
@DeD1rk DeD1rk added this to the Release 51.0 milestone Nov 9, 2023
@DeD1rk DeD1rk self-assigned this Nov 9, 2023
@JobDoesburg
Copy link
Contributor

Wouldn't it be better to just run collect static upon building a container?

@DeD1rk
Copy link
Member Author

DeD1rk commented Mar 7, 2024

Can't do that when collecting to S3...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something that should be fixed optimization Issues regarding slowness priority: high Must be dealt with before the next release is deployed. server Server related issues
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants