-
Notifications
You must be signed in to change notification settings - Fork 195
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Compress Docker images with zstd #1880
Comments
I looked into it and it seems to be a broader problem than Docker for macOS. Docker, irrespective of the OS, does not seem to support zstd compression. The following lines from the AWS Fargate write up (linked in the original issue description) also suggest that zstd compression is currently only supported by OCI images and not by the Docker Engine. However, I think it is possible to continue to use Docker images for all local development (built using |
Beyond compatibility and long-term support, we should also consider what the potential savings of this are vs other image slimming techniques (like ejecting development dependencies from the deployed image). |
I'm going to close this @dhruvkb, after thinking about this a bit more. The compatibility questions aren't worth it and we aren't seeing the same issues here. The biggest improvement in deployments we would actually benefit from is switching to some kind of multi-stage deployment where a single API task is deployed first, then the rest are spun up simultaneously. We could accomplish this, potentially, by having two API services, one very small one that is only responsible for running migrations etc, never servicing requests, then a separate real one and having they deploy in order. That's just one idea. In any case, there are other ways that wouldn't cause maintainability or compatibility issues by trying. I don't think image compression is going to give us the best result compared to actually reducing the real size of our Docker image, either. |
Problem
ECS FARGATE does not cache images as they are pulled. This means it must re-pull the image from GHCR for each task that is started during a deployment. The time it takes to pull an image from GHCR has a significant impact on the total length of our deployments, especially for the API which deploys tasks one by one rather than simultaneously as it does for the frontend.
Description
Read the documentation for zstd compression here: https://aws.amazon.com/blogs/containers/reducing-aws-fargate-startup-times-with-zstd-compressed-container-images/
Update our docker builds to use zstd compression.
Docker for macOS apparently does not yet support zstd compression. Implementing this issue would remove the ability for maintainers on macOS to run deployed images locally. This could cause issues with debugging. This issue might not be possible to responsibly implement until after Docker for macOS supports zstd.
The text was updated successfully, but these errors were encountered: