Optimize the Dockerfile to reduce the size of the image. (50.76 GB to 19.04 GB) #1976
Conversation
Signed-off-by: 陳鈞 <jim60105@gmail.com>
Signed-off-by: 陳鈞 <jim60105@gmail.com>
Too big to run on GitHub free runner. Error: no space left on device This reverts commit fc029d7. Signed-off-by: 陳鈞 <jim60105@gmail.com>
…:3.10-slim - Change the base image from `python:3.10` to `python:3.10-slim` - Expose ports `7860` and `6006` in the Dockerfile Signed-off-by: 陳鈞 <jim60105@gmail.com>
- Switch to a slim variant of the Python 3.10 Docker image for the build stage. - Introduce a conditional installation of `pillow-simd` replacing `pillow`, specific to x86 architecture. - Add required dependencies for `pillow-simd` installation, along with cleanup commands to remove unnecessary package lists after installation. - Update runtime dependencies by adding `libtcl8.6` and `libtk8.6` to the final Docker image. Signed-off-by: 陳鈞 <jim60105@gmail.com>
Signed-off-by: 陳鈞 <jim60105@gmail.com>
|
Look like a great commit... I hope it will not cause issues for current users... but I will accept it. I don't use docker so I can't really test it... fingers crossed. |
|
I think there is no breaking changes in docker side. I have taken care of the file permission in the volume, user id are the same as the original image, so there is no need to make any manual modifications. |
|
@jim60105 I am seriously reqorking how kohya gui is structured. I have gotten rid of all sd-scripts file copy in my repo and I am now cloning a copy of kohya sd-scripts in the repo to run his sode... no more maintenance of his code in mine. BUT, docker does not execure the setup_linux.py code that would clone his repo... I hope you can add the missing glue to the docker file to make it properly clone a copy of the appropriate sd-scripts release during the container build. The refactored code is in the Looking forward to a PR |
|
I can handle this matter. However, have you considered using git submodule to handle this situation? If you use the git submodule solution, users will get the submodule during the |
|
Hi, I have but submodules add a level of command complexity for users when cloning and updating the main repo. To fix that more coding and handling need to be done using code similar to what I have to use to handle but with different syntax. overall I prefer to keep the user interaction with git as simple as possible and manage the cloning of the sub repo in case… unless you can convince doing submodules can be simpler for users and code update of as-script perspective. My past experience with submodules has been hell and I have avoided it ever since. |
|
The "a level of command complexity for users" is simply need to do git clone with --recursive. I think adhering to standard practices is highly crucial for open source projects. In my experience, git submodules have not caused any particular problems for my open source projects or professional projects. It is assumed that the project manager is familiar with git and knows how to operate submodules. I usually write this in the GitHub README, for your reference: Important Clone the Git repository recursively to include submodules: |
The current Dockerfile is excessively large, containing unnecessary files within the base image.
By substituting the base image with
python:3.10-slimand implementing certain multi-stage techniques, I managed to decrease the image size.The image has been installed with these package versions
I've tested with LoRA training with fp8 base option + xformers and works well!
Additionally, I made an attempt to write a CI image build; however, the size of the build still exceeds the capacity of GitHub's free runners, so I revert the commit. If GitHub increases runner disk size in the future or if you wish to use a Self-hosted runner, you can get the commit back.