Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove GNU Toolchain for RISC-V from Dockerfile #290

Merged
merged 1 commit into from
Jan 1, 2024

Conversation

henrybear327
Copy link
Collaborator

An experiment of using xpack as the GCC toolchain instead of our self-compiled docker image.

#286

@henrybear327
Copy link
Collaborator Author

Image size evolution (the total size shown is accumulative, with respect to the previously listed items)

  • Ubuntu:22:04 -> 69.4 MB
  • Copying the source code (tests folder itself is about 500 MB) -> 831 MB
  • Install extra packages (e.g. libsdl2-dev) -> 1.49GB

For the GCC toolchain

  • Using xpack toolchain -> 3.56GB
  • Copying from the self-built GCC image -> 2.12GB

@henrybear327 henrybear327 self-assigned this Dec 12, 2023
@henrybear327 henrybear327 requested a review from jserv December 12, 2023 20:25
@henrybear327 henrybear327 force-pushed the ci/reduce_image_size branch 2 times, most recently from f4b9dda to c090200 Compare December 12, 2023 21:01
@henrybear327 henrybear327 changed the title Use xpack as the GCC toolchain Remove toolchain from the dockerfile Dec 12, 2023
@henrybear327 henrybear327 changed the title Remove toolchain from the dockerfile Remove GCC toolchain from the dockerfile Dec 12, 2023
@henrybear327 henrybear327 changed the title Remove GCC toolchain from the dockerfile Remove riscv GCC toolchain from the dockerfile Dec 12, 2023
@jserv jserv changed the title Remove riscv GCC toolchain from the dockerfile Remove GNU Toolchain for RISC-V from Dockerfile Dec 13, 2023
@jserv jserv force-pushed the master branch 3 times, most recently from 62f12cc to 6d87fd6 Compare December 16, 2023 15:42
@henrybear327
Copy link
Collaborator Author

Using xpack will actually increase the image size, so maybe we should drop this PR in the end.

@jserv
Copy link
Contributor

jserv commented Dec 22, 2023

Using xpack will actually increase the image size, so maybe we should drop this PR in the end.

The goal here is to utilize scripts for downloading and extracting the toolchain from xpack, similar to those already in use by our CI pipeline. This approach should help in significantly reducing the size of our Docker image.

@henrybear327
Copy link
Collaborator Author

Using xpack will actually increase the image size, so maybe we should drop this PR in the end.

The goal here is to utilize scripts for downloading and extracting the toolchain from xpack, similar to those already in use by our CI pipeline. This approach should help in significantly reducing the size of our Docker image.

Got it. So the following changes would be made

  • add a script that can automatically install the GCC toolchain
  • remove the existing toolchain from the image
  • compile the rv32emu and relevant binaries in the intermediate images and copy it to the final image

@jserv
Copy link
Contributor

jserv commented Dec 22, 2023

Got it. So the following changes would be made

  • add a script that can automatically install the GCC toolchain
  • remove the existing toolchain from the image
  • compile the rv32emu and relevant binaries in the intermediate images and copy it to the final image

Take run #7295920920, for example. It takes over 11 minutes, yet the actual time spent preparing the build environment and launching the build process accounts for only half of this elapsed time.

Copy link
Contributor

@jserv jserv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ensure git is installed as a prerequisite; otherwise, you will encounter the following error.

 > [linux/amd64 base_gcc 5/6] RUN make:
0.108 Makefile:86: No sdl2-config in $PATH. Check SDL2 installation in advance
0.110 /bin/sh: 1: pkg-config: not found
0.180 Check the file build/.config for configured items.
0.224   CC	build/map.o
0.299   CC	build/utils.o
0.342 git submodule update --init src/softfloat/
0.351 make: git: No such file or directory
0.352 make: *** [Makefile:73: src/softfloat/build/Linux-RISCV-GCC/Makefile] Error 127

@henrybear327 henrybear327 force-pushed the ci/reduce_image_size branch 4 times, most recently from 6512e88 to 6f02c90 Compare January 1, 2024 13:50
@henrybear327
Copy link
Collaborator Author

Added libsdl2-dev libsdl2-mixer-dev to the intermediate image, so we can compile a complete rv32emu binary.

Should libsdl2-dev libsdl2-mixer-dev also be presented in the final image?

Also, since this dockerfile is not for development anymore (unless I misunderstand the intention of removal of the GCC toolchain), should we just copy the elf files in the build folders, and leave other source files, etc. out?

@henrybear327
Copy link
Collaborator Author

CI / host-arm64

It's now the same compile time as CI / host-arm64 Github CI job, since it's the QEMU emulating arm64 that's slow.

Image size is now basically 65 MB + size of directories copied in from rv32emu repo

@jserv
Copy link
Contributor

jserv commented Jan 1, 2024

Should libsdl2-dev and libsdl2-mixer-dev also be presented in the final image?

No, the Docker image should be limited to only the essential components of rv32emu, excluding SDL display and audio, as these can lead to issues in containerized environments. Therefore, just remove the libsdl2-dev and libsdl2-mixer-dev packages.

should we just copy the elf files in the build directory, and leave other source files, etc. out?

Yes, exactly. Users can always view the source code through the GitHub website. Only the built ELF files (for both the host and the RISC-V target) are preserved.

@jserv
Copy link
Contributor

jserv commented Jan 1, 2024

The recent CI record indicates that building and publishing a Docker image to Docker Hub requires 8 minutes. It looks pretty well since we can lower the elapsed time by installing less packages.

Copy link
Contributor

@jserv jserv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Drop unnecessary packages such as SDL2.

The dockerfile contains the rv32emu executable and reference emulator, 
instead of a full-fledged container for development.
@henrybear327 henrybear327 requested a review from jserv January 1, 2024 18:19
@henrybear327
Copy link
Collaborator Author

Updated as requested/discussed.

@jserv jserv merged commit 821155f into sysprog21:master Jan 1, 2024
@jserv
Copy link
Contributor

jserv commented Jan 1, 2024

Thank @henrybear327 for contributing!

@henrybear327 henrybear327 deleted the ci/reduce_image_size branch January 2, 2024 10:26
vestata pushed a commit to vestata/rv32emu that referenced this pull request Jan 24, 2025
The Dockerfile contains the executable files and reference emulator, 
instead of a full-fledged container for development.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants