Refactor Dockerfiles

[kazk]

Dockerfiles need to be refactored to reduce image sizes, follow best practices and ensure reproducibility.

Example

codewars/swift-runner

Problems

• Download size is 2.7 GB 😱
• Swift version not specified 😱

Possible Solutions

• Download (L33), build (L45) and remove (L49) in the same layer.

  # L33
  RUN git clone https://github.com/apple/swift.git && ./swift/utils/update-checkout --clone
  # 783.3 MB
  
  # L45
  RUN utils/build-script --preset-file=codewars-build-presets.ini --preset=codewars_linux_1404 install_destdir=$SWIFT_GEN_DIR installable_package=$SWIFT_GEN_DIR/swift.tar.gz
  # 944.1 MB
  
  # L49
  RUN rm -rf $SWIFT_SOURCE_DIR/*

• Find a better way to get the source at specific version. Cloning the entire history shouldn't be necessary.

  • 3.0-dev(Language Support Status)
  • swift-3.0-RELEASE (2016-09-13)
  • swift-3.0.1-RELEASE (2016-10-28)
  • swift-3.0.2-RELEASE (2016-12-13)

Resources

• Best practices for writing Dockerfiles
• Refactoring a Dockerfile for Image Size

[kazk]

@jhoffner, do you have a server for building Docker images?

I've looked into this and I think we can reduce the image size greatly by:

1. Adding --no-install-recommends for apt-get install
2. Using tagged releases for Swift source (or git clone --depth 1 if version is not important)
3. Use --skip-history for swift/utils/update-checkout script
4. Download, build, clean in the same layer

I haven't tried because building this will (most likely) make my old laptop unusable.

[jhoffner]

I'm all for reducing the size of this image. At the time it was added (OS contribution) we had tried a few things to reduce the size but didn't try moving those 3 lines into one command. This huge image actually caused a domino effect of devops issues for me to deal with.

[kazk]

Removing in the same layer should definitely help with the final image size.

I was hoping to reduce the amount of download during the build, but it's much more complicated than I thought. swift/utils/update-checkout --clone clones 13 repositories and some of them have submodules. I have no idea how to properly manage versions of these.

[kazk]

@jhoffner
How long does it take to build current swift-runner?

kazk/codewars-runner-cli@9fe17ff should reduce the image size and allow locking Swift version. I'm making Travis build them, but it takes more than 50 minutes and get timeout errors.
I was able to get to [1655/2256] by using pre-compiled CMake and tweaking build options, but I'm out of ideas.

[kazk]

Closing this because the biggest issue is resolved. I'll open separate issues for other Dockerfiles.