CI: Manage separate docker image for our own compiler

[ynfle]

The CI takes over 5 minutes to build because we rebuild the compiler for each docker build.
We do this, so we can have a significantly smaller image which hopefully means faster throughput.

Would it help to maintain a sperate docker image that we use as our base to be able to build on top of? Would that rebuild the compiler each time? (I don't think so, but I'm curious to hear what everyone else thinks)

@iHiD would we be able to have a separate repo for that? We could use it for all the nim tooling repos

[iHiD]

Yep - what should we name it nim-compiler or something?

[ynfle]

I'm fine with anything.

@ErikSchierboom Do you know if it will speed up the build process?

@ee7 Any thoughts?

[ErikSchierboom]

@ynfle It would, but the main question is: why do we even build the compiler ourselves? There are official Nim Docker images that can be used. These images also include Alpine versions.

[ynfle]

@ynfle It would, but the main question is: why do we even build the compiler ourselves? There are official Nim Docker images that can be used. These images also include Alpine versions.

It was introduced here #64

[ErikSchierboom]

It lists the following benefits:

Reduced attack surface
Reduced time to download and extract the parent image
Reduced time to transfer our built image from/to the container
registry
We can update Nim even if https://hub.docker.com/r/nimlang/nim wasn't
updated yet
We can update Alpine even if there hasn't been a new Nim version

These are all true, although IMHO I'd still go with the official Docker images, but 🤷 I noticed that there are three different Nim Docker images: regular, slim and onbuild. It might be worth investigating how much of a size difference there is with the slim version (and if it still works).

[ee7]

It might be worth investigating how much of a size difference there is with the slim version (and if it still works).

This repo was previously using the Alpine slim flavor. The slim image is still about 250 MB, because the official images have to support things like compiling Nim to C++ and JS (and supporting nim js -d:nodejs -r foo.nim means that Node.js is in the image).

Our image is about 10x smaller.

Image size with this commit: 26.74 MB

This (uncompressed) size is getting pretty close to the theoretical
minimum for a Nim image, given that we use approximately:

• 5 MB for the Alpine parent image
• 5 MB for the Nim compiler
• 5 MB for the Nim standard library
• 10 MB for musl libc
• 1 MB in total for PCRE and tcc

Part of the point was to try tcc as our C compiler until Nim gets incremental compilation (at the time, it was thought that it would be ready in Nim 1.6).

Please see these commit messages:

• fc024ce
• 282e388
• 19b656d

This image is nice and minimal, so I'd like to keep it if the only downsides are as stated.

To speed up CI, maintaining our own Docker image would work. nim-compiler is fine as a name. I have no opinion on whether nim-alpine-musl-tcc is better - it depends if we want to distinguish a possible future image that's non-Alpine (in particular, using https://github.com/GoogleContainerTools/distroless could be nice), non-musl, and/or non-tcc.

Is it sufficient/easier to cache the Docker layers properly instead? I didn't look into it, but this was my original idea for speeding up CI. Currently I think we do cache, but we only hit it when we already built with that commit ref:

nim-test-runner/.github/workflows/docker.yml

Lines 22 to 28 in 308e296

	- name: Cache Docker layers
	uses: actions/cache@937d24475381cd9c75ae6db12cb4e79714b926ed # 2.1.7
	with:
	path: /tmp/.buildx-cache
	key: ${{ runner.os }}-buildx-${{ github.sha }}
	restore-keys: |
	${{ runner.os }}-buildx-

See also these docs:

• https://github.com/docker/setup-buildx-action
• https://github.com/moby/buildkit#export-cache

Note that actions/cache deletes data after a week of no access:

Caches that are not accessed within the last week will also be evicted.

So if using GitHub's caching and this approach, you'd have to deal with a 5 minute build if there's a week of inactivity (or add a job that does nothing but access the cache every 6 days).

[iHiD]

I propose nim-docker-base as the repo name: https://github.com/exercism/nim-docker-base

[ErikSchierboom]

Thanks for explaining @ee7. As base image might be best then.

[ynfle]

Thanks I'll have a look

[ynfle]

@ee7 Can't we just "steal" the nim compiler from nimlang/nim:alpine instead of compiling it? As far as I can tell, there isn't anything special in the way we compile.

It's real pain to wait 5+ minutes every time I push to see if there is an error and what it is. I am currently not able to install docker locally to test the commands, and that long for feedback is anyways frustrating as my environment doesn't precisely match the CI's

[ErikSchierboom]

@ee7 Can't we just "steal" the nim compiler from nimlang/nim:alpine instead of compiling it? As far as I can tell, there isn't anything special in the way we compile.

Is the end rust a single binary (or directory)? If so, you could do a mult-stage build and copy things.

[ynfle]

Successful multistage build #116

Draft to "steal" nim from official nimlang alpine image #117

[ynfle]

Build and push times decreased to ~1 from 5-7.5 minutes! 🚀 💨 🚤 🐎

[ynfle]

I'm gonna close this as we have a working base image