Smaller docker build images

[mattwynne]

Summary

Change the build system to use per-language docker images instead of one monolithic build image.

Details

Work in progress.

The idea is to alter the Make build process so that, when building a particular package / language, we briefly delegate to a language-specific build image, instead of running the entire build inside the same image.

Motivation and Context

As Cucumber grows to support more and more languages, having a single build image is difficult to scale, and becomes a dependency magnet.

The large build image is a barrier to new developers, as it takes a long time to download and may not even run on a machine with only a modest spec.

How Has This Been Tested?

It works if make works.

Types of changes

• Bug fix (non-breaking change which fixes an issue).
• New feature (non-breaking change which adds functionality).
• Breaking change (fix or feature that would cause existing functionality to not work as expected).
• Refactoring

Checklist:

• The change has been ported to Java.
• The change has been ported to Ruby.
• The change has been ported to JavaScript.
• The change has been ported to Go.
• The change has been ported to .NET.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.

[mattwynne]

@ciaranmcnulty and I paired on this today for half and hour and made a little start. We're going to try and work on /messages/javascript first as a proof of concept.

Here's what we discovered so far:

• ✨ We can use docker build --file ../../Dockerfile to use a single shared Dockerfile in the monorepo root, and still have the build context be local to the package/language folder.
• 🤔 Should we copy files or map volumes? We don't have any long-running processes (like a web server) where you'd want to change files in your editor and see the changes, but we do need to allow for people doing TDD on these packages and make sure they run specific tests and get fast feedback. We want to try both and see what the performance and experience is like.
• 🤔 The codegen target for src/messages.ts needs Ruby, so maybe the codegen step needs to be done outside of the language build, in another container.
• 🤔 That same codegen target expects to be run in the context of the monorepo (it looks for ../jsonschema/scripts/codegen.rb) which it can't see if we only copy files from the local folder. Maybe it makes more sense to map a volume of the entire monorepo?

[ciaranmcnulty]

Should we copy files or map volumes?

We'll have to map at least some volumes, so that changes made by the build process are reflected on the host (to be committed back to the repo). My first attempt would be to copy files into the containers (for speed on non-Linux, and to do optimisations like only running installers sometimes). The proof will be in the real-world performance

The codegen target for src/messages.ts needs Ruby, so maybe the codegen step needs to be done outside of the language build, in another container.

Other tasks like rsync etc. can be handled by a top-level container that spawns the other containers (which can be allowed by mounting docker.sock into the container)

[mattwynne]

We're not working on this anymore. The strategy instead is #1724