feat: Language plugins choose the runner image #3268
Conversation
stuartwdouglas
added
the
run-all
matt2e
added
the
skip-proto-breaking
matt2e
force-pushed
the
matt2e/docker
branch
5 times, most recently
from
09336f9
to
9c1bfbf
Compare
|
Previously, the language plugin protocol had a separate field to indicate the runner image. The approach has changed to using ModuleRuntime within the schema as that felt like a natural place to store it and it already part of the schema.Module that is being returned by plugins. |
matt2e
requested review from
worstell and
wesbillman
and removed request for
a team
| @@ -23,6 +23,17 @@ jobs: | |||
| name: docker-runner-artifact | |||
| path: artifacts/ftl-runner/ftl-runner.tar | |||
| retention-days: 1 | |||
| - name: Build JVM Runner | |||
There was a problem hiding this comment.
Could/should this be a matrix instead? Compute the matrix using something like ftl language list?
There was a problem hiding this comment.
I think in the future yes, but we need to have ftl-runner available before building ftl-runner-jvm.
Once theres more than one image built on top of ftl-runner (python super soon), then those can be put in a matrix.
| @@ -0,0 +1,22 @@ | |||
| # Huge hack, we want the JVM and same runtime environment as the runner | |||
There was a problem hiding this comment.
The runner image doesn't need the JVM though?
There was a problem hiding this comment.
We need the JVM to run the java module bytecode though right?
(This hack is from before this PR btw, it's just been moved out of the normal runner to the jvm runner)
There was a problem hiding this comment.
I don't know what the intermediate state is, but the end state is that the base runner image knows nothing about the JVM, Python, or any runtime, and each runtime-specific image layers on top of the runner image.
Re. this specific comment, I don't quite understand why the base runner would still contain the JVM, but maybe it's an intermediate step?
There was a problem hiding this comment.
Ahhh yes yes. It's pulling from ftl0/ftl-runner which as of this PR contains the JVM, and ftl0/ftl-runner-jvm has not yet been published.
Once this PR makes it to release I'll move it to pulling from the jvm runner.
There was a problem hiding this comment.
This was a hack to make the image build faster and to prevent the GHA runners running out of disk space.
I don't know what the intermediate state is, but the end state is that the base runner image knows nothing about the JVM, Python, or any runtime, and each runtime-specific image layers on top of the runner image.
@matt2e we should probably change the builds to use FROM instead of basically building a new base image each time. This will reduce that amount of network traffic pulling images as they will share a base layer.
# Conflicts: # backend/protos/xyz/block/ftl/v1/schema/schema.pb.go # internal/buildengine/languageplugin/external_plugin.go
# Conflicts: # backend/protos/xyz/block/ftl/v1/language/language.pb.go
# Conflicts: # jvm-runtime/plugin/common/jvmcommon.go
Co-authored-by: Alec Thomas <aat@tbd.email>
matt2e
removed
the
run-all
matt2e
force-pushed
the
matt2e/docker
branch
2 times, most recently
from
f0df19a
to
ced8165
Compare
Language plugins can now declare the runner image to use by including a ModuleRuntime in the Module schema.
Image names do not include the tag to use, instead FTL will use the same tag that the rest of FTL is using.
If a language plugin does not choose a runner image, it is defaulted to
ftl0/ftl-runner.