docker: introduce more layers for smaller images

[yanns]

Expected behaviour

sbt docker:stage could generate a Dockerfile with multiple layers:

• one for the OS with JRE
• one for the dependencies
• one for the resources
• one for the code

The idea is to generate layers that do not change frequently and to re-use them.
For example, if we change just a line of code, only the last layer has to be pushed.

Actual behaviour

sbt docker:stage generate a Dockerfile with multiple layers:

• one for the OS with JRE
• one for all the application needs (dependencies, resources, code)

Information

https://phauer.com/2019/no-fat-jar-in-docker-image/

[nigredo-tori]

Actually dragging that information all the way from JavaAppPackaging looks ugly, and will probably break a lot... As an alternative, we can add something like

val dockerGroupFiles = settingKey[String => Int](
  "Order file mappings. Lower order means the file would be a part of an earlier layer"
)
// ...
// Flat by default.
dockerGroupFiles := Function.const(0)

And then in the user code (if required):

dockerGroupFiles := {
  case m if m.startsWith("lib/my.organization") => 1
  case _ => 0
}

and group the files by that. This is far easier to add and support, and more flexible.

• one for the resources
• one for the code

That would be even harder to add than the rest, since we wouldn't be able to just packageBin things... With the approach I have outlined above, this would be achievable by factoring out the resources to a separate subproject (so that they are in a different JAR), and then tweaking dockerGroupFiles as needed.

Disclaimer: I don't actually use DockerPlugin, and I probably won't be the one to implement that.

[CremboC]

I was actually going to open this exact same issue when I saw that blog post on Reddit.

I am going to attempt @nigredo-tori 's solution tomorrow. Only caveats that I have almost zero experience with sbt plugins and/or docker. So this is going to be interesting.

[muuki88]

Related issues and PRS

• FIX #365 ADD command per mapping entry to enable docker caching #368
• Archetype for java library (JavaLibPackaging?) #723

[muuki88]

Thanks @nigredo-tori for your input. I had similar intentions (#368) and this is the most flexible and less sbt magic heavy option I came up with as well.

Thanks @CremboC for tackeling this 😎 You will make a lot of people very happy.

[ppiotrow]

Hi, I'm playing with implementing this feature. Could you clarify if this plugin enables to build applications FROM windows images like openjdk:<version>-windowsservercore?
The hardest part for me is inability to use rm command that is not present in Windows I suppose.
The cause is poor support of the Dockerfile support for negations in COPY command.

[sideeffffect]

for the impatient ones, there's already a plugin sbt-jib, which, as the name suggests, uses the jib core library to create images with multiple layers for better use of caching

[muuki88]

@ppiotrow thanks for tackling this issue 🤗

There has already been a nice amount of work in #1268 . The rm command shouldn't be necessary? What are you trying to accomplish?

[ppiotrow]

Hi, I'm aware of #1268 but wanted to implement it from the very beginning (for fun&learn). I'm already building layered images in my daily work and wanted to migrate convention that differs from suggested above. I have code ready just need some improvements to be presented in PR today. I think having more than one option to choose is good for codebase, right?
rm no longer necessary, found nicer way of doing what I need.

[muuki88]

Sorry for the late reply 😃

but wanted to implement it from the very beginning (for fun&learn)

those are the best reasons 👍

I'm already building layered images in my daily work and wanted to migrate convention that differs from suggested above

Always happy to have different approaches. I'm not using docker for production at all. So a lot to learn for me as well 😄

I think having more than one option to choose is good for codebase, right?

Definitely 👍 I just wanted to point out all the various efforts as this has been a long standing issue with multiple attempts to fix and it's a bit messy 😂