Get rid of Docker daemon or even podman service in Maven spring-boot:build-image goal

[FredPraca]

In the case of CI/CD, we don't have a Docker daemon and don't want to turn one on in our Gitlab-runners and just want to rely on a standard Maven OCI image.
At the moment, using mvn spring-boot:build-image requires having a docker daemon running whereas Jib doesn't and run easily in our runners.
As we are also running into trouble with Jib when adding custom CA certificates to our image, Buildpacks and its ca-certificates pack is great and works perfectly on developer laptops.

Getting rid of docker would be great.
Using Buildpack lifecycle as described in this workaround can be a workaround but in this case, we don't use Maven plugin at all.

[philwebb]

The use of containers is pretty much a requirement for Buildpacks and I don't think we have any option to remove it. The builders that we invoke are themselves are published as containers. I think your only option here would be to use Jib.

[wilkinsona]

To add to what Phil's said above, the workaround is using GitLab CI's image support:

image: paketobuildpacks/builder

This results in GitLab running the build in a container created from the paketobuildpacks/builder image. That isn't possible without a container runtime such as Docker or podman.

[FredPraca]

Ok, I get it.
My reasoning was a little dumb.
If the goal can retrieve the builder and runner images without podman or docker the same way Jib does, it does not have any way to run commands inside these images to build.

[wleese]

Is this not a valid ask?

It seems that the buildpacks.io pack developers were indicating that container runtimes (e.g. Docker) are not a strict requirement in order to build containers, if you use the CNB lifecycle directly.

Like the OP:

we don't have a Docker daemon and don't want to turn one on in our Gitlab-runners

.. we too want this.

Currently we're maintaining our own Jib configuration, without benefiting from all the work Spring Boot does in/via buildpacks.

[philwebb]

@wleese AFAICT that comment is talking about using Linux or Mac primitives to remove the need for Docker or Podman. That won't remove the need for actually using containers and I don't think it will work with Windows.

If pack does implement such a feature, we could attempt to copy it but I think it's unlikely that they will.

[wleese]

@philwebb I believe there were multiple discussions going on there, but regardless, let's focus on the OP and my own ask.

Sorry I wasn't clear enough in my previous comment, let me try again.

My (and the OP's, if I read it correctly) ask revolves around restrictions of a GitLab setup, where CI pipelines can be crafted by users that use predetermined containers, but within those containers, you're unable to spin up a new one on demand for security reasons (well known host machine compromises e.g. via Docker socket).

So basically: yes I can run containers, if I put them in my predefined pipeline as a job (in a GitLab pipeline, a job == a container that does something).

I've created a test project here to show what is possible.

It demonstrates using packs builder image, by invoking the creator binary to build the application.
The creator binary will walk through all the lifecycle steps and apply the bundled buildpacks in the builder image.

This process does not involve any container spawning, apart starting the builder image, in which we execute our build. This matches the restrictions we face with GitLab, and as such could make it a viable way to continue using Spring Boots Cloud Native Buildpack support, without having to move to alternatives such as Jib.

Note that the example repo does not result in a runnable container for some reason. I believe the point about not spinning up containers was sufficiently addressed once all the buildpacks (seemingly) did their work and a container was pushed to the local Docker registry.

All that said, should Spring Boot help users do this? Probably not. At least it is hard for me to see what improvements could be provided other than documentation. And of course there's the question if what I'm doing here is a good idea or not. It's all very bespoke.

Looking a bit deeper into how Spring Boot currently implements buildpack support and the buildpack ecosystem, it's hard to think of a satisfactory solution for users with our restrictions. Spring Boot will not want to also start concerning itself with creating a special builder for the user in a non-containerized environment and keeping it up to date with pre-downloaded buildpacks (a different approach than I took btw).

Looking back at the original problem, I think that Spring Boot spawning docker images is a good match for part of the user base, but clearly not for all. More restricted environments such as mine would benefit from approaches such as Kpack (or maybe even Tekton, although its scope is much bigger), where spinning up the necessary containers to.. build a container, is performed by a Kubernetes Operator instead.

The ask for Spring Boot would be to consider these restricted environments (if they represent a big enough user base, sure) within the context of buildpacks, to see what can be improved. At the end of the day, I couldn't come up with a suggestion :)

[philwebb]

Thanks for the extra details @wleese.

The ask for Spring Boot would be to consider these restricted environments (if they represent a big enough user base, sure) within the context of buildpacks, to see what can be improved. At the end of the day, I couldn't come up with a suggestion :)

Unfortunately I don't think the overlap between Spring Boot users wanting to use the buildpack support in our plugins and those running on a restricted CI is large enough for us to invest in this area. The aim of the buildpack support was to give folks a really quick on-ramp for trying buildpacks without needing to download additional tools such as pack.

Once you hit CI restrictions as you’re describing, I think the better place to look for a solution is higher up the stack. If a solution can be provided at the buildpack level, all users can benefit. If we try and provide something in Spring Boot, we're only going to hit a subset of those users and we'll be swimming against the tide of the way that buildpacks are officially supported.

Having said that, if the buildpack team themselves start to support these features and it's easy for us to copy what they do, we can certainly consider doing so.