Use local build artifacts for docker build #268
Conversation
dimas-b
requested review from
jbonofre,
ashvina,
RussellSpitzer,
snazy,
vvcephei,
takidau and
jackye1995
as code owners
dimas-b
force-pushed
the
docker-gradle
branch
from
3c943cc to
6e2d0ac
Compare
|
@collado-mike WDYT? |
The docker build will reuse regular gradle artifacts now and rely on gradle to keep them up-to-date. The Dockerfile is simplified to remove the `build` layers. This saves downloads and compile time in local builds (at least). Details: * Add `gradle` sub-project for docker-related stuff * Use `org.jetbrains.gradle.docker` plugin to push gradle artifacts into docker build. * Add `scripts` configuration to `:polaris-service` for reusing scripts across modules.
dimas-b
force-pushed
the
docker-gradle
branch
from
6e2d0ac to
5d7859f
Compare
|
rebased, fixed conflicts, added java 21 to regtest CI |
| - name: fix permissions | ||
| run: mkdir -p regtests/output && chmod 777 regtests/output && chmod 777 regtests/t_*/ref/* | ||
| - name: Regression Test | ||
| env: | ||
| AWS_ACCESS_KEY_ID: ${{secrets.AWS_ACCESS_KEY_ID}} | ||
| AWS_SECRET_ACCESS_KEY: ${{secrets.AWS_SECRET_ACCESS_KEY}} | ||
| run: docker compose up --build --exit-code-from regtest | ||
| run: | | ||
| ./gradlew dockerBuild --info |
There was a problem hiding this comment.
Nit:
| ./gradlew dockerBuild --info | |
| ./gradlew dockerBuild --stacktrace |
| @@ -22,3 +22,4 @@ polaris-core=polaris-core | |||
| polaris-service=polaris-service | |||
| polaris-eclipselink=extension/persistence/eclipselink | |||
| aggregated-license-report=aggregated-license-report | |||
| polaris-docker=docker | |||
There was a problem hiding this comment.
Hm - can you move docker/ to somewhere else or use a nested folder structure?
I could imagine that there'll be more docker related things.
| into(project.layout.buildDirectory.dir("docker-dist")) | ||
| from(startScripts) { into("bin") } | ||
| from(shadowJar) { into("lib") } | ||
| doFirst { delete(project.layout.buildDirectory.dir("regtest-dist")) } |
There was a problem hiding this comment.
| doFirst { delete(project.layout.buildDirectory.dir("regtest-dist")) } | |
| doFirst { delete(project.layout.buildDirectory.dir("docker-dist")) } |
However, the delete shouldn't be necessary, singe it's a Sync task. Can you verify this and if that's true remove this doFirst?
There was a problem hiding this comment.
Will do (although in this PR, the change was only formatting :) )
I really prefer to have deterministic builds. Admittedly, always downloading the maven artifacts is a pain (and is non-deterministic if we don't lock the dependency versions), but I would vote for a solution that leaned on docker caching rather than just using local artifacts. The primary benefit to using docker for builds is to guarantee that the build works the same across all developers' machines. This strictly bypasses that benefit. |
In CI (GH actions) the difference is between building java code on the worker node vs. inside docker, running on the worker node. Do you mean that GH actions are not deterministic enough for java builds?
When developers run Do developers rely on the docker image (as opposed to |
I always run the e2e tests by running the docker-compose script in order to avoid any surprises when it runs in CI. For day-to-day tasks, of course, I rely on unit/integration tests in my IDE, but I think it's good practice to run the build in the docker image before pushing my branch to the public repo. |
The docker build will reuse regular gradle artifacts now and rely on gradle to keep them up-to-date.
The Dockerfile is simplified to remove the
buildlayers.This saves downloads and compile time in local builds (at least).
Details:
Add
gradlesub-project for docker-related stuffUse
org.jetbrains.gradle.dockerplugin to push gradle artifacts into docker build.Add
scriptsconfiguration to:polaris-servicefor reusing scripts across modules.