-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor(containers): container builds use GCS for cache storage #14
Conversation
This makes a few changes: 1. Before the compilation, we pull down a $repo-$branch.tgz file and uncompress it (e.g. igor-master.tgz or rosco-release-1.17.x.tgz) to the /workspace volume 2. The Dockerfile.compile image is now expected to be a build step. This means we can run the container and it will compile files on the /workspace volume. Currently, we copy all the files into the container, compile them there, and then have to copy them back out. This new method is more along the lines with how GCB is supposed to work, but it does mean every Dockerfile.compile has to be changed to use CMD instead of RUN. 3. After the compile, we tar up .gradle/caches and .gradle/wrapper to $repo-$branch.tgz and send it to gs://spinnaker-build-cache. This storage bucket is set to auto-delete files that haven't been modified in 14 days. Removing our dependence on the `gradle_cache` image means that the various microservices can define their own compile image with different build tool configurations. I'd really like to have this for the Java 11 migration. It also means we could move the Debian builds into Cloud Build, since we aren't depending on them to produce that `gradle_cache` artifact.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📦 ⚒️ 💰 🎉
eacc095
to
5623804
Compare
Thinking a bit here from helping a user compile a docker image - I wonder if there is a way we can compose the Docker images in each repo in such a way that they can Just an idea, haven't explored it at all. |
* chore(build): Update Dockerfiles for new cloudbuild configs See spinnaker/buildtool#14 * chore(dockerfile): use openjdk:8 instead of rolling our own
* chore(build): Update Dockerfiles for new cloudbuild configs See spinnaker/buildtool#14 * chore(dockerfile): use openjdk:8 instead of rolling our own
See spinnaker/buildtool#14 Co-authored-by: Adam Jordens <adam@jordens.org>
* chore(build): Update Dockerfiles for new cloudbuild configs See spinnaker/buildtool#14 * chore(dockerfile): use openjdk:8 instead of rolling our own
* chore(build): Update Dockerfiles for new cloudbuild configs See spinnaker/buildtool#14 * chore(dockerfile): use openjdk:8 instead of rolling our own
* chore(build): Update Dockerfiles for new cloudbuild configs See spinnaker/buildtool#14 * chore(dockerfile): use openjdk:8 instead of rolling our own
* chore(build): Update Dockerfiles for new cloudbuild configs See spinnaker/buildtool#14 * chore(dockerfile): use openjdk:8 instead of rolling our own
* chore(build): Update Dockerfiles for new cloudbuild configs See spinnaker/buildtool#14 * chore(dockerfile): use openjdk:8 instead of rolling our own
* chore(build): Update Dockerfiles for new cloudbuild configs See spinnaker/buildtool#14 * chore(dockerfile): use openjdk:8 instead of rolling our own
See spinnaker/buildtool#14 I forgot this one.
See spinnaker/buildtool#14 I forgot this one.
…r#533) * chore(build): Update Dockerfiles for new cloudbuild configs See spinnaker/buildtool#14 * chore(dockerfile): use openjdk:8 instead of rolling our own
…r#533) * chore(build): Update Dockerfiles for new cloudbuild configs See spinnaker/buildtool#14 * chore(dockerfile): use openjdk:8 instead of rolling our own
…r#3450) * chore(build): Update Dockerfiles for new cloudbuild configs See spinnaker/buildtool#14 * chore(dockerfile): use openjdk:8 instead of rolling our own
This makes a few changes:
$repo-$branch.tgz
file anduncompress it (e.g.
igor-master.tgz
orrosco-release-1.17.x.tgz
) tothe
/workspace
volumeDockerfile.compile
image is now expected to be a build step. Thismeans we can run the container and it will compile files on the
/workspace
volume. Currently, we copy all the files into thecontainer, compile them there, and then have to copy them back out.
This new method is more along the lines with how GCB is supposed to
work, but it does mean every
Dockerfile.compile
has to bechanged to use
CMD
instead ofRUN
..gradle/caches
and.gradle/wrapper
to$repo-$branch.tgz
and send it togs://spinnaker-build-cache
. Thisstorage bucket is set to auto-delete files that haven't been
modified in 14 days.
Removing our dependence on the
gradle_cache
image means that thevarious microservices can define their own compile image with different
build tool configurations. I'd really like to have this for the Java 11
migration.
It also means we could move the Debian builds into Cloud Build, since
we aren't depending on them to produce that
gradle_cache
artifact.