ci: add build and test cache into github actions cache #1739
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Matovidlo
force-pushed
the
mv-PSGO-561-adopt-simple-caching-of-tests-similar-to-local-cache-into-ci
branch
3 times, most recently
from
3ce96ae to
d7de716
Compare
Linter goes faster than tests, remove the linter cache so the CI is efficient and cache less poluted. Fix makefile target with auto verbose tag.
Matovidlo
force-pushed
the
mv-PSGO-561-adopt-simple-caching-of-tests-similar-to-local-cache-into-ci
branch
9 times, most recently
from
8cc48bc to
623a995
Compare
Matovidlo
commented
May 3, 2024
Matovidlo
force-pushed
the
mv-PSGO-561-adopt-simple-caching-of-tests-similar-to-local-cache-into-ci
branch
from
623a995 to
5aa193e
Compare
Matovidlo
commented
May 3, 2024
There was a problem hiding this comment.
Need to fix the CI to generate coverprofile on tagged commits and main branch/ release.
Caches currently takes up 4.5 GB instead of 3.9GB which is not bad result as it is approximatelly ~600MB per ~/.cache/go-build for each platform. So ~1.8GB was approximate increase in general and ends up only increased by ~600MB.
The workflow takes up to ~11-12 minutes when no code change is made, the caching looks currently alright and hit ratio works.
Matovidlo
force-pushed
the
mv-PSGO-561-adopt-simple-caching-of-tests-similar-to-local-cache-into-ci
branch
7 times, most recently
from
e61335a to
1751d01
Compare
Matovidlo
commented
May 7, 2024
Matovidlo
force-pushed
the
mv-PSGO-561-adopt-simple-caching-of-tests-similar-to-local-cache-into-ci
branch
from
ba19752 to
1693148
Compare
Matovidlo
force-pushed
the
mv-PSGO-561-adopt-simple-caching-of-tests-similar-to-local-cache-into-ci
branch
from
cb68d0a to
f882f85
Compare
Matovidlo
force-pushed
the
mv-PSGO-561-adopt-simple-caching-of-tests-similar-to-local-cache-into-ci
branch
4 times, most recently
from
804e077 to
8385b95
Compare
move installation of Go before `go mod download`. move installation of tools and modules into `go-tools-cache`.
Matovidlo
force-pushed
the
mv-PSGO-561-adopt-simple-caching-of-tests-similar-to-local-cache-into-ci
branch
from
8385b95 to
5ff3213
Compare
…he miss fix: touch takes different format in MacOS than Unix. fix path to work also on Windows
reduces time to install tools on windows.
Matovidlo
force-pushed
the
mv-PSGO-561-adopt-simple-caching-of-tests-similar-to-local-cache-into-ci
branch
from
5ff3213 to
6a237bf
Compare
Apps Proxy Kubernetes Diff [CI]Between Expand--- /tmp/artifacts/test-k8s-state.old.json.processed.kv 2024-05-07 12:01:30.822030694 +0000
+++ /tmp/artifacts/test-k8s-state.new.json.processed.kv 2024-05-07 12:01:30.946030185 +0000
@@ -105 +105 @@
-<Deployment/apps-proxy>.spec.template.spec.containers[0].image = "docker.io/keboola/apps-proxy:e967004";
+<Deployment/apps-proxy>.spec.template.spec.containers[0].image = "docker.io/keboola/apps-proxy:6a237bf";
@@ -461,3 +461,3 @@
-<Pod/apps-proxy-<hash>>.spec.containers[0].image = "docker.io/keboola/apps-proxy:e967004";
-<Pod/apps-proxy-<hash>>.spec.containers[0].image = "docker.io/keboola/apps-proxy:e967004";
-<Pod/apps-proxy-<hash>>.spec.containers[0].image = "docker.io/keboola/apps-proxy:e967004";
+<Pod/apps-proxy-<hash>>.spec.containers[0].image = "docker.io/keboola/apps-proxy:6a237bf";
+<Pod/apps-proxy-<hash>>.spec.containers[0].image = "docker.io/keboola/apps-proxy:6a237bf";
+<Pod/apps-proxy-<hash>>.spec.containers[0].image = "docker.io/keboola/apps-proxy:6a237bf";
@@ -879 +879 @@
-<ReplicaSet/apps-proxy-<hash>>.spec.template.spec.containers[0].image = "docker.io/keboola/apps-proxy:e967004";
+<ReplicaSet/apps-proxy-<hash>>.spec.template.spec.containers[0].image = "docker.io/keboola/apps-proxy:6a237bf";
(see artifacts in the Github Action for more information) |
Templates API Kubernetes Diff [CI]Between Expand--- /tmp/artifacts/test-k8s-state.old.json.processed.kv 2024-05-07 12:03:10.821410702 +0000
+++ /tmp/artifacts/test-k8s-state.new.json.processed.kv 2024-05-07 12:03:11.025411248 +0000
@@ -194 +194 @@
-<Deployment/templates-api>.spec.template.spec.containers[0].image = "docker.io/keboola/templates-api:e967004";
+<Deployment/templates-api>.spec.template.spec.containers[0].image = "docker.io/keboola/templates-api:6a237bf";
@@ -747,3 +747,3 @@
-<Pod/templates-api-<hash>>.spec.containers[0].image = "docker.io/keboola/templates-api:e967004";
-<Pod/templates-api-<hash>>.spec.containers[0].image = "docker.io/keboola/templates-api:e967004";
-<Pod/templates-api-<hash>>.spec.containers[0].image = "docker.io/keboola/templates-api:e967004";
+<Pod/templates-api-<hash>>.spec.containers[0].image = "docker.io/keboola/templates-api:6a237bf";
+<Pod/templates-api-<hash>>.spec.containers[0].image = "docker.io/keboola/templates-api:6a237bf";
+<Pod/templates-api-<hash>>.spec.containers[0].image = "docker.io/keboola/templates-api:6a237bf";
@@ -1406 +1406 @@
-<ReplicaSet/templates-api-<hash>>.spec.template.spec.containers[0].image = "docker.io/keboola/templates-api:e967004";
+<ReplicaSet/templates-api-<hash>>.spec.template.spec.containers[0].image = "docker.io/keboola/templates-api:6a237bf";
@@ -1477,0 +1478,12 @@
+<Secret/sh.helm.release.v1.templates-api-etcd.v2> = {};
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.apiVersion = "v1";
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.data = {};
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.kind = "Secret";
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.metadata = {};
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.metadata.labels = {};
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.metadata.labels.name = "templates-api-etcd";
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.metadata.labels.owner = "helm";
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.metadata.labels.version = "2";
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.metadata.name = "sh.helm.release.v1.templates-api-etcd.v2";
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.metadata.namespace = "templates-api";
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.type = "helm.sh/release.v1";
(see artifacts in the Github Action for more information) |
hosekpeter
approved these changes
May 9, 2024
Matovidlo
deleted the
mv-PSGO-561-adopt-simple-caching-of-tests-similar-to-local-cache-into-ci
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Jira: PSGO-561
Changes:
git clonepulls the files and changes mod time to current time. cmd/go: test cache hash inputs include ModTime, often causing cache misses on CI golang/go#58571-coverprofileflag that prevents caching of tests.-coverprofileflag for release pipeline. Tagged commits andmain.go toolscache to reduce size of cache per step.e2etests is already injected but not evaluated. Will be done in next PR.go mod downloadstep to cache also packages in tools cache.