fix: build stream dev images with race #2159

Matovidlo · 2024-11-27T07:36:21Z

Changes:

Introduce build with -race.
Introduce new dockerfile inside stream service, that is build with -race.
Adjust CI to include build of both containers and push it into docker registries for potential use
Update local development environment that we use by default images build with -race to get rid of unnecessary defects

…ace other without

Generate 2 images instead of one, push them into container registry. Update local deployment of minikube to run always with race detector

provisioning/dev/docker/Dockerfile

provisioning/stream/docker/service/Dockerfile

Add comment for CGO_ENABLED=1 related libraries. Update it according to previous apk add statement

fix: Simplify Dockerfiles

github-actions · 2024-12-03T08:52:03Z

Apps Proxy Kubernetes Diff [CI]

Between base 0908146 ⬅️ head 9257583.

Expand

--- /tmp/artifacts/test-k8s-state.old.json.processed.kv	2024-12-03 08:51:58.433144125 +0000
+++ /tmp/artifacts/test-k8s-state.new.json.processed.kv	2024-12-03 08:51:58.569145102 +0000
@@ -111 +111 @@
-<Deployment/apps-proxy>.spec.template.spec.containers[0].image = "docker.io/keboola/apps-proxy:ec4ca58-1733215352";
+<Deployment/apps-proxy>.spec.template.spec.containers[0].image = "docker.io/keboola/apps-proxy:9257583-1733215621";
@@ -485,3 +485,3 @@
-<Pod/apps-proxy-<hash>>.spec.containers[0].image = "docker.io/keboola/apps-proxy:ec4ca58-1733215352";
-<Pod/apps-proxy-<hash>>.spec.containers[0].image = "docker.io/keboola/apps-proxy:ec4ca58-1733215352";
-<Pod/apps-proxy-<hash>>.spec.containers[0].image = "docker.io/keboola/apps-proxy:ec4ca58-1733215352";
+<Pod/apps-proxy-<hash>>.spec.containers[0].image = "docker.io/keboola/apps-proxy:9257583-1733215621";
+<Pod/apps-proxy-<hash>>.spec.containers[0].image = "docker.io/keboola/apps-proxy:9257583-1733215621";
+<Pod/apps-proxy-<hash>>.spec.containers[0].image = "docker.io/keboola/apps-proxy:9257583-1733215621";
@@ -909 +909 @@
-<ReplicaSet/apps-proxy-<hash>>.spec.template.spec.containers[0].image = "docker.io/keboola/apps-proxy:ec4ca58-1733215352";
+<ReplicaSet/apps-proxy-<hash>>.spec.template.spec.containers[0].image = "docker.io/keboola/apps-proxy:9257583-1733215621";


(see artifacts in the Github Action for more information)

github-actions · 2024-12-03T08:54:05Z

Templates API Kubernetes Diff [CI]

Between base 0908146 ⬅️ head 9257583.

Expand

--- /tmp/artifacts/test-k8s-state.old.json.processed.kv	2024-12-03 08:54:01.246110664 +0000
+++ /tmp/artifacts/test-k8s-state.new.json.processed.kv	2024-12-03 08:54:01.462112007 +0000
@@ -194 +194 @@
-<Deployment/templates-api>.spec.template.spec.containers[0].image = "docker.io/keboola/templates-api:ec4ca58-1733215350";
+<Deployment/templates-api>.spec.template.spec.containers[0].image = "docker.io/keboola/templates-api:9257583-1733215669";
@@ -747,3 +747,3 @@
-<Pod/templates-api-<hash>>.spec.containers[0].image = "docker.io/keboola/templates-api:ec4ca58-1733215350";
-<Pod/templates-api-<hash>>.spec.containers[0].image = "docker.io/keboola/templates-api:ec4ca58-1733215350";
-<Pod/templates-api-<hash>>.spec.containers[0].image = "docker.io/keboola/templates-api:ec4ca58-1733215350";
+<Pod/templates-api-<hash>>.spec.containers[0].image = "docker.io/keboola/templates-api:9257583-1733215669";
+<Pod/templates-api-<hash>>.spec.containers[0].image = "docker.io/keboola/templates-api:9257583-1733215669";
+<Pod/templates-api-<hash>>.spec.containers[0].image = "docker.io/keboola/templates-api:9257583-1733215669";
@@ -1417 +1417 @@
-<ReplicaSet/templates-api-<hash>>.spec.template.spec.containers[0].image = "docker.io/keboola/templates-api:ec4ca58-1733215350";
+<ReplicaSet/templates-api-<hash>>.spec.template.spec.containers[0].image = "docker.io/keboola/templates-api:9257583-1733215669";
@@ -1488,0 +1489,12 @@
+<Secret/sh.helm.release.v1.templates-api-etcd.v2> = {};
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.apiVersion = "v1";
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.data = {};
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.kind = "Secret";
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.metadata = {};
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.metadata.labels = {};
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.metadata.labels.name = "templates-api-etcd";
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.metadata.labels.owner = "helm";
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.metadata.labels.version = "2";
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.metadata.name = "sh.helm.release.v1.templates-api-etcd.v2";
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.metadata.namespace = "templates-api";
+<Secret/sh.helm.release.v1.templates-api-etcd.v2>.type = "helm.sh/release.v1";


(see artifacts in the Github Action for more information)

github-actions · 2024-12-03T09:02:27Z

Stream Kubernetes Diff [CI]

Between base 0908146 ⬅️ head 9257583.

Expand

--- /tmp/artifacts/test-k8s-state.old.json.processed.kv	2024-12-03 09:02:21.848150724 +0000
+++ /tmp/artifacts/test-k8s-state.new.json.processed.kv	2024-12-03 09:02:22.336153682 +0000
@@ -200 +200 @@
-<Deployment/stream-api>.spec.template.spec.containers[0].image = "docker.io/keboola/stream-api:ec4ca58-1733215425";
+<Deployment/stream-api>.spec.template.spec.containers[0].image = "docker.io/keboola/stream-api:9257583-1733215944";
@@ -364 +364 @@
-<Deployment/stream-http-source>.spec.template.spec.containers[0].image = "docker.io/keboola/stream-api:ec4ca58-1733215425";
+<Deployment/stream-http-source>.spec.template.spec.containers[0].image = "docker.io/keboola/stream-api:9257583-1733215944";
@@ -525 +525 @@
-<Deployment/stream-storage-coordinator>.spec.template.spec.containers[0].image = "docker.io/keboola/stream-api:ec4ca58-1733215425";
+<Deployment/stream-storage-coordinator>.spec.template.spec.containers[0].image = "docker.io/keboola/stream-api:9257583-1733215944";
@@ -609 +609 @@
-<Endpoints/stream-etcd-headless>.subsets[0].addresses[1].hostname = "stream-etcd-0";
+<Endpoints/stream-etcd-headless>.subsets[0].addresses[1].hostname = "stream-etcd-1";
@@ -613 +613 @@
-<Endpoints/stream-etcd-headless>.subsets[0].addresses[1].targetRef.name = "stream-etcd-0";
+<Endpoints/stream-etcd-headless>.subsets[0].addresses[1].targetRef.name = "stream-etcd-1";
@@ -616 +616 @@
-<Endpoints/stream-etcd-headless>.subsets[0].addresses[2].hostname = "stream-etcd-1";
+<Endpoints/stream-etcd-headless>.subsets[0].addresses[2].hostname = "stream-etcd-0";
@@ -620 +620 @@
-<Endpoints/stream-etcd-headless>.subsets[0].addresses[2].targetRef.name = "stream-etcd-1";
+<Endpoints/stream-etcd-headless>.subsets[0].addresses[2].targetRef.name = "stream-etcd-0";
@@ -659 +659 @@
-<Endpoints/stream-etcd>.subsets[0].addresses[1].targetRef.name = "stream-etcd-0";
+<Endpoints/stream-etcd>.subsets[0].addresses[1].targetRef.name = "stream-etcd-1";
@@ -665 +665 @@
-<Endpoints/stream-etcd>.subsets[0].addresses[2].targetRef.name = "stream-etcd-1";
+<Endpoints/stream-etcd>.subsets[0].addresses[2].targetRef.name = "stream-etcd-0";
@@ -717 +717 @@
-<Endpoints/stream-storage-writer-reader>.subsets[0].addresses[0].hostname = "stream-storage-writer-reader-0";
+<Endpoints/stream-storage-writer-reader>.subsets[0].addresses[0].hostname = "stream-storage-writer-reader-1";
@@ -721 +721 @@
-<Endpoints/stream-storage-writer-reader>.subsets[0].addresses[0].targetRef.name = "stream-storage-writer-reader-0";
+<Endpoints/stream-storage-writer-reader>.subsets[0].addresses[0].targetRef.name = "stream-storage-writer-reader-1";
@@ -724 +724 @@
-<Endpoints/stream-storage-writer-reader>.subsets[0].addresses[1].hostname = "stream-storage-writer-reader-1";
+<Endpoints/stream-storage-writer-reader>.subsets[0].addresses[1].hostname = "stream-storage-writer-reader-0";
@@ -728 +728 @@
-<Endpoints/stream-storage-writer-reader>.subsets[0].addresses[1].targetRef.name = "stream-storage-writer-reader-1";
+<Endpoints/stream-storage-writer-reader>.subsets[0].addresses[1].targetRef.name = "stream-storage-writer-reader-0";
@@ -1214,2 +1214,2 @@
-<Pod/stream-api-<hash>>.spec.containers[0].image = "docker.io/keboola/stream-api:ec4ca58-1733215425";
-<Pod/stream-api-<hash>>.spec.containers[0].image = "docker.io/keboola/stream-api:ec4ca58-1733215425";
+<Pod/stream-api-<hash>>.spec.containers[0].image = "docker.io/keboola/stream-api:9257583-1733215944";
+<Pod/stream-api-<hash>>.spec.containers[0].image = "docker.io/keboola/stream-api:9257583-1733215944";
@@ -1534 +1534 @@
-<Pod/stream-etcd-0>.spec.containers[0].env[21].value = "new";
+<Pod/stream-etcd-0>.spec.containers[0].env[21].value = "existing";
@@ -1780 +1780 @@
-<Pod/stream-etcd-1>.spec.containers[0].env[21].value = "new";
+<Pod/stream-etcd-1>.spec.containers[0].env[21].value = "existing";
@@ -2026 +2026 @@
-<Pod/stream-etcd-2>.spec.containers[0].env[21].value = "new";
+<Pod/stream-etcd-2>.spec.containers[0].env[21].value = "existing";
@@ -2350,2 +2350,2 @@
-<Pod/stream-http-source-<hash>>.spec.containers[0].image = "docker.io/keboola/stream-api:ec4ca58-1733215425";
-<Pod/stream-http-source-<hash>>.spec.containers[0].image = "docker.io/keboola/stream-api:ec4ca58-1733215425";
+<Pod/stream-http-source-<hash>>.spec.containers[0].image = "docker.io/keboola/stream-api:9257583-1733215944";
+<Pod/stream-http-source-<hash>>.spec.containers[0].image = "docker.io/keboola/stream-api:9257583-1733215944";
@@ -2742,2 +2742,2 @@
-<Pod/stream-storage-coordinator-<hash>>.spec.containers[0].image = "docker.io/keboola/stream-api:ec4ca58-1733215425";
-<Pod/stream-storage-coordinator-<hash>>.spec.containers[0].image = "docker.io/keboola/stream-api:ec4ca58-1733215425";
+<Pod/stream-storage-coordinator-<hash>>.spec.containers[0].image = "docker.io/keboola/stream-api:9257583-1733215944";
+<Pod/stream-storage-coordinator-<hash>>.spec.containers[0].image = "docker.io/keboola/stream-api:9257583-1733215944";
@@ -2978 +2978 @@
-<Pod/stream-storage-writer-reader-0>.spec.containers[0].image = "docker.io/keboola/stream-api:ec4ca58-1733215425";
+<Pod/stream-storage-writer-reader-0>.spec.containers[0].image = "docker.io/keboola/stream-api:9257583-1733215944";
@@ -3061 +3061 @@
-<Pod/stream-storage-writer-reader-0>.spec.containers[1].image = "docker.io/keboola/stream-api:ec4ca58-1733215425";
+<Pod/stream-storage-writer-reader-0>.spec.containers[1].image = "docker.io/keboola/stream-api:9257583-1733215944";
@@ -3231 +3231 @@
-<Pod/stream-storage-writer-reader-1>.spec.containers[0].image = "docker.io/keboola/stream-api:ec4ca58-1733215425";
+<Pod/stream-storage-writer-reader-1>.spec.containers[0].image = "docker.io/keboola/stream-api:9257583-1733215944";
@@ -3314 +3314 @@
-<Pod/stream-storage-writer-reader-1>.spec.containers[1].image = "docker.io/keboola/stream-api:ec4ca58-1733215425";
+<Pod/stream-storage-writer-reader-1>.spec.containers[1].image = "docker.io/keboola/stream-api:9257583-1733215944";
@@ -3552 +3552 @@
-<ReplicaSet/stream-api-<hash>>.spec.template.spec.containers[0].image = "docker.io/keboola/stream-api:ec4ca58-1733215425";
+<ReplicaSet/stream-api-<hash>>.spec.template.spec.containers[0].image = "docker.io/keboola/stream-api:9257583-1733215944";
@@ -3723 +3723 @@
-<ReplicaSet/stream-http-source-<hash>>.spec.template.spec.containers[0].image = "docker.io/keboola/stream-api:ec4ca58-1733215425";
+<ReplicaSet/stream-http-source-<hash>>.spec.template.spec.containers[0].image = "docker.io/keboola/stream-api:9257583-1733215944";
@@ -3891 +3891 @@
-<ReplicaSet/stream-storage-coordinator-<hash>>.spec.template.spec.containers[0].image = "docker.io/keboola/stream-api:ec4ca58-1733215425";
+<ReplicaSet/stream-storage-coordinator-<hash>>.spec.template.spec.containers[0].image = "docker.io/keboola/stream-api:9257583-1733215944";
@@ -3932,0 +3933,12 @@
+<Secret/sh.helm.release.v1.stream-etcd.v2> = {};
+<Secret/sh.helm.release.v1.stream-etcd.v2>.apiVersion = "v1";
+<Secret/sh.helm.release.v1.stream-etcd.v2>.data = {};
+<Secret/sh.helm.release.v1.stream-etcd.v2>.kind = "Secret";
+<Secret/sh.helm.release.v1.stream-etcd.v2>.metadata = {};
+<Secret/sh.helm.release.v1.stream-etcd.v2>.metadata.labels = {};
+<Secret/sh.helm.release.v1.stream-etcd.v2>.metadata.labels.name = "stream-etcd";
+<Secret/sh.helm.release.v1.stream-etcd.v2>.metadata.labels.owner = "helm";
+<Secret/sh.helm.release.v1.stream-etcd.v2>.metadata.labels.version = "2";
+<Secret/sh.helm.release.v1.stream-etcd.v2>.metadata.name = "sh.helm.release.v1.stream-etcd.v2";
+<Secret/sh.helm.release.v1.stream-etcd.v2>.metadata.namespace = "stream";
+<Secret/sh.helm.release.v1.stream-etcd.v2>.type = "helm.sh/release.v1";
@@ -4228 +4240 @@
-<StatefulSet/stream-etcd>.spec.template.spec.containers[0].env[21].value = "new";
+<StatefulSet/stream-etcd>.spec.template.spec.containers[0].env[21].value = "existing";
@@ -4478 +4490 @@
-<StatefulSet/stream-storage-writer-reader>.spec.template.spec.containers[0].image = "docker.io/keboola/stream-api:ec4ca58-1733215425";
+<StatefulSet/stream-storage-writer-reader>.spec.template.spec.containers[0].image = "docker.io/keboola/stream-api:9257583-1733215944";
@@ -4558 +4570 @@
-<StatefulSet/stream-storage-writer-reader>.spec.template.spec.containers[1].image = "docker.io/keboola/stream-api:ec4ca58-1733215425";
+<StatefulSet/stream-storage-writer-reader>.spec.template.spec.containers[1].image = "docker.io/keboola/stream-api:9257583-1733215944";


(see artifacts in the Github Action for more information)

hosekpeter

LGTM

Matovidlo changed the title ~~Fix build dev images with race~~ fix: build dev images with race Nov 27, 2024

Matovidlo force-pushed the fix-build-dev-images-with-race branch from d96422a to 679c171 Compare November 28, 2024 10:31

Matovidlo changed the title ~~fix: build dev images with race~~ fix: build stream dev images with race Nov 28, 2024

fix: Build images with race. WIP create 2 types of images, one with r…

6488153

…ace other without

Matovidlo force-pushed the fix-build-dev-images-with-race branch from 679c171 to 6488153 Compare November 28, 2024 11:46

feat: Add new dockerfile to build the image with -race

c07461d

Generate 2 images instead of one, push them into container registry. Update local deployment of minikube to run always with race detector

Matovidlo force-pushed the fix-build-dev-images-with-race branch from e97e03d to c07461d Compare November 28, 2024 12:51

Matovidlo marked this pull request as ready for review November 28, 2024 13:53

Matovidlo requested review from jachym-tousek-keboola and hosekpeter as code owners November 28, 2024 13:53

jachym-tousek-keboola requested changes Dec 2, 2024

View reviewed changes

provisioning/dev/docker/Dockerfile Outdated Show resolved Hide resolved

provisioning/stream/docker/service/Dockerfile Outdated Show resolved Hide resolved

fix: Remove not used library libsqlite3-0.

91d1e31

Add comment for CGO_ENABLED=1 related libraries. Update it according to previous apk add statement

Matovidlo requested a review from jachym-tousek-keboola December 2, 2024 12:24

jachym-tousek-keboola and others added 2 commits December 2, 2024 14:42

fix: Simplify Dockerfiles

a9578d9

Merge pull request #2165 from keboola/jt-dev-docker

9257583

fix: Simplify Dockerfiles

jachym-tousek-keboola approved these changes Dec 3, 2024

View reviewed changes

hosekpeter approved these changes Dec 3, 2024

View reviewed changes

jachym-tousek-keboola merged commit 06cd5af into main Dec 3, 2024
15 checks passed

jachym-tousek-keboola deleted the fix-build-dev-images-with-race branch December 3, 2024 11:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: build stream dev images with race #2159

fix: build stream dev images with race #2159

Matovidlo commented Nov 27, 2024 •

edited by jira bot

Loading

github-actions bot commented Dec 3, 2024

github-actions bot commented Dec 3, 2024

github-actions bot commented Dec 3, 2024

hosekpeter left a comment

fix: build stream dev images with race #2159

fix: build stream dev images with race #2159

Conversation

Matovidlo commented Nov 27, 2024 • edited by jira bot Loading

github-actions bot commented Dec 3, 2024

Apps Proxy Kubernetes Diff [CI]

github-actions bot commented Dec 3, 2024

Templates API Kubernetes Diff [CI]

github-actions bot commented Dec 3, 2024

Stream Kubernetes Diff [CI]

hosekpeter left a comment

Choose a reason for hiding this comment

Matovidlo commented Nov 27, 2024 •

edited by jira bot

Loading