Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better go pipelines #1086

Merged
merged 7 commits into from
Mar 25, 2024
Merged

Better go pipelines #1086

merged 7 commits into from
Mar 25, 2024

Conversation

xnox
Copy link
Contributor

@xnox xnox commented Mar 13, 2024

Melange Pull Request Template

Improve go pipelines:

  • default to higher microarchitecture level on amd64 to match Wolfi C/C++ -march=x86-64-v2
  • default to current microarchitecture level on arm64 to match what golang does today v8.0 (no effective change, but gives us a toggle to be able to raise it in the future)
  • strip DWARF debug information by default
  • (updated) do not strip symbols table information by default
  • trimpath in go/install pipeline by default
  • by default opt into go build tags osusergo and netgo, meaning whilst we can keep implicit default of CGO_ENABLED=1 all binaries (when possible and don't otherwise use cgo) are compiled statically linked
  • but when using go-fips toolchain, or have any other cgo features those are automatically used

This reduces go binaries size significantly, those that have not already employed the above techniques. Only about half of go packages do strip dwarf & symbols table.

Separately this reduces friction/differences when projects have optional cgo-dependent features, or when packages are rebuilt with go-fips toolchain.

Also test cases need adding.

Behavior Changes and remediation

  • "Binary cannot be executed" because VM doesn't expose x86-64-v2 CPU flags. Resolution option 1) ensure VMs expose that, note that most clouds now offer x86-64-v3 and x86-64-v4. And the rest of Wolfi has always required x86-64-v2 like any machine from this century. Resolution option 2) set amd64: v1

  • Symbols table only needed back. Resolution: set strip: "-w"

  • Symbols table & Debug symbols needed back. Resolution: set strip: ""

  • Cgo based net or osuser is desired, instead of the go version. Resolution: set toolchaintags: ""

Functional Changes

TODO:

  • This change can build all of Wolfi without errors (describe results in notes)

Notes:

SCA Changes

  • Examining several representative APKs show no regression / the desired effect (details in notes)

Notes:

Linter

  • The new check is clean across Wolfi
  • The new check is opt-in or a warning

Notes:

Example of binary diff of caddy package:

diff --git a/caddy.yaml b/caddy.yaml
index e0c214161..e57b6a5af 100644
--- a/caddy.yaml
+++ b/caddy.yaml
@@ -11,8 +11,6 @@ environment:
     packages:
       - busybox
       - ca-certificates-bundle
-  environment:
-    CGO_ENABLED: "0"
 
 pipeline:
   - uses: git-checkout
@@ -31,7 +29,6 @@ pipeline:
 
   - uses: go/build
     with:
-      ldflags: -s -w
       output: caddy
       packages: ./cmd/caddy
$ diff -u cgo-enabled.xxd cgo-disabled.xxd 
--- cgo-enabled.xxd	2024-03-12 18:10:02.853307135 +0000
+++ cgo-disabled.xxd	2024-03-12 18:10:10.453272442 +0000
@@ -248,12 +248,12 @@
 00000f70: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00000f80: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00000f90: 0000 0000 0000 0000 0000 0000 0400 0000  ................
-00000fa0: 5300 0000 0400 0000 476f 0000 342d 6d4b  S.......Go..4-mK
-00000fb0: 6641 3758 3648 6d65 7564 7547 3354 5455  fA7X6HmeuduG3TTU
-00000fc0: 2f63 3433 3150 4237 5130 336f 3151 334e  /c431PB7Q03o1Q3N
-00000fd0: 622d 7679 462f 7565 7151 6267 3042 5172  b-vyF/ueqQbg0BQr
-00000fe0: 416c 4f45 4d6a 6946 7831 2f31 3477 6f6f  AlOEMjiFx1/14woo
-00000ff0: 412d 5863 3271 424b 7763 5542 4537 5000  A-Xc2qBKwcUBE7P.
+00000fa0: 5300 0000 0400 0000 476f 0000 475f 5f77  S.......Go..G__w
+00000fb0: 6a35 4a55 4754 6f75 6c7a 6371 6853 5079  j5JUGToulzcqhSPy
+00000fc0: 2f5f 4e35 4f4e 446d 6447 5057 4168 3175  /_N5ONDmdGPWAh1u
+00000fd0: 3767 794f 502f 7565 7151 6267 3042 5172  7gyOP/ueqQbg0BQr
+00000fe0: 416c 4f45 4d6a 6946 7831 2f39 4b47 4552  AlOEMjiFx1/9KGER
+00000ff0: 5168 3141 3957 4836 7245 4765 3162 5800  Qh1A9WH6rEGe1bX.
 00001000: 4c8d a424 40ff ffff 4d3b 6610 0f86 5302  L..$@...M;f...S.
 00001010: 0000 5548 89e5 4881 ec38 0100 0048 8984  ..UH..H..8...H..
 00001020: 2448 0100 00e8 d6e6 0300 488d 05ac c85d  $H........H....]
@@ -1774443,7 +1774443,7 @@
 01b136a0: 6167 733d 6e65 7467 6f2c 6f73 7573 6572  ags=netgo,osuser
 01b136b0: 676f 0a62 7569 6c64 092d 7472 696d 7061  go.build.-trimpa
 01b136c0: 7468 3d74 7275 650a 6275 696c 6409 4347  th=true.build.CG
-01b136d0: 4f5f 454e 4142 4c45 443d 310a 6275 696c  O_ENABLED=1.buil
+01b136d0: 4f5f 454e 4142 4c45 443d 300a 6275 696c  O_ENABLED=0.buil
 01b136e0: 6409 474f 4152 4348 3d61 6d64 3634 0a62  d.GOARCH=amd64.b
 01b136f0: 7569 6c64 0947 4f4f 533d 6c69 6e75 780a  uild.GOOS=linux.
 01b13700: 6275 696c 6409 474f 414d 4436 343d 7632  build.GOAMD64=v2
@@ -2538712,7 +2538712,7 @@
 026bcd70: 6574 676f 2c6f 7375 7365 7267 6f0a 6275  etgo,osusergo.bu
 026bcd80: 696c 6409 2d74 7269 6d70 6174 683d 7472  ild.-trimpath=tr
 026bcd90: 7565 0a62 7569 6c64 0943 474f 5f45 4e41  ue.build.CGO_ENA
-026bcda0: 424c 4544 3d31 0a62 7569 6c64 0947 4f41  BLED=1.build.GOA
+026bcda0: 424c 4544 3d30 0a62 7569 6c64 0947 4f41  BLED=0.build.GOA
 026bcdb0: 5243 483d 616d 6436 340a 6275 696c 6409  RCH=amd64.build.
 026bcdc0: 474f 4f53 3d6c 696e 7578 0a62 7569 6c64  GOOS=linux.build
 026bcdd0: 0947 4f41 4d44 3634 3d76 320a 6275 696c  .GOAMD64=v2.buil

Despite enabling CGO the binary remains the same size. The win here, is that this package can now be built as is with go-fips to gain FIPS version of the binary.

Next we look at calico:

$ du --si * | grep typha
24M	calico-typha-client-3.27.2-r2.apk
12M	calico-typha-client-3.27.2-r3.apk
30M	calico-typhad-3.27.2-r2.apk
15M	calico-typhad-3.27.2-r3.apk

Switching to these scheme of "cgo enabled, but unused, unless needed, with symbols stripped" halfs the compressed apk size. The new build settings for this binary are

--- version-r2.txt	2024-03-12 18:53:30.324912809 +0000
+++ version-r3.txt	2024-03-12 18:53:36.088889227 +0000
@@ -1,4 +1,4 @@
-r2/usr/bin/calico-typha: go1.22.1
+r3/usr/bin/calico-typha: go1.22.1
 	path	github.com/projectcalico/calico/typha/cmd/calico-typha
 	mod	github.com/projectcalico/calico	(devel)	
 	dep	github.com/beorn7/perks	v1.0.1	h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
@@ -90,13 +90,14 @@
 	dep	sigs.k8s.io/yaml	v1.3.0	h1:a2VclLzOGrwOHDiV8EfBGhvjHvP46CtW5j6POvhYGGo=
 	build	-buildmode=exe
 	build	-compiler=gc
-	build	-tags=cgo
+	build	-tags=netgo,osusergo
+	build	-trimpath=true
 	build	DefaultGODEBUG=httplaxcontentlength=1,httpmuxgo121=1,tls10server=1,tlsrsakex=1,tlsunsafeekm=1
 	build	CGO_ENABLED=1
-	build	CGO_CFLAGS=
-	build	CGO_CPPFLAGS=
-	build	CGO_CXXFLAGS=
-	build	CGO_LDFLAGS=
 	build	GOARCH=amd64
 	build	GOOS=linux
-	build	GOAMD64=v1
+	build	GOAMD64=v2
+	build	vcs=git
+	build	vcs.revision=402c0b3815dc3f2452b1427fd23a394474ee513b
+	build	vcs.time=2024-02-17T01:17:55Z
+	build	vcs.modified=true
$ ldd r*/usr/bin/calico-typha
r2/usr/bin/calico-typha:
	linux-vdso.so.1 (0x00007ffd4b3e4000)
	libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007733eb331000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007733eb000000)
	/lib64/ld-linux-x86-64.so.2 (0x00007733eb357000)
r3/usr/bin/calico-typha:
	not a dynamic executable

Despite cgo being enabled.

xnox added 7 commits March 13, 2024 01:09
Add microarchitecture level settings for AMD64 and ARM64. Set defaults
to the current golang built-in upstream defaults.

Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@chainguard.dev>
Bump GOAMD64 from upstream default v1 to v2, which matches wolfi C
binaries baseline ISA -march=x86-64-v2.

Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@chainguard.dev>
Roughly half of go packages pass in `-s -w` already.

Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@chainguard.dev>
Pass -trimpath that strips prefixes from source filenames embedeed in
go binaries. This reduces binaries size, improves privacy, and
improves reproducibility.

Fixes 4b8ab55

Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@chainguard.dev>
Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@chainguard.dev>
Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@chainguard.dev>
Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@chainguard.dev>
@xnox
Copy link
Contributor Author

xnox commented Mar 25, 2024

All builds in wolfi-dev/os are good for all packages using go/build or go/install pipelines.
This is good to merge, and then we can start submitting individual cleanups to pretty-fy all the go packages.

See test build-results in wolfi-dev/os#15071 and all the related mentioned pull requests.
The only failures in those pull requests are for out of date package versions, CVE scans, and epoch conflicts (but it was up to date builds at the time of build).

@xnox xnox merged commit 0359650 into chainguard-dev:main Mar 25, 2024
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants