Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Action build-registry-server failing #1196

Closed
theganyo opened this issue Jun 15, 2023 · 6 comments · Fixed by #1197
Closed

Action build-registry-server failing #1196

theganyo opened this issue Jun 15, 2023 · 6 comments · Fixed by #1197
Assignees
Labels
bug Something isn't working

Comments

@theganyo
Copy link
Member

https://github.com/apigee/registry/actions/runs/5273705838/jobs/9537430024

@theganyo theganyo added the bug Something isn't working label Jun 15, 2023
@theganyo theganyo self-assigned this Jun 15, 2023
@theganyo theganyo moved this from Inbox to In Progress in Apigee API Registry Jun 15, 2023
@theganyo
Copy link
Member Author

theganyo commented Jun 15, 2023

This issue is created by an incompatibility between our build and release container images.

We use golang:1.20 as our builder image as our builder - that label is tied to the latest 1.20 go build.

As of go1.20.5 (released a few days ago), this image is being built from the newly released Debian 12 image.

We use gcr.io/distroless/base:latest as our release image, which is still being built on Debian 11. Note: There is an issue to upgrade this to Debian 12.

We copy binaries from the build to the release container, but these are incompatible which causes the command chmod to fail to execute in the distroless container.

A quick fix for now would be to tie our build to go 1.20.4 - which was built on Debian 11.

Note: As distroless intentionally excludes all the binaries for security, if possible we should rethink copying them from the build container.

@timburks
Copy link
Contributor

Note: As distroless intentionally excludes all the binaries for security, if possible we should rethink copying them from the build container.

Do you have a recommendation for what we might do if we didn't copy binaries from the build container?

@theganyo
Copy link
Member Author

If we simply remove the extraneous binaries from our release, the build works but leaves us with a binary that gets the following errors when running in the container:

/registry-server: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /registry-server)
/registry-server: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /registry-server)
/registry-server: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /registry-server)

This is because we build with CGO_ENABLED=1, which ties us to the libraries. So, I suggest we do both for now:

  1. Remove the extraneous binaries
  2. Peg the builder at golang:1.20.4

Then when distroless/base is updated, we should be able to unpeg the go version.

@dlorenc
Copy link

dlorenc commented Jun 16, 2023

Another option is to use the distroless images from Chainguard here: cgr.dev/chainguard/glibc-dynamic.

These work correctly with the updated glibc in bookworm, and are more flexible than original distroless images.

I'd be happy to help! I wouldn't recommend staying on go 1.20.4 - there are several severe CVEs patched in 1.20.5: https://go.dev/doc/devel/release#go1.20

@theganyo
Copy link
Member Author

Thanks, @dlorenc, good point on the issues. Another option is mentioned on distroless here: GoogleContainerTools/distroless#1342. I've verified that both your suggestion and using golang:1.20-bullseye as our builder would work. I'm leaning toward switching to bullseye for now as it is the smallest delta. @timburks ?

@dlorenc
Copy link

dlorenc commented Jun 20, 2023

Sounds good! Let me know if we can help at all.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

3 participants