build all non-distro-packaged go binaries ourselves and respect .go-version

[BenTheElder]

Pros:

• streamlines patching go version related CVEs
• ensures binaries with CGO are linked to the library version we'll be using
• allows controlling runc version independent of containerd version
• allows retiring https://github.com/kind-ci/containerd-nightlies, we can leave it as-is but we don't need to keep maintaining this vestigial hack and any contributor can PR changes to the containerd build as normal now
• allows us to control build-tags and compile out unused containerd snapshotters for some binary size savings
• ensures we can easily include LICENSE / notice info for binaries not coming from ubuntu packages

Cons:

• building the node image is more expensive, however we don't have to do this super often and we have automated builds now so we can throw cores at it and still have quick builds. The buildx multi-stage build is a DAG that can execute steps in parallel so with enough cores this runs pretty quickly.
• we had to put another copy of gimme in the image directory ... you can't use symlinks outside of the build context with docker build and we don't want the whole repo as a build context ... this is the path of least resistance for now. gimme doesn't really need updating for the forseeable future though so this is fine.
• dockerfile is somewhat more complex with more multi-stage steps, however, on the other hand these steps are typically actually simpler than the "fetch and check shasum" inline scripts, basically just clone and then make

TODO:

• Update image build postubmit jobs to trigger on .go-version and images/Makefile.common.in
  • trigger relevant kind image builds if .go-version changes kubernetes/test-infra#29474
• Rework images/kindnetd, images/local-path-provisioner to also respect .go-version w/ gimme
  • DONE in 2nd and 3rd commits

When those are complete, go CVEs can be addressed in two PRs:

1. bump .go-version
2. bump references to images built as a result of merging 1)

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: BenTheElder

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [BenTheElder]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[aojea]

cool

[BenTheElder]

I've run make -C images/base quick locally and the image checks out, I haven't used it for a node image yet, but I don't have any reason to suspect issues there.

[BenTheElder]

I've updated this to also include LICENSE and notice files via https://github.com/google/go-license

Some other Kubernetes projects are doing this and we haven't been, this is easier to do now that we're cloning and building these packages.

debian packages that we're just installing the usual way already include comparable info in a standard way.

[BenTheElder]

I've tested all of these builds locally in quick mode (single-architecture only w/o pushing).

[aojea]

Latest?

[BenTheElder]

yes, the most recent tagged release. the API surface of this tool is small

[aojea]

Didn't containerd have a specific version for runc that we obtained from one of the files on the repo?

[BenTheElder]

We did do that, but that version is just a recommendation / what they're using in CI in containerd*, we want to be able to control the pace of upgrading runc.

* their own integration tests, maybe not k8s e2e ...

We may even have to split this up and use multiple versions depending on k8s version, see kubernetes/kubernetes#117691 (comment)

[BenTheElder]

https://github.com/containerd/containerd/blob/c6d7e45c14896addf3b1e06db3f7999026b0191f/docs/RUNC.md#runc-version-requirements-for-containerd

It's recommended to use a version >= https://github.com/containerd/containerd/blob/main/script/setup/runc-version, but that may actually be broken with Kubernetes or we may want to be ahead.

runc versions will need more thought following the thread I linked above, but for now sticking to the current stable tag.

[BenTheElder]

should add: 1.1.7 is the version used by containerd 1.6.21 https://github.com/containerd/containerd/releases/tag/v1.6.21

semi-related: they're no longer including runc with containerd releases containerd/containerd#6541

[BenTheElder]

turns out we actually need to be able to stick to an older version at times. #3220

older is fine, newer binary may not be. at least as long as we have to support cgroups v1

[aojea]

/lgtm

[BenTheElder]

This has serious perf issues in the arm64 cross-build. Working on a fix

[BenTheElder]

follow-up in #3214