feat: registry addon

[shaneutt]

This PR adds a registry addon so that a container image registry can be loaded into the cluster and be made available for pods.

In order to accomplish this several new utilities were added, highlights include:

• utilities for running commands and reading/writing files in docker containers
• certificate management utilities to simplify creating cert-manager certificates
• networking utilities for kubernetes

This new addon brings several benefits including:

• resolves A registry addon #147
• helps support testing workflows where images need to be rapidly built and deployed into the test cluster
• easily seed images from cache in CI

This is the first iteration of this addon but is incomplete: this works using a self-signed certificate and only on kind clusters. The current certificate management landscape in a KTF cluster is limited and the default cluster CA feature is desired prior to adding support for GKE so that certificate trust configuration no longer needs to be a part of the addon deployment as was done here, but instead cluster setup.

[rainest]

Some minor doc grammar stuff in suggestions. I think we should also mention the --tlsverify=false option as an alternative to loading certificates--I expect most users will be okay with that for tests.

[hutchic]

have docs / examples by chance?

[shaneutt]

have docs / examples by chance?

There's a helper doc present in the CLI which explains how to push images, but we definitely need to start thinking more holistically about documenting KTF now that our use cases are expanding beyond the KIC.

Here's an example run which includes that output:

$ go run cmd/ktf/main.go envs create --addon metallb --addon cert-manager --addon registry
building new environment kong-testing-environment
waiting for addon registry to become ready...
waiting for addon metallb to become ready...
waiting for addon cert-manager to become ready...
waiting for environment to become ready (this can take some time)...
environment kong-testing-environment was created successfully!

Registry Addon HELP:

You have installed the registry addon deployed with an SSL certificate provided
by cert-manager. The default certificate used is a self-signed certificate.
As such if you try to push images to this registry with the standard:

  $ docker push ${REGISTRY_IP}/image

Without first adding its certificate to your local docker (or other client) chain
of trust it will fail. The following provides an example of how to add the certificate
using a standard docker installation on a Linux system where "/etc/docker" is the
configuration directory for docker:

  $ REGISTRY_IP="$(kubectl -n registry get svc registry -o=go-template='{{(index .status.loadBalancer.ingress 0).ip}}')"
  $ sudo mkdir -p /etc/docker/certs.d/${REGISTRY_IP}/
  $ kubectl -n registry get secrets registry-cert-secret -o=go-template='{{index .data "ca.crt"}}' | base64 -d | sudo tee /etc/docker/certs.d/${REGISTRY_IP}/ca.crt

Note that this generally is not going to work verbatim on all systems and the
above instructions should be considered just an example. Adjust for your own
system and docker installation. You may also need to change ".ip" for ".host"
if your service is provided a DNS name instead of an IP for its LB address.

Afterwards you should be able to push images to the registry, e.g.:

  $ docker pull kennethreitz/httpbin
  $ docker tag kennethreitz/httpbin ${REGISTRY_IP}/httpbin
  $ docker push ${REGISTRY_IP}/httpbin

Images pushed this way should be immediately usable in pod configurations
on the cluster as the certificate is automatically configured on the nodes.

So the short version you'll need to trust the certificate in your local environment, and then whatever the LoadBalancer IP is for the registry you can use that to push images and you can configure your images in podspecs, e.g. image: <your lb ip>/<image>:<tag>.

So in my own environment, this is what I literally ran to push an image to the registry:

$ go run cmd/ktf/main.go envs create --addon metallb --addon cert-manager --addon registry
$ REGISTRY_IP="$(kubectl -n registry get svc registry -o=go-template='{{(index .status.loadBalancer.ingress 0).ip}}')"
$ sudo mkdir -p /etc/docker/certs.d/${REGISTRY_IP}/
$ kubectl -n registry get secrets registry-cert-secret -o=go-template='{{index .data "ca.crt"}}' | base64 -d | sudo tee /etc/docker/certs.d/${REGISTRY_IP}/ca.crt
$ docker pull nginx
$ docker tag nginx ${REGISTRY_IP}/nginx
$ docker push ${REGISTRY_IP}/nginx

Then used it for a deployment:

$ cat << EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: ${REGISTRY_IP}/nginx
        ports:
        - containerPort: 80
EOF

And then:

$ kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
nginx-57f46f4d55-clbgf   1/1     Running   0          3s

Hopefully that helps you in the short term, for the long term I've created #184 so we can start improving the documentation.