Add nerdctl command for users outside kubernetes

[afbjorklund]

The default ctr and buildctl are a bit rough around the edges (i.e. user-hostile)

There is another program called nerdctl, which looks more like docker and podman.

It doesn't have a remote protocol, so users will have to use minikube ssh to use it...

(i.e. there is no equivalent to docker-env and podman-env, except for ssh tunneling)

https://github.com/containerd/nerdctl#command-reference

https://minikube.sigs.k8s.io/docs/handbook/pushing/#6-pushing-directly-to-in-cluster-containerd-buildkitd

---

With Docker and Podman, there are both local clients as well as remote clients:

minikube ssh -- docker --help

eval $(minikube docker-env)
docker --help

minikube ssh -- sudo podman --help

eval $(minikube podman-env)
podman --remote --help

With nerdctl/containerd/buildkitd, there are no local clients but only remote client:

minikube ssh -- sudo nerdctl --help

$ eval $(./out/minikube docker-env)
❌  Exiting due to MK_USAGE: The docker-env command is only compatible with the "docker" runtime, but this cluster was configured to use the "containerd" runtime.
$ eval $(./out/minikube podman-env)
❌  Exiting due to MK_USAGE: The podman-env command is only compatible with the "crio" runtime, but this cluster was configured to use the "containerd" runtime.

"local" client would be: running on the host, i.e. remotely (to server)
"remote" client would be: running on the node, i.e. locally (to server)

[afbjorklund]

For regular use, there is kubectl and crictl as always. Or the "minikube image" wrapper.

When using the regular containers, normally it is sudo nerdctl. And perhaps -n k8s.io.

[afbjorklund]

Pre-built binaries: https://github.com/containerd/nerdctl/releases/tag/v0.15.0

https://github.com/containerd/nerdctl/releases/download/v0.15.0/nerdctl-0.15.0-linux-amd64.tar.gz

docker@minikube:~$ sudo nerdctl version
Client:
 Version:	v0.15.0
 Git commit:	b72b5ca14550b2e23a42787664b6182524c5053f

Server:
 containerd:
  Version:	1.4.9
  GitCommit:	e25210fe30a0a703442421b0f60afac609f950a3

# "list all containers running for Kubernetes" 
docker@minikube:~$ sudo nerdctl -n k8s.io ps

# "list all images loaded for Kubernetes"
docker@minikube:~$ sudo nerdctl -n k8s.io images

# "build and tag a image for Kubernetes"
docker@minikube:~$ sudo nerdctl -n k8s.io build -t myimage .

No deb packages, sadly.

[afbjorklund]

Rootless nerdctl/containerd is not supported, and will give an error message:

docker@minikube:~$ nerdctl version
WARN[0000] environment variable XDG_RUNTIME_DIR is not set, see https://rootlesscontaine.rs/getting-started/common/login/ 
WARN[0000] environment variable XDG_RUNTIME_DIR is not set, see https://rootlesscontaine.rs/getting-started/common/login/ 
FATA[0000] rootless containerd not running? (hint: use `containerd-rootless-setuptool.sh install` to start rootless containerd): environment variable XDG_RUNTIME_DIR is not set, see https://rootlesscontaine.rs/getting-started/common/login/ 

This is "normal", and not supported with the other container runtimes either.

docker@minikube:~$ podman version
Error: command required for rootless mode with multiple IDs: exec: "newuidmap": executable file not found in $PATH

https://docs.docker.com/engine/security/rootless/

docker@minikube:~$ dockerd-rootless-setuptool.sh install
-bash: dockerd-rootless-setuptool.sh: command not found

[afbjorklund]

There is a nerdctl 0.16.0 out now.

[afbjorklund]

Upgraded to nerdctl 0.17.1, and containerd (ctr) 1.5 / buildkit (buildctl) 0.10

Note: we're currently still using the same containerd version as docker uses.

[afbjorklund]

There is a nerdctl 0.18.0 out now.

https://github.com/containerd/nerdctl/releases/download/v0.18.0/nerdctl-0.18.0-linux-amd64.tar.gz

[afbjorklund]

The current release is 0.22.0, same difference.

https://github.com/containerd/nerdctl/releases/tag/v0.22.0

[afbjorklund]

/remove-lifecycle stale

[afbjorklund]

/remove-lifecycle stale

[afbjorklund]

This is a pre-requisite for nerdctl.sock, if wanting to keep the API compatibility with docker

The current ctr and buildctl "work", but they don't have any CLI compatibility with docker

[afbjorklund]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.