Running containers with podman doesn't work

[afbjorklund]

We install the podman tool into the VM, to do things such as loading or building OCI images.

But currently there are some issues with the CNI configuration, if you try to use it for running:

$ minikube ssh
                         _             _            
            _         _ ( )           ( )           
  ___ ___  (_)  ___  (_)| |/')  _   _ | |_      __  
/' _ ` _ `\| |/' _ `\| || , <  ( ) ( )| '_`\  /'__`\
| ( ) ( ) || || ( ) || || |\`\ | (_) || |_) )(  ___/
(_) (_) (_)(_)(_) (_)(_)(_) (_)`\___/'(_,__/'`\____)

$ sudo podman run -it busybox
Trying to pull docker.io/library/busybox...Getting image source signatures
Copying blob 53071b97a884 done
Copying config 64f5d945ef done
Writing manifest to image destination
Storing signatures
Error: error parsing CNI plugin result "IP4:{IP:{IP:10.1.0.7 Mask:ffff0000} Gateway:10.1.0.1 Routes:[{Dst:{IP:0.0.0.0 Mask:00000000} GW:10.1.0.1}]}, DNS:{Nameservers:[] Domain: Search:[] Options:[]}": cannot convert version ["" "0.1.0" "0.2.0"] to 0.4.0: cannot convert version ["" "0.1.0" "0.2.0"] to 0.4.0

Even though it is not the primary use-case (normally use crictl/crio), this should work...

Maybe we can get the standard minikube CNI configuration compatible with Podman ?

[afbjorklund]

Similar to cri-o/cri-o#1423

[afbjorklund]

Our current CNI configuration lives at:
https://github.com/kubernetes/minikube/blob/master/pkg/minikube/bootstrapper/kubeadm/default_cni.go

/etc/cni/net.d/k8s.conf

{
  "name": "rkt.kubernetes.io",
  "type": "bridge",
  "bridge": "mybridge",
  "mtu": 1460,
  "addIf": "true",
  "isGateway": true,
  "ipMasq": true,
  "ipam": {
    "type": "host-local",
    "subnet": "10.1.0.0/16",
    "gateway": "10.1.0.1",
    "routes": [
      {
        "dst": "0.0.0.0/0"
      }
    ]
  }
}

[afbjorklund]

Here is the example configuration, from podman v1.3.1:

{
    "cniVersion": "0.3.0",
    "name": "podman",
    "plugins": [
      {
        "type": "bridge",
        "bridge": "cni0",
        "isGateway": true,
        "ipMasq": true,
        "ipam": {
            "type": "host-local",
            "subnet": "10.88.0.0/16",
            "routes": [
                { "dst": "0.0.0.0/0" }
            ]
        }
      },
      {
        "type": "portmap",
        "capabilities": {
          "portMappings": true
        }
      }
    ]
}

So it seems we should upgrade ours to cniVersion 0.3.0

[afbjorklund]

There are some other changes needed as well:

1. We need to upgrade conmon, i.e. provide /usr/libexec/podman/conmon
   podman run is broken containers/podman#3161

$ sudo podman run -it busybox
Error: write child: broken pipe

2. We need to rebuild runc, systemd support doesn't work when built static
   container does not start because of systemd support for managing cgroups is not available containers/podman#1534
   Something is weird with the cgroup manager, thought we had systemd ?

$ sudo podman run -it busybox
Error: container create failed: systemd cgroup flag passed, but systemd support for managing cgroups is not available

3. We need to provide /etc/containers/libpod.conf, with our rootfs setting
   no_pivot_root = true

$ sudo podman run -it busybox
Error: container create failed: container_linux.go:345: starting container process caused "process_linux.go:424: container init caused \"rootfs_linux.go:109: jailing process inside rootfs caused \\\"pivot_root invalid argument\\\"\""

But after those four fixes, it works as intended.

[afbjorklund]

Needs #4421