[rootless+"privileged"] creating device nodes caused: open /..<snipped>../dev/tty: no such device or address

[AkihiroSuda]

On rootless+cgroup2(systemd)+CRI, creating a "privileged" container (which is not really privileged) fails with creating device nodes caused: open /..<snipped>.../dev/tty: no such device or address.

This error is not reproducible with docker (podman) run --privileged.

The repro step is same as containers/crun#382 but the error is different.

Repro

https://github.com/AkihiroSuda/critest-rootless-cgroup2/tree/v0.0.3

runc dbe5aca

Ubuntu 20.04

$ git clone https://github.com/AkihiroSuda/critest-rootless-cgroup2.git
$ cd critest-rootless-cgroup2
$ git checkout v0.0.3
$ ./bin-download.sh
$ cp -f /usr/local/sbin/runc ./bin/crun
$ ./containerd.sh

$ vi pod-config.json container-config.json 
$ git diff
diff --git a/container-config.json b/container-config.json
index fd33c47..0e4401b 100644
--- a/container-config.json
+++ b/container-config.json
@@ -11,7 +11,7 @@
     "log_path":"my-container.log",
     "linux": {
         "security_context": {
-            "privileged": false
+            "privileged": true
         },
         "resources": {
             "cpu_shares": 128,
diff --git a/pod-config.json b/pod-config.json
index 92b8c62..2dd0548 100644
--- a/pod-config.json
+++ b/pod-config.json
@@ -9,7 +9,7 @@
     "linux": {
         "cgroup_parent": "user.slice",
         "security_context": {
-            "privileged": false
+            "privileged": true
         }
     }
 }
$ ./run.sh runc
$ ./run.sh runc
+ RUNTIME=runc
++ ./crictl.sh pods -q
+ [[ -n '' ]]
++ ./crictl.sh run --runtime=runc container-config.json pod-config.json
FATA[0002] Running container failed: starting the container "6451ca158f106fc8dd68f168993671ef60bbe195b2cd35e997f368270d65778a" failed: rpc error: code = Unknown desc = failed to create containerd task: OCI runtime create failed: container_linux.go:353: starting container process caused: process_linux.go:459: container init caused: rootfs_linux.go:70: creating device nodes caused: open /run/.ro813277178/user/1001/containerd/io.containerd.runtime.v2.task/k8s.io/6451ca158f106fc8dd68f168993671ef60bbe195b2cd35e997f368270d65778a/rootfs/dev/tty: no such device or address: unknown 
+ id=

[AkihiroSuda]

Reproducible without CRI as well:

$ rootlesskit  runc  --systemd-cgroup run foo
WARN[0000] exit status 1                                
ERRO[0000] container_linux.go:353: starting container process caused: process_linux.go:459: container init caused: rootfs_linux.go:70: creating device nodes caused: open /home/suda/tmp/runctest/rootfs/dev/tty: no such device or address 
[rootlesskit:child ] error: command [runc --systemd-cgroup run foo] exited: exit status 1
[rootlesskit:parent] error: child exited: exit status 1

{
  "ociVersion": "1.0.2-dev",
  "process": {
    "terminal": true,
    "user": {
      "uid": 0,
      "gid": 0
    },
    "args": [
      "sh"
    ],
    "env": [
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
      "TERM=xterm"
    ],
    "cwd": "/",
    "capabilities": {
      "bounding": [
        "CAP_AUDIT_WRITE",
        "CAP_KILL",
        "CAP_NET_BIND_SERVICE"
      ],
      "effective": [
        "CAP_AUDIT_WRITE",
        "CAP_KILL",
        "CAP_NET_BIND_SERVICE"
      ],
      "inheritable": [
        "CAP_AUDIT_WRITE",
        "CAP_KILL",
        "CAP_NET_BIND_SERVICE"
      ],
      "permitted": [
        "CAP_AUDIT_WRITE",
        "CAP_KILL",
        "CAP_NET_BIND_SERVICE"
      ],
      "ambient": [
        "CAP_AUDIT_WRITE",
        "CAP_KILL",
        "CAP_NET_BIND_SERVICE"
      ]
    },
    "rlimits": [
      {
        "type": "RLIMIT_NOFILE",
        "hard": 1024,
        "soft": 1024
      }
    ],
    "noNewPrivileges": true
  },
  "root": {
    "path": "rootfs",
    "readonly": true
  },
  "hostname": "runc",
  "mounts": [
    {
      "destination": "/proc",
      "type": "proc",
      "source": "proc"
    },
    {
      "destination": "/dev",
      "type": "tmpfs",
      "source": "tmpfs",
      "options": [
        "nosuid",
        "strictatime",
        "mode=755",
        "size=65536k"
      ]
    },
    {
      "destination": "/dev/pts",
      "type": "devpts",
      "source": "devpts",
      "options": [
        "nosuid",
        "noexec",
        "newinstance",
        "ptmxmode=0666",
        "mode=0620",
        "gid=5"
      ]
    },
    {
      "destination": "/dev/shm",
      "type": "tmpfs",
      "source": "shm",
      "options": [
        "nosuid",
        "noexec",
        "nodev",
        "mode=1777",
        "size=65536k"
      ]
    },
    {
      "destination": "/dev/mqueue",
      "type": "mqueue",
      "source": "mqueue",
      "options": [
        "nosuid",
        "noexec",
        "nodev"
      ]
    },
    {
      "destination": "/sys",
      "type": "sysfs",
      "source": "sysfs",
      "options": [
        "nosuid",
        "noexec",
        "nodev",
        "ro"
      ]
    },
    {
      "destination": "/sys/fs/cgroup",
      "type": "cgroup",
      "source": "cgroup",
      "options": [
        "nosuid",
        "noexec",
        "nodev",
        "relatime",
        "ro"
      ]
    }
  ],
  "linux": {
    "cgroupsPath": "user.slice:runc:foo",
    "resources": {
      "devices": [
        {
          "allow": true,
          "access": "rwm"
        }
      ]
    },
    "devices": [
      {
        "path": "/dev/tty",
        "type": "c",
        "major": 5,
        "minor": 0,
        "fileMode": 8630,
        "uid": 65534,
        "gid": 65534
      }
    ],
    "namespaces": [
      {
        "type": "pid"
      },
      {
        "type": "network"
      },
      {
        "type": "ipc"
      },
      {
        "type": "uts"
      },
      {
        "type": "mount"
      }
    ],
    "maskedPaths": [
      "/proc/acpi",
      "/proc/asound",
      "/proc/kcore",
      "/proc/keys",
      "/proc/latency_stats",
      "/proc/timer_list",
      "/proc/timer_stats",
      "/proc/sched_debug",
      "/sys/firmware",
      "/proc/scsi"
    ],
    "readonlyPaths": [
      "/proc/bus",
      "/proc/fs",
      "/proc/irq",
      "/proc/sys",
      "/proc/sysrq-trigger"
    ]
  }
}

The issue is not reproducible with docker run because Docker removes devices for rootless: https://github.com/moby/moby/blob/89382f2f20745b9e63bed6c066f104980dff4396/rootless/specconv/specconv_linux.go#L42-L43

[AkihiroSuda]

Should have been closed in #2522