kmem.limit error when trying to run container

[crosbymichael]

container_linux.go:247: starting container process caused "process_linux.go:258: applying cgroup configuration for process caused \"failed to set memory.kmem.limit_in_bytes, because either tasks have already joined this cgroup or it has children\""

I don't have a kmem.limit set in my config

[hqhq]

Can't reproduce, is it on the latest runc? Any specific options or usage that can reproduce this?

[heartlock]

can you show me your config file? I can't reproduce

[crosbymichael]

I don't think its config related. It happens after I start a container sometimes and try to restart it. I do have a couple paths nested as the cgroupsPath so it should be the correct error message about children processes but the handling for the feature is just bad right now.

{
   "platform" : {
      "arch" : "amd64",
      "os" : "linux"
   },
   "hostname" : "redis-master",
   "ociVersion" : "1.0.0-rc2-dev",
   "mounts" : [
      {
         "source" : "proc",
         "destination" : "/proc",
         "type" : "proc"
      },
      {
         "options" : [
            "nosuid",
            "strictatime",
            "mode=755",
            "size=65536k"
         ],
         "type" : "tmpfs",
         "source" : "tmpfs",
         "destination" : "/dev"
      },
      {
         "type" : "devpts",
         "options" : [
            "nosuid",
            "noexec",
            "newinstance",
            "ptmxmode=0666",
            "mode=0620",
            "gid=5"
         ],
         "destination" : "/dev/pts",
         "source" : "devpts"
      },
      {
         "destination" : "/dev/shm",
         "source" : "shm",
         "type" : "tmpfs",
         "options" : [
            "nosuid",
            "noexec",
            "nodev",
            "mode=1777",
            "size=65536k"
         ]
      },
      {
         "type" : "mqueue",
         "options" : [
            "nosuid",
            "noexec",
            "nodev"
         ],
         "destination" : "/dev/mqueue",
         "source" : "mqueue"
      },
      {
         "source" : "sysfs",
         "destination" : "/sys",
         "options" : [
            "nosuid",
            "noexec",
            "nodev"
         ],
         "type" : "sysfs"
      },
      {
         "type" : "tmpfs",
         "options" : [
            "nosuid",
            "strictatime",
            "mode=755",
            "size=65536k"
         ],
         "destination" : "/run",
         "source" : "tmpfs"
      },
      {
         "source" : "/etc/resolv.conf",
         "destination" : "/etc/resolv.conf",
         "options" : [
            "rbind",
            "ro"
         ],
         "type" : "bind"
      },
      {
         "type" : "bind",
         "options" : [
            "rbind",
            "ro"
         ],
         "destination" : "/etc/hosts",
         "source" : "/etc/hosts"
      },
      {
         "type" : "bind",
         "options" : [
            "rbind",
            "ro"
         ],
         "destination" : "/etc/localtime",
         "source" : "/etc/localtime"
      }
   ],
   "linux" : {
      "resources" : {
         "devices" : [
            {
               "allow" : false,
               "access" : "rwm"
            }
         ],
         "cpu" : {
            "period" : 100000,
            "quota" : 100000
         },
         "memory" : {
            "limit" : 134217728
         },
         "oomScoreAdj" : 0,
         "blockIO" : {}
      },
      "cgroupsPath" : "/containers/crosbymichael/redis-master",
      "namespaces" : [
         {
            "type" : "pid"
         },
         {
            "type" : "ipc"
         },
         {
            "type" : "uts"
         },
         {
            "type" : "mount"
         }
      ]
   },
   "hooks" : {},
   "process" : {
      "rlimits" : [
         {
            "soft" : 1024,
            "hard" : 1024,
            "type" : "RLIMIT_NOFILE"
         }
      ],
      "consoleSize" : {
         "height" : 0,
         "width" : 0
      },
      "noNewPrivileges" : true,
      "cwd" : "/",
      "env" : [
         "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
      ],
      "capabilities" : [
         "CAP_AUDIT_WRITE",
         "CAP_KILL",
         "CAP_FOWNER",
         "CAP_CHOWN",
         "CAP_MKNOD",
         "CAP_FSETID",
         "CAP_DAC_OVERRIDE",
         "CAP_SETFCAP",
         "CAP_SETPCAP",
         "CAP_SETGID",
         "CAP_SETUID",
         "CAP_NET_BIND_SERVICE"
      ],
      "user" : {
         "uid" : 0,
         "gid" : 0
      },
      "args" : [
         "/run/init"
      ]
   },
   "root" : {
      "path" : "rootfs"
   }
}

[cyphar]

While #1095 solves the symptom, I don't think it actually solves the cause of the issue. @crosbymichael if you try to run your container again, but this time you do set a kmemcg limit will you get the same error (this is in the fairly dodgy kmem path, so I have no doubt there's a legitimate bug there somewhere)?

[asymfermion]

@crosbymichael I found that you have solved the problem and my code is also the latest, but the problem occur again and I do not know why. Could you please give me some advise?

[cyphar]

@mangoZJ Please open a new issue with all of the relevant information.