kill command fails with read cgroup.procs: operation not supported
#3821
Comments
|
I found that the issue is that the cgroup is in threaded mode, and in that case, reading cgroup.procs returns ENOTSUP. By patching runc with the following patch, tests work again and runc doesn't fail: diff --git a/libcontainer/cgroups/utils.go b/libcontainer/cgroups/utils.go
index b32af4ee..70080efd 100644
--- a/libcontainer/cgroups/utils.go
+++ b/libcontainer/cgroups/utils.go
@@ -19,6 +19,7 @@ import (
const (
CgroupProcesses = "cgroup.procs"
+ CgroupThreads = "cgroup.threads"
unifiedMountpoint = "/sys/fs/cgroup"
hybridMountpoint = "/sys/fs/cgroup/unified"
)
@@ -137,14 +138,16 @@ func GetAllSubsystems() ([]string, error) {
}
func readProcsFile(dir string) ([]int, error) {
- f, err := OpenFile(dir, CgroupProcesses, os.O_RDONLY)
+ contents, err := ReadFile(dir, CgroupProcesses)
+ if errors.Is(err, unix.ENOTSUP) {
+ contents, err = ReadFile(dir, CgroupThreads)
+ }
if err != nil {
return nil, err
}
- defer f.Close()
var (
- s = bufio.NewScanner(f)
+ s = bufio.NewScanner(strings.NewReader(contents))
out = []int{}
)Here is the type of the cgroups (these commands were run inside the buildkit's dev-env container: |
amurzeau
changed the title
read cgroup.procs: operation not supported
|
Hi, |
|
@Bacto we've changed this part of runc a lot in the main branch. Can you try to repro this using runc compiled from the main branch? |
|
Hi @kolyshkin, I tried with the main branch and got the same issue: # runc -v
runc version 1.1.0+dev
commit: 0c5a735
spec: 1.1.0+dev
go: go1.21.6
libseccomp: 2.5.5 |
So the problem here is This is some kind of a misconfiguration, possibly caused by buildkit. |
|
Created Debian 12 VM, checked in buildkit and ran its test suite inside a container ( I think there was something wrong originally when starting a container. Would still like to get to the bottom of it, so any suggestions of how to reproduce it (ideally a vagrant file or something like this) are welcome. |
|
The issue is fixed in main branch. I've tried to find the first fixed version and found that I can reproduce the same issue with So I'm closing this issue. For reference, I'm using |
|
After upgrade runc, docker still report this error: error: # cat /sys/fs/cgroup/docker/a77dc1be093aebb8a8f18fd634adc2ebbf2d798a7c7e8e7aa5283770b1efd9b6/cgroup.type
threaded |
|
I tried 8256a93, which fix this problem. |
I guess you quoted a wrong commit. |
|
@amurzeau could you do git-bisect to find which runc commit fixes it? |
|
The first commit without the issue is f8ad20f. The cause is that the failure occurs with this stacktrace: The commit that fixes the issue (f8ad20f) removes the call to I think this can be reproduced with this bundle: To test: Running runc kill --all yield the error at commit 9583b3d: buildkit / containerd use a pid namespace, so after f8ad20f, runc/libcontainer/container_linux.go Lines 386 to 388 in f8ad20f And thus still trigger the error (so I'm not sure the commit really fix the issue): Note: I'm running this test in a docker rootful container. |
Description
Hi,
While testing buildkit within a docker container, tests use runc.
When tring to kill a runc container, runc error out with and error like this:
read /sys/fs/cgroup/buildkit/mxv4shz9kwdm0p5u49mw971ft/cgroup.procs: operation not supportedand then return error code 1.
The command line is this one:
runc --root /run/containerd/runc/buildkit --log /tmp/bktest_containerd1141985211/state/io.containerd.runtime.v2.task/buildkit/mxv4shz9kwdm0p5u49mw971ft/log.json --log-format json kill --all mxv4shz9kwdm0p5u49mw971ft 9Steps to reproduce the issue
Describe the results you received and expected
Several tests using containerd fail with this error:
What version of runc are you using?
runc version v1.1.5
spec: 1.0.2-dev
go: go1.20.3
libseccomp: 2.5.4
Host OS information
Host:
container running dev-env target from Dockerfile from buildkit git repository:
Host kernel information
Linux DOC-PC3 6.1.0-7-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.20-1 (2023-03-19) x86_64 GNU/Linux
The text was updated successfully, but these errors were encountered: