Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"runc init" goes into a loop on RHEL 7.6 (runc 09c8266bf2fcf9519a651b04ae54c967b9ab86ec) #1988

Closed
pinacoelho opened this issue Feb 18, 2019 · 39 comments · Fixed by #1984
Closed

Comments

@pinacoelho
Copy link

pinacoelho commented Feb 18, 2019

After the last upgrade, docker hangs while starting containers.
htop shows the runc processes consuming 100% of the CPU time.

Environment
RHEL 7.6 (up to date)
from https://download.docker.com/linux/centos/7/x86_64/stable:
containerd.io-1.2.2-3.3.el7.x86_64.rpm 2019-02-11 16:03:53 22.1 MiB
docker-ce-cli-18.09.2-3.el7.x86_64
docker-ce-18.09.2-3.el7.x86_64

Steps to reproduce the issue:

1. Install docker-ce-18.09.2-3.el7.x86_64, docker-ce-cli-18.09.2-3.el7.x86_64, containerd.io-1.2.2-3.3.el7.x86_64.rpm from https://download.docker.com/linux/centos/7/x86_64/stable

2. docker run alpine

Describe the results you received:
"docker run alpine" hangs. htop shows the "runc init" process consuming 100% cpu.

Describe the results you expected:
Expected docker to run the alpine container and return.

Output of containerd --version:

[root@moykano ~]# containerd --version
containerd github.com/containerd/containerd 1.2.2 [9754871](https://github.com/containerd/containerd/commit/9754871865f7fe2f4e74d43e2fc7ccd237edcbce)

Note that containerd --version outputs the same result for 1.2.2-3 and 1.2.2-3.3

Downgrading containerd.io to containerd.io-1.2.2-3.el7.x86_64.rpm 2019-01-09 21:07:30 solves the issue.

runc --version from 1.2.2-3 (GOOD):

runc version 1.0.0-rc6+dev
commit: 96ec2177ae841256168fcf76954f7177af9446eb
spec: 1.0.1-dev

runc --version from 1.2.2-3.3 (BAD):

runc version 1.0.0-rc6+dev
commit: 09c8266bf2fcf9519a651b04ae54c967b9ab86ec
spec: 1.0.1-dev

** Delta between versions **
96ec217...09c8266

  • Kernel level: Linux moykano 3.10.0-957.5.1.el7.x86_64 Change version to 0.x #1 SMP Wed Dec 19 10:46:58 EST 2018 x86_64 x86_64 x86_64 GNU/Linux (rpm: kernel-3.10.0-957.1.3.el7.x86_64)

  • lsb_release -a
    LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
    Distributor ID: RedHatEnterpriseWorkstation
    Description: Red Hat Enterprise Linux Workstation release 7.6 (Maipo)
    Release: 7.6
    Codename: Maipo

@pinacoelho
Copy link
Author

Original issue containerd/containerd#3027

@pinacoelho
Copy link
Author

pinacoelho commented Feb 18, 2019

An strace of the looping "runc init":

strace log
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 5
fcntl(5, F_GET_SEALS)                   = -1 EINVAL (Invalid argument)
close(5)                                = 0
open("/proc/self/cmdline", O_RDONLY|O_CLOEXEC) = 5
read(5, "runc\0init\0", 4096)           = 10
brk(NULL)                               = 0x55898da97000
brk(0x55898dab8000)                     = 0x55898dab8000
brk(NULL)                               = 0x55898dab8000
read(5, "", 4096)                       = 0
close(5)                                = 0
open("/proc/self/environ", O_RDONLY|O_CLOEXEC) = 5
read(5, "GOMAXPROCS=2\0_LIBCONTAINER_INITP"..., 4096) = 93
read(5, "", 4096)                       = 0
close(5)                                = 0
memfd_create("runc_cloned:/proc/self/exe", MFD_CLOEXEC|MFD_ALLOW_SEALING) = 5
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 6
sendfile(5, 6, NULL, 2147479552)        = 17501832
close(6)                                = 0
fcntl(5, F_ADD_SEALS, F_SEAL_SEAL|F_SEAL_SHRINK|F_SEAL_GROW|F_SEAL_WRITE) = 0
execve("/proc/self/fd/5", ["runc", "init"], [/* 4 vars */]) = 0
brk(NULL)                               = 0x5593fc350000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdebf98f000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=221036, ...}) = 0
mmap(NULL, 221036, PROT_READ, MAP_PRIVATE, 5, 0) = 0x7fdebf959000
close(5)                                = 0
open("/lib64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260l\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=141968, ...}) = 0
mmap(NULL, 2208904, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fdebf553000
mprotect(0x7fdebf56a000, 2093056, PROT_NONE) = 0
mmap(0x7fdebf769000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x16000) = 0x7fdebf769000
mmap(0x7fdebf76b000, 13448, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fdebf76b000
close(5)                                = 0
open("/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\r\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=19288, ...}) = 0
mmap(NULL, 2109712, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fdebf34f000
mprotect(0x7fdebf351000, 2097152, PROT_NONE) = 0
mmap(0x7fdebf551000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2000) = 0x7fdebf551000
close(5)                                = 0
open("/lib64/libseccomp.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\360\1\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=266672, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdebf958000
mmap(NULL, 2359552, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fdebf10e000
mprotect(0x7fdebf13a000, 2093056, PROT_NONE) = 0
mmap(0x7fdebf339000, 90112, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2b000) = 0x7fdebf339000
close(5)                                = 0
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340$\2\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=2151672, ...}) = 0
mmap(NULL, 3981792, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fdebed41000
mprotect(0x7fdebef03000, 2097152, PROT_NONE) = 0
mmap(0x7fdebf103000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x1c2000) = 0x7fdebf103000
mmap(0x7fdebf109000, 16864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fdebf109000
close(5)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdebf957000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fdebf955000
arch_prctl(ARCH_SET_FS, 0x7fdebf955740) = 0
mprotect(0x7fdebf103000, 16384, PROT_READ) = 0
mprotect(0x7fdebf339000, 86016, PROT_READ) = 0
mprotect(0x7fdebf551000, 4096, PROT_READ) = 0
mprotect(0x7fdebf769000, 4096, PROT_READ) = 0
mprotect(0x5593fb931000, 3280896, PROT_READ) = 0
mprotect(0x7fdebf990000, 4096, PROT_READ) = 0
munmap(0x7fdebf959000, 221036)          = 0
set_tid_address(0x7fdebf955a10)         = 21643
set_robust_list(0x7fdebf955a20, 24)     = 0
rt_sigaction(SIGRTMIN, {0x7fdebf559790, [], SA_RESTORER|SA_SIGINFO, 0x7fdebf5625d0}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7fdebf559820, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7fdebf5625d0}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 5
fcntl(5, F_GET_SEALS)                   = -1 EINVAL (Invalid argument)
close(5)                                = 0
open("/proc/self/cmdline", O_RDONLY|O_CLOEXEC) = 5
read(5, "runc\0init\0", 4096)           = 10
brk(NULL)                               = 0x5593fc350000
brk(0x5593fc371000)                     = 0x5593fc371000
brk(NULL)                               = 0x5593fc371000
read(5, "", 4096)                       = 0
close(5)                                = 0
open("/proc/self/environ", O_RDONLY|O_CLOEXEC) = 5
read(5, "GOMAXPROCS=2\0_LIBCONTAINER_INITP"..., 4096) = 93
read(5, "", 4096)                       = 0
close(5)                                = 0
memfd_create("runc_cloned:/proc/self/exe", MFD_CLOEXEC|MFD_ALLOW_SEALING) = 5
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 6
sendfile(5, 6, NULL, 2147479552)        = 17501832
close(6)                                = 0
fcntl(5, F_ADD_SEALS, F_SEAL_SEAL|F_SEAL_SHRINK|F_SEAL_GROW|F_SEAL_WRITE) = 0
execve("/proc/self/fd/5", ["runc", "init"], [/* 4 vars */]) = 0
brk(NULL)                               = 0x560faa7b8000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9988308000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=221036, ...}) = 0
mmap(NULL, 221036, PROT_READ, MAP_PRIVATE, 5, 0) = 0x7f99882d2000
close(5)                                = 0
open("/lib64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260l\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=141968, ...}) = 0
mmap(NULL, 2208904, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7f9987ecc000
mprotect(0x7f9987ee3000, 2093056, PROT_NONE) = 0
mmap(0x7f99880e2000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x16000) = 0x7f99880e2000
mmap(0x7f99880e4000, 13448, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f99880e4000
close(5)                                = 0
open("/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\r\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=19288, ...}) = 0
mmap(NULL, 2109712, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7f9987cc8000
mprotect(0x7f9987cca000, 2097152, PROT_NONE) = 0
mmap(0x7f9987eca000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2000) = 0x7f9987eca000
close(5)                                = 0
open("/lib64/libseccomp.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\360\1\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=266672, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f99882d1000
mmap(NULL, 2359552, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7f9987a87000
mprotect(0x7f9987ab3000, 2093056, PROT_NONE) = 0
mmap(0x7f9987cb2000, 90112, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2b000) = 0x7f9987cb2000
close(5)                                = 0
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340$\2\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=2151672, ...}) = 0
mmap(NULL, 3981792, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7f99876ba000
mprotect(0x7f998787c000, 2097152, PROT_NONE) = 0
mmap(0x7f9987a7c000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x1c2000) = 0x7f9987a7c000
mmap(0x7f9987a82000, 16864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f9987a82000
close(5)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f99882d0000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f99882ce000
arch_prctl(ARCH_SET_FS, 0x7f99882ce740) = 0
mprotect(0x7f9987a7c000, 16384, PROT_READ) = 0
mprotect(0x7f9987cb2000, 86016, PROT_READ) = 0
mprotect(0x7f9987eca000, 4096, PROT_READ) = 0
mprotect(0x7f99880e2000, 4096, PROT_READ) = 0
mprotect(0x560faa258000, 3280896, PROT_READ) = 0
mprotect(0x7f9988309000, 4096, PROT_READ) = 0
munmap(0x7f99882d2000, 221036)          = 0
set_tid_address(0x7f99882cea10)         = 21643
set_robust_list(0x7f99882cea20, 24)     = 0
rt_sigaction(SIGRTMIN, {0x7f9987ed2790, [], SA_RESTORER|SA_SIGINFO, 0x7f9987edb5d0}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7f9987ed2820, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7f9987edb5d0}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 5
fcntl(5, F_GET_SEALS)                   = -1 EINVAL (Invalid argument)
close(5)                                = 0
open("/proc/self/cmdline", O_RDONLY|O_CLOEXEC) = 5
read(5, "runc\0init\0", 4096)           = 10
brk(NULL)                               = 0x560faa7b8000
brk(0x560faa7d9000)                     = 0x560faa7d9000
brk(NULL)                               = 0x560faa7d9000
read(5, "", 4096)                       = 0
close(5)                                = 0
open("/proc/self/environ", O_RDONLY|O_CLOEXEC) = 5
read(5, "GOMAXPROCS=2\0_LIBCONTAINER_INITP"..., 4096) = 93
read(5, "", 4096)                       = 0
close(5)                                = 0
memfd_create("runc_cloned:/proc/self/exe", MFD_CLOEXEC|MFD_ALLOW_SEALING) = 5
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 6
sendfile(5, 6, NULL, 2147479552)        = 17501832
close(6)                                = 0
fcntl(5, F_ADD_SEALS, F_SEAL_SEAL|F_SEAL_SHRINK|F_SEAL_GROW|F_SEAL_WRITE) = 0
execve("/proc/self/fd/5", ["runc", "init"], [/* 4 vars */]) = 0
brk(NULL)                               = 0x55ab0ddd7000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb7daa50000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=221036, ...}) = 0
mmap(NULL, 221036, PROT_READ, MAP_PRIVATE, 5, 0) = 0x7fb7daa1a000
close(5)                                = 0
open("/lib64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260l\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=141968, ...}) = 0
mmap(NULL, 2208904, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fb7da614000
mprotect(0x7fb7da62b000, 2093056, PROT_NONE) = 0
mmap(0x7fb7da82a000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x16000) = 0x7fb7da82a000
mmap(0x7fb7da82c000, 13448, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fb7da82c000
close(5)                                = 0
open("/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\r\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=19288, ...}) = 0
mmap(NULL, 2109712, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fb7da410000
mprotect(0x7fb7da412000, 2097152, PROT_NONE) = 0
mmap(0x7fb7da612000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2000) = 0x7fb7da612000
close(5)                                = 0
open("/lib64/libseccomp.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\360\1\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=266672, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb7daa19000
mmap(NULL, 2359552, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fb7da1cf000
mprotect(0x7fb7da1fb000, 2093056, PROT_NONE) = 0
mmap(0x7fb7da3fa000, 90112, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2b000) = 0x7fb7da3fa000
close(5)                                = 0
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340$\2\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=2151672, ...}) = 0
mmap(NULL, 3981792, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fb7d9e02000
mprotect(0x7fb7d9fc4000, 2097152, PROT_NONE) = 0
mmap(0x7fb7da1c4000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x1c2000) = 0x7fb7da1c4000
mmap(0x7fb7da1ca000, 16864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fb7da1ca000
close(5)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb7daa18000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb7daa16000
arch_prctl(ARCH_SET_FS, 0x7fb7daa16740) = 0
mprotect(0x7fb7da1c4000, 16384, PROT_READ) = 0
mprotect(0x7fb7da3fa000, 86016, PROT_READ) = 0
mprotect(0x7fb7da612000, 4096, PROT_READ) = 0
mprotect(0x7fb7da82a000, 4096, PROT_READ) = 0
mprotect(0x55ab0c093000, 3280896, PROT_READ) = 0
mprotect(0x7fb7daa51000, 4096, PROT_READ) = 0
munmap(0x7fb7daa1a000, 221036)          = 0
set_tid_address(0x7fb7daa16a10)         = 21643
set_robust_list(0x7fb7daa16a20, 24)     = 0
rt_sigaction(SIGRTMIN, {0x7fb7da61a790, [], SA_RESTORER|SA_SIGINFO, 0x7fb7da6235d0}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7fb7da61a820, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7fb7da6235d0}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 5
fcntl(5, F_GET_SEALS)                   = -1 EINVAL (Invalid argument)
close(5)                                = 0
open("/proc/self/cmdline", O_RDONLY|O_CLOEXEC) = 5
read(5, "runc\0init\0", 4096)           = 10
brk(NULL)                               = 0x55ab0ddd7000
brk(0x55ab0ddf8000)                     = 0x55ab0ddf8000
brk(NULL)                               = 0x55ab0ddf8000
read(5, "", 4096)                       = 0
close(5)                                = 0
open("/proc/self/environ", O_RDONLY|O_CLOEXEC) = 5
read(5, "GOMAXPROCS=2\0_LIBCONTAINER_INITP"..., 4096) = 93
read(5, "", 4096)                       = 0
close(5)                                = 0
memfd_create("runc_cloned:/proc/self/exe", MFD_CLOEXEC|MFD_ALLOW_SEALING) = 5
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 6
sendfile(5, 6, NULL, 2147479552)        = 17501832
close(6)                                = 0
fcntl(5, F_ADD_SEALS, F_SEAL_SEAL|F_SEAL_SHRINK|F_SEAL_GROW|F_SEAL_WRITE) = 0
execve("/proc/self/fd/5", ["runc", "init"], [/* 4 vars */]) = 0
brk(NULL)                               = 0x558609cad000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7effa7a7d000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=221036, ...}) = 0
mmap(NULL, 221036, PROT_READ, MAP_PRIVATE, 5, 0) = 0x7effa7a47000
close(5)                                = 0
open("/lib64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260l\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=141968, ...}) = 0
mmap(NULL, 2208904, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7effa7641000
mprotect(0x7effa7658000, 2093056, PROT_NONE) = 0
mmap(0x7effa7857000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x16000) = 0x7effa7857000
mmap(0x7effa7859000, 13448, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7effa7859000
close(5)                                = 0
open("/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\r\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=19288, ...}) = 0
mmap(NULL, 2109712, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7effa743d000
mprotect(0x7effa743f000, 2097152, PROT_NONE) = 0
mmap(0x7effa763f000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2000) = 0x7effa763f000
close(5)                                = 0
open("/lib64/libseccomp.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\360\1\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=266672, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7effa7a46000
mmap(NULL, 2359552, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7effa71fc000
mprotect(0x7effa7228000, 2093056, PROT_NONE) = 0
mmap(0x7effa7427000, 90112, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2b000) = 0x7effa7427000
close(5)                                = 0
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340$\2\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=2151672, ...}) = 0
mmap(NULL, 3981792, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7effa6e2f000
mprotect(0x7effa6ff1000, 2097152, PROT_NONE) = 0
mmap(0x7effa71f1000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x1c2000) = 0x7effa71f1000
mmap(0x7effa71f7000, 16864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7effa71f7000
close(5)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7effa7a45000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7effa7a43000
arch_prctl(ARCH_SET_FS, 0x7effa7a43740) = 0
mprotect(0x7effa71f1000, 16384, PROT_READ) = 0
mprotect(0x7effa7427000, 86016, PROT_READ) = 0
mprotect(0x7effa763f000, 4096, PROT_READ) = 0
mprotect(0x7effa7857000, 4096, PROT_READ) = 0
mprotect(0x558608ed4000, 3280896, PROT_READ) = 0
mprotect(0x7effa7a7e000, 4096, PROT_READ) = 0
munmap(0x7effa7a47000, 221036)          = 0
set_tid_address(0x7effa7a43a10)         = 21643
set_robust_list(0x7effa7a43a20, 24)     = 0
rt_sigaction(SIGRTMIN, {0x7effa7647790, [], SA_RESTORER|SA_SIGINFO, 0x7effa76505d0}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7effa7647820, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7effa76505d0}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 5
fcntl(5, F_GET_SEALS)                   = -1 EINVAL (Invalid argument)
close(5)                                = 0
open("/proc/self/cmdline", O_RDONLY|O_CLOEXEC) = 5
read(5, "runc\0init\0", 4096)           = 10
brk(NULL)                               = 0x558609cad000
brk(0x558609cce000)                     = 0x558609cce000
brk(NULL)                               = 0x558609cce000
read(5, "", 4096)                       = 0
close(5)                                = 0
open("/proc/self/environ", O_RDONLY|O_CLOEXEC) = 5
read(5, "GOMAXPROCS=2\0_LIBCONTAINER_INITP"..., 4096) = 93
read(5, "", 4096)                       = 0
close(5)                                = 0
memfd_create("runc_cloned:/proc/self/exe", MFD_CLOEXEC|MFD_ALLOW_SEALING) = 5
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 6
sendfile(5, 6, NULL, 2147479552)        = 17501832
close(6)                                = 0
fcntl(5, F_ADD_SEALS, F_SEAL_SEAL|F_SEAL_SHRINK|F_SEAL_GROW|F_SEAL_WRITE) = 0
execve("/proc/self/fd/5", ["runc", "init"], [/* 4 vars */]) = 0
brk(NULL)                               = 0x561ebbd7d000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd95b85c000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=221036, ...}) = 0
mmap(NULL, 221036, PROT_READ, MAP_PRIVATE, 5, 0) = 0x7fd95b826000
close(5)                                = 0
open("/lib64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260l\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=141968, ...}) = 0
mmap(NULL, 2208904, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fd95b420000
mprotect(0x7fd95b437000, 2093056, PROT_NONE) = 0
mmap(0x7fd95b636000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x16000) = 0x7fd95b636000
mmap(0x7fd95b638000, 13448, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fd95b638000
close(5)                                = 0
open("/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\r\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=19288, ...}) = 0
mmap(NULL, 2109712, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fd95b21c000
mprotect(0x7fd95b21e000, 2097152, PROT_NONE) = 0
mmap(0x7fd95b41e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2000) = 0x7fd95b41e000
close(5)                                = 0
open("/lib64/libseccomp.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\360\1\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=266672, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd95b825000
mmap(NULL, 2359552, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fd95afdb000
mprotect(0x7fd95b007000, 2093056, PROT_NONE) = 0
mmap(0x7fd95b206000, 90112, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2b000) = 0x7fd95b206000
close(5)                                = 0
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340$\2\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=2151672, ...}) = 0
mmap(NULL, 3981792, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fd95ac0e000
mprotect(0x7fd95add0000, 2097152, PROT_NONE) = 0
mmap(0x7fd95afd0000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x1c2000) = 0x7fd95afd0000
mmap(0x7fd95afd6000, 16864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fd95afd6000
close(5)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd95b824000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd95b822000
arch_prctl(ARCH_SET_FS, 0x7fd95b822740) = 0
mprotect(0x7fd95afd0000, 16384, PROT_READ) = 0
mprotect(0x7fd95b206000, 86016, PROT_READ) = 0
mprotect(0x7fd95b41e000, 4096, PROT_READ) = 0
mprotect(0x7fd95b636000, 4096, PROT_READ) = 0
mprotect(0x561ebb06a000, 3280896, PROT_READ) = 0
mprotect(0x7fd95b85d000, 4096, PROT_READ) = 0
munmap(0x7fd95b826000, 221036)          = 0
set_tid_address(0x7fd95b822a10)         = 21643
set_robust_list(0x7fd95b822a20, 24)     = 0
rt_sigaction(SIGRTMIN, {0x7fd95b426790, [], SA_RESTORER|SA_SIGINFO, 0x7fd95b42f5d0}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7fd95b426820, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7fd95b42f5d0}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 5
fcntl(5, F_GET_SEALS)                   = -1 EINVAL (Invalid argument)
close(5)                                = 0
open("/proc/self/cmdline", O_RDONLY|O_CLOEXEC) = 5
read(5, "runc\0init\0", 4096)           = 10
brk(NULL)                               = 0x561ebbd7d000
brk(0x561ebbd9e000)                     = 0x561ebbd9e000
brk(NULL)                               = 0x561ebbd9e000
read(5, "", 4096)                       = 0
close(5)                                = 0
open("/proc/self/environ", O_RDONLY|O_CLOEXEC) = 5
read(5, "GOMAXPROCS=2\0_LIBCONTAINER_INITP"..., 4096) = 93
read(5, "", 4096)                       = 0
close(5)                                = 0
memfd_create("runc_cloned:/proc/self/exe", MFD_CLOEXEC|MFD_ALLOW_SEALING) = 5
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 6
sendfile(5, 6, NULL, 2147479552)        = 17501832
close(6)                                = 0
fcntl(5, F_ADD_SEALS, F_SEAL_SEAL|F_SEAL_SHRINK|F_SEAL_GROW|F_SEAL_WRITE) = 0
execve("/proc/self/fd/5", ["runc", "init"], [/* 4 vars */]) = 0
brk(NULL)                               = 0x55c4cf02b000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f859b62b000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=221036, ...}) = 0
mmap(NULL, 221036, PROT_READ, MAP_PRIVATE, 5, 0) = 0x7f859b5f5000
close(5)                                = 0
open("/lib64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260l\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=141968, ...}) = 0
mmap(NULL, 2208904, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7f859b1ef000
mprotect(0x7f859b206000, 2093056, PROT_NONE) = 0
mmap(0x7f859b405000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x16000) = 0x7f859b405000
mmap(0x7f859b407000, 13448, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f859b407000
close(5)                                = 0
open("/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\r\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=19288, ...}) = 0
mmap(NULL, 2109712, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7f859afeb000
mprotect(0x7f859afed000, 2097152, PROT_NONE) = 0
mmap(0x7f859b1ed000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2000) = 0x7f859b1ed000
close(5)                                = 0
open("/lib64/libseccomp.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\360\1\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=266672, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f859b5f4000
mmap(NULL, 2359552, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7f859adaa000
mprotect(0x7f859add6000, 2093056, PROT_NONE) = 0
mmap(0x7f859afd5000, 90112, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2b000) = 0x7f859afd5000
close(5)                                = 0
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340$\2\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=2151672, ...}) = 0
mmap(NULL, 3981792, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7f859a9dd000
mprotect(0x7f859ab9f000, 2097152, PROT_NONE) = 0
mmap(0x7f859ad9f000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x1c2000) = 0x7f859ad9f000
mmap(0x7f859ada5000, 16864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f859ada5000
close(5)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f859b5f3000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f859b5f1000
arch_prctl(ARCH_SET_FS, 0x7f859b5f1740) = 0
mprotect(0x7f859ad9f000, 16384, PROT_READ) = 0
mprotect(0x7f859afd5000, 86016, PROT_READ) = 0
mprotect(0x7f859b1ed000, 4096, PROT_READ) = 0
mprotect(0x7f859b405000, 4096, PROT_READ) = 0
mprotect(0x55c4cd20f000, 3280896, PROT_READ) = 0
mprotect(0x7f859b62c000, 4096, PROT_READ) = 0
munmap(0x7f859b5f5000, 221036)          = 0
set_tid_address(0x7f859b5f1a10)         = 21643
set_robust_list(0x7f859b5f1a20, 24)     = 0
rt_sigaction(SIGRTMIN, {0x7f859b1f5790, [], SA_RESTORER|SA_SIGINFO, 0x7f859b1fe5d0}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7f859b1f5820, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7f859b1fe5d0}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 5
fcntl(5, F_GET_SEALS)                   = -1 EINVAL (Invalid argument)
close(5)                                = 0
open("/proc/self/cmdline", O_RDONLY|O_CLOEXEC) = 5
read(5, "runc\0init\0", 4096)           = 10
brk(NULL)                               = 0x55c4cf02b000
brk(0x55c4cf04c000)                     = 0x55c4cf04c000
brk(NULL)                               = 0x55c4cf04c000
read(5, "", 4096)                       = 0
close(5)                                = 0
open("/proc/self/environ", O_RDONLY|O_CLOEXEC) = 5
read(5, "GOMAXPROCS=2\0_LIBCONTAINER_INITP"..., 4096) = 93
read(5, "", 4096)                       = 0
close(5)                                = 0
memfd_create("runc_cloned:/proc/self/exe", MFD_CLOEXEC|MFD_ALLOW_SEALING) = 5
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 6
sendfile(5, 6, NULL, 2147479552)        = 17501832
close(6)                                = 0
fcntl(5, F_ADD_SEALS, F_SEAL_SEAL|F_SEAL_SHRINK|F_SEAL_GROW|F_SEAL_WRITE) = 0
execve("/proc/self/fd/5", ["runc", "init"], [/* 4 vars */]) = 0
brk(NULL)                               = 0x55e91f182000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7d038e1000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=221036, ...}) = 0
mmap(NULL, 221036, PROT_READ, MAP_PRIVATE, 5, 0) = 0x7f7d038ab000
close(5)                                = 0
open("/lib64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260l\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=141968, ...}) = 0
mmap(NULL, 2208904, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7f7d034a5000
mprotect(0x7f7d034bc000, 2093056, PROT_NONE) = 0
mmap(0x7f7d036bb000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x16000) = 0x7f7d036bb000
mmap(0x7f7d036bd000, 13448, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f7d036bd000
close(5)                                = 0
open("/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\r\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=19288, ...}) = 0
mmap(NULL, 2109712, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7f7d032a1000
mprotect(0x7f7d032a3000, 2097152, PROT_NONE) = 0
mmap(0x7f7d034a3000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2000) = 0x7f7d034a3000
close(5)                                = 0
open("/lib64/libseccomp.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\360\1\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=266672, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7d038aa000
mmap(NULL, 2359552, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7f7d03060000
mprotect(0x7f7d0308c000, 2093056, PROT_NONE) = 0
mmap(0x7f7d0328b000, 90112, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2b000) = 0x7f7d0328b000
close(5)                                = 0
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340$\2\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=2151672, ...}) = 0
mmap(NULL, 3981792, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7f7d02c93000
mprotect(0x7f7d02e55000, 2097152, PROT_NONE) = 0
mmap(0x7f7d03055000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x1c2000) = 0x7f7d03055000
mmap(0x7f7d0305b000, 16864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f7d0305b000
close(5)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7d038a9000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7d038a7000
arch_prctl(ARCH_SET_FS, 0x7f7d038a7740) = 0
mprotect(0x7f7d03055000, 16384, PROT_READ) = 0
mprotect(0x7f7d0328b000, 86016, PROT_READ) = 0
mprotect(0x7f7d034a3000, 4096, PROT_READ) = 0
mprotect(0x7f7d036bb000, 4096, PROT_READ) = 0
mprotect(0x55e91ced9000, 3280896, PROT_READ) = 0
mprotect(0x7f7d038e2000, 4096, PROT_READ) = 0
munmap(0x7f7d038ab000, 221036)          = 0
set_tid_address(0x7f7d038a7a10)         = 21643
set_robust_list(0x7f7d038a7a20, 24)     = 0
rt_sigaction(SIGRTMIN, {0x7f7d034ab790, [], SA_RESTORER|SA_SIGINFO, 0x7f7d034b45d0}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7f7d034ab820, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7f7d034b45d0}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 5
fcntl(5, F_GET_SEALS)                   = -1 EINVAL (Invalid argument)
close(5)                                = 0
open("/proc/self/cmdline", O_RDONLY|O_CLOEXEC) = 5
read(5, "runc\0init\0", 4096)           = 10
brk(NULL)                               = 0x55e91f182000
brk(0x55e91f1a3000)                     = 0x55e91f1a3000
brk(NULL)                               = 0x55e91f1a3000
read(5, "", 4096)                       = 0
close(5)                                = 0
open("/proc/self/environ", O_RDONLY|O_CLOEXEC) = 5
read(5, "GOMAXPROCS=2\0_LIBCONTAINER_INITP"..., 4096) = 93
read(5, "", 4096)                       = 0
close(5)                                = 0
memfd_create("runc_cloned:/proc/self/exe", MFD_CLOEXEC|MFD_ALLOW_SEALING) = 5
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 6
sendfile(5, 6, NULL, 2147479552)        = 17501832
close(6)                                = 0
fcntl(5, F_ADD_SEALS, F_SEAL_SEAL|F_SEAL_SHRINK|F_SEAL_GROW|F_SEAL_WRITE) = 0
execve("/proc/self/fd/5", ["runc", "init"], [/* 4 vars */]) = 0
brk(NULL)                               = 0x55ea95005000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fee8a8c0000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=221036, ...}) = 0
mmap(NULL, 221036, PROT_READ, MAP_PRIVATE, 5, 0) = 0x7fee8a88a000
close(5)                                = 0
open("/lib64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260l\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=141968, ...}) = 0
mmap(NULL, 2208904, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fee8a484000
mprotect(0x7fee8a49b000, 2093056, PROT_NONE) = 0
mmap(0x7fee8a69a000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x16000) = 0x7fee8a69a000
mmap(0x7fee8a69c000, 13448, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fee8a69c000
close(5)                                = 0
open("/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\r\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=19288, ...}) = 0
mmap(NULL, 2109712, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fee8a280000
mprotect(0x7fee8a282000, 2097152, PROT_NONE) = 0
mmap(0x7fee8a482000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2000) = 0x7fee8a482000
close(5)                                = 0
open("/lib64/libseccomp.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\360\1\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=266672, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fee8a889000
mmap(NULL, 2359552, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fee8a03f000
mprotect(0x7fee8a06b000, 2093056, PROT_NONE) = 0
mmap(0x7fee8a26a000, 90112, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2b000) = 0x7fee8a26a000
close(5)                                = 0
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340$\2\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=2151672, ...}) = 0
mmap(NULL, 3981792, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fee89c72000
mprotect(0x7fee89e34000, 2097152, PROT_NONE) = 0
mmap(0x7fee8a034000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x1c2000) = 0x7fee8a034000
mmap(0x7fee8a03a000, 16864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fee8a03a000
close(5)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fee8a888000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fee8a886000
arch_prctl(ARCH_SET_FS, 0x7fee8a886740) = 0
mprotect(0x7fee8a034000, 16384, PROT_READ) = 0
mprotect(0x7fee8a26a000, 86016, PROT_READ) = 0
mprotect(0x7fee8a482000, 4096, PROT_READ) = 0
mprotect(0x7fee8a69a000, 4096, PROT_READ) = 0
mprotect(0x55ea930c9000, 3280896, PROT_READ) = 0
mprotect(0x7fee8a8c1000, 4096, PROT_READ) = 0
munmap(0x7fee8a88a000, 221036)          = 0
set_tid_address(0x7fee8a886a10)         = 21643
set_robust_list(0x7fee8a886a20, 24)     = 0
rt_sigaction(SIGRTMIN, {0x7fee8a48a790, [], SA_RESTORER|SA_SIGINFO, 0x7fee8a4935d0}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7fee8a48a820, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7fee8a4935d0}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 5
fcntl(5, F_GET_SEALS)                   = -1 EINVAL (Invalid argument)
close(5)                                = 0
open("/proc/self/cmdline", O_RDONLY|O_CLOEXEC) = 5
read(5, "runc\0init\0", 4096)           = 10
brk(NULL)                               = 0x55ea95005000
brk(0x55ea95026000)                     = 0x55ea95026000
brk(NULL)                               = 0x55ea95026000
read(5, "", 4096)                       = 0
close(5)                                = 0
open("/proc/self/environ", O_RDONLY|O_CLOEXEC) = 5
read(5, "GOMAXPROCS=2\0_LIBCONTAINER_INITP"..., 4096) = 93
read(5, "", 4096)                       = 0
close(5)                                = 0
memfd_create("runc_cloned:/proc/self/exe", MFD_CLOEXEC|MFD_ALLOW_SEALING) = 5
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 6
sendfile(5, 6, NULL, 2147479552)        = 17501832
close(6)                                = 0
fcntl(5, F_ADD_SEALS, F_SEAL_SEAL|F_SEAL_SHRINK|F_SEAL_GROW|F_SEAL_WRITE) = 0
execve("/proc/self/fd/5", ["runc", "init"], [/* 4 vars */]) = 0
brk(NULL)                               = 0x558a7bd81000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fefec72d000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=221036, ...}) = 0
mmap(NULL, 221036, PROT_READ, MAP_PRIVATE, 5, 0) = 0x7fefec6f7000
close(5)                                = 0
open("/lib64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260l\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=141968, ...}) = 0
mmap(NULL, 2208904, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fefec2f1000
mprotect(0x7fefec308000, 2093056, PROT_NONE) = 0
mmap(0x7fefec507000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x16000) = 0x7fefec507000
mmap(0x7fefec509000, 13448, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fefec509000
close(5)                                = 0
open("/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\r\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=19288, ...}) = 0
mmap(NULL, 2109712, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fefec0ed000
mprotect(0x7fefec0ef000, 2097152, PROT_NONE) = 0
mmap(0x7fefec2ef000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2000) = 0x7fefec2ef000
close(5)                                = 0
open("/lib64/libseccomp.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\360\1\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=266672, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fefec6f6000
mmap(NULL, 2359552, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fefebeac000
mprotect(0x7fefebed8000, 2093056, PROT_NONE) = 0
mmap(0x7fefec0d7000, 90112, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2b000) = 0x7fefec0d7000
close(5)                                = 0
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340$\2\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=2151672, ...}) = 0
mmap(NULL, 3981792, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fefebadf000
mprotect(0x7fefebca1000, 2097152, PROT_NONE) = 0
mmap(0x7fefebea1000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x1c2000) = 0x7fefebea1000
mmap(0x7fefebea7000, 16864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fefebea7000
close(5)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fefec6f5000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fefec6f3000
arch_prctl(ARCH_SET_FS, 0x7fefec6f3740) = 0
mprotect(0x7fefebea1000, 16384, PROT_READ) = 0
mprotect(0x7fefec0d7000, 86016, PROT_READ) = 0
mprotect(0x7fefec2ef000, 4096, PROT_READ) = 0
mprotect(0x7fefec507000, 4096, PROT_READ) = 0
mprotect(0x558a7a114000, 3280896, PROT_READ) = 0
mprotect(0x7fefec72e000, 4096, PROT_READ) = 0
munmap(0x7fefec6f7000, 221036)          = 0
set_tid_address(0x7fefec6f3a10)         = 21643
set_robust_list(0x7fefec6f3a20, 24)     = 0
rt_sigaction(SIGRTMIN, {0x7fefec2f7790, [], SA_RESTORER|SA_SIGINFO, 0x7fefec3005d0}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7fefec2f7820, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7fefec3005d0}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 5
fcntl(5, F_GET_SEALS)                   = -1 EINVAL (Invalid argument)
close(5)                                = 0
open("/proc/self/cmdline", O_RDONLY|O_CLOEXEC) = 5
read(5, "runc\0init\0", 4096)           = 10
brk(NULL)                               = 0x558a7bd81000
brk(0x558a7bda2000)                     = 0x558a7bda2000
brk(NULL)                               = 0x558a7bda2000
read(5, "", 4096)                       = 0
close(5)                                = 0
open("/proc/self/environ", O_RDONLY|O_CLOEXEC) = 5
read(5, "GOMAXPROCS=2\0_LIBCONTAINER_INITP"..., 4096) = 93
read(5, "", 4096)                       = 0
close(5)                                = 0
memfd_create("runc_cloned:/proc/self/exe", MFD_CLOEXEC|MFD_ALLOW_SEALING) = 5
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 6
sendfile(5, 6, NULL, 2147479552)        = 17501832
close(6)                                = 0
fcntl(5, F_ADD_SEALS, F_SEAL_SEAL|F_SEAL_SHRINK|F_SEAL_GROW|F_SEAL_WRITE) = 0
execve("/proc/self/fd/5", ["runc", "init"], [/* 4 vars */]) = 0
brk(NULL)                               = 0x562f03ea0000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fda0c7aa000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=221036, ...}) = 0
mmap(NULL, 221036, PROT_READ, MAP_PRIVATE, 5, 0) = 0x7fda0c774000
close(5)                                = 0
open("/lib64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260l\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=141968, ...}) = 0
mmap(NULL, 2208904, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fda0c36e000
mprotect(0x7fda0c385000, 2093056, PROT_NONE) = 0
mmap(0x7fda0c584000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x16000) = 0x7fda0c584000
mmap(0x7fda0c586000, 13448, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fda0c586000
close(5)                                = 0
open("/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\r\0\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=19288, ...}) = 0
mmap(NULL, 2109712, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fda0c16a000
mprotect(0x7fda0c16c000, 2097152, PROT_NONE) = 0
mmap(0x7fda0c36c000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2000) = 0x7fda0c36c000
close(5)                                = 0
open("/lib64/libseccomp.so.2", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\360\1\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=266672, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fda0c773000
mmap(NULL, 2359552, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fda0bf29000
mprotect(0x7fda0bf55000, 2093056, PROT_NONE) = 0
mmap(0x7fda0c154000, 90112, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x2b000) = 0x7fda0c154000
close(5)                                = 0
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 5
read(5, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340$\2\0\0\0\0\0"..., 832) = 832
fstat(5, {st_mode=S_IFREG|0755, st_size=2151672, ...}) = 0
mmap(NULL, 3981792, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 5, 0) = 0x7fda0bb5c000
mprotect(0x7fda0bd1e000, 2097152, PROT_NONE) = 0
mmap(0x7fda0bf1e000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 5, 0x1c2000) = 0x7fda0bf1e000
mmap(0x7fda0bf24000, 16864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fda0bf24000
close(5)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fda0c772000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fda0c770000
arch_prctl(ARCH_SET_FS, 0x7fda0c770740) = 0
mprotect(0x7fda0bf1e000, 16384, PROT_READ) = 0
mprotect(0x7fda0c154000, 86016, PROT_READ) = 0
mprotect(0x7fda0c36c000, 4096, PROT_READ) = 0
mprotect(0x7fda0c584000, 4096, PROT_READ) = 0
mprotect(0x562f01bff000, 3280896, PROT_READ) = 0
mprotect(0x7fda0c7ab000, 4096, PROT_READ) = 0
munmap(0x7fda0c774000, 221036)          = 0
set_tid_address(0x7fda0c770a10)         = 21643
set_robust_list(0x7fda0c770a20, 24)     = 0
rt_sigaction(SIGRTMIN, {0x7fda0c374790, [], SA_RESTORER|SA_SIGINFO, 0x7fda0c37d5d0}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7fda0c374820, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7fda0c37d5d0}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 5
fcntl(5, F_GET_SEALS)                   = -1 EINVAL (Invalid argument)
close(5)                                = 0
open("/proc/self/cmdline", O_RDONLY|O_CLOEXEC) = 5
read(5, "runc\0init\0", 4096)           = 10
brk(NULL)                               = 0x562f03ea0000
brk(0x562f03ec1000)                     = 0x562f03ec1000
brk(NULL)                               = 0x562f03ec1000
read(5, "", 4096)                       = 0
close(5)                                = 0
open("/proc/self/environ", O_RDONLY|O_CLOEXEC) = 5
read(5, "GOMAXPROCS=2\0_LIBCONTAINER_INITP"..., 4096) = 93
read(5, "", 4096)                       = 0
close(5)                                = 0
memfd_create("runc_cloned:/proc/self/exe", MFD_CLOEXEC|MFD_ALLOW_SEALING) = 5
open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 6
sendfile(5, 6, NULL, 2147479552^Cstrace: Process 21643 detached
 <detached ...>

@cyphar
Copy link
Member

cyphar commented Feb 18, 2019

To me this looks like a RHEL kernel bug -- fcntl(F_ADD_SEALS) works but fcntl(F_GET_SEALS) wasn't backported correctly? #1984 is going to be improving the fallback mechanism, but if we need to handle F_GET_SEALS being broken on RHEL this is going to be a whole different level of pain...

@pinacoelho
Copy link
Author

@cyphar do you have a code fragment that I can run to test that ? Alternatively, a flag that disables this functionality would allow runc to continue until the causing issue is fixed.

@cyphar
Copy link
Member

cyphar commented Feb 18, 2019

#1984 adds a "flag" (rather an environment variable) to disable memfd_create(2) and instead fall-back to creating a temporary file. The code in question is very central to the fix for CVE-2019-5736 so you can't really turn it off.

A simple code fragment would be something like the following (it should not give you any failures if memfd_create(2) is supported, or it should give you an error when trying to do memfd_create(2)).

Check script.
#define _GNU_SOURCE
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <errno.h>
#include <sys/syscall.h>

/* Use our own wrapper for memfd_create. */
#if !defined(SYS_memfd_create) && defined(__NR_memfd_create)
#  define SYS_memfd_create __NR_memfd_create
#endif
/* memfd_create(2) flags -- copied from <linux/memfd.h>. */
#ifndef MFD_CLOEXEC
#  define MFD_CLOEXEC       0x0001U
#  define MFD_ALLOW_SEALING 0x0002U
#endif
int memfd_create(const char *name, unsigned int flags)
{
#ifdef SYS_memfd_create
	return syscall(SYS_memfd_create, name, flags);
#else
	errno = ENOSYS;
	return -1;
#endif
}

/* This comes directly from <linux/fcntl.h>. */
#ifndef F_LINUX_SPECIFIC_BASE
#  define F_LINUX_SPECIFIC_BASE 1024
#endif
#ifndef F_ADD_SEALS
#  define F_ADD_SEALS (F_LINUX_SPECIFIC_BASE + 9)
#  define F_GET_SEALS (F_LINUX_SPECIFIC_BASE + 10)
#endif
#ifndef F_SEAL_SEAL
#  define F_SEAL_SEAL   0x0001	/* prevent further seals from being set */
#  define F_SEAL_SHRINK 0x0002	/* prevent file from shrinking */
#  define F_SEAL_GROW   0x0004	/* prevent file from growing */
#  define F_SEAL_WRITE  0x0008	/* prevent writes */
#endif

#define RUNC_MEMFD_COMMENT "runc_cloned:/proc/self/exe"
#define RUNC_MEMFD_SEALS \
	(F_SEAL_SEAL | F_SEAL_SHRINK | F_SEAL_GROW | F_SEAL_WRITE)

#define bail(msg) \
	do { perror(msg); exit(1); } while (0)

int main(void)
{
	int fd = memfd_create(RUNC_MEMFD_COMMENT, MFD_CLOEXEC | MFD_ALLOW_SEALING);
	if (fd < 0)
		bail("memfd_create failure");

	if (fcntl(fd, F_ADD_SEALS, RUNC_MEMFD_SEALS) < 0)
		bail("f_add_seals failure");

	int seals = fcntl(fd, F_GET_SEALS);
	if (seals < 0)
		bail("f_get_seals failure");
	if (seals != RUNC_MEMFD_SEALS)
		bail("f_get_seals incorrect result");

	return 0;
}

@pinacoelho
Copy link
Author

Compiled with "cc -o checkseal checkseal.c", RC=0 (both as root and as a normal(*) user)

    • id => includes wheel, docker and context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
      Is it possible that it depends on how runc is lauched by docker ? (i.e. some priviledge that is droped before runc is invoked?)

@cyphar
Copy link
Member

cyphar commented Feb 18, 2019

The reason why it's infinite looping is because it keeps copying the binary, because it thinks that /proc/self/exe isn't a memfd:

fcntl(5, F_GET_SEALS) = -1 EINVAL (Invalid argument)

There really isn't another explanation. It's possible that there is a permission issue with SELinux that makes it not want to give F_GET_SEALS information -- can you try with setenforce 0 just to check whether it's an SELinux violation?

@pinacoelho
Copy link
Author

pinacoelho commented Feb 18, 2019

Upgraded to containerd.io 1.2.2-3.3

setenforce 0 ; getenforce ; runc --version
Permissive
runc version 1.0.0-rc6+dev
commit: 09c8266
spec: 1.0.1-dev
docker run alpine (hung again)

An lsof of the "runc init" shows (Note /memfd:runc_cloned:/proc/self/exe (deleted) on 3rd entry):

lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/21780/gvfs
      Output information may be incomplete.
COMMAND   PID USER   FD   TYPE             DEVICE SIZE/OFF    NODE NAME
5       26318 root  cwd    DIR               0,42     4096 7735336 /var/lib/docker/overlay2/32c33e9832c2e6a57c449b0f4f23efc5e2c07171dacf7f9a5dbb000324695013/merged
5       26318 root  rtd    DIR              253,2     4096       2 /
5       26318 root  txt    REG                0,4 17501832 8002804 /memfd:runc_cloned:/proc/self/exe (deleted)
5       26318 root  mem    REG              253,2  2151672 4326728 /usr/lib64/libc-2.17.so
5       26318 root  mem    REG              253,2   266672 4337037 /usr/lib64/libseccomp.so.2.3.1
5       26318 root  mem    REG              253,2    19288 4373930 /usr/lib64/libdl-2.17.so
5       26318 root  mem    REG              253,2   141968 4335778 /usr/lib64/libpthread-2.17.so
5       26318 root  mem    REG              253,2   163400 4333485 /usr/lib64/ld-2.17.so
5       26318 root  mem    REG              253,2   220837 1049900 /etc/ld.so.cache
5       26318 root    0r   CHR                1,3      0t0    2051 /dev/null
5       26318 root    1w  FIFO                0,9      0t0 7799544 pipe
5       26318 root    2w  FIFO                0,9      0t0 7799545 pipe
5       26318 root    3u  unix 0xffff996881638000      0t0 7926530 socket
5       26318 root    4u  FIFO               0,20      0t0 7926529 /run/docker/runtime-runc/moby/c1e3fce4862eac2bf3327a9831fb13dd74b192d0566149a761ecfc55de7b06c0/exec.fifo

@cyphar
Copy link
Member

cyphar commented Feb 19, 2019

This looks reasonable, and matches what I was thinking:

5 26318 root txt REG 0,4 17501832 8002804 /memfd:runc_cloned:/proc/self/exe (deleted)

Have you tried to run with SELinux disabled, or checked whether there was an AVC denial logged? Then again, the code definitely works on Fedora (I tested the mitigation on Fedora with SELinux before the patch was posted).

@cyphar
Copy link
Member

cyphar commented Feb 19, 2019

Looking at my patch in #1984 it actually looks like I already have a solution for this issue without needing to deal with the bad RHEL backport (as part of the O_TMPFILE fallback we check for st_nlink == 0 which is true for memfds as well) -- can you try the current version of #1984 and let me know whether it fixes the issue?

@pinacoelho
Copy link
Author

@cyphar I'm uncertain on how to use git to pull the correct version into my machine. Can you help me ?
(I'm assuming afterwards it's just compile and copy the runc executable over the one from the rpm)

@cyphar
Copy link
Member

cyphar commented Feb 19, 2019

I'm not sure if you already have a working Go environment set up, but if you don't here is how you'd do it:

% mkdir -p $HOME/go/src
% export GOPATH="$HOME/go"

And then you fetch runc:

% mkdir -p $GOPATH/src/github.com/opencontainers
% git clone https://github.com/opencontainers/runc $GOPATH/src/github.com/opencontainers/runc
% cd $GOPATH/src/github.com/opencontainers/runc

Then fetch my PR:

% git fetch origin pull/1984/head:pr-1984
% git checkout pr-1984

And then you can build it:

% make BUILDTAGS="apparmor selinux seccomp"
go build -buildmode=pie  -ldflags "-X main.gitCommit="89c07553177c66cfe07de11fcdc17b59621ead07" -X main.version=1.0.0-rc6+dev " -tags "apparmor selinux seccomp" -o runc .
make BUILDTAGS="apparmor selinux seccomp"  26.43s user 3.12s system 269% cpu 10.980 total

And finally, install it -- you want to replace the docker-runc that comes from the RPM.

@pinacoelho
Copy link
Author

pinacoelho commented Feb 19, 2019

Went well until the make:

[jpc@moykano ~/go/github.com/opencontainers/runc]$ make BUILDTAGS="apparmor selinux seccomp"
go build -buildmode=pie  -ldflags "-X main.gitCommit="89c07553177c66cfe07de11fcdc17b59621ead07" -X main.version=1.0.0-rc6+dev " -tags "apparmor selinux seccomp" -o runc .
tty.go:12:2: cannot find package "github.com/containerd/console" in any of:
	/usr/lib/golang/src/github.com/containerd/console (from $GOROOT)
	/home/jpc/go/src/github.com/containerd/console (from $GOPATH)
utils_linux.go:22:2: cannot find package "github.com/coreos/go-systemd/activation" in any of:
	/usr/lib/golang/src/github.com/coreos/go-systemd/activation (from $GOROOT)
	/home/jpc/go/src/github.com/coreos/go-systemd/activation (from $GOPATH)
update.go:11:2: cannot find package "github.com/docker/go-units" in any of:
	/usr/lib/golang/src/github.com/docker/go-units (from $GOROOT)
	/home/jpc/go/src/github.com/docker/go-units (from $GOPATH)
checkpoint.go:11:2: cannot find package "github.com/opencontainers/runc/libcontainer" in any of:
	/usr/lib/golang/src/github.com/opencontainers/runc/libcontainer (from $GOROOT)
	/home/jpc/go/src/github.com/opencontainers/runc/libcontainer (from $GOPATH)
events.go:13:2: cannot find package "github.com/opencontainers/runc/libcontainer/cgroups" in any of:
	/usr/lib/golang/src/github.com/opencontainers/runc/libcontainer/cgroups (from $GOROOT)
	/home/jpc/go/src/github.com/opencontainers/runc/libcontainer/cgroups (from $GOPATH)
utils_linux.go:15:2: cannot find package "github.com/opencontainers/runc/libcontainer/cgroups/systemd" in any of:
	/usr/lib/golang/src/github.com/opencontainers/runc/libcontainer/cgroups/systemd (from $GOROOT)
	/home/jpc/go/src/github.com/opencontainers/runc/libcontainer/cgroups/systemd (from $GOPATH)
spec.go:11:2: cannot find package "github.com/opencontainers/runc/libcontainer/configs" in any of:
	/usr/lib/golang/src/github.com/opencontainers/runc/libcontainer/configs (from $GOROOT)
	/home/jpc/go/src/github.com/opencontainers/runc/libcontainer/configs (from $GOPATH)
events.go:14:2: cannot find package "github.com/opencontainers/runc/libcontainer/intelrdt" in any of:
	/usr/lib/golang/src/github.com/opencontainers/runc/libcontainer/intelrdt (from $GOROOT)
	/home/jpc/go/src/github.com/opencontainers/runc/libcontainer/intelrdt (from $GOPATH)
init.go:8:2: cannot find package "github.com/opencontainers/runc/libcontainer/nsenter" in any of:
	/usr/lib/golang/src/github.com/opencontainers/runc/libcontainer/nsenter (from $GOROOT)
	/home/jpc/go/src/github.com/opencontainers/runc/libcontainer/nsenter (from $GOPATH)
spec.go:12:2: cannot find package "github.com/opencontainers/runc/libcontainer/specconv" in any of:
	/usr/lib/golang/src/github.com/opencontainers/runc/libcontainer/specconv (from $GOROOT)
	/home/jpc/go/src/github.com/opencontainers/runc/libcontainer/specconv (from $GOPATH)
checkpoint.go:12:2: cannot find package "github.com/opencontainers/runc/libcontainer/system" in any of:
	/usr/lib/golang/src/github.com/opencontainers/runc/libcontainer/system (from $GOROOT)
	/home/jpc/go/src/github.com/opencontainers/runc/libcontainer/system (from $GOPATH)
list.go:17:2: cannot find package "github.com/opencontainers/runc/libcontainer/user" in any of:
	/usr/lib/golang/src/github.com/opencontainers/runc/libcontainer/user (from $GOROOT)
	/home/jpc/go/src/github.com/opencontainers/runc/libcontainer/user (from $GOPATH)
exec.go:13:2: cannot find package "github.com/opencontainers/runc/libcontainer/utils" in any of:
	/usr/lib/golang/src/github.com/opencontainers/runc/libcontainer/utils (from $GOROOT)
	/home/jpc/go/src/github.com/opencontainers/runc/libcontainer/utils (from $GOPATH)
checkpoint.go:13:2: cannot find package "github.com/opencontainers/runtime-spec/specs-go" in any of:
	/usr/lib/golang/src/github.com/opencontainers/runtime-spec/specs-go (from $GOROOT)
	/home/jpc/go/src/github.com/opencontainers/runtime-spec/specs-go (from $GOPATH)
checkpoint.go:15:2: cannot find package "github.com/urfave/cli" in any of:
	/usr/lib/golang/src/github.com/urfave/cli (from $GOROOT)
	/home/jpc/go/src/github.com/urfave/cli (from $GOPATH)
make: *** [runc] Error 1

@pinacoelho
Copy link
Author

pinacoelho commented Feb 19, 2019

Installed packages with "go get -v github.com/urfave/cli github.com/containerd/console github.com/coreos/go-systemd/activation github.com/opencontainers/runc/libcontainer github.com/opencontainers/runc/libcontainer/cgroups github.com/opencontainers/runc/libcontainer/cgroups/systemd github.com/opencontainers/runc/libcontainer/configs github.com/opencontainers/runc/libcontainer/configs github.com/opencontainers/runc/libcontainer/intelrdt github.com/opencontainers/runc/libcontainer/nsenter github.com/opencontainers/runc/libcontainer/specconv github.com/opencontainers/runc/libcontainer/specconv github.com/opencontainers/runc/libcontainer/system github.com/opencontainers/runc/libcontainer/user github.com/opencontainers/runc/libcontainer/utils github.com/opencontainers/runtime-spec/specs-go github.com/docker/go-units"

Now make yields:

[jpc@moykano ~/go/github.com/opencontainers/runc]$ make
go build -buildmode=pie  -ldflags "-X main.gitCommit="89c07553177c66cfe07de11fcdc17b59621ead07" -X main.version=1.0.0-rc6+dev " -tags "seccomp" -o runc .
# _/home/jpc/go/github.com/opencontainers/runc
./signals.go:136:28: cannot use ws (type "golang.org/x/sys/unix".WaitStatus) as type "github.com/opencontainers/runc/vendor/golang.org/x/sys/unix".WaitStatus in argument to utils.ExitStatus
./utils_linux.go:238:3: cannot use spec (type *"github.com/opencontainers/runtime-spec/specs-go".Spec) as type *"github.com/opencontainers/runc/vendor/github.com/opencontainers/runtime-spec/specs-go".Spec in field value
make: *** [runc] Error 2
$ go version
go version go1.11.4 linux/amd64

@cyphar
Copy link
Member

cyphar commented Feb 19, 2019

I made a mistake in my instructions, you should clone runc to $GOPATH/src/github.com/opencontainers/runc not $GOPATH/github.com/opencontainers/runc (on my machine I have a bunch of workarounds for this so I always forget how Go wants you to organise things). The go get workaround isn't right, and will mess around with things more than necessary.

This should fix your issue:

% mv $GOPATH/github.com $GOPATH/src

@pinacoelho
Copy link
Author

Now it compiles. Replaced /usr/sbin/runc with the new image.
[root@moykano runc]# mv /usr/sbin/runc /usr/sbin/runc.rpm
[root@moykano runc]# cp ~jpc/go/src/github.com/opencontainers/runc/runc /usr/sbin/runc
[root@moykano runc]# /usr/sbin/runc --version
runc version 1.0.0-rc6+dev
commit: 89c0755
spec: 1.0.1-dev

runc seems to run ok now (0.09s user+sys to exec an alpine container).
[root@moykano runc]# time docker run alpine
real 0m1.075s
user 0m0.053s
sys 0m0.039s

@pinacoelho
Copy link
Author

Did some more tests and looks like it's enjoying itself. :-)

What does the path forward look like ? Especially getting this translated into docker-ce, docker/for-linux, docker/runc and containerd & friends ? (I'm still trying to figure out the workflow)

@cyphar
Copy link
Member

cyphar commented Feb 19, 2019

Especially getting this translated into docker-ce, docker/for-linux, docker/runc and containerd & friends ? (I'm still trying to figure out the workflow)

Since runc is a separate binary (and completely separate project) to containerd and Docker, they have very little to do. Once it gets merged, then Docker will presumably update the pinned version they're using for their releases and so it'll be available in the next version (of docker-ce and Docker EE) -- distributions usually carry patches like this separately. containerd would do something similar. docker/runc is a mirror of this repository (with some patches added when Docker needs them) so nothing new there, and I don't know what docker/for-linux is.

@pinacoelho
Copy link
Author

The https://github.com/docker/docker-ce repository doesn't have issues.
Instead we're supposed to open them on per-platform issue-only repos (docker/for-{aws,azure,linux,mac,win})

Note: I'm still learning to use git/github, and the intersection of git & multiple projects is still a bit overwhelming, so:
@thaJeztah - Do you think I should ask for this to get pulled into docker-ce ? If so, in what repository should I open it ?
@cyphar - What's the next step I should take on this one: close with "89c0755 solves this" ? something else ?

@cyphar
Copy link
Member

cyphar commented Feb 20, 2019

Do you think I should ask for this to get pulled into docker-ce ? If so, in what repository should I open it?

Wait until #1984 is merged, and then you can send an update to https://github.com/moby/moby (which is where Docker is actually developed).

What's the next step I should take on this one: close with "89c0755 solves this" ? something else ?

You can leave it open, it'll be automatically closed when #1984 is merged.

@thaJeztah
Copy link
Member

Since runc is a separate binary (and completely separate project) to containerd and Docker, they have very little to do. Once it gets merged, then Docker will presumably update the pinned version they're using for their releases and so it'll be available in the next version (of docker-ce and Docker EE)

Correct; yes,

  • Docker 18.09 and up ship containerd and runc in a separate package (the containerd.io package currently bundles runc, which may change once runc reaches 1.0). The version of runc that is bundled will generally align with the version that is used in that version of containerd. For Docker 18.09.2 (which requires the containerd.io-1.2.2-3 package), a hot fix was applied to the runc binary, as the fix was still under embargo. (the package will re-align with upstream in the next release).
  • Older versions of docker bundle runc, and pin to a specific version of runc (they don't usually "bump" to the latest version of runc, as runc is still pre-1.0, and there have been some non-compatible changes in runc). For those releases, a fork is maintained that has long-living release-branches for those versions; https://github.com/docker/runc/branches). Docker back ports specific fixes to those branches if needed

Do you think I should ask for this to get pulled into docker-ce ? If so, in what repository should I open it?

This issue (as well as some other issues related to the runc CVE fix) is being tracked internally at Docker, and will be included in a future update. Depending on timing, that may either be as part of a new release of containerd, or as a packaging-only change for the containerd.io package.

@thaJeztah
Copy link
Member

Independent of the fix here in runc; even though the patch that's being worked on here would address the problem, there still looks to be a bug in the RHEL kernel.

I think that bug should be reported with Red Hat so that it doesn't get lost; @pinacoelho if you have an active Red Hat subscription, could you open a ticket with them? https://bugzilla.redhat.com/index.cgi

/cc @vbatts perhaps you know if this problem is already tracked?

@justincormack
Copy link
Contributor

justincormack commented Feb 20, 2019

I cannot replicate this with kernel 3.10.0-957.el7.x86_64 or 3.10.0-957.5.1.el7.x86_64, which is odd.

Tracing shows

fcntl(5, F_GET_SEALS)             = 0xf (seals F_SEAL_SEAL|F_SEAL_SHRINK|F_SEAL_GROW|F_SEAL_WRITE)

ie working as expected.

@cyphar
Copy link
Member

cyphar commented Feb 20, 2019

@justincormack Are you testing on the RHEL version of those kernels or the CentOS ones? My experience has shown that RHEL backports can be very different to CentOS ones.

@cyphar
Copy link
Member

cyphar commented Feb 21, 2019

@kolyshkin Managed to reproduce it in #1984.

@pinacoelho
Copy link
Author

I'm trying to figure out a minimal testcase to send to redhat. I expect it to run ok in cosmic cuttlefish (kernel V4) and rhel 7.6.

Trying not to get lost:
memfd_create("key",...), clone /proc/self/exe to memfd, seal memfd, execve memfd
After exec, /proc/self/exe is supposed to be a link to a name "/memfd:key") AND keep the F_SEAL_SEAL.

@cyphar
Copy link
Member

cyphar commented Feb 21, 2019

Sure, but the problem appears to be the F_ADD_SEALS and F_GET_SEALS aren't fully working on RHEL -- you could just point them to the kernel self-tests for memfd_create (which fail on RHEL kernels). See @kolyshkin's comment for more details.

@kolyshkin
Copy link
Contributor

I'm trying to figure out a minimal testcase to send to redhat.

@pinacoelho if that helps, I filed a bug to red hat, using the kernel selftest code: https://bugzilla.redhat.com/show_bug.cgi?id=1679829

@pinacoelho
Copy link
Author

pinacoelho commented Feb 22, 2019 via email

@vbatts
Copy link
Member

vbatts commented Feb 22, 2019

To be clear, the runc build that is having an issue, is it the runc package provided with RHEL?

@pinacoelho
Copy link
Author

pinacoelho commented Feb 22, 2019 via email

@vbatts
Copy link
Member

vbatts commented Feb 22, 2019

Have we confirmed that runc master has the issue? I've double checked this with the dev and test teams, and they're not seeing this loop.

The kernel passing a test-suite is another issue, and I'm glad that has been filed separately.

@pinacoelho
Copy link
Author

Have we confirmed that runc master has the issue?
If you mean opencontainers/runc master branch, I can test that during the weekend.

@jblaine
Copy link

jblaine commented Feb 23, 2019

Confirming that the following environment has the issue as well as I have only seen people saying "RHEL" in the various issues about this.

CentOS Linux release 7.6.1810 (Core)
kernel 3.10.0-957.1.3.el7.x86_64
containerd.io-1.2.2-3.3.el7.x86_64
docker-ce-18.09.2-3.el7.x86_64

docker run hello-world just goes nuts looping as described.

@ijumps
Copy link

ijumps commented Feb 24, 2019

I've tested this patch on centos(7.3 to 7.6) and rhel(7.3 and 7.6), all goes well without going looping.

You can reproduce this to get a vagrant rhel7 box (e.g. from https://app.vagrantup.com/generic/boxes/rhel7), run a vm and install docker and all other packages from centos7 repo.

[root@rhel7 ~]# uname -a
Linux rhel7.localdomain 3.10.0-957.el7.x86_64 #1 SMP Thu Oct 4 20:48:51 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

[root@rhel7 ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.6 (Maipo)

[root@rhel7 vagrant]# docker info
Containers: 8
 Running: 1
 Paused: 0
 Stopped: 7
Images: 2
Server Version: 18.09.2
Storage Driver: overlay2
 Backing Filesystem: xfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9754871865f7fe2f4e74d43e2fc7ccd237edcbce
runc version: 09c8266bf2fcf9519a651b04ae54c967b9ab86ec
init version: fec3683
Security Options:
 seccomp
  Profile: default
Kernel Version: 3.10.0-957.el7.x86_64
Operating System: Red Hat Enterprise Linux Server 7.6 (Maipo)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.795GiB
Name: rhel7.localdomain
ID: 6PGR:VPGP:MRVX:IBX4:XWP2:IONL:NU75:XLWQ:K52Y:IBFY:XZDF:KOVU
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine

[root@rhel7 vagrant]# docker run alpine

I noticed that your strace show fcntl(5, F_GET_SEALS) = -1 EINVAL (Invalid argument) all the way, which cause the loop. But my strace shows like this(ignore F_GETFL is another one), one normal, one EINVAL, as desired:

~~fcntl(3, F_GETFL)                       = 0x8000 (flags O_RDONLY|O_LARGEFILE)~~
...
[pid  9104] open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 5
[pid  9104] fcntl(5, F_GET_SEALS)       = 0xf (seals F_SEAL_SEAL|F_SEAL_SHRINK|F_SEAL_GROW|F_SEAL_WRITE)
[pid  9104] close(5)                    = 0
...
[pid  9104] open("/proc/self/exe", O_RDONLY|O_CLOEXEC) = 5
[pid  9104] fcntl(5, F_GET_SEALS)       = -1 EINVAL (Invalid argument)
[pid  9104] close(5)                    = 0

Although I use centos yum repo as rhel repo mirror, the kernel is from rhel. AFAIK, centos build their kernel form rhel kernel source(you can get it from rhel kernel srpm) with only a little change. May well not the kernel side cause this issue.

@pinacoelho
Copy link
Author

@kolyshkin @cyphar @vbatts
I've managed to create a minimal testcase based on the F_SEAL test code, that shows the seals are lost on the pexecve syscall.

https://gist.github.com/pinacoelho/396ef3302ae60b17662e29fd57172953

@vbatts
Copy link
Member

vbatts commented Mar 6, 2019

@pinacoelho cool. I linked it in the bugzilla for the mem_fd on rhel.

@pinacoelho
Copy link
Author

Just a FYI:
yum upgrade brough the following along:
containerd.io.x86_64 1.2.5-3.1.el7 @docker-ce-stable
docker-ce.x86_64 3:18.09.4-3.el7 @docker-ce-stable
docker-ce-cli.x86_64 1:18.09.4-3.el7 @docker-ce-stable
kernel.x86_64 3.10.0-957.10.1.el7 @RHEL-76-x86_64-updates

The runc included in containerd.io 1.2.5-3.1.el7 is working correctly with that kernel.

runc version 1.0.0-rc6+dev
commit: 2b18fe1
spec: 1.0.1-dev

https://gist.github.com/pinacoelho/396ef3302ae60b17662e29fd57172953 still shows seals getting lost after the pexecve.

@guunergooner
Copy link

mark

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants