release 1.2.0-rc.1

[cyphar]

This is the first release candidate for the 1.2.0 branch of runc. It includes
all patches and bugfixes included in runc 1.1 patch releases (up to and
including 1.1.12). A fair few new features have been added, and some changes
have been made which may affect users. Please help us thoroughly test this
release before we release 1.2.0.

1.2.0-rc.1 - 2024-04-01

There's a frood who really knows where his towel is.

runc now requires a minimum of Go 1.20 to compile.

NOTE: runc currently will not work properly when compiled with Go 1.22 or
newer. This is due to some unfortunate glibc behaviour that Go 1.22
exacerbates in a way that results in containers not being able to start on
some systems. See this issue for more information.

Breaking

• Several aspects of how mount options work has been adjusted in a way that
  could theoretically break users that have very strange mount option strings.
  This was necessary to fix glaring issues in how mount options were being
  treated. The key changes are:

  • Mount options on bind-mounts that clear a mount flag are now always
    applied. Previously, if a user requested a bind-mount with only clearing
    options (such as rw,exec,dev) the options would be ignored and the
    original bind-mount options would be set. Unfortunately this also means
    that container configurations which specified only clearing mount options
    will now actually get what they asked for, which could break existing
    containers (though it seems unlikely that a user who requested a specific
    mount option would consider it "broken" to get the mount options they
    asked foruser who requested a specific mount option would consider it
    "broken" to get the mount options they asked for). This also allows us to
    silently add locked mount flags the user did not explicitly request to be
    cleared in rootless mode, allowing for easier use of bind-mounts for
    rootless containers. (rootfs: remove --no-mount-fallback and finally fix MS_REMOUNT #3967)

  • Container configurations using bind-mounts with superblock mount flags
    (i.e. filesystem-specific mount flags, referred to as "data" in
    mount(2), as opposed to VFS generic mount flags like MS_NODEV) will
    now return an error. This is because superblock mount flags will also
    affect the host mount (as the superblock is shared when bind-mounting),
    which is obviously not acceptable. Previously, these flags were silently
    ignored so this change simply tells users that runc cannot fulfil their
    request rather than just ignoring it. (configs: validate: add validation for bind-mount fsflags #3990)

  If any of these changes cause problems in real-world workloads, please open
  an issue so we
  can adjust the behaviour to avoid compatibility issues.

Added

• runc has been updated to OCI runtime-spec 1.2.0, and supports all Linux
  features with a few minor exceptions. See
  docs/spec-conformance.md
  for more details.
• runc now supports id-mapped mounts for bind-mounts (with no restrictions on
  the mapping used for each mount). Other mount types are not currently
  supported. This feature requires MOUNT_ATTR_IDMAP kernel support (Linux
  5.12 or newer) as well as kernel support for the underlying filesystem used
  for the bind-mount. See mount_setattr(2) for a list of
  supported filesystems and other restrictions. (Support idmap mounts for volumes #3717, libcontainer: remove all mount logic from nsexec #3985, features: Expose idmap support #3993)
• Two new mechanisms for reducing the memory usage of our protections against
  CVE-2019-5736 have been introduced:
  • runc-dmz is a minimal binary (~8K) which acts as an additional execve
    stage, allowing us to only need to protect the smaller binary. It should
    be noted that there have been several compatibility issues reported with
    the usage of runc-dmz (namely related to capabilities and SELinux). As
    such, this mechanism is opt-in and can be enabled by running runc
    with the environment variable RUNC_DMZ=true (setting this environment
    variable in config.json will have no effect). This feature can be
    disabled at build time using the runc_nodmz build tag. ([Proposal] Use runc-dmz to defeat CVE-2019-5736 #3983, nsexec: cloned binary rework #3987)
  • contrib/memfd-bind is a helper daemon which will bind-mount a memfd copy
    of /usr/bin/runc on top of /usr/bin/runc. This entirely eliminates
    per-container copies of the binary, but requires care to ensure that
    upgrades to runc are handled properly, and requires a long-running daemon
    (unfortunately memfds cannot be bind-mounted directly and thus require a
    daemon to keep them alive). (nsexec: cloned binary rework #3987)
• runc will now use cgroup.kill if available to kill all processes in a
  container (such as when doing runc kill). (Make use of cgroup.kill #3135, runc kill: add support for cgroup.kill #3825)
• Add support for setting the umask for runc exec. (Add support for umask when exec container #3661)
• libct/cg: support SCHED_IDLE for runc cgroupfs. (libct/cg: support SCHED_IDLE for runc cgroupfs #3377)
• checkpoint/restore: implement --manage-cgroups-mode=ignore. (checkpoint/restore: implement --manage-cgroups-mode ignore #3546)
• seccomp: refactor flags support; add flags to features, set SPEC_ALLOW by
  default. (seccomp: refactor flags support; add flags to features, set SPEC_ALLOW by default #3588)
• libct/cg/sd: use systemd v240+ new MAJOR:* syntax. (libct/cg/sd: use systemd v240+ new MAJOR:* syntax #3843)
• Support CFS bandwidth burst for CPU. ([Carry #3205] libct/cg: add CFS bandwidth burst for CPU #3749, Support CFS bandwidth burst #3145)
• Support time namespaces. (Support time namespace #3876)
• Reduce the runc binary size by ~11% by updating
  github.com/checkpoint-restore/go-criu. (deps: bump github.com/checkpoint-restore/go-criu to 6.3.0 #3652)
• Add --pidfd-socket to runc run and runc exec to allow for management
  processes to receive a pidfd for the new process, allowing them to avoid pid
  reuse attacks. ([feature request] *: introduce pidfd-socket flag #4045)

Deprecated

• runc option --criu is now ignored (with a warning), and the option will
  be removed entirely in a future release. Users who need a non-standard
  criu binary should rely on the standard way of looking up binaries in
  $PATH. (Less interfaces #3316)
• runc kill option -a is now deprecated. Previously, it had to be specified
  to kill a container (with SIGKILL) which does not have its own private PID
  namespace (so that runc would send SIGKILL to all processes). Now, this is
  done automatically. (RFC: drop -a from runc kill #3864, runc kill: add support for cgroup.kill #3825)
• github.com/opencontainers/runc/libcontainer/user is now deprecated, please
  use github.com/moby/sys/user instead. It will be removed in a future
  release. (Deprecate libcontainer/user, and migrate to github.com/moby/sys/user #4017)

Changed

• When Intel RDT feature is not available, its initialization is skipped,
  resulting in slightly faster runc exec and runc run. (Faster Intel RDT init if the feature is unavailable #3306)
• runc features is no longer experimental. (features: graduate from experimental #3861)
• libcontainer users that create and kill containers from a daemon process
  (so that the container init is a child of that process) must now implement
  a proper child reaper in case a container does not have its own private PID
  namespace, as documented in container.Signal. (runc kill: add support for cgroup.kill #3825)
• Sum anon and file from memory.stat for cgroupv2 root usage,
  as the root does not have memory.current for cgroupv2.
  This aligns cgroupv2 root usage more closely with cgroupv1 reporting.
  Additionally, report root swap usage as sum of swap and memory usage,
  aligned with v1 and existing non-root v2 reporting. (libct/cg/fs2: use file + anon + swap for usage #3933)
• Add swapOnlyUsage in MemoryStats. This field reports swap-only usage.
  For cgroupv1, Usage and Failcnt are set by subtracting memory usage
  from memory+swap usage. For cgroupv2, Usage, Limit, and MaxUsage
  are set. (feat: add swapOnlyUsage in MemoryStats #4010)
• libcontainer users that create and kill containers from a daemon process
  (so that the container init is a child of that process) must now implement
  a proper child reaper in case a container does not have its own private PID
  namespace, as documented in container.Signal. (runc kill: add support for cgroup.kill #3825)
• libcontainer: container.Signal no longer takes an all argument. Whether
  or not it is necessary to kill all processes in the container individually
  is now determined automatically. (runc kill: add support for cgroup.kill #3825, runc clone binary mount too slow boot shim boot timeout，then runc.XXXXXX residual #3885)
• seccomp: enable seccomp binary tree optimization. (libct/seccomp: enable seccomp binary tree optimization #3405)
• runc run/runc exec: ignore SIGURG. (runc run/exec: ignore SIGURG #3368)
• Remove tun/tap from the default device allowlist. (Remove tun/tap from the default device rules #3468)
• runc --root non-existent-dir list now reports an error for non-existent
  root directory. (Assorted CLI nitpicks #3374)

Fixed

• In case the runc binary resides on tmpfs, runc init no longer re-execs
  itself twice. (libct/nsenter: fix extra runc re-exec on tmpfs #3342)
• Our seccomp -ENOSYS stub now correctly handles multiplexed syscalls on
  s390 and s390x. This solves the issue where syscalls the host kernel did not
  support would return -EPERM despite the existence of the -ENOSYS stub
  code (this was due to how s390x does syscall multiplexing). (seccomp: enosys: always return -ENOSYS for setup(2) #3474)
• Remove tun/tap from the default device rules. (Remove tun/tap from the default device rules #3468)
• specconv: avoid mapping "acl" to MS_POSIXACL. (specconv: avoid mapping "acl" to MS_POSIXACL #3739)
• libcontainer: fix private PID namespace detection when killing the
  container. (Shared pidns detection is wrong in initProcess.wait #3866, runc kill: add support for cgroup.kill #3825)
• systemd socket notification: fix race where runc exited before systemd
  properly handled the READY notification. (notify_socket.go: use sd_notify_barrier mechanism #3291, READY notification sometimes not accepted by systemd #3293)
• The -ENOSYS seccomp stub is now always generated for the native
  architecture that runc is running on. This is needed to work around some
  arguably specification-incompliant behaviour from Docker on architectures
  such as ppc64le, where the allowed architecture list is set to null. This
  ensures that we always generate at least one -ENOSYS stub for the native
  architecture even with these weird configs. (seccomp: patchbpf: always include native architecture in stub #4219)

Removed

• In order to fix performance issues in the "lightweight" bindfd protection
  against CVE-2019-5736, the temporary ro bind-mount of
  /proc/self/exe has been removed. runc now creates a binary copy in all
  cases. See the above notes about memfd-bind and runc-dmz as well as
  contrib/cmd/memfd-bind/README.md for more information about how this
  (minor) change in memory usage can be further reduced. (nsexec: cloned binary rework #3987, libct/nsenter: namespace the bindfd shuffle #3599, too many mount/umount syscalls #2532,
  nsexec: cloned_binary: remove bindfd logic entirely #3931)
• libct/cg: Remove EnterPid (a function with no users). (libct/cg: rm EnterPid #3797)
• libcontainer: Remove {Pre,Post}MountCmds which were never used and are
  obsoleted by more generic container hooks. (libct: Mount: rm {Pre,Post}mountCmds #3350)

[lifubang]

I think #4210 is a bug introduced by #3749 which is included in this release, how about waiting this issue fixed?
But I don't know we should only fix this special issue, or cherry-pick from c9ba576 to fix all potential issues about numeric pointer.

[lifubang]

Maybe we should include #4222 .

[rata]

Some simple questions, but this LGTM after those clarifications.

[rata]

@lifubang

how about waiting this issue fixed?

IMHO those are minor things, with small PRs. I'm fine with either merging or just releasing the rc now and adding those small things later. There is definitely lot of things merged already and getting this out would be very useful in any case (with or without those small fixes)

[lifubang]

IMHO those are minor things, with small PRs.

I have updated #4210 , I think this is not a small bug. Maybe I wrote this issue not clearly before.
I think before we draft a release, if we have realized that there is a small bug, we should fix it.

[rata]

LGTM

[lifubang]

When I was debugging #4211 , there was another bug: #4225 .
I think I can't solve it in a short time.
So let's record this bug and release 1.2.0-rc.1.

[cyphar]

@kolyshkin WDYT? Should we wait for #4227 or release this first and make sure we get #4227 for 1.2.0?

[kolyshkin]

Should we wait for #4227 or release this first and make sure we get #4227 for 1.2.0?

I found out that it requires some major rework, so I can't really fix it right now.

Let'd do rc1 soon. I will review this PR tomorrow.

[kolyshkin]

LGTM overall.

I'm concerned about Go 1.22 vs glibc vs nsenter incompatibility issue (described in golang/go#65625).

I think we should at least add a note to Changelog about it, and at most don't allow runc to be compiled with Go 1.22. Something like this should suffice:

$ cat <<< EOF > ./libcontainer/nsenter/nsenter_go122.go 
//go:build go1.22

package nsenter

import "runc does not work with Go 1.22"
EOF

Result:

go1.22.0 build -trimpath "-buildmode=pie"  -tags "seccomp urfave_cli_no_docs " -ldflags "-X main.gitCommit=v1.1.0-987-g59078892 -X main.version=1.1.0+dev " -o runc .
init.go:7:2: /home/kir/git/runc/libcontainer/nsenter/nsenter_go122.go:5:8: invalid import path: runc does not work with Go 1.22
make: *** [Makefile:71: runc-bin] Error 1

[SuperQ]

@kolyshkin Looking over the Go issue. It seems like the issue is not Go, but glibc and nsenter. And then, only specific versions of those. It seems like they improved the error handling in Go, but I have not checked the 1.22.1 changelog to see if those patches are there.

Hard dropping Go 1.22 like that is, IMO, not an acceptable way to deal with this.

Kubernetes, which directly uses this project, is already building with 1.22 as a minimum required version.

[cyphar]

@SuperQ The issue with Go 1.22 is a little more complicated than that AFAIK.

Yes, the core issue is that arguably nsenter needs to be reworked to do another re-exec after it sets up the namespaces (yay, yet another thing to make Go container runtimes slower!) but the reason that Go 1.22 will aggravate this issue is because they now explicitly detect the broken pthread state case and always panic (this patch wasn't merged AFAICS but it must've been carried -- I haven't looked into it much after I saw this patch).

Given the severity of the issue, I will see if I can get a fix for this before 1.2.0. But we shouldn't block 1.2.0-rc.1 for this issue IMHO.

(I suspect dropping CLONE_PARENT would be a workable "hacky" fix, but doing so will make zombie reaping more annoying and is a temporary fix since future glibc changes could cause issues.)

Kubernetes, which directly uses this project, is already building with 1.22 as a minimum required version.

Adding it to libcontainer/nsenter means that users which do not import nsenter will not have build issues. Kubernetes does not import nsenter so this will not impact Kubernetes. The only downstreams it will impact are those that use nsenter -- which is precisely the users that need to avoid building with Go 1.22 for the time being.

[cyphar]

@kolyshkin Now that we refuse to build with Go 1.22, does this look okay to release to you?

[kolyshkin]

LGTM 🎉