-
Notifications
You must be signed in to change notification settings - Fork 18.4k
Description
Proposal Details
We propose to add a new Fill
method to struct CPUSet
, which sets all CPU bits to 1.
The implementation can look like this (see CL 698015):
// Fill adds all possible CPU bits to the set s. On Linux, [SchedSetaffinity]
// will silently ignore any invalid CPU bits in [CPUSet] so this is an
// efficient way of resetting the CPU affinity of a process.
func (s *CPUSet) Fill() {
for i := range s {
s[i] = ^cpuMask(0)
}
}
Context
Some programs (such as container runtimes) want to reset their CPU
affinity if they are spawned by processes with a particular CPU
affinity. Container runtimes didn't really have to deal with this issue
until Linux 6.2 when the cpuset cgroup was changed to no longer
auto-reset CPU affinity in this case.
A naive approach to resetting your CPU affinity would be to get the
number of CPUs by looking at "/proc/stat" or "/sys/devices/system/cpu"
(note that runtime.NumCPU() actually returns the CPU affinity of the
process at startup time, which isn't useful for this purpose) and then
asking for all of those CPUs.
However, sched_setaffinity(2) will silently ignore any CPU bits set in
the provided CPUSet if they do not exist or are not enabled in the
cpuset cgroup of the process. This means that you can reset your CPU
affinity by just setting every CPU bit in CPUSet and passing it to
sched_setaffinity(2).
Unfortunately, setting every CPU bit in CPUSet with (*CPUSet).Set() is
very inefficient. If it were possible to just memset(0xFF) the CPUSet
array, users would be able to reset their CPU affinity even more
cheaply. However, Go doesn't have a memset primitive that can be used in
that way.
Obvious solutions like setting the array elements of CPUSet to (^0) do
not work because CPUSet is an array of a private newtype and so the
compiler complains if you try to use a constant like (^0) without a
cast (and we cannot use a cast because the type is private):
cannot use ^0 (untyped int constant -1) as
"golang.org/x/sys/unix".cpuMask value in assignment (overflows)
The only real alternative is to do something quite hacky like:
cpuset := unix.CPUSet{}
for i := range cpuset {
cpuset[i]-- // underflow to 0xFF..FF
}
... which is the solution we use in runc.
It would be much nicer to have a helper that does this memset for us in
a less hacky way, since resetting CPU affinity seems like a fairly
common operation.
Ref: Linux kernel commit da019032819a ("sched: Enforce user requested affinity")
(Description and implementation by @cyphar)