[Question] Why use "cpushares" instead of "cpus" with the Docker driver?

[migueleliasweb]

Hey everyone!

This question came to me as I was trying to setup a Nomad cluster with different server types. It's quite hard to setup properly the Job resources considering the frequency of the CPU in Mhz.

Choosing the "cpus" option would make more sense because it's easier to think in the number of cores* instead of the frequency**. It would also help properly constrainting the process to not overuse resources in the server even in the case of it being the only job at that specific server.

Docker docs about cpu resource constaints: https://docs.docker.com/config/containers/resource_constraints/#cpu

* --cpus=4.5 would mean 4 full cores and a shared core "50%". Interesting here is that --cpus=4.5 is meaningful no matter the CPU type or count.

** --cpu-shares=4500 means "4.5Ghz" but in a server with 2Ghz CPU cores that does not translate to something meaningful...maybe 2.5 CPU cores? ¯\_(ツ)_/¯. The idea ends up being a bit too abstract in this sense.

[preetapan]

@migueleliasweb Converting to Mhz allows CPU fingerprinting to work in clusters with heterogeneous CPUs, for example, hardware with cores that could be 1Ghz or 2 Ghz. Given recent community feedback and other internal discussions, we do plan to support CPU as a unit (with a documented conversion like 1 CPU=1024Mhz) in a future release since its easier to reason about.

[migueleliasweb]

Hey @preetapan , thanks for clarifying!

My only concern is that specially in the case of having heterogenous servers in a cluster, an application with a CPU limit of 2.5Ghz could have, depending on the server type, 1 or 2 Cores. That's a bit troublesome for multithreaded applications as they might get one or two cpu threads depending on the underlying infrastructure (which they shouldn't care/be aware of). On the other hand, using CPU as a unit and not considering the underlying frequency, these applications would have a more even threading experience.

[kcwong-verseon]

@preetapan: It's important to understand the semantic differences between --cpus (and the underlying --cpu-period and --cpu-quota) and --cpu-shares. The former is about asking for a specific cpu-slice duration, whereas the latter is about relative sharing weight across other cgroups. A simple translation of "1 CPU = 1024 MHz" will not bridge the two different semantic.
@migueleliasweb You don't really get "half" a cpu core/thread; it's all about time slices.

[camerondavison]

Converting to Mhz allows CPU fingerprinting to work in clusters with heterogeneous CPUs, for example, hardware with cores that could be 1Ghz or 2 Ghz. Given recent community feedback and other internal discussions, we do plan to support CPU as a unit (with a documented conversion like 1 CPU=1024Mhz) in a future release since its easier to reason about.

@preetapan I know this is an old issue but I was just looking for something along this line because I came across https://bugs.openjdk.java.net/browse/JDK-8146115 and was trying to figure out how this would translate in nomad terms. It's almost like nomad would need to normalize the cpu Mhz config to 1024 increments per cpu.

Namely talking about this quote

number_of_cpus() will be calculated based on cpu_shares()/1024. 1024 is the default and standard unit for calculating relative cpu usage in cloud based container management software.

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.