Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update doc for kubelet option --reserved-cpus #17648

Merged
merged 1 commit into from
Nov 27, 2019
Merged

update doc for kubelet option --reserved-cpus #17648

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions content/en/docs/tasks/administer-cluster/cpu-management-policies.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,10 @@ state file `cpu_manager_state` in the kubelet root directory.
This policy manages a shared pool of CPUs that initially contains all CPUs in the
node. The amount of exclusively allocatable CPUs is equal to the total
number of CPUs in the node minus any CPU reservations by the kubelet `--kube-reserved` or
`--system-reserved` options. CPUs reserved by these options are taken, in
`--system-reserved` options. From 1.17, the CPU reservation list can be specified
explictly by kubelet `--reserved-cpus` option. The explicit CPU list specified by
`--reserved-cpus` takes precedence over the CPU reservation specified by
`--kube-reserved` and `--system-reserved`. CPUs reserved by these options are taken, in
integer quantity, from the initial shared pool in ascending order by physical
core ID.  This shared pool is the set of CPUs on which any containers in
`BestEffort` and `Burstable` pods run. Containers in `Guaranteed` pods with fractional
Expand All @@ -103,8 +106,8 @@ exclusive CPUs.

{{< note >}}
The kubelet requires a CPU reservation greater than zero be made
using either `--kube-reserved` and/or `--system-reserved` when the static
policy is enabled. This is because zero CPU reservation would allow the shared
using either `--kube-reserved` and/or `--system-reserved` or `--reserved-cpus` when
the static policy is enabled. This is because zero CPU reservation would allow the shared
pool to become empty.
{{< /note >}}

Expand Down
26 changes: 26 additions & 0 deletions content/en/docs/tasks/administer-cluster/reserve-compute-resources.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,28 @@ control group (`system.slice` on systemd machines for example).
Note that Kubelet **does not** create `--system-reserved-cgroup` if it doesn't
exist. Kubelet will fail if an invalid cgroup is specified.

### Explictly Reserved CPU List
{{< feature-state for_k8s_version="v1.17" state="stable" >}}

- **Kubelet Flag**: `--reserved-cpus=0-3`

`reserved-cpus` is meant to define an explict CPU set for OS system daemons and
kubernetes system daemons. This option is added in 1.17 release. `reserved-cpus`
is for systems that do not intent to define seperate top level cgroups for
OS system daemons and kubernetes system daemons with regard to cpuset resource.
If the Kubelet **does not** have `--system-reserved-cgroup` and `--kube-reserved-cgroup`,
the explicit cpuset provided by `reserved-cpus` will take precedence over the CPUs
defined by `--kube-reserved` and `--system-reserved` options.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What goes after the = sign? I'd guess that I could specify, eg: --reserved-cpus=8-15 but I'm not sure if I've guessed correctly.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's right. Updated.

This option is specifically designed for Telco/NFV use cases where uncontrolled
interrupts/timers may impact the workload performance. you can use this option
to define the explict cpuset for the system/kubernetes daemons as well as the
interrupts/timers, so the rest CPUs on the system can be used exclusively for
workloads, with less impact from uncontrolled interrupts/timers. To move the
system daemon, kubernetes daemons and interrupts/timers to the explict cpuset
defined by this option, other mechanism outside Kubernetes should be used.
For example: in Centos, you can do this using the tuned toolset.

### Eviction Thresholds

- **Kubelet Flag**: `--eviction-hard=[memory.available<500Mi]`
Expand Down Expand Up @@ -259,4 +281,8 @@ for `kube-reserved` and `system-reserved`.
As of Kubernetes version 1.8, the `storage` key name was changed to `ephemeral-storage`
for the alpha release.

As of Kubernetes version 1.17, you can optionally specify
explicit cpuset by `reserved-cpus` as CPUs reserved for OS system
daemons/interrupts/timers and Kubernetes daemons.

{{% /capture %}}