Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

config-linux: add CFS bandwidth burst #1120

Merged
merged 1 commit into from
Jan 23, 2023

Conversation

kailun-qin
Copy link
Contributor

Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

Fixes #1119

Signed-off-by: Kailun Qin kailun.qin@intel.com

@rata rata mentioned this pull request Aug 31, 2021
kailun-qin added a commit to kailun-qin/runc that referenced this pull request Sep 9, 2021
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Signed-off-by: Kailun Qin <kailun.qin@intel.com>
config-linux.md Outdated Show resolved Hide resolved
@kailun-qin
Copy link
Contributor Author

kailun-qin commented Feb 28, 2022

Dear spec maintainers @crosbymichael @cyphar @dqminh @giuseppe @hqhq @mrunalp @tianon @vbatts,

I received this feature request several times from runtime users of different orgnizations, would you please kindly take a look at this one so that we can move forward (also opencontainers/runc#3205)?

Many thanks!

config-linux.md Outdated
@@ -359,7 +359,12 @@ For more information, see the kernel cgroups documentation about [cpusets][cgrou
The following parameters can be specified to set up the controller:

* **`shares`** *(uint64, OPTIONAL)* - specifies a relative share of CPU time available to the tasks in a cgroup
* **`quota`** *(int64, OPTIONAL)* - specifies the total amount of time in microseconds for which all tasks in a cgroup can run during one period (as defined by **`period`** below)
* **`quota`** *(int64, OPTIONAL)* - specifies the replenished amount of time in microseconds for which all tasks in a cgroup can run during one period (as defined by **`period`** below)
A value of `-1` indicates that the cgroup does not have any bandwidth restriction in place (i.e., an unconstrained bandwidth group).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure about having -1 here. On cgroup v2 quota and period are written together. Does -1 mean it has the same value as period?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For cgroup v2, I suppose -1 means max, which can be written together w/ period.

According to the the cpu.max section of cgroup v2 doc:

cpu.max
A read-write two value file which exists on non-root cgroups. The default is "max 100000".

The maximum bandwidth limit. It's in the following format:
$MAX $PERIOD
which indicates that the group may consume upto $MAX in each $PERIOD duration. "max" for $MAX indicates no limit. If only one number is written, $MAX is updated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this applies to all negative integers actually (i.e., not only -1). Any suggestions? Thanks!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would guess the question here is really why it's important to burst support that we include support for -1 here too -- I'd say this is a breaking change, since the spec doesn't currently say anything about negative numbers, and instead implies that this optional field be left unspecified for no restriction. Does cgroupv1 support negative values?

To be clear, I agree that we should clarify this, especially that unspecified needs to be max in cgroupv2, but I'm not sure how it relates to burst except that it's adjacent.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be clear, I agree that we should clarify this, especially that unspecified needs to be max in cgroupv2, but I'm not sure how it relates to burst except that it's adjacent.

Thanks, I agree we might clarify it in a separate thread.

@Zheaoli
Copy link

Zheaoli commented Mar 2, 2022

In my opinion, I think cpu_burst mode is a cgroupv2 only feature. Is there any compatibility issue if we change all schema that is used in both cgroupv1 and v2?

@giuseppe
Copy link
Member

giuseppe commented Mar 2, 2022

In my opinion, I think cpu_burst mode is a cgroupv2 only feature. Is there any compatibility issue if we change all schema that is used in both cgroupv1 and v2?

I see it is present with cgroup v1 too:

# ls /sys/fs/cgroup/cpu/cpu.cfs_burst_us
/sys/fs/cgroup/cpu/cpu.cfs_burst_us

If it was a cgroup v2 only feature then we could just use the unified map.

@kailun-qin
Copy link
Contributor Author

In my opinion, I think cpu_burst mode is a cgroupv2 only feature. Is there any compatibility issue if we change all schema that is used in both cgroupv1 and v2?

I see it is present with cgroup v1 too:

Yes, it supports both cgroup v1 and v2 IMO.

@Zheaoli Any special reason for considering it as cgroup v2 only?

@Zheaoli
Copy link

Zheaoli commented Mar 2, 2022

Sorry for the bad information, my memory is wrong

I have confirmed the burst mode support both v1 and v2 since Linux 5.14, FYI https://www.kernel.org/doc/html/latest/scheduler/sched-bwc.html

@Zheaoli
Copy link

Zheaoli commented Mar 2, 2022

But for this, maybe we should think about the kernel issue? The last LTS kernel is 5.10 which is not supported for burst mode(People may use 510 with the newest version of runc . Maybe we should make burst mode as an experimental feature before the Linux 5.15 has been released

config-linux.md Outdated Show resolved Hide resolved
config-linux.md Show resolved Hide resolved
@gregkh
Copy link
Member

gregkh commented Mar 2, 2022

But for this, maybe we should think about the kernel issue? The last LTS kernel is 5.10 which is not supported for burst mode(People may use 510 with the newest version of runc . Maybe we should make burst mode as an experimental feature before the Linux 5.15 has been released

The 5.15 kernel was released last year, and has already been marked as "LTS" for quite some time now. I do not understand why you are not using it now already :)

@Zheaoli
Copy link

Zheaoli commented Mar 2, 2022

The 5.15 kernel was released last year, and has already been marked as "LTS" for quite some time now. I do not understand why you are not using it now already :)

Many of our machines are using 510 not 515, We can upgrade the runc version easier but it's a tough way to upgrade the kernel version(And my bad again, 515 has been released in 2021-10-31).

@Zheaoli
Copy link

Zheaoli commented Mar 2, 2022

I think the spec is LGTM to me now.

For my personal idea, I'm very glad to see this spec will be accepted ASAP, because it's very useful for some performance-needed circumstances.

Copy link
Member

@giuseppe giuseppe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Zheaoli
Copy link

Zheaoli commented Mar 9, 2022

Hello all, would you mind telling me if this PR is mergeable?

Copy link
Member

@tianon tianon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, I agree we should add this, and I think the suggested name/format are fine (they seem to match what the kernel uses, from what I can tell), but I think we need to make some adjustments to the description -- I'm getting mixed signals about how this feature works from this PR, and I think we can improve that. 😄

config-linux.md Show resolved Hide resolved
config-linux.md Outdated
@@ -359,7 +359,12 @@ For more information, see the kernel cgroups documentation about [cpusets][cgrou
The following parameters can be specified to set up the controller:

* **`shares`** *(uint64, OPTIONAL)* - specifies a relative share of CPU time available to the tasks in a cgroup
* **`quota`** *(int64, OPTIONAL)* - specifies the total amount of time in microseconds for which all tasks in a cgroup can run during one period (as defined by **`period`** below)
* **`quota`** *(int64, OPTIONAL)* - specifies the replenished amount of time in microseconds for which all tasks in a cgroup can run during one period (as defined by **`period`** below)
A value of `-1` indicates that the cgroup does not have any bandwidth restriction in place (i.e., an unconstrained bandwidth group).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would guess the question here is really why it's important to burst support that we include support for -1 here too -- I'd say this is a breaking change, since the spec doesn't currently say anything about negative numbers, and instead implies that this optional field be left unspecified for no restriction. Does cgroupv1 support negative values?

To be clear, I agree that we should clarify this, especially that unspecified needs to be max in cgroupv2, but I'm not sure how it relates to burst except that it's adjacent.

config-linux.md Outdated
@@ -359,7 +359,12 @@ For more information, see the kernel cgroups documentation about [cpusets][cgrou
The following parameters can be specified to set up the controller:

* **`shares`** *(uint64, OPTIONAL)* - specifies a relative share of CPU time available to the tasks in a cgroup
* **`quota`** *(int64, OPTIONAL)* - specifies the total amount of time in microseconds for which all tasks in a cgroup can run during one period (as defined by **`period`** below)
* **`quota`** *(int64, OPTIONAL)* - specifies the replenished amount of time in microseconds for which all tasks in a cgroup can run during one period (as defined by **`period`** below)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, "the total amount ... during one period" already implies that this is "replenished" and changing this word makes it less clear 🙈

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion. emm... I re-read this again. I agree w/ you.
(FYI, my intention was to highlight that the the quota is actually the "replenished" amount while the actual usage can be "unreplenished" due to the introduction of the burst concept.)

config-linux.md Outdated
* **`quota`** *(int64, OPTIONAL)* - specifies the total amount of time in microseconds for which all tasks in a cgroup can run during one period (as defined by **`period`** below)
* **`quota`** *(int64, OPTIONAL)* - specifies the replenished amount of time in microseconds for which all tasks in a cgroup can run during one period (as defined by **`period`** below)
A value of `-1` indicates that the cgroup does not have any bandwidth restriction in place (i.e., an unconstrained bandwidth group).
Any (valid) positive value(s) no smaller than `burst` will enact the specified bandwidth limit.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding so far is that burst is either excess time "earned" from past under-usage or borrowed from future periods where we expect/hope we'll be under-utilizing (I'm not actually clear from this PR which it is). Either way, in this specific case, I think the restriction is really on the value of burst, not a restriction on the value of quota (so doesn't seem helpful to mention here).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding so far is that burst is either excess time "earned" from past under-usage or borrowed from future periods where we expect/hope we'll be under-utilizing (I'm not actually clear from this PR which it is). Either way, in this specific case, I think the restriction is really on the value of burst, not a restriction on the value of quota (so doesn't seem helpful to mention here).

In fact, CPU burst is "earned" from past under-usage . Though in this case it's more related to the value of burst, writing a quota smaller than burst will be considered as an EINVAL by the kerenl. It's mentioned in the kernel doc as well. So IMO it's worth mentioning here which may help guide the right configuration.

* **`quota`** *(int64, OPTIONAL)* - specifies the replenished amount of time in microseconds for which all tasks in a cgroup can run during one period (as defined by **`period`** below)
A value of `-1` indicates that the cgroup does not have any bandwidth restriction in place (i.e., an unconstrained bandwidth group).
Any (valid) positive value(s) no smaller than `burst` will enact the specified bandwidth limit.
* **`burst`** *(uint64, OPTIONAL)* - specifies the maximum amount of accumulated time in microseconds for which all tasks in a cgroup can run additionally for burst during one period (as defined by **`period`** below)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feature borrows time now against the future underrun ...

Given this, I don't think "accumulated" is the right choice of word here -- this feature is about borrowing from the future, not accumulating from the past:

Suggested change
* **`burst`** *(uint64, OPTIONAL)* - specifies the maximum amount of accumulated time in microseconds for which all tasks in a cgroup can run additionally for burst during one period (as defined by **`period`** below)
* **`burst`** *(uint64, OPTIONAL)* - specifies the maximum amount of burst time in microseconds for which all tasks in a cgroup can borrow from future periods during one period (as defined by **`period`** below)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feature borrows time now against the future underrun ... is directly from the kernel doc and it's actually accumulating from the past rather than borrowing from future. It intends to say that borrowing time now to prepare for the future underestimation.

config-linux.md Outdated
Comment on lines 366 to 367
A value of `0` for indicates that the cgroup can not accumulate any unused bandwidth. It makes the traditional bandwidth control behavior unchanged.
Any (valid) positive value(s) no larger than `quota` will enact the cap on unused bandwidth accumulation.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this would be more consistent if we assume that not specifying this optional field is the way to get "no limit", and I think it would also make sense to be more clear about values bigger than quota are not just ignored but rather an error, perhaps something like this?

Suggested change
A value of `0` for indicates that the cgroup can not accumulate any unused bandwidth. It makes the traditional bandwidth control behavior unchanged.
Any (valid) positive value(s) no larger than `quota` will enact the cap on unused bandwidth accumulation.
If specified, this value MUST be no larger than `quota` (runtimes MAY generate an error).

Edit: coming back to this after opencontainers/runc#3205 (comment), I'm guessing that's related? (And likely why the "negative values" bits were added to quota?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, indeed it's related.

@tianon
Copy link
Member

tianon commented Mar 9, 2022

Perhaps it would help me (and other spec reviewers) if we level-set on how the feature works. Here's my current understanding -- is this accurate?

there's at most quota+burst in any one given period, and any use of burst in a period counts against a future period's available quota

Zheaoli pushed a commit to Zheaoli/runc that referenced this pull request Mar 19, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Signed-off-by: Kailun Qin <kailun.qin@intel.com>

Signed-off-by: Manjusaka <me@manjusaka.me>
Zheaoli pushed a commit to Zheaoli/runc that referenced this pull request Mar 19, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Signed-off-by: Kailun Qin <kailun.qin@intel.com>

Signed-off-by: Manjusaka <me@manjusaka.me>
Zheaoli pushed a commit to Zheaoli/runc that referenced this pull request Mar 19, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Signed-off-by: Kailun Qin <kailun.qin@intel.com>

Signed-off-by: Manjusaka <me@manjusaka.me>
Zheaoli pushed a commit to Zheaoli/runc that referenced this pull request Mar 19, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Signed-off-by: Kailun Qin <kailun.qin@intel.com>

Signed-off-by: Manjusaka <me@manjusaka.me>
Zheaoli pushed a commit to Zheaoli/runc that referenced this pull request Mar 19, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Signed-off-by: Kailun Qin <kailun.qin@intel.com>

Signed-off-by: Manjusaka <me@manjusaka.me>
Zheaoli pushed a commit to Zheaoli/runc that referenced this pull request Mar 21, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Signed-off-by: Kailun Qin <kailun.qin@intel.com>

Signed-off-by: Manjusaka <me@manjusaka.me>
Zheaoli pushed a commit to Zheaoli/runc that referenced this pull request Apr 26, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Signed-off-by: Kailun Qin <kailun.qin@intel.com>

Signed-off-by: Manjusaka <me@manjusaka.me>
Zheaoli pushed a commit to Zheaoli/runc that referenced this pull request Apr 27, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Signed-off-by: Kailun Qin <kailun.qin@intel.com>

Signed-off-by: Manjusaka <me@manjusaka.me>
Zheaoli pushed a commit to Zheaoli/runc that referenced this pull request May 1, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Signed-off-by: Kailun Qin <kailun.qin@intel.com>

Signed-off-by: Manjusaka <me@manjusaka.me>
Zheaoli pushed a commit to Zheaoli/runc that referenced this pull request May 1, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Signed-off-by: Kailun Qin <kailun.qin@intel.com>

Signed-off-by: Manjusaka <me@manjusaka.me>
Zheaoli pushed a commit to Zheaoli/runc that referenced this pull request May 1, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Signed-off-by: Kailun Qin <kailun.qin@intel.com>

Signed-off-by: Manjusaka <me@manjusaka.me>
Zheaoli pushed a commit to Zheaoli/runc that referenced this pull request May 2, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Signed-off-by: Kailun Qin <kailun.qin@intel.com>

Signed-off-by: Manjusaka <me@manjusaka.me>
Zheaoli pushed a commit to Zheaoli/runc that referenced this pull request May 2, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Signed-off-by: Kailun Qin <kailun.qin@intel.com>

Signed-off-by: Manjusaka <me@manjusaka.me>
Zheaoli pushed a commit to Zheaoli/runc that referenced this pull request May 3, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Co-authored-by: Nadeshiko Manju <lizheao940510@gmail.com>
Signed-off-by: Kailun Qin <kailun.qin@intel.com>

Update libcontainer/configs/cgroup_linux.go
Zheaoli added a commit to Zheaoli/runc that referenced this pull request May 3, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Co-authored-by: Nadeshiko Manju <lizheao940510@gmail.com>
Signed-off-by: Kailun Qin <kailun.qin@intel.com>
Zheaoli added a commit to Zheaoli/runc that referenced this pull request May 30, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Co-authored-by: Nadeshiko Manju <lizheao940510@gmail.com>
Signed-off-by: Kailun Qin <kailun.qin@intel.com>
Zheaoli added a commit to Zheaoli/runc that referenced this pull request Jun 15, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Co-authored-by: Nadeshiko Manju <lizheao940510@gmail.com>
Signed-off-by: Kailun Qin <kailun.qin@intel.com>
Zheaoli added a commit to Zheaoli/runc that referenced this pull request Jun 22, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Co-authored-by: Nadeshiko Manju <lizheao940510@gmail.com>
Signed-off-by: Kailun Qin <kailun.qin@intel.com>
@AkihiroSuda AkihiroSuda mentioned this pull request Jun 26, 2023
12 tasks
Zheaoli added a commit to Zheaoli/runc that referenced this pull request Jun 28, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Co-authored-by: Nadeshiko Manju <lizheao940510@gmail.com>
Signed-off-by: Kailun Qin <kailun.qin@intel.com>
Zheaoli added a commit to Zheaoli/runc that referenced this pull request Aug 10, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Co-authored-by: Nadeshiko Manju <lizheao940510@gmail.com>
Signed-off-by: Kailun Qin <kailun.qin@intel.com>
Zheaoli added a commit to Zheaoli/runc that referenced this pull request Aug 14, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Co-authored-by: Nadeshiko Manju <me@manjusaka.me>
Signed-off-by: Kailun Qin <kailun.qin@intel.com>
Zheaoli added a commit to Zheaoli/runc that referenced this pull request Aug 18, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Co-authored-by: Nadeshiko Manju <me@manjusaka.me>
Signed-off-by: Kailun Qin <kailun.qin@intel.com>
Zheaoli added a commit to Zheaoli/runc that referenced this pull request Aug 24, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Co-authored-by: Nadeshiko Manju <me@manjusaka.me>
Signed-off-by: Kailun Qin <kailun.qin@intel.com>
Zheaoli added a commit to Zheaoli/runc that referenced this pull request Aug 25, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Co-authored-by: Nadeshiko Manju <me@manjusaka.me>
Signed-off-by: Kailun Qin <kailun.qin@intel.com>
Zheaoli added a commit to Zheaoli/runc that referenced this pull request Aug 26, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Co-authored-by: Nadeshiko Manju <me@manjusaka.me>
Signed-off-by: Kailun Qin <kailun.qin@intel.com>
Zheaoli added a commit to Zheaoli/runc that referenced this pull request Aug 26, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Co-authored-by: Nadeshiko Manju <me@manjusaka.me>
Signed-off-by: Kailun Qin <kailun.qin@intel.com>
Zheaoli added a commit to Zheaoli/runc that referenced this pull request Sep 6, 2023
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Co-authored-by: Nadeshiko Manju <me@manjusaka.me>
Signed-off-by: Kailun Qin <kailun.qin@intel.com>
dims pushed a commit to dims/libcontainer that referenced this pull request Oct 19, 2024
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Co-authored-by: Nadeshiko Manju <me@manjusaka.me>
Signed-off-by: Kailun Qin <kailun.qin@intel.com>
kolyshkin pushed a commit to kolyshkin/containerd-cgroups that referenced this pull request Nov 6, 2024
Burstable CFS controller is introduced in Linux 5.14. This helps with
parallel workloads that might be bursty. They can get throttled even
when their average utilization is under quota. And they may be latency
sensitive at the same time so that throttling them is undesired.

This feature borrows time now against the future underrun, at the cost
of increased interference against the other system users, by introducing
cfs_burst_us into CFS bandwidth control to enact the cap on unused
bandwidth accumulation, which will then used additionally for burst.

The patch adds the support/control for CFS bandwidth burst.

runtime-spec: opencontainers/runtime-spec#1120

Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>
Co-authored-by: Nadeshiko Manju <me@manjusaka.me>
Signed-off-by: Kailun Qin <kailun.qin@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Proposal: add control for CFS bandwidth burst
9 participants