Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libcontainer/cgroups: do not enable kmem on broken kernels #1920

Closed
wants to merge 2 commits into from

Conversation

kolyshkin
Copy link
Contributor

Commit fe898e7 (PR #1350) enables kernel memory accounting for all cgroups created by libcontainer even if kmem limit is not configured.

Kernel memory accounting is known to be broken in RHEL7 kernels,
including the latest RHEL 7.5 kernel. It does not support reclaim
and can lead to kernel oopses while removing cgroup (merging it
with its parent). Unconditionally enabling kmem acct on RHEL7
leads to bugs:

I am not aware of any good way to figure out whether the kernel
memory accounting in the given kernel is working or broken.
For the lack of a better way, let's check if the running kernel
is RHEL7, and disable initial setting of kmem.

Alternative to #1898

@thaJeztah
Copy link
Member

Wondering if it should be a compile time option, so that packagers for a specific distro can enable/disable it

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Commit fe898e7 (PR opencontainers#1350) enables kernel memory accounting
for all cgroups created by libcontainer even if kmem limit is
not configured.

Kernel memory accounting is known to be broken in RHEL7 kernels,
including the latest RHEL 7.5 kernel. It does not support reclaim
and can lead to kernel oopses while removing cgroup (merging it
with its parent). Unconditionally enabling kmem acct on RHEL7
leads to bugs:

* opencontainers#1725
* kubernetes/kubernetes#61937
* moby/moby#29638

I am not aware of any good way to figure out whether the kernel
memory accounting in the given kernel is working or broken.
For the lack of a better way, let's check if the running kernel
is RHEL7, and disable initial setting of kmem.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
@crosbymichael
Copy link
Member

cc @mrunalp

@kolyshkin
Copy link
Contributor Author

Apparently the kernel bug(s) we work around here are not fixed in latest-n-greatest RHEL7.6 kernels (see latest comments in https://bugzilla.redhat.com/show_bug.cgi?id=1507149).

@kolyshkin
Copy link
Contributor Author

Alternative to #1898

cc @ryarnyah

@mrunalp
Copy link
Contributor

mrunalp commented Nov 1, 2018

I prefer #1921 and have LGTM'ed it. Thanks!

@cyphar
Copy link
Member

cyphar commented Nov 1, 2018

NACK. I prefer #1921 as well.

@kolyshkin
Copy link
Contributor Author

Closed in favor of #1921

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants