kubelet evicts resources when observing disk pressure

[derekwaynecarr]

Description

As a cluster operator, I want the kubelet to monitor local disk usage and respond accordingly to maintain node stability. If the kubelet observes available disk and/or inodes (rootfs or imagefs) are under pressure, the kubelet should pro-actively reclaim related resource to maintain node stability by deleting images, logs, and evicting pods.

Progress Tracker

• Before Alpha
  • Write and maintain draft quality doc
    • During development keep a doc up-to-date about the desired experience of the feature and how someone can try the feature in its current state. Think of it as the README of your new feature and a skeleton for the docs to be written before the Kubernetes release. Paste link to Google Doc: DOC-LINK
  • Design Approval
    • Design Proposal. This goes under docs/proposals. Doing a proposal as a PR allows line-by-line commenting from community, and creates the basis for later design documentation. Paste link to merged design proposal here: PROPOSAL-NUMBER
    • Initial API review (if API). Maybe same PR as design doc. PR-NUMBER
      • Any code that changes an API (/pkg/apis/...)
      • cc @kubernetes/api
    • Identify shepherd (your SIG lead and/or kubernetes-pm@googlegroups.com will be able to help you). My Shepherd is: replace.me@replaceme.com (and/or GH Handle)
      • A shepherd is an individual who will help acquaint you with the process of getting your feature into the repo, identify reviewers and provide feedback on the feature. They are not (necessarily) the code reviewer of the feature, or tech lead for the area.
      • The shepherd is not responsible for showing up to Kubernetes-PM meetings and/or communicating if the feature is on-track to make the release goals. That is still your responsibility.
    • Identify secondary/backup contact point. My Secondary Contact Point is: replace.me@replaceme.com (and/or GH Handle)
  • Write (code + tests + docs) then get them merged. ALL-PR-NUMBERS
    • Code needs to be disabled by default. Verified by code OWNERS
    • Minimal testing
    • Minimal docs
      • cc @kubernetes/docs on docs PR
      • cc @kubernetes/feature-reviewers on this issue to get approval before checking this off
      • New apis: Glossary Section Item in the docs repo: kubernetes/kubernetes.github.io
    • Update release notes
• Before Beta
  • Testing is sufficient for beta
  • User docs with tutorials
    • Updated walkthrough / tutorial in the docs repo: kubernetes/kubernetes.github.io
    • cc @kubernetes/docs on docs PR
    • cc @kubernetes/feature-reviewers on this issue to get approval before checking this off
  • Thorough API review
    • cc @kubernetes/api
• Before Stable
  • docs/proposals/foo.md moved to docs/design/foo.md
    • cc @kubernetes/feature-reviewers on this issue to get approval before checking this off
  • Soak, load testing
  • detailed user docs and examples
    • cc @kubernetes/docs
    • cc @kubernetes/feature-reviewers on this issue to get approval before checking this off

FEATURE_STATUS is used for feature tracking and to be updated by @kubernetes/feature-reviewers.
FEATURE_STATUS: IN_DEVELOPMENT

More advice:

Design

• Once you get LGTM from a @kubernetes/feature-reviewers member, you can check this checkbox, and the reviewer will apply the "design-complete" label.

Coding

• Use as many PRs as you need. Write tests in the same or different PRs, as is convenient for you.
• As each PR is merged, add a comment to this issue referencing the PRs. Code goes in the http://github.com/kubernetes/kubernetes repository,
  and sometimes http://github.com/kubernetes/contrib, or other repos.
• When you are done with the code, apply the "code-complete" label.
• When the feature has user docs, please add a comment mentioning @kubernetes/feature-reviewers and they will
  check that the code matches the proposed feature and design, and that everything is done, and that there is adequate
  testing. They won't do detailed code review: that already happened when your PRs were reviewed.
  When that is done, you can check this box and the reviewer will apply the "code-complete" label.

Docs

• Write user docs and get them merged in.
• User docs go into http://github.com/kubernetes/kubernetes.github.io.
• When the feature has user docs, please add a comment mentioning @kubernetes/docs.
• When you get LGTM, you can check this checkbox, and the reviewer will apply the "docs-complete" label.

[derekwaynecarr]

/cc @kubernetes/sig-node

[vishh]

cc @ronnielai

[timothysc]

Given todays conversation, perhaps we could have a default policy of: "should pro-actively reclaim related resource to maintain node stability by deleting images, logs, and evicting pods." with the potential of firing an administrator controlled script which could also apply to other resource dimensions.

/cc @nqn

[derekwaynecarr]

I think that is feature creep. The goal is disk. Other resource
dimensions can be monitored outside of the Kubelet. We can discuss in
sig-node.

On Monday, July 25, 2016, Timothy St. Clair notifications@github.com
wrote:

Given todays conversation, perhaps we could have a default policy of:
"should pro-actively reclaim related resource to maintain node stability by
deleting images, logs, and evicting pods." with the potential of firing an
administrator controlled script which could also apply to other resource
dimensions.

/cc @nqn https://github.com/nqn

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
#39 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF8dbKgYiMydscDH1bMwo52nrkK3105qks5qZRiWgaJpZM4JR21_
.

[derekwaynecarr]

All PRs planned for 1.4 have merged in time for feature freeze, I will update the check-list next week.

[janetkuo]

@derekwaynecarr Are the docs ready? Please update the docs in https://github.com/kubernetes/kubernetes.github.io, and then add PR numbers and check the docs box in the issue description

[derekwaynecarr]

Docs are coming next week unless @ronnielai has anything yet? I think we
can update the existing eviction doc pretty quickly from the design doc.
They are pretty close for a reason :-)

On Friday, September 2, 2016, Janet Kuo notifications@github.com wrote:

@derekwaynecarr https://github.com/derekwaynecarr Are the docs ready?
Please update the docs in https://github.com/kubernetes/
kubernetes.github.io, and then add PR numbers and check the docs box in
the issue description

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#39 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF8dbLRdJAt5Uq_Wqlj9DscMukPPJCWeks5qmGY6gaJpZM4JR21_
.

[jaredbhatti]

@derekwaynecarr Can you add your docs PR here when you have it ready?

[vishh]

@derekwaynecarr My assumption is that this feature is alpha or beta in
v1.4. I hope the docs will reflect that!

On Wed, Sep 7, 2016 at 2:02 PM, Jared notifications@github.com wrote:

@derekwaynecarr https://github.com/derekwaynecarr Can you add your docs
PR here when you have it ready?

—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub
#39 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGvIKNRzEQaiP_bJjF_BFGfRRDM_3UIrks5qnyZsgaJpZM4JR21_
.

[derekwaynecarr]

@vishh @jaredbhatti docs PR: kubernetes/website#1196

[derekwaynecarr]

@kubernetes/docs -- added feature doc pr kubernetes/website#1196

[jaredbhatti]

@derekwaynecarr Is this feature Stable or Beta?

[idvoretskyi]

@derekwaynecarr can you provide us with the actual feature status?
Thanks.

[derekwaynecarr]

It's not alpha, I would say beta (when dealing with inodes) but stable for
disk capacity.

On Wednesday, September 21, 2016, Ihor Dvoretskyi notifications@github.com
wrote:

@derekwaynecarr https://github.com/derekwaynecarr can you provide us
with the actual feature status?
Thanks.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#39 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF8dbPWzNauUTXcsgIeSn96pW8bv_C3Mks5qsahfgaJpZM4JR21_
.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

[ezware]

how to disable disk pressure observe?

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close