Kubelet support for Split Image Filesystem

[kannon92]

Enhancement Description

• One-line enhancement description (can be used as a release note): Kubelet can support ImageFilesystem being separated between writeable and read-only layer

• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/4191-split-image-filesystem/README.md

• Discussion Link: Garbage Collection Working group.

• Primary contact (assignee): @kannon92

• Responsible SIGs: sig-node

• Enhancement target (which target equals to which milestone):

  • Alpha CRI and Kubelet Implementation release target (x.y): 1.29
  • Alpha Container Runtime Support: 1.30
  • Beta release target (x.y): 1.31
  • Stable release target (x.y):

• Alpha - CRI Updates

  • KEP (k/enhancements) update PR(s):
    • KEP-4191 Split Image Filesystem #4198
  • Code (k/k) update PR(s):
    • Kubelet disk api cri update kubernetes#120914
    • Kubelet disk api changes kubernetes#120616

• Beta - container runtime support and ecosystem

  • Code (crio/crio):

    • update the crio api to reflect image store cri-o/cri-o#7401
    • CRIO changes to support split filesystem  cri-o/cri-o#7269

  • CAdvisor (google/cadvisor):

    • allow for cadvisor to detect disk stats when the read-only and writeable layers are on separate disks. google/cadvisor#3395

  • CRITools (kubernetes-sigs/critools):

    • adding imagefsinfo endpoint for both container and image filesystems kubernetes-sigs/cri-tools#1295

  • Kubernetes (kubernetes/kubernetes):

    • [KEP - 4191]: Use Cadvisor labels rather than hard coding them into kubelet kubernetes#122438

  • Testing

    • Split disk crio setup test-infra#31502
    • KEP-4191: Split Image Filesystem add end-to-end tests kubernetes#125417

  • Docs (k/website) update PR(s):

    • KEP-4191: Split Image Filesystem add documentation website#46951
    • Add blog post for Separate Image Filesystem for Kubelet website#43971
    • Blog post about Split Image Filesystem

  • Feedback

    • [KEP-4191] Support case when writeable layer and readable layers are on same mount but in different locations kubernetes#126559

• Stable

  • KEP (k/enhancements) update PR(s):
  • Code (k/k) update PR(s):
  • Docs (k/website) update(s):
    -->

Nice to haves:

• Separate SeparateDiskTests from eviction kubernetes#123346

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

[kannon92]

/sig node

[SergeyKanzhelev]

/stage alpha
/milestone v1.29
/label lead-opted-in

[kannon92]

/assign

[sreeram-venkitesh]

Hello @kannon92 👋, Enhancements team here.

Just checking in as we approach enhancements freeze on 01:00 UTC, Friday, 6th October, 2023.

This enhancement is targeting for stage alpha for v1.29 (correct me, if otherwise)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: 1.29. KEPs targeting stable will need to be marked as implemented after code PRs are merged and the feature gates are removed.
• KEP readme has up-to-date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here).

For this KEP #4198 seems to take care of everything. Please make sure that the PR is merged in time. Will move the KEP to tracked for enhancement freeze once everything is merged into k/enhancements.

The status of this enhancement is marked as at risk for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[sreeram-venkitesh]

Hi @kannon92, checking in once more as we approach the 1.29 enhancement freeze deadline on 01:00 UTC Friday, 6th October 2023. The status of this enhancement is marked as at risk. It looks like #4198 will address most of the requirements. Please make sure that the changes are merged in time. Let me know if I missed anything. Thanks!

[kannon92]

Changes were merged so we hopefully should be good.

[npolshakova]

With KEP PR #4198 approved, the enhancement is ready for the enhancements freeze. The status is now marked as tracked for enhancement freeze for 1.29. 🚀 Thank you!

[katcosgrove]

Hey there @kannon92! 👋, v1.29 Docs Lead here.
Does this enhancement work planned for v1.29 require any new docs or modification to existing docs?
If so, please follows the steps here to open a PR against dev-1.29 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday, 19 October 2023.
Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.
Thank you!

[katcosgrove]

Hi again @kannon92! The deadline to open a placeholder PR against k/website for required documentation is this Thursday, 19 October. Could you please update me on the status of docs for this enhancement? Thank you!

[kannon92]

This first alpha release won’t have any corresponding doc changes yet.

[James-Quigley]

Hi @kannon92 👋 from the v1.29 Communications Release Team! We would like to check if you have any plans to publish blogs for this KEP regarding new features, removals, and deprecations for this release.
If so, you need to open a PR placeholder in the website repository.
The deadline will be on Tuesday 14th November 2023 (after the Docs deadline PR ready for review)
Here's the 1.29 Calendar

[sreeram-venkitesh]

Hey again @kannon92 👋 v1.29 Enhancements team here,

Just checking in as we approach code freeze at 01:00 UTC Wednesday 1st November 2023 .

Here's where this enhancement currently stands:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• All PR/s are ready to be merged (they have approved and lgtm labels applied) by the code freeze deadline. This includes tests.

The status of this KEP is currently at risk for Code Freeze. From what I understand, kubernetes/kubernetes#120914 and kubernetes/kubernetes#120616 are the code PRs that is planned for the v1.29 release. Please make sure that kubernetes/kubernetes#120616 is merged in time for the code freeze.

As always, we are here to help if any questions come up. Thanks!

[npolshakova]

With kubernetes/kubernetes#120616 this is now marked as tracked for code freeze for 1.29! 🚀

[sreeram-venkitesh]

Moving the KEP to At risk for code freeze for now because of kubernetes/kubernetes#126313

x-ref: kubernetes/kubernetes#126314 (comment)

[sreeram-venkitesh]

Removed from milestone as mentioned in kubernetes/kubernetes#126314 (comment).

/milestone clear

[kwilczynski]

@sreeram-venkitesh, we were able to land a fix quickly. Which is already merged and the problem with failing tests has been fixed. Would you be able to move us back into the milestone for 1.31? We don't want to miss it... if it can be helped.

[sreeram-venkitesh]

/milestone v1.31

Status updated to tracked for code freeze 🥳 with kubernetes/kubernetes#126335 and kubernetes/kubernetes#126205 merged.

@kwilczynski: kubernetes/kubernetes#126429 is not part of the exception request for this KEP right?

[kannon92]

#126429 is not part of the exception request.

[kannon92]

Posted a query about potential support in containerd: containerd/containerd#10517

[haircommander]

Since this went in beta in 1.32 and doesn't have adoptors yet, I think we will not make progress on this in 1.32 (outside of potential containerd support)

[kannon92]

@haircommander one thing I don't understand (which is why I put in proposed for consideration) is that I have one bug fix for beta I would like reviews for 1.32.

If we want to only include phase changes than that is fine. But it doesn't really track the work that is done within a stage.

I have kubernetes/kubernetes#126559 which has a PR open.

[haircommander]

yeah to simplify planning i figure we can use the board to track enhancements that the release team will follow (that require a change to the KEP, documentation, etc).

[tjons]

Hi, enhancements lead here - I inadvertently added this to the 1.32 tracking board 😀. Please readd it if you wish to progress this enhancement in 1.32.

/remove-label lead-opted-in

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[kannon92]

/remove-lifecycle stale