Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

User namespaces doc changes for 1.30 #45178

Merged
merged 5 commits into from
Mar 21, 2024

Conversation

rata
Copy link
Member

@rata rata commented Feb 16, 2024

This PR updates the documentation for changes coming in 1.30 and some other small fixes that I'll backport once this is merged.

cc @giuseppe

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Feb 16, 2024
Copy link

netlify bot commented Feb 16, 2024

👷 Deploy Preview for kubernetes-io-vnext-staging processing.

Name Link
🔨 Latest commit 69b9e71
🔍 Latest deploy log https://app.netlify.com/sites/kubernetes-io-vnext-staging/deploys/65f99a6fd00dc60008bcdb27

@k8s-ci-robot k8s-ci-robot added the language/en Issues or PRs related to English language label Feb 16, 2024
@k8s-ci-robot k8s-ci-robot added the sig/docs Categorizes an issue or PR as relevant to SIG Docs. label Feb 16, 2024
@dipesh-rawat
Copy link
Member

/retitle [WIP] User namespaces doc changes for 1.30

@k8s-ci-robot k8s-ci-robot changed the title User namespaces doc changes for 1.30 [WIP] User namespaces doc changes for 1.30 Feb 17, 2024
@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 17, 2024
@dipesh-rawat
Copy link
Member

/sig node

@k8s-ci-robot k8s-ci-robot added the sig/node Categorizes an issue or PR as relevant to SIG Node. label Feb 17, 2024
@drewhagen
Copy link
Member

Hello @rata 👋 please take a look at Documenting for a release - PR Ready for Review to get your PR ready for review before Tuesday March 12th 2024 18:00 PST. Thank you!

@rata rata force-pushed the rata/userns-1.30 branch from 1e42da9 to 0b2f9bd Compare March 7, 2024 18:45
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Mar 7, 2024
@rata rata force-pushed the rata/userns-1.30 branch 2 times, most recently from a8ab40e to f3656a9 Compare March 7, 2024 19:02
@rata rata changed the title [WIP] User namespaces doc changes for 1.30 User namespaces doc changes for 1.30 Mar 7, 2024
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 7, 2024
@rata
Copy link
Member Author

rata commented Mar 7, 2024

PTAL, this is ready for review

Copy link
Contributor

@sftim sftim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks.

As this is graduating to beta, please align with the style guide a bit more (see inline feedback).

Comment on lines 49 to 50
In addition, support is needed in the container runtime and in the OCI container
runtime.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Readers are likely to not know what you mean here. Can you clarify?

We might say “CRI-level runtime” and “OCI container runtime”, but then I think we'd do well to add an explanation about the two (maybe to our glossary?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nah, that seems to complex and we are already using all over the place (not just here) the term "container runtime", I don't think we should change it.

I've adjusted to refer to the "OCI runtime" and added a clarification here too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've found a place where we need to explain two kinds of container runtimes. You and I know the difference, but plenty of readers will not.

If we don't explain it here, we should expect to be explaining it in GitHub issues (and I'd like to avoid that happening).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sftim What? I've changed it and explained, have you read what I changed? I don't think using "CRI runtime" is worth it, for the things I explained. It is just that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewing on behalf of SIG Docs, I'm disagreeing - I think we should help readers more. Can you convince me / us that we won't have confused readers here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sftim sure. What do you think is not clear or could cause confusion? It would be great if you share that, to explain.

Let me explain in general, though, what has been already done:

First, I don't think we should use "CRI container runtime". We already use the term "container runtime" and we have been using it for several years, I don't see why we would change that. Furthermore, it is all over the place, it is not easy to change.

Secondly, the OCI runtime is only relevant for documentation in the userns context and maybe even for a few months only. So using a very specific term, instead of changing "container runtime" to "CRI container runtime" seems better.

Third, you asked to clarify and I chose a better name and clarified here what it is already.

What do you think is not clear and could cause confusion?

content/en/docs/concepts/workloads/pods/user-namespaces.md Outdated Show resolved Hide resolved
content/en/docs/concepts/workloads/pods/user-namespaces.md Outdated Show resolved Hide resolved
The kubelet can use a custom range for pods too. To configure a custom range,
the node needs to have:

* A user `kubelet` in the system
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* A user `kubelet` in the system
* A local POSIX user named `kubelet` (you cannot use any other username here)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not true, it is not a "local" user. Why would you think that?

Added the part about you cannot use another username here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd expect to find kubelet listed as a local user using getent. The first column in /etc/subuid is a login name that you can find using NSS on a typical Linux system; that's different from a username that you'd find in your cluster's audit logs.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean with "a local user using getent"? And why is it relevant?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean that you could run getent passwd | grep kubelet and I would expect to learn the UID for kubelet by doing that. Is that the case?

This is because /etc/subuid and /etc/subgid map subordinate ranges to users, and if there isn't a user, what are we mapping it to?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that is the case. However, those are not local users. That command includes local users and, for example, LDAP users.

content/en/docs/concepts/workloads/pods/user-namespaces.md Outdated Show resolved Hide resolved
content/en/docs/concepts/workloads/pods/user-namespaces.md Outdated Show resolved Hide resolved
content/en/docs/concepts/workloads/pods/user-namespaces.md Outdated Show resolved Hide resolved
content/en/docs/concepts/workloads/pods/user-namespaces.md Outdated Show resolved Hide resolved
content/en/docs/concepts/workloads/pods/user-namespaces.md Outdated Show resolved Hide resolved
Copy link
Member Author

@rata rata left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sftim fixed and answered to all your comments. But it's quite exhausting to answer to your reviews. Let me explain why.

It is exhausting because of the following (not all of them happened in this PR, but all of them happened already, I think several times):

  • You suggest changes to things you suggested to write in the first place. We can all change our mind, that is okay, but when the content is the same and you keep doing it, it is really not helpful nor easy to address
  • You make suggestions that applying them will do the opposite of another comment you did, so it is really not helpful the suggestion in the first place. It would be better if you suggest things that would just address your concerns.
  • If you don't know something, ask. I can't stress this enough. Don't assume it works in a very specific way (when there are no pointers to even suggest that AFAIK), make tens of suggestion to change things in that way and most of the times you don't even answer when people ask why you thought it was that way. Answering that would be very helpful to understand if something in the doc might suggest that.
  • If you don't know something, instead of making assumptions and tens of comments/suggestions, do it once and ask. This will make it way more simpler to act on it. If I missed to update something, you can mention it later.
  • Explain why you suggest a change. As your suggestions are loaded with assumptions you for things didn't ask, it is very hard to understand why you suggested a change: was it because you assumed something and wanted to clarify your assumption? was it to clarify something else and you added your assumption while changing the text?

content/en/docs/concepts/workloads/pods/user-namespaces.md Outdated Show resolved Hide resolved
content/en/docs/concepts/workloads/pods/user-namespaces.md Outdated Show resolved Hide resolved
The kubelet can use a custom range for pods too. To configure a custom range,
the node needs to have:

* A user `kubelet` in the system
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not true, it is not a "local" user. Why would you think that?

Added the part about you cannot use another username here.

content/en/docs/concepts/workloads/pods/user-namespaces.md Outdated Show resolved Hide resolved
content/en/docs/concepts/workloads/pods/user-namespaces.md Outdated Show resolved Hide resolved
content/en/docs/concepts/workloads/pods/user-namespaces.md Outdated Show resolved Hide resolved
content/en/docs/concepts/workloads/pods/user-namespaces.md Outdated Show resolved Hide resolved
content/en/docs/concepts/workloads/pods/user-namespaces.md Outdated Show resolved Hide resolved
Comment on lines 49 to 50
In addition, support is needed in the container runtime and in the OCI container
runtime.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nah, that seems to complex and we are already using all over the place (not just here) the term "container runtime", I don't think we should change it.

I've adjusted to refer to the "OCI runtime" and added a clarification here too.

content/en/docs/concepts/workloads/pods/user-namespaces.md Outdated Show resolved Hide resolved
@rata rata force-pushed the rata/userns-1.30 branch from f3656a9 to 7c2757a Compare March 13, 2024 11:46
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Mar 13, 2024
@sftim
Copy link
Contributor

sftim commented Mar 13, 2024

@sftim fixed and answered to all your comments. But it's quite exhausting to answer to your reviews. Let me explain why.

It is exhausting because of the following (not all of them happened in this PR, but all of them happened already, I think several times):

* You suggest changes to things you suggested to write in the first place. We can all change our mind, that is okay, but when the content is the same and you keep doing it, it is really not helpful nor easy to address

* You make suggestions that applying them will do the opposite of another comment you did, so it is really not helpful the suggestion in the first place. It would be better if you suggest things that would just address your concerns.

* If you don't know something, ask. I can't stress this enough. Don't assume it works in a very specific way (when there are no pointers to even suggest that AFAIK), make tens of suggestion to change things in that way and most of the times you don't even answer when people ask why you thought it was that way. Answering that would be very helpful to understand if something in the doc might suggest that.
* If you don't know something, instead of making assumptions and tens of comments/suggestions, do it once and ask. This will make it way more simpler to act on it. If I missed to update something, you can mention it later.

* Explain why you suggest a change. As your suggestions are loaded with assumptions you for things didn't ask, it is very hard to understand why you suggested a change: was it because you assumed something and wanted to clarify your assumption? was it to clarify something else and you added your assumption while changing the text?

Thanks for the feedback.

Here's what's important for the documentation side: we want beta-quality (or better) documentation. This means taking docs that met the criteria for an alpha release, and improving them. In places, this might mean revising text I suggested a while ago - and I get that this can be frustrating.

We're short on reviewer capacity so it might well be me that provides further reviews. I'll try to provide more context about the suggestions and to explain my thinking, inline.

@sftim
Copy link
Contributor

sftim commented Mar 13, 2024

Why? runc is the default contianer runtime and I think it is relevant for users to know it won't work. It is written in a timeless way... what is the issue with keeping it as it is?

I don't know of any place where Kubernetes hard codes runc as a kubelet default, or Pod API default, or RuntimeClass default; if is is, though, I can raise an issue (we should document this defaulting).

So far as I know, runc is very popular but Kubernetes is agnostic about which OCI runtime you use.

Copy link
Member

@drewhagen drewhagen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @rata 👋, it looks like you've already engaged a lot of feedback. Great job! Keep this up. 🚀

@giuseppe, @saschagrunert, @mrunalp, @SergeyKanzhelev, @thockin, @derekwaynecarr When one of y'all have a moment, please give a review for technical accuracy. A friendly reminder that this PR needs doc review complete by Doc Freeze on March 26th 18:00 PT to get this into the release.

Thank you @sftim for a thoroughly review detailed review, even with a shortage of reviewer capacity. Here are a few other suggestions I have for conciseness, comprehension and active voice:

@@ -46,7 +46,28 @@ tmpfs, Secrets use a tmpfs, etc.)
Some popular filesystems that support idmap mounts in Linux 6.3 are: btrfs,
Copy link
Member

@drewhagen drewhagen Mar 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious, what do we think about presenting the concept of idmap mounts for newcomers that aren't familiar with this from Linux?
https://lwn.net/Articles/896255/

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How I'd cover that:

  • have a blog article that explains Kubernetes and idmapped mounts
  • publish that article a week or so before the release
  • mark the article as evergreen (ie: we'll maintain it; usually we don't maintain blog articles once they reach a year old)
  • link from these user namespaces docs to that evergreen blog article

An example of an evergreen article: https://kubernetes.io/blog/2020/09/03/warnings/

Copy link
Member Author

@rata rata Mar 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO that seems like too much overhead. It's not clear the value it would have to have such an ongoing investment to keep it up to date either, as it is suggested. There are other places that do document idmap mounts, not sure I see the point of a k8s specific documentation (there isn't anything k8s-specific that I can think of).

Furthermore, Kubernetes users just need to know if a filesystem supports idmap mounts or not. Nothing else is really relevant.

Copy link
Member

@saschagrunert saschagrunert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just two nits, otherwise LGTM especially content-wise.

@rata rata force-pushed the rata/userns-1.30 branch 2 times, most recently from 041082a to d3ecab4 Compare March 18, 2024 10:50
@rata
Copy link
Member Author

rata commented Mar 18, 2024

@drewhagen Thanks! I've addressed all the comments, PTAL

@rata rata force-pushed the rata/userns-1.30 branch from 89eedb2 to 4f375b8 Compare March 19, 2024 12:22
rata added 5 commits March 19, 2024 14:59
Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
We were showing 4294967295 for the uid_map file, that is how it looks on
the host (not the container). Let's fix that.

While we are there, let's improve the explanation too.

Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
Now the kubelet asks the runtime for the features it supports and if it
doesn't report user namespaces support, then the kubelet will fail the
pod creation.

Therefore, it is no longer possible for the field to be ignored.

Signed-off-by: Rodrigo Campos <rodrigoca@microsoft.com>
@rata rata force-pushed the rata/userns-1.30 branch from 4f375b8 to 69b9e71 Compare March 19, 2024 14:00
Copy link
Contributor

@sftim sftim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

This is OK to publish for beta

(I judge that the) changes since #45178 (review) don't affect the technical accuracy.

/lgtm

Thanks for sticking at it @rata. The journey to GA should be smoother for this effort.

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 21, 2024
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 43606d45ecf81fad61f7ab97f028d09e5a47aa68

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sftim

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 21, 2024
@k8s-ci-robot k8s-ci-robot merged commit c7cd6c5 into kubernetes:dev-1.30 Mar 21, 2024
6 checks passed
@rata rata deleted the rata/userns-1.30 branch March 21, 2024 09:46
@drewhagen
Copy link
Member

🎉

@drewhagen
Copy link
Member

/milestone 1.30

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. language/en Issues or PRs related to English language lgtm "Looks good to me", indicates that a PR is ready to be merged. sig/docs Categorizes an issue or PR as relevant to SIG Docs. sig/node Categorizes an issue or PR as relevant to SIG Node. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
Development

Successfully merging this pull request may close these issues.

6 participants