Recommend runtimeclasses for scheduling #29062

ravisantoshgudimetla · 2021-07-21T17:52:14Z

In future, we will enforce runtimeclasses to be used. Pods having just nodeSelector
and tolerations are going to be rejected during apiserver admission. This commit
ensures we recommend using runtimeclasses in the current release so that the
transition will be smooth for end-users when we make it mandatory in future releases

In future, we will enforce runtimeclasses to be used. Pods having just nodeSelector and tolerations are going to be rejected during apiserver admission. This commit ensures we recommend using runtimeclasses in the current release so that the transition will be smooth for end-users when we make it mandatory in future releases

k8s-ci-robot · 2021-07-21T17:52:21Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign tengqm after the PR has been reviewed.
You can assign the PR to them by writing /assign @tengqm in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

content/en/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

aravindhp · 2021-07-21T18:00:16Z

content/en/docs/setup/production-environment/windows/user-guide-windows-containers.md

+Please note that this is the recommended way to schedule pods onto Windows nodes instead of using nodeSelector and
+tolerations as they can be added to pod spec by any user. In future, we will enforce runtimeclasses to be used. Pods
+having just nodeSelector and tolerations are going to be rejected during apiserver admission.


I suggest calling this out earlier under Taints and Tolerations. Otherwise the reader is going think this might come in only if they are using multiple Windows versions in the same cluster. IOW, move RuntimeClasses to the same level as Taints and Tolerations and reword appropriately.

Agree to move RuntimeClasses to be a peer of Taints and Tolerations

Please remove the "In future, ..." statement. We don't predict/document future.

Here's the style guide on future statements.

Maybe the heading should change? RuntimeClass stops being a simplification and is instead the primary, recommended way to define the OS-level scheduling constraints for Pods.

Reorganized the file to bring in some more clarity. PTAL

@tengqm @sftim - This is sort of a deprecation notice. So, it should be good as per the link you mentioned above?

I don't think should say anything about In future, we will enforce runtimeclasses to be used. Pods having just nodeSelector and tolerations are going to be rejected during apiserver admission. In this PR.

We should get approval of kubernetes/enhancements#2803 from all participating SIGs before saying anything definitive.

That's a fair argument. I'll remove that statement.

Also - because of this maybe we shouldn't say using runtime classes are recommended over nodeSelector generally.
If we can provide specific reasons why it is better (that we can commit to) great.
If not should we just document it as an alternative right now?

I think the argument is other way around, we know that the runtimeclasses are better, so we are proposing to make them authoritative in the linked enhancement.

I'll go with @sftim's suggestion that using runtimeclasses is recommended and better.

Updated. PTAL

I more meant that we shouldn't just say they are better but we should list some reasons why they are better.
Right now nodeSelectors are ubiquitous for identifying Windows pods I can see a lot of people asking why they should switch.
Since the KEP isn't approved and we cannot say nodeSelectors won't work in the future can we give some examples of why people should use them today?

I tried doing that here. I also mentioned that runtimeclasses are created by cluster admin. That is the reason for the switch. Do you want me to add more? Something from historical context that runtimeclasses didn't exist back then...

As you also need to be a cluster admin to register a Windows node / configure automation to allow admission, and this is the production Windows setup guide, I think it's not only OK to recommend using a RuntimeClass for scheduling, I strongly think we should.

netlify · 2021-07-21T18:10:49Z

✔️ Deploy Preview for kubernetes-io-main-staging ready!

🔨 Explore the source changes: ecafc37

🔍 Inspect the deploy log: https://app.netlify.com/sites/kubernetes-io-main-staging/deploys/60faf8066a259f00070e6f24

😎 Browse the preview: https://deploy-preview-29062--kubernetes-io-main-staging.netlify.app/docs/setup/production-environment/windows/user-guide-windows-containers

marosset · 2021-07-21T19:53:28Z

The example at https://kubernetes.io/docs/setup/production-environment/windows/user-guide-windows-containers/#simplifying-with-runtimeclass uses the docker runtime class handler.

Can we add an example for containerd that uses the default runtime class handler runhcs-wcow-process and then link to the https://kubernetes.io/docs/concepts/containers/runtime-class/#cri-configuration for more information?

jsturtevant · 2021-07-21T20:25:27Z

How are we going to handle something like the getting started section where node selectors are used: https://kubernetes.io/docs/setup/production-environment/windows/user-guide-windows-containers/#getting-started-deploying-a-windows-container?

sftim

I'd like to make sure we address my query about deprecation of nodeSelector.

/hold
to highlight that

marosset · 2021-07-22T18:29:59Z

content/en/docs/setup/production-environment/windows/user-guide-windows-containers.md


 ```yaml
 apiVersion: node.k8s.io/v1
 kind: RuntimeClass
 metadata:
  name: windows-2019
-handler: 'docker'
+handler: 'runhcs-wcow-process'


Can we use tabs to show handles for docker and the containerd default handler similar to OS tabs at https://kubernetes.io/docs/setup/production-environment/container-runtimes/#cri-o?

A lot of windows users are still running with docker (unfortunately) and this can be a pretty big change for them.

I need some help from @sftim or someone from familiar with that <tab> tag. I am not sure if I am having correct tags.

perithompson · 2021-07-23T15:20:51Z

How are we going to handle something like the getting started section where node selectors are used: https://kubernetes.io/docs/setup/production-environment/windows/user-guide-windows-containers/#getting-started-deploying-a-windows-container?

I feel like we should address this at the same time? If it's windows and someone is just getting started, it would be good to set them on the RuntimeClass path in the getting started guide

jsturtevant · 2021-07-23T15:38:45Z

content/en/docs/setup/production-environment/windows/user-guide-windows-containers.md

-     nodeSelector:
-      kubernetes.io/os: windows
-```
+Create a file named `win-webserver.yaml` with the contents from [here](#Use RuntimeClass(Recommended approach))


Suggested change

Create a file named `win-webserver.yaml` with the contents from [here](#Use RuntimeClass(Recommended approach))

Create a file named `win-webserver.yaml` with the contents from [using RuntimeClasses](#Use RuntimeClass(Recommended approach))

I think it is a better to use descriptions over here for screen readers

jsturtevant · 2021-07-23T15:40:58Z

content/en/docs/setup/production-environment/windows/user-guide-windows-containers.md


-### Ensuring OS-specific workloads land on the appropriate container host
+For example:  `--register-with-taints='os=windows:NoSchedule'`


Suggested change

For example: `--register-with-taints='os=windows:NoSchedule'`

For example adding the flag `--register-with-taints='os=windows:NoSchedule'` to kubelet parameters cuases kubelet to register the taint.

ravisantoshgudimetla

@jsturtevant - I addressed your comments

ravisantoshgudimetla · 2021-07-23T17:12:17Z

content/en/docs/setup/production-environment/windows/user-guide-windows-containers.md


 ```yaml
 apiVersion: node.k8s.io/v1
 kind: RuntimeClass
 metadata:
  name: windows-2019
-handler: 'docker'
+handler: 'runhcs-wcow-process'


I need some help from @sftim or someone from familiar with that <tab> tag. I am not sure if I am having correct tags.

k8s-triage-robot · 2021-10-21T18:01:38Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot · 2021-11-20T18:43:41Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-ci-robot · 2021-12-09T19:10:28Z

@ravisantoshgudimetla: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

reylejano · 2021-12-14T20:58:10Z

Hi @ravisantoshgudimetla, this PR has had no action and the bot labelled this PR as lifecycle/rotten
This PR needs a rebase
I'm closing this PR due to inactivity and it needs to rebase but if you would like to continue working on this PR, feel free to reopen the PR and rebase
/close

k8s-ci-robot · 2021-12-14T20:58:30Z

@reylejano: Closed this PR.

In response to this:

Hi @ravisantoshgudimetla, this PR has had no action and the bot labelled this PR as lifecycle/rotten
This PR needs a rebase
I'm closing this PR due to inactivity and it needs to rebase but if you would like to continue working on this PR, feel free to reopen the PR and rebase
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot requested review from jsturtevant and perithompson July 21, 2021 17:52

k8s-ci-robot added the language/en Issues or PRs related to English language label Jul 21, 2021

k8s-ci-robot added sig/docs Categorizes an issue or PR as relevant to SIG Docs. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jul 21, 2021

aravindhp reviewed Jul 21, 2021

View reviewed changes

Address reviewers comments

0daed0a

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jul 22, 2021

sftim reviewed Jul 22, 2021

View reviewed changes

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 22, 2021

Address reviewers comments

02e7dfb

marosset reviewed Jul 22, 2021

View reviewed changes

jsturtevant reviewed Jul 23, 2021

View reviewed changes

ravisantoshgudimetla force-pushed the patch-11 branch from 326664c to bbb4681 Compare July 23, 2021 16:58

Address reviewers comments

ecafc37

ravisantoshgudimetla force-pushed the patch-11 branch from bbb4681 to ecafc37 Compare July 23, 2021 17:10

ravisantoshgudimetla commented Jul 23, 2021

View reviewed changes

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Oct 21, 2021

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 20, 2021

k8s-ci-robot added the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Nov 20, 2021

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Dec 9, 2021

k8s-ci-robot closed this Dec 14, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Recommend runtimeclasses for scheduling #29062

Recommend runtimeclasses for scheduling #29062

ravisantoshgudimetla commented Jul 21, 2021

k8s-ci-robot commented Jul 21, 2021

aravindhp Jul 21, 2021

marosset Jul 21, 2021

tengqm Jul 22, 2021

sftim Jul 22, 2021

ravisantoshgudimetla Jul 22, 2021

ravisantoshgudimetla Jul 22, 2021

ravisantoshgudimetla Jul 22, 2021

marosset Jul 22, 2021

ravisantoshgudimetla Jul 22, 2021

sftim Jul 22, 2021

netlify bot commented Jul 21, 2021 •

edited

Loading

marosset commented Jul 21, 2021

jsturtevant commented Jul 21, 2021

sftim left a comment

marosset Jul 22, 2021

ravisantoshgudimetla Jul 23, 2021

perithompson commented Jul 23, 2021

jsturtevant Jul 23, 2021

jsturtevant Jul 23, 2021

ravisantoshgudimetla left a comment

ravisantoshgudimetla Jul 23, 2021

k8s-triage-robot commented Oct 21, 2021

k8s-triage-robot commented Nov 20, 2021

k8s-ci-robot commented Dec 9, 2021

reylejano commented Dec 14, 2021

k8s-ci-robot commented Dec 14, 2021

	Create a file named `win-webserver.yaml` with the contents from [here](#Use RuntimeClass(Recommended approach))
	Create a file named `win-webserver.yaml` with the contents from [using RuntimeClasses](#Use RuntimeClass(Recommended approach))


		### Ensuring OS-specific workloads land on the appropriate container host
		For example: `--register-with-taints='os=windows:NoSchedule'`

	For example: `--register-with-taints='os=windows:NoSchedule'`
	For example adding the flag `--register-with-taints='os=windows:NoSchedule'` to kubelet parameters cuases kubelet to register the taint.

Recommend runtimeclasses for scheduling #29062

Recommend runtimeclasses for scheduling #29062

Conversation

ravisantoshgudimetla commented Jul 21, 2021

k8s-ci-robot commented Jul 21, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

netlify bot commented Jul 21, 2021 • edited Loading

marosset commented Jul 21, 2021

jsturtevant commented Jul 21, 2021

sftim left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

perithompson commented Jul 23, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ravisantoshgudimetla left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

k8s-triage-robot commented Oct 21, 2021

k8s-triage-robot commented Nov 20, 2021

k8s-ci-robot commented Dec 9, 2021

reylejano commented Dec 14, 2021

k8s-ci-robot commented Dec 14, 2021

netlify bot commented Jul 21, 2021 •

edited

Loading