-
Notifications
You must be signed in to change notification settings - Fork 14.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Documentation for NodeLease feature being promoted to GA #17189
Conversation
Deploy preview for kubernetes-io-vnext-staging processing. Building with commit 718d644 https://app.netlify.com/sites/kubernetes-io-vnext-staging/deploys/5dd284e1b4847e0008bd7af2 |
/milestone v1.17 |
@wojtek-t: You must be a member of the kubernetes/website-milestone-maintainers GitHub team to set the milestone. If you believe you should be able to issue the /milestone command, please contact your Website milestone maintainers and have them propose you as an additional delegate for this responsibility. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
@@ -202,6 +200,9 @@ The following table contains feature gates for graduated or deprecated features. | |||
| `MountPropagation` | `false` | Alpha | 1.8 | 1.9 | | |||
| `MountPropagation` | `true` | Beta | 1.10 | 1.11 | | |||
| `MountPropagation` | `true` | GA | 1.12 | - | | |||
| `NodeLease` | `false` | Alpha | 1.12 | 1.13 | | |||
| `NodeLease` | `true` | Beta | 1.14 | 1.16 | | |||
| `NodeLease` | `true` | Beta | 1.17 | - | | |||
| `PersistentLocalVolumes` | `false` | Alpha | 1.7 | 1.9 | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hello @wojtek-t . Some initial suggestions:
- Line 205: Is the state GA?
- Line 177: the
kubelet
or the Kubelet? - Line 182: This sentence could be reworded and possibly add more explanation, spelling too. Does this make sense?
Compared to the Node resource, the Lease object is lightweight. The Lease resource improves the performance of the node heartbeats as the cluster scales?
From the KEP:
We will use that object to represent node heartbeat - for each Node there will be a corresponding Lease object with Name equal to Node name in a newly created dedicated namespace
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
rephrased
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Informal feedback
node lease is much more lightweight than NodeStatus, this feature makes node | ||
heartbeat significantly cheaper from both scalability and performance | ||
perspectives. | ||
Each node has an associated `Lease` object in `kube-node-lease` namespace. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit
Each node has an associated `Lease` object in `kube-node-lease` namespace. | |
Each Node has an associated `Lease` object in the `kube-node-lease` namespace. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alternatively (see other comment) omit this line.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
Node leases are renewed frequently while NodeStatus is reported from node to | ||
master only where there is some change or enough time has passed (default is | ||
5 minutes, which is longer than the default timeout of 40 seconds for | ||
unreachable nodes). Since Lease is much more lighweigh object than Node, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Typo: “lighweigh”
- This page is about Nodes, and the reader could be totally new to Kubernetes. This page is a lot of detail to explain what a Node is. If anything, I'd prefer to slim it further.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about a heading: “Node leases” plus 100 words on the control plane issuing Node leases to keep track of Node health / failures.
Does NodeStatus even need mentioning here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it does - the reason is that for the last 5 years, NodeStatus was the only signal for heartbeat. Now this is only treated as an additional one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That said, I tried to rephrase it a bit (applied suggestions by @kbhawkey ) and slightly better organize.
/milestone 1.17 |
9b56495
to
3bd6c53
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Comments applied - PTAL
node lease is much more lightweight than NodeStatus, this feature makes node | ||
heartbeat significantly cheaper from both scalability and performance | ||
perspectives. | ||
Each node has an associated `Lease` object in `kube-node-lease` namespace. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
Node leases are renewed frequently while NodeStatus is reported from node to | ||
master only where there is some change or enough time has passed (default is | ||
5 minutes, which is longer than the default timeout of 40 seconds for | ||
unreachable nodes). Since Lease is much more lighweigh object than Node, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it does - the reason is that for the last 5 years, NodeStatus was the only signal for heartbeat. Now this is only treated as an additional one.
@@ -202,6 +200,9 @@ The following table contains feature gates for graduated or deprecated features. | |||
| `MountPropagation` | `false` | Alpha | 1.8 | 1.9 | | |||
| `MountPropagation` | `true` | Beta | 1.10 | 1.11 | | |||
| `MountPropagation` | `true` | GA | 1.12 | - | | |||
| `NodeLease` | `false` | Alpha | 1.12 | 1.13 | | |||
| `NodeLease` | `true` | Beta | 1.14 | 1.16 | | |||
| `NodeLease` | `true` | Beta | 1.17 | - | | |||
| `PersistentLocalVolumes` | `false` | Alpha | 1.7 | 1.9 | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
rephrased
Node leases are renewed frequently while NodeStatus is reported from node to | ||
master only where there is some change or enough time has passed (default is | ||
5 minutes, which is longer than the default timeout of 40 seconds for | ||
unreachable nodes). Since Lease is much more lighweigh object than Node, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That said, I tried to rephrase it a bit (applied suggestions by @kbhawkey ) and slightly better organize.
3bd6c53
to
ee251d9
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some feedback - hope it's useful.
#### Heartbeats | ||
|
||
Each Node has an associated `Lease` object in `kube-node-lease` namespace. | ||
It is periodically renewed by the kubelet and both NodeStatus and the Lease |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is periodically renewed by the kubelet and both NodeStatus and the Lease | |
The kubelet updates its Lease object frequently to show that its Node is healthy. In the control plane, both NodeStatus and the Lease |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
Each Node has an associated `Lease` object in `kube-node-lease` namespace. | ||
It is periodically renewed by the kubelet and both NodeStatus and the Lease | ||
are treated as heartbeats from the node. | ||
Node leases are renewed frequently while NodeStatus is reported from node to |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If accepting the previous suggestion, how about this wording?
The kubelet updates NodeStatus either when there is change in status, or
if there has been no update for a configured interval. The default interval
for NodeStatus updates is 5 minutes (much longer than the 40 second
default timeout for unreachable nodes).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
Node leases are renewed frequently while NodeStatus is reported from node to | ||
master only where there is some change or enough time has passed (default is | ||
5 minutes, which is longer than the default timeout of 40 seconds for | ||
unreachable nodes). Compared to the Node resource, the Lease object is |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Compared to the Node resource, the Lease object is lightweight. The Lease resource improves the performance of the node heartbeats as the cluster scales.
As NodeLease is GA (once this is merged, at least), and the oldest supported Kubernetes version will have NodeLease enabled by default, it's less important to explain the older method. I think it's OK to assume that readers are using NodeLease and that it pretty much “just works” for them.
I would cut out these 2 sentences.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure - this shows why this is even needed.
perspectives. | ||
#### Heartbeats | ||
|
||
Each Node has an associated `Lease` object in `kube-node-lease` namespace. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Each Node has an associated `Lease` object in `kube-node-lease` namespace. | |
Each Node has an associated Lease object in the `kube-node-lease` {{< glossary_tooltip term_id="namespace" text="namespace">}}. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
ee251d9
to
a8d83ff
Compare
PTAL |
a8d83ff
to
906b9f9
Compare
@kbhawkey - PTAL |
5308363
to
2ef0bc8
Compare
PTAL |
2ef0bc8
to
09b1000
Compare
@sftim review from your side? Is it a lgtm? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm happy with these changes.
Thanks! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I notice a bunch of tiny, nit changes (mainly extra backticks). Also fine to merge as is, IMO.
Compared to the Node resource, the Lease is a lightweight resource, which improves | ||
the performance of the node heartbeats as the cluster scales. | ||
|
||
The `kubelet` is responsible for creating and updating `NodeStatus` and `Lease`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Late, nit tweaks:
The `kubelet` is responsible for creating and updating `NodeStatus` and `Lease`. | |
The kubelet is responsible for creating and updating `NodeStatus` and a Lease. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
|
||
The `kubelet` is responsible for creating and updating `NodeStatus` and `Lease`. | ||
|
||
- The `kubelet` updates the `NodeStatus` either when there is change in status, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- The `kubelet` updates the `NodeStatus` either when there is change in status, | |
- The kubelet updates the `NodeStatus` either when there is change in status, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
or if there has been no update for a configured interval. The default interval | ||
for `NodeStatus` updates is 5 minutes (much longer than the 40 second default | ||
timeout for unreachable nodes). | ||
- The `kubelet` creates and then updates the `Lease` object every 10 seconds |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- The `kubelet` creates and then updates the `Lease` object every 10 seconds | |
- The kubelet creates and then updates its Lease object every 10 seconds |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
for `NodeStatus` updates is 5 minutes (much longer than the 40 second default | ||
timeout for unreachable nodes). | ||
- The `kubelet` creates and then updates the `Lease` object every 10 seconds | ||
(the default update interval). `Lease` updates occur independently from the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(the default update interval). `Lease` updates occur independently from the | |
(the default update interval). Lease updates occur independently from the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
09b1000
to
573a86e
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@sftim - done; PTAL
|
||
The `kubelet` is responsible for creating and updating `NodeStatus` and `Lease`. | ||
|
||
- The `kubelet` updates the `NodeStatus` either when there is change in status, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
or if there has been no update for a configured interval. The default interval | ||
for `NodeStatus` updates is 5 minutes (much longer than the 40 second default | ||
timeout for unreachable nodes). | ||
- The `kubelet` creates and then updates the `Lease` object every 10 seconds |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
for `NodeStatus` updates is 5 minutes (much longer than the 40 second default | ||
timeout for unreachable nodes). | ||
- The `kubelet` creates and then updates the `Lease` object every 10 seconds | ||
(the default update interval). `Lease` updates occur independently from the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
Compared to the Node resource, the Lease is a lightweight resource, which improves | ||
the performance of the node heartbeats as the cluster scales. | ||
|
||
The `kubelet` is responsible for creating and updating `NodeStatus` and `Lease`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
Heartbeats, sent by Kubernetes nodes, help determine the availability of a node. | ||
There are two forms of heartbeats: updates of `NodeStatus` and the | ||
[Lease object](/docs/reference/generated/kubernetes-api/{{< latest-version >}}/#lease-v1-coordination-k8s-io). | ||
Each Node has an associated `Lease` object in the `kube-node-lease` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These changes look good. If you are cleaning up nits ...
- There is one more Lease to clean up the backticks on line 181.
- Also, I think command tools (kubelet, kubectl) typically are quoted with backticks, throughout the docs.
For example: https://kubernetes.io/docs/tasks/administer-cluster/out-of-resource/
Though, there are other places where the kubelet is not emphasized.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
kubectl
is 100% a command line tool. kubelet
, whilst also an executable, isn't a tool that people usually run in a command line, at least not during typical cluster operation. I see kubelet
fitting into the same slot as kube-scheduler
, kube-controller-manager
, and kube-proxy
.
(I've used backticks for those names in this very comment but in the docs I typically wouldn't).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed the Lease.
I agree with @sftim that kubelet is in the same category as kube-scheduler, etc.
[in this file kubelet is not in backsticks anywhere]
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hello @wojtek-t . It feels like these changes should be ready 😀 . Okay to override these comments, but if you have the energy for one more change, I'd suggest splitting/adjusting lines 183 - 184.
Lease is a lightweight resource, which improves the performance of the node heartbeats as the cluster scales.
OR
Lease is a smaller resource than the Node, which improves the performance of the node heartbeats as the cluster scales.
Line 186: The kubelet is responsible for creating and updating the NodeStatus
and a Lease object.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
573a86e
to
75102e4
Compare
75102e4
to
718d644
Compare
/lgtm |
😁 |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: kbhawkey The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
@kbhawkey - thanks a lot! |
* feat: graduate TaintNodesByCondition to GA (#17073) * Promote StartupProbe to beta (enabled by default). (#17164) * Watch bookmarks to GA (#17026) * feat: graduate ScheduleDaemonSetPods to GA (#17350) * Update Docker installation instructions (#17405) * Use exact version numbers for installing Docker in Ubuntu (#17428) * Move CSIMigration and CSIMigrationGCE to Beta in Kubernetes v1.17 (#17478) * Promote NodeLease feature to GA (#17189) * Update docs for csi topology ga (#17408) * Update RunAsUsername to beta (#17460) * doc:Update RunAsUsername to beta * doc: update samples - kubernetes.io/os is no longer beta * Updating based on review feedback * Promote Node-specific volume limits to GA (#17432) * Promote PodShareProcessNamespace to stable (#17192) * Promote PodShareProcessNamespace to stable * Add for_k8s_version to feature-state label Co-Authored-By: Tim Bannister <tim@scalefactory.com> * Readd version-check to shareProcessNamespace task * Update service load balancer finalizer doc for GA (#17438) * Update Topology Manager docs (#17451) * Added information on how device plugins can take advantage of Topology Manager * Updated the Topology Manager documentation to include additionalinformation and update some out of date sections * Fix broken Topology Manager link (#17746) Part of What's Next Device Plugin section * Update CRD defaulting docs for GA (#17450) * Add documentation for VolumeSnapshot Beta (#17233) * Updating EndpointSlice documentation for beta release in 1.17 (#17411) * (docs/dualstack): v1.17 updates (#17457) * Add placehold doc updates for dualstack in 1.17 Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com> * Add Downward API and /etc/hosts Pod IP validation Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com> * remove addressed known issue via k/k pr 85246 Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com> * Remove known issue and add flag as part of k/k 79993 Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com> * remove follow up placeholders Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com> * Update verbiage Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com> * Make IP addressing consistent throughout the task Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com> * Update to status.podIPs Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com> * Update content/en/docs/tasks/network/validate-dual-stack.md Use set instead of env Co-Authored-By: Khaled Henidak (Kal) <khnidk@outlook.com> * add topology.kubernetes.io/zone, topology.kubernetes.io/region and node.kubernetes.io/instance-type labels to docs (#17498) Signed-off-by: Andrew Sy Kim <kiman@vmware.com> * Service topology alpha documentation (#17459) * Update list of feature flags for in-tree plugins migrated to CSI (#17533) Signed-off-by: Deep Debroy <ddebroy@docker.com> * Update Node concept for TaintNodesByCondition going GA (#17577) * feat: graduate ResourceQuotaScopeSelectors to GA in 1.17 (#17554) * kubeadm: update the upgrade documentation for 1.17 (#17587) * doc: Simplify Windows deployments with RuntimeClass (#16697) * doc: Simplify Windows deployments with RuntimeClass * Updating on review feedback * doc: Adding windows-build label from enhancement 1301 * update doc for kubelet option --reserved-cpus (#17648) * feat: update TaintNodesByCondition in feature gates table (#17377) * Update docs for v1 resource quota configuration (#17547) * AdmissionConfiguration v1 (#17548) * Update WebhookAdmissionConfiguration examples (#17549) * Update AWS EBS Migration Feature state (#16126) * Add resource version section to api-concepts documentation (#16910) * Add Resource Version semantics section to api concepts * Clarify risks of going back in time, add details about compaction and watch cache sizes * Apply suggestions from liggitt Co-Authored-By: Jordan Liggitt <jordan@liggitt.net> * remove pesudocode, apply feedback * Fix typo * Clarify equality rules * Cleanup kubectl generators docs (#17609) * Write ReplicationController without a space * Drop mentioning unsupported cluster versions * Fix capitalization for “API group” * Tweak wording * Avoid using deprecated generator in example * add Antrea description in dev-1.17 (#17919) * Promote VolumeSubpathEnvExpansion to GA * Reference Documentation for the Kubernetes API for 1.17 (#18019) * Update feature-gates.md (#18033) * Reference Documentation for kubectl Commands for 1.17 (#18017) * Update for v1.17 (#18034) * Update config.toml(release-1.17) for 1.17 (#18031)
Ref kubernetes/enhancements#589