Skip to content

Commit 7bedc9b

Browse files
ndixitandixita
committed
Adding documentation for pod-level resource resize with a simple example for alpha stage of InPlacePodLevelResourcesVerticalScaling feature
Signed-off-by: ndixita <ndixita@google.com>
1 parent 16efe7f commit 7bedc9b

File tree

3 files changed

+204
-0
lines changed

3 files changed

+204
-0
lines changed

‎content/en/docs/reference/command-line-tools-reference/feature-gates/InPlacePodLevelResourcesVerticalScaling.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
title: InPlacePodLevelResourcesVerticalScaling
3+
content_type: feature_gate
4+
_build:
5+
list: never
6+
render: false
7+
8+
stages:
9+
- stage: alpha
10+
defaultValue: false
11+
fromVersion: "1.35"
12+
---
13+
Enables in-place pod-level resources vertical scaling.

‎content/en/docs/tasks/configure-pod-container/resize-pod-resources.md‎

Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
---
2+
title: Resize CPU and Memory Resources assigned to Containers
3+
content_type: task
4+
weight: 30
5+
min-kubernetes-server-version: 1.35
6+
---
7+
8+
<!-- overview -->
9+
10+
{{< feature-state feature_gate_name="InPlacePodLevelResourcesVerticalScaling" >}}
11+
12+
This page explains how to change the CPU and memory resources set at the Pod level without recreating the Pod.
13+
14+
The In-place Pod Resize feature allows modifying resource allocations for a running Pod, avoiding application disruption. The process for resizing individual container resources is covered in the documentation [Resize CPU and Memory Resources assigned to Containers](/docs/tasks/configure-pod-container/resize-container-resources.md).
15+
16+
This document focuses on the new capability introduced in Kubernetes v1.35: In-place
17+
Pod-Level Resources Resize. Pod-level resources are defined in spec.resources and
18+
act as an upper bound constraint on the aggregate resources used by all containers
19+
within the Pod. In-place Pod-Level Resources Resize feature allows you to change
20+
these overall aggregate CPU/memory allocations for a running Pod directly.
21+
22+
## {{% heading "prerequisites" %}}
23+
24+
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
25+
26+
Your Kubernetes server must be at or later than version 1.35.
27+
To check the version, enter kubectl version.
28+
29+
The following [feature
30+
gates](/docs/reference/command-line-tools-reference/feature-gates/) must be enabled
31+
for your control plane and for all nodes in your cluster:
32+
33+
* `InPlacePodLevelResourcesVerticalScaling`
34+
* `PodLevelResources`
35+
* `InPlacePodVerticalScaling`
36+
37+
The kubectl client version must be at least v1.32 to use the
38+
--subresource=resize flag.
39+
40+
## Pod Resize Status and Retry Logic
41+
42+
The mechanism the Kubelet uses to track and retry resource changes is shared between container-level and Pod-level resize requests.
43+
44+
The statuses, reasons, and retry priorities are identical to those defined for container resize:
45+
46+
* Status Conditions: The Kubelet uses PodResizePending (with reasons like Infeasible or Deferred) and PodResizeInProgress to communicate the state of the request.
47+
48+
* Retry Priority: Deferred resizes are retried based on PriorityClass, then QoS class (Guaranteed over Burstable), and finally by the duration they have been deferred.
49+
50+
* Tracking: You can use the observedGeneration fields to track which Pod specification (metadata.generation) corresponds to the status of the latest processed resize request.
51+
52+
For a full description of these conditions and retry logic, please refer to the [Pod resize status](/docs/tasks/configure-pod-container/resize-container-resources/#pod-resize-status) section in the container resize documentation.
53+
54+
## Container Resize Policy and Pod-Level Resize
55+
56+
Pod-level resource resize does not support or require its own restart policy.
57+
58+
* No Pod-Level Policy: Changes to the Pod's aggregate resources (spec.resources) are always applied in-place without triggering a restart. This is because Pod-level resources act as an overall constraint on the Pod's cgroup and do not directly manage the application runtime within containers.
59+
60+
* [Container Policy](/docs/tasks/configure-pod-container/resize-container-resources/#container-resize-policies) Still Governs: The resizePolicy must still be configured at the container level (spec.containers[*].resizePolicy). This policy governs whether an individual container is restarted when its resource requests or limits change, regardless of whether that change was initiated by a direct container-level resize or by an update to the overall Pod-level resource envelope.
61+
62+
## Limitations
63+
64+
For Kubernetes {{< skew currentVersion >}}, resizing Pod-level resources in-place is subject to all the limitations described for container-level resource resize, which you can find here: (Resize CPU and Memory Resources assigned to Containers: Limitations)[docs/tasks/configure-pod-container/resize-container-resources/#limitations].
65+
66+
Additionally, the following constraint is specific to Pod-level resource resize:
67+
* Container Requests Validation: A resize is only permitted if the resulting
68+
Pod-level resource requests (spec.resources.requests) are greater than or equal to
69+
the sum of the corresponding resource requests from all individual containers
70+
within the Pod. This maintains the minimum guaranteed resource availability for
71+
the Pod.
72+
73+
* Container Limits Validation: A resize is permitted if individual container limits
74+
are less than or equal to the Pod-level resource limits (spec.resources.limits).
75+
The Pod-level limit serves as a boundary that no single container may exceed, but
76+
the sum of container limits is permitted to exceed the Pod-level limit, enabling
77+
resource sharing across containers within the Pod.
78+
79+
## Example: Resizing Pod-Level Resources
80+
81+
First, create a Pod designed for in-place CPU resize and restart-required memory resize.
82+
83+
{{% code_sample file="pods/resource/pod-level-resize.yaml" %}}
84+
85+
Create the pod:
86+
87+
```shell
88+
kubectl create -f pod-level-resize.yaml
89+
```
90+
91+
This pod starts in the Guaranteed QoS class as pod-level requests are equal to limits. Verify its initial state:
92+
93+
```shell
94+
# Wait a moment for the pod to be running
95+
kubectl get pod pod-level-resize-demo --output=yaml
96+
```
97+
98+
Observe the `spec.resources`(200m CPU, 200Mi memory). Note the
99+
`status.containerStatuses[0].restartCount` (should be 0) and
100+
`status.containerStatuses[1].restartCount` (should be 0).
101+
102+
Now, increase the pod-level CPU request and limit to `300m`. You use `kubectl patch` with the `--subresource resize` command line argument.
103+
104+
```shell
105+
kubectl patch pod resize-demo --subresource resize --patch \
106+
'{"spec":{"resources":{"requests":{"cpu":"300m"}, "limits":{"cpu":"300m"}}}}'
107+
108+
# Alternative methods:
109+
# kubectl -n qos-example edit pod resize-demo --subresource resize
110+
# kubectl -n qos-example apply -f <updated-manifest> --subresource resize --server-side
111+
```
112+
113+
{{< note >}}
114+
The `--subresource resize` command line argument requires `kubectl` client version v1.32.0 or later.
115+
Older versions will report an `invalid subresource` error.
116+
{{< /note >}}
117+
118+
Check the pod status again after patching:
119+
120+
```shell
121+
kubectl get pod pod-level-resize-demo --output=yaml
122+
```
123+
124+
You should see:
125+
* `spec.resources.requests` and `spec.resources.limits` now show `cpu: 300m`.
126+
* `status.containerStatuses[0].restartCount` remains `0`, because the CPU
127+
`resizePolicy` was `NotRequired`.
128+
* `status.containerStatuses[1].restartCount` increased to `1` indicating the
129+
container was restarted to apply the CPU change. The restart occurred in Container 1 despite the resize being applied at the Pod level, due to the intricate relationship between Pod-level limits and container-level policies. Because Container 1 did not specify an explicit CPU limit, its underlying resource configuration (e.g., cgroups) implicitly adopted the Pod's overall CPU limit as its effective maximum consumption boundary. When the Pod-level CPU limit was patched from 200m to 300m, this action consequently changed the implicit limit enforced on Container 1. Since Container 1 had its resizePolicy explicitly set to RestartContainer for CPU, the Kubelet was obligated to restart the container to correctly apply this change in the underlying resource enforcement mechanism, thus confirming that altering Pod-level limits can trigger container restart policies even when container limits are not directly defined.
130+
131+
## Clean up
132+
133+
Delete the pod:
134+
135+
```shell
136+
kubectl pod-level-resize-demo
137+
```
138+
139+
## {{% heading "whatsnext" %}}
140+
141+
142+
### For application developers
143+
144+
* [Assign Memory Resources to Containers and Pods](/docs/tasks/configure-pod-container/assign-memory-resource/)
145+
146+
* [Assign CPU Resources to Containers and Pods](/docs/tasks/configure-pod-container/assign-cpu-resource/)
147+
148+
* [Assign Pod-level CPU and memory resources](/docs/tasks/configure-pod-container/assign-pod-level-resources/)
149+
150+
### For cluster administrators
151+
152+
* [Configure Default Memory Requests and Limits for a Namespace](/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/)
153+
154+
* [Configure Default CPU Requests and Limits for a Namespace](/docs/tasks/administer-cluster/manage-resources/cpu-default-namespace/)
155+
156+
* [Configure Minimum and Maximum Memory Constraints for a Namespace](/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace/)
157+
158+
* [Configure Minimum and Maximum CPU Constraints for a Namespace](/docs/tasks/administer-cluster/manage-resources/cpu-constraint-namespace/)
159+
160+
* [Configure Memory and CPU Quotas for a Namespace](/docs/tasks/administer-cluster/manage-resources/quota-memory-cpu-namespace/)

‎content/en/examples/pods/resource/pod-level-resize.yaml‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
apiVersion: v1
2+
kind: Pod
3+
metadata:
4+
name: pod-level-resize-demo
5+
spec:
6+
containers:
7+
- name: pause
8+
image: registry.k8s.io/pause:3.9
9+
resizePolicy:
10+
- resourceName: cpu
11+
restartPolicy: NotRequired # Default, but explicit here
12+
- resourceName: memory
13+
restartPolicy: RestartContainer
14+
resources:
15+
requests:
16+
cpu: 100m
17+
memory: 100Mi
18+
- name: nginx-server
19+
image: registry.k8s.io/nginx:latest
20+
resizePolicy:
21+
- resourceName: cpu
22+
restartPolicy: RestartContainer
23+
- resourceName: memory
24+
restartPolicy: RestartContainer
25+
resources: # Pod-level resources
26+
requests:
27+
cpu: 200m
28+
memory: 200Mi
29+
limits:
30+
cpu: 200m
31+
memory: 200Mi

0 commit comments

Comments
 (0)