Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update cluster autoscaler addon documentation for newly added flags #1714

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 29 additions & 1 deletion content/kubeone/main/guides/autoscaler-addon/_index.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,37 @@ addons:
- name: cluster-autoscaler
```

{{% notice note %}}

If you're facing an issue where your [cluster is below the minimum or above the maximum number of nodes, but the cluster-autoscaler is not fixing it][enforce-node-group-min-size], you need to pass down the `CLUSTER_AUTOSCALER_ENFORCE_NODE_GROUP_MIN_SIZE` parameter to the cluster-autoscaler addon as shown below.

If you're running a cluster with nodes in the multiple zones for the HA purposes, consider setting the `CLUSTER_AUTOSCALER_BALANCE_SIMILAR_NODE_GROUP` parameter to `true` as shown below. For more information about this parameter, check the [Cluster Autoscaler FAQ][balance-similar-node-groups].

```yaml
apiVersion: kubeone.k8c.io/v1beta2
kind: KubeOneCluster
versions:
kubernetes: '1.29.4' ## kubernetes version
cloudProvider: ## This field is sourced automatically if terraform is used for the cluster
aws: {}
addons:
enable: true
addons:
- name: cluster-autoscaler
params:
# remove the parameter which you don't need.
CLUSTER_AUTOSCALER_ENFORCE_NODE_GROUP_MIN_SIZE: "true"
CLUSTER_AUTOSCALER_BALANCE_SIMILAR_NODE_GROUP: "true"
```
{{% /notice %}}

### (Optional) Step 2: Modifying the cluster-autoscaler addon

If you wish to change some of the properties, such as timeout for scaling up/down, you’ll need to provide the appropriate command-line flags to cluster-autoscaler. For that, you’ll need to override the cluster-autoscaler addon embedded in the KubeOne binary with your addon.

To find out how to override embedded add-ons, please check the [Addons document][embedded-addons]. For more information regarding available configuration parameters and options, please check the [Cluster Autoscaler FAQ][ca-faq].
To find out how to override embedded add-ons, please check the [Addons document][embedded-addons].

For more information regarding available configuration parameters and options, please check the [Cluster Autoscaler FAQ][ca-faq].

### Step 3: Deploying Kubernetes Cluster Autoscaler

Expand Down Expand Up @@ -212,3 +238,5 @@ That is it! You have successfully deployed Kubernetes autoscaler on the KubeOne
[embedded-addons]: {{< ref "../../guides/addons/#overriding-embedded-eddons" >}}
[ca-faq]: https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md
[ca-faq-what-is]: https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#what-is-cluster-autoscaler
[enforce-node-group-min-size]: https://github.com/kubernetes/autoscaler/blob/aff50d773e42f95baaae300f27e3b2e9cba1ea1b/cluster-autoscaler/FAQ.md#my-cluster-is-below-minimum--above-maximum-number-of-nodes-but-ca-did-not-fix-that-why
[balance-similar-node-groups]: https://github.com/kubernetes/autoscaler/blob/aff50d773e42f95baaae300f27e3b2e9cba1ea1b/cluster-autoscaler/FAQ.md#im-running-cluster-with-nodes-in-multiple-zones-for-ha-purposes-is-that-supported-by-cluster-autoscaler
30 changes: 29 additions & 1 deletion content/kubeone/v1.8/guides/autoscaler-addon/_index.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,37 @@ addons:
- name: cluster-autoscaler
```

{{% notice note %}}

If you're facing an issue where your [cluster is below the minimum or above the maximum number of nodes, but the cluster-autoscaler is not fixing it][enforce-node-group-min-size], you need to pass down the `CLUSTER_AUTOSCALER_ENFORCE_NODE_GROUP_MIN_SIZE` parameter to the cluster-autoscaler addon as shown below.

If you're running a cluster with nodes in the multiple zones for the HA purposes, consider setting the `CLUSTER_AUTOSCALER_BALANCE_SIMILAR_NODE_GROUP` parameter to `true` as shown below. For more information about this parameter, check the [Cluster Autoscaler FAQ][balance-similar-node-groups].

```yaml
apiVersion: kubeone.k8c.io/v1beta2
kind: KubeOneCluster
versions:
kubernetes: '1.29.4' ## kubernetes version
cloudProvider: ## This field is sourced automatically if terraform is used for the cluster
aws: {}
addons:
enable: true
addons:
- name: cluster-autoscaler
params:
# remove the parameter which you don't need.
CLUSTER_AUTOSCALER_ENFORCE_NODE_GROUP_MIN_SIZE: "true"
CLUSTER_AUTOSCALER_BALANCE_SIMILAR_NODE_GROUP: "true"
```
{{% /notice %}}

### (Optional) Step 2: Modifying the cluster-autoscaler addon

If you wish to change some of the properties, such as timeout for scaling up/down, you’ll need to provide the appropriate command-line flags to cluster-autoscaler. For that, you’ll need to override the cluster-autoscaler addon embedded in the KubeOne binary with your addon.

To find out how to override embedded add-ons, please check the [Addons document][embedded-addons]. For more information regarding available configuration parameters and options, please check the [Cluster Autoscaler FAQ][ca-faq].
To find out how to override embedded add-ons, please check the [Addons document][embedded-addons].

For more information regarding available configuration parameters and options, please check the [Cluster Autoscaler FAQ][ca-faq].

### Step 3: Deploying Kubernetes Cluster Autoscaler

Expand Down Expand Up @@ -212,3 +238,5 @@ That is it! You have successfully deployed Kubernetes autoscaler on the KubeOne
[embedded-addons]: {{< ref "../../guides/addons/#overriding-embedded-eddons" >}}
[ca-faq]: https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md
[ca-faq-what-is]: https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#what-is-cluster-autoscaler
[enforce-node-group-min-size]: https://github.com/kubernetes/autoscaler/blob/aff50d773e42f95baaae300f27e3b2e9cba1ea1b/cluster-autoscaler/FAQ.md#my-cluster-is-below-minimum--above-maximum-number-of-nodes-but-ca-did-not-fix-that-why
[balance-similar-node-groups]: https://github.com/kubernetes/autoscaler/blob/aff50d773e42f95baaae300f27e3b2e9cba1ea1b/cluster-autoscaler/FAQ.md#im-running-cluster-with-nodes-in-multiple-zones-for-ha-purposes-is-that-supported-by-cluster-autoscaler
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ This section deals with the usage of Kubernetes Cluster Autoscaler in a KKP User

## What is a Cluster Autoscaler in Kubernetes?

Kubernetes Cluster Autoscaler is a tool that automatically adjusts the size of the worker’s node up or down depending on the consumption. This means that the autoscaler, for example, automatically scale up a cluster by increasing the node count when there are not enough node resources for cluster workload scheduling and scale down when the node resources have continuously staying idle, or there are more than enough node resources available for cluster workload scheduling. In a nutshell, it is a component that automatically adjusts the size of a Kubernetes cluster so that all pods have a place to run and there are no unneeded nodes.
Kubernetes Cluster Autoscaler is a tool that automatically adjusts the size of the worker’s node up or down depending on the consumption. This means that the cluster autoscaler, for example, automatically scale up a cluster by increasing the node count when there are not enough node resources for cluster workload scheduling and scale down when the node resources have continuously staying idle, or there are more than enough node resources available for cluster workload scheduling. In a nutshell, it is a component that automatically adjusts the size of a Kubernetes cluster so that all pods have a place to run and there are no unneeded nodes.

## KKP Cluster Autoscaler Usage

Expand Down Expand Up @@ -43,7 +43,7 @@ kube-proxy-tstvd 1/1 Running 0 21m
node-local-dns-4p8jr 1/1 Running 0 21m
```

As shown above, the autoscaler is not part of the running Kubernetes components within the namespace.
As shown above, the cluster autoscaler is not part of the running Kubernetes components within the namespace.

**Step 3**

Expand Down Expand Up @@ -82,7 +82,7 @@ coredns-666448b887-s8wv8 1/1 Running 0 36m
coredns-666448b887-vldzz 1/1 Running 0 36m
```

As shown above, the autoscaler has been provisioned and running.
As shown above, the cluster autoscaler has been provisioned and running.


## Annotating MachineDeployments for Autoscaling
Expand Down Expand Up @@ -171,7 +171,7 @@ Metadata:
……………………
```

As shown above, the MachineDeployment has been annotated with a minimum of 1 and a maximum of 5. Therefore, the autoscaler will consider only the annotated MachineDeployment on the cluster.
As shown above, the MachineDeployment has been annotated with a minimum of 1 and a maximum of 5. Therefore, the cluster autoscaler will consider only the annotated MachineDeployment on the cluster.

## Edit KKP Autoscaler

Expand All @@ -187,7 +187,16 @@ You can delete autoscaler from where you edit it above and select delete.
![Delete Autoscaler](images/delete-autoscaler.png?classes=shadow,border "Delete Autoscaler")


Once it has been deleted, you can check the cluster to ensure that the autoscaler has been deleted using the command `kubectl get pods -n kube-system`.
Once it has been deleted, you can check the cluster to ensure that the cluster autoscaler has been deleted using the command `kubectl get pods -n kube-system`.


## Customize KKP Autoscaler

You can customize the cluster autoscaler addon in order to override the cluster autoscaler deployment definition to set or pass the required flag(s) by following the instructions provided [in the Addons document]({{< relref "../../architecture/concept/kkp-concepts/addons/#custom-addons" >}}).

* [My cluster is below minimum / above maximum number of nodes, but CA did not fix that! Why?](https://github.com/kubernetes/autoscaler/blob/aff50d773e42f95baaae300f27e3b2e9cba1ea1b/cluster-autoscaler/FAQ.md#my-cluster-is-below-minimum--above-maximum-number-of-nodes-but-ca-did-not-fix-that-why)

* [I'm running cluster with nodes in multiple zones for HA purposes. Is that supported by Cluster Autoscaler?](https://github.com/kubernetes/autoscaler/blob/aff50d773e42f95baaae300f27e3b2e9cba1ea1b/cluster-autoscaler/FAQ.md#im-running-cluster-with-nodes-in-multiple-zones-for-ha-purposes-is-that-supported-by-cluster-autoscaler)


## Summary
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ This section deals with the usage of Kubernetes Cluster Autoscaler in a KKP User

## What is a Cluster Autoscaler in Kubernetes?

Kubernetes Cluster Autoscaler is a tool that automatically adjusts the size of the worker’s node up or down depending on the consumption. This means that the autoscaler, for example, automatically scale up a cluster by increasing the node count when there are not enough node resources for cluster workload scheduling and scale down when the node resources have continuously staying idle, or there are more than enough node resources available for cluster workload scheduling. In a nutshell, it is a component that automatically adjusts the size of a Kubernetes cluster so that all pods have a place to run and there are no unneeded nodes.
Kubernetes Cluster Autoscaler is a tool that automatically adjusts the size of the worker’s node up or down depending on the consumption. This means that the cluster autoscaler, for example, automatically scale up a cluster by increasing the node count when there are not enough node resources for cluster workload scheduling and scale down when the node resources have continuously staying idle, or there are more than enough node resources available for cluster workload scheduling. In a nutshell, it is a component that automatically adjusts the size of a Kubernetes cluster so that all pods have a place to run and there are no unneeded nodes.

## KKP Cluster Autoscaler Usage

Expand Down Expand Up @@ -43,7 +43,7 @@ kube-proxy-tstvd 1/1 Running 0 21m
node-local-dns-4p8jr 1/1 Running 0 21m
```

As shown above, the autoscaler is not part of the running Kubernetes components within the namespace.
As shown above, the cluster autoscaler is not part of the running Kubernetes components within the namespace.

**Step 3**

Expand Down Expand Up @@ -82,7 +82,7 @@ coredns-666448b887-s8wv8 1/1 Running 0 36m
coredns-666448b887-vldzz 1/1 Running 0 36m
```

As shown above, the autoscaler has been provisioned and running.
As shown above, the cluster autoscaler has been provisioned and running.


## Annotating MachineDeployments for Autoscaling
Expand Down Expand Up @@ -171,7 +171,7 @@ Metadata:
……………………
```

As shown above, the MachineDeployment has been annotated with a minimum of 1 and a maximum of 5. Therefore, the autoscaler will consider only the annotated MachineDeployment on the cluster.
As shown above, the MachineDeployment has been annotated with a minimum of 1 and a maximum of 5. Therefore, the cluster autoscaler will consider only the annotated MachineDeployment on the cluster.

## Edit KKP Autoscaler

Expand All @@ -187,7 +187,16 @@ You can delete autoscaler from where you edit it above and select delete.
![Delete Autoscaler](/img/kubermatic/v2.25/tutorials/cluster-autoscaler/delete-autoscaler.png?classes=shadow,border "Delete Autoscaler")


Once it has been deleted, you can check the cluster to ensure that the autoscaler has been deleted using the command `kubectl get pods -n kube-system`.
Once it has been deleted, you can check the cluster to ensure that the cluster autoscaler has been deleted using the command `kubectl get pods -n kube-system`.


## Customize KKP Autoscaler
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as for the KubeOne document, I'd recommend using permalinks here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed!


You can customize the cluster autoscaler addon in order to override the cluster autoscaler deployment definition to set or pass the required flag(s) by following the instructions provided [in the Addons document]({{< relref "../../architecture/concept/kkp-concepts/addons/#custom-addons" >}}).

* [My cluster is below minimum / above maximum number of nodes, but CA did not fix that! Why?](https://github.com/kubernetes/autoscaler/blob/aff50d773e42f95baaae300f27e3b2e9cba1ea1b/cluster-autoscaler/FAQ.md#my-cluster-is-below-minimum--above-maximum-number-of-nodes-but-ca-did-not-fix-that-why)

* [I'm running cluster with nodes in multiple zones for HA purposes. Is that supported by Cluster Autoscaler?](https://github.com/kubernetes/autoscaler/blob/aff50d773e42f95baaae300f27e3b2e9cba1ea1b/cluster-autoscaler/FAQ.md#im-running-cluster-with-nodes-in-multiple-zones-for-ha-purposes-is-that-supported-by-cluster-autoscaler)


## Summary
Expand Down