Override --ignore-daemonsets-utilization for Cluster Autoscaler on a per AWS ASG basis using ASG label

[youwalther65]

Which component are you using?:
Cluster Autoscaler on AWS

Is your feature request designed to solve a problem? If so describe the problem this feature should solve.:

Especially for smaller node types DaemonSet CPU/memory requests contribute a lot to node CPU/memory requests and therefore sometimes blocking node termination for scale-down. --ignore-daemonsets-utilization is already available as a global CAS option to solve this.

Describe the solution you'd like.:

Similar to the already available approach of AWS ASG labels which override the global cluster-autoscaler settings for the labeled ASGs , see https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/README.md. I 'd like to introduce:

k8s.io/cluster-autoscaler/node-template/autoscaling-options/ignoredaemonsetsutilization: true (overrides ignore-daemonsets-utilization value for that specific ASG)

to override global setting per ASG using this label

[vadasambar]

Possible alternatives

Set IgnoreDaemonSetsUtilization to true globally so that it applies to all the nodegroups/ASGs. One problem with this (I think) is scale down wouldn't consider node resource utilization accurately because it would always ignore daemonset resource utilization.