Support Aggregation Layer

[david92rl]

In order to take advantage of the new Horizontal Pod Autoscaling based on Custom Metrics we have to cover some requisites.

The main one is Configure Aggregation Layer. Basically set some flags on the API Server:

--requestheader-client-ca-file=<path to aggregator CA cert>
--requestheader-allowed-names=aggregator
--requestheader-extra-headers-prefix=X-Remote-Extra-
--requestheader-group-headers=X-Remote-Group
--requestheader-username-headers=X-Remote-User
--proxy-client-cert-file=<path to aggregator proxy cert>
--proxy-client-key-file=<path to aggregator proxy key>

I'm not sure if those flags must be in the KubeAPIServerConfig section or Kops has to take care of it without user interaction via cluster.spec

My question is: Is there any way to make it work with the current release (1.7.0)?

[kumudt]

Facing the same issue. I couldn't find these flags in the Kops documentation.

[kumudt]

--horizontal-pod-autoscaler-use-rest-clients has to be added on kube-controller-manager
and
--requestheader-client-ca-file on kube-apiserver
has to be supported as well.

[chrislovecnm]

#3165 will support some of this

But we will need to get

--requestheader-client-ca-file=<path to aggregator CA cert>
--requestheader-allowed-names=aggregator
--requestheader-extra-headers-prefix=X-Remote-Extra-
--requestheader-group-headers=X-Remote-Group
--requestheader-username-headers=X-Remote-User

Added to the KubeAPIServerConfig as well.

[aponomarenko]

Hi,

Is it possible to pass the option --requestheader-client-ca-file (or any other custom option) to the apiserver currently by editing the cluster config file?

Thank you.

[bensussman]

@justinsb This issue is exactly what i'm running into trying to get HPA to work on my cluster. I'm using kops 1.11 so the fix from #3679 should have long since merged. I applied this manifest https://raw.githubusercontent.com/kubernetes/kops/master/addons/metrics-server/v1.8.x.yaml as recommended here: https://github.com/kubernetes/kops/blob/master/addons/metrics-server/README.md but the metrics service is reporting:

unable to fetch metrics from Kubelet ip-*.us-west-2.compute.internal (ip-*.us-west-2.compute.internal): Get https://ip-*us-west-2.compute.internal:10250/stats/summary/: x509: certificate signed by unknown authority]

I suspect it's because the value of --requestheader-client-ca-file=<path to aggregator CA cert> is not set correctly as recommended here: https://github.com/kubernetes/kops/blob/master/docs/horizontal_pod_autoscaling.md but I would expect that kops has a CA with a path for this? What should we set this to in order to get working? Is there any documentation I can follow that explains how to correctly "enable the aggregation layer" as required for HPA described here: https://github.com/kubernetes/kops/blob/master/docs/horizontal_pod_autoscaling.md ?

[bensussman]

Actually, I think #6201 might be the fix for my issue, but there seems to be some controversy about whether or not it works with kops v1.11 (which is what i have installed, the current version available from brew). I will follow along over there cc @itskingori