Kubernetes API Server Dashboard is using wrong aggregations

[simianhacker]

The following visualizations are using sum and derivative together

• API Server Top clients by number of requests [Metricbeat Kubernetes] ECS
• API Server Requests [Metricbeat Kubernetes] ECS
• API Server Top clients by resource [Metricbeat Kubernetes] ECS

Each of these visualization is taking a sum of kubernetes.apiserver.request.count and then a derivative.

• If kubernetes.apiserver.request.count IS a counter then you should be using max with the derivative.
• If kubernetes.apiserver.request.count is NOT a counter and you're trying to produce a rate, you will need to add a cumlative_sum to convert it to a monotonically increasing number then take the derivative of that result; you will also need to provide a unit so that it scales correctly.
• If it's just a "gauge" and you're trying to display a total count for the bucket, remove the derivative.

I don't have any "actual" data for this metric but based on the field description this appears to be a counter. Either way, I'm confident that sum and derivative are probably wrong.

[ChrsMark]

It seems to be a counter indeed: https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/apiserver/pkg/endpoints/metrics/metrics.go#L66. I will open a PR for this.