Fix metrics

[maciaszczykm]

Fixes #4145.

[maciaszczykm]

I have managed to fix the sparklines that are displayed right now even when there are completed pods:

The problem with the main graphs still persists though.

[codecov]

Codecov Report

Merging #4506 into master will increase coverage by 0.01%.
The diff coverage is 58.33%.

@@            Coverage Diff             @@
##           master    #4506      +/-   ##
==========================================
+ Coverage   46.42%   46.44%   +0.01%     
==========================================
  Files         209      209              
  Lines        9641     9646       +5     
  Branches      107      107              
==========================================
+ Hits         4476     4480       +4     
- Misses       4892     4893       +1     
  Partials      273      273

Impacted Files	Coverage Δ
src/app/backend/resource/dataselect/dataselect.go	23.38% <ø> (ø)	⬆️
...c/app/backend/integration/metric/sidecar/client.go	73.68% <0%> (ø)	⬆️
src/app/backend/resource/pod/list.go	71.15% <25%> (-1.4%)	⬇️
src/app/backend/resource/pod/metrics.go	30.3% <85.71%> (+9.61%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a3b240a...3298d66. Read the comment docs.

[maciaszczykm]

The issue is fixed, but still, some improvements need to be done in next PRs:

[maciaszczykm]

/assign @floreks @jeefy

[maciaszczykm]

I think that in the long run, we have two options:

• do not ask about the resources that don't have metrics
• when a resource doesn't have metrics (sidecar is not returning them) then don't error

I have done the first to show sparklines. The second approach is used to silence the single errors that occur during scraping cumulative metrics.

[jeefy]

@maciaszczykm Solution lgtm for now, but CI is squawking on a goimports issue: https://travis-ci.org/kubernetes/dashboard/jobs/608704284#L1314

[maciaszczykm]

PTAL

[jeefy]

/lgtm

Thanks again for tackling this

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jeefy, maciaszczykm

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [jeefy,maciaszczykm]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[RedzRedz]

I use v2.0.0-beta6, but i still can't see cpu and mem usage in dashboard.
This is some logs in pod kubernetes-dashboard-****.

2019/12/03 15:35:36 Cannot get metrics: received 0 resources from sidecar instead of 10
2019/12/03 15:35:36 Cannot get metrics: received 0 resources from sidecar instead of 10
2019/12/03 15:35:36 Cannot get metrics: received 0 resources from sidecar instead of 10
2019/12/03 15:35:36 Cannot get metrics: received 0 resources from sidecar instead of 10
2019/12/03 15:35:36 Cannot get metrics: received 0 resources from sidecar instead of 10
2019/12/03 15:35:36 Cannot get metrics: received 0 resources from sidecar instead of 10
2019/12/03 15:35:36 Cannot get metrics: received 0 resources from sidecar instead of 10
2019/12/03 15:35:36 Cannot get metrics: received 0 resources from sidecar instead of 10
2019/12/03 15:35:36 Cannot get metrics: received 0 resources from sidecar instead of 10
2019/12/03 15:35:36 Cannot get metrics: received 0 resources from sidecar instead of 10
2019/12/03 15:35:36 Skipping metric because of error: Metric label not set.

@maciaszczykm

[pkesso]

@RedzRedz same on v2.0.0-beta7

[strainovic]

Same error on v2.0.0-beta8

2019/12/08 20:32:03 Cannot get metrics: received 34 resources from sidecar instead of 36

I have 1 Job and 1 CronJob Pod in Terminated: Completed status.

[maciaszczykm]

The error itself is not a problem if rest of the graphs show up. There is no data for completed resources anymore and that is this "error".

2019/12/08 20:32:03 Cannot get metrics: received 34 resources from sidecar instead of 36

In that case you should be able to see graphs for 34 resources which is totally fine.

We will work on improving this later, the message should not be even there and we should not "expect" data for completed resources.

[strainovic]

@maciaszczykm
Unfortunately, I see no graphics if namespace has CronJobs/Jobs.

[maciaszczykm]

That is strange, can you open a new issue?

[pkesso]

Still getting Cannot get metrics: received 0 resources from sidecar instead of 10 on v2.0.0b8, no CPU usage and memory usage graphs on dashboard.
Any tips?

[floreks]

Have you installed metrics-server?

[pkesso]

Have you installed metrics-server?

Yes, metrics-server is running.

[ysaakpr]

Same with me, running the latest release beta8, Unable to get any metrics on any page, Attached sample logs from dashboard.

2019/12/17 05:28:18 Getting list of all deployments in the cluster 2019/12/17 05:28:18 [2019-12-17T05:28:18Z] Incoming HTTP/1.1 GET /api/v1/csrftoken/token request from 100.101.0.28:56098: 2019/12/17 05:28:18 [2019-12-17T05:28:18Z] Outcoming response to 100.101.0.28:56098 with 200 status code 2019/12/17 05:28:18 Cannot get metrics: received 0 resources from sidecar instead of 2 2019/12/17 05:28:18 Cannot get metrics: received 0 resources from sidecar instead of 2 2019/12/17 05:28:18 Cannot get metrics: received 0 resources from sidecar instead of 2 2019/12/17 05:28:18 Cannot get metrics: received 0 resources from sidecar instead of 2

Note: I have installed cleanly, by following the recommended yaml file mentioned in the release section.

[RedzRedz]

Thank you sir, i fix the problem. Because i dont install metrics server. After i install metrics server, i can see cpu and memory usage in dashboard. Its my problem.

…

------------------&nbsp;原始邮件&nbsp;------------------ 发件人: "Paul K."<notifications@github.com&gt;; 发送时间: 2019年12月9日(星期一) 晚上11:39 收件人: "kubernetes/dashboard"<dashboard@noreply.github.com&gt;; 抄送: "1019778654"<1019778654@qq.com&gt;; "Mention"<mention@noreply.github.com&gt;; 主题: Re: [kubernetes/dashboard] Fix metrics (#4506) Still getting Cannot get metrics: received 0 resources from sidecar instead of 10 on v2.0.0b8, no CPU usage and memory usage graphs on dashboard. Any tips? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[ysaakpr]

I have confirmed, I have installed the metric-server, in kube-system namespace, following helm charts - https://github.com/helm/charts/tree/master/stable/metrics-server, And updated to latest. Still no metrics on the dashboard. I have verified that kubectl top also working, which means the metric server is working as expected.

[ysaakpr]

My metric server is running with --kubelet-insecure-tls. Unable to run it without that, since the kubelets are using self-signed certs(Default one created while Kops initialize the cluster)