Add grafana gitea-mixin #17758

v-zhuravlev · 2021-11-22T12:49:59Z

This PR adds gitea-mixin, configurable Grafana dashboards (and potentially prometheus alerts+recording rules) based on Gitea metrics.

The overview dashboard is described using jsonnet and grafonnet library: https://grafana.github.io/grafonnet-lib/

Mixins help to define dashboard and alerts as code so they can be collaboratively improved by the users.

__

Generate config files

You can manually generate dashboards, but first you should install some tools:

go get github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb
go get github.com/google/go-jsonnet/cmd/jsonnet
# or in brew: brew install go-jsonnet

For linting and formatting, you would also need mixtool and jsonnetfmt installed. If you
have a working Go development environment, it's easiest to run the following:

go get github.com/monitoring-mixins/mixtool/cmd/mixtool
go get github.com/google/go-jsonnet/cmd/jsonnetfmt

The files in dashboards_out need to be imported
into your Grafana server. The exact details will be depending on your environment.

Edit config.libsonnet (for example, list of Gitea metrics to be shown under stats can be adjusted). if required and then build JSON dashboard files for Grafana:

make

For more about mixins, please see:
https://github.com/monitoring-mixins/docs
https://www.youtube.com/watch?v=GDdnL5R_l-Y

wxiaoguang · 2021-11-22T14:05:24Z

It seems that https://gitea.com/gitea/awesome-gitea is the better place to list this dashboard.

Usually, users do not read files in gitea/contrib directory, files in this directory are mostly development related.

So listing this dashboard and document in awesome-gitea will help more users.

v-zhuravlev · 2021-11-22T16:23:34Z

HI! Thanks for the reply,
Awesome Gitea list is the great place to link to this mixin and dashboard repo for sure! The mixin' code s still has to be placed in git and is generally recommended that this is hosted together with the app itself like here:
https://github.com/prometheus/prometheus/tree/main/documentation/prometheus-mixin
https://github.com/ceph/ceph-mixins

techknowlogick · 2021-11-23T07:25:57Z

I think accepting this into contrib/ and then adding to awesome-gitea for visibility would be reasonable.

…vents

silverwind · 2021-11-25T16:15:51Z

contrib/gitea-mixin/Makefile

+	@mkdir -p dashboards_out
+	jsonnet -J vendor -m dashboards_out lib/dashboards.jsonnet
+
+build: vendor


This Makefile is missing declaration of .PHONY targets.

thanks, added .PHONY targets

silverwind · 2021-11-25T16:50:15Z

Also maybe rename folder to grafana-mixin or gitea-grafana-mixin. Just gitea-mixin seems ambiguos to me.

silverwind · 2021-11-25T16:53:06Z

contrib/gitea-mixin/dashboards/overview.libsonnet

+
+    local giteaSelector = 'job="$job", instance="$instance"',
+    local giteaStatsPanel = grafana.statPanel.new(
+                              'Gitea stats',


Too much indentation? If libsonnet syntax does not require this, I would suggest follow 2-space indendation rule to make this much more readable.

Agree, too much, it had been generated by jsonnetfmt so I made some adjustments to make suremake fmt stick 2-spcae indendation as much as possible.

…panel

codecov-commenter · 2021-12-13T21:22:18Z

Codecov Report

Merging #17758 (102de09) into main (1c7db70) will decrease coverage by 0.01%.
The diff coverage is 64.66%.

@@            Coverage Diff             @@
##             main   #17758      +/-   ##
==========================================
- Coverage   45.20%   45.19%   -0.01%     
==========================================
  Files         823      823              
  Lines       91371    91389      +18     
==========================================
- Hits        41305    41304       -1     
- Misses      43471    43492      +21     
+ Partials     6595     6593       -2

Impacted Files	Coverage Δ
cmd/dump.go	`0.89% <0.00%> (ø)`
modules/log/colors.go	`69.86% <0.00%> (ø)`
modules/util/path.go	`56.17% <0.00%> (ø)`
routers/api/v1/repo/pull.go	`28.69% <0.00%> (-0.15%)`	⬇️
routers/web/repo/pull.go	`30.38% <0.00%> (-0.17%)`	⬇️
routers/web/user/oauth.go	`35.38% <0.00%> (ø)`
services/forms/repo_form.go	`41.25% <ø> (ø)`
services/migrations/dump.go	`0.00% <0.00%> (ø)`
services/migrations/restore.go	`0.00% <0.00%> (ø)`
services/pull/merge.go	`48.33% <25.00%> (-1.06%)`	⬇️
... and 59 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ff2fd08...102de09. Read the comment docs.

…how/hide corresponding panels

v-zhuravlev · 2021-12-15T14:41:15Z

Also maybe rename folder to grafana-mixin or gitea-grafana-mixin. Just gitea-mixin seems ambiguos to me.

Thanks for the suggestion, to make it more clear, I renamed this dir to gitea-monitoring-mixin:
( https://monitoring.mixins.dev/) , as grafana dashboard is not the only possible part of the monitoring-mixin.

v-zhuravlev · 2021-12-15T20:41:49Z

Thanks for your reviews, I made changes as you requested as well as more improvements to the Grafana dashboard:

Add optional (on by default) Issue by labels, Issues by repositories panels ( represented in Add metrics to get issues by label #17201, Add metrics to get issues by repository #17225 made by @romdum
1.1) Issue by labels panels: use colors that correspond to colors of default gitea labels set (configurable)
Add support (on by default) for metrics gitea_issues_open, gitea_issues_closed introduced here (Separate open and closed issue in metrics #16637)

contrib/gitea-monitoring-mixin/README.md

v-zhuravlev · 2021-12-20T12:58:43Z

Thank you! Opened PR for awesome-gitea, as had been discussed
https://gitea.com/gitea/awesome-gitea/pulls/63

This PR adds gitea-mixin, configurable Grafana dashboards (and potentially prometheus alerts+recording rules) based on Gitea [metrics](https://docs.gitea.io/en-us/config-cheat-sheet/#metrics-metrics). The overview dashboard is described using jsonnet and grafonnet library: https://grafana.github.io/grafonnet-lib/ Mixins help to define dashboard and alerts as code so they can be collaboratively improved by the users. ![image](https://user-images.githubusercontent.com/14870891/142862822-fe57b384-c74a-4103-8548-033e92f90751.png) __ ## Generate config files You can manually generate dashboards, but first you should install some tools: ```bash go get github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb go get github.com/google/go-jsonnet/cmd/jsonnet # or in brew: brew install go-jsonnet ``` For linting and formatting, you would also need `mixtool` and `jsonnetfmt` installed. If you have a working Go development environment, it's easiest to run the following: ```bash go get github.com/monitoring-mixins/mixtool/cmd/mixtool go get github.com/google/go-jsonnet/cmd/jsonnetfmt ``` The files in `dashboards_out` need to be imported into your Grafana server. The exact details will be depending on your environment. Edit `config.libsonnet` (for example, list of Gitea metrics to be shown under stats can be adjusted). if required and then build JSON dashboard files for Grafana: ```bash make ``` For more about mixins, please see: https://github.com/monitoring-mixins/docs https://www.youtube.com/watch?v=GDdnL5R_l-Y* add gitea mixin * remove alerts/rules * gitea-mixin: add interval factor of 1/2 to remove duplicated change events * gitea-mixin: fix changes panel, add aggregation interval for changes panel * gitea-mixin: add totals singlestat * gitea mixin: switch change graph to timeseries type * add color overrides for issue labels * bump grafonnet version * gitea-mixin: convert graphs to timeseries * gitea-mixin: make fmt * gitea-mixin: add .PHONE in Makefile * gitea-mixin: add time configration * gitea-mixin: make fmt and collapse addPanel grid * gitea-mixin: add static ids for shared panels * gitea-mixin: add flags showIssuesByRepository, showIssuesByLabel to show/hide corresponding panels * gitea-mixin: update aggregation interval * gitea-mixin: update defaults * gitea-mixin: update panel names * rename dir to gitea-monitoring-mixin * gitea-mixin: add gitea_issues_open, gitea_issues_closed metrics * gitea-mixin: update visible name for datasource * gitea-mixin: update README Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: zeripath <art27@cantab.net>

v-zhuravlev added 2 commits November 22, 2021 15:30

add gitea mixin

7a32f37

remove alerts/rules

9d7f1b6

GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Nov 22, 2021

techknowlogick approved these changes Nov 23, 2021

View reviewed changes

GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Nov 23, 2021

gitea-mixin: add interval factor of 1/2 to remove duplicated change e…

8d78b7a

…vents

silverwind reviewed Nov 25, 2021

View reviewed changes

v-zhuravlev added 6 commits December 13, 2021 15:17

gitea-mixin: fix changes panel, add aggregation interval for changes …

1246988

…panel

gitea-mixin: add totals singlestat

6f6622a

gitea mixin: switch change graph to timeseries type

f4a07bd

add color overrides for issue labels

e70fd58

bump grafonnet version

7ccd873

gitea-mixin: convert graphs to timeseries

55023bd

v-zhuravlev added 10 commits December 14, 2021 15:09

gitea-mixin: make fmt

1dcc141

gitea-mixin: add .PHONE in Makefile

dc51d8f

gitea-mixin: add time configration

7739dcb

gitea-mixin: make fmt and collapse addPanel grid

15c17ac

gitea-mixin: add static ids for shared panels

c8b305f

gitea-mixin: add flags showIssuesByRepository, showIssuesByLabel to s…

74c4a8d

…how/hide corresponding panels

gitea-mixin: update aggregation interval

3b32e19

gitea-mixin: update defaults

e549661

gitea-mixin: update panel names

493b4d6

rename dir to gitea-monitoring-mixin

8d44068

gitea-mixin: add gitea_issues_open, gitea_issues_closed metrics

b7b822c

v-zhuravlev requested a review from silverwind December 15, 2021 21:14

gitea-mixin: update visible name for datasource

2a14eba

silverwind reviewed Dec 15, 2021

View reviewed changes

contrib/gitea-monitoring-mixin/README.md Outdated Show resolved Hide resolved

silverwind reviewed Dec 15, 2021

View reviewed changes

contrib/gitea-monitoring-mixin/README.md Outdated Show resolved Hide resolved

gitea-mixin: update README

a75b139

v-zhuravlev force-pushed the main branch from cf4fdd4 to a75b139 Compare December 15, 2021 22:41

silverwind approved these changes Dec 16, 2021

View reviewed changes

GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Dec 16, 2021

lunny and others added 2 commits December 16, 2021 15:14

Merge branch 'main' into main

e628e20

Merge branch 'main' into main

102de09

zeripath merged commit 2cd1479 into go-gitea:main Dec 20, 2021

zeripath changed the title ~~Add gitea-mixin~~ Add grafana gitea-mixin Dec 20, 2021

go-gitea locked and limited conversation to collaborators Apr 28, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add grafana gitea-mixin #17758

Add grafana gitea-mixin #17758

v-zhuravlev commented Nov 22, 2021

wxiaoguang commented Nov 22, 2021

v-zhuravlev commented Nov 22, 2021

techknowlogick commented Nov 23, 2021

silverwind Nov 25, 2021 •

edited

Loading

v-zhuravlev Dec 15, 2021

silverwind commented Nov 25, 2021

silverwind Nov 25, 2021 •

edited

Loading

v-zhuravlev Dec 15, 2021

codecov-commenter commented Dec 13, 2021 •

edited

Loading

v-zhuravlev commented Dec 15, 2021

v-zhuravlev commented Dec 15, 2021 •

edited

Loading

v-zhuravlev commented Dec 20, 2021

Add grafana gitea-mixin #17758

Add grafana gitea-mixin #17758

Conversation

v-zhuravlev commented Nov 22, 2021

Generate config files

wxiaoguang commented Nov 22, 2021

v-zhuravlev commented Nov 22, 2021

techknowlogick commented Nov 23, 2021

silverwind Nov 25, 2021 • edited Loading

Choose a reason for hiding this comment

v-zhuravlev Dec 15, 2021

Choose a reason for hiding this comment

silverwind commented Nov 25, 2021

silverwind Nov 25, 2021 • edited Loading

Choose a reason for hiding this comment

v-zhuravlev Dec 15, 2021

Choose a reason for hiding this comment

codecov-commenter commented Dec 13, 2021 • edited Loading

Codecov Report

v-zhuravlev commented Dec 15, 2021

v-zhuravlev commented Dec 15, 2021 • edited Loading

v-zhuravlev commented Dec 20, 2021

silverwind Nov 25, 2021 •

edited

Loading

silverwind Nov 25, 2021 •

edited

Loading

codecov-commenter commented Dec 13, 2021 •

edited

Loading

v-zhuravlev commented Dec 15, 2021 •

edited

Loading