Record stats by service #966
Comments
|
Started by moving the code into a module and adding a test of the analytics endpoint: #970. |
paulmelnikow
added
operations
|
Just be careful not to overanalyze it. There is still |
paulmelnikow
removed
developer-experience
paulmelnikow
added
core
This picks up #2068 by adding per-badge stats as discussed in #966. It ensures every service has a unique `name` property. By default this comes from the class name, and is overridden in all the various places where the class names are duplicated. (Some of those don't seem that useful, like the various download interval services, though those need to be refactored down into a single service anyway.) Tests enforce the names are unique. These are the names used by the service-test runner, so it's a good idea to make them unique anyway. (It was sort of strange before that you had to specify `nuget` instead of e.g. `resharper`.) I've added validation to `deprecatedService` and `redirector`, and required that every `route` has a `base`, even if it's an empty string. The name is used to generate unique metric labels, generating metrics like these: ``` service_requests_total{category="activity",family="eclipse-marketplace",service="eclipse_marketplace_update"} 2 service_requests_total{category="activity",family="npm",service="npm_collaborators"} 3 service_requests_total{category="activity",family="steam",service="steam_file_release_date"} 2 service_requests_total{category="analysis",family="ansible",service="ansible_galaxy_content_quality_score"} 2 service_requests_total{category="analysis",family="cii-best-practices",service="cii_best_practices_service"} 4 service_requests_total{category="analysis",family="cocoapods",service="cocoapods_docs"} 2 service_requests_total{category="analysis",family="codacy",service="codacy_grade"} 3 service_requests_total{category="analysis",family="coverity",service="coverity_scan"} 2 service_requests_total{category="analysis",family="coverity",service="deprecated_coverity_ondemand"} 2 service_requests_total{category="analysis",family="dependabot",service="dependabot_semver_compatibility"} 3 service_requests_total{category="analysis",family="lgtm",service="lgtm_alerts"} 2 service_requests_total{category="analysis",family="lgtm",service="lgtm_grade"} 3 service_requests_total{category="analysis",family="snyk",service="snyk_vulnerability_git_hub"} 4 service_requests_total{category="analysis",family="snyk",service="snyk_vulnerability_npm"} 5 service_requests_total{category="analysis",family="symfony",service="sensiolabs_i_redirector"} 1 service_requests_total{category="analysis",family="symfony",service="symfony_insight_grade"} 1 service_requests_total{category="build",family="appveyor",service="app_veyor_ci"} 3 service_requests_total{category="build",family="appveyor",service="app_veyor_tests"} 6 service_requests_total{category="build",family="azure-devops",service="azure_dev_ops_build"} 6 service_requests_total{category="build",family="azure-devops",service="azure_dev_ops_release"} 5 service_requests_total{category="build",family="azure-devops",service="azure_dev_ops_tests"} 6 service_requests_total{category="build",family="azure-devops",service="vso_build_redirector"} 2 service_requests_total{category="build",family="azure-devops",service="vso_release_redirector"} 1 service_requests_total{category="build",family="bitbucket",service="bitbucket_pipelines"} 5 service_requests_total{category="build",family="circleci",service="circle_ci"} 5 ``` This is predicated on being able to use Prometheus's [`rate()`](https://prometheus.io/docs/prometheus/latest/querying/functions/#rate) function to visualize a counter's rate of change, as mentioned at #2068 (comment). Otherwise the stats will be disrupted every time a server restarts. The metrics only appear on new-style services.
|
It's really fascinating to see all this data! It's always been such a mystery of what people are really using and I have to say, it's really fun to see the numbers climb! Some badges which I personally don't notice that often are more popular than I might have expected! It's interesting to see. 🍬 @platan I tried adding some dashboards but I'm finding the process a bit confusing! These are some of the ideas I had:
|
|
Yeah well done on getting this shipped. Its amazing to finally get some understanding of which integrations are popular and what sort of numbers are involved. There are some quite surprising results in there. |
|
@paulmelnikow thanks for ideas for graphs. It would be pleasure for me to add them to grafana. I can do it at the beginning of the next week. This week I don't have access to my pc. |
|
@platan That sounds awesome! |
|
I've prepared first version of a dashboard with service stats: https://metrics.shields.io/d/aESRBSjmz/services?orgId=2. Feel free to update it (especially names of the graphs/tables). All graph except "Total requests per day per server" show data for selected time range (you can select a time range using |
|
It is so fascinating to see this information! I will think about some more graphs :) |
Right now they're showing up in "other," though I expect they make up most of that category. #966 (comment)
Right now they're showing up in "other," though I expect they make up most of that category. #966 (comment)
|
Seems like this is done! Thanks @platan! Let's open a new issue for any follow-on work. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Logging stats by service would allow us to identify which badges are used the most often. This would be useful for prioritizing fixes and feature requests. It would also help with questions like the one I raised here:
Currently we're recording stats by format. Those stats were used to drive the choice of the default format. Since that's now been established, there's no need to keep doing that.
In the process of doing that, it'd be great to move the stats code into separate modules, and get some tests around that code.
We could consider removing support for Redis, which is not used on the production servers.
(Much of above comes from an offline conversation with @espadrine)
The text was updated successfully, but these errors were encountered: