Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace Sentry metrics with Prometheus metrics #814

Merged
merged 6 commits into from
Oct 24, 2024

Conversation

tony-codecov
Copy link
Contributor

@tony-codecov tony-codecov commented Oct 23, 2024

Replace Sentry metrics with Prometheus metrics.

Closes https://github.com/codecov/infrastructure-team/issues/461.

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. In 2022 this entity acquired Codecov and as result Sentry is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

Copy link

This PR includes changes to shared. Please review them here: codecov/shared@4b93700...cde9892

Copy link

codecov bot commented Oct 23, 2024

Codecov Report

Attention: Patch coverage is 94.11765% with 1 line in your changes missing coverage. Please review.

Project coverage is 98.01%. Comparing base (43358fc) to head (4292546).
Report is 1 commits behind head on main.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
tasks/sync_pull.py 75.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #814   +/-   ##
=======================================
  Coverage   98.01%   98.01%           
=======================================
  Files         444      444           
  Lines       36469    36473    +4     
=======================================
+ Hits        35745    35749    +4     
  Misses        724      724           
Flag Coverage Δ
integration 98.01% <94.11%> (+<0.01%) ⬆️
unit 98.01% <94.11%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 95.90% <94.11%> (+<0.01%) ⬆️
OutsideTasks 98.00% <100.00%> (+<0.01%) ⬆️
Files with missing lines Coverage Δ
services/bundle_analysis/report.py 100.00% <ø> (ø)
services/failure_normalizer.py 100.00% <100.00%> (ø)
...ervices/notification/notifiers/comment/__init__.py 99.43% <100.00%> (+<0.01%) ⬆️
tasks/parallel_verification.py 92.95% <100.00%> (+0.20%) ⬆️
tasks/sync_pull.py 94.38% <75.00%> (+0.02%) ⬆️

@codecov-notifications
Copy link

codecov-notifications bot commented Oct 23, 2024

Codecov Report

Attention: Patch coverage is 94.11765% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
tasks/sync_pull.py 75.00% 1 Missing ⚠️

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #814   +/-   ##
=======================================
  Coverage   98.01%   98.01%           
=======================================
  Files         444      444           
  Lines       36469    36473    +4     
=======================================
+ Hits        35745    35749    +4     
  Misses        724      724           
Flag Coverage Δ
integration 98.01% <94.11%> (+<0.01%) ⬆️
unit 98.01% <94.11%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 95.90% <94.11%> (+<0.01%) ⬆️
OutsideTasks 98.00% <100.00%> (+<0.01%) ⬆️
Files with missing lines Coverage Δ
services/bundle_analysis/report.py 100.00% <ø> (ø)
services/failure_normalizer.py 100.00% <100.00%> (ø)
...ervices/notification/notifiers/comment/__init__.py 99.43% <100.00%> (+<0.01%) ⬆️
tasks/parallel_verification.py 92.95% <100.00%> (+0.20%) ⬆️
tasks/sync_pull.py 94.38% <75.00%> (+0.02%) ⬆️

@codecov-qa
Copy link

codecov-qa bot commented Oct 23, 2024

Codecov Report

Attention: Patch coverage is 94.11765% with 1 line in your changes missing coverage. Please review.

Project coverage is 98.01%. Comparing base (43358fc) to head (4292546).
Report is 1 commits behind head on main.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
tasks/sync_pull.py 75.00% 1 Missing ⚠️

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #814   +/-   ##
=======================================
  Coverage   98.01%   98.01%           
=======================================
  Files         444      444           
  Lines       36469    36473    +4     
=======================================
+ Hits        35745    35749    +4     
  Misses        724      724           
Flag Coverage Δ
integration 98.01% <94.11%> (+<0.01%) ⬆️
unit 98.01% <94.11%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 95.90% <94.11%> (+<0.01%) ⬆️
OutsideTasks 98.00% <100.00%> (+<0.01%) ⬆️
Files with missing lines Coverage Δ
services/bundle_analysis/report.py 100.00% <ø> (ø)
services/failure_normalizer.py 100.00% <100.00%> (ø)
...ervices/notification/notifiers/comment/__init__.py 99.43% <100.00%> (+<0.01%) ⬆️
tasks/parallel_verification.py 92.95% <100.00%> (+0.20%) ⬆️
tasks/sync_pull.py 94.38% <75.00%> (+0.02%) ⬆️

Copy link

codecov-public-qa bot commented Oct 23, 2024

Codecov Report

Attention: Patch coverage is 94.11765% with 1 line in your changes missing coverage. Please review.

Project coverage is 98.01%. Comparing base (43358fc) to head (4292546).

✅ All tests successful. No failed tests found.

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #814   +/-   ##
=======================================
  Coverage   98.01%   98.01%           
=======================================
  Files         444      444           
  Lines       36469    36473    +4     
=======================================
+ Hits        35745    35749    +4     
  Misses        724      724           
Flag Coverage Δ
integration 98.01% <94.11%> (+<0.01%) ⬆️
unit 98.01% <94.11%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
NonTestCode 95.90% <94.11%> (+<0.01%) ⬆️
OutsideTasks 98.00% <100.00%> (+<0.01%) ⬆️
Files Coverage Δ
services/bundle_analysis/report.py 100.00% <ø> (ø)
services/failure_normalizer.py 100.00% <100.00%> (ø)
...ervices/notification/notifiers/comment/__init__.py 99.43% <100.00%> (+<0.01%) ⬆️
tasks/parallel_verification.py 92.95% <100.00%> (+0.20%) ⬆️
tasks/sync_pull.py 94.38% <75.00%> (+0.02%) ⬆️

@tony-codecov tony-codecov requested a review from a team October 23, 2024 20:58
result="upload_error" if self.error else "processed",
plugin_name="n/a",
repository=self.commit.repository.repoid,
).inc()
except Exception:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this is actually fallible?
having a fn indirection just to slap a try/except on this call seems a bit weird, in particular as the call with the extra labels argument is a bit more unwieldy than just using the prometheus API directly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point, I think the existing code with the try/except block should work as is, and the inc_counter change here is not necessary.

Another thing I'm curious about this metric though is that there are 4 different results, some number of unique plugin_name and so for each unique repo_id there will be 4*count(plugin_name) time series created, and such high cardinality might be a performance issue with Prometheus. What do you think about this or am I overlooking anything?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are absolutely right.
IMO its okay to just remove the repo_id tag here. You can also check in with @adrian-codecov who has recently worked on getting some limited form of repo-tagged metrics going using an allowlist to only tag very specific repos.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've removed repo_id for now. The allowlist method can probably be implemented later when we have determined which label(s) a repo_id should be classified into.

Copy link

This PR includes changes to shared. Please review them here: codecov/shared@4b93700...cde9892

@tony-codecov tony-codecov added this pull request to the merge queue Oct 24, 2024
Merged via the queue into main with commit bbbbb6d Oct 24, 2024
24 of 27 checks passed
@tony-codecov tony-codecov deleted the tony/prometheus-metrics branch October 24, 2024 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants