Add a metric to count `carryforward_mode=labels` usage #812

Swatinem · 2024-10-23T10:23:23Z

This introduces a metric to count the various usages of carryforward_mode=labels. It also adds some comments to those places explaining whether the logic might be over- or undercounting the actual real-world usage.

codecov · 2024-10-23T10:29:13Z

Codecov Report

Attention: Patch coverage is 88.88889% with 1 line in your changes missing coverage. Please review.

Project coverage is 97.98%. Comparing base (ad3621c) to head (973f0f1).
Report is 1 commits behind head on main.

✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
services/report/raw_upload_processor.py	80.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #812      +/-   ##
==========================================
- Coverage   97.99%   97.98%   -0.01%     
==========================================
  Files         443      444       +1     
  Lines       36448    36457       +9     
==========================================
+ Hits        35716    35724       +8     
- Misses        732      733       +1

Flag	Coverage Δ
integration	`97.98% <88.88%> (-0.01%)`	⬇️
unit	`97.98% <88.88%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
NonTestCode	`95.83% <88.88%> (-0.01%)`	⬇️
OutsideTasks	`97.96% <85.71%> (-0.01%)`	⬇️

Files with missing lines	Coverage Δ
services/processing/metrics.py	`100.00% <100.00%> (ø)`
tasks/upload_finisher.py	`97.75% <100.00%> (+0.02%)`	⬆️
services/report/raw_upload_processor.py	`95.29% <80.00%> (-0.47%)`	⬇️

codecov-notifications · 2024-10-23T10:29:19Z

Codecov Report

Attention: Patch coverage is 88.88889% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
services/report/raw_upload_processor.py	80.00%	1 Missing ⚠️

@@            Coverage Diff             @@
##             main     #812      +/-   ##
==========================================
- Coverage   97.99%   97.98%   -0.01%     
==========================================
  Files         443      444       +1     
  Lines       36448    36457       +9     
==========================================
+ Hits        35716    35724       +8     
- Misses        732      733       +1

Flag	Coverage Δ
integration	`97.98% <88.88%> (-0.01%)`	⬇️
unit	`97.98% <88.88%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
NonTestCode	`95.83% <88.88%> (-0.01%)`	⬇️
OutsideTasks	`97.96% <85.71%> (-0.01%)`	⬇️

Files with missing lines	Coverage Δ
services/processing/metrics.py	`100.00% <100.00%> (ø)`
tasks/upload_finisher.py	`97.75% <100.00%> (+0.02%)`	⬆️
services/report/raw_upload_processor.py	`95.29% <80.00%> (-0.47%)`	⬇️

codecov-qa · 2024-10-23T10:29:20Z

Codecov Report

Attention: Patch coverage is 88.88889% with 1 line in your changes missing coverage. Please review.

Project coverage is 97.98%. Comparing base (ad3621c) to head (973f0f1).
Report is 1 commits behind head on main.

✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
services/report/raw_upload_processor.py	80.00%	1 Missing ⚠️

@@            Coverage Diff             @@
##             main     #812      +/-   ##
==========================================
- Coverage   97.99%   97.98%   -0.01%     
==========================================
  Files         443      444       +1     
  Lines       36448    36457       +9     
==========================================
+ Hits        35716    35724       +8     
- Misses        732      733       +1

Flag	Coverage Δ
integration	`97.98% <88.88%> (-0.01%)`	⬇️
unit	`97.98% <88.88%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
NonTestCode	`95.83% <88.88%> (-0.01%)`	⬇️
OutsideTasks	`97.96% <85.71%> (-0.01%)`	⬇️

Files with missing lines	Coverage Δ
services/processing/metrics.py	`100.00% <100.00%> (ø)`
tasks/upload_finisher.py	`97.75% <100.00%> (+0.02%)`	⬆️
services/report/raw_upload_processor.py	`95.29% <80.00%> (-0.47%)`	⬇️

codecov-public-qa · 2024-10-23T10:29:26Z

Codecov Report

Attention: Patch coverage is 88.88889% with 1 line in your changes missing coverage. Please review.

Project coverage is 97.98%. Comparing base (ad3621c) to head (973f0f1).

✅ All tests successful. No failed tests found.

@@            Coverage Diff             @@
##             main     #812      +/-   ##
==========================================
- Coverage   97.99%   97.98%   -0.01%     
==========================================
  Files         443      444       +1     
  Lines       36448    36457       +9     
==========================================
+ Hits        35716    35724       +8     
- Misses        732      733       +1

Flag	Coverage Δ
integration	`97.98% <88.88%> (-0.01%)`	⬇️
unit	`97.98% <88.88%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
NonTestCode	`95.83% <88.88%> (-0.01%)`	⬇️
OutsideTasks	`97.96% <85.71%> (-0.01%)`	⬇️

Files	Coverage Δ
services/processing/metrics.py	`100.00% <100.00%> (ø)`
tasks/upload_finisher.py	`97.75% <100.00%> (+0.02%)`	⬆️
services/report/raw_upload_processor.py	`95.29% <80.00%> (-0.47%)`	⬇️

giovanni-guidini

Excited to see how this turns out

giovanni-guidini · 2024-10-23T10:42:47Z

tasks/upload_finisher.py

@@ -401,6 +402,10 @@ def finish_reports_processing(
            commit.state = "skipped"

        if self.should_clean_labels_index(commit_yaml, processing_results):
+            # NOTE: according to tracing, the cleanup task is never actually run.
+            # reasons for that might be that we indeed *never* have any flags
+            # configured for `carryforward_mode=labels`, or the logic is somehow wrong?


I think it might be because no repos use the label index at all
(see here)

well yes, though the conditional above in UploadFinisher is not looking at that feature flag at all.

This introduces a metric to count the various usages of `carryforward_mode=labels`. It also adds some comments to those places explaining whether the logic might be over- or undercounting the actual real-world usage.

Swatinem requested a review from a team October 23, 2024 10:23

Swatinem self-assigned this Oct 23, 2024

giovanni-guidini approved these changes Oct 23, 2024

View reviewed changes

Add a metric to count carryforward_mode=labels usage

973f0f1

This introduces a metric to count the various usages of `carryforward_mode=labels`. It also adds some comments to those places explaining whether the logic might be over- or undercounting the actual real-world usage.

Swatinem force-pushed the swatinem/labels-metric branch from caf0ab1 to 973f0f1 Compare October 23, 2024 10:58

Swatinem enabled auto-merge October 23, 2024 10:59

Swatinem added this pull request to the merge queue Oct 23, 2024

Merged via the queue into main with commit 8217e96 Oct 23, 2024
24 of 27 checks passed

Swatinem deleted the swatinem/labels-metric branch October 23, 2024 11:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a metric to count `carryforward_mode=labels` usage #812

Add a metric to count `carryforward_mode=labels` usage #812

Swatinem commented Oct 23, 2024

codecov bot commented Oct 23, 2024 •

edited

Loading

codecov-notifications bot commented Oct 23, 2024 •

edited

Loading

codecov-qa bot commented Oct 23, 2024 •

edited

Loading

codecov-public-qa bot commented Oct 23, 2024 •

edited

Loading

giovanni-guidini left a comment

giovanni-guidini Oct 23, 2024

Swatinem Oct 23, 2024

Add a metric to count carryforward_mode=labels usage #812

Add a metric to count carryforward_mode=labels usage #812

Conversation

Swatinem commented Oct 23, 2024

codecov bot commented Oct 23, 2024 • edited Loading

Codecov Report

codecov-notifications bot commented Oct 23, 2024 • edited Loading

Codecov Report

codecov-qa bot commented Oct 23, 2024 • edited Loading

Codecov Report

codecov-public-qa bot commented Oct 23, 2024 • edited Loading

Codecov Report

giovanni-guidini left a comment

Choose a reason for hiding this comment

giovanni-guidini Oct 23, 2024

Choose a reason for hiding this comment

Swatinem Oct 23, 2024

Choose a reason for hiding this comment

Add a metric to count `carryforward_mode=labels` usage #812

Add a metric to count `carryforward_mode=labels` usage #812

codecov bot commented Oct 23, 2024 •

edited

Loading

codecov-notifications bot commented Oct 23, 2024 •

edited

Loading

codecov-qa bot commented Oct 23, 2024 •

edited

Loading

codecov-public-qa bot commented Oct 23, 2024 •

edited

Loading