make telemetry helper resilient to exceptions

[matt-codecov]

https://github.com/codecov/internal-issues/issues/461

[codecov-notifications]

Codecov Report

Attention: Patch coverage is 50.00000% with 4 lines in your changes are missing coverage. Please review.

✅ All tests successful. No failed tests found.

@@            Coverage Diff             @@
##             main     #452      +/-   ##
==========================================
- Coverage   97.35%   97.33%   -0.02%     
==========================================
  Files         401      401              
  Lines       33665    33671       +6     
==========================================
+ Hits        32773    32775       +2     
- Misses        892      896       +4     

Flag	Coverage Δ
integration	97.33% <50.00%> (-0.02%)	⬇️
latest-uploader-overall	97.33% <50.00%> (-0.02%)	⬇️
unit	97.33% <50.00%> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
NonTestCode	94.56% <50.00%> (-0.03%)	⬇️
OutsideTasks	97.48% <50.00%> (-0.02%)	⬇️

Files	Coverage Δ
helpers/telemetry.py	89.55% <50.00%> (-5.53%)	⬇️

[codecov-qa]

Codecov Report

Attention: Patch coverage is 50.00000% with 4 lines in your changes are missing coverage. Please review.

Project coverage is 97.33%. Comparing base (90cdb7b) to head (2043376).

✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #452      +/-   ##
==========================================
- Coverage   97.35%   97.33%   -0.02%     
==========================================
  Files         401      401              
  Lines       33665    33671       +6     
==========================================
+ Hits        32773    32775       +2     
- Misses        892      896       +4     

Flag	Coverage Δ
integration	97.33% <50.00%> (-0.02%)	⬇️
latest-uploader-overall	97.33% <50.00%> (-0.02%)	⬇️
unit	97.33% <50.00%> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
NonTestCode	94.56% <50.00%> (-0.03%)	⬇️
OutsideTasks	97.48% <50.00%> (-0.02%)	⬇️

Files	Coverage Δ
helpers/telemetry.py	89.55% <50.00%> (-5.53%)	⬇️

[codecov-public-qa]

Codecov Report

Attention: Patch coverage is 50.00000% with 4 lines in your changes are missing coverage. Please review.

Project coverage is 97.33%. Comparing base (90cdb7b) to head (2043376).

✅ All tests successful. No failed tests found ☺️

@@            Coverage Diff             @@
##             main     #452      +/-   ##
==========================================
- Coverage   97.35%   97.33%   -0.02%     
==========================================
  Files         401      401              
  Lines       33665    33671       +6     
==========================================
+ Hits        32773    32775       +2     
- Misses        892      896       +4     

Flag	Coverage Δ
integration	97.33% <50.00%> (-0.02%)	⬇️
latest-uploader-overall	97.33% <50.00%> (-0.02%)	⬇️
unit	97.33% <50.00%> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
NonTestCode	94.56% <50.00%> (-0.03%)	⬇️
OutsideTasks	97.48% <50.00%> (-0.02%)	⬇️

Files	Coverage Δ
helpers/telemetry.py	89.55% <50.00%> (-5.53%)	⬇️

[codecov]

Codecov Report

Attention: Patch coverage is 50.00000% with 4 lines in your changes are missing coverage. Please review.

Project coverage is 97.36%. Comparing base (90cdb7b) to head (2043376).

Changes have been made to critical files, which contain lines commonly executed in production. Learn more

✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #452      +/-   ##
==========================================
- Coverage   97.37%   97.36%   -0.02%     
==========================================
  Files         432      432              
  Lines       34355    34361       +6     
==========================================
+ Hits        33454    33456       +2     
- Misses        901      905       +4     

Flag	Coverage Δ
integration	97.33% <50.00%> (-0.02%)	⬇️
latest-uploader-overall	97.33% <50.00%> (-0.02%)	⬇️
unit	97.33% <50.00%> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
NonTestCode	94.59% <50.00%> (-0.03%)	⬇️
OutsideTasks	97.48% <50.00%> (-0.02%)	⬇️

Files	Coverage Δ
helpers/telemetry.py Critical	91.30% <50.00%> (-5.53%)	⬇️

Related Entrypoints
run/app.tasks.bundle_analysis.BundleAnalysisProcessor
run/app.tasks.upload.Upload
run/app.tasks.status.SetError
run/app.tasks.notify.Notify
run/app.tasks.timeseries.save_commit_measurements
run/app.tasks.profiling.normalizer
run/app.tasks.pulls.Sync
run/app.tasks.compute_comparison.ComputeComparison
run/app.tasks.upload.UploadFinisher
run/app.tasks.upload.UploadProcessor
run/app.tasks.commit_update.CommitUpdate
run/app.tasks.upload.PreProcessUpload
run/app.tasks.profiling.collection
run/app.tasks.upload.ParallelVerification
run/app.tasks.sync_repo_languages_gql.SyncLanguagesGQL
run/app.tasks.bundle_analysis.BundleAnalysisNotify
run/app.tasks.test_results.TestResultsFinisherTask
run/app.tasks.sync_repo_languages.SyncLanguages
run/app.cron.profiling.findinguncollected
run/app.tasks.profiling.summarization
run/app.tasks.static_analysis.check_suite
run/app.tasks.sync_repos.SyncRepos
run/app.cron.hourly_check.HourlyCheckTask
run/app.tasks.test_results.TestResultsProcessor
run/app.tasks.new_user_activated.NewUserActivated
run/app.tasks.sync_teams.SyncTeams
run/app.tasks.timeseries.backfill_commits
run/app.cron.plan.TrialExpirationCronTask
run/app.cron.daily.GitHubAppWebhooksCheckTask
run/app.tasks.timeseries.backfill_dataset
run/app.tasks.label_analysis.process_request
run/app.tasks.flush_repo.FlushRepo
run/app.tasks.upload.ManualUploadCompletionTrigger
run/app.cron.daily.BrollyStatsRollupTask

[giovanni-guidini]

Are we fine with having the catch-all exceptions?
Out of curiosity, does having log.exception creates a sentry issue?

[matt-codecov]

Are we fine with having the catch-all exceptions?

i think for this component it makes sense. if telemetry fails, there isn't really anything we want the caller to do to retry or recover, and we certainly don't want to fail with an uncaught exception. should just log and continue

Out of curiosity, does having log.exception creates a sentry issue?

don't actually know, but the errors i was seeing were "DB transaction was already rolled back because of another exception: {other exception}" and the other exception has errors in sentry so they at least will not be lost

[sentry-io]

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

• ‼️ StatementError: (sqlalchemy.exc.InvalidRequestError) Can't reconnect until invalid transaction is rolled back app.tasks.upload.UploadProcessor View Issue
• ‼️ StatementError: (sqlalchemy.exc.InvalidRequestError) Can't reconnect until invalid transaction is rolled back app.tasks.upload.Upload View Issue
• ‼️ StatementError: (sqlalchemy.exc.InvalidRequestError) Can't reconnect until invalid transaction is rolled back app.tasks.upload.UploadFinisher View Issue
• ‼️ OperationalError: connection to server at "mailchimp-db.codecov.dev" (10.217.48.3), port 5432 failed: Connection re... app.cron.profiling.findinguncollected View Issue
• ‼️ StatementError: (sqlalchemy.exc.InvalidRequestError) Can't reconnect until invalid transaction is rolled back app.tasks.pulls.Sync View Issue

_{Did you find this useful? React with a 👍 or 👎}