Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Weekly escalating task work #82478

Open
armenzg opened this issue Dec 20, 2024 · 0 comments
Open

Weekly escalating task work #82478

armenzg opened this issue Dec 20, 2024 · 0 comments

Comments

@armenzg
Copy link
Member

armenzg commented Dec 20, 2024

I'm seeing a bunch of issues here:
https://sentry.my.sentry.io/organizations/sentry/issues/?project=2&query=weekly_escalating&referrer=issue-list&statsPeriod=7d

Getting ahead of it will save us from having trouble down the road.
I believe fixing the slow DB issue would reduce some of the timeouts we're seeing on the cron monitor.

Notes from Josh:

  • modify this referrer snuba side to not be a customer rate limit, but a global rate limit based on the referrer (this is what we did for the snuba part of ML backfill)
  • determine if we're making the queries too quickly. If that's the case, we can decrease the concurrency of queries, add intentional slowness to the task, and/or try to improve the queries that it makes
@armenzg armenzg changed the title Fix errors and perf issues for weekly escalating tasks Weekly escalating task work Dec 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant